* [PATCH net-next v3 0/8] ice: support dynamic interrupt allocation
@ 2023-03-23 12:24 ` Piotr Raczynski
0 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, michal.swiatkowski, shiraz.saleem, jacob.e.keller,
sridhar.samudrala, jesse.brandeburg, aleksander.lobakin,
lukasz.czapnik, Piotr Raczynski
This patchset reimplements MSIX interrupt allocation logic to allow dynamic
interrupt allocation after MSIX has been initially enabled. This allows
current and future features to allocate and free interrupts as needed and
will help to drastically decrease number of initially preallocated
interrupts (even down to the API hard limit of 1). Although this patchset
does not change behavior in terms of actual number of allocated interrupts
during probe, it will be subject to change.
First few patches prepares to introduce dynamic allocation by moving
interrupt allocation code to separate file and update allocation API used
in the driver to the currently preferred one.
Due to the current contract between ice and irdma driver which is directly
accessing msix entries allocated by ice driver, even after moving away from
older pci_enable_msix_range function, still keep msix_entries array for
irdma use.
Next patches refactors and removes redundant code from SRIOV related logic
as it also make it easier to move away from static allocation scheme.
Last patches actually enables dynamic allocation of MSIX interrupts. First,
introduce functions to allocate and free interrupts individually. This sets
ground for the rest of the changes even if that patch still allocates the
interrupts from the preallocated pool. Since this patch starts to keep
interrupt details in ice_q_vector structure we can get rid of functions
that calculates base vector number and register offset for the interrupt
as it is equal to the interrupt index. Only keep separate register offset
functions for the VF VSIs.
Next, replace homegrown interrupt tracker with much simpler xarray based
approach. As new API always allocate interrupts one by one, also track
interrupts in the same manner.
Lastly, extend the interrupt tracker to deal both with preallocated and
dynamically allocated vectors and use pci_msix_alloc_irq_at and
pci_msix_free_irq functions. Since not all architecture supports dynamic
allocation, check it before trying to allocate a new interrupt.
As previously mentioned, this patchset does not change number of initially
allocated interrupts during init phase but now it can and will likely be
changed.
Patch 1-3 -> move code around and use newer API
Patch 4-5 -> refactor and remove redundant SRIOV code
Patch 6 -> allocate every interrupt individually
Patch 7 -> replace homegrown interrupt tracker with xarray
Patch 8 -> allow dynamic interrupt allocation
Change history:
v1 -> v2:
- ice: refactor VF control VSI interrupt handling
- move ice_get_vf_ctrl_vsi to ice_lib.c (ice_vf_lib.c depends on
CONFIG_PCI_IOV)
v2 -> v3:
- ice: refactor VF control VSI interrupt handling
- revert v2 change and add no-op function in case of CONFIG_PCI_IOV=n
- ice: add dynamic interrupt allocation
- fix commit message
Piotr Raczynski (8):
ice: move interrupt related code to separate file
ice: use pci_irq_vector helper function
ice: use preferred MSIX allocation api
ice: refactor VF control VSI interrupt handling
ice: remove redundant SRIOV code
ice: add individual interrupt allocation
ice: track interrupt vectors with xarray
ice: add dynamic interrupt allocation
drivers/net/ethernet/intel/ice/Makefile | 1 +
drivers/net/ethernet/intel/ice/ice.h | 24 +-
drivers/net/ethernet/intel/ice/ice_arfs.c | 5 +-
drivers/net/ethernet/intel/ice/ice_base.c | 36 +-
drivers/net/ethernet/intel/ice/ice_ethtool.c | 2 +-
drivers/net/ethernet/intel/ice/ice_idc.c | 54 ++-
drivers/net/ethernet/intel/ice/ice_irq.c | 377 +++++++++++++++++++
drivers/net/ethernet/intel/ice/ice_irq.h | 25 ++
drivers/net/ethernet/intel/ice/ice_lib.c | 288 +-------------
drivers/net/ethernet/intel/ice/ice_lib.h | 5 -
drivers/net/ethernet/intel/ice/ice_main.c | 268 ++-----------
drivers/net/ethernet/intel/ice/ice_ptp.c | 2 +-
drivers/net/ethernet/intel/ice/ice_sriov.c | 43 +--
drivers/net/ethernet/intel/ice/ice_vf_lib.c | 32 ++
drivers/net/ethernet/intel/ice/ice_vf_lib.h | 7 +
drivers/net/ethernet/intel/ice/ice_xsk.c | 5 +-
16 files changed, 557 insertions(+), 617 deletions(-)
create mode 100644 drivers/net/ethernet/intel/ice/ice_irq.c
create mode 100644 drivers/net/ethernet/intel/ice/ice_irq.h
--
2.38.1
^ permalink raw reply [flat|nested] 55+ messages in thread
* [Intel-wired-lan] [PATCH net-next v3 0/8] ice: support dynamic interrupt allocation
@ 2023-03-23 12:24 ` Piotr Raczynski
0 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: michal.swiatkowski, netdev, jesse.brandeburg, shiraz.saleem
This patchset reimplements MSIX interrupt allocation logic to allow dynamic
interrupt allocation after MSIX has been initially enabled. This allows
current and future features to allocate and free interrupts as needed and
will help to drastically decrease number of initially preallocated
interrupts (even down to the API hard limit of 1). Although this patchset
does not change behavior in terms of actual number of allocated interrupts
during probe, it will be subject to change.
First few patches prepares to introduce dynamic allocation by moving
interrupt allocation code to separate file and update allocation API used
in the driver to the currently preferred one.
Due to the current contract between ice and irdma driver which is directly
accessing msix entries allocated by ice driver, even after moving away from
older pci_enable_msix_range function, still keep msix_entries array for
irdma use.
Next patches refactors and removes redundant code from SRIOV related logic
as it also make it easier to move away from static allocation scheme.
Last patches actually enables dynamic allocation of MSIX interrupts. First,
introduce functions to allocate and free interrupts individually. This sets
ground for the rest of the changes even if that patch still allocates the
interrupts from the preallocated pool. Since this patch starts to keep
interrupt details in ice_q_vector structure we can get rid of functions
that calculates base vector number and register offset for the interrupt
as it is equal to the interrupt index. Only keep separate register offset
functions for the VF VSIs.
Next, replace homegrown interrupt tracker with much simpler xarray based
approach. As new API always allocate interrupts one by one, also track
interrupts in the same manner.
Lastly, extend the interrupt tracker to deal both with preallocated and
dynamically allocated vectors and use pci_msix_alloc_irq_at and
pci_msix_free_irq functions. Since not all architecture supports dynamic
allocation, check it before trying to allocate a new interrupt.
As previously mentioned, this patchset does not change number of initially
allocated interrupts during init phase but now it can and will likely be
changed.
Patch 1-3 -> move code around and use newer API
Patch 4-5 -> refactor and remove redundant SRIOV code
Patch 6 -> allocate every interrupt individually
Patch 7 -> replace homegrown interrupt tracker with xarray
Patch 8 -> allow dynamic interrupt allocation
Change history:
v1 -> v2:
- ice: refactor VF control VSI interrupt handling
- move ice_get_vf_ctrl_vsi to ice_lib.c (ice_vf_lib.c depends on
CONFIG_PCI_IOV)
v2 -> v3:
- ice: refactor VF control VSI interrupt handling
- revert v2 change and add no-op function in case of CONFIG_PCI_IOV=n
- ice: add dynamic interrupt allocation
- fix commit message
Piotr Raczynski (8):
ice: move interrupt related code to separate file
ice: use pci_irq_vector helper function
ice: use preferred MSIX allocation api
ice: refactor VF control VSI interrupt handling
ice: remove redundant SRIOV code
ice: add individual interrupt allocation
ice: track interrupt vectors with xarray
ice: add dynamic interrupt allocation
drivers/net/ethernet/intel/ice/Makefile | 1 +
drivers/net/ethernet/intel/ice/ice.h | 24 +-
drivers/net/ethernet/intel/ice/ice_arfs.c | 5 +-
drivers/net/ethernet/intel/ice/ice_base.c | 36 +-
drivers/net/ethernet/intel/ice/ice_ethtool.c | 2 +-
drivers/net/ethernet/intel/ice/ice_idc.c | 54 ++-
drivers/net/ethernet/intel/ice/ice_irq.c | 377 +++++++++++++++++++
drivers/net/ethernet/intel/ice/ice_irq.h | 25 ++
drivers/net/ethernet/intel/ice/ice_lib.c | 288 +-------------
drivers/net/ethernet/intel/ice/ice_lib.h | 5 -
drivers/net/ethernet/intel/ice/ice_main.c | 268 ++-----------
drivers/net/ethernet/intel/ice/ice_ptp.c | 2 +-
drivers/net/ethernet/intel/ice/ice_sriov.c | 43 +--
drivers/net/ethernet/intel/ice/ice_vf_lib.c | 32 ++
drivers/net/ethernet/intel/ice/ice_vf_lib.h | 7 +
drivers/net/ethernet/intel/ice/ice_xsk.c | 5 +-
16 files changed, 557 insertions(+), 617 deletions(-)
create mode 100644 drivers/net/ethernet/intel/ice/ice_irq.c
create mode 100644 drivers/net/ethernet/intel/ice/ice_irq.h
--
2.38.1
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* [PATCH net-next v3 1/8] ice: move interrupt related code to separate file
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-03-23 12:24 ` Piotr Raczynski
-1 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, michal.swiatkowski, shiraz.saleem, jacob.e.keller,
sridhar.samudrala, jesse.brandeburg, aleksander.lobakin,
lukasz.czapnik, Piotr Raczynski
Keep interrupt handling code in a dedicated file. This helps keep driver
structured better and prepares for more functionality added to this file.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
---
drivers/net/ethernet/intel/ice/Makefile | 1 +
drivers/net/ethernet/intel/ice/ice.h | 1 +
drivers/net/ethernet/intel/ice/ice_irq.c | 226 ++++++++++++++++++++++
drivers/net/ethernet/intel/ice/ice_irq.h | 10 +
drivers/net/ethernet/intel/ice/ice_main.c | 218 ---------------------
5 files changed, 238 insertions(+), 218 deletions(-)
create mode 100644 drivers/net/ethernet/intel/ice/ice_irq.c
create mode 100644 drivers/net/ethernet/intel/ice/ice_irq.h
diff --git a/drivers/net/ethernet/intel/ice/Makefile b/drivers/net/ethernet/intel/ice/Makefile
index 3290f594286e..db96ec26fdbf 100644
--- a/drivers/net/ethernet/intel/ice/Makefile
+++ b/drivers/net/ethernet/intel/ice/Makefile
@@ -18,6 +18,7 @@ ice-y := ice_main.o \
ice_txrx_lib.o \
ice_txrx.o \
ice_fltr.o \
+ ice_irq.o \
ice_pf_vsi_vlan_ops.o \
ice_vsi_vlan_ops.o \
ice_vsi_vlan_lib.o \
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 2a5632ff8081..2dc180385976 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -74,6 +74,7 @@
#include "ice_lag.h"
#include "ice_vsi_vlan_ops.h"
#include "ice_gnss.h"
+#include "ice_irq.h"
#define ICE_BAR0 0
#define ICE_REQ_DESC_MULTIPLE 32
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
new file mode 100644
index 000000000000..1fc7daec9732
--- /dev/null
+++ b/drivers/net/ethernet/intel/ice/ice_irq.c
@@ -0,0 +1,226 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2023, Intel Corporation. */
+
+#include "ice.h"
+#include "ice_lib.h"
+#include "ice_irq.h"
+
+/**
+ * ice_reduce_msix_usage - Reduce usage of MSI-X vectors
+ * @pf: board private structure
+ * @v_remain: number of remaining MSI-X vectors to be distributed
+ *
+ * Reduce the usage of MSI-X vectors when entire request cannot be fulfilled.
+ * pf->num_lan_msix and pf->num_rdma_msix values are set based on number of
+ * remaining vectors.
+ */
+static void ice_reduce_msix_usage(struct ice_pf *pf, int v_remain)
+{
+ int v_rdma;
+
+ if (!ice_is_rdma_ena(pf)) {
+ pf->num_lan_msix = v_remain;
+ return;
+ }
+
+ /* RDMA needs at least 1 interrupt in addition to AEQ MSIX */
+ v_rdma = ICE_RDMA_NUM_AEQ_MSIX + 1;
+
+ if (v_remain < ICE_MIN_LAN_TXRX_MSIX + ICE_MIN_RDMA_MSIX) {
+ dev_warn(ice_pf_to_dev(pf), "Not enough MSI-X vectors to support RDMA.\n");
+ clear_bit(ICE_FLAG_RDMA_ENA, pf->flags);
+
+ pf->num_rdma_msix = 0;
+ pf->num_lan_msix = ICE_MIN_LAN_TXRX_MSIX;
+ } else if ((v_remain < ICE_MIN_LAN_TXRX_MSIX + v_rdma) ||
+ (v_remain - v_rdma < v_rdma)) {
+ /* Support minimum RDMA and give remaining vectors to LAN MSIX
+ */
+ pf->num_rdma_msix = ICE_MIN_RDMA_MSIX;
+ pf->num_lan_msix = v_remain - ICE_MIN_RDMA_MSIX;
+ } else {
+ /* Split remaining MSIX with RDMA after accounting for AEQ MSIX
+ */
+ pf->num_rdma_msix = (v_remain - ICE_RDMA_NUM_AEQ_MSIX) / 2 +
+ ICE_RDMA_NUM_AEQ_MSIX;
+ pf->num_lan_msix = v_remain - pf->num_rdma_msix;
+ }
+}
+
+/**
+ * ice_ena_msix_range - Request a range of MSIX vectors from the OS
+ * @pf: board private structure
+ *
+ * Compute the number of MSIX vectors wanted and request from the OS. Adjust
+ * device usage if there are not enough vectors. Return the number of vectors
+ * reserved or negative on failure.
+ */
+static int ice_ena_msix_range(struct ice_pf *pf)
+{
+ int num_cpus, hw_num_msix, v_other, v_wanted, v_actual;
+ struct device *dev = ice_pf_to_dev(pf);
+ int err, i;
+
+ hw_num_msix = pf->hw.func_caps.common_cap.num_msix_vectors;
+ num_cpus = num_online_cpus();
+
+ /* LAN miscellaneous handler */
+ v_other = ICE_MIN_LAN_OICR_MSIX;
+
+ /* Flow Director */
+ if (test_bit(ICE_FLAG_FD_ENA, pf->flags))
+ v_other += ICE_FDIR_MSIX;
+
+ /* switchdev */
+ v_other += ICE_ESWITCH_MSIX;
+
+ v_wanted = v_other;
+
+ /* LAN traffic */
+ pf->num_lan_msix = num_cpus;
+ v_wanted += pf->num_lan_msix;
+
+ /* RDMA auxiliary driver */
+ if (ice_is_rdma_ena(pf)) {
+ pf->num_rdma_msix = num_cpus + ICE_RDMA_NUM_AEQ_MSIX;
+ v_wanted += pf->num_rdma_msix;
+ }
+
+ if (v_wanted > hw_num_msix) {
+ int v_remain;
+
+ dev_warn(dev, "not enough device MSI-X vectors. wanted = %d, available = %d\n",
+ v_wanted, hw_num_msix);
+
+ if (hw_num_msix < ICE_MIN_MSIX) {
+ err = -ERANGE;
+ goto exit_err;
+ }
+
+ v_remain = hw_num_msix - v_other;
+ if (v_remain < ICE_MIN_LAN_TXRX_MSIX) {
+ v_other = ICE_MIN_MSIX - ICE_MIN_LAN_TXRX_MSIX;
+ v_remain = ICE_MIN_LAN_TXRX_MSIX;
+ }
+
+ ice_reduce_msix_usage(pf, v_remain);
+ v_wanted = pf->num_lan_msix + pf->num_rdma_msix + v_other;
+
+ dev_notice(dev, "Reducing request to %d MSI-X vectors for LAN traffic.\n",
+ pf->num_lan_msix);
+ if (ice_is_rdma_ena(pf))
+ dev_notice(dev, "Reducing request to %d MSI-X vectors for RDMA.\n",
+ pf->num_rdma_msix);
+ }
+
+ pf->msix_entries = devm_kcalloc(dev, v_wanted,
+ sizeof(*pf->msix_entries), GFP_KERNEL);
+ if (!pf->msix_entries) {
+ err = -ENOMEM;
+ goto exit_err;
+ }
+
+ for (i = 0; i < v_wanted; i++)
+ pf->msix_entries[i].entry = i;
+
+ /* actually reserve the vectors */
+ v_actual = pci_enable_msix_range(pf->pdev, pf->msix_entries,
+ ICE_MIN_MSIX, v_wanted);
+ if (v_actual < 0) {
+ dev_err(dev, "unable to reserve MSI-X vectors\n");
+ err = v_actual;
+ goto msix_err;
+ }
+
+ if (v_actual < v_wanted) {
+ dev_warn(dev, "not enough OS MSI-X vectors. requested = %d, obtained = %d\n",
+ v_wanted, v_actual);
+
+ if (v_actual < ICE_MIN_MSIX) {
+ /* error if we can't get minimum vectors */
+ pci_disable_msix(pf->pdev);
+ err = -ERANGE;
+ goto msix_err;
+ } else {
+ int v_remain = v_actual - v_other;
+
+ if (v_remain < ICE_MIN_LAN_TXRX_MSIX)
+ v_remain = ICE_MIN_LAN_TXRX_MSIX;
+
+ ice_reduce_msix_usage(pf, v_remain);
+
+ dev_notice(dev, "Enabled %d MSI-X vectors for LAN traffic.\n",
+ pf->num_lan_msix);
+
+ if (ice_is_rdma_ena(pf))
+ dev_notice(dev, "Enabled %d MSI-X vectors for RDMA.\n",
+ pf->num_rdma_msix);
+ }
+ }
+
+ return v_actual;
+
+msix_err:
+ devm_kfree(dev, pf->msix_entries);
+
+exit_err:
+ pf->num_rdma_msix = 0;
+ pf->num_lan_msix = 0;
+ return err;
+}
+
+/**
+ * ice_dis_msix - Disable MSI-X interrupt setup in OS
+ * @pf: board private structure
+ */
+static void ice_dis_msix(struct ice_pf *pf)
+{
+ pci_disable_msix(pf->pdev);
+ devm_kfree(ice_pf_to_dev(pf), pf->msix_entries);
+ pf->msix_entries = NULL;
+}
+
+/**
+ * ice_clear_interrupt_scheme - Undo things done by ice_init_interrupt_scheme
+ * @pf: board private structure
+ */
+void ice_clear_interrupt_scheme(struct ice_pf *pf)
+{
+ ice_dis_msix(pf);
+
+ if (pf->irq_tracker) {
+ devm_kfree(ice_pf_to_dev(pf), pf->irq_tracker);
+ pf->irq_tracker = NULL;
+ }
+}
+
+/**
+ * ice_init_interrupt_scheme - Determine proper interrupt scheme
+ * @pf: board private structure to initialize
+ */
+int ice_init_interrupt_scheme(struct ice_pf *pf)
+{
+ int vectors;
+
+ vectors = ice_ena_msix_range(pf);
+
+ if (vectors < 0)
+ return vectors;
+
+ /* set up vector assignment tracking */
+ pf->irq_tracker = devm_kzalloc(ice_pf_to_dev(pf),
+ struct_size(pf->irq_tracker, list,
+ vectors),
+ GFP_KERNEL);
+ if (!pf->irq_tracker) {
+ ice_dis_msix(pf);
+ return -ENOMEM;
+ }
+
+ /* populate SW interrupts pool with number of OS granted IRQs. */
+ pf->num_avail_sw_msix = (u16)vectors;
+ pf->irq_tracker->num_entries = (u16)vectors;
+ pf->irq_tracker->end = pf->irq_tracker->num_entries;
+
+ return 0;
+}
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.h b/drivers/net/ethernet/intel/ice/ice_irq.h
new file mode 100644
index 000000000000..82475162ab70
--- /dev/null
+++ b/drivers/net/ethernet/intel/ice/ice_irq.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2023, Intel Corporation. */
+
+#ifndef _ICE_IRQ_H_
+#define _ICE_IRQ_H_
+
+int ice_init_interrupt_scheme(struct ice_pf *pf);
+void ice_clear_interrupt_scheme(struct ice_pf *pf);
+
+#endif
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index faaf25b7dfcb..6ed5a7a68653 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -3938,224 +3938,6 @@ static int ice_init_pf(struct ice_pf *pf)
return 0;
}
-/**
- * ice_reduce_msix_usage - Reduce usage of MSI-X vectors
- * @pf: board private structure
- * @v_remain: number of remaining MSI-X vectors to be distributed
- *
- * Reduce the usage of MSI-X vectors when entire request cannot be fulfilled.
- * pf->num_lan_msix and pf->num_rdma_msix values are set based on number of
- * remaining vectors.
- */
-static void ice_reduce_msix_usage(struct ice_pf *pf, int v_remain)
-{
- int v_rdma;
-
- if (!ice_is_rdma_ena(pf)) {
- pf->num_lan_msix = v_remain;
- return;
- }
-
- /* RDMA needs at least 1 interrupt in addition to AEQ MSIX */
- v_rdma = ICE_RDMA_NUM_AEQ_MSIX + 1;
-
- if (v_remain < ICE_MIN_LAN_TXRX_MSIX + ICE_MIN_RDMA_MSIX) {
- dev_warn(ice_pf_to_dev(pf), "Not enough MSI-X vectors to support RDMA.\n");
- clear_bit(ICE_FLAG_RDMA_ENA, pf->flags);
-
- pf->num_rdma_msix = 0;
- pf->num_lan_msix = ICE_MIN_LAN_TXRX_MSIX;
- } else if ((v_remain < ICE_MIN_LAN_TXRX_MSIX + v_rdma) ||
- (v_remain - v_rdma < v_rdma)) {
- /* Support minimum RDMA and give remaining vectors to LAN MSIX */
- pf->num_rdma_msix = ICE_MIN_RDMA_MSIX;
- pf->num_lan_msix = v_remain - ICE_MIN_RDMA_MSIX;
- } else {
- /* Split remaining MSIX with RDMA after accounting for AEQ MSIX
- */
- pf->num_rdma_msix = (v_remain - ICE_RDMA_NUM_AEQ_MSIX) / 2 +
- ICE_RDMA_NUM_AEQ_MSIX;
- pf->num_lan_msix = v_remain - pf->num_rdma_msix;
- }
-}
-
-/**
- * ice_ena_msix_range - Request a range of MSIX vectors from the OS
- * @pf: board private structure
- *
- * Compute the number of MSIX vectors wanted and request from the OS. Adjust
- * device usage if there are not enough vectors. Return the number of vectors
- * reserved or negative on failure.
- */
-static int ice_ena_msix_range(struct ice_pf *pf)
-{
- int num_cpus, hw_num_msix, v_other, v_wanted, v_actual;
- struct device *dev = ice_pf_to_dev(pf);
- int err, i;
-
- hw_num_msix = pf->hw.func_caps.common_cap.num_msix_vectors;
- num_cpus = num_online_cpus();
-
- /* LAN miscellaneous handler */
- v_other = ICE_MIN_LAN_OICR_MSIX;
-
- /* Flow Director */
- if (test_bit(ICE_FLAG_FD_ENA, pf->flags))
- v_other += ICE_FDIR_MSIX;
-
- /* switchdev */
- v_other += ICE_ESWITCH_MSIX;
-
- v_wanted = v_other;
-
- /* LAN traffic */
- pf->num_lan_msix = num_cpus;
- v_wanted += pf->num_lan_msix;
-
- /* RDMA auxiliary driver */
- if (ice_is_rdma_ena(pf)) {
- pf->num_rdma_msix = num_cpus + ICE_RDMA_NUM_AEQ_MSIX;
- v_wanted += pf->num_rdma_msix;
- }
-
- if (v_wanted > hw_num_msix) {
- int v_remain;
-
- dev_warn(dev, "not enough device MSI-X vectors. wanted = %d, available = %d\n",
- v_wanted, hw_num_msix);
-
- if (hw_num_msix < ICE_MIN_MSIX) {
- err = -ERANGE;
- goto exit_err;
- }
-
- v_remain = hw_num_msix - v_other;
- if (v_remain < ICE_MIN_LAN_TXRX_MSIX) {
- v_other = ICE_MIN_MSIX - ICE_MIN_LAN_TXRX_MSIX;
- v_remain = ICE_MIN_LAN_TXRX_MSIX;
- }
-
- ice_reduce_msix_usage(pf, v_remain);
- v_wanted = pf->num_lan_msix + pf->num_rdma_msix + v_other;
-
- dev_notice(dev, "Reducing request to %d MSI-X vectors for LAN traffic.\n",
- pf->num_lan_msix);
- if (ice_is_rdma_ena(pf))
- dev_notice(dev, "Reducing request to %d MSI-X vectors for RDMA.\n",
- pf->num_rdma_msix);
- }
-
- pf->msix_entries = devm_kcalloc(dev, v_wanted,
- sizeof(*pf->msix_entries), GFP_KERNEL);
- if (!pf->msix_entries) {
- err = -ENOMEM;
- goto exit_err;
- }
-
- for (i = 0; i < v_wanted; i++)
- pf->msix_entries[i].entry = i;
-
- /* actually reserve the vectors */
- v_actual = pci_enable_msix_range(pf->pdev, pf->msix_entries,
- ICE_MIN_MSIX, v_wanted);
- if (v_actual < 0) {
- dev_err(dev, "unable to reserve MSI-X vectors\n");
- err = v_actual;
- goto msix_err;
- }
-
- if (v_actual < v_wanted) {
- dev_warn(dev, "not enough OS MSI-X vectors. requested = %d, obtained = %d\n",
- v_wanted, v_actual);
-
- if (v_actual < ICE_MIN_MSIX) {
- /* error if we can't get minimum vectors */
- pci_disable_msix(pf->pdev);
- err = -ERANGE;
- goto msix_err;
- } else {
- int v_remain = v_actual - v_other;
-
- if (v_remain < ICE_MIN_LAN_TXRX_MSIX)
- v_remain = ICE_MIN_LAN_TXRX_MSIX;
-
- ice_reduce_msix_usage(pf, v_remain);
-
- dev_notice(dev, "Enabled %d MSI-X vectors for LAN traffic.\n",
- pf->num_lan_msix);
-
- if (ice_is_rdma_ena(pf))
- dev_notice(dev, "Enabled %d MSI-X vectors for RDMA.\n",
- pf->num_rdma_msix);
- }
- }
-
- return v_actual;
-
-msix_err:
- devm_kfree(dev, pf->msix_entries);
-
-exit_err:
- pf->num_rdma_msix = 0;
- pf->num_lan_msix = 0;
- return err;
-}
-
-/**
- * ice_dis_msix - Disable MSI-X interrupt setup in OS
- * @pf: board private structure
- */
-static void ice_dis_msix(struct ice_pf *pf)
-{
- pci_disable_msix(pf->pdev);
- devm_kfree(ice_pf_to_dev(pf), pf->msix_entries);
- pf->msix_entries = NULL;
-}
-
-/**
- * ice_clear_interrupt_scheme - Undo things done by ice_init_interrupt_scheme
- * @pf: board private structure
- */
-static void ice_clear_interrupt_scheme(struct ice_pf *pf)
-{
- ice_dis_msix(pf);
-
- if (pf->irq_tracker) {
- devm_kfree(ice_pf_to_dev(pf), pf->irq_tracker);
- pf->irq_tracker = NULL;
- }
-}
-
-/**
- * ice_init_interrupt_scheme - Determine proper interrupt scheme
- * @pf: board private structure to initialize
- */
-static int ice_init_interrupt_scheme(struct ice_pf *pf)
-{
- int vectors;
-
- vectors = ice_ena_msix_range(pf);
-
- if (vectors < 0)
- return vectors;
-
- /* set up vector assignment tracking */
- pf->irq_tracker = devm_kzalloc(ice_pf_to_dev(pf),
- struct_size(pf->irq_tracker, list, vectors),
- GFP_KERNEL);
- if (!pf->irq_tracker) {
- ice_dis_msix(pf);
- return -ENOMEM;
- }
-
- /* populate SW interrupts pool with number of OS granted IRQs. */
- pf->num_avail_sw_msix = (u16)vectors;
- pf->irq_tracker->num_entries = (u16)vectors;
- pf->irq_tracker->end = pf->irq_tracker->num_entries;
-
- return 0;
-}
-
/**
* ice_is_wol_supported - check if WoL is supported
* @hw: pointer to hardware info
--
2.38.1
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Intel-wired-lan] [PATCH net-next v3 1/8] ice: move interrupt related code to separate file
@ 2023-03-23 12:24 ` Piotr Raczynski
0 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: michal.swiatkowski, netdev, jesse.brandeburg, shiraz.saleem
Keep interrupt handling code in a dedicated file. This helps keep driver
structured better and prepares for more functionality added to this file.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
---
drivers/net/ethernet/intel/ice/Makefile | 1 +
drivers/net/ethernet/intel/ice/ice.h | 1 +
drivers/net/ethernet/intel/ice/ice_irq.c | 226 ++++++++++++++++++++++
drivers/net/ethernet/intel/ice/ice_irq.h | 10 +
drivers/net/ethernet/intel/ice/ice_main.c | 218 ---------------------
5 files changed, 238 insertions(+), 218 deletions(-)
create mode 100644 drivers/net/ethernet/intel/ice/ice_irq.c
create mode 100644 drivers/net/ethernet/intel/ice/ice_irq.h
diff --git a/drivers/net/ethernet/intel/ice/Makefile b/drivers/net/ethernet/intel/ice/Makefile
index 3290f594286e..db96ec26fdbf 100644
--- a/drivers/net/ethernet/intel/ice/Makefile
+++ b/drivers/net/ethernet/intel/ice/Makefile
@@ -18,6 +18,7 @@ ice-y := ice_main.o \
ice_txrx_lib.o \
ice_txrx.o \
ice_fltr.o \
+ ice_irq.o \
ice_pf_vsi_vlan_ops.o \
ice_vsi_vlan_ops.o \
ice_vsi_vlan_lib.o \
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 2a5632ff8081..2dc180385976 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -74,6 +74,7 @@
#include "ice_lag.h"
#include "ice_vsi_vlan_ops.h"
#include "ice_gnss.h"
+#include "ice_irq.h"
#define ICE_BAR0 0
#define ICE_REQ_DESC_MULTIPLE 32
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
new file mode 100644
index 000000000000..1fc7daec9732
--- /dev/null
+++ b/drivers/net/ethernet/intel/ice/ice_irq.c
@@ -0,0 +1,226 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2023, Intel Corporation. */
+
+#include "ice.h"
+#include "ice_lib.h"
+#include "ice_irq.h"
+
+/**
+ * ice_reduce_msix_usage - Reduce usage of MSI-X vectors
+ * @pf: board private structure
+ * @v_remain: number of remaining MSI-X vectors to be distributed
+ *
+ * Reduce the usage of MSI-X vectors when entire request cannot be fulfilled.
+ * pf->num_lan_msix and pf->num_rdma_msix values are set based on number of
+ * remaining vectors.
+ */
+static void ice_reduce_msix_usage(struct ice_pf *pf, int v_remain)
+{
+ int v_rdma;
+
+ if (!ice_is_rdma_ena(pf)) {
+ pf->num_lan_msix = v_remain;
+ return;
+ }
+
+ /* RDMA needs at least 1 interrupt in addition to AEQ MSIX */
+ v_rdma = ICE_RDMA_NUM_AEQ_MSIX + 1;
+
+ if (v_remain < ICE_MIN_LAN_TXRX_MSIX + ICE_MIN_RDMA_MSIX) {
+ dev_warn(ice_pf_to_dev(pf), "Not enough MSI-X vectors to support RDMA.\n");
+ clear_bit(ICE_FLAG_RDMA_ENA, pf->flags);
+
+ pf->num_rdma_msix = 0;
+ pf->num_lan_msix = ICE_MIN_LAN_TXRX_MSIX;
+ } else if ((v_remain < ICE_MIN_LAN_TXRX_MSIX + v_rdma) ||
+ (v_remain - v_rdma < v_rdma)) {
+ /* Support minimum RDMA and give remaining vectors to LAN MSIX
+ */
+ pf->num_rdma_msix = ICE_MIN_RDMA_MSIX;
+ pf->num_lan_msix = v_remain - ICE_MIN_RDMA_MSIX;
+ } else {
+ /* Split remaining MSIX with RDMA after accounting for AEQ MSIX
+ */
+ pf->num_rdma_msix = (v_remain - ICE_RDMA_NUM_AEQ_MSIX) / 2 +
+ ICE_RDMA_NUM_AEQ_MSIX;
+ pf->num_lan_msix = v_remain - pf->num_rdma_msix;
+ }
+}
+
+/**
+ * ice_ena_msix_range - Request a range of MSIX vectors from the OS
+ * @pf: board private structure
+ *
+ * Compute the number of MSIX vectors wanted and request from the OS. Adjust
+ * device usage if there are not enough vectors. Return the number of vectors
+ * reserved or negative on failure.
+ */
+static int ice_ena_msix_range(struct ice_pf *pf)
+{
+ int num_cpus, hw_num_msix, v_other, v_wanted, v_actual;
+ struct device *dev = ice_pf_to_dev(pf);
+ int err, i;
+
+ hw_num_msix = pf->hw.func_caps.common_cap.num_msix_vectors;
+ num_cpus = num_online_cpus();
+
+ /* LAN miscellaneous handler */
+ v_other = ICE_MIN_LAN_OICR_MSIX;
+
+ /* Flow Director */
+ if (test_bit(ICE_FLAG_FD_ENA, pf->flags))
+ v_other += ICE_FDIR_MSIX;
+
+ /* switchdev */
+ v_other += ICE_ESWITCH_MSIX;
+
+ v_wanted = v_other;
+
+ /* LAN traffic */
+ pf->num_lan_msix = num_cpus;
+ v_wanted += pf->num_lan_msix;
+
+ /* RDMA auxiliary driver */
+ if (ice_is_rdma_ena(pf)) {
+ pf->num_rdma_msix = num_cpus + ICE_RDMA_NUM_AEQ_MSIX;
+ v_wanted += pf->num_rdma_msix;
+ }
+
+ if (v_wanted > hw_num_msix) {
+ int v_remain;
+
+ dev_warn(dev, "not enough device MSI-X vectors. wanted = %d, available = %d\n",
+ v_wanted, hw_num_msix);
+
+ if (hw_num_msix < ICE_MIN_MSIX) {
+ err = -ERANGE;
+ goto exit_err;
+ }
+
+ v_remain = hw_num_msix - v_other;
+ if (v_remain < ICE_MIN_LAN_TXRX_MSIX) {
+ v_other = ICE_MIN_MSIX - ICE_MIN_LAN_TXRX_MSIX;
+ v_remain = ICE_MIN_LAN_TXRX_MSIX;
+ }
+
+ ice_reduce_msix_usage(pf, v_remain);
+ v_wanted = pf->num_lan_msix + pf->num_rdma_msix + v_other;
+
+ dev_notice(dev, "Reducing request to %d MSI-X vectors for LAN traffic.\n",
+ pf->num_lan_msix);
+ if (ice_is_rdma_ena(pf))
+ dev_notice(dev, "Reducing request to %d MSI-X vectors for RDMA.\n",
+ pf->num_rdma_msix);
+ }
+
+ pf->msix_entries = devm_kcalloc(dev, v_wanted,
+ sizeof(*pf->msix_entries), GFP_KERNEL);
+ if (!pf->msix_entries) {
+ err = -ENOMEM;
+ goto exit_err;
+ }
+
+ for (i = 0; i < v_wanted; i++)
+ pf->msix_entries[i].entry = i;
+
+ /* actually reserve the vectors */
+ v_actual = pci_enable_msix_range(pf->pdev, pf->msix_entries,
+ ICE_MIN_MSIX, v_wanted);
+ if (v_actual < 0) {
+ dev_err(dev, "unable to reserve MSI-X vectors\n");
+ err = v_actual;
+ goto msix_err;
+ }
+
+ if (v_actual < v_wanted) {
+ dev_warn(dev, "not enough OS MSI-X vectors. requested = %d, obtained = %d\n",
+ v_wanted, v_actual);
+
+ if (v_actual < ICE_MIN_MSIX) {
+ /* error if we can't get minimum vectors */
+ pci_disable_msix(pf->pdev);
+ err = -ERANGE;
+ goto msix_err;
+ } else {
+ int v_remain = v_actual - v_other;
+
+ if (v_remain < ICE_MIN_LAN_TXRX_MSIX)
+ v_remain = ICE_MIN_LAN_TXRX_MSIX;
+
+ ice_reduce_msix_usage(pf, v_remain);
+
+ dev_notice(dev, "Enabled %d MSI-X vectors for LAN traffic.\n",
+ pf->num_lan_msix);
+
+ if (ice_is_rdma_ena(pf))
+ dev_notice(dev, "Enabled %d MSI-X vectors for RDMA.\n",
+ pf->num_rdma_msix);
+ }
+ }
+
+ return v_actual;
+
+msix_err:
+ devm_kfree(dev, pf->msix_entries);
+
+exit_err:
+ pf->num_rdma_msix = 0;
+ pf->num_lan_msix = 0;
+ return err;
+}
+
+/**
+ * ice_dis_msix - Disable MSI-X interrupt setup in OS
+ * @pf: board private structure
+ */
+static void ice_dis_msix(struct ice_pf *pf)
+{
+ pci_disable_msix(pf->pdev);
+ devm_kfree(ice_pf_to_dev(pf), pf->msix_entries);
+ pf->msix_entries = NULL;
+}
+
+/**
+ * ice_clear_interrupt_scheme - Undo things done by ice_init_interrupt_scheme
+ * @pf: board private structure
+ */
+void ice_clear_interrupt_scheme(struct ice_pf *pf)
+{
+ ice_dis_msix(pf);
+
+ if (pf->irq_tracker) {
+ devm_kfree(ice_pf_to_dev(pf), pf->irq_tracker);
+ pf->irq_tracker = NULL;
+ }
+}
+
+/**
+ * ice_init_interrupt_scheme - Determine proper interrupt scheme
+ * @pf: board private structure to initialize
+ */
+int ice_init_interrupt_scheme(struct ice_pf *pf)
+{
+ int vectors;
+
+ vectors = ice_ena_msix_range(pf);
+
+ if (vectors < 0)
+ return vectors;
+
+ /* set up vector assignment tracking */
+ pf->irq_tracker = devm_kzalloc(ice_pf_to_dev(pf),
+ struct_size(pf->irq_tracker, list,
+ vectors),
+ GFP_KERNEL);
+ if (!pf->irq_tracker) {
+ ice_dis_msix(pf);
+ return -ENOMEM;
+ }
+
+ /* populate SW interrupts pool with number of OS granted IRQs. */
+ pf->num_avail_sw_msix = (u16)vectors;
+ pf->irq_tracker->num_entries = (u16)vectors;
+ pf->irq_tracker->end = pf->irq_tracker->num_entries;
+
+ return 0;
+}
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.h b/drivers/net/ethernet/intel/ice/ice_irq.h
new file mode 100644
index 000000000000..82475162ab70
--- /dev/null
+++ b/drivers/net/ethernet/intel/ice/ice_irq.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (C) 2023, Intel Corporation. */
+
+#ifndef _ICE_IRQ_H_
+#define _ICE_IRQ_H_
+
+int ice_init_interrupt_scheme(struct ice_pf *pf);
+void ice_clear_interrupt_scheme(struct ice_pf *pf);
+
+#endif
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index faaf25b7dfcb..6ed5a7a68653 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -3938,224 +3938,6 @@ static int ice_init_pf(struct ice_pf *pf)
return 0;
}
-/**
- * ice_reduce_msix_usage - Reduce usage of MSI-X vectors
- * @pf: board private structure
- * @v_remain: number of remaining MSI-X vectors to be distributed
- *
- * Reduce the usage of MSI-X vectors when entire request cannot be fulfilled.
- * pf->num_lan_msix and pf->num_rdma_msix values are set based on number of
- * remaining vectors.
- */
-static void ice_reduce_msix_usage(struct ice_pf *pf, int v_remain)
-{
- int v_rdma;
-
- if (!ice_is_rdma_ena(pf)) {
- pf->num_lan_msix = v_remain;
- return;
- }
-
- /* RDMA needs at least 1 interrupt in addition to AEQ MSIX */
- v_rdma = ICE_RDMA_NUM_AEQ_MSIX + 1;
-
- if (v_remain < ICE_MIN_LAN_TXRX_MSIX + ICE_MIN_RDMA_MSIX) {
- dev_warn(ice_pf_to_dev(pf), "Not enough MSI-X vectors to support RDMA.\n");
- clear_bit(ICE_FLAG_RDMA_ENA, pf->flags);
-
- pf->num_rdma_msix = 0;
- pf->num_lan_msix = ICE_MIN_LAN_TXRX_MSIX;
- } else if ((v_remain < ICE_MIN_LAN_TXRX_MSIX + v_rdma) ||
- (v_remain - v_rdma < v_rdma)) {
- /* Support minimum RDMA and give remaining vectors to LAN MSIX */
- pf->num_rdma_msix = ICE_MIN_RDMA_MSIX;
- pf->num_lan_msix = v_remain - ICE_MIN_RDMA_MSIX;
- } else {
- /* Split remaining MSIX with RDMA after accounting for AEQ MSIX
- */
- pf->num_rdma_msix = (v_remain - ICE_RDMA_NUM_AEQ_MSIX) / 2 +
- ICE_RDMA_NUM_AEQ_MSIX;
- pf->num_lan_msix = v_remain - pf->num_rdma_msix;
- }
-}
-
-/**
- * ice_ena_msix_range - Request a range of MSIX vectors from the OS
- * @pf: board private structure
- *
- * Compute the number of MSIX vectors wanted and request from the OS. Adjust
- * device usage if there are not enough vectors. Return the number of vectors
- * reserved or negative on failure.
- */
-static int ice_ena_msix_range(struct ice_pf *pf)
-{
- int num_cpus, hw_num_msix, v_other, v_wanted, v_actual;
- struct device *dev = ice_pf_to_dev(pf);
- int err, i;
-
- hw_num_msix = pf->hw.func_caps.common_cap.num_msix_vectors;
- num_cpus = num_online_cpus();
-
- /* LAN miscellaneous handler */
- v_other = ICE_MIN_LAN_OICR_MSIX;
-
- /* Flow Director */
- if (test_bit(ICE_FLAG_FD_ENA, pf->flags))
- v_other += ICE_FDIR_MSIX;
-
- /* switchdev */
- v_other += ICE_ESWITCH_MSIX;
-
- v_wanted = v_other;
-
- /* LAN traffic */
- pf->num_lan_msix = num_cpus;
- v_wanted += pf->num_lan_msix;
-
- /* RDMA auxiliary driver */
- if (ice_is_rdma_ena(pf)) {
- pf->num_rdma_msix = num_cpus + ICE_RDMA_NUM_AEQ_MSIX;
- v_wanted += pf->num_rdma_msix;
- }
-
- if (v_wanted > hw_num_msix) {
- int v_remain;
-
- dev_warn(dev, "not enough device MSI-X vectors. wanted = %d, available = %d\n",
- v_wanted, hw_num_msix);
-
- if (hw_num_msix < ICE_MIN_MSIX) {
- err = -ERANGE;
- goto exit_err;
- }
-
- v_remain = hw_num_msix - v_other;
- if (v_remain < ICE_MIN_LAN_TXRX_MSIX) {
- v_other = ICE_MIN_MSIX - ICE_MIN_LAN_TXRX_MSIX;
- v_remain = ICE_MIN_LAN_TXRX_MSIX;
- }
-
- ice_reduce_msix_usage(pf, v_remain);
- v_wanted = pf->num_lan_msix + pf->num_rdma_msix + v_other;
-
- dev_notice(dev, "Reducing request to %d MSI-X vectors for LAN traffic.\n",
- pf->num_lan_msix);
- if (ice_is_rdma_ena(pf))
- dev_notice(dev, "Reducing request to %d MSI-X vectors for RDMA.\n",
- pf->num_rdma_msix);
- }
-
- pf->msix_entries = devm_kcalloc(dev, v_wanted,
- sizeof(*pf->msix_entries), GFP_KERNEL);
- if (!pf->msix_entries) {
- err = -ENOMEM;
- goto exit_err;
- }
-
- for (i = 0; i < v_wanted; i++)
- pf->msix_entries[i].entry = i;
-
- /* actually reserve the vectors */
- v_actual = pci_enable_msix_range(pf->pdev, pf->msix_entries,
- ICE_MIN_MSIX, v_wanted);
- if (v_actual < 0) {
- dev_err(dev, "unable to reserve MSI-X vectors\n");
- err = v_actual;
- goto msix_err;
- }
-
- if (v_actual < v_wanted) {
- dev_warn(dev, "not enough OS MSI-X vectors. requested = %d, obtained = %d\n",
- v_wanted, v_actual);
-
- if (v_actual < ICE_MIN_MSIX) {
- /* error if we can't get minimum vectors */
- pci_disable_msix(pf->pdev);
- err = -ERANGE;
- goto msix_err;
- } else {
- int v_remain = v_actual - v_other;
-
- if (v_remain < ICE_MIN_LAN_TXRX_MSIX)
- v_remain = ICE_MIN_LAN_TXRX_MSIX;
-
- ice_reduce_msix_usage(pf, v_remain);
-
- dev_notice(dev, "Enabled %d MSI-X vectors for LAN traffic.\n",
- pf->num_lan_msix);
-
- if (ice_is_rdma_ena(pf))
- dev_notice(dev, "Enabled %d MSI-X vectors for RDMA.\n",
- pf->num_rdma_msix);
- }
- }
-
- return v_actual;
-
-msix_err:
- devm_kfree(dev, pf->msix_entries);
-
-exit_err:
- pf->num_rdma_msix = 0;
- pf->num_lan_msix = 0;
- return err;
-}
-
-/**
- * ice_dis_msix - Disable MSI-X interrupt setup in OS
- * @pf: board private structure
- */
-static void ice_dis_msix(struct ice_pf *pf)
-{
- pci_disable_msix(pf->pdev);
- devm_kfree(ice_pf_to_dev(pf), pf->msix_entries);
- pf->msix_entries = NULL;
-}
-
-/**
- * ice_clear_interrupt_scheme - Undo things done by ice_init_interrupt_scheme
- * @pf: board private structure
- */
-static void ice_clear_interrupt_scheme(struct ice_pf *pf)
-{
- ice_dis_msix(pf);
-
- if (pf->irq_tracker) {
- devm_kfree(ice_pf_to_dev(pf), pf->irq_tracker);
- pf->irq_tracker = NULL;
- }
-}
-
-/**
- * ice_init_interrupt_scheme - Determine proper interrupt scheme
- * @pf: board private structure to initialize
- */
-static int ice_init_interrupt_scheme(struct ice_pf *pf)
-{
- int vectors;
-
- vectors = ice_ena_msix_range(pf);
-
- if (vectors < 0)
- return vectors;
-
- /* set up vector assignment tracking */
- pf->irq_tracker = devm_kzalloc(ice_pf_to_dev(pf),
- struct_size(pf->irq_tracker, list, vectors),
- GFP_KERNEL);
- if (!pf->irq_tracker) {
- ice_dis_msix(pf);
- return -ENOMEM;
- }
-
- /* populate SW interrupts pool with number of OS granted IRQs. */
- pf->num_avail_sw_msix = (u16)vectors;
- pf->irq_tracker->num_entries = (u16)vectors;
- pf->irq_tracker->end = pf->irq_tracker->num_entries;
-
- return 0;
-}
-
/**
* ice_is_wol_supported - check if WoL is supported
* @hw: pointer to hardware info
--
2.38.1
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH net-next v3 2/8] ice: use pci_irq_vector helper function
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-03-23 12:24 ` Piotr Raczynski
-1 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, michal.swiatkowski, shiraz.saleem, jacob.e.keller,
sridhar.samudrala, jesse.brandeburg, aleksander.lobakin,
lukasz.czapnik, Piotr Raczynski, Michal Swiatkowski
Currently, driver gets interrupt number directly from ice_pf::msix_entries
array. Use helper function dedicated to do just that.
While at it use a variable to store interrupt number in
ice_free_irq_msix_misc instead of calling the helper function twice.
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
---
drivers/net/ethernet/intel/ice/ice_arfs.c | 2 +-
drivers/net/ethernet/intel/ice/ice_lib.c | 4 ++--
drivers/net/ethernet/intel/ice/ice_main.c | 12 ++++++------
drivers/net/ethernet/intel/ice/ice_ptp.c | 2 +-
drivers/net/ethernet/intel/ice/ice_xsk.c | 2 +-
5 files changed, 11 insertions(+), 11 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_arfs.c b/drivers/net/ethernet/intel/ice/ice_arfs.c
index b38710b2540f..59146e4bdf3b 100644
--- a/drivers/net/ethernet/intel/ice/ice_arfs.c
+++ b/drivers/net/ethernet/intel/ice/ice_arfs.c
@@ -654,7 +654,7 @@ int ice_set_cpu_rx_rmap(struct ice_vsi *vsi)
base_idx = vsi->base_vector;
ice_for_each_q_vector(vsi, i)
if (irq_cpu_rmap_add(netdev->rx_cpu_rmap,
- pf->msix_entries[base_idx + i].vector)) {
+ pci_irq_vector(pf->pdev, base_idx + i))) {
ice_free_cpu_rx_rmap(vsi);
return -EINVAL;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 8d229011ffc3..a707c0f7ae29 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -3058,7 +3058,7 @@ void ice_vsi_free_irq(struct ice_vsi *vsi)
u16 vector = i + base;
int irq_num;
- irq_num = pf->msix_entries[vector].vector;
+ irq_num = pci_irq_vector(pf->pdev, vector);
/* free only the irqs that were actually requested */
if (!vsi->q_vectors[i] ||
@@ -3237,7 +3237,7 @@ void ice_vsi_dis_irq(struct ice_vsi *vsi)
return;
ice_for_each_q_vector(vsi, i)
- synchronize_irq(pf->msix_entries[i + base].vector);
+ synchronize_irq(pci_irq_vector(pf->pdev, i + base));
}
/**
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 6ed5a7a68653..f3032096e5dd 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -2539,7 +2539,7 @@ static int ice_vsi_req_irq_msix(struct ice_vsi *vsi, char *basename)
for (vector = 0; vector < q_vectors; vector++) {
struct ice_q_vector *q_vector = vsi->q_vectors[vector];
- irq_num = pf->msix_entries[base + vector].vector;
+ irq_num = pci_irq_vector(pf->pdev, base + vector);
if (q_vector->tx.tx_ring && q_vector->rx.rx_ring) {
snprintf(q_vector->name, sizeof(q_vector->name) - 1,
@@ -2595,7 +2595,7 @@ static int ice_vsi_req_irq_msix(struct ice_vsi *vsi, char *basename)
free_q_irqs:
while (vector) {
vector--;
- irq_num = pf->msix_entries[base + vector].vector;
+ irq_num = pci_irq_vector(pf->pdev, base + vector);
if (!IS_ENABLED(CONFIG_RFS_ACCEL))
irq_set_affinity_notifier(irq_num, NULL);
irq_set_affinity_hint(irq_num, NULL);
@@ -3272,6 +3272,7 @@ static void ice_dis_ctrlq_interrupts(struct ice_hw *hw)
*/
static void ice_free_irq_msix_misc(struct ice_pf *pf)
{
+ int misc_irq_num = pci_irq_vector(pf->pdev, pf->oicr_idx);
struct ice_hw *hw = &pf->hw;
ice_dis_ctrlq_interrupts(hw);
@@ -3281,9 +3282,8 @@ static void ice_free_irq_msix_misc(struct ice_pf *pf)
ice_flush(hw);
if (pf->msix_entries) {
- synchronize_irq(pf->msix_entries[pf->oicr_idx].vector);
- devm_free_irq(ice_pf_to_dev(pf),
- pf->msix_entries[pf->oicr_idx].vector, pf);
+ synchronize_irq(misc_irq_num);
+ devm_free_irq(ice_pf_to_dev(pf), misc_irq_num, pf);
}
pf->num_avail_sw_msix += 1;
@@ -3355,7 +3355,7 @@ static int ice_req_irq_msix_misc(struct ice_pf *pf)
pf->oicr_idx = (u16)oicr_idx;
err = devm_request_threaded_irq(dev,
- pf->msix_entries[pf->oicr_idx].vector,
+ pci_irq_vector(pf->pdev, pf->oicr_idx),
ice_misc_intr, ice_misc_intr_thread_fn,
0, pf->int_name, pf);
if (err) {
diff --git a/drivers/net/ethernet/intel/ice/ice_ptp.c b/drivers/net/ethernet/intel/ice/ice_ptp.c
index ac6f06f9a2ed..972d4f6fd615 100644
--- a/drivers/net/ethernet/intel/ice/ice_ptp.c
+++ b/drivers/net/ethernet/intel/ice/ice_ptp.c
@@ -911,7 +911,7 @@ ice_ptp_release_tx_tracker(struct ice_pf *pf, struct ice_ptp_tx *tx)
spin_unlock(&tx->lock);
/* wait for potentially outstanding interrupt to complete */
- synchronize_irq(pf->msix_entries[pf->oicr_idx].vector);
+ synchronize_irq(pci_irq_vector(pf->pdev, pf->oicr_idx));
ice_ptp_flush_tx_tracker(pf, tx);
diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c
index d1e489da7363..4102416d7a41 100644
--- a/drivers/net/ethernet/intel/ice/ice_xsk.c
+++ b/drivers/net/ethernet/intel/ice/ice_xsk.c
@@ -107,7 +107,7 @@ ice_qvec_dis_irq(struct ice_vsi *vsi, struct ice_rx_ring *rx_ring,
wr32(hw, GLINT_DYN_CTL(q_vector->reg_idx), 0);
ice_flush(hw);
- synchronize_irq(pf->msix_entries[v_idx + base].vector);
+ synchronize_irq(pci_irq_vector(pf->pdev, v_idx + base));
}
}
--
2.38.1
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Intel-wired-lan] [PATCH net-next v3 2/8] ice: use pci_irq_vector helper function
@ 2023-03-23 12:24 ` Piotr Raczynski
0 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: michal.swiatkowski, netdev, jesse.brandeburg, shiraz.saleem
Currently, driver gets interrupt number directly from ice_pf::msix_entries
array. Use helper function dedicated to do just that.
While at it use a variable to store interrupt number in
ice_free_irq_msix_misc instead of calling the helper function twice.
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
---
drivers/net/ethernet/intel/ice/ice_arfs.c | 2 +-
drivers/net/ethernet/intel/ice/ice_lib.c | 4 ++--
drivers/net/ethernet/intel/ice/ice_main.c | 12 ++++++------
drivers/net/ethernet/intel/ice/ice_ptp.c | 2 +-
drivers/net/ethernet/intel/ice/ice_xsk.c | 2 +-
5 files changed, 11 insertions(+), 11 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_arfs.c b/drivers/net/ethernet/intel/ice/ice_arfs.c
index b38710b2540f..59146e4bdf3b 100644
--- a/drivers/net/ethernet/intel/ice/ice_arfs.c
+++ b/drivers/net/ethernet/intel/ice/ice_arfs.c
@@ -654,7 +654,7 @@ int ice_set_cpu_rx_rmap(struct ice_vsi *vsi)
base_idx = vsi->base_vector;
ice_for_each_q_vector(vsi, i)
if (irq_cpu_rmap_add(netdev->rx_cpu_rmap,
- pf->msix_entries[base_idx + i].vector)) {
+ pci_irq_vector(pf->pdev, base_idx + i))) {
ice_free_cpu_rx_rmap(vsi);
return -EINVAL;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 8d229011ffc3..a707c0f7ae29 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -3058,7 +3058,7 @@ void ice_vsi_free_irq(struct ice_vsi *vsi)
u16 vector = i + base;
int irq_num;
- irq_num = pf->msix_entries[vector].vector;
+ irq_num = pci_irq_vector(pf->pdev, vector);
/* free only the irqs that were actually requested */
if (!vsi->q_vectors[i] ||
@@ -3237,7 +3237,7 @@ void ice_vsi_dis_irq(struct ice_vsi *vsi)
return;
ice_for_each_q_vector(vsi, i)
- synchronize_irq(pf->msix_entries[i + base].vector);
+ synchronize_irq(pci_irq_vector(pf->pdev, i + base));
}
/**
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 6ed5a7a68653..f3032096e5dd 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -2539,7 +2539,7 @@ static int ice_vsi_req_irq_msix(struct ice_vsi *vsi, char *basename)
for (vector = 0; vector < q_vectors; vector++) {
struct ice_q_vector *q_vector = vsi->q_vectors[vector];
- irq_num = pf->msix_entries[base + vector].vector;
+ irq_num = pci_irq_vector(pf->pdev, base + vector);
if (q_vector->tx.tx_ring && q_vector->rx.rx_ring) {
snprintf(q_vector->name, sizeof(q_vector->name) - 1,
@@ -2595,7 +2595,7 @@ static int ice_vsi_req_irq_msix(struct ice_vsi *vsi, char *basename)
free_q_irqs:
while (vector) {
vector--;
- irq_num = pf->msix_entries[base + vector].vector;
+ irq_num = pci_irq_vector(pf->pdev, base + vector);
if (!IS_ENABLED(CONFIG_RFS_ACCEL))
irq_set_affinity_notifier(irq_num, NULL);
irq_set_affinity_hint(irq_num, NULL);
@@ -3272,6 +3272,7 @@ static void ice_dis_ctrlq_interrupts(struct ice_hw *hw)
*/
static void ice_free_irq_msix_misc(struct ice_pf *pf)
{
+ int misc_irq_num = pci_irq_vector(pf->pdev, pf->oicr_idx);
struct ice_hw *hw = &pf->hw;
ice_dis_ctrlq_interrupts(hw);
@@ -3281,9 +3282,8 @@ static void ice_free_irq_msix_misc(struct ice_pf *pf)
ice_flush(hw);
if (pf->msix_entries) {
- synchronize_irq(pf->msix_entries[pf->oicr_idx].vector);
- devm_free_irq(ice_pf_to_dev(pf),
- pf->msix_entries[pf->oicr_idx].vector, pf);
+ synchronize_irq(misc_irq_num);
+ devm_free_irq(ice_pf_to_dev(pf), misc_irq_num, pf);
}
pf->num_avail_sw_msix += 1;
@@ -3355,7 +3355,7 @@ static int ice_req_irq_msix_misc(struct ice_pf *pf)
pf->oicr_idx = (u16)oicr_idx;
err = devm_request_threaded_irq(dev,
- pf->msix_entries[pf->oicr_idx].vector,
+ pci_irq_vector(pf->pdev, pf->oicr_idx),
ice_misc_intr, ice_misc_intr_thread_fn,
0, pf->int_name, pf);
if (err) {
diff --git a/drivers/net/ethernet/intel/ice/ice_ptp.c b/drivers/net/ethernet/intel/ice/ice_ptp.c
index ac6f06f9a2ed..972d4f6fd615 100644
--- a/drivers/net/ethernet/intel/ice/ice_ptp.c
+++ b/drivers/net/ethernet/intel/ice/ice_ptp.c
@@ -911,7 +911,7 @@ ice_ptp_release_tx_tracker(struct ice_pf *pf, struct ice_ptp_tx *tx)
spin_unlock(&tx->lock);
/* wait for potentially outstanding interrupt to complete */
- synchronize_irq(pf->msix_entries[pf->oicr_idx].vector);
+ synchronize_irq(pci_irq_vector(pf->pdev, pf->oicr_idx));
ice_ptp_flush_tx_tracker(pf, tx);
diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c
index d1e489da7363..4102416d7a41 100644
--- a/drivers/net/ethernet/intel/ice/ice_xsk.c
+++ b/drivers/net/ethernet/intel/ice/ice_xsk.c
@@ -107,7 +107,7 @@ ice_qvec_dis_irq(struct ice_vsi *vsi, struct ice_rx_ring *rx_ring,
wr32(hw, GLINT_DYN_CTL(q_vector->reg_idx), 0);
ice_flush(hw);
- synchronize_irq(pf->msix_entries[v_idx + base].vector);
+ synchronize_irq(pci_irq_vector(pf->pdev, v_idx + base));
}
}
--
2.38.1
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH net-next v3 3/8] ice: use preferred MSIX allocation api
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-03-23 12:24 ` Piotr Raczynski
-1 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, michal.swiatkowski, shiraz.saleem, jacob.e.keller,
sridhar.samudrala, jesse.brandeburg, aleksander.lobakin,
lukasz.czapnik, Piotr Raczynski, Michal Swiatkowski
Move away from using pci_enable_msix_range/pci_disable_msix and use
pci_alloc_irq_vectors/pci_free_irq_vectors instead.
As a result stop tracking msix_entries since with newer API entries are
handled by MSIX core. However, due to current design of communication
with RDMA driver which accesses ice_pf::msix_entries directly, keep
using the array just for RDMA driver use.
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
---
drivers/net/ethernet/intel/ice/ice_idc.c | 29 ++++++++++++++--
drivers/net/ethernet/intel/ice/ice_irq.c | 40 +++++------------------
drivers/net/ethernet/intel/ice/ice_main.c | 6 ++--
3 files changed, 37 insertions(+), 38 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_idc.c b/drivers/net/ethernet/intel/ice/ice_idc.c
index e6bc2285071e..1000759505d7 100644
--- a/drivers/net/ethernet/intel/ice/ice_idc.c
+++ b/drivers/net/ethernet/intel/ice/ice_idc.c
@@ -235,14 +235,33 @@ EXPORT_SYMBOL_GPL(ice_get_qos_params);
static int ice_reserve_rdma_qvector(struct ice_pf *pf)
{
if (ice_is_rdma_ena(pf)) {
- int index;
+ int index, i;
index = ice_get_res(pf, pf->irq_tracker, pf->num_rdma_msix,
ICE_RES_RDMA_VEC_ID);
if (index < 0)
return index;
+
+ pf->msix_entries = kcalloc(pf->num_rdma_msix,
+ sizeof(*pf->msix_entries),
+ GFP_KERNEL);
+ if (!pf->msix_entries) {
+ ice_free_res(pf->irq_tracker, pf->rdma_base_vector,
+ ICE_RES_RDMA_VEC_ID);
+ return -ENOMEM;
+ }
+
pf->num_avail_sw_msix -= pf->num_rdma_msix;
- pf->rdma_base_vector = (u16)index;
+
+ /* RDMA is the only user of pf->msix_entries array */
+ pf->rdma_base_vector = 0;
+
+ for (i = 0; i < pf->num_rdma_msix; i++, index++) {
+ struct msix_entry *entry = &pf->msix_entries[i];
+
+ entry->entry = index;
+ entry->vector = pci_irq_vector(pf->pdev, index);
+ }
}
return 0;
}
@@ -253,6 +272,12 @@ static int ice_reserve_rdma_qvector(struct ice_pf *pf)
*/
static void ice_free_rdma_qvector(struct ice_pf *pf)
{
+ if (!pf->msix_entries)
+ return;
+
+ kfree(pf->msix_entries);
+ pf->msix_entries = NULL;
+
pf->num_avail_sw_msix -= pf->num_rdma_msix;
ice_free_res(pf->irq_tracker, pf->rdma_base_vector,
ICE_RES_RDMA_VEC_ID);
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
index 1fc7daec9732..f61be5d76373 100644
--- a/drivers/net/ethernet/intel/ice/ice_irq.c
+++ b/drivers/net/ethernet/intel/ice/ice_irq.c
@@ -59,7 +59,7 @@ static int ice_ena_msix_range(struct ice_pf *pf)
{
int num_cpus, hw_num_msix, v_other, v_wanted, v_actual;
struct device *dev = ice_pf_to_dev(pf);
- int err, i;
+ int err;
hw_num_msix = pf->hw.func_caps.common_cap.num_msix_vectors;
num_cpus = num_online_cpus();
@@ -113,23 +113,13 @@ static int ice_ena_msix_range(struct ice_pf *pf)
pf->num_rdma_msix);
}
- pf->msix_entries = devm_kcalloc(dev, v_wanted,
- sizeof(*pf->msix_entries), GFP_KERNEL);
- if (!pf->msix_entries) {
- err = -ENOMEM;
- goto exit_err;
- }
-
- for (i = 0; i < v_wanted; i++)
- pf->msix_entries[i].entry = i;
-
/* actually reserve the vectors */
- v_actual = pci_enable_msix_range(pf->pdev, pf->msix_entries,
- ICE_MIN_MSIX, v_wanted);
+ v_actual = pci_alloc_irq_vectors(pf->pdev, ICE_MIN_MSIX, v_wanted,
+ PCI_IRQ_MSIX);
if (v_actual < 0) {
dev_err(dev, "unable to reserve MSI-X vectors\n");
err = v_actual;
- goto msix_err;
+ goto exit_err;
}
if (v_actual < v_wanted) {
@@ -138,9 +128,9 @@ static int ice_ena_msix_range(struct ice_pf *pf)
if (v_actual < ICE_MIN_MSIX) {
/* error if we can't get minimum vectors */
- pci_disable_msix(pf->pdev);
+ pci_free_irq_vectors(pf->pdev);
err = -ERANGE;
- goto msix_err;
+ goto exit_err;
} else {
int v_remain = v_actual - v_other;
@@ -160,33 +150,19 @@ static int ice_ena_msix_range(struct ice_pf *pf)
return v_actual;
-msix_err:
- devm_kfree(dev, pf->msix_entries);
-
exit_err:
pf->num_rdma_msix = 0;
pf->num_lan_msix = 0;
return err;
}
-/**
- * ice_dis_msix - Disable MSI-X interrupt setup in OS
- * @pf: board private structure
- */
-static void ice_dis_msix(struct ice_pf *pf)
-{
- pci_disable_msix(pf->pdev);
- devm_kfree(ice_pf_to_dev(pf), pf->msix_entries);
- pf->msix_entries = NULL;
-}
-
/**
* ice_clear_interrupt_scheme - Undo things done by ice_init_interrupt_scheme
* @pf: board private structure
*/
void ice_clear_interrupt_scheme(struct ice_pf *pf)
{
- ice_dis_msix(pf);
+ pci_free_irq_vectors(pf->pdev);
if (pf->irq_tracker) {
devm_kfree(ice_pf_to_dev(pf), pf->irq_tracker);
@@ -213,7 +189,7 @@ int ice_init_interrupt_scheme(struct ice_pf *pf)
vectors),
GFP_KERNEL);
if (!pf->irq_tracker) {
- ice_dis_msix(pf);
+ pci_free_irq_vectors(pf->pdev);
return -ENOMEM;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index f3032096e5dd..9ccb6092b937 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -3281,10 +3281,8 @@ static void ice_free_irq_msix_misc(struct ice_pf *pf)
wr32(hw, PFINT_OICR_ENA, 0);
ice_flush(hw);
- if (pf->msix_entries) {
- synchronize_irq(misc_irq_num);
- devm_free_irq(ice_pf_to_dev(pf), misc_irq_num, pf);
- }
+ synchronize_irq(misc_irq_num);
+ devm_free_irq(ice_pf_to_dev(pf), misc_irq_num, pf);
pf->num_avail_sw_msix += 1;
ice_free_res(pf->irq_tracker, pf->oicr_idx, ICE_RES_MISC_VEC_ID);
--
2.38.1
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Intel-wired-lan] [PATCH net-next v3 3/8] ice: use preferred MSIX allocation api
@ 2023-03-23 12:24 ` Piotr Raczynski
0 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: michal.swiatkowski, netdev, jesse.brandeburg, shiraz.saleem
Move away from using pci_enable_msix_range/pci_disable_msix and use
pci_alloc_irq_vectors/pci_free_irq_vectors instead.
As a result stop tracking msix_entries since with newer API entries are
handled by MSIX core. However, due to current design of communication
with RDMA driver which accesses ice_pf::msix_entries directly, keep
using the array just for RDMA driver use.
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
---
drivers/net/ethernet/intel/ice/ice_idc.c | 29 ++++++++++++++--
drivers/net/ethernet/intel/ice/ice_irq.c | 40 +++++------------------
drivers/net/ethernet/intel/ice/ice_main.c | 6 ++--
3 files changed, 37 insertions(+), 38 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_idc.c b/drivers/net/ethernet/intel/ice/ice_idc.c
index e6bc2285071e..1000759505d7 100644
--- a/drivers/net/ethernet/intel/ice/ice_idc.c
+++ b/drivers/net/ethernet/intel/ice/ice_idc.c
@@ -235,14 +235,33 @@ EXPORT_SYMBOL_GPL(ice_get_qos_params);
static int ice_reserve_rdma_qvector(struct ice_pf *pf)
{
if (ice_is_rdma_ena(pf)) {
- int index;
+ int index, i;
index = ice_get_res(pf, pf->irq_tracker, pf->num_rdma_msix,
ICE_RES_RDMA_VEC_ID);
if (index < 0)
return index;
+
+ pf->msix_entries = kcalloc(pf->num_rdma_msix,
+ sizeof(*pf->msix_entries),
+ GFP_KERNEL);
+ if (!pf->msix_entries) {
+ ice_free_res(pf->irq_tracker, pf->rdma_base_vector,
+ ICE_RES_RDMA_VEC_ID);
+ return -ENOMEM;
+ }
+
pf->num_avail_sw_msix -= pf->num_rdma_msix;
- pf->rdma_base_vector = (u16)index;
+
+ /* RDMA is the only user of pf->msix_entries array */
+ pf->rdma_base_vector = 0;
+
+ for (i = 0; i < pf->num_rdma_msix; i++, index++) {
+ struct msix_entry *entry = &pf->msix_entries[i];
+
+ entry->entry = index;
+ entry->vector = pci_irq_vector(pf->pdev, index);
+ }
}
return 0;
}
@@ -253,6 +272,12 @@ static int ice_reserve_rdma_qvector(struct ice_pf *pf)
*/
static void ice_free_rdma_qvector(struct ice_pf *pf)
{
+ if (!pf->msix_entries)
+ return;
+
+ kfree(pf->msix_entries);
+ pf->msix_entries = NULL;
+
pf->num_avail_sw_msix -= pf->num_rdma_msix;
ice_free_res(pf->irq_tracker, pf->rdma_base_vector,
ICE_RES_RDMA_VEC_ID);
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
index 1fc7daec9732..f61be5d76373 100644
--- a/drivers/net/ethernet/intel/ice/ice_irq.c
+++ b/drivers/net/ethernet/intel/ice/ice_irq.c
@@ -59,7 +59,7 @@ static int ice_ena_msix_range(struct ice_pf *pf)
{
int num_cpus, hw_num_msix, v_other, v_wanted, v_actual;
struct device *dev = ice_pf_to_dev(pf);
- int err, i;
+ int err;
hw_num_msix = pf->hw.func_caps.common_cap.num_msix_vectors;
num_cpus = num_online_cpus();
@@ -113,23 +113,13 @@ static int ice_ena_msix_range(struct ice_pf *pf)
pf->num_rdma_msix);
}
- pf->msix_entries = devm_kcalloc(dev, v_wanted,
- sizeof(*pf->msix_entries), GFP_KERNEL);
- if (!pf->msix_entries) {
- err = -ENOMEM;
- goto exit_err;
- }
-
- for (i = 0; i < v_wanted; i++)
- pf->msix_entries[i].entry = i;
-
/* actually reserve the vectors */
- v_actual = pci_enable_msix_range(pf->pdev, pf->msix_entries,
- ICE_MIN_MSIX, v_wanted);
+ v_actual = pci_alloc_irq_vectors(pf->pdev, ICE_MIN_MSIX, v_wanted,
+ PCI_IRQ_MSIX);
if (v_actual < 0) {
dev_err(dev, "unable to reserve MSI-X vectors\n");
err = v_actual;
- goto msix_err;
+ goto exit_err;
}
if (v_actual < v_wanted) {
@@ -138,9 +128,9 @@ static int ice_ena_msix_range(struct ice_pf *pf)
if (v_actual < ICE_MIN_MSIX) {
/* error if we can't get minimum vectors */
- pci_disable_msix(pf->pdev);
+ pci_free_irq_vectors(pf->pdev);
err = -ERANGE;
- goto msix_err;
+ goto exit_err;
} else {
int v_remain = v_actual - v_other;
@@ -160,33 +150,19 @@ static int ice_ena_msix_range(struct ice_pf *pf)
return v_actual;
-msix_err:
- devm_kfree(dev, pf->msix_entries);
-
exit_err:
pf->num_rdma_msix = 0;
pf->num_lan_msix = 0;
return err;
}
-/**
- * ice_dis_msix - Disable MSI-X interrupt setup in OS
- * @pf: board private structure
- */
-static void ice_dis_msix(struct ice_pf *pf)
-{
- pci_disable_msix(pf->pdev);
- devm_kfree(ice_pf_to_dev(pf), pf->msix_entries);
- pf->msix_entries = NULL;
-}
-
/**
* ice_clear_interrupt_scheme - Undo things done by ice_init_interrupt_scheme
* @pf: board private structure
*/
void ice_clear_interrupt_scheme(struct ice_pf *pf)
{
- ice_dis_msix(pf);
+ pci_free_irq_vectors(pf->pdev);
if (pf->irq_tracker) {
devm_kfree(ice_pf_to_dev(pf), pf->irq_tracker);
@@ -213,7 +189,7 @@ int ice_init_interrupt_scheme(struct ice_pf *pf)
vectors),
GFP_KERNEL);
if (!pf->irq_tracker) {
- ice_dis_msix(pf);
+ pci_free_irq_vectors(pf->pdev);
return -ENOMEM;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index f3032096e5dd..9ccb6092b937 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -3281,10 +3281,8 @@ static void ice_free_irq_msix_misc(struct ice_pf *pf)
wr32(hw, PFINT_OICR_ENA, 0);
ice_flush(hw);
- if (pf->msix_entries) {
- synchronize_irq(misc_irq_num);
- devm_free_irq(ice_pf_to_dev(pf), misc_irq_num, pf);
- }
+ synchronize_irq(misc_irq_num);
+ devm_free_irq(ice_pf_to_dev(pf), misc_irq_num, pf);
pf->num_avail_sw_msix += 1;
ice_free_res(pf->irq_tracker, pf->oicr_idx, ICE_RES_MISC_VEC_ID);
--
2.38.1
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Intel-wired-lan] [PATCH net-next v3 4/8] ice: refactor VF control VSI interrupt handling
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-03-23 12:24 ` Piotr Raczynski
-1 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: michal.swiatkowski, netdev, jesse.brandeburg, shiraz.saleem
All VF contrl VSIs share the same interrupt vector. Currently, a helper
function dedicated for that directly sets ice_vsi::base_vector.
Use helper that returns pointer to first found VF control VSI instead.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
---
drivers/net/ethernet/intel/ice/ice_lib.c | 76 ++++-----------------
drivers/net/ethernet/intel/ice/ice_vf_lib.c | 32 +++++++++
drivers/net/ethernet/intel/ice/ice_vf_lib.h | 7 ++
3 files changed, 52 insertions(+), 63 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index a707c0f7ae29..3bb557219ee4 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -1475,36 +1475,6 @@ ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res, u16 needed, u16 id)
return ice_search_res(res, needed, id);
}
-/**
- * ice_get_vf_ctrl_res - Get VF control VSI resource
- * @pf: pointer to the PF structure
- * @vsi: the VSI to allocate a resource for
- *
- * Look up whether another VF has already allocated the control VSI resource.
- * If so, re-use this resource so that we share it among all VFs.
- *
- * Otherwise, allocate the resource and return it.
- */
-static int ice_get_vf_ctrl_res(struct ice_pf *pf, struct ice_vsi *vsi)
-{
- struct ice_vf *vf;
- unsigned int bkt;
- int base;
-
- rcu_read_lock();
- ice_for_each_vf_rcu(pf, bkt, vf) {
- if (vf != vsi->vf && vf->ctrl_vsi_idx != ICE_NO_VSI) {
- base = pf->vsi[vf->ctrl_vsi_idx]->base_vector;
- rcu_read_unlock();
- return base;
- }
- }
- rcu_read_unlock();
-
- return ice_get_res(pf, pf->irq_tracker, vsi->num_q_vectors,
- ICE_RES_VF_CTRL_VEC_ID);
-}
-
/**
* ice_vsi_setup_vector_base - Set up the base vector for the given VSI
* @vsi: ptr to the VSI
@@ -1538,7 +1508,14 @@ static int ice_vsi_setup_vector_base(struct ice_vsi *vsi)
num_q_vectors = vsi->num_q_vectors;
/* reserve slots from OS requested IRQs */
if (vsi->type == ICE_VSI_CTRL && vsi->vf) {
- base = ice_get_vf_ctrl_res(pf, vsi);
+ struct ice_vsi *ctrl_vsi = ice_get_vf_ctrl_vsi(pf, vsi);
+
+ if (ctrl_vsi)
+ base = ctrl_vsi->base_vector;
+ else
+ base = ice_get_res(pf, pf->irq_tracker,
+ vsi->num_q_vectors,
+ ICE_RES_VF_CTRL_VEC_ID);
} else {
base = ice_get_res(pf, pf->irq_tracker, num_q_vectors,
vsi->idx);
@@ -2613,37 +2590,6 @@ static void ice_set_agg_vsi(struct ice_vsi *vsi)
vsi->agg_node->num_vsis);
}
-/**
- * ice_free_vf_ctrl_res - Free the VF control VSI resource
- * @pf: pointer to PF structure
- * @vsi: the VSI to free resources for
- *
- * Check if the VF control VSI resource is still in use. If no VF is using it
- * any more, release the VSI resource. Otherwise, leave it to be cleaned up
- * once no other VF uses it.
- */
-static void ice_free_vf_ctrl_res(struct ice_pf *pf, struct ice_vsi *vsi)
-{
- struct ice_vf *vf;
- unsigned int bkt;
-
- rcu_read_lock();
- ice_for_each_vf_rcu(pf, bkt, vf) {
- if (vf != vsi->vf && vf->ctrl_vsi_idx != ICE_NO_VSI) {
- rcu_read_unlock();
- return;
- }
- }
- rcu_read_unlock();
-
- /* No other VFs left that have control VSI. It is now safe to reclaim
- * SW interrupts back to the common pool.
- */
- ice_free_res(pf->irq_tracker, vsi->base_vector,
- ICE_RES_VF_CTRL_VEC_ID);
- pf->num_avail_sw_msix += vsi->num_q_vectors;
-}
-
static int ice_vsi_cfg_tc_lan(struct ice_pf *pf, struct ice_vsi *vsi)
{
u16 max_txqs[ICE_MAX_TRAFFIC_CLASS] = { 0 };
@@ -2918,7 +2864,11 @@ void ice_vsi_decfg(struct ice_vsi *vsi)
* cleared in the same manner.
*/
if (vsi->type == ICE_VSI_CTRL && vsi->vf) {
- ice_free_vf_ctrl_res(pf, vsi);
+ struct ice_vsi *ctrl_vsi = ice_get_vf_ctrl_vsi(pf, vsi);
+
+ if (!ctrl_vsi)
+ ice_free_res(pf->irq_tracker, vsi->base_vector,
+ ICE_RES_VF_CTRL_VEC_ID);
} else if (vsi->type != ICE_VSI_VF) {
/* reclaim SW interrupts back to the common pool */
ice_free_res(pf->irq_tracker, vsi->base_vector, vsi->idx);
diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.c b/drivers/net/ethernet/intel/ice/ice_vf_lib.c
index 89fd6982df09..68142facc85d 100644
--- a/drivers/net/ethernet/intel/ice/ice_vf_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.c
@@ -1310,3 +1310,35 @@ void ice_vf_set_initialized(struct ice_vf *vf)
set_bit(ICE_VF_STATE_INIT, vf->vf_states);
memset(&vf->vlan_v2_caps, 0, sizeof(vf->vlan_v2_caps));
}
+
+/**
+ * ice_get_vf_ctrl_vsi - Get first VF control VSI pointer
+ * @pf: the PF private structure
+ * @vsi: pointer to the VSI
+ *
+ * Return first found VF control VSI other than the vsi
+ * passed by parameter. This function is used to determine
+ * whether new resources have to be allocated for control VSI
+ * or they can be shared with existing one.
+ *
+ * Return found VF control VSI pointer other itself. Return
+ * NULL Otherwise.
+ *
+ */
+struct ice_vsi *ice_get_vf_ctrl_vsi(struct ice_pf *pf, struct ice_vsi *vsi)
+{
+ struct ice_vsi *ctrl_vsi = NULL;
+ struct ice_vf *vf;
+ unsigned int bkt;
+
+ rcu_read_lock();
+ ice_for_each_vf_rcu(pf, bkt, vf) {
+ if (vf != vsi->vf && vf->ctrl_vsi_idx != ICE_NO_VSI) {
+ ctrl_vsi = pf->vsi[vf->ctrl_vsi_idx];
+ break;
+ }
+ }
+
+ rcu_read_unlock();
+ return ctrl_vsi;
+}
diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.h b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
index e3cda6fb71ab..48fea6fa0362 100644
--- a/drivers/net/ethernet/intel/ice/ice_vf_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
@@ -226,6 +226,7 @@ int
ice_vf_clear_vsi_promisc(struct ice_vf *vf, struct ice_vsi *vsi, u8 promisc_m);
int ice_reset_vf(struct ice_vf *vf, u32 flags);
void ice_reset_all_vfs(struct ice_pf *pf);
+struct ice_vsi *ice_get_vf_ctrl_vsi(struct ice_pf *pf, struct ice_vsi *vsi);
#else /* CONFIG_PCI_IOV */
static inline struct ice_vf *ice_get_vf_by_id(struct ice_pf *pf, u16 vf_id)
{
@@ -290,6 +291,12 @@ static inline int ice_reset_vf(struct ice_vf *vf, u32 flags)
static inline void ice_reset_all_vfs(struct ice_pf *pf)
{
}
+
+static inline struct ice_vsi *
+ice_get_vf_ctrl_vsi(struct ice_pf *pf, struct ice_vsi *vsi)
+{
+ return NULL;
+}
#endif /* !CONFIG_PCI_IOV */
#endif /* _ICE_VF_LIB_H_ */
--
2.38.1
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH net-next v3 4/8] ice: refactor VF control VSI interrupt handling
@ 2023-03-23 12:24 ` Piotr Raczynski
0 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, michal.swiatkowski, shiraz.saleem, jacob.e.keller,
sridhar.samudrala, jesse.brandeburg, aleksander.lobakin,
lukasz.czapnik, Piotr Raczynski
All VF contrl VSIs share the same interrupt vector. Currently, a helper
function dedicated for that directly sets ice_vsi::base_vector.
Use helper that returns pointer to first found VF control VSI instead.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
---
drivers/net/ethernet/intel/ice/ice_lib.c | 76 ++++-----------------
drivers/net/ethernet/intel/ice/ice_vf_lib.c | 32 +++++++++
drivers/net/ethernet/intel/ice/ice_vf_lib.h | 7 ++
3 files changed, 52 insertions(+), 63 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index a707c0f7ae29..3bb557219ee4 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -1475,36 +1475,6 @@ ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res, u16 needed, u16 id)
return ice_search_res(res, needed, id);
}
-/**
- * ice_get_vf_ctrl_res - Get VF control VSI resource
- * @pf: pointer to the PF structure
- * @vsi: the VSI to allocate a resource for
- *
- * Look up whether another VF has already allocated the control VSI resource.
- * If so, re-use this resource so that we share it among all VFs.
- *
- * Otherwise, allocate the resource and return it.
- */
-static int ice_get_vf_ctrl_res(struct ice_pf *pf, struct ice_vsi *vsi)
-{
- struct ice_vf *vf;
- unsigned int bkt;
- int base;
-
- rcu_read_lock();
- ice_for_each_vf_rcu(pf, bkt, vf) {
- if (vf != vsi->vf && vf->ctrl_vsi_idx != ICE_NO_VSI) {
- base = pf->vsi[vf->ctrl_vsi_idx]->base_vector;
- rcu_read_unlock();
- return base;
- }
- }
- rcu_read_unlock();
-
- return ice_get_res(pf, pf->irq_tracker, vsi->num_q_vectors,
- ICE_RES_VF_CTRL_VEC_ID);
-}
-
/**
* ice_vsi_setup_vector_base - Set up the base vector for the given VSI
* @vsi: ptr to the VSI
@@ -1538,7 +1508,14 @@ static int ice_vsi_setup_vector_base(struct ice_vsi *vsi)
num_q_vectors = vsi->num_q_vectors;
/* reserve slots from OS requested IRQs */
if (vsi->type == ICE_VSI_CTRL && vsi->vf) {
- base = ice_get_vf_ctrl_res(pf, vsi);
+ struct ice_vsi *ctrl_vsi = ice_get_vf_ctrl_vsi(pf, vsi);
+
+ if (ctrl_vsi)
+ base = ctrl_vsi->base_vector;
+ else
+ base = ice_get_res(pf, pf->irq_tracker,
+ vsi->num_q_vectors,
+ ICE_RES_VF_CTRL_VEC_ID);
} else {
base = ice_get_res(pf, pf->irq_tracker, num_q_vectors,
vsi->idx);
@@ -2613,37 +2590,6 @@ static void ice_set_agg_vsi(struct ice_vsi *vsi)
vsi->agg_node->num_vsis);
}
-/**
- * ice_free_vf_ctrl_res - Free the VF control VSI resource
- * @pf: pointer to PF structure
- * @vsi: the VSI to free resources for
- *
- * Check if the VF control VSI resource is still in use. If no VF is using it
- * any more, release the VSI resource. Otherwise, leave it to be cleaned up
- * once no other VF uses it.
- */
-static void ice_free_vf_ctrl_res(struct ice_pf *pf, struct ice_vsi *vsi)
-{
- struct ice_vf *vf;
- unsigned int bkt;
-
- rcu_read_lock();
- ice_for_each_vf_rcu(pf, bkt, vf) {
- if (vf != vsi->vf && vf->ctrl_vsi_idx != ICE_NO_VSI) {
- rcu_read_unlock();
- return;
- }
- }
- rcu_read_unlock();
-
- /* No other VFs left that have control VSI. It is now safe to reclaim
- * SW interrupts back to the common pool.
- */
- ice_free_res(pf->irq_tracker, vsi->base_vector,
- ICE_RES_VF_CTRL_VEC_ID);
- pf->num_avail_sw_msix += vsi->num_q_vectors;
-}
-
static int ice_vsi_cfg_tc_lan(struct ice_pf *pf, struct ice_vsi *vsi)
{
u16 max_txqs[ICE_MAX_TRAFFIC_CLASS] = { 0 };
@@ -2918,7 +2864,11 @@ void ice_vsi_decfg(struct ice_vsi *vsi)
* cleared in the same manner.
*/
if (vsi->type == ICE_VSI_CTRL && vsi->vf) {
- ice_free_vf_ctrl_res(pf, vsi);
+ struct ice_vsi *ctrl_vsi = ice_get_vf_ctrl_vsi(pf, vsi);
+
+ if (!ctrl_vsi)
+ ice_free_res(pf->irq_tracker, vsi->base_vector,
+ ICE_RES_VF_CTRL_VEC_ID);
} else if (vsi->type != ICE_VSI_VF) {
/* reclaim SW interrupts back to the common pool */
ice_free_res(pf->irq_tracker, vsi->base_vector, vsi->idx);
diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.c b/drivers/net/ethernet/intel/ice/ice_vf_lib.c
index 89fd6982df09..68142facc85d 100644
--- a/drivers/net/ethernet/intel/ice/ice_vf_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.c
@@ -1310,3 +1310,35 @@ void ice_vf_set_initialized(struct ice_vf *vf)
set_bit(ICE_VF_STATE_INIT, vf->vf_states);
memset(&vf->vlan_v2_caps, 0, sizeof(vf->vlan_v2_caps));
}
+
+/**
+ * ice_get_vf_ctrl_vsi - Get first VF control VSI pointer
+ * @pf: the PF private structure
+ * @vsi: pointer to the VSI
+ *
+ * Return first found VF control VSI other than the vsi
+ * passed by parameter. This function is used to determine
+ * whether new resources have to be allocated for control VSI
+ * or they can be shared with existing one.
+ *
+ * Return found VF control VSI pointer other itself. Return
+ * NULL Otherwise.
+ *
+ */
+struct ice_vsi *ice_get_vf_ctrl_vsi(struct ice_pf *pf, struct ice_vsi *vsi)
+{
+ struct ice_vsi *ctrl_vsi = NULL;
+ struct ice_vf *vf;
+ unsigned int bkt;
+
+ rcu_read_lock();
+ ice_for_each_vf_rcu(pf, bkt, vf) {
+ if (vf != vsi->vf && vf->ctrl_vsi_idx != ICE_NO_VSI) {
+ ctrl_vsi = pf->vsi[vf->ctrl_vsi_idx];
+ break;
+ }
+ }
+
+ rcu_read_unlock();
+ return ctrl_vsi;
+}
diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.h b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
index e3cda6fb71ab..48fea6fa0362 100644
--- a/drivers/net/ethernet/intel/ice/ice_vf_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.h
@@ -226,6 +226,7 @@ int
ice_vf_clear_vsi_promisc(struct ice_vf *vf, struct ice_vsi *vsi, u8 promisc_m);
int ice_reset_vf(struct ice_vf *vf, u32 flags);
void ice_reset_all_vfs(struct ice_pf *pf);
+struct ice_vsi *ice_get_vf_ctrl_vsi(struct ice_pf *pf, struct ice_vsi *vsi);
#else /* CONFIG_PCI_IOV */
static inline struct ice_vf *ice_get_vf_by_id(struct ice_pf *pf, u16 vf_id)
{
@@ -290,6 +291,12 @@ static inline int ice_reset_vf(struct ice_vf *vf, u32 flags)
static inline void ice_reset_all_vfs(struct ice_pf *pf)
{
}
+
+static inline struct ice_vsi *
+ice_get_vf_ctrl_vsi(struct ice_pf *pf, struct ice_vsi *vsi)
+{
+ return NULL;
+}
#endif /* !CONFIG_PCI_IOV */
#endif /* _ICE_VF_LIB_H_ */
--
2.38.1
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Intel-wired-lan] [PATCH net-next v3 5/8] ice: remove redundant SRIOV code
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-03-23 12:24 ` Piotr Raczynski
-1 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: michal.swiatkowski, netdev, jesse.brandeburg, shiraz.saleem
Remove redundant code from ice_get_max_valid_res_idx that has no effect.
ice_pf::irq_tracker is initialized during driver probe, there is no reason
to check it again. Also it is not possible for pf::sriov_base_vector to be
lower than the tracker length, remove WARN_ON that will never happen.
Get rid of ice_get_max_valid_res_idx helper function completely since it
can never return negative value.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
---
drivers/net/ethernet/intel/ice/ice_sriov.c | 36 ----------------------
1 file changed, 36 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c
index f1dca59bd844..65f971b74717 100644
--- a/drivers/net/ethernet/intel/ice/ice_sriov.c
+++ b/drivers/net/ethernet/intel/ice/ice_sriov.c
@@ -135,18 +135,9 @@ static void ice_dis_vf_mappings(struct ice_vf *vf)
*/
static int ice_sriov_free_msix_res(struct ice_pf *pf)
{
- struct ice_res_tracker *res;
-
if (!pf)
return -EINVAL;
- res = pf->irq_tracker;
- if (!res)
- return -EINVAL;
-
- /* give back irq_tracker resources used */
- WARN_ON(pf->sriov_base_vector < res->num_entries);
-
pf->sriov_base_vector = 0;
return 0;
@@ -409,29 +400,6 @@ int ice_calc_vf_reg_idx(struct ice_vf *vf, struct ice_q_vector *q_vector)
q_vector->v_idx + 1;
}
-/**
- * ice_get_max_valid_res_idx - Get the max valid resource index
- * @res: pointer to the resource to find the max valid index for
- *
- * Start from the end of the ice_res_tracker and return right when we find the
- * first res->list entry with the ICE_RES_VALID_BIT set. This function is only
- * valid for SR-IOV because it is the only consumer that manipulates the
- * res->end and this is always called when res->end is set to res->num_entries.
- */
-static int ice_get_max_valid_res_idx(struct ice_res_tracker *res)
-{
- int i;
-
- if (!res)
- return -EINVAL;
-
- for (i = res->num_entries - 1; i >= 0; i--)
- if (res->list[i] & ICE_RES_VALID_BIT)
- return i;
-
- return 0;
-}
-
/**
* ice_sriov_set_msix_res - Set any used MSIX resources
* @pf: pointer to PF structure
@@ -490,7 +458,6 @@ static int ice_sriov_set_msix_res(struct ice_pf *pf, u16 num_msix_needed)
*/
static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs)
{
- int max_valid_res_idx = ice_get_max_valid_res_idx(pf->irq_tracker);
u16 num_msix_per_vf, num_txq, num_rxq, avail_qs;
int msix_avail_per_vf, msix_avail_for_sriov;
struct device *dev = ice_pf_to_dev(pf);
@@ -501,9 +468,6 @@ static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs)
if (!num_vfs)
return -EINVAL;
- if (max_valid_res_idx < 0)
- return -ENOSPC;
-
/* determine MSI-X resources per VF */
msix_avail_for_sriov = pf->hw.func_caps.common_cap.num_msix_vectors -
pf->irq_tracker->num_entries;
--
2.38.1
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH net-next v3 5/8] ice: remove redundant SRIOV code
@ 2023-03-23 12:24 ` Piotr Raczynski
0 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, michal.swiatkowski, shiraz.saleem, jacob.e.keller,
sridhar.samudrala, jesse.brandeburg, aleksander.lobakin,
lukasz.czapnik, Piotr Raczynski, Michal Swiatkowski
Remove redundant code from ice_get_max_valid_res_idx that has no effect.
ice_pf::irq_tracker is initialized during driver probe, there is no reason
to check it again. Also it is not possible for pf::sriov_base_vector to be
lower than the tracker length, remove WARN_ON that will never happen.
Get rid of ice_get_max_valid_res_idx helper function completely since it
can never return negative value.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
---
drivers/net/ethernet/intel/ice/ice_sriov.c | 36 ----------------------
1 file changed, 36 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c
index f1dca59bd844..65f971b74717 100644
--- a/drivers/net/ethernet/intel/ice/ice_sriov.c
+++ b/drivers/net/ethernet/intel/ice/ice_sriov.c
@@ -135,18 +135,9 @@ static void ice_dis_vf_mappings(struct ice_vf *vf)
*/
static int ice_sriov_free_msix_res(struct ice_pf *pf)
{
- struct ice_res_tracker *res;
-
if (!pf)
return -EINVAL;
- res = pf->irq_tracker;
- if (!res)
- return -EINVAL;
-
- /* give back irq_tracker resources used */
- WARN_ON(pf->sriov_base_vector < res->num_entries);
-
pf->sriov_base_vector = 0;
return 0;
@@ -409,29 +400,6 @@ int ice_calc_vf_reg_idx(struct ice_vf *vf, struct ice_q_vector *q_vector)
q_vector->v_idx + 1;
}
-/**
- * ice_get_max_valid_res_idx - Get the max valid resource index
- * @res: pointer to the resource to find the max valid index for
- *
- * Start from the end of the ice_res_tracker and return right when we find the
- * first res->list entry with the ICE_RES_VALID_BIT set. This function is only
- * valid for SR-IOV because it is the only consumer that manipulates the
- * res->end and this is always called when res->end is set to res->num_entries.
- */
-static int ice_get_max_valid_res_idx(struct ice_res_tracker *res)
-{
- int i;
-
- if (!res)
- return -EINVAL;
-
- for (i = res->num_entries - 1; i >= 0; i--)
- if (res->list[i] & ICE_RES_VALID_BIT)
- return i;
-
- return 0;
-}
-
/**
* ice_sriov_set_msix_res - Set any used MSIX resources
* @pf: pointer to PF structure
@@ -490,7 +458,6 @@ static int ice_sriov_set_msix_res(struct ice_pf *pf, u16 num_msix_needed)
*/
static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs)
{
- int max_valid_res_idx = ice_get_max_valid_res_idx(pf->irq_tracker);
u16 num_msix_per_vf, num_txq, num_rxq, avail_qs;
int msix_avail_per_vf, msix_avail_for_sriov;
struct device *dev = ice_pf_to_dev(pf);
@@ -501,9 +468,6 @@ static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs)
if (!num_vfs)
return -EINVAL;
- if (max_valid_res_idx < 0)
- return -ENOSPC;
-
/* determine MSI-X resources per VF */
msix_avail_for_sriov = pf->hw.func_caps.common_cap.num_msix_vectors -
pf->irq_tracker->num_entries;
--
2.38.1
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH net-next v3 6/8] ice: add individual interrupt allocation
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-03-23 12:24 ` Piotr Raczynski
-1 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, michal.swiatkowski, shiraz.saleem, jacob.e.keller,
sridhar.samudrala, jesse.brandeburg, aleksander.lobakin,
lukasz.czapnik, Piotr Raczynski, Michal Swiatkowski
Currently interrupt allocations, depending on a feature are distributed
in batches. Also, after allocation there is a series of operations that
distributes per irq settings through that batch of interrupts.
Although driver does not yet support dynamic interrupt allocation, keep
allocated interrupts in a pool and add allocation abstraction logic to
make code more flexible. Keep per interrupt information in the
ice_q_vector structure, which yields ice_vsi::base_vector redundant.
Also, as a result there are a few functions that can be removed.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
---
drivers/net/ethernet/intel/ice/ice.h | 11 +-
drivers/net/ethernet/intel/ice/ice_arfs.c | 5 +-
drivers/net/ethernet/intel/ice/ice_base.c | 36 ++-
drivers/net/ethernet/intel/ice/ice_ethtool.c | 2 +-
drivers/net/ethernet/intel/ice/ice_idc.c | 45 ++--
drivers/net/ethernet/intel/ice/ice_irq.c | 46 +++-
drivers/net/ethernet/intel/ice/ice_irq.h | 3 +
drivers/net/ethernet/intel/ice/ice_lib.c | 225 ++-----------------
drivers/net/ethernet/intel/ice/ice_lib.h | 4 +-
drivers/net/ethernet/intel/ice/ice_main.c | 44 ++--
drivers/net/ethernet/intel/ice/ice_ptp.c | 2 +-
drivers/net/ethernet/intel/ice/ice_sriov.c | 2 +-
drivers/net/ethernet/intel/ice/ice_xsk.c | 5 +-
13 files changed, 154 insertions(+), 276 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 2dc180385976..89d80a2b5feb 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -105,10 +105,6 @@
#define ICE_Q_WAIT_MAX_RETRY (5 * ICE_Q_WAIT_RETRY_LIMIT)
#define ICE_MAX_LG_RSS_QS 256
#define ICE_RES_VALID_BIT 0x8000
-#define ICE_RES_MISC_VEC_ID (ICE_RES_VALID_BIT - 1)
-#define ICE_RES_RDMA_VEC_ID (ICE_RES_MISC_VEC_ID - 1)
-/* All VF control VSIs share the same IRQ, so assign a unique ID for them */
-#define ICE_RES_VF_CTRL_VEC_ID (ICE_RES_RDMA_VEC_ID - 1)
#define ICE_INVAL_Q_INDEX 0xffff
#define ICE_MAX_RXQS_PER_TC 256 /* Used when setting VSI context per TC Rx queues */
@@ -349,7 +345,6 @@ struct ice_vsi {
u32 rx_buf_failed;
u32 rx_page_failed;
u16 num_q_vectors;
- u16 base_vector; /* IRQ base for OS reserved vectors */
enum ice_vsi_type type;
u16 vsi_num; /* HW (absolute) index of this VSI */
u16 idx; /* software index in pf->vsi[] */
@@ -480,6 +475,7 @@ struct ice_q_vector {
char name[ICE_INT_NAME_STR_LEN];
u16 total_events; /* net_dim(): number of interrupts processed */
+ struct msi_map irq;
} ____cacheline_internodealigned_in_smp;
enum ice_pf_flags {
@@ -585,8 +581,7 @@ struct ice_pf {
u32 hw_csum_rx_error;
u32 oicr_err_reg;
- u16 oicr_idx; /* Other interrupt cause MSIX vector index */
- u16 num_avail_sw_msix; /* remaining MSIX SW vectors left unclaimed */
+ struct msi_map oicr_irq; /* Other interrupt cause MSIX vector */
u16 max_pf_txqs; /* Total Tx queues PF wide */
u16 max_pf_rxqs; /* Total Rx queues PF wide */
u16 num_lan_msix; /* Total MSIX vectors for base driver */
@@ -683,7 +678,7 @@ ice_irq_dynamic_ena(struct ice_hw *hw, struct ice_vsi *vsi,
struct ice_q_vector *q_vector)
{
u32 vector = (vsi && q_vector) ? q_vector->reg_idx :
- ((struct ice_pf *)hw->back)->oicr_idx;
+ ((struct ice_pf *)hw->back)->oicr_irq.index;
int itr = ICE_ITR_NONE;
u32 val;
diff --git a/drivers/net/ethernet/intel/ice/ice_arfs.c b/drivers/net/ethernet/intel/ice/ice_arfs.c
index 59146e4bdf3b..a37aa16de281 100644
--- a/drivers/net/ethernet/intel/ice/ice_arfs.c
+++ b/drivers/net/ethernet/intel/ice/ice_arfs.c
@@ -634,7 +634,7 @@ int ice_set_cpu_rx_rmap(struct ice_vsi *vsi)
{
struct net_device *netdev;
struct ice_pf *pf;
- int base_idx, i;
+ int i;
if (!vsi || vsi->type != ICE_VSI_PF)
return 0;
@@ -651,10 +651,9 @@ int ice_set_cpu_rx_rmap(struct ice_vsi *vsi)
if (unlikely(!netdev->rx_cpu_rmap))
return -EINVAL;
- base_idx = vsi->base_vector;
ice_for_each_q_vector(vsi, i)
if (irq_cpu_rmap_add(netdev->rx_cpu_rmap,
- pci_irq_vector(pf->pdev, base_idx + i))) {
+ vsi->q_vectors[i]->irq.virq)) {
ice_free_cpu_rx_rmap(vsi);
return -EINVAL;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c
index 1911d644dfa8..e5db23eaa3f4 100644
--- a/drivers/net/ethernet/intel/ice/ice_base.c
+++ b/drivers/net/ethernet/intel/ice/ice_base.c
@@ -118,9 +118,31 @@ static int ice_vsi_alloc_q_vector(struct ice_vsi *vsi, u16 v_idx)
q_vector->rx.itr_mode = ITR_DYNAMIC;
q_vector->tx.type = ICE_TX_CONTAINER;
q_vector->rx.type = ICE_RX_CONTAINER;
+ q_vector->irq.index = -ENOENT;
- if (vsi->type == ICE_VSI_VF)
+ if (vsi->type == ICE_VSI_VF) {
+ q_vector->reg_idx = ice_calc_vf_reg_idx(vsi->vf, q_vector);
goto out;
+ } else if (vsi->type == ICE_VSI_CTRL && vsi->vf) {
+ struct ice_vsi *ctrl_vsi = ice_get_vf_ctrl_vsi(pf, vsi);
+
+ if (ctrl_vsi) {
+ if (unlikely(!ctrl_vsi->q_vectors))
+ return -ENOENT;
+ q_vector->irq = ctrl_vsi->q_vectors[0]->irq;
+ goto skip_alloc;
+ }
+ }
+
+ q_vector->irq = ice_alloc_irq(pf);
+ if (q_vector->irq.index < 0) {
+ kfree(q_vector);
+ return -ENOMEM;
+ }
+
+skip_alloc:
+ q_vector->reg_idx = q_vector->irq.index;
+
/* only set affinity_mask if the CPU is online */
if (cpu_online(v_idx))
cpumask_set_cpu(v_idx, &q_vector->affinity_mask);
@@ -168,6 +190,18 @@ static void ice_free_q_vector(struct ice_vsi *vsi, int v_idx)
if (vsi->netdev)
netif_napi_del(&q_vector->napi);
+ /* release MSIX interrupt if q_vector had interrupt allocated */
+ if (q_vector->irq.index < 0)
+ goto free_q_vector;
+
+ /* only free last VF ctrl vsi interrupt */
+ if (vsi->type == ICE_VSI_CTRL && vsi->vf &&
+ ice_get_vf_ctrl_vsi(pf, vsi))
+ goto free_q_vector;
+
+ ice_free_irq(pf, q_vector->irq);
+
+free_q_vector:
devm_kfree(dev, q_vector);
vsi->q_vectors[v_idx] = NULL;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c
index f86e814354a3..8407c7175cf6 100644
--- a/drivers/net/ethernet/intel/ice/ice_ethtool.c
+++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c
@@ -956,7 +956,7 @@ static u64 ice_intr_test(struct net_device *netdev)
netdev_info(netdev, "interrupt test\n");
- wr32(&pf->hw, GLINT_DYN_CTL(pf->oicr_idx),
+ wr32(&pf->hw, GLINT_DYN_CTL(pf->oicr_irq.index),
GLINT_DYN_CTL_SW_ITR_INDX_M |
GLINT_DYN_CTL_INTENA_MSK_M |
GLINT_DYN_CTL_SWINT_TRIG_M);
diff --git a/drivers/net/ethernet/intel/ice/ice_idc.c b/drivers/net/ethernet/intel/ice/ice_idc.c
index 1000759505d7..bc016bb4440c 100644
--- a/drivers/net/ethernet/intel/ice/ice_idc.c
+++ b/drivers/net/ethernet/intel/ice/ice_idc.c
@@ -229,38 +229,33 @@ void ice_get_qos_params(struct ice_pf *pf, struct iidc_qos_params *qos)
EXPORT_SYMBOL_GPL(ice_get_qos_params);
/**
- * ice_reserve_rdma_qvector - Reserve vector resources for RDMA driver
+ * ice_alloc_rdma_qvectors - Allocate vector resources for RDMA driver
* @pf: board private structure to initialize
*/
-static int ice_reserve_rdma_qvector(struct ice_pf *pf)
+static int ice_alloc_rdma_qvectors(struct ice_pf *pf)
{
if (ice_is_rdma_ena(pf)) {
- int index, i;
-
- index = ice_get_res(pf, pf->irq_tracker, pf->num_rdma_msix,
- ICE_RES_RDMA_VEC_ID);
- if (index < 0)
- return index;
+ int i;
pf->msix_entries = kcalloc(pf->num_rdma_msix,
sizeof(*pf->msix_entries),
GFP_KERNEL);
- if (!pf->msix_entries) {
- ice_free_res(pf->irq_tracker, pf->rdma_base_vector,
- ICE_RES_RDMA_VEC_ID);
+ if (!pf->msix_entries)
return -ENOMEM;
- }
-
- pf->num_avail_sw_msix -= pf->num_rdma_msix;
/* RDMA is the only user of pf->msix_entries array */
pf->rdma_base_vector = 0;
- for (i = 0; i < pf->num_rdma_msix; i++, index++) {
+ for (i = 0; i < pf->num_rdma_msix; i++) {
struct msix_entry *entry = &pf->msix_entries[i];
+ struct msi_map map;
- entry->entry = index;
- entry->vector = pci_irq_vector(pf->pdev, index);
+ map = ice_alloc_irq(pf);
+ if (map.index < 0)
+ break;
+
+ entry->entry = map.index;
+ entry->vector = map.virq;
}
}
return 0;
@@ -272,15 +267,21 @@ static int ice_reserve_rdma_qvector(struct ice_pf *pf)
*/
static void ice_free_rdma_qvector(struct ice_pf *pf)
{
+ int i;
+
if (!pf->msix_entries)
return;
+ for (i = 0; i < pf->num_rdma_msix; i++) {
+ struct msi_map map;
+
+ map.index = pf->msix_entries[i].entry;
+ map.virq = pf->msix_entries[i].vector;
+ ice_free_irq(pf, map);
+ }
+
kfree(pf->msix_entries);
pf->msix_entries = NULL;
-
- pf->num_avail_sw_msix -= pf->num_rdma_msix;
- ice_free_res(pf->irq_tracker, pf->rdma_base_vector,
- ICE_RES_RDMA_VEC_ID);
}
/**
@@ -382,7 +383,7 @@ int ice_init_rdma(struct ice_pf *pf)
}
/* Reserve vector resources */
- ret = ice_reserve_rdma_qvector(pf);
+ ret = ice_alloc_rdma_qvectors(pf);
if (ret < 0) {
dev_err(dev, "failed to reserve vectors for RDMA\n");
goto err_reserve_rdma_qvector;
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
index f61be5d76373..ca1a1de26766 100644
--- a/drivers/net/ethernet/intel/ice/ice_irq.c
+++ b/drivers/net/ethernet/intel/ice/ice_irq.c
@@ -194,9 +194,53 @@ int ice_init_interrupt_scheme(struct ice_pf *pf)
}
/* populate SW interrupts pool with number of OS granted IRQs. */
- pf->num_avail_sw_msix = (u16)vectors;
pf->irq_tracker->num_entries = (u16)vectors;
pf->irq_tracker->end = pf->irq_tracker->num_entries;
return 0;
}
+
+/**
+ * ice_alloc_irq - Allocate new interrupt vector
+ * @pf: board private structure
+ *
+ * Allocate new interrupt vector for a given owner id.
+ * return struct msi_map with interrupt details and track
+ * allocated interrupt appropriately.
+ *
+ * This function mimics individual interrupt allocation,
+ * even interrupts are actually already allocated with
+ * pci_alloc_irq_vectors. Individual allocation helps
+ * to track interrupts and simplifies interrupt related
+ * handling.
+ *
+ * On failure, return map with negative .index. The caller
+ * is expected to check returned map index.
+ *
+ */
+struct msi_map ice_alloc_irq(struct ice_pf *pf)
+{
+ struct msi_map map = { .index = -ENOENT };
+ int entry;
+
+ entry = ice_get_res(pf, pf->irq_tracker);
+ if (entry < 0)
+ return map;
+
+ map.index = entry;
+ map.virq = pci_irq_vector(pf->pdev, map.index);
+
+ return map;
+}
+
+/**
+ * ice_free_irq - Free interrupt vector
+ * @pf: board private structure
+ * @map: map with interrupt details
+ *
+ * Remove allocated interrupt from the interrupt tracker
+ */
+void ice_free_irq(struct ice_pf *pf, struct msi_map map)
+{
+ ice_free_res(pf->irq_tracker, map.index);
+}
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.h b/drivers/net/ethernet/intel/ice/ice_irq.h
index 82475162ab70..26e80dfe22b5 100644
--- a/drivers/net/ethernet/intel/ice/ice_irq.h
+++ b/drivers/net/ethernet/intel/ice/ice_irq.h
@@ -7,4 +7,7 @@
int ice_init_interrupt_scheme(struct ice_pf *pf);
void ice_clear_interrupt_scheme(struct ice_pf *pf);
+struct msi_map ice_alloc_irq(struct ice_pf *pf);
+void ice_free_irq(struct ice_pf *pf, struct msi_map map);
+
#endif
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 3bb557219ee4..4d29d54ed35d 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -1376,161 +1376,45 @@ static int ice_vsi_init(struct ice_vsi *vsi, u32 vsi_flags)
* ice_free_res - free a block of resources
* @res: pointer to the resource
* @index: starting index previously returned by ice_get_res
- * @id: identifier to track owner
*
* Returns number of resources freed
*/
-int ice_free_res(struct ice_res_tracker *res, u16 index, u16 id)
+int ice_free_res(struct ice_res_tracker *res, u16 index)
{
- int count = 0;
- int i;
-
if (!res || index >= res->end)
return -EINVAL;
- id |= ICE_RES_VALID_BIT;
- for (i = index; i < res->end && res->list[i] == id; i++) {
- res->list[i] = 0;
- count++;
- }
-
- return count;
-}
-
-/**
- * ice_search_res - Search the tracker for a block of resources
- * @res: pointer to the resource
- * @needed: size of the block needed
- * @id: identifier to track owner
- *
- * Returns the base item index of the block, or -ENOMEM for error
- */
-static int ice_search_res(struct ice_res_tracker *res, u16 needed, u16 id)
-{
- u16 start = 0, end = 0;
-
- if (needed > res->end)
- return -ENOMEM;
-
- id |= ICE_RES_VALID_BIT;
-
- do {
- /* skip already allocated entries */
- if (res->list[end++] & ICE_RES_VALID_BIT) {
- start = end;
- if ((start + needed) > res->end)
- break;
- }
-
- if (end == (start + needed)) {
- int i = start;
-
- /* there was enough, so assign it to the requestor */
- while (i != end)
- res->list[i++] = id;
-
- return start;
- }
- } while (end < res->end);
+ res->list[index] = 0;
- return -ENOMEM;
-}
-
-/**
- * ice_get_free_res_count - Get free count from a resource tracker
- * @res: Resource tracker instance
- */
-static u16 ice_get_free_res_count(struct ice_res_tracker *res)
-{
- u16 i, count = 0;
-
- for (i = 0; i < res->end; i++)
- if (!(res->list[i] & ICE_RES_VALID_BIT))
- count++;
-
- return count;
+ return 0;
}
/**
- * ice_get_res - get a block of resources
+ * ice_get_res - get a resource from the tracker
* @pf: board private structure
* @res: pointer to the resource
- * @needed: size of the block needed
- * @id: identifier to track owner
*
- * Returns the base item index of the block, or negative for error
+ * Returns the item index, or negative for error
*/
int
-ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res, u16 needed, u16 id)
+ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res)
{
- if (!res || !pf)
- return -EINVAL;
+ u16 i;
- if (!needed || needed > res->num_entries || id >= ICE_RES_VALID_BIT) {
- dev_err(ice_pf_to_dev(pf), "param err: needed=%d, num_entries = %d id=0x%04x\n",
- needed, res->num_entries, id);
+ if (!res || !pf)
return -EINVAL;
- }
- return ice_search_res(res, needed, id);
-}
-
-/**
- * ice_vsi_setup_vector_base - Set up the base vector for the given VSI
- * @vsi: ptr to the VSI
- *
- * This should only be called after ice_vsi_alloc_def() which allocates the
- * corresponding SW VSI structure and initializes num_queue_pairs for the
- * newly allocated VSI.
- *
- * Returns 0 on success or negative on failure
- */
-static int ice_vsi_setup_vector_base(struct ice_vsi *vsi)
-{
- struct ice_pf *pf = vsi->back;
- struct device *dev;
- u16 num_q_vectors;
- int base;
-
- dev = ice_pf_to_dev(pf);
- /* SRIOV doesn't grab irq_tracker entries for each VSI */
- if (vsi->type == ICE_VSI_VF)
- return 0;
- if (vsi->type == ICE_VSI_CHNL)
- return 0;
-
- if (vsi->base_vector) {
- dev_dbg(dev, "VSI %d has non-zero base vector %d\n",
- vsi->vsi_num, vsi->base_vector);
- return -EEXIST;
- }
-
- num_q_vectors = vsi->num_q_vectors;
- /* reserve slots from OS requested IRQs */
- if (vsi->type == ICE_VSI_CTRL && vsi->vf) {
- struct ice_vsi *ctrl_vsi = ice_get_vf_ctrl_vsi(pf, vsi);
+ /* skip already allocated entries */
+ for (i = 0; i < res->end; i++)
+ if (!(res->list[i] & ICE_RES_VALID_BIT))
+ break;
- if (ctrl_vsi)
- base = ctrl_vsi->base_vector;
- else
- base = ice_get_res(pf, pf->irq_tracker,
- vsi->num_q_vectors,
- ICE_RES_VF_CTRL_VEC_ID);
+ if (i < res->end) {
+ res->list[i] = ICE_RES_VALID_BIT;
+ return i;
} else {
- base = ice_get_res(pf, pf->irq_tracker, num_q_vectors,
- vsi->idx);
- }
-
- if (base < 0) {
- dev_err(dev, "%d MSI-X interrupts available. %s %d failed to get %d MSI-X vectors\n",
- ice_get_free_res_count(pf->irq_tracker),
- ice_vsi_type_str(vsi->type), vsi->idx, num_q_vectors);
- return -ENOENT;
+ return -ENOMEM;
}
- vsi->base_vector = (u16)base;
- pf->num_avail_sw_msix -= num_q_vectors;
-
- return 0;
}
/**
@@ -2388,50 +2272,6 @@ static void ice_vsi_set_tc_cfg(struct ice_vsi *vsi)
ice_vsi_set_dcb_tc_cfg(vsi);
}
-/**
- * ice_vsi_set_q_vectors_reg_idx - set the HW register index for all q_vectors
- * @vsi: VSI to set the q_vectors register index on
- */
-static int
-ice_vsi_set_q_vectors_reg_idx(struct ice_vsi *vsi)
-{
- u16 i;
-
- if (!vsi || !vsi->q_vectors)
- return -EINVAL;
-
- ice_for_each_q_vector(vsi, i) {
- struct ice_q_vector *q_vector = vsi->q_vectors[i];
-
- if (!q_vector) {
- dev_err(ice_pf_to_dev(vsi->back), "Failed to set reg_idx on q_vector %d VSI %d\n",
- i, vsi->vsi_num);
- goto clear_reg_idx;
- }
-
- if (vsi->type == ICE_VSI_VF) {
- struct ice_vf *vf = vsi->vf;
-
- q_vector->reg_idx = ice_calc_vf_reg_idx(vf, q_vector);
- } else {
- q_vector->reg_idx =
- q_vector->v_idx + vsi->base_vector;
- }
- }
-
- return 0;
-
-clear_reg_idx:
- ice_for_each_q_vector(vsi, i) {
- struct ice_q_vector *q_vector = vsi->q_vectors[i];
-
- if (q_vector)
- q_vector->reg_idx = 0;
- }
-
- return -EINVAL;
-}
-
/**
* ice_cfg_sw_lldp - Config switch rules for LLDP packet handling
* @vsi: the VSI being configured
@@ -2676,14 +2516,6 @@ ice_vsi_cfg_def(struct ice_vsi *vsi, struct ice_vsi_cfg_params *params)
if (ret)
goto unroll_vsi_init;
- ret = ice_vsi_setup_vector_base(vsi);
- if (ret)
- goto unroll_alloc_q_vector;
-
- ret = ice_vsi_set_q_vectors_reg_idx(vsi);
- if (ret)
- goto unroll_vector_base;
-
ret = ice_vsi_alloc_rings(vsi);
if (ret)
goto unroll_vector_base;
@@ -2734,10 +2566,6 @@ ice_vsi_cfg_def(struct ice_vsi *vsi, struct ice_vsi_cfg_params *params)
if (ret)
goto unroll_alloc_q_vector;
- ret = ice_vsi_set_q_vectors_reg_idx(vsi);
- if (ret)
- goto unroll_vector_base;
-
ret = ice_vsi_alloc_ring_stats(vsi);
if (ret)
goto unroll_vector_base;
@@ -2770,8 +2598,6 @@ ice_vsi_cfg_def(struct ice_vsi *vsi, struct ice_vsi_cfg_params *params)
unroll_vector_base:
/* reclaim SW interrupts back to the common pool */
- ice_free_res(pf->irq_tracker, vsi->base_vector, vsi->idx);
- pf->num_avail_sw_msix += vsi->num_q_vectors;
unroll_alloc_q_vector:
ice_vsi_free_q_vectors(vsi);
unroll_vsi_init:
@@ -2863,18 +2689,6 @@ void ice_vsi_decfg(struct ice_vsi *vsi)
* many interrupts each VF needs. SR-IOV MSIX resources are also
* cleared in the same manner.
*/
- if (vsi->type == ICE_VSI_CTRL && vsi->vf) {
- struct ice_vsi *ctrl_vsi = ice_get_vf_ctrl_vsi(pf, vsi);
-
- if (!ctrl_vsi)
- ice_free_res(pf->irq_tracker, vsi->base_vector,
- ICE_RES_VF_CTRL_VEC_ID);
- } else if (vsi->type != ICE_VSI_VF) {
- /* reclaim SW interrupts back to the common pool */
- ice_free_res(pf->irq_tracker, vsi->base_vector, vsi->idx);
- pf->num_avail_sw_msix += vsi->num_q_vectors;
- vsi->base_vector = 0;
- }
if (vsi->type == ICE_VSI_VF &&
vsi->agg_node && vsi->agg_node->valid)
@@ -2991,7 +2805,6 @@ static void ice_vsi_release_msix(struct ice_vsi *vsi)
void ice_vsi_free_irq(struct ice_vsi *vsi)
{
struct ice_pf *pf = vsi->back;
- int base = vsi->base_vector;
int i;
if (!vsi->q_vectors || !vsi->irqs_ready)
@@ -3005,10 +2818,9 @@ void ice_vsi_free_irq(struct ice_vsi *vsi)
ice_free_cpu_rx_rmap(vsi);
ice_for_each_q_vector(vsi, i) {
- u16 vector = i + base;
int irq_num;
- irq_num = pci_irq_vector(pf->pdev, vector);
+ irq_num = vsi->q_vectors[i]->irq.virq;
/* free only the irqs that were actually requested */
if (!vsi->q_vectors[i] ||
@@ -3140,7 +2952,6 @@ void ice_dis_vsi(struct ice_vsi *vsi, bool locked)
*/
void ice_vsi_dis_irq(struct ice_vsi *vsi)
{
- int base = vsi->base_vector;
struct ice_pf *pf = vsi->back;
struct ice_hw *hw = &pf->hw;
u32 val;
@@ -3187,7 +2998,7 @@ void ice_vsi_dis_irq(struct ice_vsi *vsi)
return;
ice_for_each_q_vector(vsi, i)
- synchronize_irq(pci_irq_vector(pf->pdev, i + base));
+ synchronize_irq(vsi->q_vectors[i]->irq.virq);
}
/**
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.h b/drivers/net/ethernet/intel/ice/ice_lib.h
index 75221478f2dc..2f52f9e32858 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_lib.h
@@ -104,10 +104,10 @@ int ice_ena_vsi(struct ice_vsi *vsi, bool locked);
void ice_vsi_decfg(struct ice_vsi *vsi);
void ice_dis_vsi(struct ice_vsi *vsi, bool locked);
-int ice_free_res(struct ice_res_tracker *res, u16 index, u16 id);
+int ice_free_res(struct ice_res_tracker *res, u16 index);
int
-ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res, u16 needed, u16 id);
+ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res);
int ice_vsi_rebuild(struct ice_vsi *vsi, u32 vsi_flags);
int ice_vsi_cfg(struct ice_vsi *vsi, struct ice_vsi_cfg_params *params);
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 9ccb6092b937..8e62ec08f582 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -2528,7 +2528,6 @@ static int ice_vsi_req_irq_msix(struct ice_vsi *vsi, char *basename)
{
int q_vectors = vsi->num_q_vectors;
struct ice_pf *pf = vsi->back;
- int base = vsi->base_vector;
struct device *dev;
int rx_int_idx = 0;
int tx_int_idx = 0;
@@ -2539,7 +2538,7 @@ static int ice_vsi_req_irq_msix(struct ice_vsi *vsi, char *basename)
for (vector = 0; vector < q_vectors; vector++) {
struct ice_q_vector *q_vector = vsi->q_vectors[vector];
- irq_num = pci_irq_vector(pf->pdev, base + vector);
+ irq_num = q_vector->irq.virq;
if (q_vector->tx.tx_ring && q_vector->rx.rx_ring) {
snprintf(q_vector->name, sizeof(q_vector->name) - 1,
@@ -2593,9 +2592,8 @@ static int ice_vsi_req_irq_msix(struct ice_vsi *vsi, char *basename)
return 0;
free_q_irqs:
- while (vector) {
- vector--;
- irq_num = pci_irq_vector(pf->pdev, base + vector);
+ while (vector--) {
+ irq_num = vsi->q_vectors[vector]->irq.virq;
if (!IS_ENABLED(CONFIG_RFS_ACCEL))
irq_set_affinity_notifier(irq_num, NULL);
irq_set_affinity_hint(irq_num, NULL);
@@ -3085,7 +3083,7 @@ static void ice_ena_misc_vector(struct ice_pf *pf)
wr32(hw, PFINT_OICR_ENA, val);
/* SW_ITR_IDX = 0, but don't change INTENA */
- wr32(hw, GLINT_DYN_CTL(pf->oicr_idx),
+ wr32(hw, GLINT_DYN_CTL(pf->oicr_irq.index),
GLINT_DYN_CTL_SW_ITR_INDX_M | GLINT_DYN_CTL_INTENA_MSK_M);
}
@@ -3272,7 +3270,7 @@ static void ice_dis_ctrlq_interrupts(struct ice_hw *hw)
*/
static void ice_free_irq_msix_misc(struct ice_pf *pf)
{
- int misc_irq_num = pci_irq_vector(pf->pdev, pf->oicr_idx);
+ int misc_irq_num = pf->oicr_irq.virq;
struct ice_hw *hw = &pf->hw;
ice_dis_ctrlq_interrupts(hw);
@@ -3284,8 +3282,7 @@ static void ice_free_irq_msix_misc(struct ice_pf *pf)
synchronize_irq(misc_irq_num);
devm_free_irq(ice_pf_to_dev(pf), misc_irq_num, pf);
- pf->num_avail_sw_msix += 1;
- ice_free_res(pf->irq_tracker, pf->oicr_idx, ICE_RES_MISC_VEC_ID);
+ ice_free_irq(pf, pf->oicr_irq);
}
/**
@@ -3331,7 +3328,8 @@ static int ice_req_irq_msix_misc(struct ice_pf *pf)
{
struct device *dev = ice_pf_to_dev(pf);
struct ice_hw *hw = &pf->hw;
- int oicr_idx, err = 0;
+ struct msi_map oicr_irq;
+ int err = 0;
if (!pf->int_name[0])
snprintf(pf->int_name, sizeof(pf->int_name) - 1, "%s-%s:misc",
@@ -3345,30 +3343,26 @@ static int ice_req_irq_msix_misc(struct ice_pf *pf)
goto skip_req_irq;
/* reserve one vector in irq_tracker for misc interrupts */
- oicr_idx = ice_get_res(pf, pf->irq_tracker, 1, ICE_RES_MISC_VEC_ID);
- if (oicr_idx < 0)
- return oicr_idx;
-
- pf->num_avail_sw_msix -= 1;
- pf->oicr_idx = (u16)oicr_idx;
-
- err = devm_request_threaded_irq(dev,
- pci_irq_vector(pf->pdev, pf->oicr_idx),
- ice_misc_intr, ice_misc_intr_thread_fn,
- 0, pf->int_name, pf);
+ oicr_irq = ice_alloc_irq(pf);
+ if (oicr_irq.index < 0)
+ return oicr_irq.index;
+
+ pf->oicr_irq = oicr_irq;
+ err = devm_request_threaded_irq(dev, pf->oicr_irq.virq, ice_misc_intr,
+ ice_misc_intr_thread_fn, 0,
+ pf->int_name, pf);
if (err) {
dev_err(dev, "devm_request_threaded_irq for %s failed: %d\n",
pf->int_name, err);
- ice_free_res(pf->irq_tracker, 1, ICE_RES_MISC_VEC_ID);
- pf->num_avail_sw_msix += 1;
+ ice_free_irq(pf, pf->oicr_irq);
return err;
}
skip_req_irq:
ice_ena_misc_vector(pf);
- ice_ena_ctrlq_interrupts(hw, pf->oicr_idx);
- wr32(hw, GLINT_ITR(ICE_RX_ITR, pf->oicr_idx),
+ ice_ena_ctrlq_interrupts(hw, pf->oicr_irq.index);
+ wr32(hw, GLINT_ITR(ICE_RX_ITR, pf->oicr_irq.index),
ITR_REG_ALIGN(ICE_ITR_8K) >> ICE_ITR_GRAN_S);
ice_flush(hw);
diff --git a/drivers/net/ethernet/intel/ice/ice_ptp.c b/drivers/net/ethernet/intel/ice/ice_ptp.c
index 972d4f6fd615..d4b6c997141d 100644
--- a/drivers/net/ethernet/intel/ice/ice_ptp.c
+++ b/drivers/net/ethernet/intel/ice/ice_ptp.c
@@ -911,7 +911,7 @@ ice_ptp_release_tx_tracker(struct ice_pf *pf, struct ice_ptp_tx *tx)
spin_unlock(&tx->lock);
/* wait for potentially outstanding interrupt to complete */
- synchronize_irq(pci_irq_vector(pf->pdev, pf->oicr_idx));
+ synchronize_irq(pf->oicr_irq.virq);
ice_ptp_flush_tx_tracker(pf, tx);
diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c
index 65f971b74717..0fc2b26a2fa6 100644
--- a/drivers/net/ethernet/intel/ice/ice_sriov.c
+++ b/drivers/net/ethernet/intel/ice/ice_sriov.c
@@ -835,7 +835,7 @@ static int ice_ena_vfs(struct ice_pf *pf, u16 num_vfs)
int ret;
/* Disable global interrupt 0 so we don't try to handle the VFLR. */
- wr32(hw, GLINT_DYN_CTL(pf->oicr_idx),
+ wr32(hw, GLINT_DYN_CTL(pf->oicr_irq.index),
ICE_ITR_NONE << GLINT_DYN_CTL_ITR_INDX_S);
set_bit(ICE_OICR_INTR_DIS, pf->state);
ice_flush(hw);
diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c
index 4102416d7a41..a7fe2b4ce655 100644
--- a/drivers/net/ethernet/intel/ice/ice_xsk.c
+++ b/drivers/net/ethernet/intel/ice/ice_xsk.c
@@ -90,7 +90,6 @@ ice_qvec_dis_irq(struct ice_vsi *vsi, struct ice_rx_ring *rx_ring,
{
struct ice_pf *pf = vsi->back;
struct ice_hw *hw = &pf->hw;
- int base = vsi->base_vector;
u16 reg;
u32 val;
@@ -103,11 +102,9 @@ ice_qvec_dis_irq(struct ice_vsi *vsi, struct ice_rx_ring *rx_ring,
wr32(hw, QINT_RQCTL(reg), val);
if (q_vector) {
- u16 v_idx = q_vector->v_idx;
-
wr32(hw, GLINT_DYN_CTL(q_vector->reg_idx), 0);
ice_flush(hw);
- synchronize_irq(pci_irq_vector(pf->pdev, v_idx + base));
+ synchronize_irq(q_vector->irq.virq);
}
}
--
2.38.1
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Intel-wired-lan] [PATCH net-next v3 6/8] ice: add individual interrupt allocation
@ 2023-03-23 12:24 ` Piotr Raczynski
0 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: michal.swiatkowski, netdev, jesse.brandeburg, shiraz.saleem
Currently interrupt allocations, depending on a feature are distributed
in batches. Also, after allocation there is a series of operations that
distributes per irq settings through that batch of interrupts.
Although driver does not yet support dynamic interrupt allocation, keep
allocated interrupts in a pool and add allocation abstraction logic to
make code more flexible. Keep per interrupt information in the
ice_q_vector structure, which yields ice_vsi::base_vector redundant.
Also, as a result there are a few functions that can be removed.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
---
drivers/net/ethernet/intel/ice/ice.h | 11 +-
drivers/net/ethernet/intel/ice/ice_arfs.c | 5 +-
drivers/net/ethernet/intel/ice/ice_base.c | 36 ++-
drivers/net/ethernet/intel/ice/ice_ethtool.c | 2 +-
drivers/net/ethernet/intel/ice/ice_idc.c | 45 ++--
drivers/net/ethernet/intel/ice/ice_irq.c | 46 +++-
drivers/net/ethernet/intel/ice/ice_irq.h | 3 +
drivers/net/ethernet/intel/ice/ice_lib.c | 225 ++-----------------
drivers/net/ethernet/intel/ice/ice_lib.h | 4 +-
drivers/net/ethernet/intel/ice/ice_main.c | 44 ++--
drivers/net/ethernet/intel/ice/ice_ptp.c | 2 +-
drivers/net/ethernet/intel/ice/ice_sriov.c | 2 +-
drivers/net/ethernet/intel/ice/ice_xsk.c | 5 +-
13 files changed, 154 insertions(+), 276 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 2dc180385976..89d80a2b5feb 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -105,10 +105,6 @@
#define ICE_Q_WAIT_MAX_RETRY (5 * ICE_Q_WAIT_RETRY_LIMIT)
#define ICE_MAX_LG_RSS_QS 256
#define ICE_RES_VALID_BIT 0x8000
-#define ICE_RES_MISC_VEC_ID (ICE_RES_VALID_BIT - 1)
-#define ICE_RES_RDMA_VEC_ID (ICE_RES_MISC_VEC_ID - 1)
-/* All VF control VSIs share the same IRQ, so assign a unique ID for them */
-#define ICE_RES_VF_CTRL_VEC_ID (ICE_RES_RDMA_VEC_ID - 1)
#define ICE_INVAL_Q_INDEX 0xffff
#define ICE_MAX_RXQS_PER_TC 256 /* Used when setting VSI context per TC Rx queues */
@@ -349,7 +345,6 @@ struct ice_vsi {
u32 rx_buf_failed;
u32 rx_page_failed;
u16 num_q_vectors;
- u16 base_vector; /* IRQ base for OS reserved vectors */
enum ice_vsi_type type;
u16 vsi_num; /* HW (absolute) index of this VSI */
u16 idx; /* software index in pf->vsi[] */
@@ -480,6 +475,7 @@ struct ice_q_vector {
char name[ICE_INT_NAME_STR_LEN];
u16 total_events; /* net_dim(): number of interrupts processed */
+ struct msi_map irq;
} ____cacheline_internodealigned_in_smp;
enum ice_pf_flags {
@@ -585,8 +581,7 @@ struct ice_pf {
u32 hw_csum_rx_error;
u32 oicr_err_reg;
- u16 oicr_idx; /* Other interrupt cause MSIX vector index */
- u16 num_avail_sw_msix; /* remaining MSIX SW vectors left unclaimed */
+ struct msi_map oicr_irq; /* Other interrupt cause MSIX vector */
u16 max_pf_txqs; /* Total Tx queues PF wide */
u16 max_pf_rxqs; /* Total Rx queues PF wide */
u16 num_lan_msix; /* Total MSIX vectors for base driver */
@@ -683,7 +678,7 @@ ice_irq_dynamic_ena(struct ice_hw *hw, struct ice_vsi *vsi,
struct ice_q_vector *q_vector)
{
u32 vector = (vsi && q_vector) ? q_vector->reg_idx :
- ((struct ice_pf *)hw->back)->oicr_idx;
+ ((struct ice_pf *)hw->back)->oicr_irq.index;
int itr = ICE_ITR_NONE;
u32 val;
diff --git a/drivers/net/ethernet/intel/ice/ice_arfs.c b/drivers/net/ethernet/intel/ice/ice_arfs.c
index 59146e4bdf3b..a37aa16de281 100644
--- a/drivers/net/ethernet/intel/ice/ice_arfs.c
+++ b/drivers/net/ethernet/intel/ice/ice_arfs.c
@@ -634,7 +634,7 @@ int ice_set_cpu_rx_rmap(struct ice_vsi *vsi)
{
struct net_device *netdev;
struct ice_pf *pf;
- int base_idx, i;
+ int i;
if (!vsi || vsi->type != ICE_VSI_PF)
return 0;
@@ -651,10 +651,9 @@ int ice_set_cpu_rx_rmap(struct ice_vsi *vsi)
if (unlikely(!netdev->rx_cpu_rmap))
return -EINVAL;
- base_idx = vsi->base_vector;
ice_for_each_q_vector(vsi, i)
if (irq_cpu_rmap_add(netdev->rx_cpu_rmap,
- pci_irq_vector(pf->pdev, base_idx + i))) {
+ vsi->q_vectors[i]->irq.virq)) {
ice_free_cpu_rx_rmap(vsi);
return -EINVAL;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c
index 1911d644dfa8..e5db23eaa3f4 100644
--- a/drivers/net/ethernet/intel/ice/ice_base.c
+++ b/drivers/net/ethernet/intel/ice/ice_base.c
@@ -118,9 +118,31 @@ static int ice_vsi_alloc_q_vector(struct ice_vsi *vsi, u16 v_idx)
q_vector->rx.itr_mode = ITR_DYNAMIC;
q_vector->tx.type = ICE_TX_CONTAINER;
q_vector->rx.type = ICE_RX_CONTAINER;
+ q_vector->irq.index = -ENOENT;
- if (vsi->type == ICE_VSI_VF)
+ if (vsi->type == ICE_VSI_VF) {
+ q_vector->reg_idx = ice_calc_vf_reg_idx(vsi->vf, q_vector);
goto out;
+ } else if (vsi->type == ICE_VSI_CTRL && vsi->vf) {
+ struct ice_vsi *ctrl_vsi = ice_get_vf_ctrl_vsi(pf, vsi);
+
+ if (ctrl_vsi) {
+ if (unlikely(!ctrl_vsi->q_vectors))
+ return -ENOENT;
+ q_vector->irq = ctrl_vsi->q_vectors[0]->irq;
+ goto skip_alloc;
+ }
+ }
+
+ q_vector->irq = ice_alloc_irq(pf);
+ if (q_vector->irq.index < 0) {
+ kfree(q_vector);
+ return -ENOMEM;
+ }
+
+skip_alloc:
+ q_vector->reg_idx = q_vector->irq.index;
+
/* only set affinity_mask if the CPU is online */
if (cpu_online(v_idx))
cpumask_set_cpu(v_idx, &q_vector->affinity_mask);
@@ -168,6 +190,18 @@ static void ice_free_q_vector(struct ice_vsi *vsi, int v_idx)
if (vsi->netdev)
netif_napi_del(&q_vector->napi);
+ /* release MSIX interrupt if q_vector had interrupt allocated */
+ if (q_vector->irq.index < 0)
+ goto free_q_vector;
+
+ /* only free last VF ctrl vsi interrupt */
+ if (vsi->type == ICE_VSI_CTRL && vsi->vf &&
+ ice_get_vf_ctrl_vsi(pf, vsi))
+ goto free_q_vector;
+
+ ice_free_irq(pf, q_vector->irq);
+
+free_q_vector:
devm_kfree(dev, q_vector);
vsi->q_vectors[v_idx] = NULL;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_ethtool.c b/drivers/net/ethernet/intel/ice/ice_ethtool.c
index f86e814354a3..8407c7175cf6 100644
--- a/drivers/net/ethernet/intel/ice/ice_ethtool.c
+++ b/drivers/net/ethernet/intel/ice/ice_ethtool.c
@@ -956,7 +956,7 @@ static u64 ice_intr_test(struct net_device *netdev)
netdev_info(netdev, "interrupt test\n");
- wr32(&pf->hw, GLINT_DYN_CTL(pf->oicr_idx),
+ wr32(&pf->hw, GLINT_DYN_CTL(pf->oicr_irq.index),
GLINT_DYN_CTL_SW_ITR_INDX_M |
GLINT_DYN_CTL_INTENA_MSK_M |
GLINT_DYN_CTL_SWINT_TRIG_M);
diff --git a/drivers/net/ethernet/intel/ice/ice_idc.c b/drivers/net/ethernet/intel/ice/ice_idc.c
index 1000759505d7..bc016bb4440c 100644
--- a/drivers/net/ethernet/intel/ice/ice_idc.c
+++ b/drivers/net/ethernet/intel/ice/ice_idc.c
@@ -229,38 +229,33 @@ void ice_get_qos_params(struct ice_pf *pf, struct iidc_qos_params *qos)
EXPORT_SYMBOL_GPL(ice_get_qos_params);
/**
- * ice_reserve_rdma_qvector - Reserve vector resources for RDMA driver
+ * ice_alloc_rdma_qvectors - Allocate vector resources for RDMA driver
* @pf: board private structure to initialize
*/
-static int ice_reserve_rdma_qvector(struct ice_pf *pf)
+static int ice_alloc_rdma_qvectors(struct ice_pf *pf)
{
if (ice_is_rdma_ena(pf)) {
- int index, i;
-
- index = ice_get_res(pf, pf->irq_tracker, pf->num_rdma_msix,
- ICE_RES_RDMA_VEC_ID);
- if (index < 0)
- return index;
+ int i;
pf->msix_entries = kcalloc(pf->num_rdma_msix,
sizeof(*pf->msix_entries),
GFP_KERNEL);
- if (!pf->msix_entries) {
- ice_free_res(pf->irq_tracker, pf->rdma_base_vector,
- ICE_RES_RDMA_VEC_ID);
+ if (!pf->msix_entries)
return -ENOMEM;
- }
-
- pf->num_avail_sw_msix -= pf->num_rdma_msix;
/* RDMA is the only user of pf->msix_entries array */
pf->rdma_base_vector = 0;
- for (i = 0; i < pf->num_rdma_msix; i++, index++) {
+ for (i = 0; i < pf->num_rdma_msix; i++) {
struct msix_entry *entry = &pf->msix_entries[i];
+ struct msi_map map;
- entry->entry = index;
- entry->vector = pci_irq_vector(pf->pdev, index);
+ map = ice_alloc_irq(pf);
+ if (map.index < 0)
+ break;
+
+ entry->entry = map.index;
+ entry->vector = map.virq;
}
}
return 0;
@@ -272,15 +267,21 @@ static int ice_reserve_rdma_qvector(struct ice_pf *pf)
*/
static void ice_free_rdma_qvector(struct ice_pf *pf)
{
+ int i;
+
if (!pf->msix_entries)
return;
+ for (i = 0; i < pf->num_rdma_msix; i++) {
+ struct msi_map map;
+
+ map.index = pf->msix_entries[i].entry;
+ map.virq = pf->msix_entries[i].vector;
+ ice_free_irq(pf, map);
+ }
+
kfree(pf->msix_entries);
pf->msix_entries = NULL;
-
- pf->num_avail_sw_msix -= pf->num_rdma_msix;
- ice_free_res(pf->irq_tracker, pf->rdma_base_vector,
- ICE_RES_RDMA_VEC_ID);
}
/**
@@ -382,7 +383,7 @@ int ice_init_rdma(struct ice_pf *pf)
}
/* Reserve vector resources */
- ret = ice_reserve_rdma_qvector(pf);
+ ret = ice_alloc_rdma_qvectors(pf);
if (ret < 0) {
dev_err(dev, "failed to reserve vectors for RDMA\n");
goto err_reserve_rdma_qvector;
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
index f61be5d76373..ca1a1de26766 100644
--- a/drivers/net/ethernet/intel/ice/ice_irq.c
+++ b/drivers/net/ethernet/intel/ice/ice_irq.c
@@ -194,9 +194,53 @@ int ice_init_interrupt_scheme(struct ice_pf *pf)
}
/* populate SW interrupts pool with number of OS granted IRQs. */
- pf->num_avail_sw_msix = (u16)vectors;
pf->irq_tracker->num_entries = (u16)vectors;
pf->irq_tracker->end = pf->irq_tracker->num_entries;
return 0;
}
+
+/**
+ * ice_alloc_irq - Allocate new interrupt vector
+ * @pf: board private structure
+ *
+ * Allocate new interrupt vector for a given owner id.
+ * return struct msi_map with interrupt details and track
+ * allocated interrupt appropriately.
+ *
+ * This function mimics individual interrupt allocation,
+ * even interrupts are actually already allocated with
+ * pci_alloc_irq_vectors. Individual allocation helps
+ * to track interrupts and simplifies interrupt related
+ * handling.
+ *
+ * On failure, return map with negative .index. The caller
+ * is expected to check returned map index.
+ *
+ */
+struct msi_map ice_alloc_irq(struct ice_pf *pf)
+{
+ struct msi_map map = { .index = -ENOENT };
+ int entry;
+
+ entry = ice_get_res(pf, pf->irq_tracker);
+ if (entry < 0)
+ return map;
+
+ map.index = entry;
+ map.virq = pci_irq_vector(pf->pdev, map.index);
+
+ return map;
+}
+
+/**
+ * ice_free_irq - Free interrupt vector
+ * @pf: board private structure
+ * @map: map with interrupt details
+ *
+ * Remove allocated interrupt from the interrupt tracker
+ */
+void ice_free_irq(struct ice_pf *pf, struct msi_map map)
+{
+ ice_free_res(pf->irq_tracker, map.index);
+}
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.h b/drivers/net/ethernet/intel/ice/ice_irq.h
index 82475162ab70..26e80dfe22b5 100644
--- a/drivers/net/ethernet/intel/ice/ice_irq.h
+++ b/drivers/net/ethernet/intel/ice/ice_irq.h
@@ -7,4 +7,7 @@
int ice_init_interrupt_scheme(struct ice_pf *pf);
void ice_clear_interrupt_scheme(struct ice_pf *pf);
+struct msi_map ice_alloc_irq(struct ice_pf *pf);
+void ice_free_irq(struct ice_pf *pf, struct msi_map map);
+
#endif
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 3bb557219ee4..4d29d54ed35d 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -1376,161 +1376,45 @@ static int ice_vsi_init(struct ice_vsi *vsi, u32 vsi_flags)
* ice_free_res - free a block of resources
* @res: pointer to the resource
* @index: starting index previously returned by ice_get_res
- * @id: identifier to track owner
*
* Returns number of resources freed
*/
-int ice_free_res(struct ice_res_tracker *res, u16 index, u16 id)
+int ice_free_res(struct ice_res_tracker *res, u16 index)
{
- int count = 0;
- int i;
-
if (!res || index >= res->end)
return -EINVAL;
- id |= ICE_RES_VALID_BIT;
- for (i = index; i < res->end && res->list[i] == id; i++) {
- res->list[i] = 0;
- count++;
- }
-
- return count;
-}
-
-/**
- * ice_search_res - Search the tracker for a block of resources
- * @res: pointer to the resource
- * @needed: size of the block needed
- * @id: identifier to track owner
- *
- * Returns the base item index of the block, or -ENOMEM for error
- */
-static int ice_search_res(struct ice_res_tracker *res, u16 needed, u16 id)
-{
- u16 start = 0, end = 0;
-
- if (needed > res->end)
- return -ENOMEM;
-
- id |= ICE_RES_VALID_BIT;
-
- do {
- /* skip already allocated entries */
- if (res->list[end++] & ICE_RES_VALID_BIT) {
- start = end;
- if ((start + needed) > res->end)
- break;
- }
-
- if (end == (start + needed)) {
- int i = start;
-
- /* there was enough, so assign it to the requestor */
- while (i != end)
- res->list[i++] = id;
-
- return start;
- }
- } while (end < res->end);
+ res->list[index] = 0;
- return -ENOMEM;
-}
-
-/**
- * ice_get_free_res_count - Get free count from a resource tracker
- * @res: Resource tracker instance
- */
-static u16 ice_get_free_res_count(struct ice_res_tracker *res)
-{
- u16 i, count = 0;
-
- for (i = 0; i < res->end; i++)
- if (!(res->list[i] & ICE_RES_VALID_BIT))
- count++;
-
- return count;
+ return 0;
}
/**
- * ice_get_res - get a block of resources
+ * ice_get_res - get a resource from the tracker
* @pf: board private structure
* @res: pointer to the resource
- * @needed: size of the block needed
- * @id: identifier to track owner
*
- * Returns the base item index of the block, or negative for error
+ * Returns the item index, or negative for error
*/
int
-ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res, u16 needed, u16 id)
+ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res)
{
- if (!res || !pf)
- return -EINVAL;
+ u16 i;
- if (!needed || needed > res->num_entries || id >= ICE_RES_VALID_BIT) {
- dev_err(ice_pf_to_dev(pf), "param err: needed=%d, num_entries = %d id=0x%04x\n",
- needed, res->num_entries, id);
+ if (!res || !pf)
return -EINVAL;
- }
- return ice_search_res(res, needed, id);
-}
-
-/**
- * ice_vsi_setup_vector_base - Set up the base vector for the given VSI
- * @vsi: ptr to the VSI
- *
- * This should only be called after ice_vsi_alloc_def() which allocates the
- * corresponding SW VSI structure and initializes num_queue_pairs for the
- * newly allocated VSI.
- *
- * Returns 0 on success or negative on failure
- */
-static int ice_vsi_setup_vector_base(struct ice_vsi *vsi)
-{
- struct ice_pf *pf = vsi->back;
- struct device *dev;
- u16 num_q_vectors;
- int base;
-
- dev = ice_pf_to_dev(pf);
- /* SRIOV doesn't grab irq_tracker entries for each VSI */
- if (vsi->type == ICE_VSI_VF)
- return 0;
- if (vsi->type == ICE_VSI_CHNL)
- return 0;
-
- if (vsi->base_vector) {
- dev_dbg(dev, "VSI %d has non-zero base vector %d\n",
- vsi->vsi_num, vsi->base_vector);
- return -EEXIST;
- }
-
- num_q_vectors = vsi->num_q_vectors;
- /* reserve slots from OS requested IRQs */
- if (vsi->type == ICE_VSI_CTRL && vsi->vf) {
- struct ice_vsi *ctrl_vsi = ice_get_vf_ctrl_vsi(pf, vsi);
+ /* skip already allocated entries */
+ for (i = 0; i < res->end; i++)
+ if (!(res->list[i] & ICE_RES_VALID_BIT))
+ break;
- if (ctrl_vsi)
- base = ctrl_vsi->base_vector;
- else
- base = ice_get_res(pf, pf->irq_tracker,
- vsi->num_q_vectors,
- ICE_RES_VF_CTRL_VEC_ID);
+ if (i < res->end) {
+ res->list[i] = ICE_RES_VALID_BIT;
+ return i;
} else {
- base = ice_get_res(pf, pf->irq_tracker, num_q_vectors,
- vsi->idx);
- }
-
- if (base < 0) {
- dev_err(dev, "%d MSI-X interrupts available. %s %d failed to get %d MSI-X vectors\n",
- ice_get_free_res_count(pf->irq_tracker),
- ice_vsi_type_str(vsi->type), vsi->idx, num_q_vectors);
- return -ENOENT;
+ return -ENOMEM;
}
- vsi->base_vector = (u16)base;
- pf->num_avail_sw_msix -= num_q_vectors;
-
- return 0;
}
/**
@@ -2388,50 +2272,6 @@ static void ice_vsi_set_tc_cfg(struct ice_vsi *vsi)
ice_vsi_set_dcb_tc_cfg(vsi);
}
-/**
- * ice_vsi_set_q_vectors_reg_idx - set the HW register index for all q_vectors
- * @vsi: VSI to set the q_vectors register index on
- */
-static int
-ice_vsi_set_q_vectors_reg_idx(struct ice_vsi *vsi)
-{
- u16 i;
-
- if (!vsi || !vsi->q_vectors)
- return -EINVAL;
-
- ice_for_each_q_vector(vsi, i) {
- struct ice_q_vector *q_vector = vsi->q_vectors[i];
-
- if (!q_vector) {
- dev_err(ice_pf_to_dev(vsi->back), "Failed to set reg_idx on q_vector %d VSI %d\n",
- i, vsi->vsi_num);
- goto clear_reg_idx;
- }
-
- if (vsi->type == ICE_VSI_VF) {
- struct ice_vf *vf = vsi->vf;
-
- q_vector->reg_idx = ice_calc_vf_reg_idx(vf, q_vector);
- } else {
- q_vector->reg_idx =
- q_vector->v_idx + vsi->base_vector;
- }
- }
-
- return 0;
-
-clear_reg_idx:
- ice_for_each_q_vector(vsi, i) {
- struct ice_q_vector *q_vector = vsi->q_vectors[i];
-
- if (q_vector)
- q_vector->reg_idx = 0;
- }
-
- return -EINVAL;
-}
-
/**
* ice_cfg_sw_lldp - Config switch rules for LLDP packet handling
* @vsi: the VSI being configured
@@ -2676,14 +2516,6 @@ ice_vsi_cfg_def(struct ice_vsi *vsi, struct ice_vsi_cfg_params *params)
if (ret)
goto unroll_vsi_init;
- ret = ice_vsi_setup_vector_base(vsi);
- if (ret)
- goto unroll_alloc_q_vector;
-
- ret = ice_vsi_set_q_vectors_reg_idx(vsi);
- if (ret)
- goto unroll_vector_base;
-
ret = ice_vsi_alloc_rings(vsi);
if (ret)
goto unroll_vector_base;
@@ -2734,10 +2566,6 @@ ice_vsi_cfg_def(struct ice_vsi *vsi, struct ice_vsi_cfg_params *params)
if (ret)
goto unroll_alloc_q_vector;
- ret = ice_vsi_set_q_vectors_reg_idx(vsi);
- if (ret)
- goto unroll_vector_base;
-
ret = ice_vsi_alloc_ring_stats(vsi);
if (ret)
goto unroll_vector_base;
@@ -2770,8 +2598,6 @@ ice_vsi_cfg_def(struct ice_vsi *vsi, struct ice_vsi_cfg_params *params)
unroll_vector_base:
/* reclaim SW interrupts back to the common pool */
- ice_free_res(pf->irq_tracker, vsi->base_vector, vsi->idx);
- pf->num_avail_sw_msix += vsi->num_q_vectors;
unroll_alloc_q_vector:
ice_vsi_free_q_vectors(vsi);
unroll_vsi_init:
@@ -2863,18 +2689,6 @@ void ice_vsi_decfg(struct ice_vsi *vsi)
* many interrupts each VF needs. SR-IOV MSIX resources are also
* cleared in the same manner.
*/
- if (vsi->type == ICE_VSI_CTRL && vsi->vf) {
- struct ice_vsi *ctrl_vsi = ice_get_vf_ctrl_vsi(pf, vsi);
-
- if (!ctrl_vsi)
- ice_free_res(pf->irq_tracker, vsi->base_vector,
- ICE_RES_VF_CTRL_VEC_ID);
- } else if (vsi->type != ICE_VSI_VF) {
- /* reclaim SW interrupts back to the common pool */
- ice_free_res(pf->irq_tracker, vsi->base_vector, vsi->idx);
- pf->num_avail_sw_msix += vsi->num_q_vectors;
- vsi->base_vector = 0;
- }
if (vsi->type == ICE_VSI_VF &&
vsi->agg_node && vsi->agg_node->valid)
@@ -2991,7 +2805,6 @@ static void ice_vsi_release_msix(struct ice_vsi *vsi)
void ice_vsi_free_irq(struct ice_vsi *vsi)
{
struct ice_pf *pf = vsi->back;
- int base = vsi->base_vector;
int i;
if (!vsi->q_vectors || !vsi->irqs_ready)
@@ -3005,10 +2818,9 @@ void ice_vsi_free_irq(struct ice_vsi *vsi)
ice_free_cpu_rx_rmap(vsi);
ice_for_each_q_vector(vsi, i) {
- u16 vector = i + base;
int irq_num;
- irq_num = pci_irq_vector(pf->pdev, vector);
+ irq_num = vsi->q_vectors[i]->irq.virq;
/* free only the irqs that were actually requested */
if (!vsi->q_vectors[i] ||
@@ -3140,7 +2952,6 @@ void ice_dis_vsi(struct ice_vsi *vsi, bool locked)
*/
void ice_vsi_dis_irq(struct ice_vsi *vsi)
{
- int base = vsi->base_vector;
struct ice_pf *pf = vsi->back;
struct ice_hw *hw = &pf->hw;
u32 val;
@@ -3187,7 +2998,7 @@ void ice_vsi_dis_irq(struct ice_vsi *vsi)
return;
ice_for_each_q_vector(vsi, i)
- synchronize_irq(pci_irq_vector(pf->pdev, i + base));
+ synchronize_irq(vsi->q_vectors[i]->irq.virq);
}
/**
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.h b/drivers/net/ethernet/intel/ice/ice_lib.h
index 75221478f2dc..2f52f9e32858 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_lib.h
@@ -104,10 +104,10 @@ int ice_ena_vsi(struct ice_vsi *vsi, bool locked);
void ice_vsi_decfg(struct ice_vsi *vsi);
void ice_dis_vsi(struct ice_vsi *vsi, bool locked);
-int ice_free_res(struct ice_res_tracker *res, u16 index, u16 id);
+int ice_free_res(struct ice_res_tracker *res, u16 index);
int
-ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res, u16 needed, u16 id);
+ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res);
int ice_vsi_rebuild(struct ice_vsi *vsi, u32 vsi_flags);
int ice_vsi_cfg(struct ice_vsi *vsi, struct ice_vsi_cfg_params *params);
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 9ccb6092b937..8e62ec08f582 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -2528,7 +2528,6 @@ static int ice_vsi_req_irq_msix(struct ice_vsi *vsi, char *basename)
{
int q_vectors = vsi->num_q_vectors;
struct ice_pf *pf = vsi->back;
- int base = vsi->base_vector;
struct device *dev;
int rx_int_idx = 0;
int tx_int_idx = 0;
@@ -2539,7 +2538,7 @@ static int ice_vsi_req_irq_msix(struct ice_vsi *vsi, char *basename)
for (vector = 0; vector < q_vectors; vector++) {
struct ice_q_vector *q_vector = vsi->q_vectors[vector];
- irq_num = pci_irq_vector(pf->pdev, base + vector);
+ irq_num = q_vector->irq.virq;
if (q_vector->tx.tx_ring && q_vector->rx.rx_ring) {
snprintf(q_vector->name, sizeof(q_vector->name) - 1,
@@ -2593,9 +2592,8 @@ static int ice_vsi_req_irq_msix(struct ice_vsi *vsi, char *basename)
return 0;
free_q_irqs:
- while (vector) {
- vector--;
- irq_num = pci_irq_vector(pf->pdev, base + vector);
+ while (vector--) {
+ irq_num = vsi->q_vectors[vector]->irq.virq;
if (!IS_ENABLED(CONFIG_RFS_ACCEL))
irq_set_affinity_notifier(irq_num, NULL);
irq_set_affinity_hint(irq_num, NULL);
@@ -3085,7 +3083,7 @@ static void ice_ena_misc_vector(struct ice_pf *pf)
wr32(hw, PFINT_OICR_ENA, val);
/* SW_ITR_IDX = 0, but don't change INTENA */
- wr32(hw, GLINT_DYN_CTL(pf->oicr_idx),
+ wr32(hw, GLINT_DYN_CTL(pf->oicr_irq.index),
GLINT_DYN_CTL_SW_ITR_INDX_M | GLINT_DYN_CTL_INTENA_MSK_M);
}
@@ -3272,7 +3270,7 @@ static void ice_dis_ctrlq_interrupts(struct ice_hw *hw)
*/
static void ice_free_irq_msix_misc(struct ice_pf *pf)
{
- int misc_irq_num = pci_irq_vector(pf->pdev, pf->oicr_idx);
+ int misc_irq_num = pf->oicr_irq.virq;
struct ice_hw *hw = &pf->hw;
ice_dis_ctrlq_interrupts(hw);
@@ -3284,8 +3282,7 @@ static void ice_free_irq_msix_misc(struct ice_pf *pf)
synchronize_irq(misc_irq_num);
devm_free_irq(ice_pf_to_dev(pf), misc_irq_num, pf);
- pf->num_avail_sw_msix += 1;
- ice_free_res(pf->irq_tracker, pf->oicr_idx, ICE_RES_MISC_VEC_ID);
+ ice_free_irq(pf, pf->oicr_irq);
}
/**
@@ -3331,7 +3328,8 @@ static int ice_req_irq_msix_misc(struct ice_pf *pf)
{
struct device *dev = ice_pf_to_dev(pf);
struct ice_hw *hw = &pf->hw;
- int oicr_idx, err = 0;
+ struct msi_map oicr_irq;
+ int err = 0;
if (!pf->int_name[0])
snprintf(pf->int_name, sizeof(pf->int_name) - 1, "%s-%s:misc",
@@ -3345,30 +3343,26 @@ static int ice_req_irq_msix_misc(struct ice_pf *pf)
goto skip_req_irq;
/* reserve one vector in irq_tracker for misc interrupts */
- oicr_idx = ice_get_res(pf, pf->irq_tracker, 1, ICE_RES_MISC_VEC_ID);
- if (oicr_idx < 0)
- return oicr_idx;
-
- pf->num_avail_sw_msix -= 1;
- pf->oicr_idx = (u16)oicr_idx;
-
- err = devm_request_threaded_irq(dev,
- pci_irq_vector(pf->pdev, pf->oicr_idx),
- ice_misc_intr, ice_misc_intr_thread_fn,
- 0, pf->int_name, pf);
+ oicr_irq = ice_alloc_irq(pf);
+ if (oicr_irq.index < 0)
+ return oicr_irq.index;
+
+ pf->oicr_irq = oicr_irq;
+ err = devm_request_threaded_irq(dev, pf->oicr_irq.virq, ice_misc_intr,
+ ice_misc_intr_thread_fn, 0,
+ pf->int_name, pf);
if (err) {
dev_err(dev, "devm_request_threaded_irq for %s failed: %d\n",
pf->int_name, err);
- ice_free_res(pf->irq_tracker, 1, ICE_RES_MISC_VEC_ID);
- pf->num_avail_sw_msix += 1;
+ ice_free_irq(pf, pf->oicr_irq);
return err;
}
skip_req_irq:
ice_ena_misc_vector(pf);
- ice_ena_ctrlq_interrupts(hw, pf->oicr_idx);
- wr32(hw, GLINT_ITR(ICE_RX_ITR, pf->oicr_idx),
+ ice_ena_ctrlq_interrupts(hw, pf->oicr_irq.index);
+ wr32(hw, GLINT_ITR(ICE_RX_ITR, pf->oicr_irq.index),
ITR_REG_ALIGN(ICE_ITR_8K) >> ICE_ITR_GRAN_S);
ice_flush(hw);
diff --git a/drivers/net/ethernet/intel/ice/ice_ptp.c b/drivers/net/ethernet/intel/ice/ice_ptp.c
index 972d4f6fd615..d4b6c997141d 100644
--- a/drivers/net/ethernet/intel/ice/ice_ptp.c
+++ b/drivers/net/ethernet/intel/ice/ice_ptp.c
@@ -911,7 +911,7 @@ ice_ptp_release_tx_tracker(struct ice_pf *pf, struct ice_ptp_tx *tx)
spin_unlock(&tx->lock);
/* wait for potentially outstanding interrupt to complete */
- synchronize_irq(pci_irq_vector(pf->pdev, pf->oicr_idx));
+ synchronize_irq(pf->oicr_irq.virq);
ice_ptp_flush_tx_tracker(pf, tx);
diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c
index 65f971b74717..0fc2b26a2fa6 100644
--- a/drivers/net/ethernet/intel/ice/ice_sriov.c
+++ b/drivers/net/ethernet/intel/ice/ice_sriov.c
@@ -835,7 +835,7 @@ static int ice_ena_vfs(struct ice_pf *pf, u16 num_vfs)
int ret;
/* Disable global interrupt 0 so we don't try to handle the VFLR. */
- wr32(hw, GLINT_DYN_CTL(pf->oicr_idx),
+ wr32(hw, GLINT_DYN_CTL(pf->oicr_irq.index),
ICE_ITR_NONE << GLINT_DYN_CTL_ITR_INDX_S);
set_bit(ICE_OICR_INTR_DIS, pf->state);
ice_flush(hw);
diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c
index 4102416d7a41..a7fe2b4ce655 100644
--- a/drivers/net/ethernet/intel/ice/ice_xsk.c
+++ b/drivers/net/ethernet/intel/ice/ice_xsk.c
@@ -90,7 +90,6 @@ ice_qvec_dis_irq(struct ice_vsi *vsi, struct ice_rx_ring *rx_ring,
{
struct ice_pf *pf = vsi->back;
struct ice_hw *hw = &pf->hw;
- int base = vsi->base_vector;
u16 reg;
u32 val;
@@ -103,11 +102,9 @@ ice_qvec_dis_irq(struct ice_vsi *vsi, struct ice_rx_ring *rx_ring,
wr32(hw, QINT_RQCTL(reg), val);
if (q_vector) {
- u16 v_idx = q_vector->v_idx;
-
wr32(hw, GLINT_DYN_CTL(q_vector->reg_idx), 0);
ice_flush(hw);
- synchronize_irq(pci_irq_vector(pf->pdev, v_idx + base));
+ synchronize_irq(q_vector->irq.virq);
}
}
--
2.38.1
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH net-next v3 7/8] ice: track interrupt vectors with xarray
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-03-23 12:24 ` Piotr Raczynski
-1 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, michal.swiatkowski, shiraz.saleem, jacob.e.keller,
sridhar.samudrala, jesse.brandeburg, aleksander.lobakin,
lukasz.czapnik, Piotr Raczynski, Michal Swiatkowski
Replace custom interrupt tracker with generic xarray data structure.
Remove all code responsible for searching for a new entry with xa_alloc,
which always tries to allocate at the lowes possible index. As a result
driver is always using a contiguous region of the MSIX vector table.
New tracker keeps ice_irq_entry entries in xarray as opaque for the rest
of the driver hiding the entry details from the caller.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
---
drivers/net/ethernet/intel/ice/ice.h | 9 +-
drivers/net/ethernet/intel/ice/ice_irq.c | 102 ++++++++++++++++-----
drivers/net/ethernet/intel/ice/ice_irq.h | 9 ++
drivers/net/ethernet/intel/ice/ice_lib.c | 45 ---------
drivers/net/ethernet/intel/ice/ice_lib.h | 5 -
drivers/net/ethernet/intel/ice/ice_sriov.c | 4 +-
6 files changed, 90 insertions(+), 84 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 89d80a2b5feb..b7398abda26a 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -104,7 +104,6 @@
#define ICE_Q_WAIT_RETRY_LIMIT 10
#define ICE_Q_WAIT_MAX_RETRY (5 * ICE_Q_WAIT_RETRY_LIMIT)
#define ICE_MAX_LG_RSS_QS 256
-#define ICE_RES_VALID_BIT 0x8000
#define ICE_INVAL_Q_INDEX 0xffff
#define ICE_MAX_RXQS_PER_TC 256 /* Used when setting VSI context per TC Rx queues */
@@ -242,12 +241,6 @@ struct ice_tc_cfg {
struct ice_tc_info tc_info[ICE_MAX_TRAFFIC_CLASS];
};
-struct ice_res_tracker {
- u16 num_entries;
- u16 end;
- u16 list[];
-};
-
struct ice_qs_cfg {
struct mutex *qs_mutex; /* will be assigned to &pf->avail_q_mutex */
unsigned long *pf_map;
@@ -536,7 +529,7 @@ struct ice_pf {
/* OS reserved IRQ details */
struct msix_entry *msix_entries;
- struct ice_res_tracker *irq_tracker;
+ struct ice_irq_tracker irq_tracker;
/* First MSIX vector used by SR-IOV VFs. Calculated by subtracting the
* number of MSIX vectors needed for all SR-IOV VFs from the number of
* MSIX vectors allowed on this PF.
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
index ca1a1de26766..20d4e9a6aefb 100644
--- a/drivers/net/ethernet/intel/ice/ice_irq.c
+++ b/drivers/net/ethernet/intel/ice/ice_irq.c
@@ -5,6 +5,76 @@
#include "ice_lib.h"
#include "ice_irq.h"
+/**
+ * ice_init_irq_tracker - initialize interrupt tracker
+ * @pf: board private structure
+ * @max_vectors: maximum number of vectors that tracker can hold
+ */
+static void
+ice_init_irq_tracker(struct ice_pf *pf, unsigned int max_vectors)
+{
+ pf->irq_tracker.num_entries = max_vectors;
+ xa_init_flags(&pf->irq_tracker.entries, XA_FLAGS_ALLOC);
+}
+
+/**
+ * ice_deinit_irq_tracker - free xarray tracker
+ * @pf: board private structure
+ */
+static void ice_deinit_irq_tracker(struct ice_pf *pf)
+{
+ xa_destroy(&pf->irq_tracker.entries);
+}
+
+/**
+ * ice_free_irq_res - free a block of resources
+ * @pf: board private structure
+ * @index: starting index previously returned by ice_get_res
+ */
+static void ice_free_irq_res(struct ice_pf *pf, u16 index)
+{
+ struct ice_irq_entry *entry;
+
+ entry = xa_erase(&pf->irq_tracker.entries, index);
+ kfree(entry);
+}
+
+/**
+ * ice_get_irq_res - get an interrupt resource
+ * @pf: board private structure
+ *
+ * Allocate new irq entry in the free slot of the tracker. Since xarray
+ * is used, always allocate new entry at the lowest possible index. Set
+ * proper allocation limit for maximum tracker entries.
+ *
+ * Returns allocated irq entry or NULL on failure.
+ */
+static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf)
+{
+ struct xa_limit limit = { .max = pf->irq_tracker.num_entries,
+ .min = 0 };
+ struct ice_irq_entry *entry;
+ unsigned int index;
+ int ret;
+
+ entry = kzalloc(sizeof(*entry), GFP_KERNEL);
+ if (!entry)
+ goto exit;
+
+ ret = xa_alloc(&pf->irq_tracker.entries, &index, entry, limit,
+ GFP_KERNEL);
+
+ if (ret) {
+ kfree(entry);
+ entry = NULL;
+ } else {
+ entry->index = index;
+ }
+
+exit:
+ return entry;
+}
+
/**
* ice_reduce_msix_usage - Reduce usage of MSI-X vectors
* @pf: board private structure
@@ -163,11 +233,7 @@ static int ice_ena_msix_range(struct ice_pf *pf)
void ice_clear_interrupt_scheme(struct ice_pf *pf)
{
pci_free_irq_vectors(pf->pdev);
-
- if (pf->irq_tracker) {
- devm_kfree(ice_pf_to_dev(pf), pf->irq_tracker);
- pf->irq_tracker = NULL;
- }
+ ice_deinit_irq_tracker(pf);
}
/**
@@ -183,19 +249,7 @@ int ice_init_interrupt_scheme(struct ice_pf *pf)
if (vectors < 0)
return vectors;
- /* set up vector assignment tracking */
- pf->irq_tracker = devm_kzalloc(ice_pf_to_dev(pf),
- struct_size(pf->irq_tracker, list,
- vectors),
- GFP_KERNEL);
- if (!pf->irq_tracker) {
- pci_free_irq_vectors(pf->pdev);
- return -ENOMEM;
- }
-
- /* populate SW interrupts pool with number of OS granted IRQs. */
- pf->irq_tracker->num_entries = (u16)vectors;
- pf->irq_tracker->end = pf->irq_tracker->num_entries;
+ ice_init_irq_tracker(pf, vectors);
return 0;
}
@@ -221,13 +275,13 @@ int ice_init_interrupt_scheme(struct ice_pf *pf)
struct msi_map ice_alloc_irq(struct ice_pf *pf)
{
struct msi_map map = { .index = -ENOENT };
- int entry;
+ struct ice_irq_entry *entry;
- entry = ice_get_res(pf, pf->irq_tracker);
- if (entry < 0)
+ entry = ice_get_irq_res(pf);
+ if (!entry)
return map;
- map.index = entry;
+ map.index = entry->index;
map.virq = pci_irq_vector(pf->pdev, map.index);
return map;
@@ -238,9 +292,9 @@ struct msi_map ice_alloc_irq(struct ice_pf *pf)
* @pf: board private structure
* @map: map with interrupt details
*
- * Remove allocated interrupt from the interrupt tracker
+ * Remove allocated interrupt from the interrupt tracker.
*/
void ice_free_irq(struct ice_pf *pf, struct msi_map map)
{
- ice_free_res(pf->irq_tracker, map.index);
+ ice_free_irq_res(pf, map.index);
}
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.h b/drivers/net/ethernet/intel/ice/ice_irq.h
index 26e80dfe22b5..da5cdb1f0d3a 100644
--- a/drivers/net/ethernet/intel/ice/ice_irq.h
+++ b/drivers/net/ethernet/intel/ice/ice_irq.h
@@ -4,6 +4,15 @@
#ifndef _ICE_IRQ_H_
#define _ICE_IRQ_H_
+struct ice_irq_entry {
+ unsigned int index;
+};
+
+struct ice_irq_tracker {
+ struct xarray entries;
+ u16 num_entries; /* total vectors available */
+};
+
int ice_init_interrupt_scheme(struct ice_pf *pf);
void ice_clear_interrupt_scheme(struct ice_pf *pf);
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 4d29d54ed35d..3d35ee8bdb6c 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -1372,51 +1372,6 @@ static int ice_vsi_init(struct ice_vsi *vsi, u32 vsi_flags)
return ret;
}
-/**
- * ice_free_res - free a block of resources
- * @res: pointer to the resource
- * @index: starting index previously returned by ice_get_res
- *
- * Returns number of resources freed
- */
-int ice_free_res(struct ice_res_tracker *res, u16 index)
-{
- if (!res || index >= res->end)
- return -EINVAL;
-
- res->list[index] = 0;
-
- return 0;
-}
-
-/**
- * ice_get_res - get a resource from the tracker
- * @pf: board private structure
- * @res: pointer to the resource
- *
- * Returns the item index, or negative for error
- */
-int
-ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res)
-{
- u16 i;
-
- if (!res || !pf)
- return -EINVAL;
-
- /* skip already allocated entries */
- for (i = 0; i < res->end; i++)
- if (!(res->list[i] & ICE_RES_VALID_BIT))
- break;
-
- if (i < res->end) {
- res->list[i] = ICE_RES_VALID_BIT;
- return i;
- } else {
- return -ENOMEM;
- }
-}
-
/**
* ice_vsi_clear_rings - Deallocates the Tx and Rx rings for VSI
* @vsi: the VSI having rings deallocated
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.h b/drivers/net/ethernet/intel/ice/ice_lib.h
index 2f52f9e32858..e985766e6bb5 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_lib.h
@@ -104,11 +104,6 @@ int ice_ena_vsi(struct ice_vsi *vsi, bool locked);
void ice_vsi_decfg(struct ice_vsi *vsi);
void ice_dis_vsi(struct ice_vsi *vsi, bool locked);
-int ice_free_res(struct ice_res_tracker *res, u16 index);
-
-int
-ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res);
-
int ice_vsi_rebuild(struct ice_vsi *vsi, u32 vsi_flags);
int ice_vsi_cfg(struct ice_vsi *vsi, struct ice_vsi_cfg_params *params);
diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c
index 0fc2b26a2fa6..195105ce9039 100644
--- a/drivers/net/ethernet/intel/ice/ice_sriov.c
+++ b/drivers/net/ethernet/intel/ice/ice_sriov.c
@@ -418,7 +418,7 @@ int ice_calc_vf_reg_idx(struct ice_vf *vf, struct ice_q_vector *q_vector)
static int ice_sriov_set_msix_res(struct ice_pf *pf, u16 num_msix_needed)
{
u16 total_vectors = pf->hw.func_caps.common_cap.num_msix_vectors;
- int vectors_used = pf->irq_tracker->num_entries;
+ int vectors_used = pf->irq_tracker.num_entries;
int sriov_base_vector;
sriov_base_vector = total_vectors - num_msix_needed;
@@ -470,7 +470,7 @@ static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs)
/* determine MSI-X resources per VF */
msix_avail_for_sriov = pf->hw.func_caps.common_cap.num_msix_vectors -
- pf->irq_tracker->num_entries;
+ pf->irq_tracker.num_entries;
msix_avail_per_vf = msix_avail_for_sriov / num_vfs;
if (msix_avail_per_vf >= ICE_NUM_VF_MSIX_MED) {
num_msix_per_vf = ICE_NUM_VF_MSIX_MED;
--
2.38.1
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Intel-wired-lan] [PATCH net-next v3 7/8] ice: track interrupt vectors with xarray
@ 2023-03-23 12:24 ` Piotr Raczynski
0 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: michal.swiatkowski, netdev, jesse.brandeburg, shiraz.saleem
Replace custom interrupt tracker with generic xarray data structure.
Remove all code responsible for searching for a new entry with xa_alloc,
which always tries to allocate at the lowes possible index. As a result
driver is always using a contiguous region of the MSIX vector table.
New tracker keeps ice_irq_entry entries in xarray as opaque for the rest
of the driver hiding the entry details from the caller.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
---
drivers/net/ethernet/intel/ice/ice.h | 9 +-
drivers/net/ethernet/intel/ice/ice_irq.c | 102 ++++++++++++++++-----
drivers/net/ethernet/intel/ice/ice_irq.h | 9 ++
drivers/net/ethernet/intel/ice/ice_lib.c | 45 ---------
drivers/net/ethernet/intel/ice/ice_lib.h | 5 -
drivers/net/ethernet/intel/ice/ice_sriov.c | 4 +-
6 files changed, 90 insertions(+), 84 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 89d80a2b5feb..b7398abda26a 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -104,7 +104,6 @@
#define ICE_Q_WAIT_RETRY_LIMIT 10
#define ICE_Q_WAIT_MAX_RETRY (5 * ICE_Q_WAIT_RETRY_LIMIT)
#define ICE_MAX_LG_RSS_QS 256
-#define ICE_RES_VALID_BIT 0x8000
#define ICE_INVAL_Q_INDEX 0xffff
#define ICE_MAX_RXQS_PER_TC 256 /* Used when setting VSI context per TC Rx queues */
@@ -242,12 +241,6 @@ struct ice_tc_cfg {
struct ice_tc_info tc_info[ICE_MAX_TRAFFIC_CLASS];
};
-struct ice_res_tracker {
- u16 num_entries;
- u16 end;
- u16 list[];
-};
-
struct ice_qs_cfg {
struct mutex *qs_mutex; /* will be assigned to &pf->avail_q_mutex */
unsigned long *pf_map;
@@ -536,7 +529,7 @@ struct ice_pf {
/* OS reserved IRQ details */
struct msix_entry *msix_entries;
- struct ice_res_tracker *irq_tracker;
+ struct ice_irq_tracker irq_tracker;
/* First MSIX vector used by SR-IOV VFs. Calculated by subtracting the
* number of MSIX vectors needed for all SR-IOV VFs from the number of
* MSIX vectors allowed on this PF.
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
index ca1a1de26766..20d4e9a6aefb 100644
--- a/drivers/net/ethernet/intel/ice/ice_irq.c
+++ b/drivers/net/ethernet/intel/ice/ice_irq.c
@@ -5,6 +5,76 @@
#include "ice_lib.h"
#include "ice_irq.h"
+/**
+ * ice_init_irq_tracker - initialize interrupt tracker
+ * @pf: board private structure
+ * @max_vectors: maximum number of vectors that tracker can hold
+ */
+static void
+ice_init_irq_tracker(struct ice_pf *pf, unsigned int max_vectors)
+{
+ pf->irq_tracker.num_entries = max_vectors;
+ xa_init_flags(&pf->irq_tracker.entries, XA_FLAGS_ALLOC);
+}
+
+/**
+ * ice_deinit_irq_tracker - free xarray tracker
+ * @pf: board private structure
+ */
+static void ice_deinit_irq_tracker(struct ice_pf *pf)
+{
+ xa_destroy(&pf->irq_tracker.entries);
+}
+
+/**
+ * ice_free_irq_res - free a block of resources
+ * @pf: board private structure
+ * @index: starting index previously returned by ice_get_res
+ */
+static void ice_free_irq_res(struct ice_pf *pf, u16 index)
+{
+ struct ice_irq_entry *entry;
+
+ entry = xa_erase(&pf->irq_tracker.entries, index);
+ kfree(entry);
+}
+
+/**
+ * ice_get_irq_res - get an interrupt resource
+ * @pf: board private structure
+ *
+ * Allocate new irq entry in the free slot of the tracker. Since xarray
+ * is used, always allocate new entry at the lowest possible index. Set
+ * proper allocation limit for maximum tracker entries.
+ *
+ * Returns allocated irq entry or NULL on failure.
+ */
+static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf)
+{
+ struct xa_limit limit = { .max = pf->irq_tracker.num_entries,
+ .min = 0 };
+ struct ice_irq_entry *entry;
+ unsigned int index;
+ int ret;
+
+ entry = kzalloc(sizeof(*entry), GFP_KERNEL);
+ if (!entry)
+ goto exit;
+
+ ret = xa_alloc(&pf->irq_tracker.entries, &index, entry, limit,
+ GFP_KERNEL);
+
+ if (ret) {
+ kfree(entry);
+ entry = NULL;
+ } else {
+ entry->index = index;
+ }
+
+exit:
+ return entry;
+}
+
/**
* ice_reduce_msix_usage - Reduce usage of MSI-X vectors
* @pf: board private structure
@@ -163,11 +233,7 @@ static int ice_ena_msix_range(struct ice_pf *pf)
void ice_clear_interrupt_scheme(struct ice_pf *pf)
{
pci_free_irq_vectors(pf->pdev);
-
- if (pf->irq_tracker) {
- devm_kfree(ice_pf_to_dev(pf), pf->irq_tracker);
- pf->irq_tracker = NULL;
- }
+ ice_deinit_irq_tracker(pf);
}
/**
@@ -183,19 +249,7 @@ int ice_init_interrupt_scheme(struct ice_pf *pf)
if (vectors < 0)
return vectors;
- /* set up vector assignment tracking */
- pf->irq_tracker = devm_kzalloc(ice_pf_to_dev(pf),
- struct_size(pf->irq_tracker, list,
- vectors),
- GFP_KERNEL);
- if (!pf->irq_tracker) {
- pci_free_irq_vectors(pf->pdev);
- return -ENOMEM;
- }
-
- /* populate SW interrupts pool with number of OS granted IRQs. */
- pf->irq_tracker->num_entries = (u16)vectors;
- pf->irq_tracker->end = pf->irq_tracker->num_entries;
+ ice_init_irq_tracker(pf, vectors);
return 0;
}
@@ -221,13 +275,13 @@ int ice_init_interrupt_scheme(struct ice_pf *pf)
struct msi_map ice_alloc_irq(struct ice_pf *pf)
{
struct msi_map map = { .index = -ENOENT };
- int entry;
+ struct ice_irq_entry *entry;
- entry = ice_get_res(pf, pf->irq_tracker);
- if (entry < 0)
+ entry = ice_get_irq_res(pf);
+ if (!entry)
return map;
- map.index = entry;
+ map.index = entry->index;
map.virq = pci_irq_vector(pf->pdev, map.index);
return map;
@@ -238,9 +292,9 @@ struct msi_map ice_alloc_irq(struct ice_pf *pf)
* @pf: board private structure
* @map: map with interrupt details
*
- * Remove allocated interrupt from the interrupt tracker
+ * Remove allocated interrupt from the interrupt tracker.
*/
void ice_free_irq(struct ice_pf *pf, struct msi_map map)
{
- ice_free_res(pf->irq_tracker, map.index);
+ ice_free_irq_res(pf, map.index);
}
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.h b/drivers/net/ethernet/intel/ice/ice_irq.h
index 26e80dfe22b5..da5cdb1f0d3a 100644
--- a/drivers/net/ethernet/intel/ice/ice_irq.h
+++ b/drivers/net/ethernet/intel/ice/ice_irq.h
@@ -4,6 +4,15 @@
#ifndef _ICE_IRQ_H_
#define _ICE_IRQ_H_
+struct ice_irq_entry {
+ unsigned int index;
+};
+
+struct ice_irq_tracker {
+ struct xarray entries;
+ u16 num_entries; /* total vectors available */
+};
+
int ice_init_interrupt_scheme(struct ice_pf *pf);
void ice_clear_interrupt_scheme(struct ice_pf *pf);
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index 4d29d54ed35d..3d35ee8bdb6c 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -1372,51 +1372,6 @@ static int ice_vsi_init(struct ice_vsi *vsi, u32 vsi_flags)
return ret;
}
-/**
- * ice_free_res - free a block of resources
- * @res: pointer to the resource
- * @index: starting index previously returned by ice_get_res
- *
- * Returns number of resources freed
- */
-int ice_free_res(struct ice_res_tracker *res, u16 index)
-{
- if (!res || index >= res->end)
- return -EINVAL;
-
- res->list[index] = 0;
-
- return 0;
-}
-
-/**
- * ice_get_res - get a resource from the tracker
- * @pf: board private structure
- * @res: pointer to the resource
- *
- * Returns the item index, or negative for error
- */
-int
-ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res)
-{
- u16 i;
-
- if (!res || !pf)
- return -EINVAL;
-
- /* skip already allocated entries */
- for (i = 0; i < res->end; i++)
- if (!(res->list[i] & ICE_RES_VALID_BIT))
- break;
-
- if (i < res->end) {
- res->list[i] = ICE_RES_VALID_BIT;
- return i;
- } else {
- return -ENOMEM;
- }
-}
-
/**
* ice_vsi_clear_rings - Deallocates the Tx and Rx rings for VSI
* @vsi: the VSI having rings deallocated
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.h b/drivers/net/ethernet/intel/ice/ice_lib.h
index 2f52f9e32858..e985766e6bb5 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_lib.h
@@ -104,11 +104,6 @@ int ice_ena_vsi(struct ice_vsi *vsi, bool locked);
void ice_vsi_decfg(struct ice_vsi *vsi);
void ice_dis_vsi(struct ice_vsi *vsi, bool locked);
-int ice_free_res(struct ice_res_tracker *res, u16 index);
-
-int
-ice_get_res(struct ice_pf *pf, struct ice_res_tracker *res);
-
int ice_vsi_rebuild(struct ice_vsi *vsi, u32 vsi_flags);
int ice_vsi_cfg(struct ice_vsi *vsi, struct ice_vsi_cfg_params *params);
diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c
index 0fc2b26a2fa6..195105ce9039 100644
--- a/drivers/net/ethernet/intel/ice/ice_sriov.c
+++ b/drivers/net/ethernet/intel/ice/ice_sriov.c
@@ -418,7 +418,7 @@ int ice_calc_vf_reg_idx(struct ice_vf *vf, struct ice_q_vector *q_vector)
static int ice_sriov_set_msix_res(struct ice_pf *pf, u16 num_msix_needed)
{
u16 total_vectors = pf->hw.func_caps.common_cap.num_msix_vectors;
- int vectors_used = pf->irq_tracker->num_entries;
+ int vectors_used = pf->irq_tracker.num_entries;
int sriov_base_vector;
sriov_base_vector = total_vectors - num_msix_needed;
@@ -470,7 +470,7 @@ static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs)
/* determine MSI-X resources per VF */
msix_avail_for_sriov = pf->hw.func_caps.common_cap.num_msix_vectors -
- pf->irq_tracker->num_entries;
+ pf->irq_tracker.num_entries;
msix_avail_per_vf = msix_avail_for_sriov / num_vfs;
if (msix_avail_per_vf >= ICE_NUM_VF_MSIX_MED) {
num_msix_per_vf = ICE_NUM_VF_MSIX_MED;
--
2.38.1
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH net-next v3 8/8] ice: add dynamic interrupt allocation
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-03-23 12:24 ` Piotr Raczynski
-1 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: netdev, michal.swiatkowski, shiraz.saleem, jacob.e.keller,
sridhar.samudrala, jesse.brandeburg, aleksander.lobakin,
lukasz.czapnik, Piotr Raczynski, Michal Swiatkowski
Currently driver can only allocate interrupt vectors during init phase by
calling pci_alloc_irq_vectors. Change that and make use of new
pci_msix_alloc_irq_at/pci_msix_free_irq API and enable to allocate and free
more interrupts after MSIX has been enabled. Since not all platforms
supports dynamic allocation, check it with pci_msix_can_alloc_dyn.
Extend the tracker to keep track how many interrupts are allocated
initially so when all such vectors are already used, additional interrupts
are automatically allocated dynamically. Remember each interrupt allocation
method to then free appropriately. Since some features may require
interrupts allocated dynamically add appropriate VSI flag and take it into
account when allocating new interrupt.
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
---
drivers/net/ethernet/intel/ice/ice.h | 3 +
drivers/net/ethernet/intel/ice/ice_base.c | 2 +-
drivers/net/ethernet/intel/ice/ice_idc.c | 2 +-
drivers/net/ethernet/intel/ice/ice_irq.c | 107 ++++++++++++++++++---
drivers/net/ethernet/intel/ice/ice_irq.h | 5 +-
drivers/net/ethernet/intel/ice/ice_main.c | 2 +-
drivers/net/ethernet/intel/ice/ice_sriov.c | 5 +-
7 files changed, 105 insertions(+), 21 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index b7398abda26a..26fa176dc1cb 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -338,6 +338,9 @@ struct ice_vsi {
u32 rx_buf_failed;
u32 rx_page_failed;
u16 num_q_vectors;
+ /* tell if only dynamic irq allocation is allowed */
+ bool irq_dyn_alloc;
+
enum ice_vsi_type type;
u16 vsi_num; /* HW (absolute) index of this VSI */
u16 idx; /* software index in pf->vsi[] */
diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c
index e5db23eaa3f4..a0c0129c995d 100644
--- a/drivers/net/ethernet/intel/ice/ice_base.c
+++ b/drivers/net/ethernet/intel/ice/ice_base.c
@@ -134,7 +134,7 @@ static int ice_vsi_alloc_q_vector(struct ice_vsi *vsi, u16 v_idx)
}
}
- q_vector->irq = ice_alloc_irq(pf);
+ q_vector->irq = ice_alloc_irq(pf, vsi->irq_dyn_alloc);
if (q_vector->irq.index < 0) {
kfree(q_vector);
return -ENOMEM;
diff --git a/drivers/net/ethernet/intel/ice/ice_idc.c b/drivers/net/ethernet/intel/ice/ice_idc.c
index bc016bb4440c..145b27f2a4ce 100644
--- a/drivers/net/ethernet/intel/ice/ice_idc.c
+++ b/drivers/net/ethernet/intel/ice/ice_idc.c
@@ -250,7 +250,7 @@ static int ice_alloc_rdma_qvectors(struct ice_pf *pf)
struct msix_entry *entry = &pf->msix_entries[i];
struct msi_map map;
- map = ice_alloc_irq(pf);
+ map = ice_alloc_irq(pf, false);
if (map.index < 0)
break;
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
index 20d4e9a6aefb..61120d4194f1 100644
--- a/drivers/net/ethernet/intel/ice/ice_irq.c
+++ b/drivers/net/ethernet/intel/ice/ice_irq.c
@@ -9,11 +9,14 @@
* ice_init_irq_tracker - initialize interrupt tracker
* @pf: board private structure
* @max_vectors: maximum number of vectors that tracker can hold
+ * @num_static: number of preallocated interrupts
*/
static void
-ice_init_irq_tracker(struct ice_pf *pf, unsigned int max_vectors)
+ice_init_irq_tracker(struct ice_pf *pf, unsigned int max_vectors,
+ unsigned int num_static)
{
pf->irq_tracker.num_entries = max_vectors;
+ pf->irq_tracker.num_static = num_static;
xa_init_flags(&pf->irq_tracker.entries, XA_FLAGS_ALLOC);
}
@@ -42,6 +45,7 @@ static void ice_free_irq_res(struct ice_pf *pf, u16 index)
/**
* ice_get_irq_res - get an interrupt resource
* @pf: board private structure
+ * @dyn_only: force entry to be dynamically allocated
*
* Allocate new irq entry in the free slot of the tracker. Since xarray
* is used, always allocate new entry at the lowest possible index. Set
@@ -49,10 +53,11 @@ static void ice_free_irq_res(struct ice_pf *pf, u16 index)
*
* Returns allocated irq entry or NULL on failure.
*/
-static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf)
+static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf, bool dyn_only)
{
struct xa_limit limit = { .max = pf->irq_tracker.num_entries,
.min = 0 };
+ unsigned int num_static = pf->irq_tracker.num_static;
struct ice_irq_entry *entry;
unsigned int index;
int ret;
@@ -61,6 +66,10 @@ static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf)
if (!entry)
goto exit;
+ /* skip preallocated entries if the caller says so */
+ if (dyn_only)
+ limit.min = num_static;
+
ret = xa_alloc(&pf->irq_tracker.entries, &index, entry, limit,
GFP_KERNEL);
@@ -69,6 +78,7 @@ static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf)
entry = NULL;
} else {
entry->index = index;
+ entry->dynamic = index >= num_static;
}
exit:
@@ -242,14 +252,20 @@ void ice_clear_interrupt_scheme(struct ice_pf *pf)
*/
int ice_init_interrupt_scheme(struct ice_pf *pf)
{
- int vectors;
+ int total_vectors = pf->hw.func_caps.common_cap.num_msix_vectors;
+ int vectors, max_vectors;
vectors = ice_ena_msix_range(pf);
if (vectors < 0)
- return vectors;
+ return -ENOMEM;
+
+ if (pci_msix_can_alloc_dyn(pf->pdev))
+ max_vectors = total_vectors;
+ else
+ max_vectors = vectors;
- ice_init_irq_tracker(pf, vectors);
+ ice_init_irq_tracker(pf, max_vectors, vectors);
return 0;
}
@@ -257,33 +273,55 @@ int ice_init_interrupt_scheme(struct ice_pf *pf)
/**
* ice_alloc_irq - Allocate new interrupt vector
* @pf: board private structure
+ * @dyn_only: force dynamic allocation of the interrupt
*
* Allocate new interrupt vector for a given owner id.
* return struct msi_map with interrupt details and track
* allocated interrupt appropriately.
*
- * This function mimics individual interrupt allocation,
- * even interrupts are actually already allocated with
- * pci_alloc_irq_vectors. Individual allocation helps
- * to track interrupts and simplifies interrupt related
- * handling.
+ * This function reserves new irq entry from the irq_tracker.
+ * if according to the tracker information all interrupts that
+ * were allocated with ice_pci_alloc_irq_vectors are already used
+ * and dynamically allocated interrupts are supported then new
+ * interrupt will be allocated with pci_msix_alloc_irq_at.
+ *
+ * Some callers may only support dynamically allocated interrupts.
+ * This is indicated with dyn_only flag.
*
* On failure, return map with negative .index. The caller
* is expected to check returned map index.
*
*/
-struct msi_map ice_alloc_irq(struct ice_pf *pf)
+struct msi_map ice_alloc_irq(struct ice_pf *pf, bool dyn_only)
{
+ int sriov_base_vector = pf->sriov_base_vector;
struct msi_map map = { .index = -ENOENT };
+ struct device *dev = ice_pf_to_dev(pf);
struct ice_irq_entry *entry;
- entry = ice_get_irq_res(pf);
+ entry = ice_get_irq_res(pf, dyn_only);
if (!entry)
return map;
- map.index = entry->index;
- map.virq = pci_irq_vector(pf->pdev, map.index);
+ /* fail if we're about to violate SRIOV vectors space */
+ if (sriov_base_vector && entry->index >= sriov_base_vector)
+ goto exit_free_res;
+
+ if (pci_msix_can_alloc_dyn(pf->pdev) && entry->dynamic) {
+ map = pci_msix_alloc_irq_at(pf->pdev, entry->index, NULL);
+ if (map.index < 0)
+ goto exit_free_res;
+ dev_dbg(dev, "allocated new irq at index %d\n", map.index);
+ } else {
+ map.index = entry->index;
+ map.virq = pci_irq_vector(pf->pdev, map.index);
+ }
+
+ return map;
+exit_free_res:
+ dev_err(dev, "Could not allocate irq at idx %d\n", entry->index);
+ ice_free_irq_res(pf, entry->index);
return map;
}
@@ -292,9 +330,48 @@ struct msi_map ice_alloc_irq(struct ice_pf *pf)
* @pf: board private structure
* @map: map with interrupt details
*
- * Remove allocated interrupt from the interrupt tracker.
+ * Remove allocated interrupt from the interrupt tracker. If interrupt was
+ * allocated dynamically, free respective interrupt vector.
*/
void ice_free_irq(struct ice_pf *pf, struct msi_map map)
{
+ struct ice_irq_entry *entry;
+
+ entry = xa_load(&pf->irq_tracker.entries, map.index);
+
+ if (!entry)
+ dev_err(ice_pf_to_dev(pf), "Failed to get MSIX interrupt entry at index %d",
+ map.index);
+
+ dev_dbg(ice_pf_to_dev(pf), "Free irq at index %d\n", map.index);
+
+ if (entry->dynamic)
+ pci_msix_free_irq(pf->pdev, map);
+
ice_free_irq_res(pf, map.index);
}
+
+/**
+ * ice_get_max_used_msix_vector - Get the max used interrupt vector
+ * @pf: board private structure
+ *
+ * Return index of maximum used interrupt vectors with respect to the
+ * beginning of the MSIX table. Take into account that some interrupts
+ * may have been dynamically allocated after MSIX was initially enabled.
+ */
+int ice_get_max_used_msix_vector(struct ice_pf *pf)
+{
+ unsigned long start, index, max_idx;
+ void *entry;
+
+ /* Treat all preallocated interrupts as used */
+ start = pf->irq_tracker.num_static;
+ max_idx = start - 1;
+
+ xa_for_each_start(&pf->irq_tracker.entries, index, entry, start) {
+ if (index > max_idx)
+ max_idx = index;
+ }
+
+ return max_idx;
+}
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.h b/drivers/net/ethernet/intel/ice/ice_irq.h
index da5cdb1f0d3a..f35efc08575e 100644
--- a/drivers/net/ethernet/intel/ice/ice_irq.h
+++ b/drivers/net/ethernet/intel/ice/ice_irq.h
@@ -6,17 +6,20 @@
struct ice_irq_entry {
unsigned int index;
+ bool dynamic; /* allocation type flag */
};
struct ice_irq_tracker {
struct xarray entries;
u16 num_entries; /* total vectors available */
+ u16 num_static; /* preallocated entries */
};
int ice_init_interrupt_scheme(struct ice_pf *pf);
void ice_clear_interrupt_scheme(struct ice_pf *pf);
-struct msi_map ice_alloc_irq(struct ice_pf *pf);
+struct msi_map ice_alloc_irq(struct ice_pf *pf, bool dyn_only);
void ice_free_irq(struct ice_pf *pf, struct msi_map map);
+int ice_get_max_used_msix_vector(struct ice_pf *pf);
#endif
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 8e62ec08f582..68ecb80ec0c8 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -3343,7 +3343,7 @@ static int ice_req_irq_msix_misc(struct ice_pf *pf)
goto skip_req_irq;
/* reserve one vector in irq_tracker for misc interrupts */
- oicr_irq = ice_alloc_irq(pf);
+ oicr_irq = ice_alloc_irq(pf, false);
if (oicr_irq.index < 0)
return oicr_irq.index;
diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c
index 195105ce9039..80c643fb9f2f 100644
--- a/drivers/net/ethernet/intel/ice/ice_sriov.c
+++ b/drivers/net/ethernet/intel/ice/ice_sriov.c
@@ -418,7 +418,7 @@ int ice_calc_vf_reg_idx(struct ice_vf *vf, struct ice_q_vector *q_vector)
static int ice_sriov_set_msix_res(struct ice_pf *pf, u16 num_msix_needed)
{
u16 total_vectors = pf->hw.func_caps.common_cap.num_msix_vectors;
- int vectors_used = pf->irq_tracker.num_entries;
+ int vectors_used = ice_get_max_used_msix_vector(pf);
int sriov_base_vector;
sriov_base_vector = total_vectors - num_msix_needed;
@@ -458,6 +458,7 @@ static int ice_sriov_set_msix_res(struct ice_pf *pf, u16 num_msix_needed)
*/
static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs)
{
+ int vectors_used = ice_get_max_used_msix_vector(pf);
u16 num_msix_per_vf, num_txq, num_rxq, avail_qs;
int msix_avail_per_vf, msix_avail_for_sriov;
struct device *dev = ice_pf_to_dev(pf);
@@ -470,7 +471,7 @@ static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs)
/* determine MSI-X resources per VF */
msix_avail_for_sriov = pf->hw.func_caps.common_cap.num_msix_vectors -
- pf->irq_tracker.num_entries;
+ vectors_used;
msix_avail_per_vf = msix_avail_for_sriov / num_vfs;
if (msix_avail_per_vf >= ICE_NUM_VF_MSIX_MED) {
num_msix_per_vf = ICE_NUM_VF_MSIX_MED;
--
2.38.1
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [Intel-wired-lan] [PATCH net-next v3 8/8] ice: add dynamic interrupt allocation
@ 2023-03-23 12:24 ` Piotr Raczynski
0 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-23 12:24 UTC (permalink / raw)
To: intel-wired-lan
Cc: michal.swiatkowski, netdev, jesse.brandeburg, shiraz.saleem
Currently driver can only allocate interrupt vectors during init phase by
calling pci_alloc_irq_vectors. Change that and make use of new
pci_msix_alloc_irq_at/pci_msix_free_irq API and enable to allocate and free
more interrupts after MSIX has been enabled. Since not all platforms
supports dynamic allocation, check it with pci_msix_can_alloc_dyn.
Extend the tracker to keep track how many interrupts are allocated
initially so when all such vectors are already used, additional interrupts
are automatically allocated dynamically. Remember each interrupt allocation
method to then free appropriately. Since some features may require
interrupts allocated dynamically add appropriate VSI flag and take it into
account when allocating new interrupt.
Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
---
drivers/net/ethernet/intel/ice/ice.h | 3 +
drivers/net/ethernet/intel/ice/ice_base.c | 2 +-
drivers/net/ethernet/intel/ice/ice_idc.c | 2 +-
drivers/net/ethernet/intel/ice/ice_irq.c | 107 ++++++++++++++++++---
drivers/net/ethernet/intel/ice/ice_irq.h | 5 +-
drivers/net/ethernet/intel/ice/ice_main.c | 2 +-
drivers/net/ethernet/intel/ice/ice_sriov.c | 5 +-
7 files changed, 105 insertions(+), 21 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index b7398abda26a..26fa176dc1cb 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -338,6 +338,9 @@ struct ice_vsi {
u32 rx_buf_failed;
u32 rx_page_failed;
u16 num_q_vectors;
+ /* tell if only dynamic irq allocation is allowed */
+ bool irq_dyn_alloc;
+
enum ice_vsi_type type;
u16 vsi_num; /* HW (absolute) index of this VSI */
u16 idx; /* software index in pf->vsi[] */
diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c
index e5db23eaa3f4..a0c0129c995d 100644
--- a/drivers/net/ethernet/intel/ice/ice_base.c
+++ b/drivers/net/ethernet/intel/ice/ice_base.c
@@ -134,7 +134,7 @@ static int ice_vsi_alloc_q_vector(struct ice_vsi *vsi, u16 v_idx)
}
}
- q_vector->irq = ice_alloc_irq(pf);
+ q_vector->irq = ice_alloc_irq(pf, vsi->irq_dyn_alloc);
if (q_vector->irq.index < 0) {
kfree(q_vector);
return -ENOMEM;
diff --git a/drivers/net/ethernet/intel/ice/ice_idc.c b/drivers/net/ethernet/intel/ice/ice_idc.c
index bc016bb4440c..145b27f2a4ce 100644
--- a/drivers/net/ethernet/intel/ice/ice_idc.c
+++ b/drivers/net/ethernet/intel/ice/ice_idc.c
@@ -250,7 +250,7 @@ static int ice_alloc_rdma_qvectors(struct ice_pf *pf)
struct msix_entry *entry = &pf->msix_entries[i];
struct msi_map map;
- map = ice_alloc_irq(pf);
+ map = ice_alloc_irq(pf, false);
if (map.index < 0)
break;
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
index 20d4e9a6aefb..61120d4194f1 100644
--- a/drivers/net/ethernet/intel/ice/ice_irq.c
+++ b/drivers/net/ethernet/intel/ice/ice_irq.c
@@ -9,11 +9,14 @@
* ice_init_irq_tracker - initialize interrupt tracker
* @pf: board private structure
* @max_vectors: maximum number of vectors that tracker can hold
+ * @num_static: number of preallocated interrupts
*/
static void
-ice_init_irq_tracker(struct ice_pf *pf, unsigned int max_vectors)
+ice_init_irq_tracker(struct ice_pf *pf, unsigned int max_vectors,
+ unsigned int num_static)
{
pf->irq_tracker.num_entries = max_vectors;
+ pf->irq_tracker.num_static = num_static;
xa_init_flags(&pf->irq_tracker.entries, XA_FLAGS_ALLOC);
}
@@ -42,6 +45,7 @@ static void ice_free_irq_res(struct ice_pf *pf, u16 index)
/**
* ice_get_irq_res - get an interrupt resource
* @pf: board private structure
+ * @dyn_only: force entry to be dynamically allocated
*
* Allocate new irq entry in the free slot of the tracker. Since xarray
* is used, always allocate new entry at the lowest possible index. Set
@@ -49,10 +53,11 @@ static void ice_free_irq_res(struct ice_pf *pf, u16 index)
*
* Returns allocated irq entry or NULL on failure.
*/
-static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf)
+static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf, bool dyn_only)
{
struct xa_limit limit = { .max = pf->irq_tracker.num_entries,
.min = 0 };
+ unsigned int num_static = pf->irq_tracker.num_static;
struct ice_irq_entry *entry;
unsigned int index;
int ret;
@@ -61,6 +66,10 @@ static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf)
if (!entry)
goto exit;
+ /* skip preallocated entries if the caller says so */
+ if (dyn_only)
+ limit.min = num_static;
+
ret = xa_alloc(&pf->irq_tracker.entries, &index, entry, limit,
GFP_KERNEL);
@@ -69,6 +78,7 @@ static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf)
entry = NULL;
} else {
entry->index = index;
+ entry->dynamic = index >= num_static;
}
exit:
@@ -242,14 +252,20 @@ void ice_clear_interrupt_scheme(struct ice_pf *pf)
*/
int ice_init_interrupt_scheme(struct ice_pf *pf)
{
- int vectors;
+ int total_vectors = pf->hw.func_caps.common_cap.num_msix_vectors;
+ int vectors, max_vectors;
vectors = ice_ena_msix_range(pf);
if (vectors < 0)
- return vectors;
+ return -ENOMEM;
+
+ if (pci_msix_can_alloc_dyn(pf->pdev))
+ max_vectors = total_vectors;
+ else
+ max_vectors = vectors;
- ice_init_irq_tracker(pf, vectors);
+ ice_init_irq_tracker(pf, max_vectors, vectors);
return 0;
}
@@ -257,33 +273,55 @@ int ice_init_interrupt_scheme(struct ice_pf *pf)
/**
* ice_alloc_irq - Allocate new interrupt vector
* @pf: board private structure
+ * @dyn_only: force dynamic allocation of the interrupt
*
* Allocate new interrupt vector for a given owner id.
* return struct msi_map with interrupt details and track
* allocated interrupt appropriately.
*
- * This function mimics individual interrupt allocation,
- * even interrupts are actually already allocated with
- * pci_alloc_irq_vectors. Individual allocation helps
- * to track interrupts and simplifies interrupt related
- * handling.
+ * This function reserves new irq entry from the irq_tracker.
+ * if according to the tracker information all interrupts that
+ * were allocated with ice_pci_alloc_irq_vectors are already used
+ * and dynamically allocated interrupts are supported then new
+ * interrupt will be allocated with pci_msix_alloc_irq_at.
+ *
+ * Some callers may only support dynamically allocated interrupts.
+ * This is indicated with dyn_only flag.
*
* On failure, return map with negative .index. The caller
* is expected to check returned map index.
*
*/
-struct msi_map ice_alloc_irq(struct ice_pf *pf)
+struct msi_map ice_alloc_irq(struct ice_pf *pf, bool dyn_only)
{
+ int sriov_base_vector = pf->sriov_base_vector;
struct msi_map map = { .index = -ENOENT };
+ struct device *dev = ice_pf_to_dev(pf);
struct ice_irq_entry *entry;
- entry = ice_get_irq_res(pf);
+ entry = ice_get_irq_res(pf, dyn_only);
if (!entry)
return map;
- map.index = entry->index;
- map.virq = pci_irq_vector(pf->pdev, map.index);
+ /* fail if we're about to violate SRIOV vectors space */
+ if (sriov_base_vector && entry->index >= sriov_base_vector)
+ goto exit_free_res;
+
+ if (pci_msix_can_alloc_dyn(pf->pdev) && entry->dynamic) {
+ map = pci_msix_alloc_irq_at(pf->pdev, entry->index, NULL);
+ if (map.index < 0)
+ goto exit_free_res;
+ dev_dbg(dev, "allocated new irq at index %d\n", map.index);
+ } else {
+ map.index = entry->index;
+ map.virq = pci_irq_vector(pf->pdev, map.index);
+ }
+
+ return map;
+exit_free_res:
+ dev_err(dev, "Could not allocate irq at idx %d\n", entry->index);
+ ice_free_irq_res(pf, entry->index);
return map;
}
@@ -292,9 +330,48 @@ struct msi_map ice_alloc_irq(struct ice_pf *pf)
* @pf: board private structure
* @map: map with interrupt details
*
- * Remove allocated interrupt from the interrupt tracker.
+ * Remove allocated interrupt from the interrupt tracker. If interrupt was
+ * allocated dynamically, free respective interrupt vector.
*/
void ice_free_irq(struct ice_pf *pf, struct msi_map map)
{
+ struct ice_irq_entry *entry;
+
+ entry = xa_load(&pf->irq_tracker.entries, map.index);
+
+ if (!entry)
+ dev_err(ice_pf_to_dev(pf), "Failed to get MSIX interrupt entry at index %d",
+ map.index);
+
+ dev_dbg(ice_pf_to_dev(pf), "Free irq at index %d\n", map.index);
+
+ if (entry->dynamic)
+ pci_msix_free_irq(pf->pdev, map);
+
ice_free_irq_res(pf, map.index);
}
+
+/**
+ * ice_get_max_used_msix_vector - Get the max used interrupt vector
+ * @pf: board private structure
+ *
+ * Return index of maximum used interrupt vectors with respect to the
+ * beginning of the MSIX table. Take into account that some interrupts
+ * may have been dynamically allocated after MSIX was initially enabled.
+ */
+int ice_get_max_used_msix_vector(struct ice_pf *pf)
+{
+ unsigned long start, index, max_idx;
+ void *entry;
+
+ /* Treat all preallocated interrupts as used */
+ start = pf->irq_tracker.num_static;
+ max_idx = start - 1;
+
+ xa_for_each_start(&pf->irq_tracker.entries, index, entry, start) {
+ if (index > max_idx)
+ max_idx = index;
+ }
+
+ return max_idx;
+}
diff --git a/drivers/net/ethernet/intel/ice/ice_irq.h b/drivers/net/ethernet/intel/ice/ice_irq.h
index da5cdb1f0d3a..f35efc08575e 100644
--- a/drivers/net/ethernet/intel/ice/ice_irq.h
+++ b/drivers/net/ethernet/intel/ice/ice_irq.h
@@ -6,17 +6,20 @@
struct ice_irq_entry {
unsigned int index;
+ bool dynamic; /* allocation type flag */
};
struct ice_irq_tracker {
struct xarray entries;
u16 num_entries; /* total vectors available */
+ u16 num_static; /* preallocated entries */
};
int ice_init_interrupt_scheme(struct ice_pf *pf);
void ice_clear_interrupt_scheme(struct ice_pf *pf);
-struct msi_map ice_alloc_irq(struct ice_pf *pf);
+struct msi_map ice_alloc_irq(struct ice_pf *pf, bool dyn_only);
void ice_free_irq(struct ice_pf *pf, struct msi_map map);
+int ice_get_max_used_msix_vector(struct ice_pf *pf);
#endif
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 8e62ec08f582..68ecb80ec0c8 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -3343,7 +3343,7 @@ static int ice_req_irq_msix_misc(struct ice_pf *pf)
goto skip_req_irq;
/* reserve one vector in irq_tracker for misc interrupts */
- oicr_irq = ice_alloc_irq(pf);
+ oicr_irq = ice_alloc_irq(pf, false);
if (oicr_irq.index < 0)
return oicr_irq.index;
diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c
index 195105ce9039..80c643fb9f2f 100644
--- a/drivers/net/ethernet/intel/ice/ice_sriov.c
+++ b/drivers/net/ethernet/intel/ice/ice_sriov.c
@@ -418,7 +418,7 @@ int ice_calc_vf_reg_idx(struct ice_vf *vf, struct ice_q_vector *q_vector)
static int ice_sriov_set_msix_res(struct ice_pf *pf, u16 num_msix_needed)
{
u16 total_vectors = pf->hw.func_caps.common_cap.num_msix_vectors;
- int vectors_used = pf->irq_tracker.num_entries;
+ int vectors_used = ice_get_max_used_msix_vector(pf);
int sriov_base_vector;
sriov_base_vector = total_vectors - num_msix_needed;
@@ -458,6 +458,7 @@ static int ice_sriov_set_msix_res(struct ice_pf *pf, u16 num_msix_needed)
*/
static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs)
{
+ int vectors_used = ice_get_max_used_msix_vector(pf);
u16 num_msix_per_vf, num_txq, num_rxq, avail_qs;
int msix_avail_per_vf, msix_avail_for_sriov;
struct device *dev = ice_pf_to_dev(pf);
@@ -470,7 +471,7 @@ static int ice_set_per_vf_res(struct ice_pf *pf, u16 num_vfs)
/* determine MSI-X resources per VF */
msix_avail_for_sriov = pf->hw.func_caps.common_cap.num_msix_vectors -
- pf->irq_tracker.num_entries;
+ vectors_used;
msix_avail_per_vf = msix_avail_for_sriov / num_vfs;
if (msix_avail_per_vf >= ICE_NUM_VF_MSIX_MED) {
num_msix_per_vf = ICE_NUM_VF_MSIX_MED;
--
2.38.1
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply related [flat|nested] 55+ messages in thread
* Re: [PATCH net-next v3 6/8] ice: add individual interrupt allocation
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-03-26 13:18 ` Simon Horman
-1 siblings, 0 replies; 55+ messages in thread
From: Simon Horman @ 2023-03-26 13:18 UTC (permalink / raw)
To: Piotr Raczynski
Cc: intel-wired-lan, netdev, michal.swiatkowski, shiraz.saleem,
jacob.e.keller, sridhar.samudrala, jesse.brandeburg,
aleksander.lobakin, lukasz.czapnik, Michal Swiatkowski
On Thu, Mar 23, 2023 at 01:24:38PM +0100, Piotr Raczynski wrote:
> Currently interrupt allocations, depending on a feature are distributed
> in batches. Also, after allocation there is a series of operations that
> distributes per irq settings through that batch of interrupts.
>
> Although driver does not yet support dynamic interrupt allocation, keep
> allocated interrupts in a pool and add allocation abstraction logic to
> make code more flexible. Keep per interrupt information in the
> ice_q_vector structure, which yields ice_vsi::base_vector redundant.
> Also, as a result there are a few functions that can be removed.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
I've made a few minor observations inline.
I don't think there is a need to respin for any of them.
> ---
> drivers/net/ethernet/intel/ice/ice.h | 11 +-
> drivers/net/ethernet/intel/ice/ice_arfs.c | 5 +-
> drivers/net/ethernet/intel/ice/ice_base.c | 36 ++-
> drivers/net/ethernet/intel/ice/ice_ethtool.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_idc.c | 45 ++--
> drivers/net/ethernet/intel/ice/ice_irq.c | 46 +++-
> drivers/net/ethernet/intel/ice/ice_irq.h | 3 +
> drivers/net/ethernet/intel/ice/ice_lib.c | 225 ++-----------------
Nice code removal from ice_lib.c :)
> drivers/net/ethernet/intel/ice/ice_lib.h | 4 +-
> drivers/net/ethernet/intel/ice/ice_main.c | 44 ++--
> drivers/net/ethernet/intel/ice/ice_ptp.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_sriov.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_xsk.c | 5 +-
> 13 files changed, 154 insertions(+), 276 deletions(-)
...
> diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c
> index 1911d644dfa8..e5db23eaa3f4 100644
> --- a/drivers/net/ethernet/intel/ice/ice_base.c
> +++ b/drivers/net/ethernet/intel/ice/ice_base.c
> @@ -118,9 +118,31 @@ static int ice_vsi_alloc_q_vector(struct ice_vsi *vsi, u16 v_idx)
> q_vector->rx.itr_mode = ITR_DYNAMIC;
> q_vector->tx.type = ICE_TX_CONTAINER;
> q_vector->rx.type = ICE_RX_CONTAINER;
> + q_vector->irq.index = -ENOENT;
>
> - if (vsi->type == ICE_VSI_VF)
> + if (vsi->type == ICE_VSI_VF) {
> + q_vector->reg_idx = ice_calc_vf_reg_idx(vsi->vf, q_vector);
> goto out;
> + } else if (vsi->type == ICE_VSI_CTRL && vsi->vf) {
> + struct ice_vsi *ctrl_vsi = ice_get_vf_ctrl_vsi(pf, vsi);
> +
> + if (ctrl_vsi) {
> + if (unlikely(!ctrl_vsi->q_vectors))
> + return -ENOENT;
> + q_vector->irq = ctrl_vsi->q_vectors[0]->irq;
> + goto skip_alloc;
nit: I think goto for error paths is very much the norm.
But, FWIIW, I would have avoided using goto here.
> + }
> + }
> +
> + q_vector->irq = ice_alloc_irq(pf);
> + if (q_vector->irq.index < 0) {
> + kfree(q_vector);
> + return -ENOMEM;
> + }
> +
> +skip_alloc:
> + q_vector->reg_idx = q_vector->irq.index;
> +
> /* only set affinity_mask if the CPU is online */
> if (cpu_online(v_idx))
> cpumask_set_cpu(v_idx, &q_vector->affinity_mask);
> @@ -168,6 +190,18 @@ static void ice_free_q_vector(struct ice_vsi *vsi, int v_idx)
> if (vsi->netdev)
> netif_napi_del(&q_vector->napi);
>
> + /* release MSIX interrupt if q_vector had interrupt allocated */
> + if (q_vector->irq.index < 0)
> + goto free_q_vector;
> +
> + /* only free last VF ctrl vsi interrupt */
> + if (vsi->type == ICE_VSI_CTRL && vsi->vf &&
> + ice_get_vf_ctrl_vsi(pf, vsi))
> + goto free_q_vector;
Ditto (x2).
> +
> + ice_free_irq(pf, q_vector->irq);
> +
> +free_q_vector:
> devm_kfree(dev, q_vector);
> vsi->q_vectors[v_idx] = NULL;
> }
...
> diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
> index f61be5d76373..ca1a1de26766 100644
> --- a/drivers/net/ethernet/intel/ice/ice_irq.c
> +++ b/drivers/net/ethernet/intel/ice/ice_irq.c
> @@ -194,9 +194,53 @@ int ice_init_interrupt_scheme(struct ice_pf *pf)
> }
>
> /* populate SW interrupts pool with number of OS granted IRQs. */
> - pf->num_avail_sw_msix = (u16)vectors;
> pf->irq_tracker->num_entries = (u16)vectors;
> pf->irq_tracker->end = pf->irq_tracker->num_entries;
>
> return 0;
> }
> +
> +/**
> + * ice_alloc_irq - Allocate new interrupt vector
> + * @pf: board private structure
> + *
> + * Allocate new interrupt vector for a given owner id.
> + * return struct msi_map with interrupt details and track
> + * allocated interrupt appropriately.
> + *
> + * This function mimics individual interrupt allocation,
> + * even interrupts are actually already allocated with
> + * pci_alloc_irq_vectors. Individual allocation helps
> + * to track interrupts and simplifies interrupt related
> + * handling.
> + *
> + * On failure, return map with negative .index. The caller
> + * is expected to check returned map index.
> + *
> + */
> +struct msi_map ice_alloc_irq(struct ice_pf *pf)
> +{
> + struct msi_map map = { .index = -ENOENT };
> + int entry;
> +
> + entry = ice_get_res(pf, pf->irq_tracker);
> + if (entry < 0)
nit: map.index could be initialised here.
> + return map;
> +
> + map.index = entry;
> + map.virq = pci_irq_vector(pf->pdev, map.index);
> +
> + return map;
> +}
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 6/8] ice: add individual interrupt allocation
@ 2023-03-26 13:18 ` Simon Horman
0 siblings, 0 replies; 55+ messages in thread
From: Simon Horman @ 2023-03-26 13:18 UTC (permalink / raw)
To: Piotr Raczynski
Cc: michal.swiatkowski, netdev, jesse.brandeburg, intel-wired-lan,
shiraz.saleem
On Thu, Mar 23, 2023 at 01:24:38PM +0100, Piotr Raczynski wrote:
> Currently interrupt allocations, depending on a feature are distributed
> in batches. Also, after allocation there is a series of operations that
> distributes per irq settings through that batch of interrupts.
>
> Although driver does not yet support dynamic interrupt allocation, keep
> allocated interrupts in a pool and add allocation abstraction logic to
> make code more flexible. Keep per interrupt information in the
> ice_q_vector structure, which yields ice_vsi::base_vector redundant.
> Also, as a result there are a few functions that can be removed.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
I've made a few minor observations inline.
I don't think there is a need to respin for any of them.
> ---
> drivers/net/ethernet/intel/ice/ice.h | 11 +-
> drivers/net/ethernet/intel/ice/ice_arfs.c | 5 +-
> drivers/net/ethernet/intel/ice/ice_base.c | 36 ++-
> drivers/net/ethernet/intel/ice/ice_ethtool.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_idc.c | 45 ++--
> drivers/net/ethernet/intel/ice/ice_irq.c | 46 +++-
> drivers/net/ethernet/intel/ice/ice_irq.h | 3 +
> drivers/net/ethernet/intel/ice/ice_lib.c | 225 ++-----------------
Nice code removal from ice_lib.c :)
> drivers/net/ethernet/intel/ice/ice_lib.h | 4 +-
> drivers/net/ethernet/intel/ice/ice_main.c | 44 ++--
> drivers/net/ethernet/intel/ice/ice_ptp.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_sriov.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_xsk.c | 5 +-
> 13 files changed, 154 insertions(+), 276 deletions(-)
...
> diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c
> index 1911d644dfa8..e5db23eaa3f4 100644
> --- a/drivers/net/ethernet/intel/ice/ice_base.c
> +++ b/drivers/net/ethernet/intel/ice/ice_base.c
> @@ -118,9 +118,31 @@ static int ice_vsi_alloc_q_vector(struct ice_vsi *vsi, u16 v_idx)
> q_vector->rx.itr_mode = ITR_DYNAMIC;
> q_vector->tx.type = ICE_TX_CONTAINER;
> q_vector->rx.type = ICE_RX_CONTAINER;
> + q_vector->irq.index = -ENOENT;
>
> - if (vsi->type == ICE_VSI_VF)
> + if (vsi->type == ICE_VSI_VF) {
> + q_vector->reg_idx = ice_calc_vf_reg_idx(vsi->vf, q_vector);
> goto out;
> + } else if (vsi->type == ICE_VSI_CTRL && vsi->vf) {
> + struct ice_vsi *ctrl_vsi = ice_get_vf_ctrl_vsi(pf, vsi);
> +
> + if (ctrl_vsi) {
> + if (unlikely(!ctrl_vsi->q_vectors))
> + return -ENOENT;
> + q_vector->irq = ctrl_vsi->q_vectors[0]->irq;
> + goto skip_alloc;
nit: I think goto for error paths is very much the norm.
But, FWIIW, I would have avoided using goto here.
> + }
> + }
> +
> + q_vector->irq = ice_alloc_irq(pf);
> + if (q_vector->irq.index < 0) {
> + kfree(q_vector);
> + return -ENOMEM;
> + }
> +
> +skip_alloc:
> + q_vector->reg_idx = q_vector->irq.index;
> +
> /* only set affinity_mask if the CPU is online */
> if (cpu_online(v_idx))
> cpumask_set_cpu(v_idx, &q_vector->affinity_mask);
> @@ -168,6 +190,18 @@ static void ice_free_q_vector(struct ice_vsi *vsi, int v_idx)
> if (vsi->netdev)
> netif_napi_del(&q_vector->napi);
>
> + /* release MSIX interrupt if q_vector had interrupt allocated */
> + if (q_vector->irq.index < 0)
> + goto free_q_vector;
> +
> + /* only free last VF ctrl vsi interrupt */
> + if (vsi->type == ICE_VSI_CTRL && vsi->vf &&
> + ice_get_vf_ctrl_vsi(pf, vsi))
> + goto free_q_vector;
Ditto (x2).
> +
> + ice_free_irq(pf, q_vector->irq);
> +
> +free_q_vector:
> devm_kfree(dev, q_vector);
> vsi->q_vectors[v_idx] = NULL;
> }
...
> diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
> index f61be5d76373..ca1a1de26766 100644
> --- a/drivers/net/ethernet/intel/ice/ice_irq.c
> +++ b/drivers/net/ethernet/intel/ice/ice_irq.c
> @@ -194,9 +194,53 @@ int ice_init_interrupt_scheme(struct ice_pf *pf)
> }
>
> /* populate SW interrupts pool with number of OS granted IRQs. */
> - pf->num_avail_sw_msix = (u16)vectors;
> pf->irq_tracker->num_entries = (u16)vectors;
> pf->irq_tracker->end = pf->irq_tracker->num_entries;
>
> return 0;
> }
> +
> +/**
> + * ice_alloc_irq - Allocate new interrupt vector
> + * @pf: board private structure
> + *
> + * Allocate new interrupt vector for a given owner id.
> + * return struct msi_map with interrupt details and track
> + * allocated interrupt appropriately.
> + *
> + * This function mimics individual interrupt allocation,
> + * even interrupts are actually already allocated with
> + * pci_alloc_irq_vectors. Individual allocation helps
> + * to track interrupts and simplifies interrupt related
> + * handling.
> + *
> + * On failure, return map with negative .index. The caller
> + * is expected to check returned map index.
> + *
> + */
> +struct msi_map ice_alloc_irq(struct ice_pf *pf)
> +{
> + struct msi_map map = { .index = -ENOENT };
> + int entry;
> +
> + entry = ice_get_res(pf, pf->irq_tracker);
> + if (entry < 0)
nit: map.index could be initialised here.
> + return map;
> +
> + map.index = entry;
> + map.virq = pci_irq_vector(pf->pdev, map.index);
> +
> + return map;
> +}
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH net-next v3 7/8] ice: track interrupt vectors with xarray
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-03-26 13:19 ` Simon Horman
-1 siblings, 0 replies; 55+ messages in thread
From: Simon Horman @ 2023-03-26 13:19 UTC (permalink / raw)
To: Piotr Raczynski
Cc: intel-wired-lan, netdev, michal.swiatkowski, shiraz.saleem,
jacob.e.keller, sridhar.samudrala, jesse.brandeburg,
aleksander.lobakin, lukasz.czapnik, Michal Swiatkowski
On Thu, Mar 23, 2023 at 01:24:39PM +0100, Piotr Raczynski wrote:
> Replace custom interrupt tracker with generic xarray data structure.
> Remove all code responsible for searching for a new entry with xa_alloc,
> which always tries to allocate at the lowes possible index. As a result
> driver is always using a contiguous region of the MSIX vector table.
>
> New tracker keeps ice_irq_entry entries in xarray as opaque for the rest
> of the driver hiding the entry details from the caller.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
I've added a few comments inline for your consideration
if you need to respin for some other reason.
...
> diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
> index 89d80a2b5feb..b7398abda26a 100644
> --- a/drivers/net/ethernet/intel/ice/ice.h
> +++ b/drivers/net/ethernet/intel/ice/ice.h
> @@ -104,7 +104,6 @@
> #define ICE_Q_WAIT_RETRY_LIMIT 10
> #define ICE_Q_WAIT_MAX_RETRY (5 * ICE_Q_WAIT_RETRY_LIMIT)
> #define ICE_MAX_LG_RSS_QS 256
> -#define ICE_RES_VALID_BIT 0x8000
nit: BIT() could be used here.
> #define ICE_INVAL_Q_INDEX 0xffff
>
> #define ICE_MAX_RXQS_PER_TC 256 /* Used when setting VSI context per TC Rx queues */
...
> diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
> index ca1a1de26766..20d4e9a6aefb 100644
> --- a/drivers/net/ethernet/intel/ice/ice_irq.c
> +++ b/drivers/net/ethernet/intel/ice/ice_irq.c
...
> +/**
> + * ice_get_irq_res - get an interrupt resource
> + * @pf: board private structure
> + *
> + * Allocate new irq entry in the free slot of the tracker. Since xarray
> + * is used, always allocate new entry at the lowest possible index. Set
> + * proper allocation limit for maximum tracker entries.
> + *
> + * Returns allocated irq entry or NULL on failure.
> + */
> +static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf)
> +{
> + struct xa_limit limit = { .max = pf->irq_tracker.num_entries,
> + .min = 0 };
> + struct ice_irq_entry *entry;
> + unsigned int index;
> + int ret;
> +
> + entry = kzalloc(sizeof(*entry), GFP_KERNEL);
> + if (!entry)
> + goto exit;
nit: maybe it is simpler to return NULL here.
> +
> + ret = xa_alloc(&pf->irq_tracker.entries, &index, entry, limit,
> + GFP_KERNEL);
> +
> + if (ret) {
> + kfree(entry);
> + entry = NULL;
and here.
> + } else {
> + entry->index = index;
Which allows for more idiomatic code by moving this out of the else clause.
> + }
> +
> +exit:
And removal of this label.
> + return entry;
> +}
> +
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 7/8] ice: track interrupt vectors with xarray
@ 2023-03-26 13:19 ` Simon Horman
0 siblings, 0 replies; 55+ messages in thread
From: Simon Horman @ 2023-03-26 13:19 UTC (permalink / raw)
To: Piotr Raczynski
Cc: michal.swiatkowski, netdev, jesse.brandeburg, intel-wired-lan,
shiraz.saleem
On Thu, Mar 23, 2023 at 01:24:39PM +0100, Piotr Raczynski wrote:
> Replace custom interrupt tracker with generic xarray data structure.
> Remove all code responsible for searching for a new entry with xa_alloc,
> which always tries to allocate at the lowes possible index. As a result
> driver is always using a contiguous region of the MSIX vector table.
>
> New tracker keeps ice_irq_entry entries in xarray as opaque for the rest
> of the driver hiding the entry details from the caller.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
I've added a few comments inline for your consideration
if you need to respin for some other reason.
...
> diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
> index 89d80a2b5feb..b7398abda26a 100644
> --- a/drivers/net/ethernet/intel/ice/ice.h
> +++ b/drivers/net/ethernet/intel/ice/ice.h
> @@ -104,7 +104,6 @@
> #define ICE_Q_WAIT_RETRY_LIMIT 10
> #define ICE_Q_WAIT_MAX_RETRY (5 * ICE_Q_WAIT_RETRY_LIMIT)
> #define ICE_MAX_LG_RSS_QS 256
> -#define ICE_RES_VALID_BIT 0x8000
nit: BIT() could be used here.
> #define ICE_INVAL_Q_INDEX 0xffff
>
> #define ICE_MAX_RXQS_PER_TC 256 /* Used when setting VSI context per TC Rx queues */
...
> diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
> index ca1a1de26766..20d4e9a6aefb 100644
> --- a/drivers/net/ethernet/intel/ice/ice_irq.c
> +++ b/drivers/net/ethernet/intel/ice/ice_irq.c
...
> +/**
> + * ice_get_irq_res - get an interrupt resource
> + * @pf: board private structure
> + *
> + * Allocate new irq entry in the free slot of the tracker. Since xarray
> + * is used, always allocate new entry at the lowest possible index. Set
> + * proper allocation limit for maximum tracker entries.
> + *
> + * Returns allocated irq entry or NULL on failure.
> + */
> +static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf)
> +{
> + struct xa_limit limit = { .max = pf->irq_tracker.num_entries,
> + .min = 0 };
> + struct ice_irq_entry *entry;
> + unsigned int index;
> + int ret;
> +
> + entry = kzalloc(sizeof(*entry), GFP_KERNEL);
> + if (!entry)
> + goto exit;
nit: maybe it is simpler to return NULL here.
> +
> + ret = xa_alloc(&pf->irq_tracker.entries, &index, entry, limit,
> + GFP_KERNEL);
> +
> + if (ret) {
> + kfree(entry);
> + entry = NULL;
and here.
> + } else {
> + entry->index = index;
Which allows for more idiomatic code by moving this out of the else clause.
> + }
> +
> +exit:
And removal of this label.
> + return entry;
> +}
> +
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 1/8] ice: move interrupt related code to separate file
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-03-26 13:34 ` Simon Horman
-1 siblings, 0 replies; 55+ messages in thread
From: Simon Horman @ 2023-03-26 13:34 UTC (permalink / raw)
To: Piotr Raczynski
Cc: michal.swiatkowski, netdev, jesse.brandeburg, intel-wired-lan,
shiraz.saleem
On Thu, Mar 23, 2023 at 01:24:33PM +0100, Piotr Raczynski wrote:
> Keep interrupt handling code in a dedicated file. This helps keep driver
> structured better and prepares for more functionality added to this file.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH net-next v3 1/8] ice: move interrupt related code to separate file
@ 2023-03-26 13:34 ` Simon Horman
0 siblings, 0 replies; 55+ messages in thread
From: Simon Horman @ 2023-03-26 13:34 UTC (permalink / raw)
To: Piotr Raczynski
Cc: intel-wired-lan, netdev, michal.swiatkowski, shiraz.saleem,
jacob.e.keller, sridhar.samudrala, jesse.brandeburg,
aleksander.lobakin, lukasz.czapnik
On Thu, Mar 23, 2023 at 01:24:33PM +0100, Piotr Raczynski wrote:
> Keep interrupt handling code in a dedicated file. This helps keep driver
> structured better and prepares for more functionality added to this file.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 2/8] ice: use pci_irq_vector helper function
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-03-26 13:35 ` Simon Horman
-1 siblings, 0 replies; 55+ messages in thread
From: Simon Horman @ 2023-03-26 13:35 UTC (permalink / raw)
To: Piotr Raczynski
Cc: michal.swiatkowski, netdev, jesse.brandeburg, intel-wired-lan,
shiraz.saleem
On Thu, Mar 23, 2023 at 01:24:34PM +0100, Piotr Raczynski wrote:
> Currently, driver gets interrupt number directly from ice_pf::msix_entries
> array. Use helper function dedicated to do just that.
>
> While at it use a variable to store interrupt number in
> ice_free_irq_msix_misc instead of calling the helper function twice.
>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH net-next v3 2/8] ice: use pci_irq_vector helper function
@ 2023-03-26 13:35 ` Simon Horman
0 siblings, 0 replies; 55+ messages in thread
From: Simon Horman @ 2023-03-26 13:35 UTC (permalink / raw)
To: Piotr Raczynski
Cc: intel-wired-lan, netdev, michal.swiatkowski, shiraz.saleem,
jacob.e.keller, sridhar.samudrala, jesse.brandeburg,
aleksander.lobakin, lukasz.czapnik, Michal Swiatkowski
On Thu, Mar 23, 2023 at 01:24:34PM +0100, Piotr Raczynski wrote:
> Currently, driver gets interrupt number directly from ice_pf::msix_entries
> array. Use helper function dedicated to do just that.
>
> While at it use a variable to store interrupt number in
> ice_free_irq_msix_misc instead of calling the helper function twice.
>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 3/8] ice: use preferred MSIX allocation api
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-03-26 13:35 ` Simon Horman
-1 siblings, 0 replies; 55+ messages in thread
From: Simon Horman @ 2023-03-26 13:35 UTC (permalink / raw)
To: Piotr Raczynski
Cc: michal.swiatkowski, netdev, jesse.brandeburg, intel-wired-lan,
shiraz.saleem
On Thu, Mar 23, 2023 at 01:24:35PM +0100, Piotr Raczynski wrote:
> Move away from using pci_enable_msix_range/pci_disable_msix and use
> pci_alloc_irq_vectors/pci_free_irq_vectors instead.
>
> As a result stop tracking msix_entries since with newer API entries are
> handled by MSIX core. However, due to current design of communication
> with RDMA driver which accesses ice_pf::msix_entries directly, keep
> using the array just for RDMA driver use.
>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH net-next v3 3/8] ice: use preferred MSIX allocation api
@ 2023-03-26 13:35 ` Simon Horman
0 siblings, 0 replies; 55+ messages in thread
From: Simon Horman @ 2023-03-26 13:35 UTC (permalink / raw)
To: Piotr Raczynski
Cc: intel-wired-lan, netdev, michal.swiatkowski, shiraz.saleem,
jacob.e.keller, sridhar.samudrala, jesse.brandeburg,
aleksander.lobakin, lukasz.czapnik, Michal Swiatkowski
On Thu, Mar 23, 2023 at 01:24:35PM +0100, Piotr Raczynski wrote:
> Move away from using pci_enable_msix_range/pci_disable_msix and use
> pci_alloc_irq_vectors/pci_free_irq_vectors instead.
>
> As a result stop tracking msix_entries since with newer API entries are
> handled by MSIX core. However, due to current design of communication
> with RDMA driver which accesses ice_pf::msix_entries directly, keep
> using the array just for RDMA driver use.
>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH net-next v3 4/8] ice: refactor VF control VSI interrupt handling
2023-03-23 12:24 ` Piotr Raczynski
@ 2023-03-26 13:36 ` Simon Horman
-1 siblings, 0 replies; 55+ messages in thread
From: Simon Horman @ 2023-03-26 13:36 UTC (permalink / raw)
To: Piotr Raczynski
Cc: intel-wired-lan, netdev, michal.swiatkowski, shiraz.saleem,
jacob.e.keller, sridhar.samudrala, jesse.brandeburg,
aleksander.lobakin, lukasz.czapnik
On Thu, Mar 23, 2023 at 01:24:36PM +0100, Piotr Raczynski wrote:
> All VF contrl VSIs share the same interrupt vector. Currently, a helper
> function dedicated for that directly sets ice_vsi::base_vector.
>
> Use helper that returns pointer to first found VF control VSI instead.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 4/8] ice: refactor VF control VSI interrupt handling
@ 2023-03-26 13:36 ` Simon Horman
0 siblings, 0 replies; 55+ messages in thread
From: Simon Horman @ 2023-03-26 13:36 UTC (permalink / raw)
To: Piotr Raczynski
Cc: michal.swiatkowski, netdev, jesse.brandeburg, intel-wired-lan,
shiraz.saleem
On Thu, Mar 23, 2023 at 01:24:36PM +0100, Piotr Raczynski wrote:
> All VF contrl VSIs share the same interrupt vector. Currently, a helper
> function dedicated for that directly sets ice_vsi::base_vector.
>
> Use helper that returns pointer to first found VF control VSI instead.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH net-next v3 5/8] ice: remove redundant SRIOV code
2023-03-23 12:24 ` Piotr Raczynski
@ 2023-03-26 13:36 ` Simon Horman
-1 siblings, 0 replies; 55+ messages in thread
From: Simon Horman @ 2023-03-26 13:36 UTC (permalink / raw)
To: Piotr Raczynski
Cc: intel-wired-lan, netdev, michal.swiatkowski, shiraz.saleem,
jacob.e.keller, sridhar.samudrala, jesse.brandeburg,
aleksander.lobakin, lukasz.czapnik, Michal Swiatkowski
On Thu, Mar 23, 2023 at 01:24:37PM +0100, Piotr Raczynski wrote:
> Remove redundant code from ice_get_max_valid_res_idx that has no effect.
> ice_pf::irq_tracker is initialized during driver probe, there is no reason
> to check it again. Also it is not possible for pf::sriov_base_vector to be
> lower than the tracker length, remove WARN_ON that will never happen.
>
> Get rid of ice_get_max_valid_res_idx helper function completely since it
> can never return negative value.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 5/8] ice: remove redundant SRIOV code
@ 2023-03-26 13:36 ` Simon Horman
0 siblings, 0 replies; 55+ messages in thread
From: Simon Horman @ 2023-03-26 13:36 UTC (permalink / raw)
To: Piotr Raczynski
Cc: michal.swiatkowski, netdev, jesse.brandeburg, intel-wired-lan,
shiraz.saleem
On Thu, Mar 23, 2023 at 01:24:37PM +0100, Piotr Raczynski wrote:
> Remove redundant code from ice_get_max_valid_res_idx that has no effect.
> ice_pf::irq_tracker is initialized during driver probe, there is no reason
> to check it again. Also it is not possible for pf::sriov_base_vector to be
> lower than the tracker length, remove WARN_ON that will never happen.
>
> Get rid of ice_get_max_valid_res_idx helper function completely since it
> can never return negative value.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH net-next v3 8/8] ice: add dynamic interrupt allocation
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-03-26 13:37 ` Simon Horman
-1 siblings, 0 replies; 55+ messages in thread
From: Simon Horman @ 2023-03-26 13:37 UTC (permalink / raw)
To: Piotr Raczynski
Cc: intel-wired-lan, netdev, michal.swiatkowski, shiraz.saleem,
jacob.e.keller, sridhar.samudrala, jesse.brandeburg,
aleksander.lobakin, lukasz.czapnik, Michal Swiatkowski
On Thu, Mar 23, 2023 at 01:24:40PM +0100, Piotr Raczynski wrote:
> Currently driver can only allocate interrupt vectors during init phase by
> calling pci_alloc_irq_vectors. Change that and make use of new
> pci_msix_alloc_irq_at/pci_msix_free_irq API and enable to allocate and free
> more interrupts after MSIX has been enabled. Since not all platforms
> supports dynamic allocation, check it with pci_msix_can_alloc_dyn.
>
> Extend the tracker to keep track how many interrupts are allocated
> initially so when all such vectors are already used, additional interrupts
> are automatically allocated dynamically. Remember each interrupt allocation
> method to then free appropriately. Since some features may require
> interrupts allocated dynamically add appropriate VSI flag and take it into
> account when allocating new interrupt.
>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 8/8] ice: add dynamic interrupt allocation
@ 2023-03-26 13:37 ` Simon Horman
0 siblings, 0 replies; 55+ messages in thread
From: Simon Horman @ 2023-03-26 13:37 UTC (permalink / raw)
To: Piotr Raczynski
Cc: michal.swiatkowski, netdev, jesse.brandeburg, intel-wired-lan,
shiraz.saleem
On Thu, Mar 23, 2023 at 01:24:40PM +0100, Piotr Raczynski wrote:
> Currently driver can only allocate interrupt vectors during init phase by
> calling pci_alloc_irq_vectors. Change that and make use of new
> pci_msix_alloc_irq_at/pci_msix_free_irq API and enable to allocate and free
> more interrupts after MSIX has been enabled. Since not all platforms
> supports dynamic allocation, check it with pci_msix_can_alloc_dyn.
>
> Extend the tracker to keep track how many interrupts are allocated
> initially so when all such vectors are already used, additional interrupts
> are automatically allocated dynamically. Remember each interrupt allocation
> method to then free appropriately. Since some features may require
> interrupts allocated dynamically add appropriate VSI flag and take it into
> account when allocating new interrupt.
>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
Reviewed-by: Simon Horman <simon.horman@corigine.com>
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH net-next v3 7/8] ice: track interrupt vectors with xarray
2023-03-26 13:19 ` [Intel-wired-lan] " Simon Horman
@ 2023-03-28 16:12 ` Piotr Raczynski
-1 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-28 16:12 UTC (permalink / raw)
To: Simon Horman
Cc: intel-wired-lan, netdev, michal.swiatkowski, shiraz.saleem,
jacob.e.keller, sridhar.samudrala, jesse.brandeburg,
aleksander.lobakin, lukasz.czapnik, Michal Swiatkowski
On Sun, Mar 26, 2023 at 03:19:58PM +0200, Simon Horman wrote:
> On Thu, Mar 23, 2023 at 01:24:39PM +0100, Piotr Raczynski wrote:
> > Replace custom interrupt tracker with generic xarray data structure.
> > Remove all code responsible for searching for a new entry with xa_alloc,
> > which always tries to allocate at the lowes possible index. As a result
> > driver is always using a contiguous region of the MSIX vector table.
> >
> > New tracker keeps ice_irq_entry entries in xarray as opaque for the rest
> > of the driver hiding the entry details from the caller.
> >
> > Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> > Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> > Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
>
> Reviewed-by: Simon Horman <simon.horman@corigine.com>
>
> I've added a few comments inline for your consideration
> if you need to respin for some other reason.
>
Thanks for reviewing.
> ...
>
> > diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
> > index 89d80a2b5feb..b7398abda26a 100644
> > --- a/drivers/net/ethernet/intel/ice/ice.h
> > +++ b/drivers/net/ethernet/intel/ice/ice.h
> > @@ -104,7 +104,6 @@
> > #define ICE_Q_WAIT_RETRY_LIMIT 10
> > #define ICE_Q_WAIT_MAX_RETRY (5 * ICE_Q_WAIT_RETRY_LIMIT)
> > #define ICE_MAX_LG_RSS_QS 256
> > -#define ICE_RES_VALID_BIT 0x8000
>
> nit: BIT() could be used here.
>
This piece is gone anyway.
> > #define ICE_INVAL_Q_INDEX 0xffff
> >
> > #define ICE_MAX_RXQS_PER_TC 256 /* Used when setting VSI context per TC Rx queues */
>
> ...
>
> > diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
> > index ca1a1de26766..20d4e9a6aefb 100644
> > --- a/drivers/net/ethernet/intel/ice/ice_irq.c
> > +++ b/drivers/net/ethernet/intel/ice/ice_irq.c
>
> ...
>
> > +/**
> > + * ice_get_irq_res - get an interrupt resource
> > + * @pf: board private structure
> > + *
> > + * Allocate new irq entry in the free slot of the tracker. Since xarray
> > + * is used, always allocate new entry at the lowest possible index. Set
> > + * proper allocation limit for maximum tracker entries.
> > + *
> > + * Returns allocated irq entry or NULL on failure.
> > + */
> > +static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf)
> > +{
> > + struct xa_limit limit = { .max = pf->irq_tracker.num_entries,
> > + .min = 0 };
> > + struct ice_irq_entry *entry;
> > + unsigned int index;
> > + int ret;
> > +
> > + entry = kzalloc(sizeof(*entry), GFP_KERNEL);
> > + if (!entry)
> > + goto exit;
>
> nit: maybe it is simpler to return NULL here.
>
> > +
> > + ret = xa_alloc(&pf->irq_tracker.entries, &index, entry, limit,
> > + GFP_KERNEL);
> > +
> > + if (ret) {
> > + kfree(entry);
> > + entry = NULL;
>
> and here.
>
> > + } else {
> > + entry->index = index;
>
> Which allows for more idiomatic code by moving this out of the else clause.
>
> > + }
> > +
> > +exit:
>
> And removal of this label.
Good idea, thanks.
>
> > + return entry;
> > +}
> > +
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 7/8] ice: track interrupt vectors with xarray
@ 2023-03-28 16:12 ` Piotr Raczynski
0 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-28 16:12 UTC (permalink / raw)
To: Simon Horman
Cc: michal.swiatkowski, netdev, jesse.brandeburg, intel-wired-lan,
shiraz.saleem
On Sun, Mar 26, 2023 at 03:19:58PM +0200, Simon Horman wrote:
> On Thu, Mar 23, 2023 at 01:24:39PM +0100, Piotr Raczynski wrote:
> > Replace custom interrupt tracker with generic xarray data structure.
> > Remove all code responsible for searching for a new entry with xa_alloc,
> > which always tries to allocate at the lowes possible index. As a result
> > driver is always using a contiguous region of the MSIX vector table.
> >
> > New tracker keeps ice_irq_entry entries in xarray as opaque for the rest
> > of the driver hiding the entry details from the caller.
> >
> > Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> > Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> > Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
>
> Reviewed-by: Simon Horman <simon.horman@corigine.com>
>
> I've added a few comments inline for your consideration
> if you need to respin for some other reason.
>
Thanks for reviewing.
> ...
>
> > diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
> > index 89d80a2b5feb..b7398abda26a 100644
> > --- a/drivers/net/ethernet/intel/ice/ice.h
> > +++ b/drivers/net/ethernet/intel/ice/ice.h
> > @@ -104,7 +104,6 @@
> > #define ICE_Q_WAIT_RETRY_LIMIT 10
> > #define ICE_Q_WAIT_MAX_RETRY (5 * ICE_Q_WAIT_RETRY_LIMIT)
> > #define ICE_MAX_LG_RSS_QS 256
> > -#define ICE_RES_VALID_BIT 0x8000
>
> nit: BIT() could be used here.
>
This piece is gone anyway.
> > #define ICE_INVAL_Q_INDEX 0xffff
> >
> > #define ICE_MAX_RXQS_PER_TC 256 /* Used when setting VSI context per TC Rx queues */
>
> ...
>
> > diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
> > index ca1a1de26766..20d4e9a6aefb 100644
> > --- a/drivers/net/ethernet/intel/ice/ice_irq.c
> > +++ b/drivers/net/ethernet/intel/ice/ice_irq.c
>
> ...
>
> > +/**
> > + * ice_get_irq_res - get an interrupt resource
> > + * @pf: board private structure
> > + *
> > + * Allocate new irq entry in the free slot of the tracker. Since xarray
> > + * is used, always allocate new entry at the lowest possible index. Set
> > + * proper allocation limit for maximum tracker entries.
> > + *
> > + * Returns allocated irq entry or NULL on failure.
> > + */
> > +static struct ice_irq_entry *ice_get_irq_res(struct ice_pf *pf)
> > +{
> > + struct xa_limit limit = { .max = pf->irq_tracker.num_entries,
> > + .min = 0 };
> > + struct ice_irq_entry *entry;
> > + unsigned int index;
> > + int ret;
> > +
> > + entry = kzalloc(sizeof(*entry), GFP_KERNEL);
> > + if (!entry)
> > + goto exit;
>
> nit: maybe it is simpler to return NULL here.
>
> > +
> > + ret = xa_alloc(&pf->irq_tracker.entries, &index, entry, limit,
> > + GFP_KERNEL);
> > +
> > + if (ret) {
> > + kfree(entry);
> > + entry = NULL;
>
> and here.
>
> > + } else {
> > + entry->index = index;
>
> Which allows for more idiomatic code by moving this out of the else clause.
>
> > + }
> > +
> > +exit:
>
> And removal of this label.
Good idea, thanks.
>
> > + return entry;
> > +}
> > +
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH net-next v3 6/8] ice: add individual interrupt allocation
2023-03-26 13:18 ` [Intel-wired-lan] " Simon Horman
@ 2023-03-28 16:16 ` Piotr Raczynski
-1 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-28 16:16 UTC (permalink / raw)
To: Simon Horman
Cc: intel-wired-lan, netdev, michal.swiatkowski, shiraz.saleem,
jacob.e.keller, sridhar.samudrala, jesse.brandeburg,
aleksander.lobakin, lukasz.czapnik, Michal Swiatkowski
On Sun, Mar 26, 2023 at 03:18:03PM +0200, Simon Horman wrote:
> On Thu, Mar 23, 2023 at 01:24:38PM +0100, Piotr Raczynski wrote:
> > Currently interrupt allocations, depending on a feature are distributed
> > in batches. Also, after allocation there is a series of operations that
> > distributes per irq settings through that batch of interrupts.
> >
> > Although driver does not yet support dynamic interrupt allocation, keep
> > allocated interrupts in a pool and add allocation abstraction logic to
> > make code more flexible. Keep per interrupt information in the
> > ice_q_vector structure, which yields ice_vsi::base_vector redundant.
> > Also, as a result there are a few functions that can be removed.
> >
> > Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> > Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> > Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
>
> Reviewed-by: Simon Horman <simon.horman@corigine.com>
>
> I've made a few minor observations inline.
> I don't think there is a need to respin for any of them.
>
Thanks.
> > ---
> > drivers/net/ethernet/intel/ice/ice.h | 11 +-
> > drivers/net/ethernet/intel/ice/ice_arfs.c | 5 +-
> > drivers/net/ethernet/intel/ice/ice_base.c | 36 ++-
> > drivers/net/ethernet/intel/ice/ice_ethtool.c | 2 +-
> > drivers/net/ethernet/intel/ice/ice_idc.c | 45 ++--
> > drivers/net/ethernet/intel/ice/ice_irq.c | 46 +++-
> > drivers/net/ethernet/intel/ice/ice_irq.h | 3 +
> > drivers/net/ethernet/intel/ice/ice_lib.c | 225 ++-----------------
>
> Nice code removal from ice_lib.c :)
>
> > drivers/net/ethernet/intel/ice/ice_lib.h | 4 +-
> > drivers/net/ethernet/intel/ice/ice_main.c | 44 ++--
> > drivers/net/ethernet/intel/ice/ice_ptp.c | 2 +-
> > drivers/net/ethernet/intel/ice/ice_sriov.c | 2 +-
> > drivers/net/ethernet/intel/ice/ice_xsk.c | 5 +-
> > 13 files changed, 154 insertions(+), 276 deletions(-)
>
> ...
>
> > diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c
> > index 1911d644dfa8..e5db23eaa3f4 100644
> > --- a/drivers/net/ethernet/intel/ice/ice_base.c
> > +++ b/drivers/net/ethernet/intel/ice/ice_base.c
> > @@ -118,9 +118,31 @@ static int ice_vsi_alloc_q_vector(struct ice_vsi *vsi, u16 v_idx)
> > q_vector->rx.itr_mode = ITR_DYNAMIC;
> > q_vector->tx.type = ICE_TX_CONTAINER;
> > q_vector->rx.type = ICE_RX_CONTAINER;
> > + q_vector->irq.index = -ENOENT;
> >
> > - if (vsi->type == ICE_VSI_VF)
> > + if (vsi->type == ICE_VSI_VF) {
> > + q_vector->reg_idx = ice_calc_vf_reg_idx(vsi->vf, q_vector);
> > goto out;
> > + } else if (vsi->type == ICE_VSI_CTRL && vsi->vf) {
> > + struct ice_vsi *ctrl_vsi = ice_get_vf_ctrl_vsi(pf, vsi);
> > +
> > + if (ctrl_vsi) {
> > + if (unlikely(!ctrl_vsi->q_vectors))
> > + return -ENOENT;
> > + q_vector->irq = ctrl_vsi->q_vectors[0]->irq;
> > + goto skip_alloc;
>
> nit: I think goto for error paths is very much the norm.
> But, FWIIW, I would have avoided using goto here.
Thanks will take a look.
>
> > + }
> > + }
> > +
> > + q_vector->irq = ice_alloc_irq(pf);
> > + if (q_vector->irq.index < 0) {
> > + kfree(q_vector);
> > + return -ENOMEM;
> > + }
> > +
> > +skip_alloc:
> > + q_vector->reg_idx = q_vector->irq.index;
> > +
> > /* only set affinity_mask if the CPU is online */
> > if (cpu_online(v_idx))
> > cpumask_set_cpu(v_idx, &q_vector->affinity_mask);
> > @@ -168,6 +190,18 @@ static void ice_free_q_vector(struct ice_vsi *vsi, int v_idx)
> > if (vsi->netdev)
> > netif_napi_del(&q_vector->napi);
> >
> > + /* release MSIX interrupt if q_vector had interrupt allocated */
> > + if (q_vector->irq.index < 0)
> > + goto free_q_vector;
> > +
> > + /* only free last VF ctrl vsi interrupt */
> > + if (vsi->type == ICE_VSI_CTRL && vsi->vf &&
> > + ice_get_vf_ctrl_vsi(pf, vsi))
> > + goto free_q_vector;
>
> Ditto (x2).
Will also take a look.
>
> > +
> > + ice_free_irq(pf, q_vector->irq);
> > +
> > +free_q_vector:
> > devm_kfree(dev, q_vector);
> > vsi->q_vectors[v_idx] = NULL;
> > }
>
> ...
>
> > diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
> > index f61be5d76373..ca1a1de26766 100644
> > --- a/drivers/net/ethernet/intel/ice/ice_irq.c
> > +++ b/drivers/net/ethernet/intel/ice/ice_irq.c
> > @@ -194,9 +194,53 @@ int ice_init_interrupt_scheme(struct ice_pf *pf)
> > }
> >
> > /* populate SW interrupts pool with number of OS granted IRQs. */
> > - pf->num_avail_sw_msix = (u16)vectors;
> > pf->irq_tracker->num_entries = (u16)vectors;
> > pf->irq_tracker->end = pf->irq_tracker->num_entries;
> >
> > return 0;
> > }
> > +
> > +/**
> > + * ice_alloc_irq - Allocate new interrupt vector
> > + * @pf: board private structure
> > + *
> > + * Allocate new interrupt vector for a given owner id.
> > + * return struct msi_map with interrupt details and track
> > + * allocated interrupt appropriately.
> > + *
> > + * This function mimics individual interrupt allocation,
> > + * even interrupts are actually already allocated with
> > + * pci_alloc_irq_vectors. Individual allocation helps
> > + * to track interrupts and simplifies interrupt related
> > + * handling.
> > + *
> > + * On failure, return map with negative .index. The caller
> > + * is expected to check returned map index.
> > + *
> > + */
> > +struct msi_map ice_alloc_irq(struct ice_pf *pf)
> > +{
> > + struct msi_map map = { .index = -ENOENT };
> > + int entry;
> > +
> > + entry = ice_get_res(pf, pf->irq_tracker);
> > + if (entry < 0)
>
> nit: map.index could be initialised here.
>
> > + return map;
> > +
> > + map.index = entry;
> > + map.virq = pci_irq_vector(pf->pdev, map.index);
> > +
> > + return map;
> > +}
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 6/8] ice: add individual interrupt allocation
@ 2023-03-28 16:16 ` Piotr Raczynski
0 siblings, 0 replies; 55+ messages in thread
From: Piotr Raczynski @ 2023-03-28 16:16 UTC (permalink / raw)
To: Simon Horman
Cc: michal.swiatkowski, netdev, jesse.brandeburg, intel-wired-lan,
shiraz.saleem
On Sun, Mar 26, 2023 at 03:18:03PM +0200, Simon Horman wrote:
> On Thu, Mar 23, 2023 at 01:24:38PM +0100, Piotr Raczynski wrote:
> > Currently interrupt allocations, depending on a feature are distributed
> > in batches. Also, after allocation there is a series of operations that
> > distributes per irq settings through that batch of interrupts.
> >
> > Although driver does not yet support dynamic interrupt allocation, keep
> > allocated interrupts in a pool and add allocation abstraction logic to
> > make code more flexible. Keep per interrupt information in the
> > ice_q_vector structure, which yields ice_vsi::base_vector redundant.
> > Also, as a result there are a few functions that can be removed.
> >
> > Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> > Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> > Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
>
> Reviewed-by: Simon Horman <simon.horman@corigine.com>
>
> I've made a few minor observations inline.
> I don't think there is a need to respin for any of them.
>
Thanks.
> > ---
> > drivers/net/ethernet/intel/ice/ice.h | 11 +-
> > drivers/net/ethernet/intel/ice/ice_arfs.c | 5 +-
> > drivers/net/ethernet/intel/ice/ice_base.c | 36 ++-
> > drivers/net/ethernet/intel/ice/ice_ethtool.c | 2 +-
> > drivers/net/ethernet/intel/ice/ice_idc.c | 45 ++--
> > drivers/net/ethernet/intel/ice/ice_irq.c | 46 +++-
> > drivers/net/ethernet/intel/ice/ice_irq.h | 3 +
> > drivers/net/ethernet/intel/ice/ice_lib.c | 225 ++-----------------
>
> Nice code removal from ice_lib.c :)
>
> > drivers/net/ethernet/intel/ice/ice_lib.h | 4 +-
> > drivers/net/ethernet/intel/ice/ice_main.c | 44 ++--
> > drivers/net/ethernet/intel/ice/ice_ptp.c | 2 +-
> > drivers/net/ethernet/intel/ice/ice_sriov.c | 2 +-
> > drivers/net/ethernet/intel/ice/ice_xsk.c | 5 +-
> > 13 files changed, 154 insertions(+), 276 deletions(-)
>
> ...
>
> > diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c
> > index 1911d644dfa8..e5db23eaa3f4 100644
> > --- a/drivers/net/ethernet/intel/ice/ice_base.c
> > +++ b/drivers/net/ethernet/intel/ice/ice_base.c
> > @@ -118,9 +118,31 @@ static int ice_vsi_alloc_q_vector(struct ice_vsi *vsi, u16 v_idx)
> > q_vector->rx.itr_mode = ITR_DYNAMIC;
> > q_vector->tx.type = ICE_TX_CONTAINER;
> > q_vector->rx.type = ICE_RX_CONTAINER;
> > + q_vector->irq.index = -ENOENT;
> >
> > - if (vsi->type == ICE_VSI_VF)
> > + if (vsi->type == ICE_VSI_VF) {
> > + q_vector->reg_idx = ice_calc_vf_reg_idx(vsi->vf, q_vector);
> > goto out;
> > + } else if (vsi->type == ICE_VSI_CTRL && vsi->vf) {
> > + struct ice_vsi *ctrl_vsi = ice_get_vf_ctrl_vsi(pf, vsi);
> > +
> > + if (ctrl_vsi) {
> > + if (unlikely(!ctrl_vsi->q_vectors))
> > + return -ENOENT;
> > + q_vector->irq = ctrl_vsi->q_vectors[0]->irq;
> > + goto skip_alloc;
>
> nit: I think goto for error paths is very much the norm.
> But, FWIIW, I would have avoided using goto here.
Thanks will take a look.
>
> > + }
> > + }
> > +
> > + q_vector->irq = ice_alloc_irq(pf);
> > + if (q_vector->irq.index < 0) {
> > + kfree(q_vector);
> > + return -ENOMEM;
> > + }
> > +
> > +skip_alloc:
> > + q_vector->reg_idx = q_vector->irq.index;
> > +
> > /* only set affinity_mask if the CPU is online */
> > if (cpu_online(v_idx))
> > cpumask_set_cpu(v_idx, &q_vector->affinity_mask);
> > @@ -168,6 +190,18 @@ static void ice_free_q_vector(struct ice_vsi *vsi, int v_idx)
> > if (vsi->netdev)
> > netif_napi_del(&q_vector->napi);
> >
> > + /* release MSIX interrupt if q_vector had interrupt allocated */
> > + if (q_vector->irq.index < 0)
> > + goto free_q_vector;
> > +
> > + /* only free last VF ctrl vsi interrupt */
> > + if (vsi->type == ICE_VSI_CTRL && vsi->vf &&
> > + ice_get_vf_ctrl_vsi(pf, vsi))
> > + goto free_q_vector;
>
> Ditto (x2).
Will also take a look.
>
> > +
> > + ice_free_irq(pf, q_vector->irq);
> > +
> > +free_q_vector:
> > devm_kfree(dev, q_vector);
> > vsi->q_vectors[v_idx] = NULL;
> > }
>
> ...
>
> > diff --git a/drivers/net/ethernet/intel/ice/ice_irq.c b/drivers/net/ethernet/intel/ice/ice_irq.c
> > index f61be5d76373..ca1a1de26766 100644
> > --- a/drivers/net/ethernet/intel/ice/ice_irq.c
> > +++ b/drivers/net/ethernet/intel/ice/ice_irq.c
> > @@ -194,9 +194,53 @@ int ice_init_interrupt_scheme(struct ice_pf *pf)
> > }
> >
> > /* populate SW interrupts pool with number of OS granted IRQs. */
> > - pf->num_avail_sw_msix = (u16)vectors;
> > pf->irq_tracker->num_entries = (u16)vectors;
> > pf->irq_tracker->end = pf->irq_tracker->num_entries;
> >
> > return 0;
> > }
> > +
> > +/**
> > + * ice_alloc_irq - Allocate new interrupt vector
> > + * @pf: board private structure
> > + *
> > + * Allocate new interrupt vector for a given owner id.
> > + * return struct msi_map with interrupt details and track
> > + * allocated interrupt appropriately.
> > + *
> > + * This function mimics individual interrupt allocation,
> > + * even interrupts are actually already allocated with
> > + * pci_alloc_irq_vectors. Individual allocation helps
> > + * to track interrupts and simplifies interrupt related
> > + * handling.
> > + *
> > + * On failure, return map with negative .index. The caller
> > + * is expected to check returned map index.
> > + *
> > + */
> > +struct msi_map ice_alloc_irq(struct ice_pf *pf)
> > +{
> > + struct msi_map map = { .index = -ENOENT };
> > + int entry;
> > +
> > + entry = ice_get_res(pf, pf->irq_tracker);
> > + if (entry < 0)
>
> nit: map.index could be initialised here.
>
> > + return map;
> > +
> > + map.index = entry;
> > + map.virq = pci_irq_vector(pf->pdev, map.index);
> > +
> > + return map;
> > +}
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH net-next v3 0/8] ice: support dynamic interrupt allocation
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-03-29 11:35 ` Leon Romanovsky
-1 siblings, 0 replies; 55+ messages in thread
From: Leon Romanovsky @ 2023-03-29 11:35 UTC (permalink / raw)
To: Piotr Raczynski
Cc: intel-wired-lan, netdev, michal.swiatkowski, shiraz.saleem,
jacob.e.keller, sridhar.samudrala, jesse.brandeburg,
aleksander.lobakin, lukasz.czapnik
On Thu, Mar 23, 2023 at 01:24:32PM +0100, Piotr Raczynski wrote:
> This patchset reimplements MSIX interrupt allocation logic to allow dynamic
> interrupt allocation after MSIX has been initially enabled. This allows
> current and future features to allocate and free interrupts as needed and
> will help to drastically decrease number of initially preallocated
> interrupts (even down to the API hard limit of 1). Although this patchset
> does not change behavior in terms of actual number of allocated interrupts
> during probe, it will be subject to change.
>
> First few patches prepares to introduce dynamic allocation by moving
> interrupt allocation code to separate file and update allocation API used
> in the driver to the currently preferred one.
>
> Due to the current contract between ice and irdma driver which is directly
> accessing msix entries allocated by ice driver, even after moving away from
> older pci_enable_msix_range function, still keep msix_entries array for
> irdma use.
>
> Next patches refactors and removes redundant code from SRIOV related logic
> as it also make it easier to move away from static allocation scheme.
>
> Last patches actually enables dynamic allocation of MSIX interrupts. First,
> introduce functions to allocate and free interrupts individually. This sets
> ground for the rest of the changes even if that patch still allocates the
> interrupts from the preallocated pool. Since this patch starts to keep
> interrupt details in ice_q_vector structure we can get rid of functions
> that calculates base vector number and register offset for the interrupt
> as it is equal to the interrupt index. Only keep separate register offset
> functions for the VF VSIs.
>
> Next, replace homegrown interrupt tracker with much simpler xarray based
> approach. As new API always allocate interrupts one by one, also track
> interrupts in the same manner.
>
> Lastly, extend the interrupt tracker to deal both with preallocated and
> dynamically allocated vectors and use pci_msix_alloc_irq_at and
> pci_msix_free_irq functions. Since not all architecture supports dynamic
> allocation, check it before trying to allocate a new interrupt.
>
> As previously mentioned, this patchset does not change number of initially
> allocated interrupts during init phase but now it can and will likely be
> changed.
>
> Patch 1-3 -> move code around and use newer API
> Patch 4-5 -> refactor and remove redundant SRIOV code
> Patch 6 -> allocate every interrupt individually
> Patch 7 -> replace homegrown interrupt tracker with xarray
> Patch 8 -> allow dynamic interrupt allocation
>
> Change history:
> v1 -> v2:
> - ice: refactor VF control VSI interrupt handling
> - move ice_get_vf_ctrl_vsi to ice_lib.c (ice_vf_lib.c depends on
> CONFIG_PCI_IOV)
> v2 -> v3:
> - ice: refactor VF control VSI interrupt handling
> - revert v2 change and add no-op function in case of CONFIG_PCI_IOV=n
> - ice: add dynamic interrupt allocation
> - fix commit message
>
> Piotr Raczynski (8):
> ice: move interrupt related code to separate file
> ice: use pci_irq_vector helper function
> ice: use preferred MSIX allocation api
> ice: refactor VF control VSI interrupt handling
> ice: remove redundant SRIOV code
> ice: add individual interrupt allocation
> ice: track interrupt vectors with xarray
> ice: add dynamic interrupt allocation
>
Thanks,
Reviewed-by: Leon Romanovsky <leonro@nvidia.com>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 0/8] ice: support dynamic interrupt allocation
@ 2023-03-29 11:35 ` Leon Romanovsky
0 siblings, 0 replies; 55+ messages in thread
From: Leon Romanovsky @ 2023-03-29 11:35 UTC (permalink / raw)
To: Piotr Raczynski
Cc: michal.swiatkowski, netdev, jesse.brandeburg, intel-wired-lan,
shiraz.saleem
On Thu, Mar 23, 2023 at 01:24:32PM +0100, Piotr Raczynski wrote:
> This patchset reimplements MSIX interrupt allocation logic to allow dynamic
> interrupt allocation after MSIX has been initially enabled. This allows
> current and future features to allocate and free interrupts as needed and
> will help to drastically decrease number of initially preallocated
> interrupts (even down to the API hard limit of 1). Although this patchset
> does not change behavior in terms of actual number of allocated interrupts
> during probe, it will be subject to change.
>
> First few patches prepares to introduce dynamic allocation by moving
> interrupt allocation code to separate file and update allocation API used
> in the driver to the currently preferred one.
>
> Due to the current contract between ice and irdma driver which is directly
> accessing msix entries allocated by ice driver, even after moving away from
> older pci_enable_msix_range function, still keep msix_entries array for
> irdma use.
>
> Next patches refactors and removes redundant code from SRIOV related logic
> as it also make it easier to move away from static allocation scheme.
>
> Last patches actually enables dynamic allocation of MSIX interrupts. First,
> introduce functions to allocate and free interrupts individually. This sets
> ground for the rest of the changes even if that patch still allocates the
> interrupts from the preallocated pool. Since this patch starts to keep
> interrupt details in ice_q_vector structure we can get rid of functions
> that calculates base vector number and register offset for the interrupt
> as it is equal to the interrupt index. Only keep separate register offset
> functions for the VF VSIs.
>
> Next, replace homegrown interrupt tracker with much simpler xarray based
> approach. As new API always allocate interrupts one by one, also track
> interrupts in the same manner.
>
> Lastly, extend the interrupt tracker to deal both with preallocated and
> dynamically allocated vectors and use pci_msix_alloc_irq_at and
> pci_msix_free_irq functions. Since not all architecture supports dynamic
> allocation, check it before trying to allocate a new interrupt.
>
> As previously mentioned, this patchset does not change number of initially
> allocated interrupts during init phase but now it can and will likely be
> changed.
>
> Patch 1-3 -> move code around and use newer API
> Patch 4-5 -> refactor and remove redundant SRIOV code
> Patch 6 -> allocate every interrupt individually
> Patch 7 -> replace homegrown interrupt tracker with xarray
> Patch 8 -> allow dynamic interrupt allocation
>
> Change history:
> v1 -> v2:
> - ice: refactor VF control VSI interrupt handling
> - move ice_get_vf_ctrl_vsi to ice_lib.c (ice_vf_lib.c depends on
> CONFIG_PCI_IOV)
> v2 -> v3:
> - ice: refactor VF control VSI interrupt handling
> - revert v2 change and add no-op function in case of CONFIG_PCI_IOV=n
> - ice: add dynamic interrupt allocation
> - fix commit message
>
> Piotr Raczynski (8):
> ice: move interrupt related code to separate file
> ice: use pci_irq_vector helper function
> ice: use preferred MSIX allocation api
> ice: refactor VF control VSI interrupt handling
> ice: remove redundant SRIOV code
> ice: add individual interrupt allocation
> ice: track interrupt vectors with xarray
> ice: add dynamic interrupt allocation
>
Thanks,
Reviewed-by: Leon Romanovsky <leonro@nvidia.com>
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 4/8] ice: refactor VF control VSI interrupt handling
2023-03-26 13:36 ` [Intel-wired-lan] " Simon Horman
(?)
@ 2023-03-30 16:53 ` Romanowski, Rafal
-1 siblings, 0 replies; 55+ messages in thread
From: Romanowski, Rafal @ 2023-03-30 16:53 UTC (permalink / raw)
To: intel-wired-lan
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of
> Simon Horman
> Sent: niedziela, 26 marca 2023 15:36
> To: Raczynski, Piotr <piotr.raczynski@intel.com>
> Cc: Swiatkowski, Michal <michal.swiatkowski@intel.com>;
> netdev@vger.kernel.org; Brandeburg, Jesse
> <jesse.brandeburg@intel.com>; intel-wired-lan@lists.osuosl.org; Saleem,
> Shiraz <shiraz.saleem@intel.com>
> Subject: Re: [Intel-wired-lan] [PATCH net-next v3 4/8] ice: refactor VF control
> VSI interrupt handling
>
> On Thu, Mar 23, 2023 at 01:24:36PM +0100, Piotr Raczynski wrote:
> > All VF contrl VSIs share the same interrupt vector. Currently, a
> > helper function dedicated for that directly sets ice_vsi::base_vector.
> >
> > Use helper that returns pointer to first found VF control VSI instead.
> >
> > Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> > Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
>
> Reviewed-by: Simon Horman <simon.horman@corigine.com>
Tested-by: Rafal Romanowski <rafal.romanowski@intel.com>
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 5/8] ice: remove redundant SRIOV code
2023-03-26 13:36 ` [Intel-wired-lan] " Simon Horman
@ 2023-04-06 9:50 ` Romanowski, Rafal
-1 siblings, 0 replies; 55+ messages in thread
From: Romanowski, Rafal @ 2023-04-06 9:50 UTC (permalink / raw)
To: Simon Horman, Raczynski, Piotr
Cc: netdev, Saleem, Shiraz, intel-wired-lan, Brandeburg, Jesse,
Swiatkowski, Michal
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of
> Simon Horman
> Sent: niedziela, 26 marca 2023 15:36
> To: Raczynski, Piotr <piotr.raczynski@intel.com>
> Cc: Swiatkowski, Michal <michal.swiatkowski@intel.com>;
> netdev@vger.kernel.org; Brandeburg, Jesse
> <jesse.brandeburg@intel.com>; intel-wired-lan@lists.osuosl.org; Saleem,
> Shiraz <shiraz.saleem@intel.com>
> Subject: Re: [Intel-wired-lan] [PATCH net-next v3 5/8] ice: remove
> redundant SRIOV code
>
> On Thu, Mar 23, 2023 at 01:24:37PM +0100, Piotr Raczynski wrote:
> > Remove redundant code from ice_get_max_valid_res_idx that has no
> effect.
> > ice_pf::irq_tracker is initialized during driver probe, there is no
> > reason to check it again. Also it is not possible for
> > pf::sriov_base_vector to be lower than the tracker length, remove
> WARN_ON that will never happen.
> >
> > Get rid of ice_get_max_valid_res_idx helper function completely since
> > it can never return negative value.
> >
> > Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> > Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> > Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
>
> Reviewed-by: Simon Horman <simon.horman@corigine.com>
Tested-by: Rafal Romanowski <rafal.romanowski@intel.com>
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* RE: [Intel-wired-lan] [PATCH net-next v3 5/8] ice: remove redundant SRIOV code
@ 2023-04-06 9:50 ` Romanowski, Rafal
0 siblings, 0 replies; 55+ messages in thread
From: Romanowski, Rafal @ 2023-04-06 9:50 UTC (permalink / raw)
To: Simon Horman, Raczynski, Piotr
Cc: Swiatkowski, Michal, netdev, Brandeburg, Jesse, intel-wired-lan,
Saleem, Shiraz
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of
> Simon Horman
> Sent: niedziela, 26 marca 2023 15:36
> To: Raczynski, Piotr <piotr.raczynski@intel.com>
> Cc: Swiatkowski, Michal <michal.swiatkowski@intel.com>;
> netdev@vger.kernel.org; Brandeburg, Jesse
> <jesse.brandeburg@intel.com>; intel-wired-lan@lists.osuosl.org; Saleem,
> Shiraz <shiraz.saleem@intel.com>
> Subject: Re: [Intel-wired-lan] [PATCH net-next v3 5/8] ice: remove
> redundant SRIOV code
>
> On Thu, Mar 23, 2023 at 01:24:37PM +0100, Piotr Raczynski wrote:
> > Remove redundant code from ice_get_max_valid_res_idx that has no
> effect.
> > ice_pf::irq_tracker is initialized during driver probe, there is no
> > reason to check it again. Also it is not possible for
> > pf::sriov_base_vector to be lower than the tracker length, remove
> WARN_ON that will never happen.
> >
> > Get rid of ice_get_max_valid_res_idx helper function completely since
> > it can never return negative value.
> >
> > Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> > Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> > Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
>
> Reviewed-by: Simon Horman <simon.horman@corigine.com>
Tested-by: Rafal Romanowski <rafal.romanowski@intel.com>
^ permalink raw reply [flat|nested] 55+ messages in thread
* RE: [Intel-wired-lan] [PATCH net-next v3 1/8] ice: move interrupt related code to separate file
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-04-21 5:36 ` Pucha, HimasekharX Reddy
-1 siblings, 0 replies; 55+ messages in thread
From: Pucha, HimasekharX Reddy @ 2023-04-21 5:36 UTC (permalink / raw)
To: Raczynski, Piotr, intel-wired-lan
Cc: Swiatkowski, Michal, netdev, Brandeburg, Jesse, Saleem, Shiraz
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Raczynski, Piotr
> Sent: Thursday, March 23, 2023 5:55 PM
> To: intel-wired-lan@lists.osuosl.org
> Cc: Swiatkowski, Michal <michal.swiatkowski@intel.com>; netdev@vger.kernel.org; Brandeburg, Jesse <jesse.brandeburg@intel.com>; Saleem, Shiraz <shiraz.saleem@intel.com>
> Subject: [Intel-wired-lan] [PATCH net-next v3 1/8] ice: move interrupt related code to separate file
>
> Keep interrupt handling code in a dedicated file. This helps keep driver structured better and prepares for more functionality added to this file.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
> ---
> drivers/net/ethernet/intel/ice/Makefile | 1 +
> drivers/net/ethernet/intel/ice/ice.h | 1 +
> drivers/net/ethernet/intel/ice/ice_irq.c | 226 ++++++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_irq.h | 10 + drivers/net/ethernet/intel/ice/ice_main.c | 218 ---------------------
> 5 files changed, 238 insertions(+), 218 deletions(-) create mode 100644 drivers/net/ethernet/intel/ice/ice_irq.c
> create mode 100644 drivers/net/ethernet/intel/ice/ice_irq.h
>
Tested-by: Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 1/8] ice: move interrupt related code to separate file
@ 2023-04-21 5:36 ` Pucha, HimasekharX Reddy
0 siblings, 0 replies; 55+ messages in thread
From: Pucha, HimasekharX Reddy @ 2023-04-21 5:36 UTC (permalink / raw)
To: Raczynski, Piotr, intel-wired-lan
Cc: netdev, Saleem, Shiraz, Brandeburg, Jesse, Swiatkowski, Michal
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Raczynski, Piotr
> Sent: Thursday, March 23, 2023 5:55 PM
> To: intel-wired-lan@lists.osuosl.org
> Cc: Swiatkowski, Michal <michal.swiatkowski@intel.com>; netdev@vger.kernel.org; Brandeburg, Jesse <jesse.brandeburg@intel.com>; Saleem, Shiraz <shiraz.saleem@intel.com>
> Subject: [Intel-wired-lan] [PATCH net-next v3 1/8] ice: move interrupt related code to separate file
>
> Keep interrupt handling code in a dedicated file. This helps keep driver structured better and prepares for more functionality added to this file.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
> ---
> drivers/net/ethernet/intel/ice/Makefile | 1 +
> drivers/net/ethernet/intel/ice/ice.h | 1 +
> drivers/net/ethernet/intel/ice/ice_irq.c | 226 ++++++++++++++++++++++ drivers/net/ethernet/intel/ice/ice_irq.h | 10 + drivers/net/ethernet/intel/ice/ice_main.c | 218 ---------------------
> 5 files changed, 238 insertions(+), 218 deletions(-) create mode 100644 drivers/net/ethernet/intel/ice/ice_irq.c
> create mode 100644 drivers/net/ethernet/intel/ice/ice_irq.h
>
Tested-by: Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* RE: [Intel-wired-lan] [PATCH net-next v3 2/8] ice: use pci_irq_vector helper function
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-04-21 5:41 ` Pucha, HimasekharX Reddy
-1 siblings, 0 replies; 55+ messages in thread
From: Pucha, HimasekharX Reddy @ 2023-04-21 5:41 UTC (permalink / raw)
To: Raczynski, Piotr, intel-wired-lan
Cc: Swiatkowski, Michal, netdev, Brandeburg, Jesse, Saleem, Shiraz
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Raczynski, Piotr
> Sent: Thursday, March 23, 2023 5:55 PM
> To: intel-wired-lan@lists.osuosl.org
> Cc: Swiatkowski, Michal <michal.swiatkowski@intel.com>; netdev@vger.kernel.org; Brandeburg, Jesse <jesse.brandeburg@intel.com>; Saleem, Shiraz <shiraz.saleem@intel.com>
> Subject: [Intel-wired-lan] [PATCH net-next v3 2/8] ice: use pci_irq_vector helper function
>
> Currently, driver gets interrupt number directly from ice_pf::msix_entries array. Use helper function dedicated to do just that.
>
> While at it use a variable to store interrupt number in ice_free_irq_msix_misc instead of calling the helper function twice.
>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
> ---
> drivers/net/ethernet/intel/ice/ice_arfs.c | 2 +- drivers/net/ethernet/intel/ice/ice_lib.c | 4 ++-- drivers/net/ethernet/intel/ice/ice_main.c | 12 ++++++------ drivers/net/ethernet/intel/ice/ice_ptp.c | 2 +- > drivers/net/ethernet/intel/ice/ice_xsk.c | 2 +-
> 5 files changed, 11 insertions(+), 11 deletions(-)
>
Tested-by: Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 2/8] ice: use pci_irq_vector helper function
@ 2023-04-21 5:41 ` Pucha, HimasekharX Reddy
0 siblings, 0 replies; 55+ messages in thread
From: Pucha, HimasekharX Reddy @ 2023-04-21 5:41 UTC (permalink / raw)
To: Raczynski, Piotr, intel-wired-lan
Cc: netdev, Saleem, Shiraz, Brandeburg, Jesse, Swiatkowski, Michal
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Raczynski, Piotr
> Sent: Thursday, March 23, 2023 5:55 PM
> To: intel-wired-lan@lists.osuosl.org
> Cc: Swiatkowski, Michal <michal.swiatkowski@intel.com>; netdev@vger.kernel.org; Brandeburg, Jesse <jesse.brandeburg@intel.com>; Saleem, Shiraz <shiraz.saleem@intel.com>
> Subject: [Intel-wired-lan] [PATCH net-next v3 2/8] ice: use pci_irq_vector helper function
>
> Currently, driver gets interrupt number directly from ice_pf::msix_entries array. Use helper function dedicated to do just that.
>
> While at it use a variable to store interrupt number in ice_free_irq_msix_misc instead of calling the helper function twice.
>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
> ---
> drivers/net/ethernet/intel/ice/ice_arfs.c | 2 +- drivers/net/ethernet/intel/ice/ice_lib.c | 4 ++-- drivers/net/ethernet/intel/ice/ice_main.c | 12 ++++++------ drivers/net/ethernet/intel/ice/ice_ptp.c | 2 +- > drivers/net/ethernet/intel/ice/ice_xsk.c | 2 +-
> 5 files changed, 11 insertions(+), 11 deletions(-)
>
Tested-by: Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* RE: [Intel-wired-lan] [PATCH net-next v3 3/8] ice: use preferred MSIX allocation api
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-04-21 5:45 ` Pucha, HimasekharX Reddy
-1 siblings, 0 replies; 55+ messages in thread
From: Pucha, HimasekharX Reddy @ 2023-04-21 5:45 UTC (permalink / raw)
To: Raczynski, Piotr, intel-wired-lan
Cc: Swiatkowski, Michal, netdev, Brandeburg, Jesse, Saleem, Shiraz
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Raczynski, Piotr
> Sent: Thursday, March 23, 2023 5:55 PM
> To: intel-wired-lan@lists.osuosl.org
> Cc: Swiatkowski, Michal <michal.swiatkowski@intel.com>; netdev@vger.kernel.org; Brandeburg, Jesse <jesse.brandeburg@intel.com>; Saleem, Shiraz <shiraz.saleem@intel.com>
> Subject: [Intel-wired-lan] [PATCH net-next v3 3/8] ice: use preferred MSIX allocation api
>
> Move away from using pci_enable_msix_range/pci_disable_msix and use pci_alloc_irq_vectors/pci_free_irq_vectors instead.
>
> As a result stop tracking msix_entries since with newer API entries are handled by MSIX core. However, due to current design of communication with RDMA driver which accesses ice_pf::msix_entries directly, keep using the array just for RDMA driver use.
>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
> ---
> drivers/net/ethernet/intel/ice/ice_idc.c | 29 ++++++++++++++-- drivers/net/ethernet/intel/ice/ice_irq.c | 40 +++++------------------ drivers/net/ethernet/intel/ice/ice_main.c | 6 ++--
> 3 files changed, 37 insertions(+), 38 deletions(-)
>
Tested-by: Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 3/8] ice: use preferred MSIX allocation api
@ 2023-04-21 5:45 ` Pucha, HimasekharX Reddy
0 siblings, 0 replies; 55+ messages in thread
From: Pucha, HimasekharX Reddy @ 2023-04-21 5:45 UTC (permalink / raw)
To: Raczynski, Piotr, intel-wired-lan
Cc: netdev, Saleem, Shiraz, Brandeburg, Jesse, Swiatkowski, Michal
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Raczynski, Piotr
> Sent: Thursday, March 23, 2023 5:55 PM
> To: intel-wired-lan@lists.osuosl.org
> Cc: Swiatkowski, Michal <michal.swiatkowski@intel.com>; netdev@vger.kernel.org; Brandeburg, Jesse <jesse.brandeburg@intel.com>; Saleem, Shiraz <shiraz.saleem@intel.com>
> Subject: [Intel-wired-lan] [PATCH net-next v3 3/8] ice: use preferred MSIX allocation api
>
> Move away from using pci_enable_msix_range/pci_disable_msix and use pci_alloc_irq_vectors/pci_free_irq_vectors instead.
>
> As a result stop tracking msix_entries since with newer API entries are handled by MSIX core. However, due to current design of communication with RDMA driver which accesses ice_pf::msix_entries directly, keep using the array just for RDMA driver use.
>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
> ---
> drivers/net/ethernet/intel/ice/ice_idc.c | 29 ++++++++++++++-- drivers/net/ethernet/intel/ice/ice_irq.c | 40 +++++------------------ drivers/net/ethernet/intel/ice/ice_main.c | 6 ++--
> 3 files changed, 37 insertions(+), 38 deletions(-)
>
Tested-by: Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* RE: [Intel-wired-lan] [PATCH net-next v3 6/8] ice: add individual interrupt allocation
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-04-21 5:48 ` Pucha, HimasekharX Reddy
-1 siblings, 0 replies; 55+ messages in thread
From: Pucha, HimasekharX Reddy @ 2023-04-21 5:48 UTC (permalink / raw)
To: Raczynski, Piotr, intel-wired-lan
Cc: Swiatkowski, Michal, netdev, Brandeburg, Jesse, Saleem, Shiraz
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Raczynski, Piotr
> Sent: Thursday, March 23, 2023 5:55 PM
> To: intel-wired-lan@lists.osuosl.org
> Cc: Swiatkowski, Michal <michal.swiatkowski@intel.com>; netdev@vger.kernel.org; Brandeburg, Jesse <jesse.brandeburg@intel.com>; Saleem, Shiraz <shiraz.saleem@intel.com>
> Subject: [Intel-wired-lan] [PATCH net-next v3 6/8] ice: add individual interrupt allocation
>
> Currently interrupt allocations, depending on a feature are distributed in batches. Also, after allocation there is a series of operations that distributes per irq settings through that batch of interrupts.
>
> Although driver does not yet support dynamic interrupt allocation, keep allocated interrupts in a pool and add allocation abstraction logic to make code more flexible. Keep per interrupt information in the ice_q_vector structure, which yields ice_vsi::base_vector redundant.
> Also, as a result there are a few functions that can be removed.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
> ---
> drivers/net/ethernet/intel/ice/ice.h | 11 +-
> drivers/net/ethernet/intel/ice/ice_arfs.c | 5 +-
> drivers/net/ethernet/intel/ice/ice_base.c | 36 ++-
> drivers/net/ethernet/intel/ice/ice_ethtool.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_idc.c | 45 ++--
> drivers/net/ethernet/intel/ice/ice_irq.c | 46 +++-
> drivers/net/ethernet/intel/ice/ice_irq.h | 3 +
> drivers/net/ethernet/intel/ice/ice_lib.c | 225 ++-----------------
> drivers/net/ethernet/intel/ice/ice_lib.h | 4 +-
> drivers/net/ethernet/intel/ice/ice_main.c | 44 ++--
> drivers/net/ethernet/intel/ice/ice_ptp.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_sriov.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_xsk.c | 5 +-
> 13 files changed, 154 insertions(+), 276 deletions(-)
>
Tested-by: Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 6/8] ice: add individual interrupt allocation
@ 2023-04-21 5:48 ` Pucha, HimasekharX Reddy
0 siblings, 0 replies; 55+ messages in thread
From: Pucha, HimasekharX Reddy @ 2023-04-21 5:48 UTC (permalink / raw)
To: Raczynski, Piotr, intel-wired-lan
Cc: netdev, Saleem, Shiraz, Brandeburg, Jesse, Swiatkowski, Michal
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Raczynski, Piotr
> Sent: Thursday, March 23, 2023 5:55 PM
> To: intel-wired-lan@lists.osuosl.org
> Cc: Swiatkowski, Michal <michal.swiatkowski@intel.com>; netdev@vger.kernel.org; Brandeburg, Jesse <jesse.brandeburg@intel.com>; Saleem, Shiraz <shiraz.saleem@intel.com>
> Subject: [Intel-wired-lan] [PATCH net-next v3 6/8] ice: add individual interrupt allocation
>
> Currently interrupt allocations, depending on a feature are distributed in batches. Also, after allocation there is a series of operations that distributes per irq settings through that batch of interrupts.
>
> Although driver does not yet support dynamic interrupt allocation, keep allocated interrupts in a pool and add allocation abstraction logic to make code more flexible. Keep per interrupt information in the ice_q_vector structure, which yields ice_vsi::base_vector redundant.
> Also, as a result there are a few functions that can be removed.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
> ---
> drivers/net/ethernet/intel/ice/ice.h | 11 +-
> drivers/net/ethernet/intel/ice/ice_arfs.c | 5 +-
> drivers/net/ethernet/intel/ice/ice_base.c | 36 ++-
> drivers/net/ethernet/intel/ice/ice_ethtool.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_idc.c | 45 ++--
> drivers/net/ethernet/intel/ice/ice_irq.c | 46 +++-
> drivers/net/ethernet/intel/ice/ice_irq.h | 3 +
> drivers/net/ethernet/intel/ice/ice_lib.c | 225 ++-----------------
> drivers/net/ethernet/intel/ice/ice_lib.h | 4 +-
> drivers/net/ethernet/intel/ice/ice_main.c | 44 ++--
> drivers/net/ethernet/intel/ice/ice_ptp.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_sriov.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_xsk.c | 5 +-
> 13 files changed, 154 insertions(+), 276 deletions(-)
>
Tested-by: Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* RE: [Intel-wired-lan] [PATCH net-next v3 7/8] ice: track interrupt vectors with xarray
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-04-21 5:53 ` Pucha, HimasekharX Reddy
-1 siblings, 0 replies; 55+ messages in thread
From: Pucha, HimasekharX Reddy @ 2023-04-21 5:53 UTC (permalink / raw)
To: Raczynski, Piotr, intel-wired-lan
Cc: Swiatkowski, Michal, netdev, Brandeburg, Jesse, Saleem, Shiraz
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Raczynski, Piotr
> Sent: Thursday, March 23, 2023 5:55 PM
> To: intel-wired-lan@lists.osuosl.org
> Cc: Swiatkowski, Michal <michal.swiatkowski@intel.com>; netdev@vger.kernel.org; Brandeburg, Jesse <jesse.brandeburg@intel.com>; Saleem, Shiraz <shiraz.saleem@intel.com>
> Subject: [Intel-wired-lan] [PATCH net-next v3 7/8] ice: track interrupt vectors with xarray
>
> Replace custom interrupt tracker with generic xarray data structure.
> Remove all code responsible for searching for a new entry with xa_alloc, which always tries to allocate at the lowes possible index. As a result driver is always using a contiguous region of the MSIX vector table.
>
> New tracker keeps ice_irq_entry entries in xarray as opaque for the rest of the driver hiding the entry details from the caller.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
> ---
> drivers/net/ethernet/intel/ice/ice.h | 9 +-
> drivers/net/ethernet/intel/ice/ice_irq.c | 102 ++++++++++++++++-----
> drivers/net/ethernet/intel/ice/ice_irq.h | 9 ++
> drivers/net/ethernet/intel/ice/ice_lib.c | 45 ---------
> drivers/net/ethernet/intel/ice/ice_lib.h | 5 -
> drivers/net/ethernet/intel/ice/ice_sriov.c | 4 +-
> 6 files changed, 90 insertions(+), 84 deletions(-)
>
Tested-by: Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 7/8] ice: track interrupt vectors with xarray
@ 2023-04-21 5:53 ` Pucha, HimasekharX Reddy
0 siblings, 0 replies; 55+ messages in thread
From: Pucha, HimasekharX Reddy @ 2023-04-21 5:53 UTC (permalink / raw)
To: Raczynski, Piotr, intel-wired-lan
Cc: netdev, Saleem, Shiraz, Brandeburg, Jesse, Swiatkowski, Michal
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Raczynski, Piotr
> Sent: Thursday, March 23, 2023 5:55 PM
> To: intel-wired-lan@lists.osuosl.org
> Cc: Swiatkowski, Michal <michal.swiatkowski@intel.com>; netdev@vger.kernel.org; Brandeburg, Jesse <jesse.brandeburg@intel.com>; Saleem, Shiraz <shiraz.saleem@intel.com>
> Subject: [Intel-wired-lan] [PATCH net-next v3 7/8] ice: track interrupt vectors with xarray
>
> Replace custom interrupt tracker with generic xarray data structure.
> Remove all code responsible for searching for a new entry with xa_alloc, which always tries to allocate at the lowes possible index. As a result driver is always using a contiguous region of the MSIX vector table.
>
> New tracker keeps ice_irq_entry entries in xarray as opaque for the rest of the driver hiding the entry details from the caller.
>
> Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
> ---
> drivers/net/ethernet/intel/ice/ice.h | 9 +-
> drivers/net/ethernet/intel/ice/ice_irq.c | 102 ++++++++++++++++-----
> drivers/net/ethernet/intel/ice/ice_irq.h | 9 ++
> drivers/net/ethernet/intel/ice/ice_lib.c | 45 ---------
> drivers/net/ethernet/intel/ice/ice_lib.h | 5 -
> drivers/net/ethernet/intel/ice/ice_sriov.c | 4 +-
> 6 files changed, 90 insertions(+), 84 deletions(-)
>
Tested-by: Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
* RE: [Intel-wired-lan] [PATCH net-next v3 8/8] ice: add dynamic interrupt allocation
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
@ 2023-04-21 5:55 ` Pucha, HimasekharX Reddy
-1 siblings, 0 replies; 55+ messages in thread
From: Pucha, HimasekharX Reddy @ 2023-04-21 5:55 UTC (permalink / raw)
To: Raczynski, Piotr, intel-wired-lan
Cc: Swiatkowski, Michal, netdev, Brandeburg, Jesse, Saleem, Shiraz
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Raczynski, Piotr
> Sent: Thursday, March 23, 2023 5:55 PM
> To: intel-wired-lan@lists.osuosl.org
> Cc: Swiatkowski, Michal <michal.swiatkowski@intel.com>; netdev@vger.kernel.org; Brandeburg, Jesse <jesse.brandeburg@intel.com>; Saleem, Shiraz <shiraz.saleem@intel.com>
> Subject: [Intel-wired-lan] [PATCH net-next v3 8/8] ice: add dynamic interrupt allocation
>
> Currently driver can only allocate interrupt vectors during init phase by calling pci_alloc_irq_vectors. Change that and make use of new pci_msix_alloc_irq_at/pci_msix_free_irq API and enable to allocate and free more interrupts after MSIX has been enabled. Since not all platforms supports dynamic allocation, check it with pci_msix_can_alloc_dyn.
>
> Extend the tracker to keep track how many interrupts are allocated initially so when all such vectors are already used, additional interrupts are automatically allocated dynamically. Remember each interrupt allocation method to then free appropriately. Since some features may require interrupts allocated dynamically add appropriate VSI flag and take it into account when allocating new interrupt.
>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
> ---
> drivers/net/ethernet/intel/ice/ice.h | 3 +
> drivers/net/ethernet/intel/ice/ice_base.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_idc.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_irq.c | 107 ++++++++++++++++++---
> drivers/net/ethernet/intel/ice/ice_irq.h | 5 +-
> drivers/net/ethernet/intel/ice/ice_main.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_sriov.c | 5 +-
> 7 files changed, 105 insertions(+), 21 deletions(-)
>
Tested-by: Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [Intel-wired-lan] [PATCH net-next v3 8/8] ice: add dynamic interrupt allocation
@ 2023-04-21 5:55 ` Pucha, HimasekharX Reddy
0 siblings, 0 replies; 55+ messages in thread
From: Pucha, HimasekharX Reddy @ 2023-04-21 5:55 UTC (permalink / raw)
To: Raczynski, Piotr, intel-wired-lan
Cc: netdev, Saleem, Shiraz, Brandeburg, Jesse, Swiatkowski, Michal
> -----Original Message-----
> From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Raczynski, Piotr
> Sent: Thursday, March 23, 2023 5:55 PM
> To: intel-wired-lan@lists.osuosl.org
> Cc: Swiatkowski, Michal <michal.swiatkowski@intel.com>; netdev@vger.kernel.org; Brandeburg, Jesse <jesse.brandeburg@intel.com>; Saleem, Shiraz <shiraz.saleem@intel.com>
> Subject: [Intel-wired-lan] [PATCH net-next v3 8/8] ice: add dynamic interrupt allocation
>
> Currently driver can only allocate interrupt vectors during init phase by calling pci_alloc_irq_vectors. Change that and make use of new pci_msix_alloc_irq_at/pci_msix_free_irq API and enable to allocate and free more interrupts after MSIX has been enabled. Since not all platforms supports dynamic allocation, check it with pci_msix_can_alloc_dyn.
>
> Extend the tracker to keep track how many interrupts are allocated initially so when all such vectors are already used, additional interrupts are automatically allocated dynamically. Remember each interrupt allocation method to then free appropriately. Since some features may require interrupts allocated dynamically add appropriate VSI flag and take it into account when allocating new interrupt.
>
> Reviewed-by: Michal Swiatkowski <michal.swiatkowski@linux.intel.com>
> Signed-off-by: Piotr Raczynski <piotr.raczynski@intel.com>
> ---
> drivers/net/ethernet/intel/ice/ice.h | 3 +
> drivers/net/ethernet/intel/ice/ice_base.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_idc.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_irq.c | 107 ++++++++++++++++++---
> drivers/net/ethernet/intel/ice/ice_irq.h | 5 +-
> drivers/net/ethernet/intel/ice/ice_main.c | 2 +-
> drivers/net/ethernet/intel/ice/ice_sriov.c | 5 +-
> 7 files changed, 105 insertions(+), 21 deletions(-)
>
Tested-by: Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
^ permalink raw reply [flat|nested] 55+ messages in thread
end of thread, other threads:[~2023-04-21 16:01 UTC | newest]
Thread overview: 55+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-03-23 12:24 [PATCH net-next v3 0/8] ice: support dynamic interrupt allocation Piotr Raczynski
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
2023-03-23 12:24 ` [PATCH net-next v3 1/8] ice: move interrupt related code to separate file Piotr Raczynski
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
2023-03-26 13:34 ` Simon Horman
2023-03-26 13:34 ` Simon Horman
2023-04-21 5:36 ` [Intel-wired-lan] " Pucha, HimasekharX Reddy
2023-04-21 5:36 ` Pucha, HimasekharX Reddy
2023-03-23 12:24 ` [PATCH net-next v3 2/8] ice: use pci_irq_vector helper function Piotr Raczynski
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
2023-03-26 13:35 ` Simon Horman
2023-03-26 13:35 ` Simon Horman
2023-04-21 5:41 ` [Intel-wired-lan] " Pucha, HimasekharX Reddy
2023-04-21 5:41 ` Pucha, HimasekharX Reddy
2023-03-23 12:24 ` [PATCH net-next v3 3/8] ice: use preferred MSIX allocation api Piotr Raczynski
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
2023-03-26 13:35 ` Simon Horman
2023-03-26 13:35 ` Simon Horman
2023-04-21 5:45 ` [Intel-wired-lan] " Pucha, HimasekharX Reddy
2023-04-21 5:45 ` Pucha, HimasekharX Reddy
2023-03-23 12:24 ` [Intel-wired-lan] [PATCH net-next v3 4/8] ice: refactor VF control VSI interrupt handling Piotr Raczynski
2023-03-23 12:24 ` Piotr Raczynski
2023-03-26 13:36 ` Simon Horman
2023-03-26 13:36 ` [Intel-wired-lan] " Simon Horman
2023-03-30 16:53 ` Romanowski, Rafal
2023-03-23 12:24 ` [Intel-wired-lan] [PATCH net-next v3 5/8] ice: remove redundant SRIOV code Piotr Raczynski
2023-03-23 12:24 ` Piotr Raczynski
2023-03-26 13:36 ` Simon Horman
2023-03-26 13:36 ` [Intel-wired-lan] " Simon Horman
2023-04-06 9:50 ` Romanowski, Rafal
2023-04-06 9:50 ` Romanowski, Rafal
2023-03-23 12:24 ` [PATCH net-next v3 6/8] ice: add individual interrupt allocation Piotr Raczynski
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
2023-03-26 13:18 ` Simon Horman
2023-03-26 13:18 ` [Intel-wired-lan] " Simon Horman
2023-03-28 16:16 ` Piotr Raczynski
2023-03-28 16:16 ` [Intel-wired-lan] " Piotr Raczynski
2023-04-21 5:48 ` Pucha, HimasekharX Reddy
2023-04-21 5:48 ` Pucha, HimasekharX Reddy
2023-03-23 12:24 ` [PATCH net-next v3 7/8] ice: track interrupt vectors with xarray Piotr Raczynski
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
2023-03-26 13:19 ` Simon Horman
2023-03-26 13:19 ` [Intel-wired-lan] " Simon Horman
2023-03-28 16:12 ` Piotr Raczynski
2023-03-28 16:12 ` [Intel-wired-lan] " Piotr Raczynski
2023-04-21 5:53 ` Pucha, HimasekharX Reddy
2023-04-21 5:53 ` Pucha, HimasekharX Reddy
2023-03-23 12:24 ` [PATCH net-next v3 8/8] ice: add dynamic interrupt allocation Piotr Raczynski
2023-03-23 12:24 ` [Intel-wired-lan] " Piotr Raczynski
2023-03-26 13:37 ` Simon Horman
2023-03-26 13:37 ` [Intel-wired-lan] " Simon Horman
2023-04-21 5:55 ` Pucha, HimasekharX Reddy
2023-04-21 5:55 ` Pucha, HimasekharX Reddy
2023-03-29 11:35 ` [PATCH net-next v3 0/8] ice: support " Leon Romanovsky
2023-03-29 11:35 ` [Intel-wired-lan] " Leon Romanovsky
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.