From: Sreekanth Reddy <sreekanth.reddy@broadcom.com>
To: Nitesh Narayan Lal <nitesh@redhat.com>
Cc: linux-kernel@vger.kernel.org,
linux-scsi <linux-scsi@vger.kernel.org>,
intel-wired-lan@lists.osuosl.org, netdev@vger.kernel.org,
linux-api@vger.kernel.org, linux-pci@vger.kernel.org,
Thomas Gleixner <tglx@linutronix.de>,
jesse.brandeburg@intel.com, robin.murphy@arm.com,
mtosatti@redhat.com, Ingo Molnar <mingo@kernel.org>,
jbrandeb@kernel.org, frederic@kernel.org, juri.lelli@redhat.com,
abelits@marvell.com, Bjorn Helgaas <bhelgaas@google.com>,
rostedt@goodmis.org, Peter Zijlstra <peterz@infradead.org>,
David Miller <davem@davemloft.net>,
Andrew Morton <akpm@linux-foundation.org>,
Stephen Rothwell <sfr@canb.auug.org.au>,
stephen@networkplumber.org, rppt@linux.vnet.ibm.com,
chris.friesen@windriver.com, maz@kernel.org,
Neil Horman <nhorman@tuxdriver.com>,
pjwaskiewicz@gmail.com, sassmann@redhat.com,
Tomas Henzl <thenzl@redhat.com>,
Kashyap Desai <kashyap.desai@broadcom.com>,
Sumit Saxena <sumit.saxena@broadcom.com>,
Shivasharan Srikanteshwara
<shivasharan.srikanteshwara@broadcom.com>,
Sathya Prakash Veerichetty <sathya.prakash@broadcom.com>,
Suganath Prabu Subramani <suganath-prabu.subramani@broadcom.com>,
james.smart@broadcom.com, dick.kennedy@broadcom.com,
jkc@redhat.com, faisal.latif@intel.com, shiraz.saleem@intel.com,
tariqt@nvidia.com, ahleihel@redhat.com, kheib@redhat.com,
borisp@nvidia.com, saeedm@nvidia.com, benve@cisco.com,
govind@gmx.com, jassisinghbrar@gmail.com,
ajit.khaparde@broadcom.com, sriharsha.basavapatna@broadcom.com,
somnath.kotur@broadcom.com, nilal@redhat.com,
tatyana.e.nikolova@intel.com, mustafa.ismail@intel.com,
ahs3@redhat.com, leonro@nvidia.com,
Chandrakanth Patil <chandrakanth.patil@broadcom.com>,
bjorn.andersson@linaro.org, chunkuang.hu@kernel.org,
yongqiang.niu@mediatek.com, baolin.wang7@gmail.com,
poros@redhat.com, minlei@redhat.com,
"Ewan D. Milne" <emilne@redhat.com>,
"James E.J. Bottomley" <jejb@linux.ibm.com>,
"Martin K. Petersen" <martin.petersen@oracle.com>,
_govind@gmx.com, kabel@kernel.org, viresh.kumar@linaro.org,
Tushar.Khandelwal@arm.com, kuba@kernel.org
Subject: Re: [PATCH v5 05/14] scsi: mpt3sas: Use irq_set_affinity_and_hint
Date: Fri, 6 Aug 2021 14:24:49 +0530 [thread overview]
Message-ID: <CAK=zhgo29UsFsK_q9RYnEocmTA20kUVX4BNnrpJPh35Ssb49zw@mail.gmail.com> (raw)
In-Reply-To: <20210720232624.1493424-6-nitesh@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 4148 bytes --]
On Wed, Jul 21, 2021 at 4:57 AM Nitesh Narayan Lal <nitesh@redhat.com> wrote:
>
> The driver uses irq_set_affinity_hint() specifically for the high IOPS
> queue interrupts for two purposes:
>
> - To set the affinity_hint which is consumed by the userspace for
> distributing the interrupts
>
> - To apply an affinity that it provides
>
> The driver enforces its own affinity to bind the high IOPS queue interrupts
> to the local NUMA node. However, irq_set_affinity_hint() applying the
> provided cpumask as an affinity (if not NULL) for the interrupt is an
> undocumented side effect.
>
> To remove this side effect irq_set_affinity_hint() has been marked
> as deprecated and new interfaces have been introduced. Hence, replace the
> irq_set_affinity_hint() with the new interface irq_set_affinity_and_hint()
> where the provided mask needs to be applied as the affinity and
> affinity_hint pointer needs to be set and replace with
> irq_update_affinity_hint() where only affinity_hint needs to be updated.
>
Changes looks good and also verified that the high iops queue's IRQs
are affinitied to local numa node.
Reviewed-by: Sreekanth Reddy <sreekanth.reddy@broadcom.com>
> Signed-off-by: Nitesh Narayan Lal <nitesh@redhat.com>
> ---
> drivers/scsi/mpt3sas/mpt3sas_base.c | 21 ++++++++++-----------
> 1 file changed, 10 insertions(+), 11 deletions(-)
>
> diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.c b/drivers/scsi/mpt3sas/mpt3sas_base.c
> index c39955239d1c..c1a11962f227 100644
> --- a/drivers/scsi/mpt3sas/mpt3sas_base.c
> +++ b/drivers/scsi/mpt3sas/mpt3sas_base.c
> @@ -2991,6 +2991,7 @@ _base_check_enable_msix(struct MPT3SAS_ADAPTER *ioc)
> static void
> _base_free_irq(struct MPT3SAS_ADAPTER *ioc)
> {
> + unsigned int irq;
> struct adapter_reply_queue *reply_q, *next;
>
> if (list_empty(&ioc->reply_queue_list))
> @@ -2998,9 +2999,10 @@ _base_free_irq(struct MPT3SAS_ADAPTER *ioc)
>
> list_for_each_entry_safe(reply_q, next, &ioc->reply_queue_list, list) {
> list_del(&reply_q->list);
> - if (ioc->smp_affinity_enable)
> - irq_set_affinity_hint(pci_irq_vector(ioc->pdev,
> - reply_q->msix_index), NULL);
> + if (ioc->smp_affinity_enable) {
> + irq = pci_irq_vector(ioc->pdev, reply_q->msix_index);
> + irq_update_affinity_hint(irq, NULL);
> + }
> free_irq(pci_irq_vector(ioc->pdev, reply_q->msix_index),
> reply_q);
> kfree(reply_q);
> @@ -3056,16 +3058,13 @@ _base_request_irq(struct MPT3SAS_ADAPTER *ioc, u8 index)
> * @ioc: per adapter object
> *
> * The enduser would need to set the affinity via /proc/irq/#/smp_affinity
> - *
> - * It would nice if we could call irq_set_affinity, however it is not
> - * an exported symbol
> */
> static void
> _base_assign_reply_queues(struct MPT3SAS_ADAPTER *ioc)
> {
> - unsigned int cpu, nr_cpus, nr_msix, index = 0;
> + unsigned int cpu, nr_cpus, nr_msix, index = 0, irq;
> struct adapter_reply_queue *reply_q;
> - int local_numa_node;
> + const struct cpumask *mask;
>
> if (!_base_is_controller_msix_enabled(ioc))
> return;
> @@ -3088,11 +3087,11 @@ _base_assign_reply_queues(struct MPT3SAS_ADAPTER *ioc)
> * corresponding to high iops queues.
> */
> if (ioc->high_iops_queues) {
> - local_numa_node = dev_to_node(&ioc->pdev->dev);
> + mask = cpumask_of_node(dev_to_node(&ioc->pdev->dev));
> for (index = 0; index < ioc->high_iops_queues;
> index++) {
> - irq_set_affinity_hint(pci_irq_vector(ioc->pdev,
> - index), cpumask_of_node(local_numa_node));
> + irq = pci_irq_vector(ioc->pdev, index);
> + irq_set_affinity_and_hint(irq, mask);
> }
> }
>
> --
> 2.27.0
>
[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4218 bytes --]
next prev parent reply other threads:[~2021-08-06 8:55 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-20 23:26 [PATCH v5 00/14] genirq: Cleanup the abuse of irq_set_affinity_hint() Nitesh Narayan Lal
2021-07-20 23:26 ` [PATCH v5 01/14] genirq: Provide new interfaces for affinity hints Nitesh Narayan Lal
2021-08-06 15:51 ` Ming Lei
2021-07-20 23:26 ` [PATCH v5 02/14] iavf: Use irq_update_affinity_hint Nitesh Narayan Lal
2021-07-20 23:26 ` [PATCH v5 03/14] i40e: " Nitesh Narayan Lal
2021-07-20 23:26 ` [PATCH v5 04/14] scsi: megaraid_sas: Use irq_set_affinity_and_hint Nitesh Narayan Lal
2021-08-06 11:31 ` Sumit Saxena
2021-07-20 23:26 ` [PATCH v5 05/14] scsi: mpt3sas: " Nitesh Narayan Lal
2021-08-06 8:54 ` Sreekanth Reddy [this message]
2021-07-20 23:26 ` [PATCH v5 06/14] RDMA/irdma: Use irq_update_affinity_hint Nitesh Narayan Lal
2021-07-22 18:25 ` Nikolova, Tatyana E
2021-07-20 23:26 ` [PATCH v5 07/14] enic: " Nitesh Narayan Lal
2021-08-16 17:53 ` Christian Benvenuti (benve)
2021-07-20 23:26 ` [PATCH v5 08/14] be2net: " Nitesh Narayan Lal
2021-07-20 23:26 ` [PATCH v5 09/14] ixgbe: " Nitesh Narayan Lal
2021-07-20 23:26 ` [PATCH v5 10/14] mailbox: " Nitesh Narayan Lal
2021-08-16 18:11 ` Jassi Brar
2021-07-20 23:26 ` [PATCH v5 11/14] scsi: lpfc: Use irq_set_affinity Nitesh Narayan Lal
2021-07-23 17:15 ` James Smart
2021-07-20 23:26 ` [PATCH v5 12/14] hinic: Use irq_set_affinity_and_hint Nitesh Narayan Lal
2021-07-20 23:26 ` [PATCH v5 13/14] net/mlx5: " Nitesh Narayan Lal
2021-07-20 23:26 ` [PATCH v5 14/14] net/mlx4: Use irq_update_affinity_hint Nitesh Narayan Lal
2021-08-02 15:26 ` [PATCH v5 00/14] genirq: Cleanup the abuse of irq_set_affinity_hint() Nitesh Lal
2021-08-06 3:04 ` Martin K. Petersen
2021-08-06 14:56 ` Nitesh Lal
2021-08-16 15:50 ` Nitesh Lal
2021-08-30 15:38 ` Nitesh Lal
2021-08-30 15:40 ` Nitesh Lal
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAK=zhgo29UsFsK_q9RYnEocmTA20kUVX4BNnrpJPh35Ssb49zw@mail.gmail.com' \
--to=sreekanth.reddy@broadcom.com \
--cc=Tushar.Khandelwal@arm.com \
--cc=_govind@gmx.com \
--cc=abelits@marvell.com \
--cc=ahleihel@redhat.com \
--cc=ahs3@redhat.com \
--cc=ajit.khaparde@broadcom.com \
--cc=akpm@linux-foundation.org \
--cc=baolin.wang7@gmail.com \
--cc=benve@cisco.com \
--cc=bhelgaas@google.com \
--cc=bjorn.andersson@linaro.org \
--cc=borisp@nvidia.com \
--cc=chandrakanth.patil@broadcom.com \
--cc=chris.friesen@windriver.com \
--cc=chunkuang.hu@kernel.org \
--cc=davem@davemloft.net \
--cc=dick.kennedy@broadcom.com \
--cc=emilne@redhat.com \
--cc=faisal.latif@intel.com \
--cc=frederic@kernel.org \
--cc=govind@gmx.com \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=james.smart@broadcom.com \
--cc=jassisinghbrar@gmail.com \
--cc=jbrandeb@kernel.org \
--cc=jejb@linux.ibm.com \
--cc=jesse.brandeburg@intel.com \
--cc=jkc@redhat.com \
--cc=juri.lelli@redhat.com \
--cc=kabel@kernel.org \
--cc=kashyap.desai@broadcom.com \
--cc=kheib@redhat.com \
--cc=kuba@kernel.org \
--cc=leonro@nvidia.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=martin.petersen@oracle.com \
--cc=maz@kernel.org \
--cc=mingo@kernel.org \
--cc=minlei@redhat.com \
--cc=mtosatti@redhat.com \
--cc=mustafa.ismail@intel.com \
--cc=netdev@vger.kernel.org \
--cc=nhorman@tuxdriver.com \
--cc=nilal@redhat.com \
--cc=nitesh@redhat.com \
--cc=peterz@infradead.org \
--cc=pjwaskiewicz@gmail.com \
--cc=poros@redhat.com \
--cc=robin.murphy@arm.com \
--cc=rostedt@goodmis.org \
--cc=rppt@linux.vnet.ibm.com \
--cc=saeedm@nvidia.com \
--cc=sassmann@redhat.com \
--cc=sathya.prakash@broadcom.com \
--cc=sfr@canb.auug.org.au \
--cc=shiraz.saleem@intel.com \
--cc=shivasharan.srikanteshwara@broadcom.com \
--cc=somnath.kotur@broadcom.com \
--cc=sriharsha.basavapatna@broadcom.com \
--cc=stephen@networkplumber.org \
--cc=suganath-prabu.subramani@broadcom.com \
--cc=sumit.saxena@broadcom.com \
--cc=tariqt@nvidia.com \
--cc=tatyana.e.nikolova@intel.com \
--cc=tglx@linutronix.de \
--cc=thenzl@redhat.com \
--cc=viresh.kumar@linaro.org \
--cc=yongqiang.niu@mediatek.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).