From: Jens Axboe <axboe@kernel.dk>
To: Keith Busch <keith.busch@intel.com>
Cc: linux-block@vger.kernel.org, linux-nvme@lists.infradead.org,
Thomas Gleixner <tglx@linutronix.de>,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH 11/14] irq: add support for allocating (and affinitizing) sets of IRQs
Date: Thu, 25 Oct 2018 17:07:19 -0600 [thread overview]
Message-ID: <8d70362e-0c15-5423-9616-0ef92e8c9fe3@kernel.dk> (raw)
In-Reply-To: <20181025215213.GA8141@localhost.localdomain>
On 10/25/18 3:52 PM, Keith Busch wrote:
> On Thu, Oct 25, 2018 at 03:16:23PM -0600, Jens Axboe wrote:
>> A driver may have a need to allocate multiple sets of MSI/MSI-X
>> interrupts, and have them appropriately affinitized. Add support for
>> defining a number of sets in the irq_affinity structure, of varying
>> sizes, and get each set affinitized correctly across the machine.
>
> <>
>
>> @@ -258,13 +272,18 @@ int irq_calc_affinity_vectors(int minvec, int maxvec, const struct irq_affinity
>> {
>> int resv = affd->pre_vectors + affd->post_vectors;
>> int vecs = maxvec - resv;
>> + int i, set_vecs;
>> int ret;
>>
>> if (resv > minvec)
>> return 0;
>>
>> get_online_cpus();
>> - ret = min_t(int, cpumask_weight(cpu_possible_mask), vecs) + resv;
>> + ret = min_t(int, cpumask_weight(cpu_possible_mask), vecs);
>> put_online_cpus();
>> - return ret;
>> +
>> + for (i = 0, set_vecs = 0; i < affd->nr_sets; i++)
>> + set_vecs += affd->sets[i];
>> +
>> + return resv + max(ret, set_vecs);
>> }
>
> This is looking pretty good, but we may risk getting into an infinite
> loop in __pci_enable_msix_range() if we're requesting too many vectors
> in a set: the above code may continue returning set_vecs, overriding
> the reduced nvec that pci requested, and pci msix initialization will
> continue to fail because it is repeatedly requesting to activate the
> same vector count that failed before.
Good catch, we always want to be using min() with the passed in maxvec
in there. How about this incremental?
diff --git a/kernel/irq/affinity.c b/kernel/irq/affinity.c
index 0055e252e438..2046a0f0f0f1 100644
--- a/kernel/irq/affinity.c
+++ b/kernel/irq/affinity.c
@@ -272,18 +272,21 @@ int irq_calc_affinity_vectors(int minvec, int maxvec, const struct irq_affinity
{
int resv = affd->pre_vectors + affd->post_vectors;
int vecs = maxvec - resv;
- int i, set_vecs;
- int ret;
+ int set_vecs;
if (resv > minvec)
return 0;
- get_online_cpus();
- ret = min_t(int, cpumask_weight(cpu_possible_mask), vecs);
- put_online_cpus();
+ if (affd->nr_sets) {
+ int i;
- for (i = 0, set_vecs = 0; i < affd->nr_sets; i++)
- set_vecs += affd->sets[i];
+ for (i = 0, set_vecs = 0; i < affd->nr_sets; i++)
+ set_vecs += affd->sets[i];
+ } else {
+ get_online_cpus();
+ set_vecs = cpumask_weight(cpu_possible_mask);
+ put_online_cpus();
+ }
- return resv + max(ret, set_vecs);
+ return resv + min(set_vecs, vecs);
}
--
Jens Axboe
next prev parent reply other threads:[~2018-10-25 23:07 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20181025211626.12692-1-axboe@kernel.dk>
2018-10-25 21:16 ` [PATCH 11/14] irq: add support for allocating (and affinitizing) sets of IRQs Jens Axboe
2018-10-25 21:52 ` Keith Busch
2018-10-25 23:07 ` Jens Axboe [this message]
2018-10-29 7:43 ` Hannes Reinecke
2018-10-29 16:37 [PATCHSET v2 0/14] blk-mq: Add support for multiple queue maps Jens Axboe
2018-10-29 16:37 ` [PATCH 11/14] irq: add support for allocating (and affinitizing) sets of IRQs Jens Axboe
2018-10-29 17:08 ` Thomas Gleixner
2018-10-29 17:09 ` Jens Axboe
2018-10-30 9:25 ` Ming Lei
2018-10-30 14:26 ` Keith Busch
2018-10-30 14:36 ` Jens Axboe
2018-10-30 14:45 ` Keith Busch
2018-10-30 14:53 ` Jens Axboe
2018-10-30 15:08 ` Keith Busch
2018-10-30 15:18 ` Jens Axboe
2018-10-30 16:02 ` Keith Busch
2018-10-30 16:42 ` Jens Axboe
2018-10-30 17:09 ` Jens Axboe
2018-10-30 17:22 ` Keith Busch
2018-10-30 17:33 ` Jens Axboe
2018-10-30 17:35 ` Keith Busch
2018-10-30 17:25 ` Thomas Gleixner
2018-10-30 17:34 ` Jens Axboe
2018-10-30 17:43 ` Jens Axboe
2018-10-30 17:46 ` Thomas Gleixner
2018-10-30 17:47 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8d70362e-0c15-5423-9616-0ef92e8c9fe3@kernel.dk \
--to=axboe@kernel.dk \
--cc=keith.busch@intel.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).