From: John Garry <john.garry@huawei.com>
To: Marc Zyngier <maz@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>,
Zhou Wang <wangzhou1@hisilicon.com>,
<linux-kernel@vger.kernel.org>
Subject: Re: PCI MSI issue with reinserting a driver
Date: Tue, 2 Feb 2021 15:46:25 +0000 [thread overview]
Message-ID: <28c56995-501a-880b-e6dd-ac76b8290c2c@huawei.com> (raw)
In-Reply-To: <4848792ce8c9ed7490e2205281a3cbda@kernel.org>
On 02/02/2021 14:48, Marc Zyngier wrote:
>>>
>>> Not sure. I also now notice an error for the SAS PCI driver on D06
>>> when nr_cpus < 16, which means number of MSI vectors allocated < 32,
>>> so looks the same problem. There we try to allocate 16 + max(nr cpus,
>>> 16) MSI.
>>>
>>> Anyway, let me have a look today to see what is going wrong.
>>>
>> Could this be the problem:
>>
>> nr_cpus=11
>>
>> In alloc path, we have:
>> its_alloc_device_irq(nvecs=27 = 16+11)
>> bitmap_find_free_region(order = 5);
>> In free path, we have:
>> its_irq_domain_free(nvecs = 1) and free each 27 vecs
>> bitmap_release_region(order = 0)
>>
>> So we allocate 32 bits, but only free 27. And 2nd alloc for 32 fails.
[ ... ]
>>
>>
>> But I'm not sure that we have any requirement for those map bits to be
>> consecutive.
>
> We can't really do that. All the events must be contiguous,
> and there is also a lot of assumptions in the ITS driver that
> LPI allocations is also contiguous.
>
> But there is also the fact that for Multi-MSI, we *must*
> allocate 32 vectors. Any driver could assume that if we have
> allocated 17 vectors, then there is another 15 available.
>
> My question still stand: how was this working with the previous
> behaviour?
Because previously in this scenario we would allocate 32 bits and free
32 bits in the map; but now we allocate 32 bits, yet only free 27 - so
leak 5 bits. And this comes from how irq_domain_free_irqs_hierarchy()
now frees per-interrupt, instead of all irqs per domain.
Before:
In free path, we have:
its_irq_domain_free(nvecs = 27)
bitmap_release_region(count order = 5 == 32bits)
Current:
In free path, we have:
its_irq_domain_free(nvecs = 1) for free each 27 vecs
bitmap_release_region(count order = 0 == 1bit)
Cheers,
John
next prev parent reply other threads:[~2021-02-02 15:50 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-01 18:34 PCI MSI issue with reinserting a driver John Garry
2021-02-01 18:50 ` Marc Zyngier
2021-02-02 8:37 ` John Garry
2021-02-02 12:38 ` John Garry
2021-02-02 14:48 ` Marc Zyngier
2021-02-02 15:46 ` John Garry [this message]
2021-02-03 17:23 ` Marc Zyngier
2021-02-04 10:45 ` John Garry
2022-08-04 10:59 ` John Garry
2021-04-06 9:46 ` John Garry
2021-08-27 8:33 ` luojiaxing
2023-08-29 23:00 ` Thomas Gleixner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=28c56995-501a-880b-e6dd-ac76b8290c2c@huawei.com \
--to=john.garry@huawei.com \
--cc=linux-kernel@vger.kernel.org \
--cc=maz@kernel.org \
--cc=tglx@linutronix.de \
--cc=wangzhou1@hisilicon.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).