Re: [PATCH v3 19/19] irqdomain: Switch to per-domain locking

[Thomas Gleixner <tglx@linutronix.de>]

On Mon, Dec 12 2022 at 15:29, Johan Hovold wrote:
> On Mon, Dec 12, 2022 at 03:14:34PM +0100, Thomas Gleixner wrote:
>> On Fri, Dec 09 2022 at 15:01, Johan Hovold wrote:
>> > The IRQ domain structures are currently protected by the global
>> > irq_domain_mutex. Switch to using more fine-grained per-domain locking,
>> > which may potentially speed up parallel probing somewhat.
>> >
>> > Note that the domain lock of the root domain (innermost domain) must be
>> > used for hierarchical domains. For non-hierarchical domain (as for root
>> > domains), the new root pointer is set to the domain itself so that
>> > domain->root->mutex can be used in shared code paths.
>> >
>> > Also note that hierarchical domains should be constructed using
>> > irq_domain_create_hierarchy() (or irq_domain_add_hierarchy()) to avoid
>> > poking at irqdomain internals.
>> 
>> While I agree in principle, this change really makes me nervous.
>> 
>> Any fail in setting up domain->root correctly, e.g. by not using
>> irq_domain_create_hierarchy(), cannot be detected and creates nasty to
>> debug race conditions.
>> 
>> So we need some debug mechanism which allows to validate that
>> domain->root is set up correctly when domain->parent != NULL.
>
> Lockdep will catch that due to the
>
> 	lockdep_assert_held(&domain->root->mutex);
>
> I added to irq_domain_set_mapping() and which is is called for each
> (inner) domain in a hierarchy when allocating an IRQ.

Hmm. Indeed that should do the trick.

Some comments would be appreciated as the rules around domain->root are
far from obvious.

Thanks,

        tglx