All of lore.kernel.org
 help / color / mirror / Atom feed
* irqdomain API: how to set affinity of parent irq of chained irqs?
@ 2022-05-02  8:21 Marek Behún
  2022-05-02  9:31 ` Marc Zyngier
  0 siblings, 1 reply; 4+ messages in thread
From: Marek Behún @ 2022-05-02  8:21 UTC (permalink / raw)
  To: Marc Zyngier; +Cc: linux-kernel, Thomas Gleixner, pali

Dear Marc, Thomas,

we have encountered the following problem that can hopefully be put
some light onto: What is the intended way to set affinity (and possibly
other irq attributes) of parent IRQ of chained IRQs, when using the
irqdomain API?

We are working on a driver that
- registers an irqchip and adds an irqdomain
- calls irq_set_chained_handler_and_data(parent_irq, handler)
  where handler triggers handling of child IRQs
- but since parent_irq isn't requested for with request_thread_irq(),
  it does not show up in proc/sysfs, only in debugfs
- the HW does not support setting affinity for the chained IRQs, only 
  the parent (which comes from a GIC chip)

The problem is that he parent IRQ, as mentioned in the third point, does
not show up in proc/sysfs.

Is there some precedent for this?

Thank you.

Marek

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: irqdomain API: how to set affinity of parent irq of chained irqs?
  2022-05-02  8:21 irqdomain API: how to set affinity of parent irq of chained irqs? Marek Behún
@ 2022-05-02  9:31 ` Marc Zyngier
  2022-05-02 15:45   ` Marek Behún
  0 siblings, 1 reply; 4+ messages in thread
From: Marc Zyngier @ 2022-05-02  9:31 UTC (permalink / raw)
  To: Marek Behún; +Cc: linux-kernel, Thomas Gleixner, pali

On Mon, 02 May 2022 09:21:37 +0100,
Marek Behún <kabel@kernel.org> wrote:
> 
> Dear Marc, Thomas,
> 
> we have encountered the following problem that can hopefully be put
> some light onto: What is the intended way to set affinity (and possibly
> other irq attributes) of parent IRQ of chained IRQs, when using the
> irqdomain API?

Simples: you can't. What sense does it make to change the affinity of
the parent interrupt, given that its fate is tied to *all* of the
other interrupts that are muxed to it?

Moving the parent interrupt breaks userspace's view of how interrupt
affinity is managed (change the affinity of one interrupt, see all the
others move the same way). Which is why we don't expose this interrupt
to userspace, as this can only lead to bad things.

Note that this has nothing to do with the irqdomain API, but
everything to do with the userspace ABI.

> 
> We are working on a driver that
> - registers an irqchip and adds an irqdomain
> - calls irq_set_chained_handler_and_data(parent_irq, handler)
>   where handler triggers handling of child IRQs
> - but since parent_irq isn't requested for with request_thread_irq(),
>   it does not show up in proc/sysfs, only in debugfs
> - the HW does not support setting affinity for the chained IRQs, only 
>   the parent (which comes from a GIC chip)
> 
> The problem is that he parent IRQ, as mentioned in the third point, does
> not show up in proc/sysfs.
> 
> Is there some precedent for this?

There were precedents of irqchips doing terrible things, such as
implementing a set_affinity() callback in the chained irqchip.
Thankfully, they have been either fixed or eradicated.

	M.

-- 
Without deviation from the norm, progress is not possible.

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: irqdomain API: how to set affinity of parent irq of chained irqs?
  2022-05-02  9:31 ` Marc Zyngier
@ 2022-05-02 15:45   ` Marek Behún
  2022-05-03  9:32     ` Marc Zyngier
  0 siblings, 1 reply; 4+ messages in thread
From: Marek Behún @ 2022-05-02 15:45 UTC (permalink / raw)
  To: Marc Zyngier; +Cc: linux-kernel, Thomas Gleixner, pali

On Mon, 02 May 2022 10:31:11 +0100
Marc Zyngier <maz@kernel.org> wrote:

> On Mon, 02 May 2022 09:21:37 +0100,
> Marek Behún <kabel@kernel.org> wrote:
> > 
> > Dear Marc, Thomas,
> > 
> > we have encountered the following problem that can hopefully be put
> > some light onto: What is the intended way to set affinity (and possibly
> > other irq attributes) of parent IRQ of chained IRQs, when using the
> > irqdomain API?  
> 
> Simples: you can't. What sense does it make to change the affinity of
> the parent interrupt, given that its fate is tied to *all* of the
> other interrupts that are muxed to it?

Dear Marc,

thank you for your answer. Still:

What about when we want to set the same affinity for all the chained
interrupts?

Example: on Armada 385 there are 4 PCIe controllers. Each controller
has one interrupt from which we trigger chained interrupts. We would
like to configure each controller to trigger interrupt (and thus all
chained interrupts in the domain) on different CPU core.

Moreover we would really like to do this in runtime, through sysfs,
depending on for example whether there are cards plugged in the PCIe
ports.

Maybe there should be some mechanism to allow to change affinity for
whole irqdomain, or something?

Marek

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: irqdomain API: how to set affinity of parent irq of chained irqs?
  2022-05-02 15:45   ` Marek Behún
@ 2022-05-03  9:32     ` Marc Zyngier
  0 siblings, 0 replies; 4+ messages in thread
From: Marc Zyngier @ 2022-05-03  9:32 UTC (permalink / raw)
  To: Marek Behún; +Cc: linux-kernel, Thomas Gleixner, pali

On Mon, 02 May 2022 16:45:59 +0100,
Marek Behún <kabel@kernel.org> wrote:
> 
> On Mon, 02 May 2022 10:31:11 +0100
> Marc Zyngier <maz@kernel.org> wrote:
> 
> > On Mon, 02 May 2022 09:21:37 +0100,
> > Marek Behún <kabel@kernel.org> wrote:
> > > 
> > > Dear Marc, Thomas,
> > > 
> > > we have encountered the following problem that can hopefully be put
> > > some light onto: What is the intended way to set affinity (and possibly
> > > other irq attributes) of parent IRQ of chained IRQs, when using the
> > > irqdomain API?  
> > 
> > Simples: you can't. What sense does it make to change the affinity of
> > the parent interrupt, given that its fate is tied to *all* of the
> > other interrupts that are muxed to it?
> 
> Dear Marc,
> 
> thank you for your answer. Still:
> 
> What about when we want to set the same affinity for all the chained
> interrupts?
> 
> Example: on Armada 385 there are 4 PCIe controllers. Each controller
> has one interrupt from which we trigger chained interrupts. We would
> like to configure each controller to trigger interrupt (and thus all
> chained interrupts in the domain) on different CPU core.
> 
> Moreover we would really like to do this in runtime, through sysfs,
> depending on for example whether there are cards plugged in the PCIe
> ports.
> 
> Maybe there should be some mechanism to allow to change affinity for
> whole irqdomain, or something?

Should? Maybe. But not for an irqdomain (which really doesn't have
anything to do with interrupt affinity).

What you may want is a new sysfs interface that would allow a parent
interrupt affinity being changed, but also exposing to userspace all
the interrupts this affects *at the same time*. something like:

/sys/kernel/irq/42/smp_affinity_list
/sys/kernel/irq/42/muxed_irqs/
/sys/kernel/irq/42/muxed_irqs/56 -> ../../56
/sys/kernel/irq/42/muxed_irqs/57 -> ../../57

The main issues are that:

- we don't really track the muxing information in any of the data
  structures, so you can't just walk a short list and generate this
  information. You'd need to build the topology information at
  allocation time (or fish it out at runtime, but that's likely a
  pain).

- sysfs doesn't deal with affinities at all. procfs does, but adding
  more crap there is frowned upon.

- it *must* be a new interface. You can't repurpose the existing one,
  as something like irqbalance would be otherwise be massively
  confused by seeing interrupts moving around behind its back.

- conversely, you'll need to teach irqbalance how to deal with this
  new interface.

- this needs to be safe against CPU hotplug. It probably already is,
  but nobody ever tested it, given that userspace can't interact with
  these interrupts at the moment.

	M.

-- 
Without deviation from the norm, progress is not possible.

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2022-05-03  9:33 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-02  8:21 irqdomain API: how to set affinity of parent irq of chained irqs? Marek Behún
2022-05-02  9:31 ` Marc Zyngier
2022-05-02 15:45   ` Marek Behún
2022-05-03  9:32     ` Marc Zyngier

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.