linux-rockchip.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
From: Peter Geis <pgwipeout@gmail.com>
To: Thomas Gleixner <tglx@linutronix.de>, Marc Zyngier <maz@kernel.org>
Cc: "open list:ARM/Rockchip SoC..."
	<linux-rockchip@lists.infradead.org>,
	linux-kernel@vger.kernel.org
Subject: [RFC] ITS fails to allocate on rk3568/rk3566
Date: Mon, 12 Apr 2021 16:49:59 -0400	[thread overview]
Message-ID: <CAMdYzYrQ5f-mv_VmTq_CRf9tR=j3mwRpKHNLmPFgCF9whsGFRw@mail.gmail.com> (raw)

Good Afternoon,

I am assisting with early bringup of the rk3566 based quartz64
development board for mainline linux.
I've encountered a few issues with allocating ITS on their version of
the GIC-V3.
The first issue is the ITS controller can only use 32bit addresses.
This leads to the following error:
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] GICv3: 320 SPIs implemented
[    0.000000] GICv3: 0 Extended SPIs implemented
[    0.000000] GICv3: Distributor has no Range Selector support
[    0.000000] Root IRQ handler: gic_handle_irq
[    0.000000] GICv3: 16 PPIs implemented
[    0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000fd460000
[    0.000000] ITS [mem 0xfd440000-0xfd45ffff]
[    0.000000] ITS@0x00000000fd440000: Devices doesn't stick:
f907000100190600 f907000000190600
[    0.000000] ITS@0x00000000fd440000: failed probing (-6)
[    0.000000] ITS: No ITS available, not enabling LPIs

Downstream fixed this by adding the GFP_DMA32 flag to the memory allocations.
They also force clear the GITS_BASER_SHAREABILITY_MASK.
Unfortunately while this allowed ITS to allocate on downstream, as
soon as MSIs attempted to use it all interrupts would time out.

On upstream, we observe this during allocation:
[    0.000000] ITS [mem 0xfd440000-0xfd45ffff]
[    0.000000] ITS@0x00000000fd440000: allocated 8192 Devices @3810000
(indirect, esz 8, psz 64K, shr 1)
[    0.000000] ITS@0x00000000fd440000: allocated 32768 Interrupt
Collections @3820000 (flat, esz 2, psz 64K, shr 1)
[    0.000000] GICv3: using LPI property table @0x0000000003830000
[    0.000000] GICv3: CPU0: using allocated LPI pending table
@0x0000000003840000
[    0.000000] ITS queue timeout (64 1)
[    0.000000] ITS cmd its_build_mapc_cmd failed
[    0.000000] ITS queue timeout (96 1)
[    0.000000] ITS cmd its_build_invall_cmd failed
<snip>
[    0.150282] Platform MSI: interrupt-controller@fd440000 domain created
[    0.152409] PCI/MSI:
/interrupt-controller@fd400000/interrupt-controller@fd440000 domain
created
[    0.156158] EFI services will not be available.
[    0.161168] smp: Bringing up secondary CPUs ...
[    0.170252] Detected VIPT I-cache on CPU1
[    0.170325] GICv3: CPU1: found redistributor 100 region 0:0x00000000fd480000
[    0.170356] GICv3: GIC_READ_PMR old_pmr=0xf0
[    0.170371] GICv3: GIC_READ_PMR new_pmr=0x0
[    0.170393] GICv3: CPU1: using allocated LPI pending table
@0x0000000003850000
[    1.686709] ITS queue timeout (160 1)
[    1.686733] ITS cmd its_build_mapc_cmd failed
[    3.202979] ITS queue timeout (192 1)
[    3.202999] ITS cmd its_build_invall_cmd failed
[    3.203157] CPU1: Booted secondary processor 0x0000000100 [0x412fd050]
[    3.208303] Callback from call_rcu_tasks_rude() invoked.
[    3.228487] Detected VIPT I-cache on CPU2
[    3.228559] GICv3: CPU2: found redistributor 200 region 0:0x00000000fd4a0000
[    3.228590] GICv3: GIC_READ_PMR old_pmr=0xf0
[    3.228606] GICv3: GIC_READ_PMR new_pmr=0x0
[    3.228630] GICv3: CPU2: using allocated LPI pending table
@0x0000000003860000
[    4.744675] ITS queue timeout (256 1)
[    4.744698] ITS cmd its_build_mapc_cmd failed
[    6.260797] ITS queue timeout (288 1)
[    6.260816] ITS cmd its_build_invall_cmd failed
[    6.260948] CPU2: Booted secondary processor 0x0000000200 [0x412fd050]
[    6.269204] Callback from call_rcu_tasks() invoked.
[    6.278358] Detected VIPT I-cache on CPU3
[    6.278430] GICv3: CPU3: found redistributor 300 region 0:0x00000000fd4c0000
[    6.278460] GICv3: GIC_READ_PMR old_pmr=0xf0
[    6.278476] GICv3: GIC_READ_PMR new_pmr=0x0
[    6.278501] GICv3: CPU3: using allocated LPI pending table
@0x0000000003870000
[    7.794367] ITS queue timeout (352 1)
[    7.794390] ITS cmd its_build_mapc_cmd failed
[    9.310219] ITS queue timeout (384 1)
[    9.310237] ITS cmd its_build_invall_cmd failed
[    9.310368] CPU3: Booted secondary processor 0x0000000300 [0x412fd050]
[    9.311925] smp: Brought up 1 node, 4 CPUs
[    9.318831] SMP: Total of 4 processors activated.

Also, when MSIs activate the same timeouts seen downstream happen:
[   13.884698] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[   13.886476] pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22
[   13.887466] pcieport 0000:00:00.0: assign IRQ: got 0
[   15.414727] ITS queue timeout (416 1)
[   15.415476] ITS cmd its_build_mapd_cmd failed
[   16.957519] ITS queue timeout (480 1)
[   16.958040] ITS cmd its_build_mapti_cmd failed
[   18.482251] ITS queue timeout (544 1)
[   18.482762] ITS cmd its_build_mapti_cmd failed
<snip>
[   70.267783] ITS queue timeout (2656 1)
[   70.268363] ITS cmd its_build_inv_cmd failed
[   70.269782] pcieport 0000:00:00.0: PME: Signaling with IRQ 64

Any assistance you can provide would be greatly appreciated.

Very Respectfully,
Peter Geis

_______________________________________________
Linux-rockchip mailing list
Linux-rockchip@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-rockchip

             reply	other threads:[~2021-04-12 20:50 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-12 20:49 Peter Geis [this message]
2021-04-13  9:23 ` Marc Zyngier
     [not found]   ` <CAMdYzYruPyiT89FrbJhuV=c36PyRwZ7sT45abnv8rTv85AKRow@mail.gmail.com>
     [not found]     ` <87y2dmmggt.wl-maz@kernel.org>
2021-04-13 15:03       ` Peter Geis
2021-04-13 15:51         ` Marc Zyngier
2021-04-14 11:41           ` Peter Geis
2021-04-14 12:42             ` Marc Zyngier
2021-04-15  7:24               ` Kever Yang
2021-04-15  8:11                 ` Marc Zyngier
2021-04-16  1:13                   ` Kever Yang
2021-04-16 15:23                     ` Marc Zyngier
2021-04-21  1:40                       ` Kever Yang
2021-04-21 10:23                         ` Marc Zyngier

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAMdYzYrQ5f-mv_VmTq_CRf9tR=j3mwRpKHNLmPFgCF9whsGFRw@mail.gmail.com' \
    --to=pgwipeout@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rockchip@lists.infradead.org \
    --cc=maz@kernel.org \
    --cc=tglx@linutronix.de \
    --subject='Re: [RFC] ITS fails to allocate on rk3568/rk3566' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).