From: Robin Murphy <robin.murphy@arm.com>
To: "Christian König" <ckoenig.leichtzumerken@gmail.com>,
"Alex Deucher" <alexdeucher@gmail.com>,
"Peter Geis" <pgwipeout@gmail.com>
Cc: "Deucher, Alexander" <alexander.deucher@amd.com>,
Christian Koenig <christian.koenig@amd.com>,
amd-gfx list <amd-gfx@lists.freedesktop.org>
Subject: Re: radeon ring 0 test failed on arm64
Date: Wed, 26 May 2021 11:59:39 +0100 [thread overview]
Message-ID: <546bf682-565f-8384-ec80-201ce1c747f4@arm.com> (raw)
In-Reply-To: <599edb94-8294-c4c5-ff7f-84c7072af3dd@gmail.com>
On 2021-05-26 10:42, Christian König wrote:
> Hi Robin,
>
> Am 25.05.21 um 22:09 schrieb Robin Murphy:
>> On 2021-05-25 14:05, Alex Deucher wrote:
>>> On Tue, May 25, 2021 at 8:56 AM Peter Geis <pgwipeout@gmail.com> wrote:
>>>>
>>>> On Tue, May 25, 2021 at 8:47 AM Alex Deucher <alexdeucher@gmail.com>
>>>> wrote:
>>>>>
>>>>> On Tue, May 25, 2021 at 8:42 AM Peter Geis <pgwipeout@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Good Evening,
>>>>>>
>>>>>> I am stress testing the pcie controller on the rk3566-quartz64
>>>>>> prototype SBC.
>>>>>> This device has 1GB available at <0x3 0x00000000> for the PCIe
>>>>>> controller, which makes a dGPU theoretically possible.
>>>>>> While attempting to light off a HD7570 card I manage to get a modeset
>>>>>> console, but ring0 test fails and disables acceleration.
>>>>>>
>>>>>> Note, we do not have UEFI, so all PCIe setup is from the Linux
>>>>>> kernel.
>>>>>> Any insight you can provide would be much appreciated.
>>>>>
>>>>> Does your platform support PCIe cache coherency with the CPU? I.e.,
>>>>> does the CPU allow cache snoops from PCIe devices? That is required
>>>>> for the driver to operate.
>>>>
>>>> Ah, most likely not.
>>>> This issue has come up already as the GIC isn't permitted to snoop on
>>>> the CPUs, so I doubt the PCIe controller can either.
>>>>
>>>> Is there no way to work around this or is it dead in the water?
>>>
>>> It's required by the pcie spec. You could potentially work around it
>>> if you can allocate uncached memory for DMA, but I don't think that is
>>> possible currently. Ideally we'd figure out some way to detect if a
>>> particular platform supports cache snooping or not as well.
>>
>> There's device_get_dma_attr(), although I don't think it will work
>> currently for PCI devices without an OF or ACPI node - we could
>> perhaps do with a PCI-specific wrapper which can walk up and defer to
>> the host bridge's firmware description as necessary.
>>
>> The common DMA ops *do* correctly keep track of per-device coherency
>> internally, but drivers aren't supposed to be poking at that
>> information directly.
>
> That sounds like you underestimate the problem. ARM has unfortunately
> made the coherency for PCI an optional IP.
Sorry to be that guy, but I'm involved a lot internally with our system
IP and interconnect, and I probably understand the situation better than
99% of the community ;)
For the record, the SBSA specification (the closet thing we have to a
"system architecture") does require that PCIe is integrated in an
I/O-coherent manner, but we don't have any control over what people do
in embedded applications (note that we don't make PCIe IP at all, and
there is plenty of 3rd-party interconnect IP).
> So we are talking about a hardware limitation which potentially can't be
> fixed without replacing the hardware.
You expressed interest in "some way to detect if a particular platform
supports cache snooping or not", by which I assumed you meant a software
method for the amdgpu/radeon drivers to call, rather than, say, a
website that driver maintainers can look up SoC names on. I'm saying
that that API already exists (just may need a bit more work). Note that
it is emphatically not a platform-level thing since coherency can and
does vary per device within a system.
I wasn't suggesting that Linux could somehow make coherency magically
work when the signals don't physically exist in the interconnect - I was
assuming you'd merely want to do something like throw a big warning and
taint the kernel to help triage bug reports. Some drivers like
ahci_qoriq and panfrost simply need to know so they can program their
device to emit the appropriate memory attributes either way, and rely on
the DMA API to hide the rest of the difference, but if you want to treat
non-coherent use as unsupported because it would require too invasive
changes that's fine by me.
Robin.
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
next prev parent reply other threads:[~2021-05-26 12:57 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-25 2:34 radeon ring 0 test failed on arm64 Peter Geis
2021-05-25 12:46 ` Alex Deucher
2021-05-25 12:55 ` Peter Geis
2021-05-25 13:05 ` Alex Deucher
2021-05-25 13:18 ` Peter Geis
2021-05-25 20:09 ` Robin Murphy
2021-05-26 9:42 ` Christian König
2021-05-26 10:59 ` Robin Murphy [this message]
2021-05-26 11:21 ` Christian König
2022-03-17 0:14 ` Peter Geis
2022-03-17 0:14 ` Peter Geis
2022-03-17 3:07 ` Kever Yang
2022-03-17 3:07 ` Kever Yang
2022-03-17 12:19 ` Peter Geis
2022-03-17 12:19 ` Peter Geis
2022-03-18 7:51 ` Kever Yang
2022-03-18 7:51 ` Kever Yang
2022-03-18 8:35 ` Christian König
2022-03-18 11:24 ` Peter Geis
2022-03-18 11:24 ` Peter Geis
2022-03-18 12:31 ` Christian König
2022-03-18 12:31 ` Christian König
2022-03-18 12:45 ` Peter Geis
2022-03-18 12:45 ` Peter Geis
2022-03-17 9:14 ` Christian König
2022-03-17 9:14 ` Christian König
2022-03-17 12:21 ` Peter Geis
2022-03-17 12:21 ` Peter Geis
2022-03-17 20:27 ` Alex Deucher
2022-03-17 20:27 ` Alex Deucher
2022-03-17 10:37 ` Robin Murphy
2022-03-17 10:37 ` Robin Murphy
2022-03-17 12:26 ` Peter Geis
2022-03-17 12:26 ` Peter Geis
2022-03-17 12:51 ` Christian König
2022-03-17 12:51 ` Christian König
2022-03-17 13:17 ` Robin Murphy
2022-03-17 13:17 ` Robin Murphy
2022-03-17 14:21 ` Peter Geis
2022-03-17 14:21 ` Peter Geis
2022-03-23 21:06 ` Alex Deucher
2022-03-23 21:06 ` Alex Deucher
2021-05-25 14:08 ` Christian König
2021-05-25 14:19 ` Peter Geis
2021-05-25 15:09 ` Christian König
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=546bf682-565f-8384-ec80-201ce1c747f4@arm.com \
--to=robin.murphy@arm.com \
--cc=alexander.deucher@amd.com \
--cc=alexdeucher@gmail.com \
--cc=amd-gfx@lists.freedesktop.org \
--cc=christian.koenig@amd.com \
--cc=ckoenig.leichtzumerken@gmail.com \
--cc=pgwipeout@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.