Re: [PATCH 2/6] cacheinfo: Set cache 'id' based on DT data

[Jeremy Linton <jeremy.linton@arm.com>]

Hi,

On 12/17/21 13:26, Rob Herring wrote:
> On Fri, Dec 17, 2021 at 1:03 PM Sudeep Holla <sudeep.holla@arm.com> wrote:
>>
>> On Fri, Dec 17, 2021 at 12:14:22PM -0600, Rob Herring wrote:
>>> On Fri, Dec 17, 2021 at 10:57 AM Robin Murphy <robin.murphy@arm.com> wrote:
>>>>
>>>> Hi Rob,
>>>>
>>>> On 2021-12-16 23:31, Rob Herring wrote:
>>>>> Use the minimum CPU h/w id of the CPUs associated with the cache for the
>>>>> cache 'id'. This will provide a stable id value for a given system. As
>>
>> I am trying to follow the code. IIUC, the level one(I$ and D$) are skipped
>> in this logic and the private unified cache if any will get the cpu hwid as
>> the cache id which is all fine. But what happens if there are 2 levels of
>> unified private cache ? I am assuming we only care about shared caches for
>> MPAM and ignore private caches which sounds OK but I just wanted to confirm.
> 
> The cacheinfo 'id' is only unique to the level and type. It's the
> type, level, and ID that gives a unique identifier:
> 
>   * struct cacheinfo - represent a cache leaf node
>   * @id: This cache's id. It is unique among caches with the same (type, level).
> 
> Maybe ACPI's ID expects/allows globally unique cache IDs?

Yes, but the spec is IMHO written in a way that they may only be unique 
for a subset of the caches! The rest might not have an ID at all, 
particularly for !arm machines.


> 
>>>>> we need to check all possible CPUs, we can't use the shared_cpu_map
>>>>> which is just online CPUs. There's not a cache to CPUs mapping in DT, so
>>>>> we have to walk all CPU nodes and then walk cache levels.
>>
>> I would have preferred to add the cache IDs in DT similar to ACPI but I see
>> you have certain concerns with that which are valid as well.
>>
>>>>
>>>> I believe another expected use of the cache ID exposed in sysfs is to
>>>> program steering tags for cache stashing (typically in VFIO-based
>>>> userspace drivers like DPDK so we can't realistically mediate it any
>>>> other way). There were plans afoot last year to ensure that ACPI PPTT
>>>> could provide the necessary ID values for arm64 systems which will
>>>> typically be fairly arbitrary (but unique) due to reflecting underlying
>>>> interconnect routing IDs. Assuming that there will eventually be some
>>>> interest in cache stashing on DT-based systems too, we probably want to
>>>> allow for an explicit ID property on DT cache nodes in a similar manner.
>>>
>>> If you have a suggestion for ID values that correspond to the h/w,
>>> then we can add them. I'd like a bit more than just trusting that ID
>>> is something real.
>>>
>>
>> I agree, probably architecture must do better job at defining these. But
>> generated IDs IMO might cause issues especial if we have to change the
>> logic without breaking the backward compatibility.
>>
>>> While the ACPI folks may be willing to take an arbitrary index, it's
>>> something we (mostly) avoid for DT.
>>>
>>
>> Not sure if we can call that *arbitrary* 😄, in that case we can imagine
>> the same at several places in the firmware.
> 
> By arbitrary, I mean made up by the binding/dts author or
> documentation convention (UART0, UART1, etc.). Certainly things like
> clock IDs are often made up number spaces, but I don't see how we
> avoid that. DT had 'cell-index' which I still see attempted. But that
> property traces back to h/w having a single power ctrl register and
> cell-index was the bit index for the register. If only h/w was still
> that simple.
> 
> Rob
>