Re: [RFC PATCH v2] arm64: cpufeatures: add support for tlbi range instructions

[Hanjun Guo <guohanjun@huawei.com>]

Hi Marc,

On 2019/11/19 18:03, Marc Zyngier wrote:
> Hi Hanjun,
> 
> On 2019-11-19 01:13, Hanjun Guo wrote:
>> +Cc linux-arm-kernel mailing list and Shaokun.
>>
>> Hi Marc,
>>
>> On 2019/11/11 22:04, Marc Zyngier wrote:
>>> On 2019-11-11 14:56, Zhenyu Ye wrote:
>>>> On 2019/11/11 21:27, Will Deacon wrote:
>>>>> On Mon, Nov 11, 2019 at 09:23:55PM +0800, Zhenyu Ye wrote:
>> [...]
>>>>>
>>>>> How does this address my concerns here:
>>>>> https://lore.kernel.org/linux-arm-kernel/20191031131649.GB27196@willie-the-truck/
>>>>>
>>>>> ?
>>>>>
>>>>> Will
>>>>
>>>> I think your concern is more about the hardware level, and we can do
>>>> nothing about
>>>> this at all. The interconnect/DVM implementation is not exposed to
>>>> software layer
>>>> (and no need), and may should be constrained at hardware level.
>>>
>>> You're missing the point here: the instruction may be implemented
>>> and perfectly working at the CPU level, and yet not carried over
>>> the interconnect. In this situation, other CPUs may not observe
>>> the DVM messages instructing them of such invalidation, and you'll end
>>> up with memory corruption.
>>>
>>> So, in the absence of an architectural guarantee that range invalidation
>>> is supported and observed by all the DVM agents in the system, there must
>>> be a firmware description for it on which the kernel can rely.
>>
>> I'm thinking of how to add a firmware description for it, how about this:
>>
>> Adding a system level flag to indicate the supporting of TIBi by range,
>> which means adding a binding name for example "tlbi-by-range" at system
>> level in the dts file, or a tlbi by range flag in ACPI FADT table, then
>> we use the ID register per-cpu and the system level flag as
>>
>> if (cpus_have_const_cap(ARM64_HAS_TLBI_BY_RANGE) &&
>> system_level_tlbi_by_range)
>>     flush_tlb_by_range()
>> else
>>     flush_tlb_range()
>>
>> And this seems work for heterogeneous system (olny parts of the CPU support
>> TLBi by range) as well, correct me if anything wrong.
> 
> It could work, but it needs to come with the strongest guarantees that
> all the DVM agents in the system understand this type of invalidation,
> specially as we move into the SVM territory. It may also need to cope
> with non-compliant agents being hot-plugged, or at least discovered late.

Totally agreed, we are working on this in the system level including SMMU.

> 
> I also wonder if the ARMv8.4-TTL extension (which I have patches for in
> the nested virt series) requires the same kind of treatment (after all,
> it has an implicit range based on the base granule size and level).
> 
> In any way, this requires careful specification, and I don't think
> we can improvise this on the ML... ;-)

Sure :), the good news is that ARM officially announced will be
working with Huawei again.

So if I understand your point correctly, we need steps to take:
 - ARM spec needs to make TIBi by range crystal clear and being
   written down in the spec;
 - Firmware description of supporting TLBi by range in system level
   for both FDT and ACPI;
 - Then upstream the code.

Thanks
Hanjun