Re: [PATCH] PCI: dwc: Warn only for non-prefetchable memory resource size >4GB

From: Vidya Sagar <vidyas@nvidia.com>
To: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <helgaas@kernel.org>, <jingoohan1@gmail.com>,
	<gustavo.pimentel@synopsys.com>,
	Andrew Murray <amurray@thegoodpenguin.co.uk>,
	<bhelgaas@google.com>, <thierry.reding@gmail.com>,
	<jonathanh@nvidia.com>, <linux-pci@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <kthota@nvidia.com>,
	<mmaddireddy@nvidia.com>, <sagar.tv@gmail.com>,
	"Alan Mikhak" <alan.mikhak@sifive.com>
Subject: Re: [PATCH] PCI: dwc: Warn only for non-prefetchable memory resource size >4GB
Date: Tue, 19 May 2020 22:38:39 +0530	[thread overview]
Message-ID: <59c32bed-3a6a-70ba-0052-65d9466a0790@nvidia.com> (raw)
In-Reply-To: <20200519145816.GB21261@e121166-lin.cambridge.arm.com>


On 19-May-20 8:28 PM, Lorenzo Pieralisi wrote:
> External email: Use caution opening links or attachments
> 
> 
> On Tue, May 19, 2020 at 07:25:02PM +0530, Vidya Sagar wrote:
>>
>>
>> On 18-May-20 9:24 PM, Lorenzo Pieralisi wrote:
>>> External email: Use caution opening links or attachments
>>>
>>>
>>> On Wed, May 13, 2020 at 05:35:08PM -0500, Bjorn Helgaas wrote:
>>>> [+cc Alan; please cc authors of relevant commits,
>>>> updated Andrew's email address]
>>>>
>>>> On Thu, May 14, 2020 at 12:38:55AM +0530, Vidya Sagar wrote:
>>>>> commit 9e73fa02aa009 ("PCI: dwc: Warn if MEM resource size exceeds max for
>>>>> 32-bits") enables warning for MEM resources of size >4GB but prefetchable
>>>>>    memory resources also come under this category where sizes can go beyond
>>>>> 4GB. Avoid logging a warning for prefetchable memory resources.
>>>>>
>>>>> Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
>>>>> ---
>>>>>    drivers/pci/controller/dwc/pcie-designware-host.c | 3 ++-
>>>>>    1 file changed, 2 insertions(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
>>>>> index 42fbfe2a1b8f..a29396529ea4 100644
>>>>> --- a/drivers/pci/controller/dwc/pcie-designware-host.c
>>>>> +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
>>>>> @@ -366,7 +366,8 @@ int dw_pcie_host_init(struct pcie_port *pp)
>>>>>                       pp->mem = win->res;
>>>>>                       pp->mem->name = "MEM";
>>>>>                       mem_size = resource_size(pp->mem);
>>>>> -                   if (upper_32_bits(mem_size))
>>>>> +                   if (upper_32_bits(mem_size) &&
>>>>> +                       !(win->res->flags & IORESOURCE_PREFETCH))
>>>>>                               dev_warn(dev, "MEM resource size exceeds max for 32 bits\n");
>>>>>                       pp->mem_size = mem_size;
>>>>>                       pp->mem_bus_addr = pp->mem->start - win->offset;
>>>
>>> That warning was added for a reason - why should not we log legitimate
>>> warnings ? AFAIU having resources larger than 4GB can lead to undefined
>>> behaviour given the current ATU programming API.
>> Yeah. I'm all for a warning if the size is larger than 4GB in case of
>> non-prefetchable window as one of the ATU outbound translation
>> channels is being used,
> 
> Is it true for all DWC host controllers ? Or there may be another
> exception whereby we would be forced to disable this warning altogether
> ?I think so. As I see from the code, ATU's
Region-0 is used for config space translation
Region-1 is used for non-prefetchable memory translation
Region-2 is used for I/O translation
So, there is no region reserved for translating prefetchable memory regions.

> 
>> but, we are not employing any ATU outbound translation channel for
> 
> What does this mean ? "we are not employing any ATU outbound...", is
> this the tegra driver ? And what guarantees that this warning is not
> legitimate on DWC host controllers that do use the ATU outbound
> translation for prefetchable windows ?
Not Tegra driver but Tegra HW. Tegra HW doesn't need any ATU outbound 
translation for prefetchable (for that matter any 1-to-1 mapping to 
generate memory transactions on the PCIe bus).
The Warning is still valid for both Tegra and other DWC based 
controllers for non-prefetchable memory translation.

> 
> Can DWC maintainers chime in and clarify please ?
> 
>> prefetchable window and they can be greater than 4GB in size for all
>> right reasons. So, logging a warning for prefetchable region doesn't
>> seem correct to me. Please let me know if my understanding is wrong.
> 
> I think your patch is wrong and it is applied on top of a patch that
> is wrong too, so I won't apply yours and it is likely I will revert
> Alan's because it seems to solve nothing (and warn spuriously).
> 
> It is time for people who maintain DWC please to speak up because I
> don't have the HW details required to make a judgment.
> 
> Lorenzo
> 
>> - Vidya Sagar
>>>
>>> Alan ? I want to understand what's the best course of action before
>>> merging these patches.
>>>
>>> Lorenzo
>>>