From: Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
To: isaacm@codeaurora.org
Cc: Will Deacon <will@kernel.org>, Rob Clark <robdclark@gmail.com>,
Jordan Crouse <jcrouse@codeaurora.org>,
linux-arm-msm@vger.kernel.org, Joerg Roedel <joro@8bytes.org>,
linux-kernel@vger.kernel.org,
Akhil P Oommen <akhilpo@codeaurora.org>,
iommu@lists.linux-foundation.org,
Robin Murphy <robin.murphy@arm.com>,
linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH] iommu/io-pgtable-arm: Allow non-coherent masters to use system cache
Date: Mon, 11 Jan 2021 10:08:08 +0530 [thread overview]
Message-ID: <73b1957d0898a937e5e88c1a469352ea@codeaurora.org> (raw)
In-Reply-To: <84ff10c38e99635bc222ca2dd29be2b5@codeaurora.org>
On 2021-01-08 23:39, isaacm@codeaurora.org wrote:
> On 2021-01-07 21:47, Sai Prakash Ranjan wrote:
>> On 2021-01-07 22:27, isaacm@codeaurora.org wrote:
>>> On 2021-01-06 03:56, Will Deacon wrote:
>>>> On Thu, Dec 24, 2020 at 12:10:07PM +0530, Sai Prakash Ranjan wrote:
>>>>> commit ecd7274fb4cd ("iommu: Remove unused IOMMU_SYS_CACHE_ONLY
>>>>> flag")
>>>>> removed unused IOMMU_SYS_CACHE_ONLY prot flag and along with it
>>>>> went
>>>>> the memory type setting required for the non-coherent masters to
>>>>> use
>>>>> system cache. Now that system cache support for GPU is added, we
>>>>> will
>>>>> need to mark the memory as normal sys-cached for GPU to use system
>>>>> cache.
>>>>> Without this, the system cache lines are not allocated for GPU. We
>>>>> use
>>>>> the IO_PGTABLE_QUIRK_ARM_OUTER_WBWA quirk instead of a page
>>>>> protection
>>>>> flag as the flag cannot be exposed via DMA api because of no
>>>>> in-tree
>>>>> users.
>>>>>
>>>>> Signed-off-by: Sai Prakash Ranjan
>>>>> <saiprakash.ranjan@codeaurora.org>
>>>>> ---
>>>>> drivers/iommu/io-pgtable-arm.c | 3 +++
>>>>> 1 file changed, 3 insertions(+)
>>>>>
>>>>> diff --git a/drivers/iommu/io-pgtable-arm.c
>>>>> b/drivers/iommu/io-pgtable-arm.c
>>>>> index 7c9ea9d7874a..3fb7de8304a2 100644
>>>>> --- a/drivers/iommu/io-pgtable-arm.c
>>>>> +++ b/drivers/iommu/io-pgtable-arm.c
>>>>> @@ -415,6 +415,9 @@ static arm_lpae_iopte
>>>>> arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data,
>>>>> else if (prot & IOMMU_CACHE)
>>>>> pte |= (ARM_LPAE_MAIR_ATTR_IDX_CACHE
>>>>> << ARM_LPAE_PTE_ATTRINDX_SHIFT);
>>>>> + else if (data->iop.cfg.quirks & IO_PGTABLE_QUIRK_ARM_OUTER_WBWA)
>>>>> + pte |= (ARM_LPAE_MAIR_ATTR_IDX_INC_OCACHE
>>>>> + << ARM_LPAE_PTE_ATTRINDX_SHIFT);
>>>>> }
>>>>
>>> While this approach of enabling system cache globally for both page
>>> tables and other buffers
>>> works for the GPU usecase, this isn't ideal for other clients that
>>> use
>>> system cache. For example,
>>> video clients only want to cache a subset of their buffers in the
>>> system cache, due to the sizing constraint
>>> imposed by how much of the system cache they can use. So, it would be
>>> ideal to have
>>> a way of expressing the desire to use the system cache on a
>>> per-buffer
>>> basis. Additionally,
>>> our video clients use the DMA layer, and since the requirement is for
>>> caching in the system cache
>>> to be a per buffer attribute, it seems like we would have to have a
>>> DMA attribute to express
>>> this on a per-buffer basis.
>>>
>>
>> I did bring this up initially [1], also where is this video client
>> in upstream? AFAIK, only system cache user in upstream is GPU.
>> We cannot add any DMA attribute unless there is any user upstream
> Right, there wouldn't be an upstream user, which would be problematic,
> but I was thinking of having it so that when video or any of our other
> clients that use this attribute on a per buffer basis upstreams their
> code, it's not too much of a stretch to add the support.
Agreed.
>> as per [2], so when the support for such a client is added, wouldn't
>> ((data->iop.cfg.quirks & IO_PGTABLE_QUIRK_ARM_OUTER_WBWA) ||
>> PROT_FLAG)
>> work?
> I don't think that will work, because we currently have clients who use
> the
> system cache as follows:
> -cache only page tables in the system cache
> -cache only data buffers in the system cache
> -cache both page tables and all buffers in the system cache
> -cache both page tables and some buffers in the system cache
>
> The approach you're suggesting doesn't allow for the last case, as
> caching the
> page tables in the system cache involves setting
> IO_PGTABLE_QUIRK_ARM_OUTER_WBWA,
> so we will end up losing the flexibility to cache some data buffers in
> the system cache.
>
Ah yes, you are right, I believe Jordan mentioned the same [1].
[1]
https://lore.kernel.org/lkml/20200709161352.GC21059@jcrouse1-lnx.qualcomm.com/
> Ideally, the page table quirk would drive the settings for the TCR,
> and the prot flag
> drives the PTE for the mapping, as is done with the page table walker
> being dma-coherent,
> while buffers are mapped as cacheable based on IOMMU_CACHE. Thoughts?
>
Right, mixing the two is not correct. Will's suggestion for a new prot
flag sounds good to me, I will work on that.
Thanks,
Sai
--
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a
member
of Code Aurora Forum, hosted by The Linux Foundation
next prev parent reply other threads:[~2021-01-11 4:39 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-24 6:40 [PATCH] iommu/io-pgtable-arm: Allow non-coherent masters to use system cache Sai Prakash Ranjan
2021-01-06 11:56 ` Will Deacon
2021-01-07 6:35 ` Sai Prakash Ranjan
2021-01-07 16:57 ` isaacm
2021-01-08 5:47 ` Sai Prakash Ranjan
2021-01-08 18:09 ` isaacm
2021-01-11 4:38 ` Sai Prakash Ranjan [this message]
2021-01-08 18:18 ` Will Deacon
2021-01-08 19:50 ` isaacm
2021-01-11 4:56 ` Sai Prakash Ranjan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=73b1957d0898a937e5e88c1a469352ea@codeaurora.org \
--to=saiprakash.ranjan@codeaurora.org \
--cc=akhilpo@codeaurora.org \
--cc=iommu@lists.linux-foundation.org \
--cc=isaacm@codeaurora.org \
--cc=jcrouse@codeaurora.org \
--cc=joro@8bytes.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=robdclark@gmail.com \
--cc=robin.murphy@arm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).