All of lore.kernel.org
 help / color / mirror / Atom feed
From: Krishna Reddy <vdumpa@nvidia.com>
To: Robin Murphy <robin.murphy@arm.com>,
	Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
Cc: "linux-arm-msm@vger.kernel.org" <linux-arm-msm@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>, Will Deacon <will@kernel.org>,
	"linux-arm-kernel@lists.infradead.org" 
	<linux-arm-kernel@lists.infradead.org>,
	Thierry Reding <treding@nvidia.com>
Subject: RE: [PATCH] iommu/io-pgtable-arm: Optimize partial walk flush for large scatter-gather list
Date: Fri, 11 Jun 2021 00:37:24 +0000	[thread overview]
Message-ID: <BY5PR12MB37646698F37C00381EFF7C77B3349@BY5PR12MB3764.namprd12.prod.outlook.com> (raw)
In-Reply-To: <61c69d23-324a-85d7-2458-dfff8df9280b@arm.com>

> > No, the unmap latency is not just in some test case written, the issue
> > is very real and we have workloads where camera is reporting frame
> > drops because of this unmap latency in the order of 100s of milliseconds.
> > And hardware team recommends using ASID based invalidations for
> > anything larger than 128 TLB entries. So yes, we have taken note of
> > impacts here before going this way and hence feel more inclined to
> > make this qcom specific if required.

Seems like the real issue here is not the unmap API latency.
It should be the high number of back to back SMMU TLB invalidate register writes that is resulting
in lower ISO BW to Camera and overflow. Isn't it?
Even Tegra186 SoC has similar issue and HW team recommended to rate limit the number of
back to back SMMU tlb invalidate registers writes. The subsequent Tegra194 SoC has a dedicated SMMU for
ISO clients to avoid the impact of TLB invalidates from NISO clients on ISO BW.

>> Thinking some more, I
>> wonder if the Tegra folks might have an opinion to add here, given 
>> that their multiple-SMMU solution was seemingly about trying to get 
>> enough TLB and pagetable walk bandwidth in the first place?

While it is good to reduce the number of tlb register writes, Flushing all TLB entries at context granularity arbitrarily
can have negative impact on active traffic and BW. I don't have much data on possible impact at this point.
Can the flushing at context granularity be made a quirk than performing it as default? 

-KR


WARNING: multiple messages have this Message-ID (diff)
From: Krishna Reddy <vdumpa@nvidia.com>
To: Robin Murphy <robin.murphy@arm.com>,
	Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
Cc: "linux-arm-msm@vger.kernel.org" <linux-arm-msm@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>,
	Thierry Reding <treding@nvidia.com>,
	Will Deacon <will@kernel.org>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>
Subject: RE: [PATCH] iommu/io-pgtable-arm: Optimize partial walk flush for large scatter-gather list
Date: Fri, 11 Jun 2021 00:37:24 +0000	[thread overview]
Message-ID: <BY5PR12MB37646698F37C00381EFF7C77B3349@BY5PR12MB3764.namprd12.prod.outlook.com> (raw)
In-Reply-To: <61c69d23-324a-85d7-2458-dfff8df9280b@arm.com>

> > No, the unmap latency is not just in some test case written, the issue
> > is very real and we have workloads where camera is reporting frame
> > drops because of this unmap latency in the order of 100s of milliseconds.
> > And hardware team recommends using ASID based invalidations for
> > anything larger than 128 TLB entries. So yes, we have taken note of
> > impacts here before going this way and hence feel more inclined to
> > make this qcom specific if required.

Seems like the real issue here is not the unmap API latency.
It should be the high number of back to back SMMU TLB invalidate register writes that is resulting
in lower ISO BW to Camera and overflow. Isn't it?
Even Tegra186 SoC has similar issue and HW team recommended to rate limit the number of
back to back SMMU tlb invalidate registers writes. The subsequent Tegra194 SoC has a dedicated SMMU for
ISO clients to avoid the impact of TLB invalidates from NISO clients on ISO BW.

>> Thinking some more, I
>> wonder if the Tegra folks might have an opinion to add here, given 
>> that their multiple-SMMU solution was seemingly about trying to get 
>> enough TLB and pagetable walk bandwidth in the first place?

While it is good to reduce the number of tlb register writes, Flushing all TLB entries at context granularity arbitrarily
can have negative impact on active traffic and BW. I don't have much data on possible impact at this point.
Can the flushing at context granularity be made a quirk than performing it as default? 

-KR

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

WARNING: multiple messages have this Message-ID (diff)
From: Krishna Reddy <vdumpa@nvidia.com>
To: Robin Murphy <robin.murphy@arm.com>,
	Sai Prakash Ranjan <saiprakash.ranjan@codeaurora.org>
Cc: "linux-arm-msm@vger.kernel.org" <linux-arm-msm@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>, Will Deacon <will@kernel.org>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>,
	Thierry Reding <treding@nvidia.com>
Subject: RE: [PATCH] iommu/io-pgtable-arm: Optimize partial walk flush for large scatter-gather list
Date: Fri, 11 Jun 2021 00:37:24 +0000	[thread overview]
Message-ID: <BY5PR12MB37646698F37C00381EFF7C77B3349@BY5PR12MB3764.namprd12.prod.outlook.com> (raw)
In-Reply-To: <61c69d23-324a-85d7-2458-dfff8df9280b@arm.com>

> > No, the unmap latency is not just in some test case written, the issue
> > is very real and we have workloads where camera is reporting frame
> > drops because of this unmap latency in the order of 100s of milliseconds.
> > And hardware team recommends using ASID based invalidations for
> > anything larger than 128 TLB entries. So yes, we have taken note of
> > impacts here before going this way and hence feel more inclined to
> > make this qcom specific if required.

Seems like the real issue here is not the unmap API latency.
It should be the high number of back to back SMMU TLB invalidate register writes that is resulting
in lower ISO BW to Camera and overflow. Isn't it?
Even Tegra186 SoC has similar issue and HW team recommended to rate limit the number of
back to back SMMU tlb invalidate registers writes. The subsequent Tegra194 SoC has a dedicated SMMU for
ISO clients to avoid the impact of TLB invalidates from NISO clients on ISO BW.

>> Thinking some more, I
>> wonder if the Tegra folks might have an opinion to add here, given 
>> that their multiple-SMMU solution was seemingly about trying to get 
>> enough TLB and pagetable walk bandwidth in the first place?

While it is good to reduce the number of tlb register writes, Flushing all TLB entries at context granularity arbitrarily
can have negative impact on active traffic and BW. I don't have much data on possible impact at this point.
Can the flushing at context granularity be made a quirk than performing it as default? 

-KR

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

  parent reply	other threads:[~2021-06-11  0:37 UTC|newest]

Thread overview: 54+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-09 14:53 [PATCH] iommu/io-pgtable-arm: Optimize partial walk flush for large scatter-gather list Sai Prakash Ranjan
2021-06-09 14:53 ` Sai Prakash Ranjan
2021-06-09 18:44 ` Robin Murphy
2021-06-09 18:44   ` Robin Murphy
2021-06-09 18:44   ` Robin Murphy
2021-06-10  5:24   ` Sai Prakash Ranjan
2021-06-10  5:24     ` Sai Prakash Ranjan
2021-06-10  9:08     ` Robin Murphy
2021-06-10  9:08       ` Robin Murphy
2021-06-10  9:08       ` Robin Murphy
2021-06-10  9:36       ` Sai Prakash Ranjan
2021-06-10  9:36         ` Sai Prakash Ranjan
2021-06-10 11:33         ` Robin Murphy
2021-06-10 11:33           ` Robin Murphy
2021-06-10 11:33           ` Robin Murphy
2021-06-10 11:54           ` Sai Prakash Ranjan
2021-06-10 11:54             ` Sai Prakash Ranjan
2021-06-10 15:29             ` Robin Murphy
2021-06-10 15:29               ` Robin Murphy
2021-06-10 15:29               ` Robin Murphy
2021-06-10 15:51               ` Sai Prakash Ranjan
2021-06-10 15:51                 ` Sai Prakash Ranjan
2021-06-11  0:37               ` Krishna Reddy [this message]
2021-06-11  0:37                 ` Krishna Reddy
2021-06-11  0:37                 ` Krishna Reddy
2021-06-11  0:54                 ` Sai Prakash Ranjan
2021-06-11  0:54                   ` Sai Prakash Ranjan
2021-06-11 16:49                   ` Krishna Reddy
2021-06-11 16:49                     ` Krishna Reddy
2021-06-11 16:49                     ` Krishna Reddy
2021-06-12  2:46                     ` Sai Prakash Ranjan
2021-06-12  2:46                       ` Sai Prakash Ranjan
2021-06-14 17:48                       ` Krishna Reddy
2021-06-14 17:48                         ` Krishna Reddy
2021-06-14 17:48                         ` Krishna Reddy
2021-06-15 11:51                         ` Sai Prakash Ranjan
2021-06-15 11:51                           ` Sai Prakash Ranjan
2021-06-15 13:53                           ` Robin Murphy
2021-06-15 13:53                             ` Robin Murphy
2021-06-15 13:53                             ` Robin Murphy
2021-06-16  6:58                             ` Sai Prakash Ranjan
2021-06-16  6:58                               ` Sai Prakash Ranjan
2021-06-16  9:03                               ` Sai Prakash Ranjan
2021-06-16  9:03                                 ` Sai Prakash Ranjan
2021-06-17 21:18                                 ` Krishna Reddy
2021-06-17 21:18                                   ` Krishna Reddy
2021-06-17 21:18                                   ` Krishna Reddy
2021-06-18  2:47                                   ` Sai Prakash Ranjan
2021-06-18  2:47                                     ` Sai Prakash Ranjan
2021-06-18  4:04                           ` Sai Prakash Ranjan
2021-06-18  4:04                             ` Sai Prakash Ranjan
2021-06-10 12:03           ` Thierry Reding
2021-06-10 12:03             ` Thierry Reding
2021-06-10 12:03             ` Thierry Reding

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=BY5PR12MB37646698F37C00381EFF7C77B3349@BY5PR12MB3764.namprd12.prod.outlook.com \
    --to=vdumpa@nvidia.com \
    --cc=iommu@lists.linux-foundation.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-arm-msm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=robin.murphy@arm.com \
    --cc=saiprakash.ranjan@codeaurora.org \
    --cc=treding@nvidia.com \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.