From: John Garry <john.garry@huawei.com>
To: Robin Murphy <robin.murphy@arm.com>,
"Leizhen (ThunderTown)" <thunder.leizhen@huawei.com>,
Will Deacon <will@kernel.org>, Joerg Roedel <joro@8bytes.org>,
iommu <iommu@lists.linux-foundation.org>,
linux-kernel <linux-kernel@vger.kernel.org>
Cc: Vijayanand Jitta <vjitta@codeaurora.org>,
Linuxarm <linuxarm@huawei.com>,
chenxiang <chenxiang66@hisilicon.com>
Subject: Re: [PATCH 1/1] Revert "iommu/iova: Retry from last rb tree node if iova search fails"
Date: Tue, 2 Mar 2021 12:30:51 +0000 [thread overview]
Message-ID: <0cb64d74-0ec1-2284-f67a-b1619a3eb138@huawei.com> (raw)
In-Reply-To: <d090b869-b3ac-fecc-9efd-d870e43e0d99@huawei.com>
On 01/03/2021 15:48, John Garry wrote:
>>
>> While max32_alloc_size indirectly tracks the largest*contiguous*
>> available space, one of the ideas from which it grew was to simply keep
>> count of the total number of free PFNs. If you're really spending
>> significant time determining that the tree is full, as opposed to just
>> taking longer to eventually succeed, then it might be relatively
>> innocuous to tack on that semi-redundant extra accounting as a
>> self-contained quick fix for that worst case.
>>
>>> Anyway, we see ~50% throughput regression, which is intolerable. As seen
>>> in [0], I put this down to the fact that we have so many IOVA requests
>>> which exceed the rcache size limit, which means many RB tree accesses
>>> for non-cacheble IOVAs, which are now slower.
>
> I will attempt to prove this by increasing RCACHE RANGE, such that all
> IOVA sizes may be cached.
About this one, as expected, we restore performance by increasing the
RCACHE RANGE.
Some figures:
Baseline v5.12-rc1
strict mode:
600K IOPs
Revert "iommu/iova: Retry from last rb tree node if iova search fails":
1215K
Increase IOVA RCACHE range 6 -> 10 (All IOVAs size requests now
cacheable for this experiment):
1400K
Reduce LLDD max SGE count 124 -> 16:
1288K
non-strict mode
1650K
So ideally we can work towards something for which IOVAs of all size
could be cached.
Cheers,
John
WARNING: multiple messages have this Message-ID (diff)
From: John Garry <john.garry@huawei.com>
To: Robin Murphy <robin.murphy@arm.com>,
"Leizhen (ThunderTown)" <thunder.leizhen@huawei.com>,
Will Deacon <will@kernel.org>, Joerg Roedel <joro@8bytes.org>,
iommu <iommu@lists.linux-foundation.org>,
linux-kernel <linux-kernel@vger.kernel.org>
Cc: Vijayanand Jitta <vjitta@codeaurora.org>, Linuxarm <linuxarm@huawei.com>
Subject: Re: [PATCH 1/1] Revert "iommu/iova: Retry from last rb tree node if iova search fails"
Date: Tue, 2 Mar 2021 12:30:51 +0000 [thread overview]
Message-ID: <0cb64d74-0ec1-2284-f67a-b1619a3eb138@huawei.com> (raw)
In-Reply-To: <d090b869-b3ac-fecc-9efd-d870e43e0d99@huawei.com>
On 01/03/2021 15:48, John Garry wrote:
>>
>> While max32_alloc_size indirectly tracks the largest*contiguous*
>> available space, one of the ideas from which it grew was to simply keep
>> count of the total number of free PFNs. If you're really spending
>> significant time determining that the tree is full, as opposed to just
>> taking longer to eventually succeed, then it might be relatively
>> innocuous to tack on that semi-redundant extra accounting as a
>> self-contained quick fix for that worst case.
>>
>>> Anyway, we see ~50% throughput regression, which is intolerable. As seen
>>> in [0], I put this down to the fact that we have so many IOVA requests
>>> which exceed the rcache size limit, which means many RB tree accesses
>>> for non-cacheble IOVAs, which are now slower.
>
> I will attempt to prove this by increasing RCACHE RANGE, such that all
> IOVA sizes may be cached.
About this one, as expected, we restore performance by increasing the
RCACHE RANGE.
Some figures:
Baseline v5.12-rc1
strict mode:
600K IOPs
Revert "iommu/iova: Retry from last rb tree node if iova search fails":
1215K
Increase IOVA RCACHE range 6 -> 10 (All IOVAs size requests now
cacheable for this experiment):
1400K
Reduce LLDD max SGE count 124 -> 16:
1288K
non-strict mode
1650K
So ideally we can work towards something for which IOVAs of all size
could be cached.
Cheers,
John
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2021-03-02 14:11 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-29 9:21 [PATCH 1/1] Revert "iommu/iova: Retry from last rb tree node if iova search fails" Zhen Lei
2021-01-29 9:48 ` Leizhen (ThunderTown)
2021-01-29 12:03 ` Robin Murphy
2021-01-29 12:43 ` chenxiang (M)
2021-02-25 13:54 ` John Garry
2021-02-25 13:54 ` John Garry
2021-03-01 13:20 ` Robin Murphy
2021-03-01 13:20 ` Robin Murphy
2021-03-01 15:48 ` John Garry
2021-03-01 15:48 ` John Garry
2021-03-02 12:30 ` John Garry [this message]
2021-03-02 12:30 ` John Garry
2021-03-08 15:15 ` Robin Murphy
2021-03-08 15:15 ` Robin Murphy
2021-03-08 16:22 ` John Garry
2021-03-08 16:22 ` John Garry
2021-03-10 17:50 ` John Garry
2021-03-10 17:50 ` John Garry
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0cb64d74-0ec1-2284-f67a-b1619a3eb138@huawei.com \
--to=john.garry@huawei.com \
--cc=chenxiang66@hisilicon.com \
--cc=iommu@lists.linux-foundation.org \
--cc=joro@8bytes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxarm@huawei.com \
--cc=robin.murphy@arm.com \
--cc=thunder.leizhen@huawei.com \
--cc=vjitta@codeaurora.org \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.