From: Yicong Yang <yangyicong@huawei.com>
To: Barry Song <21cnbao@gmail.com>
Cc: <yangyicong@hisilicon.com>,
Catalin Marinas <catalin.marinas@arm.com>,
<akpm@linux-foundation.org>, <linux-mm@kvack.org>,
<linux-arm-kernel@lists.infradead.org>, <x86@kernel.org>,
<mark.rutland@arm.com>, <ryan.roberts@arm.com>, <will@kernel.org>,
<anshuman.khandual@arm.com>, <linux-doc@vger.kernel.org>,
<corbet@lwn.net>, <peterz@infradead.org>, <arnd@arndb.de>,
<punit.agrawal@bytedance.com>, <linux-kernel@vger.kernel.org>,
<darren@os.amperecomputing.com>, <huzhanyuan@oppo.com>,
<lipeifeng@oppo.com>, <zhangshiming@oppo.com>, <guojian@oppo.com>,
<realmz6@gmail.com>, <linux-mips@vger.kernel.org>,
<openrisc@lists.librecores.org>, <linuxppc-dev@lists.ozlabs.org>,
<linux-riscv@lists.infradead.org>, <linux-s390@vger.kernel.org>,
<wangkefeng.wang@huawei.com>, <xhao@linux.alibaba.com>,
<prime.zeng@hisilicon.com>, <Jonathan.Cameron@huawei.com>,
Barry Song <v-songbaohua@oppo.com>, Nadav Amit <namit@vmware.com>,
Mel Gorman <mgorman@suse.de>
Subject: Re: [RESEND PATCH v9 2/2] arm64: support batched/deferred tlb shootdown during page reclamation/migration
Date: Wed, 5 Jul 2023 18:24:12 +0800 [thread overview]
Message-ID: <124b7798-94ae-ebfc-bbe5-21ebaaa02760@huawei.com> (raw)
In-Reply-To: <CAGsJ_4zG=DT0gwC+5uN51rQKT=UudNDZ4t1BgRNoFb_3NNLOtQ@mail.gmail.com>
On 2023/7/5 16:43, Barry Song wrote:
> On Tue, Jul 4, 2023 at 10:36 PM Yicong Yang <yangyicong@huawei.com> wrote:
>>
>> On 2023/6/30 1:26, Catalin Marinas wrote:
>>> On Thu, Jun 29, 2023 at 05:31:36PM +0100, Catalin Marinas wrote:
>>>> On Thu, May 18, 2023 at 02:59:34PM +0800, Yicong Yang wrote:
>>>>> From: Barry Song <v-songbaohua@oppo.com>
>>>>>
>>>>> on x86, batched and deferred tlb shootdown has lead to 90%
>>>>> performance increase on tlb shootdown. on arm64, HW can do
>>>>> tlb shootdown without software IPI. But sync tlbi is still
>>>>> quite expensive.
>>>> [...]
>>>>> .../features/vm/TLB/arch-support.txt | 2 +-
>>>>> arch/arm64/Kconfig | 1 +
>>>>> arch/arm64/include/asm/tlbbatch.h | 12 ++++
>>>>> arch/arm64/include/asm/tlbflush.h | 33 ++++++++-
>>>>> arch/arm64/mm/flush.c | 69 +++++++++++++++++++
>>>>> arch/x86/include/asm/tlbflush.h | 5 +-
>>>>> include/linux/mm_types_task.h | 4 +-
>>>>> mm/rmap.c | 12 ++--
>>>>
>>>> First of all, this patch needs to be split in some preparatory patches
>>>> introducing/renaming functions with no functional change for x86. Once
>>>> done, you can add the arm64-only changes.
>>>>
>>
>> got it. will try to split this patch as suggested.
>>
>>>> Now, on the implementation, I had some comments on v7 but we didn't get
>>>> to a conclusion and the thread eventually died:
>>>>
>>>> https://lore.kernel.org/linux-mm/Y7cToj5mWd1ZbMyQ@arm.com/
>>>>
>>>> I know I said a command line argument is better than Kconfig or some
>>>> random number of CPUs heuristics but it would be even better if we don't
>>>> bother with any, just make this always on.
>>
>> ok, will make this always on.
>>
>>>> Barry had some comments
>>>> around mprotect() being racy and that's why we have
>>>> flush_tlb_batched_pending() but I don't think it's needed (or, for
>>>> arm64, it can be a DSB since this patch issues the TLBIs but without the
>>>> DVM Sync). So we need to clarify this (see Barry's last email on the
>>>> above thread) and before attempting new versions of this patchset. With
>>>> flush_tlb_batched_pending() removed (or DSB), I have a suspicion such
>>>> implementation would be faster on any SoC irrespective of the number of
>>>> CPUs.
>>>
>>> I think I got the need for flush_tlb_batched_pending(). If
>>> try_to_unmap() marks the pte !present and we have a pending TLBI,
>>> change_pte_range() will skip the TLB maintenance altogether since it did
>>> not change the pte. So we could be left with stale TLB entries after
>>> mprotect() before TTU does the batch flushing.
>>>
>
> Good catch.
> This could be also true for MADV_DONTNEED. after try_to_unmap, we run
> MADV_DONTNEED on this area, as pte is not present, we don't do anything
> on this PTE in zap_pte_range afterwards.
>
>>> We can have an arch-specific flush_tlb_batched_pending() that can be a
>>> DSB only on arm64 and a full mm flush on x86.
>>>
>>
>> We need to do a flush/dsb in flush_tlb_batched_pending() only in a race
>> condition so we first check whether there's a pended batched flush and
>> if so do the tlb flush. The pending checking is common and the differences
>> among the archs is how to flush the TLB here within the flush_tlb_batched_pending(),
>> on arm64 it should only be a dsb.
>>
>> As we only needs to maintain the TLBs already pended in batched flush,
>> does it make sense to only handle those TLBs in flush_tlb_batched_pending()?
>> Then we can use the arch_tlbbatch_flush() rather than flush_tlb_mm() in
>> flush_tlb_batched_pending() and no arch specific function needed.
>
> as we have issued no-sync tlbi on those pending addresses , that means
> our hardware
> has already "recorded" what should be flushed in the specific mm. so
> DSB only will flush
> them correctly. right?
>
yes it's right. I was just thought something like below. arch_tlbbatch_flush()
will only be a dsb on arm64 so this will match what Catalin wants. But as you
told that this maybe incorrect on x86 so we'd better have arch specific
implementation for flush_tlb_batched_pending() as suggested.
diff --git a/mm/rmap.c b/mm/rmap.c
index 9699c6011b0e..afa3571503a0 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -717,7 +717,7 @@ void flush_tlb_batched_pending(struct mm_struct *mm)
int flushed = batch >> TLB_FLUSH_BATCH_FLUSHED_SHIFT;
if (pending != flushed) {
- flush_tlb_mm(mm);
+ arch_tlbbatch_flush(¤t->tlb_ubc.arch);
/*
* If the new TLB flushing is pending during flushing, leave
* mm->tlb_flush_batched as is, to avoid losing flushing.
prev parent reply other threads:[~2023-07-05 10:24 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-05-18 6:59 [RESEND PATCH v9 0/2] arm64: support batched/deferred tlb shootdown during page reclamation/migration Yicong Yang
2023-05-18 6:59 ` [RESEND PATCH v9 1/2] mm/tlbbatch: Introduce arch_tlbbatch_should_defer() Yicong Yang
2023-05-18 6:59 ` [RESEND PATCH v9 2/2] arm64: support batched/deferred tlb shootdown during page reclamation/migration Yicong Yang
2023-06-29 16:31 ` Catalin Marinas
2023-06-29 17:26 ` Catalin Marinas
2023-07-04 14:36 ` Yicong Yang
2023-07-05 8:43 ` Barry Song
2023-07-05 10:24 ` Yicong Yang [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=124b7798-94ae-ebfc-bbe5-21ebaaa02760@huawei.com \
--to=yangyicong@huawei.com \
--cc=21cnbao@gmail.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=akpm@linux-foundation.org \
--cc=anshuman.khandual@arm.com \
--cc=arnd@arndb.de \
--cc=catalin.marinas@arm.com \
--cc=corbet@lwn.net \
--cc=darren@os.amperecomputing.com \
--cc=guojian@oppo.com \
--cc=huzhanyuan@oppo.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mips@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-riscv@lists.infradead.org \
--cc=linux-s390@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=lipeifeng@oppo.com \
--cc=mark.rutland@arm.com \
--cc=mgorman@suse.de \
--cc=namit@vmware.com \
--cc=openrisc@lists.librecores.org \
--cc=peterz@infradead.org \
--cc=prime.zeng@hisilicon.com \
--cc=punit.agrawal@bytedance.com \
--cc=realmz6@gmail.com \
--cc=ryan.roberts@arm.com \
--cc=v-songbaohua@oppo.com \
--cc=wangkefeng.wang@huawei.com \
--cc=will@kernel.org \
--cc=x86@kernel.org \
--cc=xhao@linux.alibaba.com \
--cc=yangyicong@hisilicon.com \
--cc=zhangshiming@oppo.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).