From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A76FC43381 for ; Fri, 15 Mar 2019 08:24:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 69A4C218CD for ; Fri, 15 Mar 2019 08:24:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727273AbfCOIYB (ORCPT ); Fri, 15 Mar 2019 04:24:01 -0400 Received: from szxga04-in.huawei.com ([45.249.212.190]:4686 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726582AbfCOIYB (ORCPT ); Fri, 15 Mar 2019 04:24:01 -0400 Received: from DGGEMS411-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id E93509FB3120A50F2BFF; Fri, 15 Mar 2019 16:23:58 +0800 (CST) Received: from [127.0.0.1] (10.177.29.32) by DGGEMS411-HUB.china.huawei.com (10.3.19.211) with Microsoft SMTP Server id 14.3.408.0; Fri, 15 Mar 2019 16:23:50 +0800 Subject: Re: [RFC] Question about TLB flush while set Stage-2 huge pages To: Zenghui Yu , Suzuki K Poulose CC: Marc Zyngier , , , , , , , , Wang Haibin , , References: <5f712cc6-0874-adbe-add6-46f5de24f36f@huawei.com> <1c0e07b9-73f0-efa4-c1b7-ad81789b42c5@huawei.com> <5188e3b9-5b5a-a6a7-7ef0-09b7b4f06af6@arm.com> <348d0b3b-c74b-7b39-ec30-85905c077c38@huawei.com> <20190314105537.GA15323@en101> <368bd218-ac1d-19b2-6e92-960b91afee8b@huawei.com> From: Zheng Xiang Message-ID: Date: Fri, 15 Mar 2019 16:21:03 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:64.0) Gecko/20100101 Thunderbird/64.0 MIME-Version: 1.0 In-Reply-To: <368bd218-ac1d-19b2-6e92-960b91afee8b@huawei.com> Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 8bit X-Originating-IP: [10.177.29.32] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Suzuki, I have tested this patch, VM doesn't hang and we get expected WARNING log: [ 526.184452] pstate: 20400009 (nzCv daif +PAN -UAO) [ 526.184454] pc : user_mem_abort+0x484/0x9e0 [ 526.184455] lr : user_mem_abort+0x478/0x9e0 [ 526.184456] sp : ffff000084a038e0 [ 526.184457] x29: ffff000084a038e0 x28: 000000012f600000 [ 526.184458] x27: ffff8a2fa27ae918 x26: 0000000000200000 [ 526.184460] x25: 0000000000000000 x24: 0000000000000000 [ 526.184461] x23: 00400a269d0007fd x22: ffff0000849cd000 [ 526.184462] x21: ffff00001181d000 x20: 00000a26eef72003 [ 526.184463] x19: ffff8a2fb41d4bd8 x18: 00004fffb8b22000 [ 526.184465] x17: 0000000000000000 x16: 0000000000000000 [ 526.184466] x15: 0000000000000001 x14: ffff000008dd12a8 [ 526.184467] x13: 0000000000000041 x12: ffff8a26eeca6e30 [ 526.184468] x11: ffff8000fe4af800 x10: 0000000000000040 [ 526.184469] x9 : ffff0000097c46c0 x8 : ffff8000ff400248 [ 526.184471] x7 : 0000001000000000 x6 : 00000000000021f8 [ 526.184472] x5 : 00000000a269d000 x4 : 0000000000000018 [ 526.184473] x3 : 000000000000000a x2 : 0000000000000004 [ 526.184474] x1 : 0000000000000000 x0 : 0000000000000000 [ 526.184476] Call trace: [ 526.184477] user_mem_abort+0x484/0x9e0 [ 526.184479] kvm_handle_guest_abort+0x11c/0x478 [ 526.184480] handle_exit+0x14c/0x1c8 [ 526.184482] kvm_arch_vcpu_ioctl_run+0x280/0x898 [ 526.184483] kvm_vcpu_ioctl+0x488/0x8a8 [ 526.184485] do_vfs_ioctl+0xc4/0x8c0 [ 526.184486] ksys_ioctl+0x8c/0xa0 [ 526.184487] __arm64_sys_ioctl+0x28/0x38 [ 526.184489] el0_svc_common+0xa0/0x180 [ 526.184491] el0_svc_handler+0x38/0x78 [ 526.184492] el0_svc+0x8/0xc However, we also get the following unexpected log: [ 908.329900] BUG: Bad page state in process qemu-kvm pfn:a2fb41cf [ 908.339415] page:ffff7e28bed073c0 count:-4 mapcount:0 mapping:0000000000000000 index:0x0 [ 908.339416] flags: 0x4ffffe0000000000() [ 908.339418] raw: 4ffffe0000000000 dead000000000100 dead000000000200 0000000000000000 [ 908.339419] raw: 0000000000000000 0000000000000000 fffffffcffffffff 0000000000000000 [ 908.339420] page dumped because: nonzero _refcount [ 908.339437] CPU: 32 PID: 72599 Comm: qemu-kvm Kdump: loaded Tainted: G B W 5.0.0+ #1 [ 908.339438] Call trace: [ 908.339439] dump_backtrace+0x0/0x188 [ 908.339441] show_stack+0x24/0x30 [ 908.339442] dump_stack+0xa8/0xcc [ 908.339443] bad_page+0xf0/0x150 [ 908.339445] free_pages_check_bad+0x84/0xa0 [ 908.339446] free_pcppages_bulk+0x4b8/0x750 [ 908.339448] free_unref_page_commit+0x13c/0x198 [ 908.339449] free_unref_page+0x84/0xa0 [ 908.339451] __free_pages+0x58/0x68 [ 908.339452] zap_huge_pmd+0x290/0x2d8 [ 908.339454] unmap_page_range+0x2b4/0x470 [ 908.339455] unmap_single_vma+0x94/0xe8 [ 908.339457] unmap_vmas+0x8c/0x108 [ 908.339458] exit_mmap+0xd4/0x178 [ 908.339459] mmput+0x74/0x180 [ 908.339460] do_exit+0x2b4/0x5b0 [ 908.339462] do_group_exit+0x3c/0xe0 [ 908.339463] __arm64_sys_exit_group+0x24/0x28 [ 908.339465] el0_svc_common+0xa0/0x180 [ 908.339466] el0_svc_handler+0x38/0x78 [ 908.339467] el0_svc+0x8/0xc >> Marc and I had a discussion about this and it looks like we may have an >> issue here. So with the cancellation of logging, we do not trigger the >> mmu_notifiers (as the userspace memory mapping hasn't changed) and thus >> have memory leaks while trying to install a huge mapping. Would it be >> possible for you to try the patch below ? It will trigger a WARNING >> to confirm our theory, but should not cause the hang. As we unmap >> the PMD/PUD range of PTE mappings before reinstalling a block map. > > Thanks for the reply. And I think this is alomst what Zheng Xiang wanted to say! We will test this patch tomorrow and give you some feedback. > > BTW, we have noticed that X86 had also suffered from the similar issue. You may want to look into commit 3ea3b7fa9af0 ("kvm: mmu: lazy collapse small sptes into large sptes" 2015) :-) > > > thanks, > > zenghui > >> >> >> ---8>--- >> >> test: kvm: arm: Fix handling of stage2 huge mappings >> >> We rely on the mmu_notifier call backs to handle the split/merging >> of huge pages and thus we are guaranteed that while creating a >> block mapping, the entire block is unmapped at stage2. However, >> we miss a case where the block mapping is split for dirty logging >> case and then could later be made block mapping, if we cancel the >> dirty logging. This not only creates inconsistent TLB entries for >> the pages in the the block, but also leakes the table pages for >> PMD level. >> >> Handle these corner cases for the huge mappings at stage2. >> >> Signed-off-by: Suzuki K Poulose >> --- >>   virt/kvm/arm/mmu.c | 51 +++++++++++++++++++++++++++++++++++---------------- >>   1 file changed, 35 insertions(+), 16 deletions(-) >> >> diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c >> index 66e0fbb5..04b0f9b 100644 >> --- a/virt/kvm/arm/mmu.c >> +++ b/virt/kvm/arm/mmu.c >> @@ -1076,24 +1076,38 @@ static int stage2_set_pmd_huge(struct kvm *kvm, struct kvm_mmu_memory_cache >>            * Skip updating the page table if the entry is >>            * unchanged. >>            */ >> -        if (pmd_val(old_pmd) == pmd_val(*new_pmd)) >> +        if (pmd_val(old_pmd) == pmd_val(*new_pmd)) { >>               return 0; >> - >> +        } else if (WARN_ON_ONCE(!pmd_thp_or_huge(old_pmd))) { >>           /* >> -         * Mapping in huge pages should only happen through a >> -         * fault.  If a page is merged into a transparent huge >> -         * page, the individual subpages of that huge page >> -         * should be unmapped through MMU notifiers before we >> -         * get here. >> -         * >> -         * Merging of CompoundPages is not supported; they >> -         * should become splitting first, unmapped, merged, >> -         * and mapped back in on-demand. >> +         * If we have PTE level mapping for this block, >> +         * we must unmap it to avoid inconsistent TLB >> +         * state. We could end up in this situation if >> +         * the memory slot was marked for dirty logging >> +         * and was reverted, leaving PTE level mappings >> +         * for the pages accessed during the period. >> +         * Normal THP split/merge follows mmu_notifier >> +         * callbacks and do get handled accordingly. >>            */ >> -        VM_BUG_ON(pmd_pfn(old_pmd) != pmd_pfn(*new_pmd)); >> +            unmap_stage2_range(kvm, (addr & S2_PMD_MASK), S2_PMD_SIZE); It seems that kvm decreases the _refcount of the page twice in transparent_hugepage_adjust() and unmap_stage2_range(). >> +        } else { >>   -        pmd_clear(pmd); >> -        kvm_tlb_flush_vmid_ipa(kvm, addr); >> +            /* >> +             * Mapping in huge pages should only happen through a >> +             * fault.  If a page is merged into a transparent huge >> +             * page, the individual subpages of that huge page >> +             * should be unmapped through MMU notifiers before we >> +             * get here. >> +             * >> +             * Merging of CompoundPages is not supported; they >> +             * should become splitting first, unmapped, merged, >> +             * and mapped back in on-demand. >> +             */ >> +            WARN_ON_ONCE(pmd_pfn(old_pmd) != pmd_pfn(*new_pmd)); >> + >> +            pmd_clear(pmd); >> +            kvm_tlb_flush_vmid_ipa(kvm, addr); >> +        } >>       } else { >>           get_page(virt_to_page(pmd)); >>       } >> @@ -1122,8 +1136,13 @@ static int stage2_set_pud_huge(struct kvm *kvm, struct kvm_mmu_memory_cache *cac >>           return 0; >>         if (stage2_pud_present(kvm, old_pud)) { >> -        stage2_pud_clear(kvm, pudp); >> -        kvm_tlb_flush_vmid_ipa(kvm, addr); >> +        /* If we have PTE level mapping, unmap the entire range */ >> +        if (WARN_ON_ONCE(!stage2_pud_huge(kvm, old_pud))) { >> +            unmap_stage2_range(kvm, addr & S2_PUD_MASK, S2_PUD_SIZE); >> +        } else { >> +            stage2_pud_clear(kvm, pudp); >> +            kvm_tlb_flush_vmid_ipa(kvm, addr); >> +        } >>       } else { >>           get_page(virt_to_page(pudp)); >>       } >> > > > . -- Thanks, Xiang From mboxrd@z Thu Jan 1 00:00:00 1970 From: Zheng Xiang Subject: Re: [RFC] Question about TLB flush while set Stage-2 huge pages Date: Fri, 15 Mar 2019 16:21:03 +0800 Message-ID: References: <5f712cc6-0874-adbe-add6-46f5de24f36f@huawei.com> <1c0e07b9-73f0-efa4-c1b7-ad81789b42c5@huawei.com> <5188e3b9-5b5a-a6a7-7ef0-09b7b4f06af6@arm.com> <348d0b3b-c74b-7b39-ec30-85905c077c38@huawei.com> <20190314105537.GA15323@en101> <368bd218-ac1d-19b2-6e92-960b91afee8b@huawei.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit Return-path: In-Reply-To: <368bd218-ac1d-19b2-6e92-960b91afee8b@huawei.com> Content-Language: en-US Sender: linux-kernel-owner@vger.kernel.org To: Zenghui Yu , Suzuki K Poulose Cc: Marc Zyngier , christoffer.dall@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, james.morse@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, Wang Haibin , lious.lilei@hisilicon.com, lishuo1@hisilicon.com List-Id: kvmarm@lists.cs.columbia.edu Hi Suzuki, I have tested this patch, VM doesn't hang and we get expected WARNING log: [ 526.184452] pstate: 20400009 (nzCv daif +PAN -UAO) [ 526.184454] pc : user_mem_abort+0x484/0x9e0 [ 526.184455] lr : user_mem_abort+0x478/0x9e0 [ 526.184456] sp : ffff000084a038e0 [ 526.184457] x29: ffff000084a038e0 x28: 000000012f600000 [ 526.184458] x27: ffff8a2fa27ae918 x26: 0000000000200000 [ 526.184460] x25: 0000000000000000 x24: 0000000000000000 [ 526.184461] x23: 00400a269d0007fd x22: ffff0000849cd000 [ 526.184462] x21: ffff00001181d000 x20: 00000a26eef72003 [ 526.184463] x19: ffff8a2fb41d4bd8 x18: 00004fffb8b22000 [ 526.184465] x17: 0000000000000000 x16: 0000000000000000 [ 526.184466] x15: 0000000000000001 x14: ffff000008dd12a8 [ 526.184467] x13: 0000000000000041 x12: ffff8a26eeca6e30 [ 526.184468] x11: ffff8000fe4af800 x10: 0000000000000040 [ 526.184469] x9 : ffff0000097c46c0 x8 : ffff8000ff400248 [ 526.184471] x7 : 0000001000000000 x6 : 00000000000021f8 [ 526.184472] x5 : 00000000a269d000 x4 : 0000000000000018 [ 526.184473] x3 : 000000000000000a x2 : 0000000000000004 [ 526.184474] x1 : 0000000000000000 x0 : 0000000000000000 [ 526.184476] Call trace: [ 526.184477] user_mem_abort+0x484/0x9e0 [ 526.184479] kvm_handle_guest_abort+0x11c/0x478 [ 526.184480] handle_exit+0x14c/0x1c8 [ 526.184482] kvm_arch_vcpu_ioctl_run+0x280/0x898 [ 526.184483] kvm_vcpu_ioctl+0x488/0x8a8 [ 526.184485] do_vfs_ioctl+0xc4/0x8c0 [ 526.184486] ksys_ioctl+0x8c/0xa0 [ 526.184487] __arm64_sys_ioctl+0x28/0x38 [ 526.184489] el0_svc_common+0xa0/0x180 [ 526.184491] el0_svc_handler+0x38/0x78 [ 526.184492] el0_svc+0x8/0xc However, we also get the following unexpected log: [ 908.329900] BUG: Bad page state in process qemu-kvm pfn:a2fb41cf [ 908.339415] page:ffff7e28bed073c0 count:-4 mapcount:0 mapping:0000000000000000 index:0x0 [ 908.339416] flags: 0x4ffffe0000000000() [ 908.339418] raw: 4ffffe0000000000 dead000000000100 dead000000000200 0000000000000000 [ 908.339419] raw: 0000000000000000 0000000000000000 fffffffcffffffff 0000000000000000 [ 908.339420] page dumped because: nonzero _refcount [ 908.339437] CPU: 32 PID: 72599 Comm: qemu-kvm Kdump: loaded Tainted: G B W 5.0.0+ #1 [ 908.339438] Call trace: [ 908.339439] dump_backtrace+0x0/0x188 [ 908.339441] show_stack+0x24/0x30 [ 908.339442] dump_stack+0xa8/0xcc [ 908.339443] bad_page+0xf0/0x150 [ 908.339445] free_pages_check_bad+0x84/0xa0 [ 908.339446] free_pcppages_bulk+0x4b8/0x750 [ 908.339448] free_unref_page_commit+0x13c/0x198 [ 908.339449] free_unref_page+0x84/0xa0 [ 908.339451] __free_pages+0x58/0x68 [ 908.339452] zap_huge_pmd+0x290/0x2d8 [ 908.339454] unmap_page_range+0x2b4/0x470 [ 908.339455] unmap_single_vma+0x94/0xe8 [ 908.339457] unmap_vmas+0x8c/0x108 [ 908.339458] exit_mmap+0xd4/0x178 [ 908.339459] mmput+0x74/0x180 [ 908.339460] do_exit+0x2b4/0x5b0 [ 908.339462] do_group_exit+0x3c/0xe0 [ 908.339463] __arm64_sys_exit_group+0x24/0x28 [ 908.339465] el0_svc_common+0xa0/0x180 [ 908.339466] el0_svc_handler+0x38/0x78 [ 908.339467] el0_svc+0x8/0xc >> Marc and I had a discussion about this and it looks like we may have an >> issue here. So with the cancellation of logging, we do not trigger the >> mmu_notifiers (as the userspace memory mapping hasn't changed) and thus >> have memory leaks while trying to install a huge mapping. Would it be >> possible for you to try the patch below ? It will trigger a WARNING >> to confirm our theory, but should not cause the hang. As we unmap >> the PMD/PUD range of PTE mappings before reinstalling a block map. > > Thanks for the reply. And I think this is alomst what Zheng Xiang wanted to say! We will test this patch tomorrow and give you some feedback. > > BTW, we have noticed that X86 had also suffered from the similar issue. You may want to look into commit 3ea3b7fa9af0 ("kvm: mmu: lazy collapse small sptes into large sptes" 2015) :-) > > > thanks, > > zenghui > >> >> >> ---8>--- >> >> test: kvm: arm: Fix handling of stage2 huge mappings >> >> We rely on the mmu_notifier call backs to handle the split/merging >> of huge pages and thus we are guaranteed that while creating a >> block mapping, the entire block is unmapped at stage2. However, >> we miss a case where the block mapping is split for dirty logging >> case and then could later be made block mapping, if we cancel the >> dirty logging. This not only creates inconsistent TLB entries for >> the pages in the the block, but also leakes the table pages for >> PMD level. >> >> Handle these corner cases for the huge mappings at stage2. >> >> Signed-off-by: Suzuki K Poulose >> --- >>   virt/kvm/arm/mmu.c | 51 +++++++++++++++++++++++++++++++++++---------------- >>   1 file changed, 35 insertions(+), 16 deletions(-) >> >> diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c >> index 66e0fbb5..04b0f9b 100644 >> --- a/virt/kvm/arm/mmu.c >> +++ b/virt/kvm/arm/mmu.c >> @@ -1076,24 +1076,38 @@ static int stage2_set_pmd_huge(struct kvm *kvm, struct kvm_mmu_memory_cache >>            * Skip updating the page table if the entry is >>            * unchanged. >>            */ >> -        if (pmd_val(old_pmd) == pmd_val(*new_pmd)) >> +        if (pmd_val(old_pmd) == pmd_val(*new_pmd)) { >>               return 0; >> - >> +        } else if (WARN_ON_ONCE(!pmd_thp_or_huge(old_pmd))) { >>           /* >> -         * Mapping in huge pages should only happen through a >> -         * fault.  If a page is merged into a transparent huge >> -         * page, the individual subpages of that huge page >> -         * should be unmapped through MMU notifiers before we >> -         * get here. >> -         * >> -         * Merging of CompoundPages is not supported; they >> -         * should become splitting first, unmapped, merged, >> -         * and mapped back in on-demand. >> +         * If we have PTE level mapping for this block, >> +         * we must unmap it to avoid inconsistent TLB >> +         * state. We could end up in this situation if >> +         * the memory slot was marked for dirty logging >> +         * and was reverted, leaving PTE level mappings >> +         * for the pages accessed during the period. >> +         * Normal THP split/merge follows mmu_notifier >> +         * callbacks and do get handled accordingly. >>            */ >> -        VM_BUG_ON(pmd_pfn(old_pmd) != pmd_pfn(*new_pmd)); >> +            unmap_stage2_range(kvm, (addr & S2_PMD_MASK), S2_PMD_SIZE); It seems that kvm decreases the _refcount of the page twice in transparent_hugepage_adjust() and unmap_stage2_range(). >> +        } else { >>   -        pmd_clear(pmd); >> -        kvm_tlb_flush_vmid_ipa(kvm, addr); >> +            /* >> +             * Mapping in huge pages should only happen through a >> +             * fault.  If a page is merged into a transparent huge >> +             * page, the individual subpages of that huge page >> +             * should be unmapped through MMU notifiers before we >> +             * get here. >> +             * >> +             * Merging of CompoundPages is not supported; they >> +             * should become splitting first, unmapped, merged, >> +             * and mapped back in on-demand. >> +             */ >> +            WARN_ON_ONCE(pmd_pfn(old_pmd) != pmd_pfn(*new_pmd)); >> + >> +            pmd_clear(pmd); >> +            kvm_tlb_flush_vmid_ipa(kvm, addr); >> +        } >>       } else { >>           get_page(virt_to_page(pmd)); >>       } >> @@ -1122,8 +1136,13 @@ static int stage2_set_pud_huge(struct kvm *kvm, struct kvm_mmu_memory_cache *cac >>           return 0; >>         if (stage2_pud_present(kvm, old_pud)) { >> -        stage2_pud_clear(kvm, pudp); >> -        kvm_tlb_flush_vmid_ipa(kvm, addr); >> +        /* If we have PTE level mapping, unmap the entire range */ >> +        if (WARN_ON_ONCE(!stage2_pud_huge(kvm, old_pud))) { >> +            unmap_stage2_range(kvm, addr & S2_PUD_MASK, S2_PUD_SIZE); >> +        } else { >> +            stage2_pud_clear(kvm, pudp); >> +            kvm_tlb_flush_vmid_ipa(kvm, addr); >> +        } >>       } else { >>           get_page(virt_to_page(pudp)); >>       } >> > > > . -- Thanks, Xiang From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DBF4C43381 for ; Fri, 15 Mar 2019 08:24:17 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3F60D2054F for ; Fri, 15 Mar 2019 08:24:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="QIohpLCY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3F60D2054F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date: Message-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description :Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=IApB6iB8XtqRbuHfvDFHv8DoPyRrwXyLIfq0fLsBxOg=; b=QIohpLCYdYklZE PLRxxpc85EKxQWDiGZg/2rkF15pVHzoLZLn3YPqNyKMbZuBwwvSvlbhfeBimr9NQNpC+B4rCx//Nm vlQ/TqVvEIjULWViw0OAvNL2SegZTeGwEEiVaukvHZIhZQVYmBXh5n4pH774kSwWqNj3OlSrewLk5 M/2/nIUeF2CcZbd31X7ndXkA0XBk7YErGc3ffxWcJcX+xjlFs6T44G2LQMqV99qqaRnVt80vul3hR c3CKJ8K7hsmhx/EsNxwKi500TF0KJc9Z1IY+Ed7qCUVUCTUWdZnnATZqVhP9ZCwvwUWZ4VrvVQO88 2ujvq/fF1/9vo8HpvHDA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1h4i8R-0007IQ-Rm; Fri, 15 Mar 2019 08:24:11 +0000 Received: from szxga04-in.huawei.com ([45.249.212.190] helo=huawei.com) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1h4i8P-0007H3-4J for linux-arm-kernel@lists.infradead.org; Fri, 15 Mar 2019 08:24:10 +0000 Received: from DGGEMS411-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id E93509FB3120A50F2BFF; Fri, 15 Mar 2019 16:23:58 +0800 (CST) Received: from [127.0.0.1] (10.177.29.32) by DGGEMS411-HUB.china.huawei.com (10.3.19.211) with Microsoft SMTP Server id 14.3.408.0; Fri, 15 Mar 2019 16:23:50 +0800 Subject: Re: [RFC] Question about TLB flush while set Stage-2 huge pages To: Zenghui Yu , Suzuki K Poulose References: <5f712cc6-0874-adbe-add6-46f5de24f36f@huawei.com> <1c0e07b9-73f0-efa4-c1b7-ad81789b42c5@huawei.com> <5188e3b9-5b5a-a6a7-7ef0-09b7b4f06af6@arm.com> <348d0b3b-c74b-7b39-ec30-85905c077c38@huawei.com> <20190314105537.GA15323@en101> <368bd218-ac1d-19b2-6e92-960b91afee8b@huawei.com> From: Zheng Xiang Message-ID: Date: Fri, 15 Mar 2019 16:21:03 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:64.0) Gecko/20100101 Thunderbird/64.0 MIME-Version: 1.0 In-Reply-To: <368bd218-ac1d-19b2-6e92-960b91afee8b@huawei.com> Content-Language: en-US X-Originating-IP: [10.177.29.32] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190315_012409_532580_4D6A0324 X-CRM114-Status: GOOD ( 21.15 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marc Zyngier , catalin.marinas@arm.com, will.deacon@arm.com, christoffer.dall@arm.com, linux-kernel@vger.kernel.org, james.morse@arm.com, lishuo1@hisilicon.com, Wang Haibin , kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, lious.lilei@hisilicon.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org SGkgU3V6dWtpLAoKSSBoYXZlIHRlc3RlZCB0aGlzIHBhdGNoLCBWTSBkb2Vzbid0IGhhbmcgYW5k IHdlIGdldCBleHBlY3RlZCBXQVJOSU5HIGxvZzoKClsgIDUyNi4xODQ0NTJdIHBzdGF0ZTogMjA0 MDAwMDkgKG56Q3YgZGFpZiArUEFOIC1VQU8pClsgIDUyNi4xODQ0NTRdIHBjIDogdXNlcl9tZW1f YWJvcnQrMHg0ODQvMHg5ZTAKWyAgNTI2LjE4NDQ1NV0gbHIgOiB1c2VyX21lbV9hYm9ydCsweDQ3 OC8weDllMApbICA1MjYuMTg0NDU2XSBzcCA6IGZmZmYwMDAwODRhMDM4ZTAKWyAgNTI2LjE4NDQ1 N10geDI5OiBmZmZmMDAwMDg0YTAzOGUwIHgyODogMDAwMDAwMDEyZjYwMDAwMApbICA1MjYuMTg0 NDU4XSB4Mjc6IGZmZmY4YTJmYTI3YWU5MTggeDI2OiAwMDAwMDAwMDAwMjAwMDAwClsgIDUyNi4x ODQ0NjBdIHgyNTogMDAwMDAwMDAwMDAwMDAwMCB4MjQ6IDAwMDAwMDAwMDAwMDAwMDAKWyAgNTI2 LjE4NDQ2MV0geDIzOiAwMDQwMGEyNjlkMDAwN2ZkIHgyMjogZmZmZjAwMDA4NDljZDAwMApbICA1 MjYuMTg0NDYyXSB4MjE6IGZmZmYwMDAwMTE4MWQwMDAgeDIwOiAwMDAwMGEyNmVlZjcyMDAzClsg IDUyNi4xODQ0NjNdIHgxOTogZmZmZjhhMmZiNDFkNGJkOCB4MTg6IDAwMDA0ZmZmYjhiMjIwMDAK WyAgNTI2LjE4NDQ2NV0geDE3OiAwMDAwMDAwMDAwMDAwMDAwIHgxNjogMDAwMDAwMDAwMDAwMDAw MApbICA1MjYuMTg0NDY2XSB4MTU6IDAwMDAwMDAwMDAwMDAwMDEgeDE0OiBmZmZmMDAwMDA4ZGQx MmE4ClsgIDUyNi4xODQ0NjddIHgxMzogMDAwMDAwMDAwMDAwMDA0MSB4MTI6IGZmZmY4YTI2ZWVj YTZlMzAKWyAgNTI2LjE4NDQ2OF0geDExOiBmZmZmODAwMGZlNGFmODAwIHgxMDogMDAwMDAwMDAw MDAwMDA0MApbICA1MjYuMTg0NDY5XSB4OSA6IGZmZmYwMDAwMDk3YzQ2YzAgeDggOiBmZmZmODAw MGZmNDAwMjQ4ClsgIDUyNi4xODQ0NzFdIHg3IDogMDAwMDAwMTAwMDAwMDAwMCB4NiA6IDAwMDAw MDAwMDAwMDIxZjgKWyAgNTI2LjE4NDQ3Ml0geDUgOiAwMDAwMDAwMGEyNjlkMDAwIHg0IDogMDAw MDAwMDAwMDAwMDAxOApbICA1MjYuMTg0NDczXSB4MyA6IDAwMDAwMDAwMDAwMDAwMGEgeDIgOiAw MDAwMDAwMDAwMDAwMDA0ClsgIDUyNi4xODQ0NzRdIHgxIDogMDAwMDAwMDAwMDAwMDAwMCB4MCA6 IDAwMDAwMDAwMDAwMDAwMDAKWyAgNTI2LjE4NDQ3Nl0gQ2FsbCB0cmFjZToKWyAgNTI2LjE4NDQ3 N10gIHVzZXJfbWVtX2Fib3J0KzB4NDg0LzB4OWUwClsgIDUyNi4xODQ0NzldICBrdm1faGFuZGxl X2d1ZXN0X2Fib3J0KzB4MTFjLzB4NDc4ClsgIDUyNi4xODQ0ODBdICBoYW5kbGVfZXhpdCsweDE0 Yy8weDFjOApbICA1MjYuMTg0NDgyXSAga3ZtX2FyY2hfdmNwdV9pb2N0bF9ydW4rMHgyODAvMHg4 OTgKWyAgNTI2LjE4NDQ4M10gIGt2bV92Y3B1X2lvY3RsKzB4NDg4LzB4OGE4ClsgIDUyNi4xODQ0 ODVdICBkb192ZnNfaW9jdGwrMHhjNC8weDhjMApbICA1MjYuMTg0NDg2XSAga3N5c19pb2N0bCsw eDhjLzB4YTAKWyAgNTI2LjE4NDQ4N10gIF9fYXJtNjRfc3lzX2lvY3RsKzB4MjgvMHgzOApbICA1 MjYuMTg0NDg5XSAgZWwwX3N2Y19jb21tb24rMHhhMC8weDE4MApbICA1MjYuMTg0NDkxXSAgZWww X3N2Y19oYW5kbGVyKzB4MzgvMHg3OApbICA1MjYuMTg0NDkyXSAgZWwwX3N2YysweDgvMHhjCgpI b3dldmVyLCB3ZSBhbHNvIGdldCB0aGUgZm9sbG93aW5nIHVuZXhwZWN0ZWQgbG9nOgoKWyAgOTA4 LjMyOTkwMF0gQlVHOiBCYWQgcGFnZSBzdGF0ZSBpbiBwcm9jZXNzIHFlbXUta3ZtICBwZm46YTJm YjQxY2YKWyAgOTA4LjMzOTQxNV0gcGFnZTpmZmZmN2UyOGJlZDA3M2MwIGNvdW50Oi00IG1hcGNv dW50OjAgbWFwcGluZzowMDAwMDAwMDAwMDAwMDAwIGluZGV4OjB4MApbICA5MDguMzM5NDE2XSBm bGFnczogMHg0ZmZmZmUwMDAwMDAwMDAwKCkKWyAgOTA4LjMzOTQxOF0gcmF3OiA0ZmZmZmUwMDAw MDAwMDAwIGRlYWQwMDAwMDAwMDAxMDAgZGVhZDAwMDAwMDAwMDIwMCAwMDAwMDAwMDAwMDAwMDAw ClsgIDkwOC4zMzk0MTldIHJhdzogMDAwMDAwMDAwMDAwMDAwMCAwMDAwMDAwMDAwMDAwMDAwIGZm ZmZmZmZjZmZmZmZmZmYgMDAwMDAwMDAwMDAwMDAwMApbICA5MDguMzM5NDIwXSBwYWdlIGR1bXBl ZCBiZWNhdXNlOiBub256ZXJvIF9yZWZjb3VudApbICA5MDguMzM5NDM3XSBDUFU6IDMyIFBJRDog NzI1OTkgQ29tbTogcWVtdS1rdm0gS2R1bXA6IGxvYWRlZCBUYWludGVkOiBHICAgIEIgIFcgICAg ICAgIDUuMC4wKyAjMQpbICA5MDguMzM5NDM4XSBDYWxsIHRyYWNlOgpbICA5MDguMzM5NDM5XSAg ZHVtcF9iYWNrdHJhY2UrMHgwLzB4MTg4ClsgIDkwOC4zMzk0NDFdICBzaG93X3N0YWNrKzB4MjQv MHgzMApbICA5MDguMzM5NDQyXSAgZHVtcF9zdGFjaysweGE4LzB4Y2MKWyAgOTA4LjMzOTQ0M10g IGJhZF9wYWdlKzB4ZjAvMHgxNTAKWyAgOTA4LjMzOTQ0NV0gIGZyZWVfcGFnZXNfY2hlY2tfYmFk KzB4ODQvMHhhMApbICA5MDguMzM5NDQ2XSAgZnJlZV9wY3BwYWdlc19idWxrKzB4NGI4LzB4NzUw ClsgIDkwOC4zMzk0NDhdICBmcmVlX3VucmVmX3BhZ2VfY29tbWl0KzB4MTNjLzB4MTk4ClsgIDkw OC4zMzk0NDldICBmcmVlX3VucmVmX3BhZ2UrMHg4NC8weGEwClsgIDkwOC4zMzk0NTFdICBfX2Zy ZWVfcGFnZXMrMHg1OC8weDY4ClsgIDkwOC4zMzk0NTJdICB6YXBfaHVnZV9wbWQrMHgyOTAvMHgy ZDgKWyAgOTA4LjMzOTQ1NF0gIHVubWFwX3BhZ2VfcmFuZ2UrMHgyYjQvMHg0NzAKWyAgOTA4LjMz OTQ1NV0gIHVubWFwX3NpbmdsZV92bWErMHg5NC8weGU4ClsgIDkwOC4zMzk0NTddICB1bm1hcF92 bWFzKzB4OGMvMHgxMDgKWyAgOTA4LjMzOTQ1OF0gIGV4aXRfbW1hcCsweGQ0LzB4MTc4ClsgIDkw OC4zMzk0NTldICBtbXB1dCsweDc0LzB4MTgwClsgIDkwOC4zMzk0NjBdICBkb19leGl0KzB4MmI0 LzB4NWIwClsgIDkwOC4zMzk0NjJdICBkb19ncm91cF9leGl0KzB4M2MvMHhlMApbICA5MDguMzM5 NDYzXSAgX19hcm02NF9zeXNfZXhpdF9ncm91cCsweDI0LzB4MjgKWyAgOTA4LjMzOTQ2NV0gIGVs MF9zdmNfY29tbW9uKzB4YTAvMHgxODAKWyAgOTA4LjMzOTQ2Nl0gIGVsMF9zdmNfaGFuZGxlcisw eDM4LzB4NzgKWyAgOTA4LjMzOTQ2N10gIGVsMF9zdmMrMHg4LzB4YwoKPj4gTWFyYyBhbmQgSSBo YWQgYSBkaXNjdXNzaW9uIGFib3V0IHRoaXMgYW5kIGl0IGxvb2tzIGxpa2Ugd2UgbWF5IGhhdmUg YW4KPj4gaXNzdWUgaGVyZS4gU28gd2l0aCB0aGUgY2FuY2VsbGF0aW9uIG9mIGxvZ2dpbmcsIHdl IGRvIG5vdCB0cmlnZ2VyIHRoZQo+PiBtbXVfbm90aWZpZXJzIChhcyB0aGUgdXNlcnNwYWNlIG1l bW9yeSBtYXBwaW5nIGhhc24ndCBjaGFuZ2VkKSBhbmQgdGh1cwo+PiBoYXZlIG1lbW9yeSBsZWFr cyB3aGlsZSB0cnlpbmcgdG8gaW5zdGFsbCBhIGh1Z2UgbWFwcGluZy4gV291bGQgaXQgYmUKPj4g cG9zc2libGUgZm9yIHlvdSB0byB0cnkgdGhlIHBhdGNoIGJlbG93ID8gSXQgd2lsbCB0cmlnZ2Vy IGEgV0FSTklORwo+PiB0byBjb25maXJtIG91ciB0aGVvcnksIGJ1dCBzaG91bGQgbm90IGNhdXNl IHRoZSBoYW5nLiBBcyB3ZSB1bm1hcAo+PiB0aGUgUE1EL1BVRCByYW5nZSBvZiBQVEUgbWFwcGlu Z3MgYmVmb3JlIHJlaW5zdGFsbGluZyBhIGJsb2NrIG1hcC4KPiAKPiBUaGFua3MgZm9yIHRoZSBy ZXBseS4gQW5kIEkgdGhpbmsgdGhpcyBpcyBhbG9tc3Qgd2hhdCBaaGVuZyBYaWFuZyB3YW50ZWQg dG8gc2F5ISBXZSB3aWxsIHRlc3QgdGhpcyBwYXRjaCB0b21vcnJvdyBhbmQgZ2l2ZSB5b3Ugc29t ZSBmZWVkYmFjay4KPiAKPiBCVFcsIHdlIGhhdmUgbm90aWNlZCB0aGF0IFg4NiBoYWQgYWxzbyBz dWZmZXJlZCBmcm9tIHRoZSBzaW1pbGFyIGlzc3VlLiBZb3UgbWF5IHdhbnQgdG8gbG9vayBpbnRv IGNvbW1pdCAzZWEzYjdmYTlhZjAgKCJrdm06IG1tdTogbGF6eSBjb2xsYXBzZSBzbWFsbCBzcHRl cyBpbnRvIGxhcmdlIHNwdGVzIiAyMDE1KSA6LSkKPiAKPiAKPiB0aGFua3MsCj4gCj4gemVuZ2h1 aQo+IAo+Pgo+Pgo+PiAtLS04Pi0tLQo+Pgo+PiB0ZXN0OiBrdm06IGFybTogRml4IGhhbmRsaW5n IG9mIHN0YWdlMiBodWdlIG1hcHBpbmdzCj4+Cj4+IFdlIHJlbHkgb24gdGhlIG1tdV9ub3RpZmll ciBjYWxsIGJhY2tzIHRvIGhhbmRsZSB0aGUgc3BsaXQvbWVyZ2luZwo+PiBvZiBodWdlIHBhZ2Vz IGFuZCB0aHVzIHdlIGFyZSBndWFyYW50ZWVkIHRoYXQgd2hpbGUgY3JlYXRpbmcgYQo+PiBibG9j ayBtYXBwaW5nLCB0aGUgZW50aXJlIGJsb2NrIGlzIHVubWFwcGVkIGF0IHN0YWdlMi4gSG93ZXZl ciwKPj4gd2UgbWlzcyBhIGNhc2Ugd2hlcmUgdGhlIGJsb2NrIG1hcHBpbmcgaXMgc3BsaXQgZm9y IGRpcnR5IGxvZ2dpbmcKPj4gY2FzZSBhbmQgdGhlbiBjb3VsZCBsYXRlciBiZSBtYWRlIGJsb2Nr IG1hcHBpbmcsIGlmIHdlIGNhbmNlbCB0aGUKPj4gZGlydHkgbG9nZ2luZy4gVGhpcyBub3Qgb25s eSBjcmVhdGVzIGluY29uc2lzdGVudCBUTEIgZW50cmllcyBmb3IKPj4gdGhlIHBhZ2VzIGluIHRo ZSB0aGUgYmxvY2ssIGJ1dCBhbHNvIGxlYWtlcyB0aGUgdGFibGUgcGFnZXMgZm9yCj4+IFBNRCBs ZXZlbC4KPj4KPj4gSGFuZGxlIHRoZXNlIGNvcm5lciBjYXNlcyBmb3IgdGhlIGh1Z2UgbWFwcGlu Z3MgYXQgc3RhZ2UyLgo+Pgo+PiBTaWduZWQtb2ZmLWJ5OiBTdXp1a2kgSyBQb3Vsb3NlIDxzdXp1 a2kucG91bG9zZUBhcm0uY29tPgo+PiAtLS0KPj4gwqAgdmlydC9rdm0vYXJtL21tdS5jIHwgNTEg KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKystLS0tLS0tLS0tLS0tLS0tCj4+IMKg IDEgZmlsZSBjaGFuZ2VkLCAzNSBpbnNlcnRpb25zKCspLCAxNiBkZWxldGlvbnMoLSkKPj4KPj4g ZGlmZiAtLWdpdCBhL3ZpcnQva3ZtL2FybS9tbXUuYyBiL3ZpcnQva3ZtL2FybS9tbXUuYwo+PiBp bmRleCA2NmUwZmJiNS4uMDRiMGY5YiAxMDA2NDQKPj4gLS0tIGEvdmlydC9rdm0vYXJtL21tdS5j Cj4+ICsrKyBiL3ZpcnQva3ZtL2FybS9tbXUuYwo+PiBAQCAtMTA3NiwyNCArMTA3NiwzOCBAQCBz dGF0aWMgaW50IHN0YWdlMl9zZXRfcG1kX2h1Z2Uoc3RydWN0IGt2bSAqa3ZtLCBzdHJ1Y3Qga3Zt X21tdV9tZW1vcnlfY2FjaGUKPj4gwqDCoMKgwqDCoMKgwqDCoMKgwqAgKiBTa2lwIHVwZGF0aW5n IHRoZSBwYWdlIHRhYmxlIGlmIHRoZSBlbnRyeSBpcwo+PiDCoMKgwqDCoMKgwqDCoMKgwqDCoCAq IHVuY2hhbmdlZC4KPj4gwqDCoMKgwqDCoMKgwqDCoMKgwqAgKi8KPj4gLcKgwqDCoMKgwqDCoMKg IGlmIChwbWRfdmFsKG9sZF9wbWQpID09IHBtZF92YWwoKm5ld19wbWQpKQo+PiArwqDCoMKgwqDC oMKgwqAgaWYgKHBtZF92YWwob2xkX3BtZCkgPT0gcG1kX3ZhbCgqbmV3X3BtZCkpIHsKPj4gwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgcmV0dXJuIDA7Cj4+IC0KPj4gK8KgwqDCoMKgwqDCoMKg IH0gZWxzZSBpZiAoV0FSTl9PTl9PTkNFKCFwbWRfdGhwX29yX2h1Z2Uob2xkX3BtZCkpKSB7Cj4+ IMKgwqDCoMKgwqDCoMKgwqDCoCAvKgo+PiAtwqDCoMKgwqDCoMKgwqDCoCAqIE1hcHBpbmcgaW4g aHVnZSBwYWdlcyBzaG91bGQgb25seSBoYXBwZW4gdGhyb3VnaCBhCj4+IC3CoMKgwqDCoMKgwqDC oMKgICogZmF1bHQuwqAgSWYgYSBwYWdlIGlzIG1lcmdlZCBpbnRvIGEgdHJhbnNwYXJlbnQgaHVn ZQo+PiAtwqDCoMKgwqDCoMKgwqDCoCAqIHBhZ2UsIHRoZSBpbmRpdmlkdWFsIHN1YnBhZ2VzIG9m IHRoYXQgaHVnZSBwYWdlCj4+IC3CoMKgwqDCoMKgwqDCoMKgICogc2hvdWxkIGJlIHVubWFwcGVk IHRocm91Z2ggTU1VIG5vdGlmaWVycyBiZWZvcmUgd2UKPj4gLcKgwqDCoMKgwqDCoMKgwqAgKiBn ZXQgaGVyZS4KPj4gLcKgwqDCoMKgwqDCoMKgwqAgKgo+PiAtwqDCoMKgwqDCoMKgwqDCoCAqIE1l cmdpbmcgb2YgQ29tcG91bmRQYWdlcyBpcyBub3Qgc3VwcG9ydGVkOyB0aGV5Cj4+IC3CoMKgwqDC oMKgwqDCoMKgICogc2hvdWxkIGJlY29tZSBzcGxpdHRpbmcgZmlyc3QsIHVubWFwcGVkLCBtZXJn ZWQsCj4+IC3CoMKgwqDCoMKgwqDCoMKgICogYW5kIG1hcHBlZCBiYWNrIGluIG9uLWRlbWFuZC4K Pj4gK8KgwqDCoMKgwqDCoMKgwqAgKiBJZiB3ZSBoYXZlIFBURSBsZXZlbCBtYXBwaW5nIGZvciB0 aGlzIGJsb2NrLAo+PiArwqDCoMKgwqDCoMKgwqDCoCAqIHdlIG11c3QgdW5tYXAgaXQgdG8gYXZv aWQgaW5jb25zaXN0ZW50IFRMQgo+PiArwqDCoMKgwqDCoMKgwqDCoCAqIHN0YXRlLiBXZSBjb3Vs ZCBlbmQgdXAgaW4gdGhpcyBzaXR1YXRpb24gaWYKPj4gK8KgwqDCoMKgwqDCoMKgwqAgKiB0aGUg bWVtb3J5IHNsb3Qgd2FzIG1hcmtlZCBmb3IgZGlydHkgbG9nZ2luZwo+PiArwqDCoMKgwqDCoMKg wqDCoCAqIGFuZCB3YXMgcmV2ZXJ0ZWQsIGxlYXZpbmcgUFRFIGxldmVsIG1hcHBpbmdzCj4+ICvC oMKgwqDCoMKgwqDCoMKgICogZm9yIHRoZSBwYWdlcyBhY2Nlc3NlZCBkdXJpbmcgdGhlIHBlcmlv ZC4KPj4gK8KgwqDCoMKgwqDCoMKgwqAgKiBOb3JtYWwgVEhQIHNwbGl0L21lcmdlIGZvbGxvd3Mg bW11X25vdGlmaWVyCj4+ICvCoMKgwqDCoMKgwqDCoMKgICogY2FsbGJhY2tzIGFuZCBkbyBnZXQg aGFuZGxlZCBhY2NvcmRpbmdseS4KPj4gwqDCoMKgwqDCoMKgwqDCoMKgwqAgKi8KPj4gLcKgwqDC oMKgwqDCoMKgIFZNX0JVR19PTihwbWRfcGZuKG9sZF9wbWQpICE9IHBtZF9wZm4oKm5ld19wbWQp KTsKPj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqAgdW5tYXBfc3RhZ2UyX3JhbmdlKGt2bSwgKGFk ZHIgJiBTMl9QTURfTUFTSyksIFMyX1BNRF9TSVpFKTsKCkl0IHNlZW1zIHRoYXQga3ZtIGRlY3Jl YXNlcyB0aGUgX3JlZmNvdW50IG9mIHRoZSBwYWdlIHR3aWNlIGluIHRyYW5zcGFyZW50X2h1Z2Vw YWdlX2FkanVzdCgpCmFuZCB1bm1hcF9zdGFnZTJfcmFuZ2UoKS4KCj4+ICvCoMKgwqDCoMKgwqDC oCB9IGVsc2Ugewo+PiDCoCAtwqDCoMKgwqDCoMKgwqAgcG1kX2NsZWFyKHBtZCk7Cj4+IC3CoMKg wqDCoMKgwqDCoCBrdm1fdGxiX2ZsdXNoX3ZtaWRfaXBhKGt2bSwgYWRkcik7Cj4+ICvCoMKgwqDC oMKgwqDCoMKgwqDCoMKgIC8qCj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgKiBNYXBwaW5n IGluIGh1Z2UgcGFnZXMgc2hvdWxkIG9ubHkgaGFwcGVuIHRocm91Z2ggYQo+PiArwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgICogZmF1bHQuwqAgSWYgYSBwYWdlIGlzIG1lcmdlZCBpbnRvIGEgdHJh bnNwYXJlbnQgaHVnZQo+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgICogcGFnZSwgdGhlIGlu ZGl2aWR1YWwgc3VicGFnZXMgb2YgdGhhdCBodWdlIHBhZ2UKPj4gK8KgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCAqIHNob3VsZCBiZSB1bm1hcHBlZCB0aHJvdWdoIE1NVSBub3RpZmllcnMgYmVmb3Jl IHdlCj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgKiBnZXQgaGVyZS4KPj4gK8KgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoCAqCj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgKiBNZXJnaW5n IG9mIENvbXBvdW5kUGFnZXMgaXMgbm90IHN1cHBvcnRlZDsgdGhleQo+PiArwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgICogc2hvdWxkIGJlY29tZSBzcGxpdHRpbmcgZmlyc3QsIHVubWFwcGVkLCBt ZXJnZWQsCj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgKiBhbmQgbWFwcGVkIGJhY2sgaW4g b24tZGVtYW5kLgo+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgICovCj4+ICvCoMKgwqDCoMKg wqDCoMKgwqDCoMKgIFdBUk5fT05fT05DRShwbWRfcGZuKG9sZF9wbWQpICE9IHBtZF9wZm4oKm5l d19wbWQpKTsKPj4gKwo+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBwbWRfY2xlYXIocG1kKTsK Pj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqAga3ZtX3RsYl9mbHVzaF92bWlkX2lwYShrdm0sIGFk ZHIpOwo+PiArwqDCoMKgwqDCoMKgwqAgfQo+PiDCoMKgwqDCoMKgIH0gZWxzZSB7Cj4+IMKgwqDC oMKgwqDCoMKgwqDCoCBnZXRfcGFnZSh2aXJ0X3RvX3BhZ2UocG1kKSk7Cj4+IMKgwqDCoMKgwqAg fQo+PiBAQCAtMTEyMiw4ICsxMTM2LDEzIEBAIHN0YXRpYyBpbnQgc3RhZ2UyX3NldF9wdWRfaHVn ZShzdHJ1Y3Qga3ZtICprdm0sIHN0cnVjdCBrdm1fbW11X21lbW9yeV9jYWNoZSAqY2FjCj4+IMKg wqDCoMKgwqDCoMKgwqDCoCByZXR1cm4gMDsKPj4gwqAgwqDCoMKgwqDCoCBpZiAoc3RhZ2UyX3B1 ZF9wcmVzZW50KGt2bSwgb2xkX3B1ZCkpIHsKPj4gLcKgwqDCoMKgwqDCoMKgIHN0YWdlMl9wdWRf Y2xlYXIoa3ZtLCBwdWRwKTsKPj4gLcKgwqDCoMKgwqDCoMKgIGt2bV90bGJfZmx1c2hfdm1pZF9p cGEoa3ZtLCBhZGRyKTsKPj4gK8KgwqDCoMKgwqDCoMKgIC8qIElmIHdlIGhhdmUgUFRFIGxldmVs IG1hcHBpbmcsIHVubWFwIHRoZSBlbnRpcmUgcmFuZ2UgKi8KPj4gK8KgwqDCoMKgwqDCoMKgIGlm IChXQVJOX09OX09OQ0UoIXN0YWdlMl9wdWRfaHVnZShrdm0sIG9sZF9wdWQpKSkgewo+PiArwqDC oMKgwqDCoMKgwqDCoMKgwqDCoCB1bm1hcF9zdGFnZTJfcmFuZ2Uoa3ZtLCBhZGRyICYgUzJfUFVE X01BU0ssIFMyX1BVRF9TSVpFKTsKPj4gK8KgwqDCoMKgwqDCoMKgIH0gZWxzZSB7Cj4+ICvCoMKg wqDCoMKgwqDCoMKgwqDCoMKgIHN0YWdlMl9wdWRfY2xlYXIoa3ZtLCBwdWRwKTsKPj4gK8KgwqDC oMKgwqDCoMKgwqDCoMKgwqAga3ZtX3RsYl9mbHVzaF92bWlkX2lwYShrdm0sIGFkZHIpOwo+PiAr wqDCoMKgwqDCoMKgwqAgfQo+PiDCoMKgwqDCoMKgIH0gZWxzZSB7Cj4+IMKgwqDCoMKgwqDCoMKg wqDCoCBnZXRfcGFnZSh2aXJ0X3RvX3BhZ2UocHVkcCkpOwo+PiDCoMKgwqDCoMKgIH0KPj4KPiAK PiAKPiAuCi0tIAoKVGhhbmtzLApYaWFuZwoKCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxpbmcgbGlzdApsaW51eC1h cm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcv bWFpbG1hbi9saXN0aW5mby9saW51eC1hcm0ta2VybmVsCg==