From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E00FC43381 for ; Sun, 17 Mar 2019 14:02:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EEC972087C for ; Sun, 17 Mar 2019 14:02:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727161AbfCQOCg (ORCPT ); Sun, 17 Mar 2019 10:02:36 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:5265 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726563AbfCQOCg (ORCPT ); Sun, 17 Mar 2019 10:02:36 -0400 Received: from DGGEMS411-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id F0911A9936BDD7847668; Sun, 17 Mar 2019 22:02:31 +0800 (CST) Received: from [127.0.0.1] (10.184.12.158) by DGGEMS411-HUB.china.huawei.com (10.3.19.211) with Microsoft SMTP Server id 14.3.408.0; Sun, 17 Mar 2019 22:02:23 +0800 Subject: Re: [RFC] Question about TLB flush while set Stage-2 huge pages To: Suzuki K Poulose , CC: , , , , , , , , , , References: <5f712cc6-0874-adbe-add6-46f5de24f36f@huawei.com> <1c0e07b9-73f0-efa4-c1b7-ad81789b42c5@huawei.com> <5188e3b9-5b5a-a6a7-7ef0-09b7b4f06af6@arm.com> <348d0b3b-c74b-7b39-ec30-85905c077c38@huawei.com> <20190314105537.GA15323@en101> <368bd218-ac1d-19b2-6e92-960b91afee8b@huawei.com> <6aea4049-7860-7144-a7be-14f856cdc789@arm.com> From: Zenghui Yu Message-ID: <0fce42bd-1295-45a4-46b6-81b128ee343e@huawei.com> Date: Sun, 17 Mar 2019 21:55:12 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:64.0) Gecko/20100101 Thunderbird/64.0 MIME-Version: 1.0 In-Reply-To: <6aea4049-7860-7144-a7be-14f856cdc789@arm.com> Content-Type: text/plain; charset="utf-8"; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Originating-IP: [10.184.12.158] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Suzuki, On 2019/3/15 22:56, Suzuki K Poulose wrote: > Hi Zhengui, > > On 15/03/2019 08:21, Zheng Xiang wrote: >> Hi Suzuki, >> >> I have tested this patch, VM doesn't hang and we get expected WARNING >> log: > > Thanks for the quick testing ! > >> However, we also get the following unexpected log: >> >> [  908.329900] BUG: Bad page state in process qemu-kvm  pfn:a2fb41cf >> [  908.339415] page:ffff7e28bed073c0 count:-4 mapcount:0 >> mapping:0000000000000000 index:0x0 >> [  908.339416] flags: 0x4ffffe0000000000() >> [  908.339418] raw: 4ffffe0000000000 dead000000000100 dead000000000200 >> 0000000000000000 >> [  908.339419] raw: 0000000000000000 0000000000000000 fffffffcffffffff >> 0000000000000000 >> [  908.339420] page dumped because: nonzero _refcount >> [  908.339437] CPU: 32 PID: 72599 Comm: qemu-kvm Kdump: loaded >> Tainted: G    B  W        5.0.0+ #1 >> [  908.339438] Call trace: >> [  908.339439]  dump_backtrace+0x0/0x188 >> [  908.339441]  show_stack+0x24/0x30 >> [  908.339442]  dump_stack+0xa8/0xcc >> [  908.339443]  bad_page+0xf0/0x150 >> [  908.339445]  free_pages_check_bad+0x84/0xa0 >> [  908.339446]  free_pcppages_bulk+0x4b8/0x750 >> [  908.339448]  free_unref_page_commit+0x13c/0x198 >> [  908.339449]  free_unref_page+0x84/0xa0 >> [  908.339451]  __free_pages+0x58/0x68 >> [  908.339452]  zap_huge_pmd+0x290/0x2d8 >> [  908.339454]  unmap_page_range+0x2b4/0x470 >> [  908.339455]  unmap_single_vma+0x94/0xe8 >> [  908.339457]  unmap_vmas+0x8c/0x108 >> [  908.339458]  exit_mmap+0xd4/0x178 >> [  908.339459]  mmput+0x74/0x180 >> [  908.339460]  do_exit+0x2b4/0x5b0 >> [  908.339462]  do_group_exit+0x3c/0xe0 >> [  908.339463]  __arm64_sys_exit_group+0x24/0x28 >> [  908.339465]  el0_svc_common+0xa0/0x180 >> [  908.339466]  el0_svc_handler+0x38/0x78 >> [  908.339467]  el0_svc+0x8/0xc > > Thats bad, we seem to be making upto 4 unbalanced put_page(). > >>>> --- >>>>    virt/kvm/arm/mmu.c | 51 >>>> +++++++++++++++++++++++++++++++++++---------------- >>>>    1 file changed, 35 insertions(+), 16 deletions(-) >>>> >>>> diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c >>>> index 66e0fbb5..04b0f9b 100644 >>>> --- a/virt/kvm/arm/mmu.c >>>> +++ b/virt/kvm/arm/mmu.c >>>> @@ -1076,24 +1076,38 @@ static int stage2_set_pmd_huge(struct kvm >>>> *kvm, struct kvm_mmu_memory_cache >>>>             * Skip updating the page table if the entry is >>>>             * unchanged. >>>>             */ >>>> -        if (pmd_val(old_pmd) == pmd_val(*new_pmd)) >>>> +        if (pmd_val(old_pmd) == pmd_val(*new_pmd)) { >>>>                return 0; >>>> - >>>> +        } else if (WARN_ON_ONCE(!pmd_thp_or_huge(old_pmd))) { >>>>            /* >>>> -         * Mapping in huge pages should only happen through a >>>> -         * fault.  If a page is merged into a transparent huge >>>> -         * page, the individual subpages of that huge page >>>> -         * should be unmapped through MMU notifiers before we >>>> -         * get here. >>>> -         * >>>> -         * Merging of CompoundPages is not supported; they >>>> -         * should become splitting first, unmapped, merged, >>>> -         * and mapped back in on-demand. >>>> +         * If we have PTE level mapping for this block, >>>> +         * we must unmap it to avoid inconsistent TLB >>>> +         * state. We could end up in this situation if >>>> +         * the memory slot was marked for dirty logging >>>> +         * and was reverted, leaving PTE level mappings >>>> +         * for the pages accessed during the period. >>>> +         * Normal THP split/merge follows mmu_notifier >>>> +         * callbacks and do get handled accordingly. >>>>             */ >>>> -        VM_BUG_ON(pmd_pfn(old_pmd) != pmd_pfn(*new_pmd)); >>>> +            unmap_stage2_range(kvm, (addr & S2_PMD_MASK), >>>> S2_PMD_SIZE); >> >> It seems that kvm decreases the _refcount of the page twice in >> transparent_hugepage_adjust() >> and unmap_stage2_range(). > > But I thought we should be doing that on the head_page already, as this > is THP. > I will take a look and get back to you on this. Btw, is it possible for you > to turn on CONFIG_DEBUG_VM and re-run with the above patch ? And for detailed debugging info: I've turned on CONFIG_DEBUG_VM and re-run with your patch -- Run a guest with stage2 PUD hugepage, enable then disable the dirty logging, and then shutdown this guest. The result is: Host hit a kernel BUG with the below log (when shutdown-ing guest): [ 486.997640] kernel BUG at ./include/linux/mm.h:547! [ 487.005524] Internal error: Oops - BUG: 0 [#1] SMP [ 487.013455] Modules linked in: ... [ 487.104072] CPU: 14 PID: 60747 Comm: qemu-kvm Kdump: loaded Tainted: G W 5.0.0+ #2 [ 487.117150] ... [ 487.135433] pstate: 40400009 (nZcv daif +PAN -UAO) [ 487.144849] pc : unmap_stage2_puds+0x480/0x6e0 [ 487.153756] lr : unmap_stage2_puds+0x480/0x6e0 [ 487.162507] sp : ffff00002c72bb10 [ 487.179630] x27: 0000000041a00000 x26: ffff8027bbb56060 [ 487.183465] openvswitch: netlink: Tunnel attr 5 has unexpected len 1 expected 0 [ 487.189184] x25: ffff802769cbe008 x24: ffff7e0000000000 [ 487.189185] x23: ffff802769cbe008 x22: ffff00004b0af000 [ 487.189186] x21: ffff80279da06060 x20: 00400027332007fd [ 487.189188] x19: 0000000080000000 x18: 0000000000000010 [ 487.189189] x17: 0000000000000000 x16: 0000000000000000 [ 487.189190] x15: ffff00001182d708 x14: 3030303030303030 [ 487.189191] x13: 3030303030302066 x12: ffff000011857000 [ 487.189192] x11: 0000000000000000 x10: ffff000011a48000 [ 487.189193] x9 : 0000000000000000 x8 : 0000000000000003 [ 487.189194] x7 : 000000000000095b x6 : 0000000212557560 [ 487.189196] x3 : ffff802fc0b08260 x2 : b2513adc3568f800 [ 487.189197] x1 : 0000000000000000 x0 : 000000000000003e [ 487.189200] Process qemu-kvm (pid: 60747, stack limit = 0x000000004342b298) [ 487.189201] Call trace: [ 487.189203] unmap_stage2_puds+0x480/0x6e0 [ 487.189205] unmap_stage2_range+0xa4/0x190 [ 487.189208] kvm_free_stage2_pgd+0x64/0x100 [ 487.363897] kvm_arch_flush_shadow_all+0x20/0x30 [ 487.372095] kvm_mmu_notifier_release+0x3c/0x80 [ 487.380092] __mmu_notifier_release+0x50/0x100 [ 487.387914] exit_mmap+0x170/0x178 [ 487.394567] mmput+0x70/0x180 [ 487.400653] do_exit+0x2b4/0x5c8 [ 487.406849] do_group_exit+0x3c/0xe0 --[ end trace 55c414a329c80b63 ]--- [ 487.454174] Kernel panic - not syncing: Fatal exception [ 487.461967] SMP: stopping secondary CPUs [ 487.468473] Kernel Offset: disabled [ 487.474520] CPU features: 0x002,22208a38 [ 487.481008] Memory Limit: none [ 487.489095] Starting crashdump kernel... [ 487.495457] Bye! > > Kind regards > Suzuki > > > > . From mboxrd@z Thu Jan 1 00:00:00 1970 From: Zenghui Yu Subject: Re: [RFC] Question about TLB flush while set Stage-2 huge pages Date: Sun, 17 Mar 2019 21:55:12 +0800 Message-ID: <0fce42bd-1295-45a4-46b6-81b128ee343e@huawei.com> References: <5f712cc6-0874-adbe-add6-46f5de24f36f@huawei.com> <1c0e07b9-73f0-efa4-c1b7-ad81789b42c5@huawei.com> <5188e3b9-5b5a-a6a7-7ef0-09b7b4f06af6@arm.com> <348d0b3b-c74b-7b39-ec30-85905c077c38@huawei.com> <20190314105537.GA15323@en101> <368bd218-ac1d-19b2-6e92-960b91afee8b@huawei.com> <6aea4049-7860-7144-a7be-14f856cdc789@arm.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <6aea4049-7860-7144-a7be-14f856cdc789@arm.com> Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org To: Suzuki K Poulose , zhengxiang9@huawei.com Cc: marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, christoffer.dall@arm.com, linux-kernel@vger.kernel.org, james.morse@arm.com, lishuo1@hisilicon.com, wanghaibin.wang@huawei.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, lious.lilei@hisilicon.com List-Id: kvmarm@lists.cs.columbia.edu SGkgU3V6dWtpLAoKT24gMjAxOS8zLzE1IDIyOjU2LCBTdXp1a2kgSyBQb3Vsb3NlIHdyb3RlOgo+ IEhpIFpoZW5ndWksCj4gCj4gT24gMTUvMDMvMjAxOSAwODoyMSwgWmhlbmcgWGlhbmcgd3JvdGU6 Cj4+IEhpIFN1enVraSwKPj4KPj4gSSBoYXZlIHRlc3RlZCB0aGlzIHBhdGNoLCBWTSBkb2Vzbid0 IGhhbmcgYW5kIHdlIGdldCBleHBlY3RlZCBXQVJOSU5HIAo+PiBsb2c6Cj4gCj4gVGhhbmtzIGZv ciB0aGUgcXVpY2sgdGVzdGluZyAhCj4gCj4+IEhvd2V2ZXIsIHdlIGFsc28gZ2V0IHRoZSBmb2xs b3dpbmcgdW5leHBlY3RlZCBsb2c6Cj4+Cj4+IFvCoCA5MDguMzI5OTAwXSBCVUc6IEJhZCBwYWdl IHN0YXRlIGluIHByb2Nlc3MgcWVtdS1rdm3CoCBwZm46YTJmYjQxY2YKPj4gW8KgIDkwOC4zMzk0 MTVdIHBhZ2U6ZmZmZjdlMjhiZWQwNzNjMCBjb3VudDotNCBtYXBjb3VudDowIAo+PiBtYXBwaW5n OjAwMDAwMDAwMDAwMDAwMDAgaW5kZXg6MHgwCj4+IFvCoCA5MDguMzM5NDE2XSBmbGFnczogMHg0 ZmZmZmUwMDAwMDAwMDAwKCkKPj4gW8KgIDkwOC4zMzk0MThdIHJhdzogNGZmZmZlMDAwMDAwMDAw MCBkZWFkMDAwMDAwMDAwMTAwIGRlYWQwMDAwMDAwMDAyMDAgCj4+IDAwMDAwMDAwMDAwMDAwMDAK Pj4gW8KgIDkwOC4zMzk0MTldIHJhdzogMDAwMDAwMDAwMDAwMDAwMCAwMDAwMDAwMDAwMDAwMDAw IGZmZmZmZmZjZmZmZmZmZmYgCj4+IDAwMDAwMDAwMDAwMDAwMDAKPj4gW8KgIDkwOC4zMzk0MjBd IHBhZ2UgZHVtcGVkIGJlY2F1c2U6IG5vbnplcm8gX3JlZmNvdW50Cj4+IFvCoCA5MDguMzM5NDM3 XSBDUFU6IDMyIFBJRDogNzI1OTkgQ29tbTogcWVtdS1rdm0gS2R1bXA6IGxvYWRlZCAKPj4gVGFp bnRlZDogR8KgwqDCoCBCwqAgV8KgwqDCoMKgwqDCoMKgIDUuMC4wKyAjMQo+PiBbwqAgOTA4LjMz OTQzOF0gQ2FsbCB0cmFjZToKPj4gW8KgIDkwOC4zMzk0MzldwqAgZHVtcF9iYWNrdHJhY2UrMHgw LzB4MTg4Cj4+IFvCoCA5MDguMzM5NDQxXcKgIHNob3dfc3RhY2srMHgyNC8weDMwCj4+IFvCoCA5 MDguMzM5NDQyXcKgIGR1bXBfc3RhY2srMHhhOC8weGNjCj4+IFvCoCA5MDguMzM5NDQzXcKgIGJh ZF9wYWdlKzB4ZjAvMHgxNTAKPj4gW8KgIDkwOC4zMzk0NDVdwqAgZnJlZV9wYWdlc19jaGVja19i YWQrMHg4NC8weGEwCj4+IFvCoCA5MDguMzM5NDQ2XcKgIGZyZWVfcGNwcGFnZXNfYnVsaysweDRi OC8weDc1MAo+PiBbwqAgOTA4LjMzOTQ0OF3CoCBmcmVlX3VucmVmX3BhZ2VfY29tbWl0KzB4MTNj LzB4MTk4Cj4+IFvCoCA5MDguMzM5NDQ5XcKgIGZyZWVfdW5yZWZfcGFnZSsweDg0LzB4YTAKPj4g W8KgIDkwOC4zMzk0NTFdwqAgX19mcmVlX3BhZ2VzKzB4NTgvMHg2OAo+PiBbwqAgOTA4LjMzOTQ1 Ml3CoCB6YXBfaHVnZV9wbWQrMHgyOTAvMHgyZDgKPj4gW8KgIDkwOC4zMzk0NTRdwqAgdW5tYXBf cGFnZV9yYW5nZSsweDJiNC8weDQ3MAo+PiBbwqAgOTA4LjMzOTQ1NV3CoCB1bm1hcF9zaW5nbGVf dm1hKzB4OTQvMHhlOAo+PiBbwqAgOTA4LjMzOTQ1N13CoCB1bm1hcF92bWFzKzB4OGMvMHgxMDgK Pj4gW8KgIDkwOC4zMzk0NThdwqAgZXhpdF9tbWFwKzB4ZDQvMHgxNzgKPj4gW8KgIDkwOC4zMzk0 NTldwqAgbW1wdXQrMHg3NC8weDE4MAo+PiBbwqAgOTA4LjMzOTQ2MF3CoCBkb19leGl0KzB4MmI0 LzB4NWIwCj4+IFvCoCA5MDguMzM5NDYyXcKgIGRvX2dyb3VwX2V4aXQrMHgzYy8weGUwCj4+IFvC oCA5MDguMzM5NDYzXcKgIF9fYXJtNjRfc3lzX2V4aXRfZ3JvdXArMHgyNC8weDI4Cj4+IFvCoCA5 MDguMzM5NDY1XcKgIGVsMF9zdmNfY29tbW9uKzB4YTAvMHgxODAKPj4gW8KgIDkwOC4zMzk0NjZd wqAgZWwwX3N2Y19oYW5kbGVyKzB4MzgvMHg3OAo+PiBbwqAgOTA4LjMzOTQ2N13CoCBlbDBfc3Zj KzB4OC8weGMKPiAKPiBUaGF0cyBiYWQsIHdlIHNlZW0gdG8gYmUgbWFraW5nIHVwdG8gNCB1bmJh bGFuY2VkIHB1dF9wYWdlKCkuCj4gCj4+Pj4gLS0tCj4+Pj4gwqDCoCB2aXJ0L2t2bS9hcm0vbW11 LmMgfCA1MSAKPj4+PiArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKy0tLS0tLS0t LS0tLS0tLS0KPj4+PiDCoMKgIDEgZmlsZSBjaGFuZ2VkLCAzNSBpbnNlcnRpb25zKCspLCAxNiBk ZWxldGlvbnMoLSkKPj4+Pgo+Pj4+IGRpZmYgLS1naXQgYS92aXJ0L2t2bS9hcm0vbW11LmMgYi92 aXJ0L2t2bS9hcm0vbW11LmMKPj4+PiBpbmRleCA2NmUwZmJiNS4uMDRiMGY5YiAxMDA2NDQKPj4+ PiAtLS0gYS92aXJ0L2t2bS9hcm0vbW11LmMKPj4+PiArKysgYi92aXJ0L2t2bS9hcm0vbW11LmMK Pj4+PiBAQCAtMTA3NiwyNCArMTA3NiwzOCBAQCBzdGF0aWMgaW50IHN0YWdlMl9zZXRfcG1kX2h1 Z2Uoc3RydWN0IGt2bSAKPj4+PiAqa3ZtLCBzdHJ1Y3Qga3ZtX21tdV9tZW1vcnlfY2FjaGUKPj4+ PiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgICogU2tpcCB1cGRhdGluZyB0aGUgcGFnZSB0YWJsZSBp ZiB0aGUgZW50cnkgaXMKPj4+PiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgICogdW5jaGFuZ2VkLgo+ Pj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgKi8KPj4+PiAtwqDCoMKgwqDCoMKgwqAgaWYgKHBt ZF92YWwob2xkX3BtZCkgPT0gcG1kX3ZhbCgqbmV3X3BtZCkpCj4+Pj4gK8KgwqDCoMKgwqDCoMKg IGlmIChwbWRfdmFsKG9sZF9wbWQpID09IHBtZF92YWwoKm5ld19wbWQpKSB7Cj4+Pj4gwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCByZXR1cm4gMDsKPj4+PiAtCj4+Pj4gK8KgwqDCoMKgwqDC oMKgIH0gZWxzZSBpZiAoV0FSTl9PTl9PTkNFKCFwbWRfdGhwX29yX2h1Z2Uob2xkX3BtZCkpKSB7 Cj4+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgwqAgLyoKPj4+PiAtwqDCoMKgwqDCoMKgwqDCoCAqIE1h cHBpbmcgaW4gaHVnZSBwYWdlcyBzaG91bGQgb25seSBoYXBwZW4gdGhyb3VnaCBhCj4+Pj4gLcKg wqDCoMKgwqDCoMKgwqAgKiBmYXVsdC7CoCBJZiBhIHBhZ2UgaXMgbWVyZ2VkIGludG8gYSB0cmFu c3BhcmVudCBodWdlCj4+Pj4gLcKgwqDCoMKgwqDCoMKgwqAgKiBwYWdlLCB0aGUgaW5kaXZpZHVh bCBzdWJwYWdlcyBvZiB0aGF0IGh1Z2UgcGFnZQo+Pj4+IC3CoMKgwqDCoMKgwqDCoMKgICogc2hv dWxkIGJlIHVubWFwcGVkIHRocm91Z2ggTU1VIG5vdGlmaWVycyBiZWZvcmUgd2UKPj4+PiAtwqDC oMKgwqDCoMKgwqDCoCAqIGdldCBoZXJlLgo+Pj4+IC3CoMKgwqDCoMKgwqDCoMKgICoKPj4+PiAt wqDCoMKgwqDCoMKgwqDCoCAqIE1lcmdpbmcgb2YgQ29tcG91bmRQYWdlcyBpcyBub3Qgc3VwcG9y dGVkOyB0aGV5Cj4+Pj4gLcKgwqDCoMKgwqDCoMKgwqAgKiBzaG91bGQgYmVjb21lIHNwbGl0dGlu ZyBmaXJzdCwgdW5tYXBwZWQsIG1lcmdlZCwKPj4+PiAtwqDCoMKgwqDCoMKgwqDCoCAqIGFuZCBt YXBwZWQgYmFjayBpbiBvbi1kZW1hbmQuCj4+Pj4gK8KgwqDCoMKgwqDCoMKgwqAgKiBJZiB3ZSBo YXZlIFBURSBsZXZlbCBtYXBwaW5nIGZvciB0aGlzIGJsb2NrLAo+Pj4+ICvCoMKgwqDCoMKgwqDC oMKgICogd2UgbXVzdCB1bm1hcCBpdCB0byBhdm9pZCBpbmNvbnNpc3RlbnQgVExCCj4+Pj4gK8Kg wqDCoMKgwqDCoMKgwqAgKiBzdGF0ZS4gV2UgY291bGQgZW5kIHVwIGluIHRoaXMgc2l0dWF0aW9u IGlmCj4+Pj4gK8KgwqDCoMKgwqDCoMKgwqAgKiB0aGUgbWVtb3J5IHNsb3Qgd2FzIG1hcmtlZCBm b3IgZGlydHkgbG9nZ2luZwo+Pj4+ICvCoMKgwqDCoMKgwqDCoMKgICogYW5kIHdhcyByZXZlcnRl ZCwgbGVhdmluZyBQVEUgbGV2ZWwgbWFwcGluZ3MKPj4+PiArwqDCoMKgwqDCoMKgwqDCoCAqIGZv ciB0aGUgcGFnZXMgYWNjZXNzZWQgZHVyaW5nIHRoZSBwZXJpb2QuCj4+Pj4gK8KgwqDCoMKgwqDC oMKgwqAgKiBOb3JtYWwgVEhQIHNwbGl0L21lcmdlIGZvbGxvd3MgbW11X25vdGlmaWVyCj4+Pj4g K8KgwqDCoMKgwqDCoMKgwqAgKiBjYWxsYmFja3MgYW5kIGRvIGdldCBoYW5kbGVkIGFjY29yZGlu Z2x5Lgo+Pj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgKi8KPj4+PiAtwqDCoMKgwqDCoMKgwqAg Vk1fQlVHX09OKHBtZF9wZm4ob2xkX3BtZCkgIT0gcG1kX3BmbigqbmV3X3BtZCkpOwo+Pj4+ICvC oMKgwqDCoMKgwqDCoMKgwqDCoMKgIHVubWFwX3N0YWdlMl9yYW5nZShrdm0sIChhZGRyICYgUzJf UE1EX01BU0spLCAKPj4+PiBTMl9QTURfU0laRSk7Cj4+Cj4+IEl0IHNlZW1zIHRoYXQga3ZtIGRl Y3JlYXNlcyB0aGUgX3JlZmNvdW50IG9mIHRoZSBwYWdlIHR3aWNlIGluIAo+PiB0cmFuc3BhcmVu dF9odWdlcGFnZV9hZGp1c3QoKQo+PiBhbmQgdW5tYXBfc3RhZ2UyX3JhbmdlKCkuCj4gCj4gQnV0 IEkgdGhvdWdodCB3ZSBzaG91bGQgYmUgZG9pbmcgdGhhdCBvbiB0aGUgaGVhZF9wYWdlIGFscmVh ZHksIGFzIHRoaXMgCj4gaXMgVEhQLgo+IEkgd2lsbCB0YWtlIGEgbG9vayBhbmQgZ2V0IGJhY2sg dG8geW91IG9uIHRoaXMuIEJ0dywgaXMgaXQgcG9zc2libGUgZm9yIHlvdQo+IHRvIHR1cm4gb24g Q09ORklHX0RFQlVHX1ZNIGFuZCByZS1ydW4gd2l0aCB0aGUgYWJvdmUgcGF0Y2ggPwoKQW5kIGZv ciBkZXRhaWxlZCBkZWJ1Z2dpbmcgaW5mbzoKCkkndmUgdHVybmVkIG9uIENPTkZJR19ERUJVR19W TSBhbmQgcmUtcnVuIHdpdGggeW91ciBwYXRjaCAtLSBSdW4gYSBndWVzdAp3aXRoIHN0YWdlMiBQ VUQgaHVnZXBhZ2UsIGVuYWJsZSB0aGVuIGRpc2FibGUgdGhlIGRpcnR5IGxvZ2dpbmcsIGFuZAp0 aGVuIHNodXRkb3duIHRoaXMgZ3Vlc3QuIFRoZSByZXN1bHQgaXM6IEhvc3QgaGl0IGEga2VybmVs IEJVRyB3aXRoIHRoZQpiZWxvdyBsb2cgKHdoZW4gc2h1dGRvd24taW5nIGd1ZXN0KToKCgpbICA0 ODYuOTk3NjQwXSBrZXJuZWwgQlVHIGF0IC4vaW5jbHVkZS9saW51eC9tbS5oOjU0NyEKWyAgNDg3 LjAwNTUyNF0gSW50ZXJuYWwgZXJyb3I6IE9vcHMgLSBCVUc6IDAgWyMxXSBTTVAKWyAgNDg3LjAx MzQ1NV0gTW9kdWxlcyBsaW5rZWQgaW46IC4uLgpbICA0ODcuMTA0MDcyXSBDUFU6IDE0IFBJRDog NjA3NDcgQ29tbTogcWVtdS1rdm0gS2R1bXA6IGxvYWRlZCBUYWludGVkOiAKRyAgICAgICAgVyAg ICAgICAgIDUuMC4wKyAjMgpbICA0ODcuMTE3MTUwXSAuLi4KWyAgNDg3LjEzNTQzM10gcHN0YXRl OiA0MDQwMDAwOSAoblpjdiBkYWlmICtQQU4gLVVBTykKWyAgNDg3LjE0NDg0OV0gcGMgOiB1bm1h cF9zdGFnZTJfcHVkcysweDQ4MC8weDZlMApbICA0ODcuMTUzNzU2XSBsciA6IHVubWFwX3N0YWdl Ml9wdWRzKzB4NDgwLzB4NmUwClsgIDQ4Ny4xNjI1MDddIHNwIDogZmZmZjAwMDAyYzcyYmIxMApb ICA0ODcuMTc5NjMwXSB4Mjc6IDAwMDAwMDAwNDFhMDAwMDAgeDI2OiBmZmZmODAyN2JiYjU2MDYw ClsgIDQ4Ny4xODM0NjVdIG9wZW52c3dpdGNoOiBuZXRsaW5rOiBUdW5uZWwgYXR0ciA1IGhhcyB1 bmV4cGVjdGVkIGxlbiAxIApleHBlY3RlZCAwClsgIDQ4Ny4xODkxODRdIHgyNTogZmZmZjgwMjc2 OWNiZTAwOCB4MjQ6IGZmZmY3ZTAwMDAwMDAwMDAKWyAgNDg3LjE4OTE4NV0geDIzOiBmZmZmODAy NzY5Y2JlMDA4IHgyMjogZmZmZjAwMDA0YjBhZjAwMApbICA0ODcuMTg5MTg2XSB4MjE6IGZmZmY4 MDI3OWRhMDYwNjAgeDIwOiAwMDQwMDAyNzMzMjAwN2ZkClsgIDQ4Ny4xODkxODhdIHgxOTogMDAw MDAwMDA4MDAwMDAwMCB4MTg6IDAwMDAwMDAwMDAwMDAwMTAKWyAgNDg3LjE4OTE4OV0geDE3OiAw MDAwMDAwMDAwMDAwMDAwIHgxNjogMDAwMDAwMDAwMDAwMDAwMApbICA0ODcuMTg5MTkwXSB4MTU6 IGZmZmYwMDAwMTE4MmQ3MDggeDE0OiAzMDMwMzAzMDMwMzAzMDMwClsgIDQ4Ny4xODkxOTFdIHgx MzogMzAzMDMwMzAzMDMwMjA2NiB4MTI6IGZmZmYwMDAwMTE4NTcwMDAKWyAgNDg3LjE4OTE5Ml0g eDExOiAwMDAwMDAwMDAwMDAwMDAwIHgxMDogZmZmZjAwMDAxMWE0ODAwMApbICA0ODcuMTg5MTkz XSB4OSA6IDAwMDAwMDAwMDAwMDAwMDAgeDggOiAwMDAwMDAwMDAwMDAwMDAzClsgIDQ4Ny4xODkx OTRdIHg3IDogMDAwMDAwMDAwMDAwMDk1YiB4NiA6IDAwMDAwMDAyMTI1NTc1NjAKWyAgNDg3LjE4 OTE5Nl0geDMgOiBmZmZmODAyZmMwYjA4MjYwIHgyIDogYjI1MTNhZGMzNTY4ZjgwMApbICA0ODcu MTg5MTk3XSB4MSA6IDAwMDAwMDAwMDAwMDAwMDAgeDAgOiAwMDAwMDAwMDAwMDAwMDNlClsgIDQ4 Ny4xODkyMDBdIFByb2Nlc3MgcWVtdS1rdm0gKHBpZDogNjA3NDcsIHN0YWNrIGxpbWl0ID0gCjB4 MDAwMDAwMDA0MzQyYjI5OCkKWyAgNDg3LjE4OTIwMV0gQ2FsbCB0cmFjZToKWyAgNDg3LjE4OTIw M10gIHVubWFwX3N0YWdlMl9wdWRzKzB4NDgwLzB4NmUwClsgIDQ4Ny4xODkyMDVdICB1bm1hcF9z dGFnZTJfcmFuZ2UrMHhhNC8weDE5MApbICA0ODcuMTg5MjA4XSAga3ZtX2ZyZWVfc3RhZ2UyX3Bn ZCsweDY0LzB4MTAwClsgIDQ4Ny4zNjM4OTddICBrdm1fYXJjaF9mbHVzaF9zaGFkb3dfYWxsKzB4 MjAvMHgzMApbICA0ODcuMzcyMDk1XSAga3ZtX21tdV9ub3RpZmllcl9yZWxlYXNlKzB4M2MvMHg4 MApbICA0ODcuMzgwMDkyXSAgX19tbXVfbm90aWZpZXJfcmVsZWFzZSsweDUwLzB4MTAwClsgIDQ4 Ny4zODc5MTRdICBleGl0X21tYXArMHgxNzAvMHgxNzgKWyAgNDg3LjM5NDU2N10gIG1tcHV0KzB4 NzAvMHgxODAKWyAgNDg3LjQwMDY1M10gIGRvX2V4aXQrMHgyYjQvMHg1YzgKWyAgNDg3LjQwNjg0 OV0gIGRvX2dyb3VwX2V4aXQrMHgzYy8weGUwCi0tWyBlbmQgdHJhY2UgNTVjNDE0YTMyOWM4MGI2 MyBdLS0tClsgIDQ4Ny40NTQxNzRdIEtlcm5lbCBwYW5pYyAtIG5vdCBzeW5jaW5nOiBGYXRhbCBl eGNlcHRpb24KWyAgNDg3LjQ2MTk2N10gU01QOiBzdG9wcGluZyBzZWNvbmRhcnkgQ1BVcwpbICA0 ODcuNDY4NDczXSBLZXJuZWwgT2Zmc2V0OiBkaXNhYmxlZApbICA0ODcuNDc0NTIwXSBDUFUgZmVh dHVyZXM6IDB4MDAyLDIyMjA4YTM4ClsgIDQ4Ny40ODEwMDhdIE1lbW9yeSBMaW1pdDogbm9uZQpb ICA0ODcuNDg5MDk1XSBTdGFydGluZyBjcmFzaGR1bXAga2VybmVsLi4uClsgIDQ4Ny40OTU0NTdd IEJ5ZSEKCgo+IAo+IEtpbmQgcmVnYXJkcwo+IFN1enVraQo+IAo+IAo+IAo+IC4KCgpfX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpsaW51eC1hcm0ta2VybmVs IG1haWxpbmcgbGlzdApsaW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDov L2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1hcm0ta2VybmVsCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3DDE6C43381 for ; Sun, 17 Mar 2019 14:02:53 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 01B3821019 for ; Sun, 17 Mar 2019 14:02:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="nweWskng" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 01B3821019 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=6yiwqajRV2xp0TU3+mQNLeXxN4WS5bNm28oAjP6+Qp8=; b=nweWskngngOkhp0dUqcE39FnW 2gILkn3CaZGX0xjYFjjWUI75rLAS8JdSsiJ9zzX98wXFLhrdRpvq5bNsrXLfNAplh3qMbuTZ8mcrk djw1lKhcQVQID/BG38Kkp5W10FPfKp/ZnXvjGt2c0OQ8Bl2eockHt62LbLQ1CxsmfhAoaFius7fXi lkf2DFWGDoNZo0+waDjhvljbmbKdpopvYv6Zf/y80FbYIo2NiRo3YSoXzmoVf5l81DITfrEkPNyyj 0NlO+AFaRYBaHac6ApNI1wNeEhRVDXqInhBvbEjqZfEJvAtr6fCxEBS9D5dudYsAUxNoGTuE5MWUU XwFMB5tmQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1h5WN9-0007HC-Ns; Sun, 17 Mar 2019 14:02:43 +0000 Received: from szxga05-in.huawei.com ([45.249.212.191] helo=huawei.com) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1h5WN6-0007Fu-I2 for linux-arm-kernel@lists.infradead.org; Sun, 17 Mar 2019 14:02:42 +0000 Received: from DGGEMS411-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id F0911A9936BDD7847668; Sun, 17 Mar 2019 22:02:31 +0800 (CST) Received: from [127.0.0.1] (10.184.12.158) by DGGEMS411-HUB.china.huawei.com (10.3.19.211) with Microsoft SMTP Server id 14.3.408.0; Sun, 17 Mar 2019 22:02:23 +0800 Subject: Re: [RFC] Question about TLB flush while set Stage-2 huge pages To: Suzuki K Poulose , References: <5f712cc6-0874-adbe-add6-46f5de24f36f@huawei.com> <1c0e07b9-73f0-efa4-c1b7-ad81789b42c5@huawei.com> <5188e3b9-5b5a-a6a7-7ef0-09b7b4f06af6@arm.com> <348d0b3b-c74b-7b39-ec30-85905c077c38@huawei.com> <20190314105537.GA15323@en101> <368bd218-ac1d-19b2-6e92-960b91afee8b@huawei.com> <6aea4049-7860-7144-a7be-14f856cdc789@arm.com> From: Zenghui Yu Message-ID: <0fce42bd-1295-45a4-46b6-81b128ee343e@huawei.com> Date: Sun, 17 Mar 2019 21:55:12 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:64.0) Gecko/20100101 Thunderbird/64.0 MIME-Version: 1.0 In-Reply-To: <6aea4049-7860-7144-a7be-14f856cdc789@arm.com> Content-Language: en-US X-Originating-IP: [10.184.12.158] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190317_070240_960858_C7EC1D5C X-CRM114-Status: GOOD ( 19.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, christoffer.dall@arm.com, linux-kernel@vger.kernel.org, james.morse@arm.com, lishuo1@hisilicon.com, wanghaibin.wang@huawei.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, lious.lilei@hisilicon.com Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org SGkgU3V6dWtpLAoKT24gMjAxOS8zLzE1IDIyOjU2LCBTdXp1a2kgSyBQb3Vsb3NlIHdyb3RlOgo+ IEhpIFpoZW5ndWksCj4gCj4gT24gMTUvMDMvMjAxOSAwODoyMSwgWmhlbmcgWGlhbmcgd3JvdGU6 Cj4+IEhpIFN1enVraSwKPj4KPj4gSSBoYXZlIHRlc3RlZCB0aGlzIHBhdGNoLCBWTSBkb2Vzbid0 IGhhbmcgYW5kIHdlIGdldCBleHBlY3RlZCBXQVJOSU5HIAo+PiBsb2c6Cj4gCj4gVGhhbmtzIGZv ciB0aGUgcXVpY2sgdGVzdGluZyAhCj4gCj4+IEhvd2V2ZXIsIHdlIGFsc28gZ2V0IHRoZSBmb2xs b3dpbmcgdW5leHBlY3RlZCBsb2c6Cj4+Cj4+IFvCoCA5MDguMzI5OTAwXSBCVUc6IEJhZCBwYWdl IHN0YXRlIGluIHByb2Nlc3MgcWVtdS1rdm3CoCBwZm46YTJmYjQxY2YKPj4gW8KgIDkwOC4zMzk0 MTVdIHBhZ2U6ZmZmZjdlMjhiZWQwNzNjMCBjb3VudDotNCBtYXBjb3VudDowIAo+PiBtYXBwaW5n OjAwMDAwMDAwMDAwMDAwMDAgaW5kZXg6MHgwCj4+IFvCoCA5MDguMzM5NDE2XSBmbGFnczogMHg0 ZmZmZmUwMDAwMDAwMDAwKCkKPj4gW8KgIDkwOC4zMzk0MThdIHJhdzogNGZmZmZlMDAwMDAwMDAw MCBkZWFkMDAwMDAwMDAwMTAwIGRlYWQwMDAwMDAwMDAyMDAgCj4+IDAwMDAwMDAwMDAwMDAwMDAK Pj4gW8KgIDkwOC4zMzk0MTldIHJhdzogMDAwMDAwMDAwMDAwMDAwMCAwMDAwMDAwMDAwMDAwMDAw IGZmZmZmZmZjZmZmZmZmZmYgCj4+IDAwMDAwMDAwMDAwMDAwMDAKPj4gW8KgIDkwOC4zMzk0MjBd IHBhZ2UgZHVtcGVkIGJlY2F1c2U6IG5vbnplcm8gX3JlZmNvdW50Cj4+IFvCoCA5MDguMzM5NDM3 XSBDUFU6IDMyIFBJRDogNzI1OTkgQ29tbTogcWVtdS1rdm0gS2R1bXA6IGxvYWRlZCAKPj4gVGFp bnRlZDogR8KgwqDCoCBCwqAgV8KgwqDCoMKgwqDCoMKgIDUuMC4wKyAjMQo+PiBbwqAgOTA4LjMz OTQzOF0gQ2FsbCB0cmFjZToKPj4gW8KgIDkwOC4zMzk0MzldwqAgZHVtcF9iYWNrdHJhY2UrMHgw LzB4MTg4Cj4+IFvCoCA5MDguMzM5NDQxXcKgIHNob3dfc3RhY2srMHgyNC8weDMwCj4+IFvCoCA5 MDguMzM5NDQyXcKgIGR1bXBfc3RhY2srMHhhOC8weGNjCj4+IFvCoCA5MDguMzM5NDQzXcKgIGJh ZF9wYWdlKzB4ZjAvMHgxNTAKPj4gW8KgIDkwOC4zMzk0NDVdwqAgZnJlZV9wYWdlc19jaGVja19i YWQrMHg4NC8weGEwCj4+IFvCoCA5MDguMzM5NDQ2XcKgIGZyZWVfcGNwcGFnZXNfYnVsaysweDRi OC8weDc1MAo+PiBbwqAgOTA4LjMzOTQ0OF3CoCBmcmVlX3VucmVmX3BhZ2VfY29tbWl0KzB4MTNj LzB4MTk4Cj4+IFvCoCA5MDguMzM5NDQ5XcKgIGZyZWVfdW5yZWZfcGFnZSsweDg0LzB4YTAKPj4g W8KgIDkwOC4zMzk0NTFdwqAgX19mcmVlX3BhZ2VzKzB4NTgvMHg2OAo+PiBbwqAgOTA4LjMzOTQ1 Ml3CoCB6YXBfaHVnZV9wbWQrMHgyOTAvMHgyZDgKPj4gW8KgIDkwOC4zMzk0NTRdwqAgdW5tYXBf cGFnZV9yYW5nZSsweDJiNC8weDQ3MAo+PiBbwqAgOTA4LjMzOTQ1NV3CoCB1bm1hcF9zaW5nbGVf dm1hKzB4OTQvMHhlOAo+PiBbwqAgOTA4LjMzOTQ1N13CoCB1bm1hcF92bWFzKzB4OGMvMHgxMDgK Pj4gW8KgIDkwOC4zMzk0NThdwqAgZXhpdF9tbWFwKzB4ZDQvMHgxNzgKPj4gW8KgIDkwOC4zMzk0 NTldwqAgbW1wdXQrMHg3NC8weDE4MAo+PiBbwqAgOTA4LjMzOTQ2MF3CoCBkb19leGl0KzB4MmI0 LzB4NWIwCj4+IFvCoCA5MDguMzM5NDYyXcKgIGRvX2dyb3VwX2V4aXQrMHgzYy8weGUwCj4+IFvC oCA5MDguMzM5NDYzXcKgIF9fYXJtNjRfc3lzX2V4aXRfZ3JvdXArMHgyNC8weDI4Cj4+IFvCoCA5 MDguMzM5NDY1XcKgIGVsMF9zdmNfY29tbW9uKzB4YTAvMHgxODAKPj4gW8KgIDkwOC4zMzk0NjZd wqAgZWwwX3N2Y19oYW5kbGVyKzB4MzgvMHg3OAo+PiBbwqAgOTA4LjMzOTQ2N13CoCBlbDBfc3Zj KzB4OC8weGMKPiAKPiBUaGF0cyBiYWQsIHdlIHNlZW0gdG8gYmUgbWFraW5nIHVwdG8gNCB1bmJh bGFuY2VkIHB1dF9wYWdlKCkuCj4gCj4+Pj4gLS0tCj4+Pj4gwqDCoCB2aXJ0L2t2bS9hcm0vbW11 LmMgfCA1MSAKPj4+PiArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKy0tLS0tLS0t LS0tLS0tLS0KPj4+PiDCoMKgIDEgZmlsZSBjaGFuZ2VkLCAzNSBpbnNlcnRpb25zKCspLCAxNiBk ZWxldGlvbnMoLSkKPj4+Pgo+Pj4+IGRpZmYgLS1naXQgYS92aXJ0L2t2bS9hcm0vbW11LmMgYi92 aXJ0L2t2bS9hcm0vbW11LmMKPj4+PiBpbmRleCA2NmUwZmJiNS4uMDRiMGY5YiAxMDA2NDQKPj4+ PiAtLS0gYS92aXJ0L2t2bS9hcm0vbW11LmMKPj4+PiArKysgYi92aXJ0L2t2bS9hcm0vbW11LmMK Pj4+PiBAQCAtMTA3NiwyNCArMTA3NiwzOCBAQCBzdGF0aWMgaW50IHN0YWdlMl9zZXRfcG1kX2h1 Z2Uoc3RydWN0IGt2bSAKPj4+PiAqa3ZtLCBzdHJ1Y3Qga3ZtX21tdV9tZW1vcnlfY2FjaGUKPj4+ PiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgICogU2tpcCB1cGRhdGluZyB0aGUgcGFnZSB0YWJsZSBp ZiB0aGUgZW50cnkgaXMKPj4+PiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgICogdW5jaGFuZ2VkLgo+ Pj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgKi8KPj4+PiAtwqDCoMKgwqDCoMKgwqAgaWYgKHBt ZF92YWwob2xkX3BtZCkgPT0gcG1kX3ZhbCgqbmV3X3BtZCkpCj4+Pj4gK8KgwqDCoMKgwqDCoMKg IGlmIChwbWRfdmFsKG9sZF9wbWQpID09IHBtZF92YWwoKm5ld19wbWQpKSB7Cj4+Pj4gwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCByZXR1cm4gMDsKPj4+PiAtCj4+Pj4gK8KgwqDCoMKgwqDC oMKgIH0gZWxzZSBpZiAoV0FSTl9PTl9PTkNFKCFwbWRfdGhwX29yX2h1Z2Uob2xkX3BtZCkpKSB7 Cj4+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgwqAgLyoKPj4+PiAtwqDCoMKgwqDCoMKgwqDCoCAqIE1h cHBpbmcgaW4gaHVnZSBwYWdlcyBzaG91bGQgb25seSBoYXBwZW4gdGhyb3VnaCBhCj4+Pj4gLcKg wqDCoMKgwqDCoMKgwqAgKiBmYXVsdC7CoCBJZiBhIHBhZ2UgaXMgbWVyZ2VkIGludG8gYSB0cmFu c3BhcmVudCBodWdlCj4+Pj4gLcKgwqDCoMKgwqDCoMKgwqAgKiBwYWdlLCB0aGUgaW5kaXZpZHVh bCBzdWJwYWdlcyBvZiB0aGF0IGh1Z2UgcGFnZQo+Pj4+IC3CoMKgwqDCoMKgwqDCoMKgICogc2hv dWxkIGJlIHVubWFwcGVkIHRocm91Z2ggTU1VIG5vdGlmaWVycyBiZWZvcmUgd2UKPj4+PiAtwqDC oMKgwqDCoMKgwqDCoCAqIGdldCBoZXJlLgo+Pj4+IC3CoMKgwqDCoMKgwqDCoMKgICoKPj4+PiAt wqDCoMKgwqDCoMKgwqDCoCAqIE1lcmdpbmcgb2YgQ29tcG91bmRQYWdlcyBpcyBub3Qgc3VwcG9y dGVkOyB0aGV5Cj4+Pj4gLcKgwqDCoMKgwqDCoMKgwqAgKiBzaG91bGQgYmVjb21lIHNwbGl0dGlu ZyBmaXJzdCwgdW5tYXBwZWQsIG1lcmdlZCwKPj4+PiAtwqDCoMKgwqDCoMKgwqDCoCAqIGFuZCBt YXBwZWQgYmFjayBpbiBvbi1kZW1hbmQuCj4+Pj4gK8KgwqDCoMKgwqDCoMKgwqAgKiBJZiB3ZSBo YXZlIFBURSBsZXZlbCBtYXBwaW5nIGZvciB0aGlzIGJsb2NrLAo+Pj4+ICvCoMKgwqDCoMKgwqDC oMKgICogd2UgbXVzdCB1bm1hcCBpdCB0byBhdm9pZCBpbmNvbnNpc3RlbnQgVExCCj4+Pj4gK8Kg wqDCoMKgwqDCoMKgwqAgKiBzdGF0ZS4gV2UgY291bGQgZW5kIHVwIGluIHRoaXMgc2l0dWF0aW9u IGlmCj4+Pj4gK8KgwqDCoMKgwqDCoMKgwqAgKiB0aGUgbWVtb3J5IHNsb3Qgd2FzIG1hcmtlZCBm b3IgZGlydHkgbG9nZ2luZwo+Pj4+ICvCoMKgwqDCoMKgwqDCoMKgICogYW5kIHdhcyByZXZlcnRl ZCwgbGVhdmluZyBQVEUgbGV2ZWwgbWFwcGluZ3MKPj4+PiArwqDCoMKgwqDCoMKgwqDCoCAqIGZv ciB0aGUgcGFnZXMgYWNjZXNzZWQgZHVyaW5nIHRoZSBwZXJpb2QuCj4+Pj4gK8KgwqDCoMKgwqDC oMKgwqAgKiBOb3JtYWwgVEhQIHNwbGl0L21lcmdlIGZvbGxvd3MgbW11X25vdGlmaWVyCj4+Pj4g K8KgwqDCoMKgwqDCoMKgwqAgKiBjYWxsYmFja3MgYW5kIGRvIGdldCBoYW5kbGVkIGFjY29yZGlu Z2x5Lgo+Pj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgKi8KPj4+PiAtwqDCoMKgwqDCoMKgwqAg Vk1fQlVHX09OKHBtZF9wZm4ob2xkX3BtZCkgIT0gcG1kX3BmbigqbmV3X3BtZCkpOwo+Pj4+ICvC oMKgwqDCoMKgwqDCoMKgwqDCoMKgIHVubWFwX3N0YWdlMl9yYW5nZShrdm0sIChhZGRyICYgUzJf UE1EX01BU0spLCAKPj4+PiBTMl9QTURfU0laRSk7Cj4+Cj4+IEl0IHNlZW1zIHRoYXQga3ZtIGRl Y3JlYXNlcyB0aGUgX3JlZmNvdW50IG9mIHRoZSBwYWdlIHR3aWNlIGluIAo+PiB0cmFuc3BhcmVu dF9odWdlcGFnZV9hZGp1c3QoKQo+PiBhbmQgdW5tYXBfc3RhZ2UyX3JhbmdlKCkuCj4gCj4gQnV0 IEkgdGhvdWdodCB3ZSBzaG91bGQgYmUgZG9pbmcgdGhhdCBvbiB0aGUgaGVhZF9wYWdlIGFscmVh ZHksIGFzIHRoaXMgCj4gaXMgVEhQLgo+IEkgd2lsbCB0YWtlIGEgbG9vayBhbmQgZ2V0IGJhY2sg dG8geW91IG9uIHRoaXMuIEJ0dywgaXMgaXQgcG9zc2libGUgZm9yIHlvdQo+IHRvIHR1cm4gb24g Q09ORklHX0RFQlVHX1ZNIGFuZCByZS1ydW4gd2l0aCB0aGUgYWJvdmUgcGF0Y2ggPwoKQW5kIGZv ciBkZXRhaWxlZCBkZWJ1Z2dpbmcgaW5mbzoKCkkndmUgdHVybmVkIG9uIENPTkZJR19ERUJVR19W TSBhbmQgcmUtcnVuIHdpdGggeW91ciBwYXRjaCAtLSBSdW4gYSBndWVzdAp3aXRoIHN0YWdlMiBQ VUQgaHVnZXBhZ2UsIGVuYWJsZSB0aGVuIGRpc2FibGUgdGhlIGRpcnR5IGxvZ2dpbmcsIGFuZAp0 aGVuIHNodXRkb3duIHRoaXMgZ3Vlc3QuIFRoZSByZXN1bHQgaXM6IEhvc3QgaGl0IGEga2VybmVs IEJVRyB3aXRoIHRoZQpiZWxvdyBsb2cgKHdoZW4gc2h1dGRvd24taW5nIGd1ZXN0KToKCgpbICA0 ODYuOTk3NjQwXSBrZXJuZWwgQlVHIGF0IC4vaW5jbHVkZS9saW51eC9tbS5oOjU0NyEKWyAgNDg3 LjAwNTUyNF0gSW50ZXJuYWwgZXJyb3I6IE9vcHMgLSBCVUc6IDAgWyMxXSBTTVAKWyAgNDg3LjAx MzQ1NV0gTW9kdWxlcyBsaW5rZWQgaW46IC4uLgpbICA0ODcuMTA0MDcyXSBDUFU6IDE0IFBJRDog NjA3NDcgQ29tbTogcWVtdS1rdm0gS2R1bXA6IGxvYWRlZCBUYWludGVkOiAKRyAgICAgICAgVyAg ICAgICAgIDUuMC4wKyAjMgpbICA0ODcuMTE3MTUwXSAuLi4KWyAgNDg3LjEzNTQzM10gcHN0YXRl OiA0MDQwMDAwOSAoblpjdiBkYWlmICtQQU4gLVVBTykKWyAgNDg3LjE0NDg0OV0gcGMgOiB1bm1h cF9zdGFnZTJfcHVkcysweDQ4MC8weDZlMApbICA0ODcuMTUzNzU2XSBsciA6IHVubWFwX3N0YWdl Ml9wdWRzKzB4NDgwLzB4NmUwClsgIDQ4Ny4xNjI1MDddIHNwIDogZmZmZjAwMDAyYzcyYmIxMApb ICA0ODcuMTc5NjMwXSB4Mjc6IDAwMDAwMDAwNDFhMDAwMDAgeDI2OiBmZmZmODAyN2JiYjU2MDYw ClsgIDQ4Ny4xODM0NjVdIG9wZW52c3dpdGNoOiBuZXRsaW5rOiBUdW5uZWwgYXR0ciA1IGhhcyB1 bmV4cGVjdGVkIGxlbiAxIApleHBlY3RlZCAwClsgIDQ4Ny4xODkxODRdIHgyNTogZmZmZjgwMjc2 OWNiZTAwOCB4MjQ6IGZmZmY3ZTAwMDAwMDAwMDAKWyAgNDg3LjE4OTE4NV0geDIzOiBmZmZmODAy NzY5Y2JlMDA4IHgyMjogZmZmZjAwMDA0YjBhZjAwMApbICA0ODcuMTg5MTg2XSB4MjE6IGZmZmY4 MDI3OWRhMDYwNjAgeDIwOiAwMDQwMDAyNzMzMjAwN2ZkClsgIDQ4Ny4xODkxODhdIHgxOTogMDAw MDAwMDA4MDAwMDAwMCB4MTg6IDAwMDAwMDAwMDAwMDAwMTAKWyAgNDg3LjE4OTE4OV0geDE3OiAw MDAwMDAwMDAwMDAwMDAwIHgxNjogMDAwMDAwMDAwMDAwMDAwMApbICA0ODcuMTg5MTkwXSB4MTU6 IGZmZmYwMDAwMTE4MmQ3MDggeDE0OiAzMDMwMzAzMDMwMzAzMDMwClsgIDQ4Ny4xODkxOTFdIHgx MzogMzAzMDMwMzAzMDMwMjA2NiB4MTI6IGZmZmYwMDAwMTE4NTcwMDAKWyAgNDg3LjE4OTE5Ml0g eDExOiAwMDAwMDAwMDAwMDAwMDAwIHgxMDogZmZmZjAwMDAxMWE0ODAwMApbICA0ODcuMTg5MTkz XSB4OSA6IDAwMDAwMDAwMDAwMDAwMDAgeDggOiAwMDAwMDAwMDAwMDAwMDAzClsgIDQ4Ny4xODkx OTRdIHg3IDogMDAwMDAwMDAwMDAwMDk1YiB4NiA6IDAwMDAwMDAyMTI1NTc1NjAKWyAgNDg3LjE4 OTE5Nl0geDMgOiBmZmZmODAyZmMwYjA4MjYwIHgyIDogYjI1MTNhZGMzNTY4ZjgwMApbICA0ODcu MTg5MTk3XSB4MSA6IDAwMDAwMDAwMDAwMDAwMDAgeDAgOiAwMDAwMDAwMDAwMDAwMDNlClsgIDQ4 Ny4xODkyMDBdIFByb2Nlc3MgcWVtdS1rdm0gKHBpZDogNjA3NDcsIHN0YWNrIGxpbWl0ID0gCjB4 MDAwMDAwMDA0MzQyYjI5OCkKWyAgNDg3LjE4OTIwMV0gQ2FsbCB0cmFjZToKWyAgNDg3LjE4OTIw M10gIHVubWFwX3N0YWdlMl9wdWRzKzB4NDgwLzB4NmUwClsgIDQ4Ny4xODkyMDVdICB1bm1hcF9z dGFnZTJfcmFuZ2UrMHhhNC8weDE5MApbICA0ODcuMTg5MjA4XSAga3ZtX2ZyZWVfc3RhZ2UyX3Bn ZCsweDY0LzB4MTAwClsgIDQ4Ny4zNjM4OTddICBrdm1fYXJjaF9mbHVzaF9zaGFkb3dfYWxsKzB4 MjAvMHgzMApbICA0ODcuMzcyMDk1XSAga3ZtX21tdV9ub3RpZmllcl9yZWxlYXNlKzB4M2MvMHg4 MApbICA0ODcuMzgwMDkyXSAgX19tbXVfbm90aWZpZXJfcmVsZWFzZSsweDUwLzB4MTAwClsgIDQ4 Ny4zODc5MTRdICBleGl0X21tYXArMHgxNzAvMHgxNzgKWyAgNDg3LjM5NDU2N10gIG1tcHV0KzB4 NzAvMHgxODAKWyAgNDg3LjQwMDY1M10gIGRvX2V4aXQrMHgyYjQvMHg1YzgKWyAgNDg3LjQwNjg0 OV0gIGRvX2dyb3VwX2V4aXQrMHgzYy8weGUwCi0tWyBlbmQgdHJhY2UgNTVjNDE0YTMyOWM4MGI2 MyBdLS0tClsgIDQ4Ny40NTQxNzRdIEtlcm5lbCBwYW5pYyAtIG5vdCBzeW5jaW5nOiBGYXRhbCBl eGNlcHRpb24KWyAgNDg3LjQ2MTk2N10gU01QOiBzdG9wcGluZyBzZWNvbmRhcnkgQ1BVcwpbICA0 ODcuNDY4NDczXSBLZXJuZWwgT2Zmc2V0OiBkaXNhYmxlZApbICA0ODcuNDc0NTIwXSBDUFUgZmVh dHVyZXM6IDB4MDAyLDIyMjA4YTM4ClsgIDQ4Ny40ODEwMDhdIE1lbW9yeSBMaW1pdDogbm9uZQpb ICA0ODcuNDg5MDk1XSBTdGFydGluZyBjcmFzaGR1bXAga2VybmVsLi4uClsgIDQ4Ny40OTU0NTdd IEJ5ZSEKCgo+IAo+IEtpbmQgcmVnYXJkcwo+IFN1enVraQo+IAo+IAo+IAo+IC4KCgpfX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpsaW51eC1hcm0ta2VybmVs IG1haWxpbmcgbGlzdApsaW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDov L2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1hcm0ta2VybmVsCg==