From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C337EB64D9 for ; Fri, 7 Jul 2023 15:14:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232723AbjGGPOA (ORCPT ); Fri, 7 Jul 2023 11:14:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52908 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229561AbjGGPN6 (ORCPT ); Fri, 7 Jul 2023 11:13:58 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 5E8772115 for ; Fri, 7 Jul 2023 08:13:56 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2F31FD75; Fri, 7 Jul 2023 08:14:38 -0700 (PDT) Received: from [10.57.77.63] (unknown [10.57.77.63]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 091EF3F762; Fri, 7 Jul 2023 08:13:53 -0700 (PDT) Message-ID: <9dd036a8-9ba3-0cc4-b791-cb3178237728@arm.com> Date: Fri, 7 Jul 2023 16:13:52 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: [PATCH v2 4/5] mm: FLEXIBLE_THP for improved performance To: David Hildenbrand , Matthew Wilcox Cc: "Huang, Ying" , Andrew Morton , "Kirill A. Shutemov" , Yin Fengwei , Yu Zhao , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20230703135330.1865927-1-ryan.roberts@arm.com> <20230703135330.1865927-5-ryan.roberts@arm.com> <87edlkgnfa.fsf@yhuang6-desk2.ccr.corp.intel.com> <44e60630-5e9d-c8df-ab79-cb0767de680e@arm.com> <524bacd2-4a47-2b8b-6685-c46e31a01631@redhat.com> <1e406f04-78ef-6573-e1f1-f0d0e0d5246a@redhat.com> From: Ryan Roberts In-Reply-To: <1e406f04-78ef-6573-e1f1-f0d0e0d5246a@redhat.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 07/07/2023 15:07, David Hildenbrand wrote: > On 07.07.23 15:57, Matthew Wilcox wrote: >> On Fri, Jul 07, 2023 at 01:29:02PM +0200, David Hildenbrand wrote: >>> On 07.07.23 11:52, Ryan Roberts wrote: >>>> On 07/07/2023 09:01, Huang, Ying wrote: >>>>> Although we can use smaller page order for FLEXIBLE_THP, it's hard to >>>>> avoid internal fragmentation completely.  So, I think that finally we >>>>> will need to provide a mechanism for the users to opt out, e.g., >>>>> something like "always madvise never" via >>>>> /sys/kernel/mm/transparent_hugepage/enabled.  I'm not sure whether it's >>>>> a good idea to reuse the existing interface of THP. >>>> >>>> I wouldn't want to tie this to the existing interface, simply because that >>>> implies that we would want to follow the "always" and "madvise" advice too; >>>> That >>>> means that on a thp=madvise system (which is certainly the case for android and >>>> other client systems) we would have to disable large anon folios for VMAs that >>>> haven't explicitly opted in. That breaks the intention that this should be an >>>> invisible performance boost. I think it's important to set the policy for >>>> use of >>> >>> It will never ever be a completely invisible performance boost, just like >>> ordinary THP. >>> >>> Using the exact same existing toggle is the right thing to do. If someone >>> specify "never" or "madvise", then do exactly that. >>> >>> It might make sense to have more modes or additional toggles, but >>> "madvise=never" means no memory waste. >> >> I hate the existing mechanisms.  They are an abdication of our >> responsibility, and an attempt to blame the user (be it the sysadmin >> or the programmer) of our code for using it wrongly.  We should not >> replicate this mistake. > > I don't agree regarding the programmer responsibility. In some cases the > programmer really doesn't want to get more memory populated than requested -- > and knows exactly why setting MADV_NOHUGEPAGE is the right thing to do. > > Regarding the madvise=never/madvise/always (sys admin decision), memory waste > (and nailing down bugs or working around them in customer setups) have been very > good reasons to let the admin have a word. > >> >> Our code should be auto-tuning.  I posted a long, detailed outline here: >> https://lore.kernel.org/linux-mm/Y%2FU8bQd15aUO97vS@casper.infradead.org/ >> > > Well, "auto-tuning" also should be perfect for everybody, but once reality > strikes you know it isn't. > > If people don't feel like using THP, let them have a word. The "madvise" config > option is probably more controversial. But the "always vs. never" absolutely > makes sense to me. > >>> I remember I raised it already in the past, but you *absolutely* have to >>> respect the MADV_NOHUGEPAGE flag. There is user space out there (for >>> example, userfaultfd) that doesn't want the kernel to populate any >>> additional page tables. So if you have to respect that already, then also >>> respect MADV_HUGEPAGE, simple. >> >> Possibly having uffd enabled on a VMA should disable using large folios, > > There are cases where we enable uffd *after* already touching memory (postcopy > live migration in QEMU being the famous example). That doesn't fly. > >> I can get behind that.  But the notion that userspace knows what it's >> doing ... hahaha.  Just ignore the madvise flags.  Userspace doesn't >> know what it's doing. > > If user space sets MADV_NOHUGEPAGE, it exactly knows what it is doing ... in > some cases. And these include cases I care about messing with sparse VM memory :) > > I have strong opinions against populating more than required when user space set > MADV_NOHUGEPAGE. I can see your point about honouring MADV_NOHUGEPAGE, so think that it is reasonable to fallback to allocating an order-0 page in a VMA that has it set. The app has gone out of its way to explicitly set it, after all. I think the correct behaviour for the global thp controls (cmdline and sysfs) are less obvious though. I could get on board with disabling large anon folios globally when thp="never". But for other situations, I would prefer to keep large anon folios enabled (treat "madvise" as "always"), with the argument that their order is much smaller than traditional THP and therefore the internal fragmentation is significantly reduced. I really don't want to end up with user space ever having to opt-in (with MADV_HUGEPAGE) to see the benefits of large anon folios. I still feel that it would be better for the thp and large anon folio controls to be independent though - what's the argument for tying them together? > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4FBFBEB64D9 for ; Fri, 7 Jul 2023 15:14:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To: Subject:MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Uv1e4wUplH0Z6/poVdaQiik0yG60XVFEre8dgY3YgGQ=; b=Sv3v1fJo1ijQcb xe1Qa1T0ToSBNz7ysDgfr3EXAlDw74dF28KJ/aZChllScENnABt6zAZgW0myZb3z57CwF2s2PeXzE xQAsqKsEDwxEnUxKy7iLwoffC3+zWaOVP84vQoIKU3IEF/Kxq1McDmixMUADAwRseYFWjvMmoSpXX bW6iup2CyiIsJaqOWJeTfeWPa/0/BVqc0rj9ZOSbytnETks3lAhCnIJKsjSVB2MVqYB+sx/gRAdnr icn2TeOBwIWY/TPyzYQyMrkh/DljtLglkEoHh39DdDa51k8DJ1mR7gzruCsSTlgxs0JZa6tR790F0 bDQhTOMM175as7fIYLHA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qHn9p-004zzZ-0M; Fri, 07 Jul 2023 15:14:05 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qHn9k-004zyH-1B for linux-arm-kernel@lists.infradead.org; Fri, 07 Jul 2023 15:14:03 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2F31FD75; Fri, 7 Jul 2023 08:14:38 -0700 (PDT) Received: from [10.57.77.63] (unknown [10.57.77.63]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 091EF3F762; Fri, 7 Jul 2023 08:13:53 -0700 (PDT) Message-ID: <9dd036a8-9ba3-0cc4-b791-cb3178237728@arm.com> Date: Fri, 7 Jul 2023 16:13:52 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: [PATCH v2 4/5] mm: FLEXIBLE_THP for improved performance To: David Hildenbrand , Matthew Wilcox Cc: "Huang, Ying" , Andrew Morton , "Kirill A. Shutemov" , Yin Fengwei , Yu Zhao , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20230703135330.1865927-1-ryan.roberts@arm.com> <20230703135330.1865927-5-ryan.roberts@arm.com> <87edlkgnfa.fsf@yhuang6-desk2.ccr.corp.intel.com> <44e60630-5e9d-c8df-ab79-cb0767de680e@arm.com> <524bacd2-4a47-2b8b-6685-c46e31a01631@redhat.com> <1e406f04-78ef-6573-e1f1-f0d0e0d5246a@redhat.com> From: Ryan Roberts In-Reply-To: <1e406f04-78ef-6573-e1f1-f0d0e0d5246a@redhat.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230707_081400_509299_FF545C29 X-CRM114-Status: GOOD ( 35.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gMDcvMDcvMjAyMyAxNTowNywgRGF2aWQgSGlsZGVuYnJhbmQgd3JvdGU6Cj4gT24gMDcuMDcu MjMgMTU6NTcsIE1hdHRoZXcgV2lsY294IHdyb3RlOgo+PiBPbiBGcmksIEp1bCAwNywgMjAyMyBh dCAwMToyOTowMlBNICswMjAwLCBEYXZpZCBIaWxkZW5icmFuZCB3cm90ZToKPj4+IE9uIDA3LjA3 LjIzIDExOjUyLCBSeWFuIFJvYmVydHMgd3JvdGU6Cj4+Pj4gT24gMDcvMDcvMjAyMyAwOTowMSwg SHVhbmcsIFlpbmcgd3JvdGU6Cj4+Pj4+IEFsdGhvdWdoIHdlIGNhbiB1c2Ugc21hbGxlciBwYWdl IG9yZGVyIGZvciBGTEVYSUJMRV9USFAsIGl0J3MgaGFyZCB0bwo+Pj4+PiBhdm9pZCBpbnRlcm5h bCBmcmFnbWVudGF0aW9uIGNvbXBsZXRlbHkuwqAgU28sIEkgdGhpbmsgdGhhdCBmaW5hbGx5IHdl Cj4+Pj4+IHdpbGwgbmVlZCB0byBwcm92aWRlIGEgbWVjaGFuaXNtIGZvciB0aGUgdXNlcnMgdG8g b3B0IG91dCwgZS5nLiwKPj4+Pj4gc29tZXRoaW5nIGxpa2UgImFsd2F5cyBtYWR2aXNlIG5ldmVy IiB2aWEKPj4+Pj4gL3N5cy9rZXJuZWwvbW0vdHJhbnNwYXJlbnRfaHVnZXBhZ2UvZW5hYmxlZC7C oCBJJ20gbm90IHN1cmUgd2hldGhlciBpdCdzCj4+Pj4+IGEgZ29vZCBpZGVhIHRvIHJldXNlIHRo ZSBleGlzdGluZyBpbnRlcmZhY2Ugb2YgVEhQLgo+Pj4+Cj4+Pj4gSSB3b3VsZG4ndCB3YW50IHRv IHRpZSB0aGlzIHRvIHRoZSBleGlzdGluZyBpbnRlcmZhY2UsIHNpbXBseSBiZWNhdXNlIHRoYXQK Pj4+PiBpbXBsaWVzIHRoYXQgd2Ugd291bGQgd2FudCB0byBmb2xsb3cgdGhlICJhbHdheXMiIGFu ZCAibWFkdmlzZSIgYWR2aWNlIHRvbzsKPj4+PiBUaGF0Cj4+Pj4gbWVhbnMgdGhhdCBvbiBhIHRo cD1tYWR2aXNlIHN5c3RlbSAod2hpY2ggaXMgY2VydGFpbmx5IHRoZSBjYXNlIGZvciBhbmRyb2lk IGFuZAo+Pj4+IG90aGVyIGNsaWVudCBzeXN0ZW1zKSB3ZSB3b3VsZCBoYXZlIHRvIGRpc2FibGUg bGFyZ2UgYW5vbiBmb2xpb3MgZm9yIFZNQXMgdGhhdAo+Pj4+IGhhdmVuJ3QgZXhwbGljaXRseSBv cHRlZCBpbi4gVGhhdCBicmVha3MgdGhlIGludGVudGlvbiB0aGF0IHRoaXMgc2hvdWxkIGJlIGFu Cj4+Pj4gaW52aXNpYmxlIHBlcmZvcm1hbmNlIGJvb3N0LiBJIHRoaW5rIGl0J3MgaW1wb3J0YW50 IHRvIHNldCB0aGUgcG9saWN5IGZvcgo+Pj4+IHVzZSBvZgo+Pj4KPj4+IEl0IHdpbGwgbmV2ZXIg ZXZlciBiZSBhIGNvbXBsZXRlbHkgaW52aXNpYmxlIHBlcmZvcm1hbmNlIGJvb3N0LCBqdXN0IGxp a2UKPj4+IG9yZGluYXJ5IFRIUC4KPj4+Cj4+PiBVc2luZyB0aGUgZXhhY3Qgc2FtZSBleGlzdGlu ZyB0b2dnbGUgaXMgdGhlIHJpZ2h0IHRoaW5nIHRvIGRvLiBJZiBzb21lb25lCj4+PiBzcGVjaWZ5 ICJuZXZlciIgb3IgIm1hZHZpc2UiLCB0aGVuIGRvIGV4YWN0bHkgdGhhdC4KPj4+Cj4+PiBJdCBt aWdodCBtYWtlIHNlbnNlIHRvIGhhdmUgbW9yZSBtb2RlcyBvciBhZGRpdGlvbmFsIHRvZ2dsZXMs IGJ1dAo+Pj4gIm1hZHZpc2U9bmV2ZXIiIG1lYW5zIG5vIG1lbW9yeSB3YXN0ZS4KPj4KPj4gSSBo YXRlIHRoZSBleGlzdGluZyBtZWNoYW5pc21zLsKgIFRoZXkgYXJlIGFuIGFiZGljYXRpb24gb2Yg b3VyCj4+IHJlc3BvbnNpYmlsaXR5LCBhbmQgYW4gYXR0ZW1wdCB0byBibGFtZSB0aGUgdXNlciAo YmUgaXQgdGhlIHN5c2FkbWluCj4+IG9yIHRoZSBwcm9ncmFtbWVyKSBvZiBvdXIgY29kZSBmb3Ig dXNpbmcgaXQgd3JvbmdseS7CoCBXZSBzaG91bGQgbm90Cj4+IHJlcGxpY2F0ZSB0aGlzIG1pc3Rh a2UuCj4gCj4gSSBkb24ndCBhZ3JlZSByZWdhcmRpbmcgdGhlIHByb2dyYW1tZXIgcmVzcG9uc2li aWxpdHkuIEluIHNvbWUgY2FzZXMgdGhlCj4gcHJvZ3JhbW1lciByZWFsbHkgZG9lc24ndCB3YW50 IHRvIGdldCBtb3JlIG1lbW9yeSBwb3B1bGF0ZWQgdGhhbiByZXF1ZXN0ZWQgLS0KPiBhbmQga25v d3MgZXhhY3RseSB3aHkgc2V0dGluZyBNQURWX05PSFVHRVBBR0UgaXMgdGhlIHJpZ2h0IHRoaW5n IHRvIGRvLgo+IAo+IFJlZ2FyZGluZyB0aGUgbWFkdmlzZT1uZXZlci9tYWR2aXNlL2Fsd2F5cyAo c3lzIGFkbWluIGRlY2lzaW9uKSwgbWVtb3J5IHdhc3RlCj4gKGFuZCBuYWlsaW5nIGRvd24gYnVn cyBvciB3b3JraW5nIGFyb3VuZCB0aGVtIGluIGN1c3RvbWVyIHNldHVwcykgaGF2ZSBiZWVuIHZl cnkKPiBnb29kIHJlYXNvbnMgdG8gbGV0IHRoZSBhZG1pbiBoYXZlIGEgd29yZC4KPiAKPj4KPj4g T3VyIGNvZGUgc2hvdWxkIGJlIGF1dG8tdHVuaW5nLsKgIEkgcG9zdGVkIGEgbG9uZywgZGV0YWls ZWQgb3V0bGluZSBoZXJlOgo+PiBodHRwczovL2xvcmUua2VybmVsLm9yZy9saW51eC1tbS9ZJTJG VThiUWQxNWFVTzk3dlNAY2FzcGVyLmluZnJhZGVhZC5vcmcvCj4+Cj4gCj4gV2VsbCwgImF1dG8t dHVuaW5nIiBhbHNvIHNob3VsZCBiZSBwZXJmZWN0IGZvciBldmVyeWJvZHksIGJ1dCBvbmNlIHJl YWxpdHkKPiBzdHJpa2VzIHlvdSBrbm93IGl0IGlzbid0Lgo+IAo+IElmIHBlb3BsZSBkb24ndCBm ZWVsIGxpa2UgdXNpbmcgVEhQLCBsZXQgdGhlbSBoYXZlIGEgd29yZC4gVGhlICJtYWR2aXNlIiBj b25maWcKPiBvcHRpb24gaXMgcHJvYmFibHkgbW9yZSBjb250cm92ZXJzaWFsLiBCdXQgdGhlICJh bHdheXMgdnMuIG5ldmVyIiBhYnNvbHV0ZWx5Cj4gbWFrZXMgc2Vuc2UgdG8gbWUuCj4gCj4+PiBJ IHJlbWVtYmVyIEkgcmFpc2VkIGl0IGFscmVhZHkgaW4gdGhlIHBhc3QsIGJ1dCB5b3UgKmFic29s dXRlbHkqIGhhdmUgdG8KPj4+IHJlc3BlY3QgdGhlIE1BRFZfTk9IVUdFUEFHRSBmbGFnLiBUaGVy ZSBpcyB1c2VyIHNwYWNlIG91dCB0aGVyZSAoZm9yCj4+PiBleGFtcGxlLCB1c2VyZmF1bHRmZCkg dGhhdCBkb2Vzbid0IHdhbnQgdGhlIGtlcm5lbCB0byBwb3B1bGF0ZSBhbnkKPj4+IGFkZGl0aW9u YWwgcGFnZSB0YWJsZXMuIFNvIGlmIHlvdSBoYXZlIHRvIHJlc3BlY3QgdGhhdCBhbHJlYWR5LCB0 aGVuIGFsc28KPj4+IHJlc3BlY3QgTUFEVl9IVUdFUEFHRSwgc2ltcGxlLgo+Pgo+PiBQb3NzaWJs eSBoYXZpbmcgdWZmZCBlbmFibGVkIG9uIGEgVk1BIHNob3VsZCBkaXNhYmxlIHVzaW5nIGxhcmdl IGZvbGlvcywKPiAKPiBUaGVyZSBhcmUgY2FzZXMgd2hlcmUgd2UgZW5hYmxlIHVmZmQgKmFmdGVy KiBhbHJlYWR5IHRvdWNoaW5nIG1lbW9yeSAocG9zdGNvcHkKPiBsaXZlIG1pZ3JhdGlvbiBpbiBR RU1VIGJlaW5nIHRoZSBmYW1vdXMgZXhhbXBsZSkuIFRoYXQgZG9lc24ndCBmbHkuCj4gCj4+IEkg Y2FuIGdldCBiZWhpbmQgdGhhdC7CoCBCdXQgdGhlIG5vdGlvbiB0aGF0IHVzZXJzcGFjZSBrbm93 cyB3aGF0IGl0J3MKPj4gZG9pbmcgLi4uIGhhaGFoYS7CoCBKdXN0IGlnbm9yZSB0aGUgbWFkdmlz ZSBmbGFncy7CoCBVc2Vyc3BhY2UgZG9lc24ndAo+PiBrbm93IHdoYXQgaXQncyBkb2luZy4KPiAK PiBJZiB1c2VyIHNwYWNlIHNldHMgTUFEVl9OT0hVR0VQQUdFLCBpdCBleGFjdGx5IGtub3dzIHdo YXQgaXQgaXMgZG9pbmcgLi4uIGluCj4gc29tZSBjYXNlcy4gQW5kIHRoZXNlIGluY2x1ZGUgY2Fz ZXMgSSBjYXJlIGFib3V0IG1lc3Npbmcgd2l0aCBzcGFyc2UgVk0gbWVtb3J5IDopCj4gCj4gSSBo YXZlIHN0cm9uZyBvcGluaW9ucyBhZ2FpbnN0IHBvcHVsYXRpbmcgbW9yZSB0aGFuIHJlcXVpcmVk IHdoZW4gdXNlciBzcGFjZSBzZXQKPiBNQURWX05PSFVHRVBBR0UuCgpJIGNhbiBzZWUgeW91ciBw b2ludCBhYm91dCBob25vdXJpbmcgTUFEVl9OT0hVR0VQQUdFLCBzbyB0aGluayB0aGF0IGl0IGlz CnJlYXNvbmFibGUgdG8gZmFsbGJhY2sgdG8gYWxsb2NhdGluZyBhbiBvcmRlci0wIHBhZ2UgaW4g YSBWTUEgdGhhdCBoYXMgaXQgc2V0LgpUaGUgYXBwIGhhcyBnb25lIG91dCBvZiBpdHMgd2F5IHRv IGV4cGxpY2l0bHkgc2V0IGl0LCBhZnRlciBhbGwuCgpJIHRoaW5rIHRoZSBjb3JyZWN0IGJlaGF2 aW91ciBmb3IgdGhlIGdsb2JhbCB0aHAgY29udHJvbHMgKGNtZGxpbmUgYW5kIHN5c2ZzKQphcmUg bGVzcyBvYnZpb3VzIHRob3VnaC4gSSBjb3VsZCBnZXQgb24gYm9hcmQgd2l0aCBkaXNhYmxpbmcg bGFyZ2UgYW5vbiBmb2xpb3MKZ2xvYmFsbHkgd2hlbiB0aHA9Im5ldmVyIi4gQnV0IGZvciBvdGhl ciBzaXR1YXRpb25zLCBJIHdvdWxkIHByZWZlciB0byBrZWVwCmxhcmdlIGFub24gZm9saW9zIGVu YWJsZWQgKHRyZWF0ICJtYWR2aXNlIiBhcyAiYWx3YXlzIiksIHdpdGggdGhlIGFyZ3VtZW50IHRo YXQKdGhlaXIgb3JkZXIgaXMgbXVjaCBzbWFsbGVyIHRoYW4gdHJhZGl0aW9uYWwgVEhQIGFuZCB0 aGVyZWZvcmUgdGhlIGludGVybmFsCmZyYWdtZW50YXRpb24gaXMgc2lnbmlmaWNhbnRseSByZWR1 Y2VkLiBJIHJlYWxseSBkb24ndCB3YW50IHRvIGVuZCB1cCB3aXRoIHVzZXIKc3BhY2UgZXZlciBo YXZpbmcgdG8gb3B0LWluICh3aXRoIE1BRFZfSFVHRVBBR0UpIHRvIHNlZSB0aGUgYmVuZWZpdHMg b2YgbGFyZ2UKYW5vbiBmb2xpb3MuCgpJIHN0aWxsIGZlZWwgdGhhdCBpdCB3b3VsZCBiZSBiZXR0 ZXIgZm9yIHRoZSB0aHAgYW5kIGxhcmdlIGFub24gZm9saW8gY29udHJvbHMKdG8gYmUgaW5kZXBl bmRlbnQgdGhvdWdoIC0gd2hhdCdzIHRoZSBhcmd1bWVudCBmb3IgdHlpbmcgdGhlbSB0b2dldGhl cj8KCj4gCgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18K bGludXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZy YWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGlu dXgtYXJtLWtlcm5lbAo=