From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C432EB64DA for ; Wed, 5 Jul 2023 17:25:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232506AbjGERZK (ORCPT ); Wed, 5 Jul 2023 13:25:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45164 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231398AbjGERZJ (ORCPT ); Wed, 5 Jul 2023 13:25:09 -0400 Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com [IPv6:2607:f8b0:4864:20::836]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E74FA188 for ; Wed, 5 Jul 2023 10:25:07 -0700 (PDT) Received: by mail-qt1-x836.google.com with SMTP id d75a77b69052e-401d1d967beso20881cf.0 for ; Wed, 05 Jul 2023 10:25:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688577907; x=1691169907; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=uvJl8XInaVm4n5AR4AD2zyrea1AdB/Yqy4IKMB7L0Iw=; b=DM87sN9etq3pLXS2B0O0x8/s69x8n8NtNg54LKnNdle2WaUIVHRKv2ggFtINDm6Wmq Tqw4LejYbfeB0xcvLh0X5mpLO2lh9LpxapgsVfLpB4qGdv5keyAOHiOqlruV+bBp6iiO a5WlNuHufIBSuqJ2SAN8sH3TvZVGH6x9hwxTX6ugTClo7DgB6YkQzk6WIqj0g+uYRm8y kXIH5o6hAztnIiTFDahGGEEUxi6AN4Z5VtkqouFT1mEOhKAkIyquMjVfHFRZRLPK2Icq 7xfArJI7fgL0ZRiqzznqyR1obr7TvIYfcjc+NfAlpM3GDURvTgdjB79Tg0SfJ8vjbbvN odYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688577907; x=1691169907; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uvJl8XInaVm4n5AR4AD2zyrea1AdB/Yqy4IKMB7L0Iw=; b=XsUd20EfKqzVrlhdcU9kht2XYTOsh/VWBoij70zModkO7CtPENzcsRH4PEnYiZO4NX MOAFvnX54tE5dVeVv9NUJRGr4RdtxSI54rK3+sODrCujRkwSRCzQFv6Jbcxa783DianV Q9yVlVXNVJ9IdjRhmvG/H1L23qmNmgHdHe6b9Twm9WUlESlanwK6UP1jZsT1EtZPLxf3 23is7O/Rhje940Kd/pbq4bm8yOfH2ewJViHVWU7tyyzhgwx/5tOliNxVwr/SpM66ocqo IRYzh8QlRl7aM/TOvdXWXutnpfK7l4PjcOIBFfexwO8jPOusv/gJV+Mv4VBUwb9PfnO9 tMSA== X-Gm-Message-State: ABy/qLbgA9jpouE1VKSuRP05Pdb2Li5YkIx9fT0vCsu8+ZrdCgJO5cPi QcXf9gcKAZcAWuSrEa1sdFg21US/FBfVKvC+/TUjdg== X-Google-Smtp-Source: APBJJlFvsrxca+yc7W6N+OGa5E4od+AwyUaACPlOJi2RIZLZQoFjwJXpzRBVXAIhOwUbSGzbP0UszSU6QIsPScInIuU= X-Received: by 2002:a05:622a:18e:b0:3f8:5b2:aeec with SMTP id s14-20020a05622a018e00b003f805b2aeecmr2782qtw.20.1688577906947; Wed, 05 Jul 2023 10:25:06 -0700 (PDT) MIME-Version: 1.0 References: <20230703135330.1865927-1-ryan.roberts@arm.com> <20230703135330.1865927-4-ryan.roberts@arm.com> <9c5f3515-ad39-e416-902e-96e9387a3b60@arm.com> In-Reply-To: <9c5f3515-ad39-e416-902e-96e9387a3b60@arm.com> From: Yu Zhao Date: Wed, 5 Jul 2023 11:24:30 -0600 Message-ID: Subject: Re: [PATCH v2 3/5] mm: Default implementation of arch_wants_pte_order() To: Ryan Roberts Cc: Andrew Morton , Matthew Wilcox , "Kirill A. Shutemov" , Yin Fengwei , David Hildenbrand , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jul 5, 2023 at 3:11=E2=80=AFAM Ryan Roberts = wrote: > > On 05/07/2023 03:07, Yu Zhao wrote: > > On Tue, Jul 4, 2023 at 7:20=E2=80=AFAM Ryan Roberts wrote: > >> > >> On 03/07/2023 20:50, Yu Zhao wrote: > >>> On Mon, Jul 3, 2023 at 7:53=E2=80=AFAM Ryan Roberts wrote: > >>>> > >>>> arch_wants_pte_order() can be overridden by the arch to return the > >>>> preferred folio order for pte-mapped memory. This is useful as some > >>>> architectures (e.g. arm64) can coalesce TLB entries when the physica= l > >>>> memory is suitably contiguous. > >>>> > >>>> The first user for this hint will be FLEXIBLE_THP, which aims to > >>>> allocate large folios for anonymous memory to reduce page faults and > >>>> other per-page operation costs. > >>>> > >>>> Here we add the default implementation of the function, used when th= e > >>>> architecture does not define it, which returns the order correspondi= ng > >>>> to 64K. > >>> > >>> I don't really mind a non-zero default value. But people would ask wh= y > >>> non-zero and why 64KB. Probably you could argue this is the large siz= e > >>> all known archs support if they have TLB coalescing. For x86, AMD CPU= s > >>> would want to override this. I'll leave it to Fengwei to decide > >>> whether Intel wants a different default value.> > >>> Also I don't like the vma parameter because it makes > >>> arch_wants_pte_order() a mix of hw preference and vma policy. From my > >>> POV, the function should be only about the former; the latter should > >>> be decided by arch-independent MM code. However, I can live with it i= f > >>> ARM MM people think this is really what you want. ATM, I'm skeptical > >>> they do. > >> > >> Here's the big picture for what I'm tryng to achieve: > >> > >> - In the common case, I'd like all programs to get a performance bump= by > >> automatically and transparently using large anon folios - so no explic= it > >> requirement on the process to opt-in. > > > > We all agree on this :) > > > >> - On arm64, in the above case, I'd like the preferred folio size to b= e 64K; > >> from the (admittedly limitted) testing I've done that's about where th= e > >> performance knee is and it doesn't appear to increase the memory wasta= ge very > >> much. It also has the benefits that for 4K base pages this is the cont= pte size > >> (order-4) so I can take full benefit of contpte mappings transparently= to the > >> process. And for 16K this is the HPA size (order-2). > > > > My highest priority is to get 16KB proven first because it would > > benefit both client and server devices. So it may be different from > > yours but I don't see any conflict. > > Do you mean 16K folios on a 4K base page system Yes. > or large folios on a 16K base > page system? I thought your focus was on speeding up 4K base page client = systems > but this statement has got me wondering? Sorry, I should have said 4x4KB. > >> - On arm64 when the process has marked the VMA for THP (or when > >> transparent_hugepage=3Dalways) but the VMA does not meet the requireme= nts for a > >> PMD-sized mapping (or we failed to allocate, ...) then I'd like to map= using > >> contpte. For 4K base pages this is 64K (order-4), for 16K this is 2M (= order-7) > >> and for 64K this is 2M (order-5). The 64K base page case is very impor= tant since > >> the PMD size for that base page is 512MB which is almost impossible to= allocate > >> in practice. > > > > Which case (server or client) are you focusing on here? For our client > > devices, I can confidently say that 64KB has to be after 16KB, if it > > happens at all. For servers in general, I don't know of any major > > memory-intensive workloads that are not THP-aware, i.e., I don't think > > "VMA does not meet the requirements" is a concern. > > For the 64K base page case, the focus is server. The problem reported by = our > partner is that the 512M huge page size is too big to reliably allocate a= nd so > the fauls always fall back to 64K base pages in practice. I would also sp= eculate > (happy to be proved wrong) that there are many THP-aware workloads that a= ssume > the THP size is 2M. In this case, their VMAs may well be too small to fit= a 512M > huge page when running on 64K base page system. Interesting. When you have something ready to share, I might be able to try it on our ARM servers as well. > But the TL;DR is that Arm has a partner for which enabling 2M THP on a 64= K base > page system is a very real requirement. Our intent is that this will be t= he > mechanism we use to enable it. Yes, contpte makes more sense for what you described. It'd fit in a lot better in the hugetlb case, but I guess your partner uses anon. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E78FBEB64DA for ; Wed, 5 Jul 2023 17:25:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=0bFQd/BgljiUJXOHsvOpYKqCqO66qFuL0UInGC3Gbf8=; b=igMTCi6fxEwcg8 xH3NfoNNDAng2krk4zswtz9ol1cwdJIVUXN7BvgOXUYoSdCp3IIMf5wMkGuZ6PGmLlAeOPbDBZIOZ HXkXN4STCFT7GuJm6f1B2GdcWfkntoCJsNEkDyyVSOFVPZt1uqoTIfY14pqVsJJw+5Fb96F6L5qsk diSY/9dZbXrTqj987Cl7h96LcGsnDoC2v5VG0budLiTMP1/jyO72WoMR6KEUasmzGggutQgX0R4ON Y7Jv0YZaku0bsM4IyuxoaBhB7OmZMF2CEx4kC6HQ5zndfx/TWNqUMT4ROTLTSfwfh1u1oUK3mgdgU R0HVYe95t+qhw0JTH2+Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qH6Fb-00GbKl-2Y; Wed, 05 Jul 2023 17:25:11 +0000 Received: from mail-qt1-x829.google.com ([2607:f8b0:4864:20::829]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qH6FY-00GbJj-1L for linux-arm-kernel@lists.infradead.org; Wed, 05 Jul 2023 17:25:09 +0000 Received: by mail-qt1-x829.google.com with SMTP id d75a77b69052e-40371070eb7so16121cf.1 for ; Wed, 05 Jul 2023 10:25:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688577907; x=1691169907; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=uvJl8XInaVm4n5AR4AD2zyrea1AdB/Yqy4IKMB7L0Iw=; b=DM87sN9etq3pLXS2B0O0x8/s69x8n8NtNg54LKnNdle2WaUIVHRKv2ggFtINDm6Wmq Tqw4LejYbfeB0xcvLh0X5mpLO2lh9LpxapgsVfLpB4qGdv5keyAOHiOqlruV+bBp6iiO a5WlNuHufIBSuqJ2SAN8sH3TvZVGH6x9hwxTX6ugTClo7DgB6YkQzk6WIqj0g+uYRm8y kXIH5o6hAztnIiTFDahGGEEUxi6AN4Z5VtkqouFT1mEOhKAkIyquMjVfHFRZRLPK2Icq 7xfArJI7fgL0ZRiqzznqyR1obr7TvIYfcjc+NfAlpM3GDURvTgdjB79Tg0SfJ8vjbbvN odYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688577907; x=1691169907; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uvJl8XInaVm4n5AR4AD2zyrea1AdB/Yqy4IKMB7L0Iw=; b=PHBSa9TTHs1mZrvoZrKMXTEHQ/102HGlBs6eVMFIICvD+PLG4rMCfaB5Jf8QEgfcxM rP6ky4MT7W5EsSlm5mZNVaZdLaEwz8rkypezl2LhiiNJzMCv/sGNl3Vu8i7475R3W5wI JzC++0YsQMWyDz9sjMpG09lQ72+NS/WqD+LrGTNwIK1wG1yoN/eS7hV+DG0vLqPj2a2U hzui2GD+DesU7WcKUncQHlWSzJki02KE8Dugp+Xf5DuYU2OdrEQG+lGq9OeYcq436Bue a17wxwl4JlqrE69a8RYEAWmPegtKklSq5uQzEwTRXOqFInF/Qknok5Fw6tUJa9kOnzD6 Ww4Q== X-Gm-Message-State: ABy/qLbNxVE3O1KNmNaiaclP4uTNjHA7TNEHaIdbApkAQo4w49goGolr KVI42Ju/R0MpdxsjYwQq7yz1SM2/Tt/sfNHtm0bq5w== X-Google-Smtp-Source: APBJJlFvsrxca+yc7W6N+OGa5E4od+AwyUaACPlOJi2RIZLZQoFjwJXpzRBVXAIhOwUbSGzbP0UszSU6QIsPScInIuU= X-Received: by 2002:a05:622a:18e:b0:3f8:5b2:aeec with SMTP id s14-20020a05622a018e00b003f805b2aeecmr2782qtw.20.1688577906947; Wed, 05 Jul 2023 10:25:06 -0700 (PDT) MIME-Version: 1.0 References: <20230703135330.1865927-1-ryan.roberts@arm.com> <20230703135330.1865927-4-ryan.roberts@arm.com> <9c5f3515-ad39-e416-902e-96e9387a3b60@arm.com> In-Reply-To: <9c5f3515-ad39-e416-902e-96e9387a3b60@arm.com> From: Yu Zhao Date: Wed, 5 Jul 2023 11:24:30 -0600 Message-ID: Subject: Re: [PATCH v2 3/5] mm: Default implementation of arch_wants_pte_order() To: Ryan Roberts Cc: Andrew Morton , Matthew Wilcox , "Kirill A. Shutemov" , Yin Fengwei , David Hildenbrand , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230705_102508_458595_AE32B935 X-CRM114-Status: GOOD ( 43.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gV2VkLCBKdWwgNSwgMjAyMyBhdCAzOjEx4oCvQU0gUnlhbiBSb2JlcnRzIDxyeWFuLnJvYmVy dHNAYXJtLmNvbT4gd3JvdGU6Cj4KPiBPbiAwNS8wNy8yMDIzIDAzOjA3LCBZdSBaaGFvIHdyb3Rl Ogo+ID4gT24gVHVlLCBKdWwgNCwgMjAyMyBhdCA3OjIw4oCvQU0gUnlhbiBSb2JlcnRzIDxyeWFu LnJvYmVydHNAYXJtLmNvbT4gd3JvdGU6Cj4gPj4KPiA+PiBPbiAwMy8wNy8yMDIzIDIwOjUwLCBZ dSBaaGFvIHdyb3RlOgo+ID4+PiBPbiBNb24sIEp1bCAzLCAyMDIzIGF0IDc6NTPigK9BTSBSeWFu IFJvYmVydHMgPHJ5YW4ucm9iZXJ0c0Bhcm0uY29tPiB3cm90ZToKPiA+Pj4+Cj4gPj4+PiBhcmNo X3dhbnRzX3B0ZV9vcmRlcigpIGNhbiBiZSBvdmVycmlkZGVuIGJ5IHRoZSBhcmNoIHRvIHJldHVy biB0aGUKPiA+Pj4+IHByZWZlcnJlZCBmb2xpbyBvcmRlciBmb3IgcHRlLW1hcHBlZCBtZW1vcnku IFRoaXMgaXMgdXNlZnVsIGFzIHNvbWUKPiA+Pj4+IGFyY2hpdGVjdHVyZXMgKGUuZy4gYXJtNjQp IGNhbiBjb2FsZXNjZSBUTEIgZW50cmllcyB3aGVuIHRoZSBwaHlzaWNhbAo+ID4+Pj4gbWVtb3J5 IGlzIHN1aXRhYmx5IGNvbnRpZ3VvdXMuCj4gPj4+Pgo+ID4+Pj4gVGhlIGZpcnN0IHVzZXIgZm9y IHRoaXMgaGludCB3aWxsIGJlIEZMRVhJQkxFX1RIUCwgd2hpY2ggYWltcyB0bwo+ID4+Pj4gYWxs b2NhdGUgbGFyZ2UgZm9saW9zIGZvciBhbm9ueW1vdXMgbWVtb3J5IHRvIHJlZHVjZSBwYWdlIGZh dWx0cyBhbmQKPiA+Pj4+IG90aGVyIHBlci1wYWdlIG9wZXJhdGlvbiBjb3N0cy4KPiA+Pj4+Cj4g Pj4+PiBIZXJlIHdlIGFkZCB0aGUgZGVmYXVsdCBpbXBsZW1lbnRhdGlvbiBvZiB0aGUgZnVuY3Rp b24sIHVzZWQgd2hlbiB0aGUKPiA+Pj4+IGFyY2hpdGVjdHVyZSBkb2VzIG5vdCBkZWZpbmUgaXQs IHdoaWNoIHJldHVybnMgdGhlIG9yZGVyIGNvcnJlc3BvbmRpbmcKPiA+Pj4+IHRvIDY0Sy4KPiA+ Pj4KPiA+Pj4gSSBkb24ndCByZWFsbHkgbWluZCBhIG5vbi16ZXJvIGRlZmF1bHQgdmFsdWUuIEJ1 dCBwZW9wbGUgd291bGQgYXNrIHdoeQo+ID4+PiBub24temVybyBhbmQgd2h5IDY0S0IuIFByb2Jh Ymx5IHlvdSBjb3VsZCBhcmd1ZSB0aGlzIGlzIHRoZSBsYXJnZSBzaXplCj4gPj4+IGFsbCBrbm93 biBhcmNocyBzdXBwb3J0IGlmIHRoZXkgaGF2ZSBUTEIgY29hbGVzY2luZy4gRm9yIHg4NiwgQU1E IENQVXMKPiA+Pj4gd291bGQgd2FudCB0byBvdmVycmlkZSB0aGlzLiBJJ2xsIGxlYXZlIGl0IHRv IEZlbmd3ZWkgdG8gZGVjaWRlCj4gPj4+IHdoZXRoZXIgSW50ZWwgd2FudHMgYSBkaWZmZXJlbnQg ZGVmYXVsdCB2YWx1ZS4+Cj4gPj4+IEFsc28gSSBkb24ndCBsaWtlIHRoZSB2bWEgcGFyYW1ldGVy IGJlY2F1c2UgaXQgbWFrZXMKPiA+Pj4gYXJjaF93YW50c19wdGVfb3JkZXIoKSBhIG1peCBvZiBo dyBwcmVmZXJlbmNlIGFuZCB2bWEgcG9saWN5LiBGcm9tIG15Cj4gPj4+IFBPViwgdGhlIGZ1bmN0 aW9uIHNob3VsZCBiZSBvbmx5IGFib3V0IHRoZSBmb3JtZXI7IHRoZSBsYXR0ZXIgc2hvdWxkCj4g Pj4+IGJlIGRlY2lkZWQgYnkgYXJjaC1pbmRlcGVuZGVudCBNTSBjb2RlLiBIb3dldmVyLCBJIGNh biBsaXZlIHdpdGggaXQgaWYKPiA+Pj4gQVJNIE1NIHBlb3BsZSB0aGluayB0aGlzIGlzIHJlYWxs eSB3aGF0IHlvdSB3YW50LiBBVE0sIEknbSBza2VwdGljYWwKPiA+Pj4gdGhleSBkby4KPiA+Pgo+ ID4+IEhlcmUncyB0aGUgYmlnIHBpY3R1cmUgZm9yIHdoYXQgSSdtIHRyeW5nIHRvIGFjaGlldmU6 Cj4gPj4KPiA+PiAgLSBJbiB0aGUgY29tbW9uIGNhc2UsIEknZCBsaWtlIGFsbCBwcm9ncmFtcyB0 byBnZXQgYSBwZXJmb3JtYW5jZSBidW1wIGJ5Cj4gPj4gYXV0b21hdGljYWxseSBhbmQgdHJhbnNw YXJlbnRseSB1c2luZyBsYXJnZSBhbm9uIGZvbGlvcyAtIHNvIG5vIGV4cGxpY2l0Cj4gPj4gcmVx dWlyZW1lbnQgb24gdGhlIHByb2Nlc3MgdG8gb3B0LWluLgo+ID4KPiA+IFdlIGFsbCBhZ3JlZSBv biB0aGlzIDopCj4gPgo+ID4+ICAtIE9uIGFybTY0LCBpbiB0aGUgYWJvdmUgY2FzZSwgSSdkIGxp a2UgdGhlIHByZWZlcnJlZCBmb2xpbyBzaXplIHRvIGJlIDY0SzsKPiA+PiBmcm9tIHRoZSAoYWRt aXR0ZWRseSBsaW1pdHRlZCkgdGVzdGluZyBJJ3ZlIGRvbmUgdGhhdCdzIGFib3V0IHdoZXJlIHRo ZQo+ID4+IHBlcmZvcm1hbmNlIGtuZWUgaXMgYW5kIGl0IGRvZXNuJ3QgYXBwZWFyIHRvIGluY3Jl YXNlIHRoZSBtZW1vcnkgd2FzdGFnZSB2ZXJ5Cj4gPj4gbXVjaC4gSXQgYWxzbyBoYXMgdGhlIGJl bmVmaXRzIHRoYXQgZm9yIDRLIGJhc2UgcGFnZXMgdGhpcyBpcyB0aGUgY29udHB0ZSBzaXplCj4g Pj4gKG9yZGVyLTQpIHNvIEkgY2FuIHRha2UgZnVsbCBiZW5lZml0IG9mIGNvbnRwdGUgbWFwcGlu Z3MgdHJhbnNwYXJlbnRseSB0byB0aGUKPiA+PiBwcm9jZXNzLiBBbmQgZm9yIDE2SyB0aGlzIGlz IHRoZSBIUEEgc2l6ZSAob3JkZXItMikuCj4gPgo+ID4gTXkgaGlnaGVzdCBwcmlvcml0eSBpcyB0 byBnZXQgMTZLQiBwcm92ZW4gZmlyc3QgYmVjYXVzZSBpdCB3b3VsZAo+ID4gYmVuZWZpdCBib3Ro IGNsaWVudCBhbmQgc2VydmVyIGRldmljZXMuIFNvIGl0IG1heSBiZSBkaWZmZXJlbnQgZnJvbQo+ ID4geW91cnMgYnV0IEkgZG9uJ3Qgc2VlIGFueSBjb25mbGljdC4KPgo+IERvIHlvdSBtZWFuIDE2 SyBmb2xpb3Mgb24gYSA0SyBiYXNlIHBhZ2Ugc3lzdGVtCgpZZXMuCgo+IG9yIGxhcmdlIGZvbGlv cyBvbiBhIDE2SyBiYXNlCj4gcGFnZSBzeXN0ZW0/IEkgdGhvdWdodCB5b3VyIGZvY3VzIHdhcyBv biBzcGVlZGluZyB1cCA0SyBiYXNlIHBhZ2UgY2xpZW50IHN5c3RlbXMKPiBidXQgdGhpcyBzdGF0 ZW1lbnQgaGFzIGdvdCBtZSB3b25kZXJpbmc/CgpTb3JyeSwgSSBzaG91bGQgaGF2ZSBzYWlkIDR4 NEtCLgoKPiA+PiAgLSBPbiBhcm02NCB3aGVuIHRoZSBwcm9jZXNzIGhhcyBtYXJrZWQgdGhlIFZN QSBmb3IgVEhQIChvciB3aGVuCj4gPj4gdHJhbnNwYXJlbnRfaHVnZXBhZ2U9YWx3YXlzKSBidXQg dGhlIFZNQSBkb2VzIG5vdCBtZWV0IHRoZSByZXF1aXJlbWVudHMgZm9yIGEKPiA+PiBQTUQtc2l6 ZWQgbWFwcGluZyAob3Igd2UgZmFpbGVkIHRvIGFsbG9jYXRlLCAuLi4pIHRoZW4gSSdkIGxpa2Ug dG8gbWFwIHVzaW5nCj4gPj4gY29udHB0ZS4gRm9yIDRLIGJhc2UgcGFnZXMgdGhpcyBpcyA2NEsg KG9yZGVyLTQpLCBmb3IgMTZLIHRoaXMgaXMgMk0gKG9yZGVyLTcpCj4gPj4gYW5kIGZvciA2NEsg dGhpcyBpcyAyTSAob3JkZXItNSkuIFRoZSA2NEsgYmFzZSBwYWdlIGNhc2UgaXMgdmVyeSBpbXBv cnRhbnQgc2luY2UKPiA+PiB0aGUgUE1EIHNpemUgZm9yIHRoYXQgYmFzZSBwYWdlIGlzIDUxMk1C IHdoaWNoIGlzIGFsbW9zdCBpbXBvc3NpYmxlIHRvIGFsbG9jYXRlCj4gPj4gaW4gcHJhY3RpY2Uu Cj4gPgo+ID4gV2hpY2ggY2FzZSAoc2VydmVyIG9yIGNsaWVudCkgYXJlIHlvdSBmb2N1c2luZyBv biBoZXJlPyBGb3Igb3VyIGNsaWVudAo+ID4gZGV2aWNlcywgSSBjYW4gY29uZmlkZW50bHkgc2F5 IHRoYXQgNjRLQiBoYXMgdG8gYmUgYWZ0ZXIgMTZLQiwgaWYgaXQKPiA+IGhhcHBlbnMgYXQgYWxs LiBGb3Igc2VydmVycyBpbiBnZW5lcmFsLCBJIGRvbid0IGtub3cgb2YgYW55IG1ham9yCj4gPiBt ZW1vcnktaW50ZW5zaXZlIHdvcmtsb2FkcyB0aGF0IGFyZSBub3QgVEhQLWF3YXJlLCBpLmUuLCBJ IGRvbid0IHRoaW5rCj4gPiAiVk1BIGRvZXMgbm90IG1lZXQgdGhlIHJlcXVpcmVtZW50cyIgaXMg YSBjb25jZXJuLgo+Cj4gRm9yIHRoZSA2NEsgYmFzZSBwYWdlIGNhc2UsIHRoZSBmb2N1cyBpcyBz ZXJ2ZXIuIFRoZSBwcm9ibGVtIHJlcG9ydGVkIGJ5IG91cgo+IHBhcnRuZXIgaXMgdGhhdCB0aGUg NTEyTSBodWdlIHBhZ2Ugc2l6ZSBpcyB0b28gYmlnIHRvIHJlbGlhYmx5IGFsbG9jYXRlIGFuZCBz bwo+IHRoZSBmYXVscyBhbHdheXMgZmFsbCBiYWNrIHRvIDY0SyBiYXNlIHBhZ2VzIGluIHByYWN0 aWNlLiBJIHdvdWxkIGFsc28gc3BlY3VsYXRlCj4gKGhhcHB5IHRvIGJlIHByb3ZlZCB3cm9uZykg dGhhdCB0aGVyZSBhcmUgbWFueSBUSFAtYXdhcmUgd29ya2xvYWRzIHRoYXQgYXNzdW1lCj4gdGhl IFRIUCBzaXplIGlzIDJNLiBJbiB0aGlzIGNhc2UsIHRoZWlyIFZNQXMgbWF5IHdlbGwgYmUgdG9v IHNtYWxsIHRvIGZpdCBhIDUxMk0KPiBodWdlIHBhZ2Ugd2hlbiBydW5uaW5nIG9uIDY0SyBiYXNl IHBhZ2Ugc3lzdGVtLgoKSW50ZXJlc3RpbmcuIFdoZW4geW91IGhhdmUgc29tZXRoaW5nIHJlYWR5 IHRvIHNoYXJlLCBJIG1pZ2h0IGJlIGFibGUKdG8gdHJ5IGl0IG9uIG91ciBBUk0gc2VydmVycyBh cyB3ZWxsLgoKPiBCdXQgdGhlIFRMO0RSIGlzIHRoYXQgQXJtIGhhcyBhIHBhcnRuZXIgZm9yIHdo aWNoIGVuYWJsaW5nIDJNIFRIUCBvbiBhIDY0SyBiYXNlCj4gcGFnZSBzeXN0ZW0gaXMgYSB2ZXJ5 IHJlYWwgcmVxdWlyZW1lbnQuIE91ciBpbnRlbnQgaXMgdGhhdCB0aGlzIHdpbGwgYmUgdGhlCj4g bWVjaGFuaXNtIHdlIHVzZSB0byBlbmFibGUgaXQuCgpZZXMsIGNvbnRwdGUgbWFrZXMgbW9yZSBz ZW5zZSBmb3Igd2hhdCB5b3UgZGVzY3JpYmVkLiBJdCdkIGZpdCBpbiBhCmxvdCBiZXR0ZXIgaW4g dGhlIGh1Z2V0bGIgY2FzZSwgYnV0IEkgZ3Vlc3MgeW91ciBwYXJ0bmVyIHVzZXMgYW5vbi4KCl9f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmxpbnV4LWFybS1r ZXJuZWwgbWFpbGluZyBsaXN0CmxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpo dHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJu ZWwK