From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D211C7618B for ; Wed, 24 Jul 2019 15:21:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id EB87321738 for ; Wed, 24 Jul 2019 15:21:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="MsOxbaLs" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727473AbfGXPVU (ORCPT ); Wed, 24 Jul 2019 11:21:20 -0400 Received: from lelv0143.ext.ti.com ([198.47.23.248]:33166 "EHLO lelv0143.ext.ti.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727083AbfGXPVU (ORCPT ); Wed, 24 Jul 2019 11:21:20 -0400 Received: from lelv0266.itg.ti.com ([10.180.67.225]) by lelv0143.ext.ti.com (8.15.2/8.15.2) with ESMTP id x6OFKXTl129002; Wed, 24 Jul 2019 10:20:33 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=ti-com-17Q1; t=1563981633; bh=kV06TLoVNsUYXenFXygLoD1KHApkQKg0DKnf9mR/P54=; h=Subject:To:CC:References:From:Date:In-Reply-To; b=MsOxbaLsddGg8FBIp7Bu36HN2Mh9pwDYMGDPBW8eyabCV5R4FO8VEDo+1WznZ87Uv EK416RLdiWYKaO8XXSijoygp9AGH0yTfWZKy52cVWyE7nfnIUkzHdYT3PJnp6WeIPJ v5fhFY+Ee+Qn/c1hRF3Kxs5auR6KAImeb8DLo38Y= Received: from DFLE111.ent.ti.com (dfle111.ent.ti.com [10.64.6.32]) by lelv0266.itg.ti.com (8.15.2/8.15.2) with ESMTPS id x6OFKXkl011982 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 24 Jul 2019 10:20:33 -0500 Received: from DFLE106.ent.ti.com (10.64.6.27) by DFLE111.ent.ti.com (10.64.6.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1713.5; Wed, 24 Jul 2019 10:20:33 -0500 Received: from fllv0039.itg.ti.com (10.64.41.19) by DFLE106.ent.ti.com (10.64.6.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1713.5 via Frontend Transport; Wed, 24 Jul 2019 10:20:33 -0500 Received: from [10.250.86.29] (ileax41-snat.itg.ti.com [10.172.224.153]) by fllv0039.itg.ti.com (8.15.2/8.15.2) with ESMTP id x6OFKVIJ026870; Wed, 24 Jul 2019 10:20:32 -0500 Subject: Re: [PATCH v6 2/5] dma-buf: heaps: Add heap helpers To: Christoph Hellwig , Rob Clark CC: John Stultz , lkml , Laura Abbott , Benjamin Gaignard , Sumit Semwal , Liam Mark , Pratik Patel , Brian Starkey , Vincent Donnefort , Sudipto Paul , Xu YiPing , "Chenfeng (puck)" , butao , "Xiaqing (A)" , Yudongbin , Chenbo Feng , Alistair Strachan , dri-devel , Hridya Valsaraju References: <20190624194908.121273-1-john.stultz@linaro.org> <20190624194908.121273-3-john.stultz@linaro.org> <20190718100654.GA19666@infradead.org> <20190724065530.GA16225@infradead.org> From: "Andrew F. Davis" Message-ID: <3966dff1-864d-cad4-565f-7c7120301265@ti.com> Date: Wed, 24 Jul 2019 11:20:31 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <20190724065530.GA16225@infradead.org> Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 7bit X-EXCLAIMER-MD-CONFIG: e1e8a2fd-e40a-4ac6-ac9b-f7e9cc9ee180 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 7/24/19 2:55 AM, Christoph Hellwig wrote: > On Tue, Jul 23, 2019 at 01:09:55PM -0700, Rob Clark wrote: >> On Mon, Jul 22, 2019 at 9:09 PM John Stultz wrote: >>> >>> On Thu, Jul 18, 2019 at 3:06 AM Christoph Hellwig wrote: >>>> >>>> Is there any exlusion between mmap / vmap and the device accessing >>>> the data? Without that you are going to run into a lot of coherency >>>> problems. >> >> dma_fence is basically the way to handle exclusion between different >> device access (since device access tends to be asynchronous). For >> device<->device access, each driver is expected to take care of any >> cache(s) that the device might have. (Ie. device writing to buffer >> should flush it's caches if needed before signalling fence to let >> reading device know that it is safe to read, etc.) >> >> _begin/end_cpu_access() is intended to be the exclusion for CPU access >> (which is synchronous) > > What I mean is that we need a clear state machine (preferably including > ownership tracking ala dma-debug) where a piece of memory has one > owner at a time that can access it. Only the owner can access is at > that time, and at any owner switch we need to flush/invalidate all > relevant caches. And with memory that is vmaped and mapped to userspace > that can get really complicated. > > The above sounds like you have some of that in place, but we'll really > need clear rules to make sure we don't have holes in the scheme. > Well then lets think on this. A given buffer can have 3 owners states (CPU-owned, Device-owned, and Un-owned). These are based on the caching state from the CPU perspective. If a buffer is CPU-owned then we (Linux) can write to the buffer safely without worry that the data is stale or that it will be accessed by the device without having been flushed. Device-owned buffers should not be accessed by the CPU, and inter-device synchronization should be handled by fencing as Rob points out. Un-owned is how a buffer starts for consistency and to prevent unneeded cache operations on unwritten buffers. We also need to track the mapping states, 4 states for this, CPU-mapped, Device-mapped, CPU/Device-mapped, unmapped. Should be self explanatory, map_dma_buf maps towards the device, mmap/vmap/kmap towards the CPU. Leaving a buffer mapped by the CPU while device access takes place is safe as long as ownership is taken before any access. One more point, we assume reference counting for the below discussion, for instance unmap_dma_buf refers to the last device unmapping, map_dma_buf refers only to the first. This gives 12 combined states, if we assume a buffer will always be owned when it has someone mapping it, either CPU or device or both, then we can drop 3 states. If a buffer is only mapped into one space, then that space owns it, this drops 2 cross-owned states. Lastly if not mapped by either space then the buffer becomes un-owned (and the backing memory can be freed or migrated as needed). Leaving us 5 valid states. * Un-Owned Un-Mapped * Device-Owned Device-Mapped * Device-Owned CPU/Device-Mapped * CPU-Owned CPU-Mapped * CPU-Owned CPU/Device-Mapped There are 6 DMA-BUF operations (classes) on a buffer: * map_dma_buf * unmap_dma_buf * begin_cpu_access * end_cpu_access * mmap/vmap/kmap * ummanp/vunmap/kunmap >From all this I've suggest the following state-machine(in DOT language): Note: Buffers start in "Un-Owned Un-Mapped" and can only be freed from that state. Note: Commented out states/transitions are not valid but here to prove completeness ------------------------------------------------------------------- digraph dma_buf_buffer_states { label = "DMA-BUF Buffer states"; uo_um [ label="Un-Owned\nUn-Mapped" ]; // uo_dm [ label="Un-Owned\nDevice-Mapped" ]; // uo_cm [ label="Un-Owned\nCPU-Mapped" ]; // uo_cdm [ label="Un-Owned\nCPU/Device-Mapped" ]; // do_um [ label="Device-Owned\nUn-Mapped" ]; do_dm [ label="Device-Owned\nDevice-Mapped" ]; // do_cm [ label="Device-Owned\nCPU-Mapped" ]; do_cdm [ label="Device-Owned\nCPU/Device-Mapped" ]; // co_um [ label="CPU-Owned\nUn-Mapped" ]; // co_dm [ label="CPU-Owned\nDevice-Mapped" ]; co_cm [ label="CPU-Owned\nCPU-Mapped" ]; co_cdm [ label="CPU-Owned\nCPU/Device-Mapped" ]; /* From Un-Owned Un-Mapped */ uo_um -> do_dm [ label="map_dma_buf" ]; // uo_um -> [ label="unmap_dma_buf" ]; // uo_um -> [ label="begin_cpu_access" ]; // uo_um -> [ label="end_cpu_access" ]; uo_um -> co_cm [ label="mmap/vmap/kmap" ]; // uo_um -> [ label="ummanp/vunmap/kunmap" ]; /* From Device-Owned Device-Mapped */ do_dm -> do_dm [ label="map_dma_buf" ]; do_dm -> uo_um [ label="unmap_dma_buf" ]; // do_dm -> [ label="begin_cpu_access" ]; // do_dm -> [ label="end_cpu_access" ]; do_dm -> do_cdm [ label="mmap/vmap/kmap" ]; // do_dm -> [ label="ummanp/vunmap/kunmap" ]; /* From Device-Owned CPU/Device-Mapped */ do_cdm -> do_cdm [ label="map_dma_buf" ]; do_cdm -> co_cm [ label="unmap_dma_buf" ]; do_cdm -> co_cdm [ label="begin_cpu_access" ]; // do_cdm -> [ label="end_cpu_access" ]; do_cdm -> do_cdm [ label="mmap/vmap/kmap" ]; do_cdm -> do_dm [ label="ummanp/vunmap/kunmap" ]; /* From CPU-Owned CPU-Mapped */ co_cm -> co_cdm [ label="map_dma_buf" ]; // co_cm -> [ label="unmap_dma_buf" ]; // co_cm -> [ label="begin_cpu_access" ]; co_cm -> co_cm [ label="end_cpu_access" ]; // co_cm -> [ label="mmap/vmap/kmap" ]; co_cm -> uo_um [ label="ummanp/vunmap/kunmap" ]; /* From CPU-Owned CPU/Device-Mapped */ co_cdm -> co_cdm [ label="map_dma_buf" ]; co_cdm -> co_cm [ label="unmap_dma_buf" ]; // co_cdm -> [ label="begin_cpu_access" ]; co_cdm -> do_cdm [ label="end_cpu_access" ]; co_cdm -> co_cdm [ label="mmap/vmap/kmap" ]; // co_cdm -> [ label="ummanp/vunmap/kunmap" ]; { rank = same; co_cm -> do_dm [ style=invis ]; rankdir = LR; } { rank = same; co_cdm -> do_cdm [ style=invis ]; rankdir = LR; } } ------------------------------------------------------------------- If we consider this the "official" model, then we can start optimizing cache operations, and start forbidding some nonsensical operations. What do y'all think? Andrew From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Andrew F. Davis" Subject: Re: [PATCH v6 2/5] dma-buf: heaps: Add heap helpers Date: Wed, 24 Jul 2019 11:20:31 -0400 Message-ID: <3966dff1-864d-cad4-565f-7c7120301265@ti.com> References: <20190624194908.121273-1-john.stultz@linaro.org> <20190624194908.121273-3-john.stultz@linaro.org> <20190718100654.GA19666@infradead.org> <20190724065530.GA16225@infradead.org> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Received: from lelv0143.ext.ti.com (lelv0143.ext.ti.com [198.47.23.248]) by gabe.freedesktop.org (Postfix) with ESMTPS id F34806E5CD for ; Wed, 24 Jul 2019 15:21:16 +0000 (UTC) In-Reply-To: <20190724065530.GA16225@infradead.org> Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: Christoph Hellwig , Rob Clark Cc: Yudongbin , Sudipto Paul , Xu YiPing , Alistair Strachan , Vincent Donnefort , "Chenfeng (puck)" , dri-devel , Chenbo Feng , lkml , Liam Mark , "Xiaqing (A)" , Hridya Valsaraju , Pratik Patel , butao List-Id: dri-devel@lists.freedesktop.org T24gNy8yNC8xOSAyOjU1IEFNLCBDaHJpc3RvcGggSGVsbHdpZyB3cm90ZToKPiBPbiBUdWUsIEp1 bCAyMywgMjAxOSBhdCAwMTowOTo1NVBNIC0wNzAwLCBSb2IgQ2xhcmsgd3JvdGU6Cj4+IE9uIE1v biwgSnVsIDIyLCAyMDE5IGF0IDk6MDkgUE0gSm9obiBTdHVsdHogPGpvaG4uc3R1bHR6QGxpbmFy by5vcmc+IHdyb3RlOgo+Pj4KPj4+IE9uIFRodSwgSnVsIDE4LCAyMDE5IGF0IDM6MDYgQU0gQ2hy aXN0b3BoIEhlbGx3aWcgPGhjaEBpbmZyYWRlYWQub3JnPiB3cm90ZToKPj4+Pgo+Pj4+IElzIHRo ZXJlIGFueSBleGx1c2lvbiBiZXR3ZWVuIG1tYXAgLyB2bWFwIGFuZCB0aGUgZGV2aWNlIGFjY2Vz c2luZwo+Pj4+IHRoZSBkYXRhPyAgV2l0aG91dCB0aGF0IHlvdSBhcmUgZ29pbmcgdG8gcnVuIGlu dG8gYSBsb3Qgb2YgY29oZXJlbmN5Cj4+Pj4gcHJvYmxlbXMuCj4+Cj4+IGRtYV9mZW5jZSBpcyBi YXNpY2FsbHkgdGhlIHdheSB0byBoYW5kbGUgZXhjbHVzaW9uIGJldHdlZW4gZGlmZmVyZW50Cj4+ IGRldmljZSBhY2Nlc3MgKHNpbmNlIGRldmljZSBhY2Nlc3MgdGVuZHMgdG8gYmUgYXN5bmNocm9u b3VzKS4gIEZvcgo+PiBkZXZpY2U8LT5kZXZpY2UgYWNjZXNzLCBlYWNoIGRyaXZlciBpcyBleHBl Y3RlZCB0byB0YWtlIGNhcmUgb2YgYW55Cj4+IGNhY2hlKHMpIHRoYXQgdGhlIGRldmljZSBtaWdo dCBoYXZlLiAgKEllLiBkZXZpY2Ugd3JpdGluZyB0byBidWZmZXIKPj4gc2hvdWxkIGZsdXNoIGl0 J3MgY2FjaGVzIGlmIG5lZWRlZCBiZWZvcmUgc2lnbmFsbGluZyBmZW5jZSB0byBsZXQKPj4gcmVh ZGluZyBkZXZpY2Uga25vdyB0aGF0IGl0IGlzIHNhZmUgdG8gcmVhZCwgZXRjLikKPj4KPj4gX2Jl Z2luL2VuZF9jcHVfYWNjZXNzKCkgaXMgaW50ZW5kZWQgdG8gYmUgdGhlIGV4Y2x1c2lvbiBmb3Ig Q1BVIGFjY2Vzcwo+PiAod2hpY2ggaXMgc3luY2hyb25vdXMpCj4gCj4gV2hhdCBJIG1lYW4gaXMg dGhhdCB3ZSBuZWVkIGEgY2xlYXIgc3RhdGUgbWFjaGluZSAocHJlZmVyYWJseSBpbmNsdWRpbmcK PiBvd25lcnNoaXAgdHJhY2tpbmcgYWxhIGRtYS1kZWJ1Zykgd2hlcmUgYSBwaWVjZSBvZiBtZW1v cnkgaGFzIG9uZQo+IG93bmVyIGF0IGEgdGltZSB0aGF0IGNhbiBhY2Nlc3MgaXQuICBPbmx5IHRo ZSBvd25lciBjYW4gYWNjZXNzIGlzIGF0Cj4gdGhhdCB0aW1lLCBhbmQgYXQgYW55IG93bmVyIHN3 aXRjaCB3ZSBuZWVkIHRvIGZsdXNoL2ludmFsaWRhdGUgYWxsCj4gcmVsZXZhbnQgY2FjaGVzLiAg QW5kIHdpdGggbWVtb3J5IHRoYXQgaXMgdm1hcGVkIGFuZCBtYXBwZWQgdG8gdXNlcnNwYWNlCj4g dGhhdCBjYW4gZ2V0IHJlYWxseSBjb21wbGljYXRlZC4KPiAKPiBUaGUgYWJvdmUgc291bmRzIGxp a2UgeW91IGhhdmUgc29tZSBvZiB0aGF0IGluIHBsYWNlLCBidXQgd2UnbGwgcmVhbGx5Cj4gbmVl ZCBjbGVhciBydWxlcyB0byBtYWtlIHN1cmUgd2UgZG9uJ3QgaGF2ZSBob2xlcyBpbiB0aGUgc2No ZW1lLgo+IAoKV2VsbCB0aGVuIGxldHMgdGhpbmsgb24gdGhpcy4gQSBnaXZlbiBidWZmZXIgY2Fu IGhhdmUgMyBvd25lcnMgc3RhdGVzCihDUFUtb3duZWQsIERldmljZS1vd25lZCwgYW5kIFVuLW93 bmVkKS4gVGhlc2UgYXJlIGJhc2VkIG9uIHRoZSBjYWNoaW5nCnN0YXRlIGZyb20gdGhlIENQVSBw ZXJzcGVjdGl2ZS4KCklmIGEgYnVmZmVyIGlzIENQVS1vd25lZCB0aGVuIHdlIChMaW51eCkgY2Fu IHdyaXRlIHRvIHRoZSBidWZmZXIgc2FmZWx5CndpdGhvdXQgd29ycnkgdGhhdCB0aGUgZGF0YSBp cyBzdGFsZSBvciB0aGF0IGl0IHdpbGwgYmUgYWNjZXNzZWQgYnkgdGhlCmRldmljZSB3aXRob3V0 IGhhdmluZyBiZWVuIGZsdXNoZWQuIERldmljZS1vd25lZCBidWZmZXJzIHNob3VsZCBub3QgYmUK YWNjZXNzZWQgYnkgdGhlIENQVSwgYW5kIGludGVyLWRldmljZSBzeW5jaHJvbml6YXRpb24gc2hv dWxkIGJlIGhhbmRsZWQKYnkgZmVuY2luZyBhcyBSb2IgcG9pbnRzIG91dC4gVW4tb3duZWQgaXMg aG93IGEgYnVmZmVyIHN0YXJ0cyBmb3IKY29uc2lzdGVuY3kgYW5kIHRvIHByZXZlbnQgdW5uZWVk ZWQgY2FjaGUgb3BlcmF0aW9ucyBvbiB1bndyaXR0ZW4gYnVmZmVycy4KCldlIGFsc28gbmVlZCB0 byB0cmFjayB0aGUgbWFwcGluZyBzdGF0ZXMsIDQgc3RhdGVzIGZvciB0aGlzLCBDUFUtbWFwcGVk LApEZXZpY2UtbWFwcGVkLCBDUFUvRGV2aWNlLW1hcHBlZCwgdW5tYXBwZWQuIFNob3VsZCBiZSBz ZWxmIGV4cGxhbmF0b3J5LAptYXBfZG1hX2J1ZiBtYXBzIHRvd2FyZHMgdGhlIGRldmljZSwgbW1h cC92bWFwL2ttYXAgdG93YXJkcyB0aGUgQ1BVLgpMZWF2aW5nIGEgYnVmZmVyIG1hcHBlZCBieSB0 aGUgQ1BVIHdoaWxlIGRldmljZSBhY2Nlc3MgdGFrZXMgcGxhY2UgaXMKc2FmZSBhcyBsb25nIGFz IG93bmVyc2hpcCBpcyB0YWtlbiBiZWZvcmUgYW55IGFjY2Vzcy4gT25lIG1vcmUgcG9pbnQsIHdl CmFzc3VtZSByZWZlcmVuY2UgY291bnRpbmcgZm9yIHRoZSBiZWxvdyBkaXNjdXNzaW9uLCBmb3Ig aW5zdGFuY2UKdW5tYXBfZG1hX2J1ZiByZWZlcnMgdG8gdGhlIGxhc3QgZGV2aWNlIHVubWFwcGlu ZywgbWFwX2RtYV9idWYgcmVmZXJzCm9ubHkgdG8gdGhlIGZpcnN0LgoKVGhpcyBnaXZlcyAxMiBj b21iaW5lZCBzdGF0ZXMsIGlmIHdlIGFzc3VtZSBhIGJ1ZmZlciB3aWxsIGFsd2F5cyBiZQpvd25l ZCB3aGVuIGl0IGhhcyBzb21lb25lIG1hcHBpbmcgaXQsIGVpdGhlciBDUFUgb3IgZGV2aWNlIG9y IGJvdGgsIHRoZW4Kd2UgY2FuIGRyb3AgMyBzdGF0ZXMuIElmIGEgYnVmZmVyIGlzIG9ubHkgbWFw cGVkIGludG8gb25lIHNwYWNlLCB0aGVuCnRoYXQgc3BhY2Ugb3ducyBpdCwgdGhpcyBkcm9wcyAy IGNyb3NzLW93bmVkIHN0YXRlcy4gTGFzdGx5IGlmIG5vdAptYXBwZWQgYnkgZWl0aGVyIHNwYWNl IHRoZW4gdGhlIGJ1ZmZlciBiZWNvbWVzIHVuLW93bmVkIChhbmQgdGhlIGJhY2tpbmcKbWVtb3J5 IGNhbiBiZSBmcmVlZCBvciBtaWdyYXRlZCBhcyBuZWVkZWQpLiBMZWF2aW5nIHVzIDUgdmFsaWQg c3RhdGVzLgoKKiBVbi1Pd25lZCBVbi1NYXBwZWQKKiBEZXZpY2UtT3duZWQgRGV2aWNlLU1hcHBl ZAoqIERldmljZS1Pd25lZCBDUFUvRGV2aWNlLU1hcHBlZAoqIENQVS1Pd25lZCBDUFUtTWFwcGVk CiogQ1BVLU93bmVkIENQVS9EZXZpY2UtTWFwcGVkCgpUaGVyZSBhcmUgNiBETUEtQlVGIG9wZXJh dGlvbnMgKGNsYXNzZXMpIG9uIGEgYnVmZmVyOgoKKiBtYXBfZG1hX2J1ZgoqIHVubWFwX2RtYV9i dWYKKiBiZWdpbl9jcHVfYWNjZXNzCiogZW5kX2NwdV9hY2Nlc3MKKiBtbWFwL3ZtYXAva21hcAoq IHVtbWFucC92dW5tYXAva3VubWFwCgpGcm9tIGFsbCB0aGlzIEkndmUgc3VnZ2VzdCB0aGUgZm9s bG93aW5nIHN0YXRlLW1hY2hpbmUoaW4gRE9UIGxhbmd1YWdlKToKCk5vdGU6IEJ1ZmZlcnMgc3Rh cnQgaW4gIlVuLU93bmVkIFVuLU1hcHBlZCIgYW5kIGNhbiBvbmx5IGJlIGZyZWVkIGZyb20KdGhh dCBzdGF0ZS4KCk5vdGU6IENvbW1lbnRlZCBvdXQgc3RhdGVzL3RyYW5zaXRpb25zIGFyZSBub3Qg dmFsaWQgYnV0IGhlcmUgdG8gcHJvdmUKY29tcGxldGVuZXNzCgotLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCgpkaWdyYXBo IGRtYV9idWZfYnVmZmVyX3N0YXRlcwp7CglsYWJlbCA9ICJETUEtQlVGIEJ1ZmZlciBzdGF0ZXMi OwoKCXVvX3VtIFsgbGFiZWw9IlVuLU93bmVkXG5Vbi1NYXBwZWQiIF07Ci8vCXVvX2RtIFsgbGFi ZWw9IlVuLU93bmVkXG5EZXZpY2UtTWFwcGVkIiBdOwovLwl1b19jbSBbIGxhYmVsPSJVbi1Pd25l ZFxuQ1BVLU1hcHBlZCIgXTsKLy8JdW9fY2RtIFsgbGFiZWw9IlVuLU93bmVkXG5DUFUvRGV2aWNl LU1hcHBlZCIgXTsKCi8vCWRvX3VtIFsgbGFiZWw9IkRldmljZS1Pd25lZFxuVW4tTWFwcGVkIiBd OwoJZG9fZG0gWyBsYWJlbD0iRGV2aWNlLU93bmVkXG5EZXZpY2UtTWFwcGVkIiBdOwovLwlkb19j bSBbIGxhYmVsPSJEZXZpY2UtT3duZWRcbkNQVS1NYXBwZWQiIF07Cglkb19jZG0gWyBsYWJlbD0i RGV2aWNlLU93bmVkXG5DUFUvRGV2aWNlLU1hcHBlZCIgXTsKCi8vCWNvX3VtIFsgbGFiZWw9IkNQ VS1Pd25lZFxuVW4tTWFwcGVkIiBdOwovLwljb19kbSBbIGxhYmVsPSJDUFUtT3duZWRcbkRldmlj ZS1NYXBwZWQiIF07Cgljb19jbSBbIGxhYmVsPSJDUFUtT3duZWRcbkNQVS1NYXBwZWQiIF07Cglj b19jZG0gWyBsYWJlbD0iQ1BVLU93bmVkXG5DUFUvRGV2aWNlLU1hcHBlZCIgXTsKCgkvKiBGcm9t IFVuLU93bmVkIFVuLU1hcHBlZCAqLwoJCXVvX3VtIC0+IGRvX2RtCQlbIGxhYmVsPSJtYXBfZG1h X2J1ZiIgXTsKLy8JCXVvX3VtIC0+CQlbIGxhYmVsPSJ1bm1hcF9kbWFfYnVmIiBdOwovLwkJdW9f dW0gLT4gCQlbIGxhYmVsPSJiZWdpbl9jcHVfYWNjZXNzIiBdOwovLwkJdW9fdW0gLT4JCVsgbGFi ZWw9ImVuZF9jcHVfYWNjZXNzIiBdOwoJCXVvX3VtIC0+IGNvX2NtCQlbIGxhYmVsPSJtbWFwL3Zt YXAva21hcCIgXTsKLy8JCXVvX3VtIC0+IAkJWyBsYWJlbD0idW1tYW5wL3Z1bm1hcC9rdW5tYXAi IF07CgoJLyogRnJvbSBEZXZpY2UtT3duZWQgRGV2aWNlLU1hcHBlZCAqLwoJCWRvX2RtIC0+IGRv X2RtCQlbIGxhYmVsPSJtYXBfZG1hX2J1ZiIgXTsKCQlkb19kbSAtPiB1b191bQkJWyBsYWJlbD0i dW5tYXBfZG1hX2J1ZiIgXTsKLy8JCWRvX2RtIC0+IAkJWyBsYWJlbD0iYmVnaW5fY3B1X2FjY2Vz cyIgXTsKLy8JCWRvX2RtIC0+CQlbIGxhYmVsPSJlbmRfY3B1X2FjY2VzcyIgXTsKCQlkb19kbSAt PiBkb19jZG0JCVsgbGFiZWw9Im1tYXAvdm1hcC9rbWFwIiBdOwovLwkJZG9fZG0gLT4gCQlbIGxh YmVsPSJ1bW1hbnAvdnVubWFwL2t1bm1hcCIgXTsKCgkvKiBGcm9tIERldmljZS1Pd25lZCBDUFUv RGV2aWNlLU1hcHBlZCAqLwoJCWRvX2NkbSAtPiBkb19jZG0JWyBsYWJlbD0ibWFwX2RtYV9idWYi IF07CgkJZG9fY2RtIC0+IGNvX2NtCQlbIGxhYmVsPSJ1bm1hcF9kbWFfYnVmIiBdOwoJCWRvX2Nk bSAtPiBjb19jZG0JWyBsYWJlbD0iYmVnaW5fY3B1X2FjY2VzcyIgXTsKLy8JCWRvX2NkbSAtPgkJ WyBsYWJlbD0iZW5kX2NwdV9hY2Nlc3MiIF07CgkJZG9fY2RtIC0+IGRvX2NkbQlbIGxhYmVsPSJt bWFwL3ZtYXAva21hcCIgXTsKCQlkb19jZG0gLT4gZG9fZG0JCVsgbGFiZWw9InVtbWFucC92dW5t YXAva3VubWFwIiBdOwoKCS8qIEZyb20gQ1BVLU93bmVkIENQVS1NYXBwZWQgKi8KCQljb19jbSAt PiBjb19jZG0JCVsgbGFiZWw9Im1hcF9kbWFfYnVmIiBdOwovLwkJY29fY20gLT4gCQlbIGxhYmVs PSJ1bm1hcF9kbWFfYnVmIiBdOwovLwkJY29fY20gLT4gCQlbIGxhYmVsPSJiZWdpbl9jcHVfYWNj ZXNzIiBdOwoJCWNvX2NtIC0+IGNvX2NtCQlbIGxhYmVsPSJlbmRfY3B1X2FjY2VzcyIgXTsKLy8J CWNvX2NtIC0+CQlbIGxhYmVsPSJtbWFwL3ZtYXAva21hcCIgXTsKCQljb19jbSAtPiB1b191bQkJ WyBsYWJlbD0idW1tYW5wL3Z1bm1hcC9rdW5tYXAiIF07CgoJLyogRnJvbSBDUFUtT3duZWQgQ1BV L0RldmljZS1NYXBwZWQgKi8KCQljb19jZG0gLT4gY29fY2RtCVsgbGFiZWw9Im1hcF9kbWFfYnVm IiBdOwoJCWNvX2NkbSAtPiBjb19jbQkJWyBsYWJlbD0idW5tYXBfZG1hX2J1ZiIgXTsKLy8JCWNv X2NkbSAtPiAJCVsgbGFiZWw9ImJlZ2luX2NwdV9hY2Nlc3MiIF07CgkJY29fY2RtIC0+IGRvX2Nk bQlbIGxhYmVsPSJlbmRfY3B1X2FjY2VzcyIgXTsKCQljb19jZG0gLT4gY29fY2RtCVsgbGFiZWw9 Im1tYXAvdm1hcC9rbWFwIiBdOwovLwkJY29fY2RtIC0+CQlbIGxhYmVsPSJ1bW1hbnAvdnVubWFw L2t1bm1hcCIgXTsKCgl7CgkJcmFuayA9IHNhbWU7CgkJY29fY20gLT4gZG9fZG0gWyBzdHlsZT1p bnZpcyBdOwoJCXJhbmtkaXIgPSBMUjsKCX0KCgl7CgkJcmFuayA9IHNhbWU7CgkJY29fY2RtIC0+ IGRvX2NkbSBbIHN0eWxlPWludmlzIF07CgkJcmFua2RpciA9IExSOwoJfQp9CgotLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t CgpJZiB3ZSBjb25zaWRlciB0aGlzIHRoZSAib2ZmaWNpYWwiIG1vZGVsLCB0aGVuIHdlIGNhbiBz dGFydCBvcHRpbWl6aW5nCmNhY2hlIG9wZXJhdGlvbnMsIGFuZCBzdGFydCBmb3JiaWRkaW5nIHNv bWUgbm9uc2Vuc2ljYWwgb3BlcmF0aW9ucy4KCldoYXQgZG8geSdhbGwgdGhpbms/CgpBbmRyZXcK X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRldmVs IG1haWxpbmcgbGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlz dHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVs