From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 139A9C433EF for ; Wed, 22 Jun 2022 08:24:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232890AbiFVIYf (ORCPT ); Wed, 22 Jun 2022 04:24:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47990 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229989AbiFVIYf (ORCPT ); Wed, 22 Jun 2022 04:24:35 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D08C11A1F; Wed, 22 Jun 2022 01:24:31 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 097D361766; Wed, 22 Jun 2022 08:24:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8F0BDC34114; Wed, 22 Jun 2022 08:24:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1655886270; bh=YbuYr5/ie+xJQcPyfNne1QY4FDSlOh+xwavMqcIsMnU=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=qeE7pxOHB1owDbNrefzoSq4pKLcFSTifGgjLZlz5Vqg0MHXH7Un2gk9NlDFQw0L8H tJeGbPX5BCTacAgEZpRXso9n9IfPTXYTgLGAMwwttl+GY4KuCmuqyFUGx7nPJS10/z 4kLiDY/g2Y8mIccxuXLTBl5Ky28evFXltp3I8A0g= Date: Wed, 22 Jun 2022 10:24:32 +0200 From: Greg Kroah-Hartman To: Zhangfei Gao Cc: Jean-Philippe Brucker , Arnd Bergmann , Herbert Xu , Wangzhou , Jonathan Cameron , linux-accelerators@lists.ozlabs.org, linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, iommu@lists.linux-foundation.org, Yang Shen Subject: Re: [PATCH] uacce: fix concurrency of fops_open and uacce_remove Message-ID: References: <20220610123423.27496-1-zhangfei.gao@linaro.org> <6fb8bed5-8d40-fd63-4537-44e9eb6aa053@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <6fb8bed5-8d40-fd63-4537-44e9eb6aa053@linaro.org> Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org On Wed, Jun 22, 2022 at 04:14:45PM +0800, Zhangfei Gao wrote: > Hi, Greg > > On 2022/6/21 下午3:44, Greg Kroah-Hartman wrote: > > On Tue, Jun 21, 2022 at 03:37:31PM +0800, Zhangfei Gao wrote: > > > > > > On 2022/6/20 下午9:36, Greg Kroah-Hartman wrote: > > > > On Mon, Jun 20, 2022 at 02:24:31PM +0100, Jean-Philippe Brucker wrote: > > > > > On Fri, Jun 17, 2022 at 02:05:21PM +0800, Zhangfei Gao wrote: > > > > > > > The refcount only ensures that the uacce_device object is not freed as > > > > > > > long as there are open fds. But uacce_remove() can run while there are > > > > > > > open fds, or fds in the process of being opened. And atfer uacce_remove() > > > > > > > runs, the uacce_device object still exists but is mostly unusable. For > > > > > > > example once the module is freed, uacce->ops is not valid anymore. But > > > > > > > currently uacce_fops_open() may dereference the ops in this case: > > > > > > > > > > > > > > uacce_fops_open() > > > > > > > if (!uacce->parent->driver) > > > > > > > /* Still valid, keep going */ > > > > > > > ... rmmod > > > > > > > uacce_remove() > > > > > > > ... free_module() > > > > > > > uacce->ops->get_queue() /* BUG */ > > > > > > uacce_remove should wait for uacce->queues_lock, until fops_open release the > > > > > > lock. > > > > > > If open happen just after the uacce_remove: unlock, uacce_bind_queue in open > > > > > > should fail. > > > > > Ah yes sorry, I lost sight of what this patch was adding. But we could > > > > > have the same issue with the patch, just in a different order, no? > > > > > > > > > > uacce_fops_open() > > > > > uacce = xa_load() > > > > > ... rmmod > > > > Um, how is rmmod called if the file descriptor is open? > > > > > > > > That should not be possible if the owner of the file descriptor is > > > > properly set. Please fix that up. > > > Thanks Greg > > > > > > Set cdev owner or use module_get/put can block rmmod once fops_open. > > > -       uacce->cdev->owner = THIS_MODULE; > > > +       uacce->cdev->owner = uacce->parent->driver->owner; > > > > > > However, still not find good method to block removing parent pci device. > > > > > > $ echo 1 > /sys/bus/pci/devices/0000:00:02.0/remove & > > > > > > [   32.563350]  uacce_remove+0x6c/0x148 > > > [   32.563353]  hisi_qm_uninit+0x12c/0x178 > > > [   32.563356]  hisi_zip_remove+0xa0/0xd0 [hisi_zip] > > > [   32.563361]  pci_device_remove+0x44/0xd8 > > > [   32.563364]  device_remove+0x54/0x88 > > > [   32.563367]  device_release_driver_internal+0xec/0x1a0 > > > [   32.563370]  device_release_driver+0x20/0x30 > > > [   32.563372]  pci_stop_bus_device+0x8c/0xe0 > > > [   32.563375]  pci_stop_and_remove_bus_device_locked+0x28/0x60 > > > [   32.563378]  remove_store+0x9c/0xb0 > > > [   32.563379]  dev_attr_store+0x20/0x38 > > Removing the parent pci device does not remove the module code, it > > removes the device itself. Don't confuse code vs. data here. > > Do you mean even parent pci device is removed immediately, the code has to > wait, like dma etc? No, reads will fail, as will DMA transfers, all PCI drivers need to handle surprise removal like this as we have had PCI hotplug systems for decades now. > Currently parent driver has to ensure all dma stopped then call > uacce_remove, > ie, after uacce_fops_open succeed, parent driver need wait fops_release, > then uacce_remove can be called. remove can be called before the file close can happen all the time, you need to handle that properly. > For example: > drivers/crypto/hisilicon/zip/zip_main.c: > hisi_qm_wait_task_finish > > If remove this wait , there may other issue, > Unable to handle kernel paging request at virtual address ffff80000b700204 > pc : hisi_qm_cache_wb.part.0+0x2c/0xa0 > > So uacce only need serialize uacce_fops_open and uacce_remove. That's not going to help much. > After uacce_fops_open, we can assume uacce_remove only happen after > uacce_fops_release? Nope, again, device remove can happen at any point in time and is independent of userspace open/close of file descriptors on the char device. This is a common problem/pattern that drivers need to handle, and unfortunatly they all need to handle it on their own. We have discussed ways of making it easier (see the ksummit discuss list archives from last year), but no one has stepped up and done the work yet :( > Then it would be much simpler. Sorry. If you treat the structures as independant, and properly grab some reference counts or a lock, you should be ok. greg k-h From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5851FC433EF for ; Wed, 22 Jun 2022 08:24:40 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id D3263409A6; Wed, 22 Jun 2022 08:24:39 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org D3263409A6 Authentication-Results: smtp2.osuosl.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=linuxfoundation.org header.i=@linuxfoundation.org header.a=rsa-sha256 header.s=korg header.b=qeE7pxOH X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GdZkZ6bP4r_V; Wed, 22 Jun 2022 08:24:39 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp2.osuosl.org (Postfix) with ESMTPS id 8F8FA40A02; Wed, 22 Jun 2022 08:24:38 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 8F8FA40A02 Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 55437C0039; Wed, 22 Jun 2022 08:24:38 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 889B0C002D for ; Wed, 22 Jun 2022 08:24:36 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id 6A431409E9 for ; Wed, 22 Jun 2022 08:24:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 6A431409E9 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8aXpe8xB6bJv for ; Wed, 22 Jun 2022 08:24:35 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org D0265409A6 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by smtp2.osuosl.org (Postfix) with ESMTPS id D0265409A6 for ; Wed, 22 Jun 2022 08:24:34 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id BD477B81CC0; Wed, 22 Jun 2022 08:24:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8F0BDC34114; Wed, 22 Jun 2022 08:24:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1655886270; bh=YbuYr5/ie+xJQcPyfNne1QY4FDSlOh+xwavMqcIsMnU=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=qeE7pxOHB1owDbNrefzoSq4pKLcFSTifGgjLZlz5Vqg0MHXH7Un2gk9NlDFQw0L8H tJeGbPX5BCTacAgEZpRXso9n9IfPTXYTgLGAMwwttl+GY4KuCmuqyFUGx7nPJS10/z 4kLiDY/g2Y8mIccxuXLTBl5Ky28evFXltp3I8A0g= Date: Wed, 22 Jun 2022 10:24:32 +0200 From: Greg Kroah-Hartman To: Zhangfei Gao Subject: Re: [PATCH] uacce: fix concurrency of fops_open and uacce_remove Message-ID: References: <20220610123423.27496-1-zhangfei.gao@linaro.org> <6fb8bed5-8d40-fd63-4537-44e9eb6aa053@linaro.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <6fb8bed5-8d40-fd63-4537-44e9eb6aa053@linaro.org> Cc: Jean-Philippe Brucker , Yang Shen , Herbert Xu , Arnd Bergmann , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, linux-crypto@vger.kernel.org, linux-accelerators@lists.ozlabs.org X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" T24gV2VkLCBKdW4gMjIsIDIwMjIgYXQgMDQ6MTQ6NDVQTSArMDgwMCwgWmhhbmdmZWkgR2FvIHdy b3RlOgo+IEhpLCBHcmVnCj4gCj4gT24gMjAyMi82LzIxIOS4i+WNiDM6NDQsIEdyZWcgS3JvYWgt SGFydG1hbiB3cm90ZToKPiA+IE9uIFR1ZSwgSnVuIDIxLCAyMDIyIGF0IDAzOjM3OjMxUE0gKzA4 MDAsIFpoYW5nZmVpIEdhbyB3cm90ZToKPiA+ID4gCj4gPiA+IE9uIDIwMjIvNi8yMCDkuIvljYg5 OjM2LCBHcmVnIEtyb2FoLUhhcnRtYW4gd3JvdGU6Cj4gPiA+ID4gT24gTW9uLCBKdW4gMjAsIDIw MjIgYXQgMDI6MjQ6MzFQTSArMDEwMCwgSmVhbi1QaGlsaXBwZSBCcnVja2VyIHdyb3RlOgo+ID4g PiA+ID4gT24gRnJpLCBKdW4gMTcsIDIwMjIgYXQgMDI6MDU6MjFQTSArMDgwMCwgWmhhbmdmZWkg R2FvIHdyb3RlOgo+ID4gPiA+ID4gPiA+IFRoZSByZWZjb3VudCBvbmx5IGVuc3VyZXMgdGhhdCB0 aGUgdWFjY2VfZGV2aWNlIG9iamVjdCBpcyBub3QgZnJlZWQgYXMKPiA+ID4gPiA+ID4gPiBsb25n IGFzIHRoZXJlIGFyZSBvcGVuIGZkcy4gQnV0IHVhY2NlX3JlbW92ZSgpIGNhbiBydW4gd2hpbGUg dGhlcmUgYXJlCj4gPiA+ID4gPiA+ID4gb3BlbiBmZHMsIG9yIGZkcyBpbiB0aGUgcHJvY2VzcyBv ZiBiZWluZyBvcGVuZWQuIEFuZCBhdGZlciB1YWNjZV9yZW1vdmUoKQo+ID4gPiA+ID4gPiA+IHJ1 bnMsIHRoZSB1YWNjZV9kZXZpY2Ugb2JqZWN0IHN0aWxsIGV4aXN0cyBidXQgaXMgbW9zdGx5IHVu dXNhYmxlLiBGb3IKPiA+ID4gPiA+ID4gPiBleGFtcGxlIG9uY2UgdGhlIG1vZHVsZSBpcyBmcmVl ZCwgdWFjY2UtPm9wcyBpcyBub3QgdmFsaWQgYW55bW9yZS4gQnV0Cj4gPiA+ID4gPiA+ID4gY3Vy cmVudGx5IHVhY2NlX2ZvcHNfb3BlbigpIG1heSBkZXJlZmVyZW5jZSB0aGUgb3BzIGluIHRoaXMg Y2FzZToKPiA+ID4gPiA+ID4gPiAKPiA+ID4gPiA+ID4gPiAJdWFjY2VfZm9wc19vcGVuKCkKPiA+ ID4gPiA+ID4gPiAJIGlmICghdWFjY2UtPnBhcmVudC0+ZHJpdmVyKQo+ID4gPiA+ID4gPiA+IAkg LyogU3RpbGwgdmFsaWQsIGtlZXAgZ29pbmcgKi8JCQo+ID4gPiA+ID4gPiA+IAkgLi4uCQkJCQly bW1vZAo+ID4gPiA+ID4gPiA+IAkJCQkJCSB1YWNjZV9yZW1vdmUoKQo+ID4gPiA+ID4gPiA+IAkg Li4uCQkJCQkgZnJlZV9tb2R1bGUoKQo+ID4gPiA+ID4gPiA+IAkgdWFjY2UtPm9wcy0+Z2V0X3F1 ZXVlKCkgLyogQlVHICovCj4gPiA+ID4gPiA+IHVhY2NlX3JlbW92ZSBzaG91bGQgd2FpdCBmb3Ig dWFjY2UtPnF1ZXVlc19sb2NrLCB1bnRpbCBmb3BzX29wZW4gcmVsZWFzZSB0aGUKPiA+ID4gPiA+ ID4gbG9jay4KPiA+ID4gPiA+ID4gSWYgb3BlbiBoYXBwZW4ganVzdCBhZnRlciB0aGUgdWFjY2Vf cmVtb3ZlOiB1bmxvY2ssIHVhY2NlX2JpbmRfcXVldWUgaW4gb3Blbgo+ID4gPiA+ID4gPiBzaG91 bGQgZmFpbC4KPiA+ID4gPiA+IEFoIHllcyBzb3JyeSwgSSBsb3N0IHNpZ2h0IG9mIHdoYXQgdGhp cyBwYXRjaCB3YXMgYWRkaW5nLiBCdXQgd2UgY291bGQKPiA+ID4gPiA+IGhhdmUgdGhlIHNhbWUg aXNzdWUgd2l0aCB0aGUgcGF0Y2gsIGp1c3QgaW4gYSBkaWZmZXJlbnQgb3JkZXIsIG5vPwo+ID4g PiA+ID4gCj4gPiA+ID4gPiAJdWFjY2VfZm9wc19vcGVuKCkKPiA+ID4gPiA+IAkgdWFjY2UgPSB4 YV9sb2FkKCkKPiA+ID4gPiA+IAkgLi4uCQkJCQlybW1vZAo+ID4gPiA+IFVtLCBob3cgaXMgcm1t b2QgY2FsbGVkIGlmIHRoZSBmaWxlIGRlc2NyaXB0b3IgaXMgb3Blbj8KPiA+ID4gPiAKPiA+ID4g PiBUaGF0IHNob3VsZCBub3QgYmUgcG9zc2libGUgaWYgdGhlIG93bmVyIG9mIHRoZSBmaWxlIGRl c2NyaXB0b3IgaXMKPiA+ID4gPiBwcm9wZXJseSBzZXQuICBQbGVhc2UgZml4IHRoYXQgdXAuCj4g PiA+IFRoYW5rcyBHcmVnCj4gPiA+IAo+ID4gPiBTZXQgY2RldiBvd25lciBvciB1c2UgbW9kdWxl X2dldC9wdXQgY2FuIGJsb2NrIHJtbW9kIG9uY2UgZm9wc19vcGVuLgo+ID4gPiAtwqDCoMKgwqDC oMKgIHVhY2NlLT5jZGV2LT5vd25lciA9IFRISVNfTU9EVUxFOwo+ID4gPiArwqDCoMKgwqDCoMKg IHVhY2NlLT5jZGV2LT5vd25lciA9IHVhY2NlLT5wYXJlbnQtPmRyaXZlci0+b3duZXI7Cj4gPiA+ IAo+ID4gPiBIb3dldmVyLCBzdGlsbCBub3QgZmluZCBnb29kIG1ldGhvZCB0byBibG9jayByZW1v dmluZyBwYXJlbnQgcGNpIGRldmljZS4KPiA+ID4gCj4gPiA+ICQgZWNobyAxID4gL3N5cy9idXMv cGNpL2RldmljZXMvMDAwMDowMDowMi4wL3JlbW92ZSAmCj4gPiA+IAo+ID4gPiBbwqDCoCAzMi41 NjMzNTBdwqAgdWFjY2VfcmVtb3ZlKzB4NmMvMHgxNDgKPiA+ID4gW8KgwqAgMzIuNTYzMzUzXcKg IGhpc2lfcW1fdW5pbml0KzB4MTJjLzB4MTc4Cj4gPiA+IFvCoMKgIDMyLjU2MzM1Nl3CoCBoaXNp X3ppcF9yZW1vdmUrMHhhMC8weGQwIFtoaXNpX3ppcF0KPiA+ID4gW8KgwqAgMzIuNTYzMzYxXcKg IHBjaV9kZXZpY2VfcmVtb3ZlKzB4NDQvMHhkOAo+ID4gPiBbwqDCoCAzMi41NjMzNjRdwqAgZGV2 aWNlX3JlbW92ZSsweDU0LzB4ODgKPiA+ID4gW8KgwqAgMzIuNTYzMzY3XcKgIGRldmljZV9yZWxl YXNlX2RyaXZlcl9pbnRlcm5hbCsweGVjLzB4MWEwCj4gPiA+IFvCoMKgIDMyLjU2MzM3MF3CoCBk ZXZpY2VfcmVsZWFzZV9kcml2ZXIrMHgyMC8weDMwCj4gPiA+IFvCoMKgIDMyLjU2MzM3Ml3CoCBw Y2lfc3RvcF9idXNfZGV2aWNlKzB4OGMvMHhlMAo+ID4gPiBbwqDCoCAzMi41NjMzNzVdwqAgcGNp X3N0b3BfYW5kX3JlbW92ZV9idXNfZGV2aWNlX2xvY2tlZCsweDI4LzB4NjAKPiA+ID4gW8KgwqAg MzIuNTYzMzc4XcKgIHJlbW92ZV9zdG9yZSsweDljLzB4YjAKPiA+ID4gW8KgwqAgMzIuNTYzMzc5 XcKgIGRldl9hdHRyX3N0b3JlKzB4MjAvMHgzOAo+ID4gUmVtb3ZpbmcgdGhlIHBhcmVudCBwY2kg ZGV2aWNlIGRvZXMgbm90IHJlbW92ZSB0aGUgbW9kdWxlIGNvZGUsIGl0Cj4gPiByZW1vdmVzIHRo ZSBkZXZpY2UgaXRzZWxmLiAgRG9uJ3QgY29uZnVzZSBjb2RlIHZzLiBkYXRhIGhlcmUuCj4gCj4g RG8geW91IG1lYW4gZXZlbiBwYXJlbnQgcGNpIGRldmljZSBpcyByZW1vdmVkIGltbWVkaWF0ZWx5 LCB0aGUgY29kZSBoYXMgdG8KPiB3YWl0LCBsaWtlIGRtYSBldGM/CgpObywgcmVhZHMgd2lsbCBm YWlsLCBhcyB3aWxsIERNQSB0cmFuc2ZlcnMsIGFsbCBQQ0kgZHJpdmVycyBuZWVkIHRvCmhhbmRs ZSBzdXJwcmlzZSByZW1vdmFsIGxpa2UgdGhpcyBhcyB3ZSBoYXZlIGhhZCBQQ0kgaG90cGx1ZyBz eXN0ZW1zIGZvcgpkZWNhZGVzIG5vdy4KCj4gQ3VycmVudGx5IHBhcmVudCBkcml2ZXIgaGFzIHRv IGVuc3VyZSBhbGwgZG1hIHN0b3BwZWQgdGhlbiBjYWxsCj4gdWFjY2VfcmVtb3ZlLAo+IGllLCBh ZnRlciB1YWNjZV9mb3BzX29wZW4gc3VjY2VlZCwgcGFyZW50IGRyaXZlciBuZWVkIHdhaXQgZm9w c19yZWxlYXNlLAo+IHRoZW4gdWFjY2VfcmVtb3ZlIGNhbiBiZSBjYWxsZWQuCgpyZW1vdmUgY2Fu IGJlIGNhbGxlZCBiZWZvcmUgdGhlIGZpbGUgY2xvc2UgY2FuIGhhcHBlbiBhbGwgdGhlIHRpbWUs IHlvdQpuZWVkIHRvIGhhbmRsZSB0aGF0IHByb3Blcmx5LgoKPiBGb3IgZXhhbXBsZToKPiBkcml2 ZXJzL2NyeXB0by9oaXNpbGljb24vemlwL3ppcF9tYWluLmM6Cj4gaGlzaV9xbV93YWl0X3Rhc2tf ZmluaXNoCj4gCj4gSWYgcmVtb3ZlIHRoaXMgd2FpdCAsIHRoZXJlIG1heSBvdGhlciBpc3N1ZSwK PiBVbmFibGUgdG8gaGFuZGxlIGtlcm5lbCBwYWdpbmcgcmVxdWVzdCBhdCB2aXJ0dWFsIGFkZHJl c3MgZmZmZjgwMDAwYjcwMDIwNAo+IHBjIDogaGlzaV9xbV9jYWNoZV93Yi5wYXJ0LjArMHgyYy8w eGEwCj4gCj4gU28gdWFjY2Ugb25seSBuZWVkIHNlcmlhbGl6ZSB1YWNjZV9mb3BzX29wZW4gYW5k IHVhY2NlX3JlbW92ZS4KClRoYXQncyBub3QgZ29pbmcgdG8gaGVscCBtdWNoLgoKPiBBZnRlciB1 YWNjZV9mb3BzX29wZW4sIHdlIGNhbiBhc3N1bWUgdWFjY2VfcmVtb3ZlIG9ubHkgaGFwcGVuIGFm dGVyCj4gdWFjY2VfZm9wc19yZWxlYXNlPwoKTm9wZSwgYWdhaW4sIGRldmljZSByZW1vdmUgY2Fu IGhhcHBlbiBhdCBhbnkgcG9pbnQgaW4gdGltZSBhbmQgaXMKaW5kZXBlbmRlbnQgb2YgdXNlcnNw YWNlIG9wZW4vY2xvc2Ugb2YgZmlsZSBkZXNjcmlwdG9ycyBvbiB0aGUgY2hhcgpkZXZpY2UuCgpU aGlzIGlzIGEgY29tbW9uIHByb2JsZW0vcGF0dGVybiB0aGF0IGRyaXZlcnMgbmVlZCB0byBoYW5k bGUsIGFuZAp1bmZvcnR1bmF0bHkgdGhleSBhbGwgbmVlZCB0byBoYW5kbGUgaXQgb24gdGhlaXIg b3duLiAgV2UgaGF2ZSBkaXNjdXNzZWQKd2F5cyBvZiBtYWtpbmcgaXQgZWFzaWVyIChzZWUgdGhl IGtzdW1taXQgZGlzY3VzcyBsaXN0IGFyY2hpdmVzIGZyb20KbGFzdCB5ZWFyKSwgYnV0IG5vIG9u ZSBoYXMgc3RlcHBlZCB1cCBhbmQgZG9uZSB0aGUgd29yayB5ZXQgOigKCj4gVGhlbiBpdCB3b3Vs ZCBiZSBtdWNoIHNpbXBsZXIuCgpTb3JyeS4gIElmIHlvdSB0cmVhdCB0aGUgc3RydWN0dXJlcyBh cyBpbmRlcGVuZGFudCwgYW5kIHByb3Blcmx5IGdyYWIKc29tZSByZWZlcmVuY2UgY291bnRzIG9y IGEgbG9jaywgeW91IHNob3VsZCBiZSBvay4KCmdyZWcgay1oCl9fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fCmlvbW11IG1haWxpbmcgbGlzdAppb21tdUBsaXN0 cy5saW51eC1mb3VuZGF0aW9uLm9yZwpodHRwczovL2xpc3RzLmxpbnV4Zm91bmRhdGlvbi5vcmcv bWFpbG1hbi9saXN0aW5mby9pb21tdQ==