From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABF63C04AAF for ; Tue, 21 May 2019 14:59:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 812C821743 for ; Tue, 21 May 2019 14:59:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728664AbfEUO7j convert rfc822-to-8bit (ORCPT ); Tue, 21 May 2019 10:59:39 -0400 Received: from mx1.redhat.com ([209.132.183.28]:57750 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728505AbfEUO7i (ORCPT ); Tue, 21 May 2019 10:59:38 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 96625C057F3E; Tue, 21 May 2019 14:59:34 +0000 (UTC) Received: from x1.home (ovpn-117-92.phx2.redhat.com [10.3.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id 63B7C19C71; Tue, 21 May 2019 14:59:31 +0000 (UTC) Date: Tue, 21 May 2019 08:59:30 -0600 From: Alex Williamson To: Pierre Morel Cc: Cornelia Huck , sebott@linux.vnet.ibm.com, gerald.schaefer@de.ibm.com, pasic@linux.vnet.ibm.com, borntraeger@de.ibm.com, walling@linux.ibm.com, linux-s390@vger.kernel.org, iommu@lists.linux-foundation.org, joro@8bytes.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, schwidefsky@de.ibm.com, heiko.carstens@de.ibm.com, robin.murphy@arm.com Subject: Re: [PATCH v2 4/4] vfio: vfio_iommu_type1: implement VFIO_IOMMU_INFO_CAPABILITIES Message-ID: <20190521085930.4d91842c@x1.home> In-Reply-To: <9dc0a8de-b850-df21-e3b7-21b7c2a373a3@linux.ibm.com> References: <1558109810-18683-1-git-send-email-pmorel@linux.ibm.com> <1558109810-18683-5-git-send-email-pmorel@linux.ibm.com> <20190517104143.240082b5@x1.home> <92b6ad4e-9a49-636b-9225-acca0bec4bb7@linux.ibm.com> <20190520162737.7560ad7c.cohuck@redhat.com> <23f6a739-be4f-7eda-2227-2994fdc2325a@linux.ibm.com> <20190520122352.73082e52@x1.home> <9dc0a8de-b850-df21-e3b7-21b7c2a373a3@linux.ibm.com> Organization: Red Hat MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 21 May 2019 14:59:37 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 21 May 2019 11:14:38 +0200 Pierre Morel wrote: > On 20/05/2019 20:23, Alex Williamson wrote: > > On Mon, 20 May 2019 18:31:08 +0200 > > Pierre Morel wrote: > > > >> On 20/05/2019 16:27, Cornelia Huck wrote: > >>> On Mon, 20 May 2019 13:19:23 +0200 > >>> Pierre Morel wrote: > >>> > >>>> On 17/05/2019 20:04, Pierre Morel wrote: > >>>>> On 17/05/2019 18:41, Alex Williamson wrote: > >>>>>> On Fri, 17 May 2019 18:16:50 +0200 > >>>>>> Pierre Morel wrote: > >>>>>> > >>>>>>> We implement the capability interface for VFIO_IOMMU_GET_INFO. > >>>>>>> > >>>>>>> When calling the ioctl, the user must specify > >>>>>>> VFIO_IOMMU_INFO_CAPABILITIES to retrieve the capabilities and > >>>>>>> must check in the answer if capabilities are supported. > >>>>>>> > >>>>>>> The iommu get_attr callback will be used to retrieve the specific > >>>>>>> attributes and fill the capabilities. > >>>>>>> > >>>>>>> Currently two Z-PCI specific capabilities will be queried and > >>>>>>> filled by the underlying Z specific s390_iommu: > >>>>>>> VFIO_IOMMU_INFO_CAP_QFN for the PCI query function attributes > >>>>>>> and > >>>>>>> VFIO_IOMMU_INFO_CAP_QGRP for the PCI query function group. > >>>>>>> > >>>>>>> Other architectures may add new capabilities in the same way > >>>>>>> after enhancing the architecture specific IOMMU driver. > >>>>>>> > >>>>>>> Signed-off-by: Pierre Morel > >>>>>>> --- > >>>>>>>   drivers/vfio/vfio_iommu_type1.c | 122 > >>>>>>> +++++++++++++++++++++++++++++++++++++++- > >>>>>>>   1 file changed, 121 insertions(+), 1 deletion(-) > >>>>>>> > >>>>>>> diff --git a/drivers/vfio/vfio_iommu_type1.c > >>>>>>> b/drivers/vfio/vfio_iommu_type1.c > >>>>>>> index d0f731c..9435647 100644 > >>>>>>> --- a/drivers/vfio/vfio_iommu_type1.c > >>>>>>> +++ b/drivers/vfio/vfio_iommu_type1.c > >>>>>>> @@ -1658,6 +1658,97 @@ static int > >>>>>>> vfio_domains_have_iommu_cache(struct vfio_iommu *iommu) > >>>>>>>       return ret; > >>>>>>>   } > >>>>>>> +static int vfio_iommu_type1_zpci_fn(struct iommu_domain *domain, > >>>>>>> +                    struct vfio_info_cap *caps, size_t size) > >>>>>>> +{ > >>>>>>> +    struct vfio_iommu_type1_info_pcifn *info_fn; > >>>>>>> +    int ret; > >>>>>>> + > >>>>>>> +    info_fn = kzalloc(size, GFP_KERNEL); > >>>>>>> +    if (!info_fn) > >>>>>>> +        return -ENOMEM; > >>>>>>> + > >>>>>>> +    ret = iommu_domain_get_attr(domain, DOMAIN_ATTR_ZPCI_FN, > >>>>>>> +                    &info_fn->response); > >>>>>> > >>>>>> What ensures that the 'struct clp_rsp_query_pci' returned from this > >>>>>> get_attr remains consistent with a 'struct vfio_iommu_pci_function'? > >>>>>> Why does the latter contains so many reserved fields (beyond simply > >>>>>> alignment) for a user API?  What fields of these structures are > >>>>>> actually useful to userspace?  Should any fields not be exposed to the > >>>>>> user?  Aren't BAR sizes redundant to what's available through the vfio > >>>>>> PCI API?  I'm afraid that simply redefining an internal structure as > >>>>>> the API leaves a lot to be desired too.  Thanks, > >>>>>> > >>>>>> Alex > >>>>>> > >>>>> Hi Alex, > >>>>> > >>>>> I simply used the structure returned by the firmware to be sure to be > >>>>> consistent with future evolutions and facilitate the copy from CLP and > >>>>> to userland. > >>>>> > >>>>> If you prefer, and I understand that this is the case, I can define a > >>>>> specific VFIO_IOMMU structure with only the fields relevant to the user, > >>>>> leaving future enhancement of the user's interface being implemented in > >>>>> another kernel patch when the time has come. > > > > TBH, I had no idea that CLP is an s390 firmware interface and this is > > just dumping that to userspace. The cover letter says: > > > > Using the PCI VFIO interface allows userland, a.k.a. QEMU, to > > retrieve ZPCI specific information without knowing Z specific > > identifiers like the function ID or the function handle of the zPCI > > function hidden behind the PCI interface. > > > > But what does this allow userland to do and what specific pieces of > > information do they need? We do have a case already where Intel > > graphics devices have a table (OpRegion) living in host system memory > > that we expose via a vfio region, so it wouldn't be unprecedented to do > > something like this, but as Connie suggests, if we knew what was being > > consumed here and why, maybe we could generalize it into something > > useful for others. > > OK, sorry I try to explain better. > > 1) A short description, of zPCI functions and groups > > IN Z, PCI cards, leave behind an adapter between subchannels and PCI. > We access PCI cards through 2 ways: > - dedicated PCI instructions (pci_load/pci_store/pci/store_block) > - DMA > We receive events through > - Adapter interrupts > - CHSC events > > The adapter propose an IOMMU to protect the DMA > and the interrupt handling goes through a MSIX like interface handled by > the adapter. > > The architecture specific PCI do the interface between the standard PCI > level and the zPCI function (PCI + DMA/IOMMU/Interrupt) > > To handle the communication through the "zPCI way" the CLP interface > provides instructions to retrieve informations from the adapters. > > There are different group of functions having same functionalities. > > clp_list give us a list from zPCI functions > clp_query_pci_function returns informations specific to a function > clp_query_group returns information on a function group > > > 2) Why do we need it in the guest > > We need to provide the guest with information on the adapters and zPCI > functions returned by the clp_query instruction so that the guest's > driver gets the right information on how the way to the zPCI function > has been built in the host. > > > When a guest issues the CLP instructions we intercept the clp command in > QEMU and we need to feed the response with the right values for the guest. > The "right" values are not the raw CLP response values: > > - some identifier must be virtualized, like UID and FID, > > - some values must match what the host received from the CLP response, > like the size of the transmited blocks, the DMA Address Space Mask, > number of interrupt, MSIA > > - some other must match what the host handled with the adapter and > function, the start and end of DMA, > > - some what the host IOMMU driver supports (frame size), This seems very reminiscent of virtualizing PCI config space... so why is this being proposed as a VFIO IOMMU ioctl extension? These are all function level characteristics, right? Should this be a capability on the VFIO device, or perhaps a region like we used for the Intel OpRegion (though the structure size seems more akin to a capability here)? As I mentioned in my previous reply, tying this into the IOMMU interface seemed to rely on (I assume) an one-to-one-to-one mapping of PCI function to IOMMU group to IOMMU domain, but that doesn't still doesn't necessarily lend itself to using the IOMMU for device level information. If there is IOMMU info, perhaps it needs to be split, ie. expose a frame size via domain_get_attr, expose device level features via a device capability, let QEMU assemble these into something coherent to emulate the clp interface. > 3) We have three different way to get This information: > > The PCI Linux interface is a standard PCI interface and some Z specific > information is available in sysfs. > Not all the information needed to be returned inside the CLP response is > available. > So we can not use the sysfs interface to get all the information. > > There is a CLP ioctl interface but this interface is not secure in that > it returns the information for all adapters in the system. > > The VFIO interface offers the advantage to point to a single PCI > function, so more secure than the clp ioctl interface. > Coupled with the s390_iommu we get access to the zPCI CLP instruction > and to the values handled by the zPCI driver. > > > 4) Until now we used to fill the CLP response to the guest inside QEMU > with fixed values corresponding to the only PCI card we supported. > To support new cards we need to get the right values from the kernel out. If it's already emulated, I much prefer figuring out how to expose the right pieces of information via an appropriate interface to virtualize fields that are actually necessary rather than simply providing an interface to dump the clp info straight to userspace and pipe it to the VM. Thanks, Alex From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F5C0C04AAF for ; Tue, 21 May 2019 14:59:46 +0000 (UTC) Received: from mail.linuxfoundation.org (mail.linuxfoundation.org [140.211.169.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E0F352173E for ; Tue, 21 May 2019 14:59:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E0F352173E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from mail.linux-foundation.org (localhost [127.0.0.1]) by mail.linuxfoundation.org (Postfix) with ESMTP id B3420BE7; Tue, 21 May 2019 14:59:45 +0000 (UTC) Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 2B0B3AC8 for ; Tue, 21 May 2019 14:59:44 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 58DA5878 for ; Tue, 21 May 2019 14:59:43 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 96625C057F3E; Tue, 21 May 2019 14:59:34 +0000 (UTC) Received: from x1.home (ovpn-117-92.phx2.redhat.com [10.3.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id 63B7C19C71; Tue, 21 May 2019 14:59:31 +0000 (UTC) Date: Tue, 21 May 2019 08:59:30 -0600 From: Alex Williamson To: Pierre Morel Subject: Re: [PATCH v2 4/4] vfio: vfio_iommu_type1: implement VFIO_IOMMU_INFO_CAPABILITIES Message-ID: <20190521085930.4d91842c@x1.home> In-Reply-To: <9dc0a8de-b850-df21-e3b7-21b7c2a373a3@linux.ibm.com> References: <1558109810-18683-1-git-send-email-pmorel@linux.ibm.com> <1558109810-18683-5-git-send-email-pmorel@linux.ibm.com> <20190517104143.240082b5@x1.home> <92b6ad4e-9a49-636b-9225-acca0bec4bb7@linux.ibm.com> <20190520162737.7560ad7c.cohuck@redhat.com> <23f6a739-be4f-7eda-2227-2994fdc2325a@linux.ibm.com> <20190520122352.73082e52@x1.home> <9dc0a8de-b850-df21-e3b7-21b7c2a373a3@linux.ibm.com> Organization: Red Hat MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.32]); Tue, 21 May 2019 14:59:37 +0000 (UTC) Cc: linux-s390@vger.kernel.org, pasic@linux.vnet.ibm.com, kvm@vger.kernel.org, heiko.carstens@de.ibm.com, Cornelia Huck , sebott@linux.vnet.ibm.com, walling@linux.ibm.com, linux-kernel@vger.kernel.org, borntraeger@de.ibm.com, iommu@lists.linux-foundation.org, schwidefsky@de.ibm.com, robin.murphy@arm.com, gerald.schaefer@de.ibm.com X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: iommu-bounces@lists.linux-foundation.org Errors-To: iommu-bounces@lists.linux-foundation.org T24gVHVlLCAyMSBNYXkgMjAxOSAxMToxNDozOCArMDIwMApQaWVycmUgTW9yZWwgPHBtb3JlbEBs aW51eC5pYm0uY29tPiB3cm90ZToKCj4gT24gMjAvMDUvMjAxOSAyMDoyMywgQWxleCBXaWxsaWFt c29uIHdyb3RlOgo+ID4gT24gTW9uLCAyMCBNYXkgMjAxOSAxODozMTowOCArMDIwMAo+ID4gUGll cnJlIE1vcmVsIDxwbW9yZWxAbGludXguaWJtLmNvbT4gd3JvdGU6Cj4gPiAgIAo+ID4+IE9uIDIw LzA1LzIwMTkgMTY6MjcsIENvcm5lbGlhIEh1Y2sgd3JvdGU6ICAKPiA+Pj4gT24gTW9uLCAyMCBN YXkgMjAxOSAxMzoxOToyMyArMDIwMAo+ID4+PiBQaWVycmUgTW9yZWwgPHBtb3JlbEBsaW51eC5p Ym0uY29tPiB3cm90ZToKPiA+Pj4gICAgICAKPiA+Pj4+IE9uIDE3LzA1LzIwMTkgMjA6MDQsIFBp ZXJyZSBNb3JlbCB3cm90ZTogIAo+ID4+Pj4+IE9uIDE3LzA1LzIwMTkgMTg6NDEsIEFsZXggV2ls bGlhbXNvbiB3cm90ZTogIAo+ID4+Pj4+PiBPbiBGcmksIDE3IE1heSAyMDE5IDE4OjE2OjUwICsw MjAwCj4gPj4+Pj4+IFBpZXJyZSBNb3JlbCA8cG1vcmVsQGxpbnV4LmlibS5jb20+IHdyb3RlOgo+ ID4+Pj4+PiAgICAgICAgCj4gPj4+Pj4+PiBXZSBpbXBsZW1lbnQgdGhlIGNhcGFiaWxpdHkgaW50 ZXJmYWNlIGZvciBWRklPX0lPTU1VX0dFVF9JTkZPLgo+ID4+Pj4+Pj4KPiA+Pj4+Pj4+IFdoZW4g Y2FsbGluZyB0aGUgaW9jdGwsIHRoZSB1c2VyIG11c3Qgc3BlY2lmeQo+ID4+Pj4+Pj4gVkZJT19J T01NVV9JTkZPX0NBUEFCSUxJVElFUyB0byByZXRyaWV2ZSB0aGUgY2FwYWJpbGl0aWVzIGFuZAo+ ID4+Pj4+Pj4gbXVzdCBjaGVjayBpbiB0aGUgYW5zd2VyIGlmIGNhcGFiaWxpdGllcyBhcmUgc3Vw cG9ydGVkLgo+ID4+Pj4+Pj4KPiA+Pj4+Pj4+IFRoZSBpb21tdSBnZXRfYXR0ciBjYWxsYmFjayB3 aWxsIGJlIHVzZWQgdG8gcmV0cmlldmUgdGhlIHNwZWNpZmljCj4gPj4+Pj4+PiBhdHRyaWJ1dGVz IGFuZCBmaWxsIHRoZSBjYXBhYmlsaXRpZXMuCj4gPj4+Pj4+Pgo+ID4+Pj4+Pj4gQ3VycmVudGx5 IHR3byBaLVBDSSBzcGVjaWZpYyBjYXBhYmlsaXRpZXMgd2lsbCBiZSBxdWVyaWVkIGFuZAo+ID4+ Pj4+Pj4gZmlsbGVkIGJ5IHRoZSB1bmRlcmx5aW5nIFogc3BlY2lmaWMgczM5MF9pb21tdToKPiA+ Pj4+Pj4+IFZGSU9fSU9NTVVfSU5GT19DQVBfUUZOIGZvciB0aGUgUENJIHF1ZXJ5IGZ1bmN0aW9u IGF0dHJpYnV0ZXMKPiA+Pj4+Pj4+IGFuZAo+ID4+Pj4+Pj4gVkZJT19JT01NVV9JTkZPX0NBUF9R R1JQIGZvciB0aGUgUENJIHF1ZXJ5IGZ1bmN0aW9uIGdyb3VwLgo+ID4+Pj4+Pj4KPiA+Pj4+Pj4+ IE90aGVyIGFyY2hpdGVjdHVyZXMgbWF5IGFkZCBuZXcgY2FwYWJpbGl0aWVzIGluIHRoZSBzYW1l IHdheQo+ID4+Pj4+Pj4gYWZ0ZXIgZW5oYW5jaW5nIHRoZSBhcmNoaXRlY3R1cmUgc3BlY2lmaWMg SU9NTVUgZHJpdmVyLgo+ID4+Pj4+Pj4KPiA+Pj4+Pj4+IFNpZ25lZC1vZmYtYnk6IFBpZXJyZSBN b3JlbCA8cG1vcmVsQGxpbnV4LmlibS5jb20+Cj4gPj4+Pj4+PiAtLS0KPiA+Pj4+Pj4+ICAgwqAg ZHJpdmVycy92ZmlvL3ZmaW9faW9tbXVfdHlwZTEuYyB8IDEyMgo+ID4+Pj4+Pj4gKysrKysrKysr KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrLQo+ID4+Pj4+Pj4gICDCoCAxIGZpbGUgY2hh bmdlZCwgMTIxIGluc2VydGlvbnMoKyksIDEgZGVsZXRpb24oLSkKPiA+Pj4+Pj4+Cj4gPj4+Pj4+ PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy92ZmlvL3ZmaW9faW9tbXVfdHlwZTEuYwo+ID4+Pj4+Pj4g Yi9kcml2ZXJzL3ZmaW8vdmZpb19pb21tdV90eXBlMS5jCj4gPj4+Pj4+PiBpbmRleCBkMGY3MzFj Li45NDM1NjQ3IDEwMDY0NAo+ID4+Pj4+Pj4gLS0tIGEvZHJpdmVycy92ZmlvL3ZmaW9faW9tbXVf dHlwZTEuYwo+ID4+Pj4+Pj4gKysrIGIvZHJpdmVycy92ZmlvL3ZmaW9faW9tbXVfdHlwZTEuYwo+ ID4+Pj4+Pj4gQEAgLTE2NTgsNiArMTY1OCw5NyBAQCBzdGF0aWMgaW50Cj4gPj4+Pj4+PiB2Zmlv X2RvbWFpbnNfaGF2ZV9pb21tdV9jYWNoZShzdHJ1Y3QgdmZpb19pb21tdSAqaW9tbXUpCj4gPj4+ Pj4+PiAgIMKgwqDCoMKgwqAgcmV0dXJuIHJldDsKPiA+Pj4+Pj4+ICAgwqAgfQo+ID4+Pj4+Pj4g K3N0YXRpYyBpbnQgdmZpb19pb21tdV90eXBlMV96cGNpX2ZuKHN0cnVjdCBpb21tdV9kb21haW4g KmRvbWFpbiwKPiA+Pj4+Pj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oCBzdHJ1Y3QgdmZpb19pbmZvX2NhcCAqY2Fwcywgc2l6ZV90IHNpemUpCj4gPj4+Pj4+PiArewo+ ID4+Pj4+Pj4gK8KgwqDCoCBzdHJ1Y3QgdmZpb19pb21tdV90eXBlMV9pbmZvX3BjaWZuICppbmZv X2ZuOwo+ID4+Pj4+Pj4gK8KgwqDCoCBpbnQgcmV0Owo+ID4+Pj4+Pj4gKwo+ID4+Pj4+Pj4gK8Kg wqDCoCBpbmZvX2ZuID0ga3phbGxvYyhzaXplLCBHRlBfS0VSTkVMKTsKPiA+Pj4+Pj4+ICvCoMKg wqAgaWYgKCFpbmZvX2ZuKQo+ID4+Pj4+Pj4gK8KgwqDCoMKgwqDCoMKgIHJldHVybiAtRU5PTUVN Owo+ID4+Pj4+Pj4gKwo+ID4+Pj4+Pj4gK8KgwqDCoCByZXQgPSBpb21tdV9kb21haW5fZ2V0X2F0 dHIoZG9tYWluLCBET01BSU5fQVRUUl9aUENJX0ZOLAo+ID4+Pj4+Pj4gK8KgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgICZpbmZvX2ZuLT5yZXNwb25zZSk7ICAKPiA+Pj4+Pj4K PiA+Pj4+Pj4gV2hhdCBlbnN1cmVzIHRoYXQgdGhlICdzdHJ1Y3QgY2xwX3JzcF9xdWVyeV9wY2kn IHJldHVybmVkIGZyb20gdGhpcwo+ID4+Pj4+PiBnZXRfYXR0ciByZW1haW5zIGNvbnNpc3RlbnQg d2l0aCBhICdzdHJ1Y3QgdmZpb19pb21tdV9wY2lfZnVuY3Rpb24nPwo+ID4+Pj4+PiBXaHkgZG9l cyB0aGUgbGF0dGVyIGNvbnRhaW5zIHNvIG1hbnkgcmVzZXJ2ZWQgZmllbGRzIChiZXlvbmQgc2lt cGx5Cj4gPj4+Pj4+IGFsaWdubWVudCkgZm9yIGEgdXNlciBBUEk/wqAgV2hhdCBmaWVsZHMgb2Yg dGhlc2Ugc3RydWN0dXJlcyBhcmUKPiA+Pj4+Pj4gYWN0dWFsbHkgdXNlZnVsIHRvIHVzZXJzcGFj ZT/CoCBTaG91bGQgYW55IGZpZWxkcyBub3QgYmUgZXhwb3NlZCB0byB0aGUKPiA+Pj4+Pj4gdXNl cj/CoCBBcmVuJ3QgQkFSIHNpemVzIHJlZHVuZGFudCB0byB3aGF0J3MgYXZhaWxhYmxlIHRocm91 Z2ggdGhlIHZmaW8KPiA+Pj4+Pj4gUENJIEFQST/CoCBJJ20gYWZyYWlkIHRoYXQgc2ltcGx5IHJl ZGVmaW5pbmcgYW4gaW50ZXJuYWwgc3RydWN0dXJlIGFzCj4gPj4+Pj4+IHRoZSBBUEkgbGVhdmVz IGEgbG90IHRvIGJlIGRlc2lyZWQgdG9vLsKgIFRoYW5rcywKPiA+Pj4+Pj4KPiA+Pj4+Pj4gQWxl eAo+ID4+Pj4+PiAgICAgICAgCj4gPj4+Pj4gSGkgQWxleCwKPiA+Pj4+Pgo+ID4+Pj4+IEkgc2lt cGx5IHVzZWQgdGhlIHN0cnVjdHVyZSByZXR1cm5lZCBieSB0aGUgZmlybXdhcmUgdG8gYmUgc3Vy ZSB0byBiZQo+ID4+Pj4+IGNvbnNpc3RlbnQgd2l0aCBmdXR1cmUgZXZvbHV0aW9ucyBhbmQgZmFj aWxpdGF0ZSB0aGUgY29weSBmcm9tIENMUCBhbmQKPiA+Pj4+PiB0byB1c2VybGFuZC4KPiA+Pj4+ Pgo+ID4+Pj4+IElmIHlvdSBwcmVmZXIsIGFuZCBJIHVuZGVyc3RhbmQgdGhhdCB0aGlzIGlzIHRo ZSBjYXNlLCBJIGNhbiBkZWZpbmUgYQo+ID4+Pj4+IHNwZWNpZmljIFZGSU9fSU9NTVUgc3RydWN0 dXJlIHdpdGggb25seSB0aGUgZmllbGRzIHJlbGV2YW50IHRvIHRoZSB1c2VyLAo+ID4+Pj4+IGxl YXZpbmcgZnV0dXJlIGVuaGFuY2VtZW50IG9mIHRoZSB1c2VyJ3MgaW50ZXJmYWNlIGJlaW5nIGlt cGxlbWVudGVkIGluCj4gPj4+Pj4gYW5vdGhlciBrZXJuZWwgcGF0Y2ggd2hlbiB0aGUgdGltZSBo YXMgY29tZS4gIAo+ID4gCj4gPiBUQkgsIEkgaGFkIG5vIGlkZWEgdGhhdCBDTFAgaXMgYW4gczM5 MCBmaXJtd2FyZSBpbnRlcmZhY2UgYW5kIHRoaXMgaXMKPiA+IGp1c3QgZHVtcGluZyB0aGF0IHRv IHVzZXJzcGFjZS4gIFRoZSBjb3ZlciBsZXR0ZXIgc2F5czoKPiA+IAo+ID4gICAgVXNpbmcgdGhl IFBDSSBWRklPIGludGVyZmFjZSBhbGxvd3MgdXNlcmxhbmQsIGEuay5hLiBRRU1VLCB0bwo+ID4g ICAgcmV0cmlldmUgWlBDSSBzcGVjaWZpYyBpbmZvcm1hdGlvbiB3aXRob3V0IGtub3dpbmcgWiBz cGVjaWZpYwo+ID4gICAgaWRlbnRpZmllcnMgbGlrZSB0aGUgZnVuY3Rpb24gSUQgb3IgdGhlIGZ1 bmN0aW9uIGhhbmRsZSBvZiB0aGUgelBDSQo+ID4gICAgZnVuY3Rpb24gaGlkZGVuIGJlaGluZCB0 aGUgUENJIGludGVyZmFjZS4KPiA+IAo+ID4gQnV0IHdoYXQgZG9lcyB0aGlzIGFsbG93IHVzZXJs YW5kIHRvIGRvIGFuZCB3aGF0IHNwZWNpZmljIHBpZWNlcyBvZgo+ID4gaW5mb3JtYXRpb24gZG8g dGhleSBuZWVkPyAgV2UgZG8gaGF2ZSBhIGNhc2UgYWxyZWFkeSB3aGVyZSBJbnRlbAo+ID4gZ3Jh cGhpY3MgZGV2aWNlcyBoYXZlIGEgdGFibGUgKE9wUmVnaW9uKSBsaXZpbmcgaW4gaG9zdCBzeXN0 ZW0gbWVtb3J5Cj4gPiB0aGF0IHdlIGV4cG9zZSB2aWEgYSB2ZmlvIHJlZ2lvbiwgc28gaXQgd291 bGRuJ3QgYmUgdW5wcmVjZWRlbnRlZCB0byBkbwo+ID4gc29tZXRoaW5nIGxpa2UgdGhpcywgYnV0 IGFzIENvbm5pZSBzdWdnZXN0cywgaWYgd2Uga25ldyB3aGF0IHdhcyBiZWluZwo+ID4gY29uc3Vt ZWQgaGVyZSBhbmQgd2h5LCBtYXliZSB3ZSBjb3VsZCBnZW5lcmFsaXplIGl0IGludG8gc29tZXRo aW5nCj4gPiB1c2VmdWwgZm9yIG90aGVycy4gIAo+IAo+IE9LLCBzb3JyeSBJIHRyeSB0byBleHBs YWluIGJldHRlci4KPiAKPiAxKSBBIHNob3J0IGRlc2NyaXB0aW9uLCBvZiB6UENJIGZ1bmN0aW9u cyBhbmQgZ3JvdXBzCj4gCj4gSU4gWiwgUENJIGNhcmRzLCBsZWF2ZSBiZWhpbmQgYW4gYWRhcHRl ciBiZXR3ZWVuIHN1YmNoYW5uZWxzIGFuZCBQQ0kuCj4gV2UgYWNjZXNzIFBDSSBjYXJkcyB0aHJv dWdoIDIgd2F5czoKPiAtIGRlZGljYXRlZCBQQ0kgaW5zdHJ1Y3Rpb25zIChwY2lfbG9hZC9wY2lf c3RvcmUvcGNpL3N0b3JlX2Jsb2NrKQo+IC0gRE1BCj4gV2UgcmVjZWl2ZSBldmVudHMgdGhyb3Vn aAo+IC0gQWRhcHRlciBpbnRlcnJ1cHRzCj4gLSBDSFNDIGV2ZW50cwo+IAo+IFRoZSBhZGFwdGVy IHByb3Bvc2UgYW4gSU9NTVUgdG8gcHJvdGVjdCB0aGUgRE1BCj4gYW5kIHRoZSBpbnRlcnJ1cHQg aGFuZGxpbmcgZ29lcyB0aHJvdWdoIGEgTVNJWCBsaWtlIGludGVyZmFjZSBoYW5kbGVkIGJ5IAo+ IHRoZSBhZGFwdGVyLgo+IAo+IFRoZSBhcmNoaXRlY3R1cmUgc3BlY2lmaWMgUENJIGRvIHRoZSBp bnRlcmZhY2UgYmV0d2VlbiB0aGUgc3RhbmRhcmQgUENJIAo+IGxldmVsIGFuZCB0aGUgelBDSSBm dW5jdGlvbiAoUENJICsgRE1BL0lPTU1VL0ludGVycnVwdCkKPiAKPiBUbyBoYW5kbGUgdGhlIGNv bW11bmljYXRpb24gdGhyb3VnaCB0aGUgInpQQ0kgd2F5IiB0aGUgQ0xQIGludGVyZmFjZSAKPiBw cm92aWRlcyBpbnN0cnVjdGlvbnMgdG8gcmV0cmlldmUgaW5mb3JtYXRpb25zIGZyb20gdGhlIGFk YXB0ZXJzLgo+IAo+IFRoZXJlIGFyZSBkaWZmZXJlbnQgZ3JvdXAgb2YgZnVuY3Rpb25zIGhhdmlu ZyBzYW1lIGZ1bmN0aW9uYWxpdGllcy4KPiAKPiBjbHBfbGlzdCBnaXZlIHVzIGEgbGlzdCBmcm9t IHpQQ0kgZnVuY3Rpb25zCj4gY2xwX3F1ZXJ5X3BjaV9mdW5jdGlvbiByZXR1cm5zIGluZm9ybWF0 aW9ucyBzcGVjaWZpYyB0byBhIGZ1bmN0aW9uCj4gY2xwX3F1ZXJ5X2dyb3VwIHJldHVybnMgaW5m b3JtYXRpb24gb24gYSBmdW5jdGlvbiBncm91cAo+IAo+IAo+IDIpIFdoeSBkbyB3ZSBuZWVkIGl0 IGluIHRoZSBndWVzdAo+IAo+IFdlIG5lZWQgdG8gcHJvdmlkZSB0aGUgZ3Vlc3Qgd2l0aCBpbmZv cm1hdGlvbiBvbiB0aGUgYWRhcHRlcnMgYW5kIHpQQ0kgCj4gZnVuY3Rpb25zIHJldHVybmVkIGJ5 IHRoZSBjbHBfcXVlcnkgaW5zdHJ1Y3Rpb24gc28gdGhhdCB0aGUgZ3Vlc3QncyAKPiBkcml2ZXIg Z2V0cyB0aGUgcmlnaHQgaW5mb3JtYXRpb24gb24gaG93IHRoZSB3YXkgdG8gdGhlIHpQQ0kgZnVu Y3Rpb24gCj4gaGFzIGJlZW4gYnVpbHQgaW4gdGhlIGhvc3QuCj4gCj4gCj4gV2hlbiBhIGd1ZXN0 IGlzc3VlcyB0aGUgQ0xQIGluc3RydWN0aW9ucyB3ZSBpbnRlcmNlcHQgdGhlIGNscCBjb21tYW5k IGluIAo+IFFFTVUgYW5kIHdlIG5lZWQgdG8gZmVlZCB0aGUgcmVzcG9uc2Ugd2l0aCB0aGUgcmln aHQgdmFsdWVzIGZvciB0aGUgZ3Vlc3QuCj4gVGhlICJyaWdodCIgdmFsdWVzIGFyZSBub3QgdGhl IHJhdyBDTFAgcmVzcG9uc2UgdmFsdWVzOgo+IAo+IC0gc29tZSBpZGVudGlmaWVyIG11c3QgYmUg dmlydHVhbGl6ZWQsIGxpa2UgVUlEIGFuZCBGSUQsCj4gCj4gLSBzb21lIHZhbHVlcyBtdXN0IG1h dGNoIHdoYXQgdGhlIGhvc3QgcmVjZWl2ZWQgZnJvbSB0aGUgQ0xQIHJlc3BvbnNlLCAKPiBsaWtl IHRoZSBzaXplIG9mIHRoZSB0cmFuc21pdGVkIGJsb2NrcywgdGhlIERNQSBBZGRyZXNzIFNwYWNl IE1hc2ssIAo+IG51bWJlciBvZiBpbnRlcnJ1cHQsIE1TSUEKPiAKPiAtIHNvbWUgb3RoZXIgbXVz dCBtYXRjaCB3aGF0IHRoZSBob3N0IGhhbmRsZWQgd2l0aCB0aGUgYWRhcHRlciBhbmQgCj4gZnVu Y3Rpb24sIHRoZSBzdGFydCBhbmQgZW5kIG9mIERNQSwKPiAKPiAtIHNvbWUgd2hhdCB0aGUgaG9z dCBJT01NVSBkcml2ZXIgc3VwcG9ydHMgKGZyYW1lIHNpemUpLAoKVGhpcyBzZWVtcyB2ZXJ5IHJl bWluaXNjZW50IG9mIHZpcnR1YWxpemluZyBQQ0kgY29uZmlnIHNwYWNlLi4uIHNvIHdoeQppcyB0 aGlzIGJlaW5nIHByb3Bvc2VkIGFzIGEgVkZJTyBJT01NVSBpb2N0bCBleHRlbnNpb24/ICBUaGVz ZSBhcmUgYWxsCmZ1bmN0aW9uIGxldmVsIGNoYXJhY3RlcmlzdGljcywgcmlnaHQ/ICBTaG91bGQg dGhpcyBiZSBhIGNhcGFiaWxpdHkgb24KdGhlIFZGSU8gZGV2aWNlLCBvciBwZXJoYXBzIGEgcmVn aW9uIGxpa2Ugd2UgdXNlZCBmb3IgdGhlIEludGVsCk9wUmVnaW9uICh0aG91Z2ggdGhlIHN0cnVj dHVyZSBzaXplIHNlZW1zIG1vcmUgYWtpbiB0byBhIGNhcGFiaWxpdHkKaGVyZSk/ICBBcyBJIG1l bnRpb25lZCBpbiBteSBwcmV2aW91cyByZXBseSwgdHlpbmcgdGhpcyBpbnRvIHRoZSBJT01NVQpp bnRlcmZhY2Ugc2VlbWVkIHRvIHJlbHkgb24gKEkgYXNzdW1lKSBhbiBvbmUtdG8tb25lLXRvLW9u ZSBtYXBwaW5nIG9mClBDSSBmdW5jdGlvbiB0byBJT01NVSBncm91cCB0byBJT01NVSBkb21haW4s IGJ1dCB0aGF0IGRvZXNuJ3Qgc3RpbGwKZG9lc24ndCBuZWNlc3NhcmlseSBsZW5kIGl0c2VsZiB0 byB1c2luZyB0aGUgSU9NTVUgZm9yIGRldmljZSBsZXZlbAppbmZvcm1hdGlvbi4gIElmIHRoZXJl IGlzIElPTU1VIGluZm8sIHBlcmhhcHMgaXQgbmVlZHMgdG8gYmUgc3BsaXQsIGllLgpleHBvc2Ug YSBmcmFtZSBzaXplIHZpYSBkb21haW5fZ2V0X2F0dHIsIGV4cG9zZSBkZXZpY2UgbGV2ZWwgZmVh dHVyZXMKdmlhIGEgZGV2aWNlIGNhcGFiaWxpdHksIGxldCBRRU1VIGFzc2VtYmxlIHRoZXNlIGlu dG8gc29tZXRoaW5nCmNvaGVyZW50IHRvIGVtdWxhdGUgdGhlIGNscCBpbnRlcmZhY2UuCgo+IDMp IFdlIGhhdmUgdGhyZWUgZGlmZmVyZW50IHdheSB0byBnZXQgVGhpcyBpbmZvcm1hdGlvbjoKPiAK PiBUaGUgUENJIExpbnV4IGludGVyZmFjZSBpcyBhIHN0YW5kYXJkIFBDSSBpbnRlcmZhY2UgYW5k IHNvbWUgWiBzcGVjaWZpYyAKPiBpbmZvcm1hdGlvbiBpcyBhdmFpbGFibGUgaW4gc3lzZnMuCj4g Tm90IGFsbCB0aGUgaW5mb3JtYXRpb24gbmVlZGVkIHRvIGJlIHJldHVybmVkIGluc2lkZSB0aGUg Q0xQIHJlc3BvbnNlIGlzIAo+IGF2YWlsYWJsZS4KPiBTbyB3ZSBjYW4gbm90IHVzZSB0aGUgc3lz ZnMgaW50ZXJmYWNlIHRvIGdldCBhbGwgdGhlIGluZm9ybWF0aW9uLgo+IAo+IFRoZXJlIGlzIGEg Q0xQIGlvY3RsIGludGVyZmFjZSBidXQgdGhpcyBpbnRlcmZhY2UgaXMgbm90IHNlY3VyZSBpbiB0 aGF0IAo+IGl0IHJldHVybnMgdGhlIGluZm9ybWF0aW9uIGZvciBhbGwgYWRhcHRlcnMgaW4gdGhl IHN5c3RlbS4KPiAKPiBUaGUgVkZJTyBpbnRlcmZhY2Ugb2ZmZXJzIHRoZSBhZHZhbnRhZ2UgdG8g cG9pbnQgdG8gYSBzaW5nbGUgUENJIAo+IGZ1bmN0aW9uLCBzbyBtb3JlIHNlY3VyZSB0aGFuIHRo ZSBjbHAgaW9jdGwgaW50ZXJmYWNlLgo+IENvdXBsZWQgd2l0aCB0aGUgczM5MF9pb21tdSB3ZSBn ZXQgYWNjZXNzIHRvIHRoZSB6UENJIENMUCBpbnN0cnVjdGlvbiAKPiBhbmQgdG8gdGhlIHZhbHVl cyBoYW5kbGVkIGJ5IHRoZSB6UENJIGRyaXZlci4KPiAKPiAKPiA0KSBVbnRpbCBub3cgd2UgdXNl ZCB0byBmaWxsIHRoZSBDTFAgcmVzcG9uc2UgdG8gdGhlIGd1ZXN0IGluc2lkZSBRRU1VIAo+IHdp dGggZml4ZWQgdmFsdWVzIGNvcnJlc3BvbmRpbmcgdG8gdGhlIG9ubHkgUENJIGNhcmQgd2Ugc3Vw cG9ydGVkLgo+IFRvIHN1cHBvcnQgbmV3IGNhcmRzIHdlIG5lZWQgdG8gZ2V0IHRoZSByaWdodCB2 YWx1ZXMgZnJvbSB0aGUga2VybmVsIG91dC4KCklmIGl0J3MgYWxyZWFkeSBlbXVsYXRlZCwgSSBt dWNoIHByZWZlciBmaWd1cmluZyBvdXQgaG93IHRvIGV4cG9zZSB0aGUKcmlnaHQgcGllY2VzIG9m IGluZm9ybWF0aW9uIHZpYSBhbiBhcHByb3ByaWF0ZSBpbnRlcmZhY2UgdG8gdmlydHVhbGl6ZQpm aWVsZHMgdGhhdCBhcmUgYWN0dWFsbHkgbmVjZXNzYXJ5IHJhdGhlciB0aGFuIHNpbXBseSBwcm92 aWRpbmcgYW4KaW50ZXJmYWNlIHRvIGR1bXAgdGhlIGNscCBpbmZvIHN0cmFpZ2h0IHRvIHVzZXJz cGFjZSBhbmQgcGlwZSBpdCB0byB0aGUKVk0uICBUaGFua3MsCgpBbGV4Cl9fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmlvbW11IG1haWxpbmcgbGlzdAppb21t dUBsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZwpodHRwczovL2xpc3RzLmxpbnV4Zm91bmRhdGlv bi5vcmcvbWFpbG1hbi9saXN0aW5mby9pb21tdQ==