From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4ACE9C433E0 for ; Wed, 1 Jul 2020 12:18:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2747620772 for ; Wed, 1 Jul 2020 12:18:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730561AbgGAMS2 (ORCPT ); Wed, 1 Jul 2020 08:18:28 -0400 Received: from foss.arm.com ([217.140.110.172]:35312 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728129AbgGAMS2 (ORCPT ); Wed, 1 Jul 2020 08:18:28 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4FBAC30E; Wed, 1 Jul 2020 05:18:27 -0700 (PDT) Received: from [10.57.21.32] (unknown [10.57.21.32]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C4C563F73C; Wed, 1 Jul 2020 05:18:25 -0700 (PDT) Subject: Re: [PATCH 1/2] iommu: Add iommu_group_get/set_domain() To: Lu Baolu , Joerg Roedel , Alex Williamson Cc: Kevin Tian , Dave Jiang , Ashok Raj , kvm@vger.kernel.org, Cornelia Huck , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org References: <20200627031532.28046-1-baolu.lu@linux.intel.com> <5dc1cece-6111-9b56-d04c-9553d592675b@linux.intel.com> <48dd9f1e-c18b-77b7-650a-c35ecbb69f2b@arm.com> From: Robin Murphy Message-ID: Date: Wed, 1 Jul 2020 13:18:23 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2020-07-01 08:32, Lu Baolu wrote: > Hi Robin, > > On 2020/7/1 0:51, Robin Murphy wrote: >> On 2020-06-30 02:03, Lu Baolu wrote: >>> Hi Robin, >>> >>> On 6/29/20 7:56 PM, Robin Murphy wrote: >>>> On 2020-06-27 04:15, Lu Baolu wrote: >>>>> The hardware assistant vfio mediated device is a use case of iommu >>>>> aux-domain. The interactions between vfio/mdev and iommu during mdev >>>>> creation and passthr are: >>>>> >>>>> - Create a group for mdev with iommu_group_alloc(); >>>>> - Add the device to the group with >>>>>          group = iommu_group_alloc(); >>>>>          if (IS_ERR(group)) >>>>>                  return PTR_ERR(group); >>>>> >>>>>          ret = iommu_group_add_device(group, &mdev->dev); >>>>>          if (!ret) >>>>>                  dev_info(&mdev->dev, "MDEV: group_id = %d\n", >>>>>                           iommu_group_id(group)); >>>>> - Allocate an aux-domain >>>>>     iommu_domain_alloc() >>>>> - Attach the aux-domain to the physical device from which the mdev is >>>>>    created. >>>>>     iommu_aux_attach_device() >>>>> >>>>> In the whole process, an iommu group was allocated for the mdev and an >>>>> iommu domain was attached to the group, but the group->domain leaves >>>>> NULL. As the result, iommu_get_domain_for_dev() doesn't work anymore. >>>>> >>>>> This adds iommu_group_get/set_domain() so that group->domain could be >>>>> managed whenever a domain is attached or detached through the >>>>> aux-domain >>>>> api's. >>>> >>>> Letting external callers poke around directly in the internals of >>>> iommu_group doesn't look right to me. >>> >>> Unfortunately, it seems that the vifo iommu abstraction is deeply bound >>> to the IOMMU subsystem. We can easily find other examples: >>> >>> iommu_group_get/set_iommudata() >>> iommu_group_get/set_name() >>> ... >> >> Sure, but those are ways for users of a group to attach useful >> information of their own to it, that doesn't matter to the IOMMU >> subsystem itself. The interface you've proposed gives callers rich new >> opportunities to fundamentally break correct operation of the API: >> >>      dom = iommu_domain_alloc(); >>      iommu_attach_group(dom, grp); >>      ... >>      iommu_group_set_domain(grp, NULL); >>      // oops, leaked and can't ever detach properly now >> >> or perhaps: >> >>      grp = iommu_group_alloc(); >>      iommu_group_add_device(grp, dev); >>      iommu_group_set_domain(grp, dom); >>      ... >>      iommu_detach_group(dom, grp); >>      // oops, IOMMU driver might not handle this >> >>>> If a regular device is attached to one or more aux domains for PASID >>>> use, iommu_get_domain_for_dev() is still going to return the primary >>>> domain, so why should it be expected to behave differently for mediated >>> >>> Unlike the normal device attach, we will encounter two devices when it >>> comes to aux-domain. >>> >>> - Parent physical device - this might be, for example, a PCIe device >>> with PASID feature support, hence it is able to tag an unique PASID >>> for DMA transfers originated from its subset. The device driver hence >>> is able to wrapper this subset into an isolated: >>> >>> - Mediated device - a fake device created by the device driver mentioned >>> above. >>> >>> Yes. All you mentioned are right for the parent device. But for mediated >>> device, iommu_get_domain_for_dev() doesn't work even it has an valid >>> iommu_group and iommu_domain. >>> >>> iommu_get_domain_for_dev() is a necessary interface for device drivers >>> which want to support aux-domain. For example, >> >> Only if they want to follow this very specific notion of using made-up >> devices and groups to represent aux attachments. Even if a driver >> managing its own aux domains entirely privately does create child >> devices for them, it's not like it can't keep its domain pointers in >> drvdata if it wants to ;) >> >> Let's not conflate the current implementation of vfio_mdev with the >> general concepts involved here. >> >>>            struct iommu_domain *domain; >>>            struct device *dev = mdev_dev(mdev); >>>        unsigned long pasid; >>> >>>            domain = iommu_get_domain_for_dev(dev); >>>            if (!domain) >>>                    return -ENODEV; >>> >>>            pasid = iommu_aux_get_pasid(domain, dev->parent); >>>        if (pasid == IOASID_INVALID) >>>            return -EINVAL; >>> >>>        /* Program the device context with the PASID value */ >>>        .... >>> >>> Without this fix, iommu_get_domain_for_dev() always returns NULL and the >>> device driver has no means to support aux-domain. >> >> So either the IOMMU API itself is missing the ability to do the right >> thing internally, or the mdev layer isn't using it appropriately. >> Either way, simply punching holes in the API for mdev to hack around >> its own mess doesn't seem like the best thing to do. >> >> The initial impression I got was that it's implicitly assumed here >> that the mdev itself is attached to exactly one aux domain and nothing >> else, at which point I would wonder why it's using aux at all, but are >> you saying that in fact no attach happens with the mdev group either >> way, only to the parent device? >> >> I'll admit I'm not hugely familiar with any of this, but it seems to >> me that the logical flow should be: >> >>      - allocate domain >>      - attach as aux to parent >>      - retrieve aux domain PASID >>      - create mdev child based on PASID >>      - attach mdev to domain (normally) >> >> Of course that might require giving the IOMMU API a proper first-class >> notion of mediated devices, such that it knows the mdev represents the >> PASID, and can recognise the mdev attach is equivalent to the earlier >> parent aux attach so not just blindly hand it down to an IOMMU driver >> that's never heard of this new device before. Or perhaps the IOMMU >> drivers do their own bookkeeping for the mdev bus, such that they do >> handle the attach call, and just validate it internally based on the >> associated parent device and PASID. Either way, the inside maintains >> self-consistency and from the outside it looks like standard API usage >> without nasty hacks. >> >> I'm pretty sure I've heard suggestions of using mediated devices >> beyond VFIO (e.g. within the kernel itself), so chances are this is a >> direction that we'll have to take at some point anyway. >> >> And, that said, even if people do want an immediate quick fix >> regardless of technical debt, I'd still be a lot happier to see >> iommu_group_set_domain() lightly respun as iommu_attach_mdev() ;) > > Get your point and I agree with your concerns. > > To maintain the relationship between mdev's iommu_group and > iommu_domain, how about extending below existing aux_attach api > > int iommu_aux_attach_device(struct iommu_domain *domain, >                 struct device *dev) > > by adding the mdev's iommu_group? > > int iommu_aux_attach_device(struct iommu_domain *domain, >                 struct device *dev, >                 struct iommu_group *group) > > And, in iommu_aux_attach_device(), we require, >  - @group only has a single device; >  - @group hasn't been attached by any devices; >  - Set the @domain to @group > > Just like what we've done in iommu_attach_device(). > > Any thoughts? Rather than pass a bare iommu_group with implicit restrictions, it might be neater to just pass an mdev_device, so that the IOMMU core can also take care of allocating and setting up the group. Then we flag the group internally as a special "mdev group" such that we can prevent callers from subsequently trying to add/remove devices or attach/detach its domain directly. That seems like it would make a pretty straightforward and robust API extension, as long as the mdev argument here is optional so that SVA and other aux users don't have to care. Other than the slightly different ordering where caller would have to allocate the mdev first, then finish it's PASID-based configuration afterwards, I guess it's not far off what I was thinking yesterday :) Robin. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83DF4C433E0 for ; Wed, 1 Jul 2020 12:18:35 +0000 (UTC) Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3B29B20760 for ; Wed, 1 Jul 2020 12:18:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3B29B20760 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id B80C08945A; Wed, 1 Jul 2020 12:18:34 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qFEMcaWQdcGK; Wed, 1 Jul 2020 12:18:32 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id 41D87893DB; Wed, 1 Jul 2020 12:18:32 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 25AFAC0865; Wed, 1 Jul 2020 12:18:32 +0000 (UTC) Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 3EC0AC0733 for ; Wed, 1 Jul 2020 12:18:30 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 3833C8BB64 for ; Wed, 1 Jul 2020 12:18:30 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QvUGYV6sAlND for ; Wed, 1 Jul 2020 12:18:28 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by whitealder.osuosl.org (Postfix) with ESMTP id 118778BB5D for ; Wed, 1 Jul 2020 12:18:27 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4FBAC30E; Wed, 1 Jul 2020 05:18:27 -0700 (PDT) Received: from [10.57.21.32] (unknown [10.57.21.32]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C4C563F73C; Wed, 1 Jul 2020 05:18:25 -0700 (PDT) Subject: Re: [PATCH 1/2] iommu: Add iommu_group_get/set_domain() To: Lu Baolu , Joerg Roedel , Alex Williamson References: <20200627031532.28046-1-baolu.lu@linux.intel.com> <5dc1cece-6111-9b56-d04c-9553d592675b@linux.intel.com> <48dd9f1e-c18b-77b7-650a-c35ecbb69f2b@arm.com> From: Robin Murphy Message-ID: Date: Wed, 1 Jul 2020 13:18:23 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-GB Cc: Kevin Tian , Dave Jiang , Ashok Raj , kvm@vger.kernel.org, Cornelia Huck , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" T24gMjAyMC0wNy0wMSAwODozMiwgTHUgQmFvbHUgd3JvdGU6Cj4gSGkgUm9iaW4sCj4gCj4gT24g MjAyMC83LzEgMDo1MSwgUm9iaW4gTXVycGh5IHdyb3RlOgo+PiBPbiAyMDIwLTA2LTMwIDAyOjAz LCBMdSBCYW9sdSB3cm90ZToKPj4+IEhpIFJvYmluLAo+Pj4KPj4+IE9uIDYvMjkvMjAgNzo1NiBQ TSwgUm9iaW4gTXVycGh5IHdyb3RlOgo+Pj4+IE9uIDIwMjAtMDYtMjcgMDQ6MTUsIEx1IEJhb2x1 IHdyb3RlOgo+Pj4+PiBUaGUgaGFyZHdhcmUgYXNzaXN0YW50IHZmaW8gbWVkaWF0ZWQgZGV2aWNl IGlzIGEgdXNlIGNhc2Ugb2YgaW9tbXUKPj4+Pj4gYXV4LWRvbWFpbi4gVGhlIGludGVyYWN0aW9u cyBiZXR3ZWVuIHZmaW8vbWRldiBhbmQgaW9tbXUgZHVyaW5nIG1kZXYKPj4+Pj4gY3JlYXRpb24g YW5kIHBhc3N0aHIgYXJlOgo+Pj4+Pgo+Pj4+PiAtIENyZWF0ZSBhIGdyb3VwIGZvciBtZGV2IHdp dGggaW9tbXVfZ3JvdXBfYWxsb2MoKTsKPj4+Pj4gLSBBZGQgdGhlIGRldmljZSB0byB0aGUgZ3Jv dXAgd2l0aAo+Pj4+PiDCoMKgwqDCoMKgwqDCoMKgIGdyb3VwID0gaW9tbXVfZ3JvdXBfYWxsb2Mo KTsKPj4+Pj4gwqDCoMKgwqDCoMKgwqDCoCBpZiAoSVNfRVJSKGdyb3VwKSkKPj4+Pj4gwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgcmV0dXJuIFBUUl9FUlIoZ3JvdXApOwo+Pj4+Pgo+ Pj4+PiDCoMKgwqDCoMKgwqDCoMKgIHJldCA9IGlvbW11X2dyb3VwX2FkZF9kZXZpY2UoZ3JvdXAs ICZtZGV2LT5kZXYpOwo+Pj4+PiDCoMKgwqDCoMKgwqDCoMKgIGlmICghcmV0KQo+Pj4+PiDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBkZXZfaW5mbygmbWRldi0+ZGV2LCAiTURFVjog Z3JvdXBfaWQgPSAlZFxuIiwKPj4+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqAgaW9tbXVfZ3JvdXBfaWQoZ3JvdXApKTsKPj4+Pj4gLSBBbGxvY2F0 ZSBhbiBhdXgtZG9tYWluCj4+Pj4+IMKgwqDCoMKgaW9tbXVfZG9tYWluX2FsbG9jKCkKPj4+Pj4g LSBBdHRhY2ggdGhlIGF1eC1kb21haW4gdG8gdGhlIHBoeXNpY2FsIGRldmljZSBmcm9tIHdoaWNo IHRoZSBtZGV2IGlzCj4+Pj4+IMKgwqAgY3JlYXRlZC4KPj4+Pj4gwqDCoMKgwqBpb21tdV9hdXhf YXR0YWNoX2RldmljZSgpCj4+Pj4+Cj4+Pj4+IEluIHRoZSB3aG9sZSBwcm9jZXNzLCBhbiBpb21t dSBncm91cCB3YXMgYWxsb2NhdGVkIGZvciB0aGUgbWRldiBhbmQgYW4KPj4+Pj4gaW9tbXUgZG9t YWluIHdhcyBhdHRhY2hlZCB0byB0aGUgZ3JvdXAsIGJ1dCB0aGUgZ3JvdXAtPmRvbWFpbiBsZWF2 ZXMKPj4+Pj4gTlVMTC4gQXMgdGhlIHJlc3VsdCwgaW9tbXVfZ2V0X2RvbWFpbl9mb3JfZGV2KCkg ZG9lc24ndCB3b3JrIGFueW1vcmUuCj4+Pj4+Cj4+Pj4+IFRoaXMgYWRkcyBpb21tdV9ncm91cF9n ZXQvc2V0X2RvbWFpbigpIHNvIHRoYXQgZ3JvdXAtPmRvbWFpbiBjb3VsZCBiZQo+Pj4+PiBtYW5h Z2VkIHdoZW5ldmVyIGEgZG9tYWluIGlzIGF0dGFjaGVkIG9yIGRldGFjaGVkIHRocm91Z2ggdGhl IAo+Pj4+PiBhdXgtZG9tYWluCj4+Pj4+IGFwaSdzLgo+Pj4+Cj4+Pj4gTGV0dGluZyBleHRlcm5h bCBjYWxsZXJzIHBva2UgYXJvdW5kIGRpcmVjdGx5IGluIHRoZSBpbnRlcm5hbHMgb2YgCj4+Pj4g aW9tbXVfZ3JvdXAgZG9lc24ndCBsb29rIHJpZ2h0IHRvIG1lLgo+Pj4KPj4+IFVuZm9ydHVuYXRl bHksIGl0IHNlZW1zIHRoYXQgdGhlIHZpZm8gaW9tbXUgYWJzdHJhY3Rpb24gaXMgZGVlcGx5IGJv dW5kCj4+PiB0byB0aGUgSU9NTVUgc3Vic3lzdGVtLiBXZSBjYW4gZWFzaWx5IGZpbmQgb3RoZXIg ZXhhbXBsZXM6Cj4+Pgo+Pj4gaW9tbXVfZ3JvdXBfZ2V0L3NldF9pb21tdWRhdGEoKQo+Pj4gaW9t bXVfZ3JvdXBfZ2V0L3NldF9uYW1lKCkKPj4+IC4uLgo+Pgo+PiBTdXJlLCBidXQgdGhvc2UgYXJl IHdheXMgZm9yIHVzZXJzIG9mIGEgZ3JvdXAgdG8gYXR0YWNoIHVzZWZ1bCAKPj4gaW5mb3JtYXRp b24gb2YgdGhlaXIgb3duIHRvIGl0LCB0aGF0IGRvZXNuJ3QgbWF0dGVyIHRvIHRoZSBJT01NVSAK Pj4gc3Vic3lzdGVtIGl0c2VsZi4gVGhlIGludGVyZmFjZSB5b3UndmUgcHJvcG9zZWQgZ2l2ZXMg Y2FsbGVycyByaWNoIG5ldyAKPj4gb3Bwb3J0dW5pdGllcyB0byBmdW5kYW1lbnRhbGx5IGJyZWFr IGNvcnJlY3Qgb3BlcmF0aW9uIG9mIHRoZSBBUEk6Cj4+Cj4+IMKgwqDCoMKgwqBkb20gPSBpb21t dV9kb21haW5fYWxsb2MoKTsKPj4gwqDCoMKgwqDCoGlvbW11X2F0dGFjaF9ncm91cChkb20sIGdy cCk7Cj4+IMKgwqDCoMKgwqAuLi4KPj4gwqDCoMKgwqDCoGlvbW11X2dyb3VwX3NldF9kb21haW4o Z3JwLCBOVUxMKTsKPj4gwqDCoMKgwqDCoC8vIG9vcHMsIGxlYWtlZCBhbmQgY2FuJ3QgZXZlciBk ZXRhY2ggcHJvcGVybHkgbm93Cj4+Cj4+IG9yIHBlcmhhcHM6Cj4+Cj4+IMKgwqDCoMKgwqBncnAg PSBpb21tdV9ncm91cF9hbGxvYygpOwo+PiDCoMKgwqDCoMKgaW9tbXVfZ3JvdXBfYWRkX2Rldmlj ZShncnAsIGRldik7Cj4+IMKgwqDCoMKgwqBpb21tdV9ncm91cF9zZXRfZG9tYWluKGdycCwgZG9t KTsKPj4gwqDCoMKgwqDCoC4uLgo+PiDCoMKgwqDCoMKgaW9tbXVfZGV0YWNoX2dyb3VwKGRvbSwg Z3JwKTsKPj4gwqDCoMKgwqDCoC8vIG9vcHMsIElPTU1VIGRyaXZlciBtaWdodCBub3QgaGFuZGxl IHRoaXMKPj4KPj4+PiBJZiBhIHJlZ3VsYXIgZGV2aWNlIGlzIGF0dGFjaGVkIHRvIG9uZSBvciBt b3JlIGF1eCBkb21haW5zIGZvciBQQVNJRCAKPj4+PiB1c2UsIGlvbW11X2dldF9kb21haW5fZm9y X2RldigpIGlzIHN0aWxsIGdvaW5nIHRvIHJldHVybiB0aGUgcHJpbWFyeSAKPj4+PiBkb21haW4s IHNvIHdoeSBzaG91bGQgaXQgYmUgZXhwZWN0ZWQgdG8gYmVoYXZlIGRpZmZlcmVudGx5IGZvciBt ZWRpYXRlZAo+Pj4KPj4+IFVubGlrZSB0aGUgbm9ybWFsIGRldmljZSBhdHRhY2gsIHdlIHdpbGwg ZW5jb3VudGVyIHR3byBkZXZpY2VzIHdoZW4gaXQKPj4+IGNvbWVzIHRvIGF1eC1kb21haW4uCj4+ Pgo+Pj4gLSBQYXJlbnQgcGh5c2ljYWwgZGV2aWNlIC0gdGhpcyBtaWdodCBiZSwgZm9yIGV4YW1w bGUsIGEgUENJZSBkZXZpY2UKPj4+IHdpdGggUEFTSUQgZmVhdHVyZSBzdXBwb3J0LCBoZW5jZSBp dCBpcyBhYmxlIHRvIHRhZyBhbiB1bmlxdWUgUEFTSUQKPj4+IGZvciBETUEgdHJhbnNmZXJzIG9y aWdpbmF0ZWQgZnJvbSBpdHMgc3Vic2V0LiBUaGUgZGV2aWNlIGRyaXZlciBoZW5jZQo+Pj4gaXMg YWJsZSB0byB3cmFwcGVyIHRoaXMgc3Vic2V0IGludG8gYW4gaXNvbGF0ZWQ6Cj4+Pgo+Pj4gLSBN ZWRpYXRlZCBkZXZpY2UgLSBhIGZha2UgZGV2aWNlIGNyZWF0ZWQgYnkgdGhlIGRldmljZSBkcml2 ZXIgbWVudGlvbmVkCj4+PiBhYm92ZS4KPj4+Cj4+PiBZZXMuIEFsbCB5b3UgbWVudGlvbmVkIGFy ZSByaWdodCBmb3IgdGhlIHBhcmVudCBkZXZpY2UuIEJ1dCBmb3IgbWVkaWF0ZWQKPj4+IGRldmlj ZSwgaW9tbXVfZ2V0X2RvbWFpbl9mb3JfZGV2KCkgZG9lc24ndCB3b3JrIGV2ZW4gaXQgaGFzIGFu IHZhbGlkCj4+PiBpb21tdV9ncm91cCBhbmQgaW9tbXVfZG9tYWluLgo+Pj4KPj4+IGlvbW11X2dl dF9kb21haW5fZm9yX2RldigpIGlzIGEgbmVjZXNzYXJ5IGludGVyZmFjZSBmb3IgZGV2aWNlIGRy aXZlcnMKPj4+IHdoaWNoIHdhbnQgdG8gc3VwcG9ydCBhdXgtZG9tYWluLiBGb3IgZXhhbXBsZSwK Pj4KPj4gT25seSBpZiB0aGV5IHdhbnQgdG8gZm9sbG93IHRoaXMgdmVyeSBzcGVjaWZpYyBub3Rp b24gb2YgdXNpbmcgbWFkZS11cCAKPj4gZGV2aWNlcyBhbmQgZ3JvdXBzIHRvIHJlcHJlc2VudCBh dXggYXR0YWNobWVudHMuIEV2ZW4gaWYgYSBkcml2ZXIgCj4+IG1hbmFnaW5nIGl0cyBvd24gYXV4 IGRvbWFpbnMgZW50aXJlbHkgcHJpdmF0ZWx5IGRvZXMgY3JlYXRlIGNoaWxkIAo+PiBkZXZpY2Vz IGZvciB0aGVtLCBpdCdzIG5vdCBsaWtlIGl0IGNhbid0IGtlZXAgaXRzIGRvbWFpbiBwb2ludGVy cyBpbiAKPj4gZHJ2ZGF0YSBpZiBpdCB3YW50cyB0byA7KQo+Pgo+PiBMZXQncyBub3QgY29uZmxh dGUgdGhlIGN1cnJlbnQgaW1wbGVtZW50YXRpb24gb2YgdmZpb19tZGV2IHdpdGggdGhlIAo+PiBn ZW5lcmFsIGNvbmNlcHRzIGludm9sdmVkIGhlcmUuCj4+Cj4+PiDCoMKgwqDCoMKgwqDCoMKgwqDC oCBzdHJ1Y3QgaW9tbXVfZG9tYWluICpkb21haW47Cj4+PiDCoMKgwqDCoMKgwqDCoMKgwqDCoCBz dHJ1Y3QgZGV2aWNlICpkZXYgPSBtZGV2X2RldihtZGV2KTsKPj4+IMKgwqDCoMKgwqDCoCB1bnNp Z25lZCBsb25nIHBhc2lkOwo+Pj4KPj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgIGRvbWFpbiA9IGlv bW11X2dldF9kb21haW5fZm9yX2RldihkZXYpOwo+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgwqAgaWYg KCFkb21haW4pCj4+PiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgcmV0dXJu IC1FTk9ERVY7Cj4+Pgo+Pj4gwqDCoMKgwqDCoMKgwqDCoMKgwqAgcGFzaWQgPSBpb21tdV9hdXhf Z2V0X3Bhc2lkKGRvbWFpbiwgZGV2LT5wYXJlbnQpOwo+Pj4gwqDCoMKgwqDCoMKgIGlmIChwYXNp ZCA9PSBJT0FTSURfSU5WQUxJRCkKPj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgIHJldHVybiAtRUlO VkFMOwo+Pj4KPj4+IMKgwqDCoMKgwqDCoCAvKiBQcm9ncmFtIHRoZSBkZXZpY2UgY29udGV4dCB3 aXRoIHRoZSBQQVNJRCB2YWx1ZSAqLwo+Pj4gwqDCoMKgwqDCoMKgIC4uLi4KPj4+Cj4+PiBXaXRo b3V0IHRoaXMgZml4LCBpb21tdV9nZXRfZG9tYWluX2Zvcl9kZXYoKSBhbHdheXMgcmV0dXJucyBO VUxMIGFuZCB0aGUKPj4+IGRldmljZSBkcml2ZXIgaGFzIG5vIG1lYW5zIHRvIHN1cHBvcnQgYXV4 LWRvbWFpbi4KPj4KPj4gU28gZWl0aGVyIHRoZSBJT01NVSBBUEkgaXRzZWxmIGlzIG1pc3Npbmcg dGhlIGFiaWxpdHkgdG8gZG8gdGhlIHJpZ2h0IAo+PiB0aGluZyBpbnRlcm5hbGx5LCBvciB0aGUg bWRldiBsYXllciBpc24ndCB1c2luZyBpdCBhcHByb3ByaWF0ZWx5LiAKPj4gRWl0aGVyIHdheSwg c2ltcGx5IHB1bmNoaW5nIGhvbGVzIGluIHRoZSBBUEkgZm9yIG1kZXYgdG8gaGFjayBhcm91bmQg Cj4+IGl0cyBvd24gbWVzcyBkb2Vzbid0IHNlZW0gbGlrZSB0aGUgYmVzdCB0aGluZyB0byBkby4K Pj4KPj4gVGhlIGluaXRpYWwgaW1wcmVzc2lvbiBJIGdvdCB3YXMgdGhhdCBpdCdzIGltcGxpY2l0 bHkgYXNzdW1lZCBoZXJlIAo+PiB0aGF0IHRoZSBtZGV2IGl0c2VsZiBpcyBhdHRhY2hlZCB0byBl eGFjdGx5IG9uZSBhdXggZG9tYWluIGFuZCBub3RoaW5nIAo+PiBlbHNlLCBhdCB3aGljaCBwb2lu dCBJIHdvdWxkIHdvbmRlciB3aHkgaXQncyB1c2luZyBhdXggYXQgYWxsLCBidXQgYXJlIAo+PiB5 b3Ugc2F5aW5nIHRoYXQgaW4gZmFjdCBubyBhdHRhY2ggaGFwcGVucyB3aXRoIHRoZSBtZGV2IGdy b3VwIGVpdGhlciAKPj4gd2F5LCBvbmx5IHRvIHRoZSBwYXJlbnQgZGV2aWNlPwo+Pgo+PiBJJ2xs IGFkbWl0IEknbSBub3QgaHVnZWx5IGZhbWlsaWFyIHdpdGggYW55IG9mIHRoaXMsIGJ1dCBpdCBz ZWVtcyB0byAKPj4gbWUgdGhhdCB0aGUgbG9naWNhbCBmbG93IHNob3VsZCBiZToKPj4KPj4gwqDC oMKgwqDCoC0gYWxsb2NhdGUgZG9tYWluCj4+IMKgwqDCoMKgwqAtIGF0dGFjaCBhcyBhdXggdG8g cGFyZW50Cj4+IMKgwqDCoMKgwqAtIHJldHJpZXZlIGF1eCBkb21haW4gUEFTSUQKPj4gwqDCoMKg wqDCoC0gY3JlYXRlIG1kZXYgY2hpbGQgYmFzZWQgb24gUEFTSUQKPj4gwqDCoMKgwqDCoC0gYXR0 YWNoIG1kZXYgdG8gZG9tYWluIChub3JtYWxseSkKPj4KPj4gT2YgY291cnNlIHRoYXQgbWlnaHQg cmVxdWlyZSBnaXZpbmcgdGhlIElPTU1VIEFQSSBhIHByb3BlciBmaXJzdC1jbGFzcyAKPj4gbm90 aW9uIG9mIG1lZGlhdGVkIGRldmljZXMsIHN1Y2ggdGhhdCBpdCBrbm93cyB0aGUgbWRldiByZXBy ZXNlbnRzIHRoZSAKPj4gUEFTSUQsIGFuZCBjYW4gcmVjb2duaXNlIHRoZSBtZGV2IGF0dGFjaCBp cyBlcXVpdmFsZW50IHRvIHRoZSBlYXJsaWVyIAo+PiBwYXJlbnQgYXV4IGF0dGFjaCBzbyBub3Qg anVzdCBibGluZGx5IGhhbmQgaXQgZG93biB0byBhbiBJT01NVSBkcml2ZXIgCj4+IHRoYXQncyBu ZXZlciBoZWFyZCBvZiB0aGlzIG5ldyBkZXZpY2UgYmVmb3JlLiBPciBwZXJoYXBzIHRoZSBJT01N VSAKPj4gZHJpdmVycyBkbyB0aGVpciBvd24gYm9va2tlZXBpbmcgZm9yIHRoZSBtZGV2IGJ1cywg c3VjaCB0aGF0IHRoZXkgZG8gCj4+IGhhbmRsZSB0aGUgYXR0YWNoIGNhbGwsIGFuZCBqdXN0IHZh bGlkYXRlIGl0IGludGVybmFsbHkgYmFzZWQgb24gdGhlIAo+PiBhc3NvY2lhdGVkIHBhcmVudCBk ZXZpY2UgYW5kIFBBU0lELiBFaXRoZXIgd2F5LCB0aGUgaW5zaWRlIG1haW50YWlucyAKPj4gc2Vs Zi1jb25zaXN0ZW5jeSBhbmQgZnJvbSB0aGUgb3V0c2lkZSBpdCBsb29rcyBsaWtlIHN0YW5kYXJk IEFQSSB1c2FnZSAKPj4gd2l0aG91dCBuYXN0eSBoYWNrcy4KPj4KPj4gSSdtIHByZXR0eSBzdXJl IEkndmUgaGVhcmQgc3VnZ2VzdGlvbnMgb2YgdXNpbmcgbWVkaWF0ZWQgZGV2aWNlcyAKPj4gYmV5 b25kIFZGSU8gKGUuZy4gd2l0aGluIHRoZSBrZXJuZWwgaXRzZWxmKSwgc28gY2hhbmNlcyBhcmUg dGhpcyBpcyBhIAo+PiBkaXJlY3Rpb24gdGhhdCB3ZSdsbCBoYXZlIHRvIHRha2UgYXQgc29tZSBw b2ludCBhbnl3YXkuCj4+Cj4+IEFuZCwgdGhhdCBzYWlkLCBldmVuIGlmIHBlb3BsZSBkbyB3YW50 IGFuIGltbWVkaWF0ZSBxdWljayBmaXggCj4+IHJlZ2FyZGxlc3Mgb2YgdGVjaG5pY2FsIGRlYnQs IEknZCBzdGlsbCBiZSBhIGxvdCBoYXBwaWVyIHRvIHNlZSAKPj4gaW9tbXVfZ3JvdXBfc2V0X2Rv bWFpbigpIGxpZ2h0bHkgcmVzcHVuIGFzIGlvbW11X2F0dGFjaF9tZGV2KCkgOykKPiAKPiBHZXQg eW91ciBwb2ludCBhbmQgSSBhZ3JlZSB3aXRoIHlvdXIgY29uY2VybnMuCj4gCj4gVG8gbWFpbnRh aW4gdGhlIHJlbGF0aW9uc2hpcCBiZXR3ZWVuIG1kZXYncyBpb21tdV9ncm91cCBhbmQKPiBpb21t dV9kb21haW4sIGhvdyBhYm91dCBleHRlbmRpbmcgYmVsb3cgZXhpc3RpbmcgYXV4X2F0dGFjaCBh cGkKPiAKPiBpbnQgaW9tbXVfYXV4X2F0dGFjaF9kZXZpY2Uoc3RydWN0IGlvbW11X2RvbWFpbiAq ZG9tYWluLAo+ICDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgc3RydWN0IGRldmljZSAq ZGV2KQo+IAo+IGJ5IGFkZGluZyB0aGUgbWRldidzIGlvbW11X2dyb3VwPwo+IAo+IGludCBpb21t dV9hdXhfYXR0YWNoX2RldmljZShzdHJ1Y3QgaW9tbXVfZG9tYWluICpkb21haW4sCj4gIMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBzdHJ1Y3QgZGV2aWNlICpkZXYsCj4gIMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBzdHJ1Y3QgaW9tbXVfZ3JvdXAgKmdyb3VwKQo+IAo+IEFu ZCwgaW4gaW9tbXVfYXV4X2F0dGFjaF9kZXZpY2UoKSwgd2UgcmVxdWlyZSwKPiAgwqAtIEBncm91 cCBvbmx5IGhhcyBhIHNpbmdsZSBkZXZpY2U7Cj4gIMKgLSBAZ3JvdXAgaGFzbid0IGJlZW4gYXR0 YWNoZWQgYnkgYW55IGRldmljZXM7Cj4gIMKgLSBTZXQgdGhlIEBkb21haW4gdG8gQGdyb3VwCj4g Cj4gSnVzdCBsaWtlIHdoYXQgd2UndmUgZG9uZSBpbiBpb21tdV9hdHRhY2hfZGV2aWNlKCkuCj4g Cj4gQW55IHRob3VnaHRzPwoKUmF0aGVyIHRoYW4gcGFzcyBhIGJhcmUgaW9tbXVfZ3JvdXAgd2l0 aCBpbXBsaWNpdCByZXN0cmljdGlvbnMsIGl0IG1pZ2h0IApiZSBuZWF0ZXIgdG8ganVzdCBwYXNz IGFuIG1kZXZfZGV2aWNlLCBzbyB0aGF0IHRoZSBJT01NVSBjb3JlIGNhbiBhbHNvIAp0YWtlIGNh cmUgb2YgYWxsb2NhdGluZyBhbmQgc2V0dGluZyB1cCB0aGUgZ3JvdXAuIFRoZW4gd2UgZmxhZyB0 aGUgZ3JvdXAgCmludGVybmFsbHkgYXMgYSBzcGVjaWFsICJtZGV2IGdyb3VwIiBzdWNoIHRoYXQg d2UgY2FuIHByZXZlbnQgY2FsbGVycyAKZnJvbSBzdWJzZXF1ZW50bHkgdHJ5aW5nIHRvIGFkZC9y ZW1vdmUgZGV2aWNlcyBvciBhdHRhY2gvZGV0YWNoIGl0cyAKZG9tYWluIGRpcmVjdGx5LiBUaGF0 IHNlZW1zIGxpa2UgaXQgd291bGQgbWFrZSBhIHByZXR0eSBzdHJhaWdodGZvcndhcmQgCmFuZCBy b2J1c3QgQVBJIGV4dGVuc2lvbiwgYXMgbG9uZyBhcyB0aGUgbWRldiBhcmd1bWVudCBoZXJlIGlz IG9wdGlvbmFsIApzbyB0aGF0IFNWQSBhbmQgb3RoZXIgYXV4IHVzZXJzIGRvbid0IGhhdmUgdG8g Y2FyZS4gT3RoZXIgdGhhbiB0aGUgCnNsaWdodGx5IGRpZmZlcmVudCBvcmRlcmluZyB3aGVyZSBj YWxsZXIgd291bGQgaGF2ZSB0byBhbGxvY2F0ZSB0aGUgbWRldiAKZmlyc3QsIHRoZW4gZmluaXNo IGl0J3MgUEFTSUQtYmFzZWQgY29uZmlndXJhdGlvbiBhZnRlcndhcmRzLCBJIGd1ZXNzIAppdCdz IG5vdCBmYXIgb2ZmIHdoYXQgSSB3YXMgdGhpbmtpbmcgeWVzdGVyZGF5IDopCgpSb2Jpbi4KX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KaW9tbXUgbWFpbGlu ZyBsaXN0CmlvbW11QGxpc3RzLmxpbnV4LWZvdW5kYXRpb24ub3JnCmh0dHBzOi8vbGlzdHMubGlu dXhmb3VuZGF0aW9uLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2lvbW11