From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C460BC43381 for ; Tue, 12 Mar 2019 17:43:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7F259214AE for ; Tue, 12 Mar 2019 17:43:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="VL9uFKtI" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728260AbfCLRnz (ORCPT ); Tue, 12 Mar 2019 13:43:55 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:45164 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728851AbfCLRPo (ORCPT ); Tue, 12 Mar 2019 13:15:44 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x2CHEJXU126442; Tue, 12 Mar 2019 17:14:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=subject : to : cc : references : from : message-id : date : mime-version : in-reply-to : content-type : content-transfer-encoding; s=corp-2018-07-02; bh=kqgYdMMnC1rX4Pe0Vj248e1LbMDASl3aQnMtGJJOrv0=; b=VL9uFKtISELnRNlRemBF35Al1+vx1gnRI6cmmIhzNXaeqLOVxSWSH40iZ/QWB+3fGKbB qXmpG9FGyuFdNL6rH7MoXft5ypl1nf7tLonT0cHysrQNtamR9DvxeQNbi0Yv0bXXw2qc 6DF9d9vw9L9L6KzhFnYK1cDed8h/VE4dP99Jx4SV4nXiDLlyZwqhUVJNfy0lGmRy2zwK XQ8899+vVWonZXrzYD2sX8Qh1QgmcsovVO0DwEozWIZkvFp1I4cXtYAmWBiUJEiLNNzI K4+kXJw8eK06YrR5/O9LdCuGOt4zsYgrA3BCYg3KJkoVVdV0/HZ6ulZyrocl2soNWeht Ew== Received: from aserv0021.oracle.com (aserv0021.oracle.com [141.146.126.233]) by userp2120.oracle.com with ESMTP id 2r464re2mb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Mar 2019 17:14:59 +0000 Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserv0021.oracle.com (8.14.4/8.14.4) with ESMTP id x2CHEweW031622 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Mar 2019 17:14:58 GMT Received: from abhmp0012.oracle.com (abhmp0012.oracle.com [141.146.116.18]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x2CHEvrJ021638; Tue, 12 Mar 2019 17:14:57 GMT Received: from [10.175.188.73] (/10.175.188.73) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 12 Mar 2019 10:14:57 -0700 Subject: Re: [PATCH RFC 00/39] x86/KVM: Xen HVM guest support To: Paolo Bonzini Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Ankur Arora , Boris Ostrovsky , =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , x86@kernel.org, Juergen Gross , Stefano Stabellini , xen-devel@lists.xenproject.org References: <20190220201609.28290-1-joao.m.martins@oracle.com> <35051310-c497-8ad5-4434-1b8426a317d2@redhat.com> <8b1f4912-4f92-69ae-ae01-d899d5640572@oracle.com> <3ee91f33-2973-c2db-386f-afbf138081b4@redhat.com> From: Joao Martins Message-ID: <59676804-786d-3df8-7752-8e45dec6d65b@oracle.com> Date: Tue, 12 Mar 2019 17:14:50 +0000 MIME-Version: 1.0 In-Reply-To: <3ee91f33-2973-c2db-386f-afbf138081b4@redhat.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9193 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=5 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1903120118 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2/22/19 4:59 PM, Paolo Bonzini wrote: > On 21/02/19 12:45, Joao Martins wrote: >> On 2/20/19 9:09 PM, Paolo Bonzini wrote: >>> On 20/02/19 21:15, Joao Martins wrote: >>>> 2. PV Driver support (patches 17 - 39) >>>> >>>> We start by redirecting hypercalls from the backend to routines >>>> which emulate the behaviour that PV backends expect i.e. grant >>>> table and interdomain events. Next, we add support for late >>>> initialization of xenbus, followed by implementing >>>> frontend/backend communication mechanisms (i.e. grant tables and >>>> interdomain event channels). Finally, introduce xen-shim.ko, >>>> which will setup a limited Xen environment. This uses the added >>>> functionality of Xen specific shared memory (grant tables) and >>>> notifications (event channels). >>> >>> I am a bit worried by the last patches, they seem really brittle and >>> prone to breakage. I don't know Xen well enough to understand if the >>> lack of support for GNTMAP_host_map is fixable, but if not, you have to >>> define a completely different hypercall. >>> >> I guess Ankur already answered this; so just to stack this on top of his comment. >> >> The xen_shim_domain() is only meant to handle the case where the backend >> has/can-have full access to guest memory [i.e. netback and blkback would work >> with similar assumptions as vhost?]. For the normal case, where a backend *in a >> guest* maps and unmaps other guest memory, this is not applicable and these >> changes don't affect that case. >> >> IOW, the PV backend here sits on the hypervisor, and the hypercalls aren't >> actual hypercalls but rather invoking shim_hypercall(). The call chain would go >> more or less like: >> >> >> gnttab_map_refs(map_ops, pages) >> HYPERVISOR_grant_table_op(GNTTABOP_map_grant_ref,...) >> shim_hypercall() >> shim_hcall_gntmap() >> >> Our reasoning was that given we are already in KVM, why mapping a page if the >> user (i.e. the kernel PV backend) is himself? The lack of GNTMAP_host_map is how >> the shim determines its user doesn't want to map the page. Also, there's another >> issue where PV backends always need a struct page to reference the device >> inflight data as Ankur pointed out. > > Ultimately it's up to the Xen people. It does make their API uglier, > especially the in/out change for the parameter. If you can at least > avoid that, it would alleviate my concerns quite a bit. In my view, we have two options overall: 1) Make it explicit, the changes the PV drivers we have to make in order to support xen_shim_domain(). This could mean e.g. a) add a callback argument to gnttab_map_refs() that is invoked for every page that gets looked up successfully, and inside this callback the PV driver may update it's tracking page. Here we no longer have this in/out parameter in gnttab_map_refs, and all shim_domain specific bits would be a little more abstracted from Xen PV backends. See netback example below the scissors mark. Or b) have sort of a translate_gref() and put_gref() API that Xen PV drivers use which make it even more explicit that there's no grant ops involved. The latter is more invasive. 2) The second option is to support guest grant mapping/unmapping [*] to allow hosting PV backends inside the guest. This would remove the Xen changes in this series completely. But it would require another guest being used as netback/blkback/xenstored, and less performance than 1) (though, in theory, it would be equivalent to what does Xen with grants/events). The only changes in Linux Xen code is adding xenstored domain support, but that is useful on its own outside the scope of this work. I think there's value on both; 1) is probably more familiar for KVM users perhaps (as it is similar to what vhost does?) while 2) equates to implementing Xen disagregation capabilities in KVM. Thoughts? Xen maintainers what's your take on this? Joao [*] Interdomain events would also have to change. ---------------- >8 ---------------- It isn't much cleaner, but PV drivers avoid/hide a bunch of xen_shim_domain() conditionals in the data path. It is more explicit while avoiding the in/out parameter change in gnttab_map_refs. diff --git a/drivers/net/xen-netback/common.h b/drivers/net/xen-netback/common.h index 936c0b3e0ba2..c6e47dcb7e10 100644 --- a/drivers/net/xen-netback/common.h +++ b/drivers/net/xen-netback/common.h @@ -158,6 +158,7 @@ struct xenvif_queue { /* Per-queue data for xenvif */ struct gnttab_copy tx_copy_ops[MAX_PENDING_REQS]; struct gnttab_map_grant_ref tx_map_ops[MAX_PENDING_REQS]; struct gnttab_unmap_grant_ref tx_unmap_ops[MAX_PENDING_REQS]; + struct gnttab_page_changed page_cb[MAX_PENDING_REQS]; /* passed to gnttab_[un]map_refs with pages under (un)mapping */ struct page *pages_to_map[MAX_PENDING_REQS]; struct page *pages_to_unmap[MAX_PENDING_REQS]; diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c index 80aae3a32c2a..56788d8cd813 100644 --- a/drivers/net/xen-netback/netback.c +++ b/drivers/net/xen-netback/netback.c @@ -324,15 +324,29 @@ struct xenvif_tx_cb { #define XENVIF_TX_CB(skb) ((struct xenvif_tx_cb *)(skb)->cb) +static inline void xenvif_tx_page_changed(phys_addr_t addr, void *opaque) +{ + struct page **page = opaque; + + *page = virt_to_page(addr); +} static inline void xenvif_tx_create_map_op(struct xenvif_queue *queue, u16 pending_idx, struct xen_netif_tx_request *txp, unsigned int extra_count, struct gnttab_map_grant_ref *mop) { - queue->pages_to_map[mop-queue->tx_map_ops] = queue->mmap_pages[pending_idx]; + u32 map_idx = mop - queue->tx_map_ops; + + queue->pages_to_map[map_idx] = queue->mmap_pages[pending_idx]; + queue->page_cb[map_idx].ctx = &queue->mmap_pages[pending_idx]; + queue->page_cb[map_idx].cb = xenvif_tx_page_changed; + gnttab_set_map_op(mop, idx_to_kaddr(queue, pending_idx), - GNTMAP_host_map | GNTMAP_readonly, + GNTTAB_host_map | GNTMAP_readonly, txp->gref, queue->vif->domid); memcpy(&queue->pending_tx_info[pending_idx].req, txp, @@ -1268,7 +1283,7 @@ static inline void xenvif_tx_dealloc_action(struct xenvif_queue *queue) queue->mmap_pages[pending_idx]; gnttab_set_unmap_op(gop, idx_to_kaddr(queue, pending_idx), - GNTMAP_host_map, + GNTTAB_host_map, queue->grant_tx_handle[pending_idx]); xenvif_grant_handle_reset(queue, pending_idx); ++gop; @@ -1322,7 +1337,7 @@ int xenvif_tx_action(struct xenvif_queue *queue, int budget) gnttab_batch_copy(queue->tx_copy_ops, nr_cops); if (nr_mops != 0) { ret = gnttab_map_refs(queue->tx_map_ops, - NULL, + NULL, queue->page_cb, queue->pages_to_map, nr_mops); BUG_ON(ret); @@ -1394,7 +1409,7 @@ void xenvif_idx_unmap(struct xenvif_queue *queue, u16 pending_idx) gnttab_set_unmap_op(&tx_unmap_op, idx_to_kaddr(queue, pending_idx), - GNTMAP_host_map, + GNTTAB_host_map, queue->grant_tx_handle[pending_idx]); xenvif_grant_handle_reset(queue, pending_idx); @@ -1622,7 +1637,7 @@ static int __init netback_init(void) { int rc = 0; - if (!xen_domain()) + if (!xen_domain() && !xen_shim_domain_get()) return -ENODEV; /* Allow as many queues as there are CPUs but max. 8 if user has not @@ -1663,6 +1678,7 @@ static void __exit netback_fini(void) debugfs_remove_recursive(xen_netback_dbg_root); #endif /* CONFIG_DEBUG_FS */ xenvif_xenbus_fini(); + xen_shim_domain_put(); } module_exit(netback_fini); diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c index 7ea6fb6a2e5d..b4c9d7ff531f 100644 --- a/drivers/xen/grant-table.c +++ b/drivers/xen/grant-table.c @@ -1031,6 +1031,7 @@ void gnttab_foreach_grant(struct page **pages, int gnttab_map_refs(struct gnttab_map_grant_ref *map_ops, struct gnttab_map_grant_ref *kmap_ops, + struct gnttab_page_changed *page_cb, struct page **pages, unsigned int count) { int i, ret; @@ -1045,6 +1046,12 @@ int gnttab_map_refs(struct gnttab_map_grant_ref *map_ops, { struct xen_page_foreign *foreign; + if (xen_shim_domain() && page_cb) { + page_cb[i].cb(map_ops[i].host_addr, + page_cb[i].ctx); + continue; + } + SetPageForeign(pages[i]); foreign = xen_page_foreign(pages[i]); foreign->domid = map_ops[i].dom; diff --git a/include/xen/grant_table.h b/include/xen/grant_table.h index 9bc5bc07d4d3..5e17fa08e779 100644 --- a/include/xen/grant_table.h +++ b/include/xen/grant_table.h @@ -55,6 +55,9 @@ /* NR_GRANT_FRAMES must be less than or equal to that configured in Xen */ #define NR_GRANT_FRAMES 4 +/* Selects host map only if on native Xen */ +#define GNTTAB_host_map (xen_shim_domain() ? 0 : GNTMAP_host_map) + struct gnttab_free_callback { struct gnttab_free_callback *next; void (*fn)(void *); @@ -78,6 +81,12 @@ struct gntab_unmap_queue_data unsigned int age; }; +struct gnttab_page_changed +{ + void (*cb)(phys_addr_t addr, void *opaque); + void *ctx; +}; + int gnttab_init(void); int gnttab_suspend(void); int gnttab_resume(void); @@ -221,6 +230,7 @@ void gnttab_pages_clear_private(int nr_pages, struct page **pages); int gnttab_map_refs(struct gnttab_map_grant_ref *map_ops, struct gnttab_map_grant_ref *kmap_ops, + struct gnttab_page_changed *cb, struct page **pages, unsigned int count); int gnttab_unmap_refs(struct gnttab_unmap_grant_ref *unmap_ops, struct gnttab_unmap_grant_ref *kunmap_ops, From mboxrd@z Thu Jan 1 00:00:00 1970 From: Joao Martins Subject: Re: [PATCH RFC 00/39] x86/KVM: Xen HVM guest support Date: Tue, 12 Mar 2019 17:14:50 +0000 Message-ID: <59676804-786d-3df8-7752-8e45dec6d65b@oracle.com> References: <20190220201609.28290-1-joao.m.martins@oracle.com> <35051310-c497-8ad5-4434-1b8426a317d2@redhat.com> <8b1f4912-4f92-69ae-ae01-d899d5640572@oracle.com> <3ee91f33-2973-c2db-386f-afbf138081b4@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Cc: Juergen Gross , Stefano Stabellini , kvm@vger.kernel.org, =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , x86@kernel.org, linux-kernel@vger.kernel.org, Ankur Arora , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , xen-devel@lists.xenproject.org, Boris Ostrovsky , Thomas Gleixner To: Paolo Bonzini Return-path: In-Reply-To: <3ee91f33-2973-c2db-386f-afbf138081b4@redhat.com> Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" List-Id: kvm.vger.kernel.org T24gMi8yMi8xOSA0OjU5IFBNLCBQYW9sbyBCb256aW5pIHdyb3RlOgo+IE9uIDIxLzAyLzE5IDEy OjQ1LCBKb2FvIE1hcnRpbnMgd3JvdGU6Cj4+IE9uIDIvMjAvMTkgOTowOSBQTSwgUGFvbG8gQm9u emluaSB3cm90ZToKPj4+IE9uIDIwLzAyLzE5IDIxOjE1LCBKb2FvIE1hcnRpbnMgd3JvdGU6Cj4+ Pj4gIDIuIFBWIERyaXZlciBzdXBwb3J0IChwYXRjaGVzIDE3IC0gMzkpCj4+Pj4KPj4+PiAgV2Ug c3RhcnQgYnkgcmVkaXJlY3RpbmcgaHlwZXJjYWxscyBmcm9tIHRoZSBiYWNrZW5kIHRvIHJvdXRp bmVzCj4+Pj4gIHdoaWNoIGVtdWxhdGUgdGhlIGJlaGF2aW91ciB0aGF0IFBWIGJhY2tlbmRzIGV4 cGVjdCBpLmUuIGdyYW50Cj4+Pj4gIHRhYmxlIGFuZCBpbnRlcmRvbWFpbiBldmVudHMuIE5leHQs IHdlIGFkZCBzdXBwb3J0IGZvciBsYXRlCj4+Pj4gIGluaXRpYWxpemF0aW9uIG9mIHhlbmJ1cywg Zm9sbG93ZWQgYnkgaW1wbGVtZW50aW5nCj4+Pj4gIGZyb250ZW5kL2JhY2tlbmQgY29tbXVuaWNh dGlvbiBtZWNoYW5pc21zIChpLmUuIGdyYW50IHRhYmxlcyBhbmQKPj4+PiAgaW50ZXJkb21haW4g ZXZlbnQgY2hhbm5lbHMpLiBGaW5hbGx5LCBpbnRyb2R1Y2UgeGVuLXNoaW0ua28sCj4+Pj4gIHdo aWNoIHdpbGwgc2V0dXAgYSBsaW1pdGVkIFhlbiBlbnZpcm9ubWVudC4gVGhpcyB1c2VzIHRoZSBh ZGRlZAo+Pj4+ICBmdW5jdGlvbmFsaXR5IG9mIFhlbiBzcGVjaWZpYyBzaGFyZWQgbWVtb3J5IChn cmFudCB0YWJsZXMpIGFuZAo+Pj4+ICBub3RpZmljYXRpb25zIChldmVudCBjaGFubmVscykuCj4+ Pgo+Pj4gSSBhbSBhIGJpdCB3b3JyaWVkIGJ5IHRoZSBsYXN0IHBhdGNoZXMsIHRoZXkgc2VlbSBy ZWFsbHkgYnJpdHRsZSBhbmQKPj4+IHByb25lIHRvIGJyZWFrYWdlLiAgSSBkb24ndCBrbm93IFhl biB3ZWxsIGVub3VnaCB0byB1bmRlcnN0YW5kIGlmIHRoZQo+Pj4gbGFjayBvZiBzdXBwb3J0IGZv ciBHTlRNQVBfaG9zdF9tYXAgaXMgZml4YWJsZSwgYnV0IGlmIG5vdCwgeW91IGhhdmUgdG8KPj4+ IGRlZmluZSBhIGNvbXBsZXRlbHkgZGlmZmVyZW50IGh5cGVyY2FsbC4KPj4+Cj4+IEkgZ3Vlc3Mg QW5rdXIgYWxyZWFkeSBhbnN3ZXJlZCB0aGlzOyBzbyBqdXN0IHRvIHN0YWNrIHRoaXMgb24gdG9w IG9mIGhpcyBjb21tZW50Lgo+Pgo+PiBUaGUgeGVuX3NoaW1fZG9tYWluKCkgaXMgb25seSBtZWFu dCB0byBoYW5kbGUgdGhlIGNhc2Ugd2hlcmUgdGhlIGJhY2tlbmQKPj4gaGFzL2Nhbi1oYXZlIGZ1 bGwgYWNjZXNzIHRvIGd1ZXN0IG1lbW9yeSBbaS5lLiBuZXRiYWNrIGFuZCBibGtiYWNrIHdvdWxk IHdvcmsKPj4gd2l0aCBzaW1pbGFyIGFzc3VtcHRpb25zIGFzIHZob3N0P10uIEZvciB0aGUgbm9y bWFsIGNhc2UsIHdoZXJlIGEgYmFja2VuZCAqaW4gYQo+PiBndWVzdCogbWFwcyBhbmQgdW5tYXBz IG90aGVyIGd1ZXN0IG1lbW9yeSwgdGhpcyBpcyBub3QgYXBwbGljYWJsZSBhbmQgdGhlc2UKPj4g Y2hhbmdlcyBkb24ndCBhZmZlY3QgdGhhdCBjYXNlLgo+Pgo+PiBJT1csIHRoZSBQViBiYWNrZW5k IGhlcmUgc2l0cyBvbiB0aGUgaHlwZXJ2aXNvciwgYW5kIHRoZSBoeXBlcmNhbGxzIGFyZW4ndAo+ PiBhY3R1YWwgaHlwZXJjYWxscyBidXQgcmF0aGVyIGludm9raW5nIHNoaW1faHlwZXJjYWxsKCku IFRoZSBjYWxsIGNoYWluIHdvdWxkIGdvCj4+IG1vcmUgb3IgbGVzcyBsaWtlOgo+Pgo+PiA8bmV0 YmFja3xibGtiYWNrfHNjc2liYWNrPgo+PiAgZ250dGFiX21hcF9yZWZzKG1hcF9vcHMsIHBhZ2Vz KQo+PiAgICBIWVBFUlZJU09SX2dyYW50X3RhYmxlX29wKEdOVFRBQk9QX21hcF9ncmFudF9yZWYs Li4uKQo+PiAgICAgIHNoaW1faHlwZXJjYWxsKCkKPj4gICAgICAgIHNoaW1faGNhbGxfZ250bWFw KCkKPj4KPj4gT3VyIHJlYXNvbmluZyB3YXMgdGhhdCBnaXZlbiB3ZSBhcmUgYWxyZWFkeSBpbiBL Vk0sIHdoeSBtYXBwaW5nIGEgcGFnZSBpZiB0aGUKPj4gdXNlciAoaS5lLiB0aGUga2VybmVsIFBW IGJhY2tlbmQpIGlzIGhpbXNlbGY/IFRoZSBsYWNrIG9mIEdOVE1BUF9ob3N0X21hcCBpcyBob3cK Pj4gdGhlIHNoaW0gZGV0ZXJtaW5lcyBpdHMgdXNlciBkb2Vzbid0IHdhbnQgdG8gbWFwIHRoZSBw YWdlLiBBbHNvLCB0aGVyZSdzIGFub3RoZXIKPj4gaXNzdWUgd2hlcmUgUFYgYmFja2VuZHMgYWx3 YXlzIG5lZWQgYSBzdHJ1Y3QgcGFnZSB0byByZWZlcmVuY2UgdGhlIGRldmljZQo+PiBpbmZsaWdo dCBkYXRhIGFzIEFua3VyIHBvaW50ZWQgb3V0Lgo+IAo+IFVsdGltYXRlbHkgaXQncyB1cCB0byB0 aGUgWGVuIHBlb3BsZS4gIEl0IGRvZXMgbWFrZSB0aGVpciBBUEkgdWdsaWVyLAo+IGVzcGVjaWFs bHkgdGhlIGluL291dCBjaGFuZ2UgZm9yIHRoZSBwYXJhbWV0ZXIuICBJZiB5b3UgY2FuIGF0IGxl YXN0Cj4gYXZvaWQgdGhhdCwgaXQgd291bGQgYWxsZXZpYXRlIG15IGNvbmNlcm5zIHF1aXRlIGEg Yml0LgoKSW4gbXkgdmlldywgd2UgaGF2ZSB0d28gb3B0aW9ucyBvdmVyYWxsOgoKMSkgTWFrZSBp dCBleHBsaWNpdCwgdGhlIGNoYW5nZXMgdGhlIFBWIGRyaXZlcnMgd2UgaGF2ZSB0byBtYWtlIGlu Cm9yZGVyIHRvIHN1cHBvcnQgeGVuX3NoaW1fZG9tYWluKCkuIFRoaXMgY291bGQgbWVhbiBlLmcu IGEpIGFkZCBhIGNhbGxiYWNrCmFyZ3VtZW50IHRvIGdudHRhYl9tYXBfcmVmcygpIHRoYXQgaXMg aW52b2tlZCBmb3IgZXZlcnkgcGFnZSB0aGF0IGdldHMgbG9va2VkIHVwCnN1Y2Nlc3NmdWxseSwg YW5kIGluc2lkZSB0aGlzIGNhbGxiYWNrIHRoZSBQViBkcml2ZXIgbWF5IHVwZGF0ZSBpdCdzIHRy YWNraW5nCnBhZ2UuIEhlcmUgd2Ugbm8gbG9uZ2VyIGhhdmUgdGhpcyBpbi9vdXQgcGFyYW1ldGVy IGluIGdudHRhYl9tYXBfcmVmcywgYW5kIGFsbApzaGltX2RvbWFpbiBzcGVjaWZpYyBiaXRzIHdv dWxkIGJlIGEgbGl0dGxlIG1vcmUgYWJzdHJhY3RlZCBmcm9tIFhlbiBQVgpiYWNrZW5kcy4gU2Vl IG5ldGJhY2sgZXhhbXBsZSBiZWxvdyB0aGUgc2Npc3NvcnMgbWFyay4gT3IgYikgaGF2ZSBzb3J0 IG9mIGEKdHJhbnNsYXRlX2dyZWYoKSBhbmQgcHV0X2dyZWYoKSBBUEkgdGhhdCBYZW4gUFYgZHJp dmVycyB1c2Ugd2hpY2ggbWFrZSBpdCBldmVuCm1vcmUgZXhwbGljaXQgdGhhdCB0aGVyZSdzIG5v IGdyYW50IG9wcyBpbnZvbHZlZC4gVGhlIGxhdHRlciBpcyBtb3JlIGludmFzaXZlLgoKMikgVGhl IHNlY29uZCBvcHRpb24gaXMgdG8gc3VwcG9ydCBndWVzdCBncmFudCBtYXBwaW5nL3VubWFwcGlu ZyBbKl0gdG8gYWxsb3cKaG9zdGluZyBQViBiYWNrZW5kcyBpbnNpZGUgdGhlIGd1ZXN0LiBUaGlz IHdvdWxkIHJlbW92ZSB0aGUgWGVuIGNoYW5nZXMgaW4gdGhpcwpzZXJpZXMgY29tcGxldGVseS4g QnV0IGl0IHdvdWxkIHJlcXVpcmUgYW5vdGhlciBndWVzdCBiZWluZyB1c2VkCmFzIG5ldGJhY2sv YmxrYmFjay94ZW5zdG9yZWQsIGFuZCBsZXNzIHBlcmZvcm1hbmNlIHRoYW4gMSkgKHRob3VnaCwg aW4gdGhlb3J5LAppdCB3b3VsZCBiZSBlcXVpdmFsZW50IHRvIHdoYXQgZG9lcyBYZW4gd2l0aCBn cmFudHMvZXZlbnRzKS4gVGhlIG9ubHkgY2hhbmdlcyBpbgpMaW51eCBYZW4gY29kZSBpcyBhZGRp bmcgeGVuc3RvcmVkIGRvbWFpbiBzdXBwb3J0LCBidXQgdGhhdCBpcyB1c2VmdWwgb24gaXRzIG93 bgpvdXRzaWRlIHRoZSBzY29wZSBvZiB0aGlzIHdvcmsuCgpJIHRoaW5rIHRoZXJlJ3MgdmFsdWUg b24gYm90aDsgMSkgaXMgcHJvYmFibHkgbW9yZSBmYW1pbGlhciBmb3IgS1ZNIHVzZXJzCnBlcmhh cHMgKGFzIGl0IGlzIHNpbWlsYXIgdG8gd2hhdCB2aG9zdCBkb2VzPykgd2hpbGUgIDIpIGVxdWF0 ZXMgdG8gaW1wbGVtZW50aW5nClhlbiBkaXNhZ3JlZ2F0aW9uIGNhcGFiaWxpdGllcyBpbiBLVk0u CgpUaG91Z2h0cz8gWGVuIG1haW50YWluZXJzIHdoYXQncyB5b3VyIHRha2Ugb24gdGhpcz8KCglK b2FvCgpbKl0gSW50ZXJkb21haW4gZXZlbnRzIHdvdWxkIGFsc28gaGF2ZSB0byBjaGFuZ2UuCgot LS0tLS0tLS0tLS0tLS0tID44IC0tLS0tLS0tLS0tLS0tLS0KCkl0IGlzbid0IG11Y2ggY2xlYW5l ciwgYnV0IFBWIGRyaXZlcnMgYXZvaWQvaGlkZSBhIGJ1bmNoIG9mIHhlbl9zaGltX2RvbWFpbigp CmNvbmRpdGlvbmFscyBpbiB0aGUgZGF0YSBwYXRoLiBJdCBpcyBtb3JlIGV4cGxpY2l0IHdoaWxl IGF2b2lkaW5nIHRoZSBpbi9vdXQKcGFyYW1ldGVyIGNoYW5nZSBpbiBnbnR0YWJfbWFwX3JlZnMu CgpkaWZmIC0tZ2l0IGEvZHJpdmVycy9uZXQveGVuLW5ldGJhY2svY29tbW9uLmggYi9kcml2ZXJz L25ldC94ZW4tbmV0YmFjay9jb21tb24uaAppbmRleCA5MzZjMGIzZTBiYTIuLmM2ZTQ3ZGNiN2Ux MCAxMDA2NDQKLS0tIGEvZHJpdmVycy9uZXQveGVuLW5ldGJhY2svY29tbW9uLmgKKysrIGIvZHJp dmVycy9uZXQveGVuLW5ldGJhY2svY29tbW9uLmgKQEAgLTE1OCw2ICsxNTgsNyBAQCBzdHJ1Y3Qg eGVudmlmX3F1ZXVlIHsgLyogUGVyLXF1ZXVlIGRhdGEgZm9yIHhlbnZpZiAqLwogCXN0cnVjdCBn bnR0YWJfY29weSB0eF9jb3B5X29wc1tNQVhfUEVORElOR19SRVFTXTsKIAlzdHJ1Y3QgZ250dGFi X21hcF9ncmFudF9yZWYgdHhfbWFwX29wc1tNQVhfUEVORElOR19SRVFTXTsKIAlzdHJ1Y3QgZ250 dGFiX3VubWFwX2dyYW50X3JlZiB0eF91bm1hcF9vcHNbTUFYX1BFTkRJTkdfUkVRU107CisJc3Ry dWN0IGdudHRhYl9wYWdlX2NoYW5nZWQgcGFnZV9jYltNQVhfUEVORElOR19SRVFTXTsKIAkvKiBw YXNzZWQgdG8gZ250dGFiX1t1bl1tYXBfcmVmcyB3aXRoIHBhZ2VzIHVuZGVyICh1biltYXBwaW5n ICovCiAJc3RydWN0IHBhZ2UgKnBhZ2VzX3RvX21hcFtNQVhfUEVORElOR19SRVFTXTsKIAlzdHJ1 Y3QgcGFnZSAqcGFnZXNfdG9fdW5tYXBbTUFYX1BFTkRJTkdfUkVRU107CmRpZmYgLS1naXQgYS9k cml2ZXJzL25ldC94ZW4tbmV0YmFjay9uZXRiYWNrLmMgYi9kcml2ZXJzL25ldC94ZW4tbmV0YmFj ay9uZXRiYWNrLmMKaW5kZXggODBhYWUzYTMyYzJhLi41Njc4OGQ4Y2Q4MTMgMTAwNjQ0Ci0tLSBh L2RyaXZlcnMvbmV0L3hlbi1uZXRiYWNrL25ldGJhY2suYworKysgYi9kcml2ZXJzL25ldC94ZW4t bmV0YmFjay9uZXRiYWNrLmMKQEAgLTMyNCwxNSArMzI0LDI5IEBAIHN0cnVjdCB4ZW52aWZfdHhf Y2IgewoKICNkZWZpbmUgWEVOVklGX1RYX0NCKHNrYikgKChzdHJ1Y3QgeGVudmlmX3R4X2NiICop KHNrYiktPmNiKQoKK3N0YXRpYyBpbmxpbmUgdm9pZCB4ZW52aWZfdHhfcGFnZV9jaGFuZ2VkKHBo eXNfYWRkcl90IGFkZHIsIHZvaWQgKm9wYXF1ZSkKK3sKKwlzdHJ1Y3QgcGFnZSAqKnBhZ2UgPSBv cGFxdWU7CisKKwkqcGFnZSA9IHZpcnRfdG9fcGFnZShhZGRyKTsKK30KIHN0YXRpYyBpbmxpbmUg dm9pZCB4ZW52aWZfdHhfY3JlYXRlX21hcF9vcChzdHJ1Y3QgeGVudmlmX3F1ZXVlICpxdWV1ZSwK IAkJCQkJICAgdTE2IHBlbmRpbmdfaWR4LAogCQkJCQkgICBzdHJ1Y3QgeGVuX25ldGlmX3R4X3Jl cXVlc3QgKnR4cCwKIAkJCQkJICAgdW5zaWduZWQgaW50IGV4dHJhX2NvdW50LAogCQkJCQkgICBz dHJ1Y3QgZ250dGFiX21hcF9ncmFudF9yZWYgKm1vcCkKIHsKLQlxdWV1ZS0+cGFnZXNfdG9fbWFw W21vcC1xdWV1ZS0+dHhfbWFwX29wc10gPSBxdWV1ZS0+bW1hcF9wYWdlc1twZW5kaW5nX2lkeF07 CisJdTMyIG1hcF9pZHggPSBtb3AgLSBxdWV1ZS0+dHhfbWFwX29wczsKKworCXF1ZXVlLT5wYWdl c190b19tYXBbbWFwX2lkeF0gPSBxdWV1ZS0+bW1hcF9wYWdlc1twZW5kaW5nX2lkeF07CisJcXVl dWUtPnBhZ2VfY2JbbWFwX2lkeF0uY3R4ID0gJnF1ZXVlLT5tbWFwX3BhZ2VzW3BlbmRpbmdfaWR4 XTsKKwlxdWV1ZS0+cGFnZV9jYlttYXBfaWR4XS5jYiA9IHhlbnZpZl90eF9wYWdlX2NoYW5nZWQ7 CisKIAlnbnR0YWJfc2V0X21hcF9vcChtb3AsIGlkeF90b19rYWRkcihxdWV1ZSwgcGVuZGluZ19p ZHgpLAotCQkJICBHTlRNQVBfaG9zdF9tYXAgfCBHTlRNQVBfcmVhZG9ubHksCisJCQkgIEdOVFRB Ql9ob3N0X21hcCB8IEdOVE1BUF9yZWFkb25seSwKIAkJCSAgdHhwLT5ncmVmLCBxdWV1ZS0+dmlm LT5kb21pZCk7CgogCW1lbWNweSgmcXVldWUtPnBlbmRpbmdfdHhfaW5mb1twZW5kaW5nX2lkeF0u cmVxLCB0eHAsCkBAIC0xMjY4LDcgKzEyODMsNyBAQCBzdGF0aWMgaW5saW5lIHZvaWQgeGVudmlm X3R4X2RlYWxsb2NfYWN0aW9uKHN0cnVjdAp4ZW52aWZfcXVldWUgKnF1ZXVlKQogCQkJCXF1ZXVl LT5tbWFwX3BhZ2VzW3BlbmRpbmdfaWR4XTsKIAkJCWdudHRhYl9zZXRfdW5tYXBfb3AoZ29wLAog CQkJCQkgICAgaWR4X3RvX2thZGRyKHF1ZXVlLCBwZW5kaW5nX2lkeCksCi0JCQkJCSAgICBHTlRN QVBfaG9zdF9tYXAsCisJCQkJCSAgICBHTlRUQUJfaG9zdF9tYXAsCiAJCQkJCSAgICBxdWV1ZS0+ Z3JhbnRfdHhfaGFuZGxlW3BlbmRpbmdfaWR4XSk7CiAJCQl4ZW52aWZfZ3JhbnRfaGFuZGxlX3Jl c2V0KHF1ZXVlLCBwZW5kaW5nX2lkeCk7CiAJCQkrK2dvcDsKQEAgLTEzMjIsNyArMTMzNyw3IEBA IGludCB4ZW52aWZfdHhfYWN0aW9uKHN0cnVjdCB4ZW52aWZfcXVldWUgKnF1ZXVlLCBpbnQgYnVk Z2V0KQogCWdudHRhYl9iYXRjaF9jb3B5KHF1ZXVlLT50eF9jb3B5X29wcywgbnJfY29wcyk7CiAJ aWYgKG5yX21vcHMgIT0gMCkgewogCQlyZXQgPSBnbnR0YWJfbWFwX3JlZnMocXVldWUtPnR4X21h cF9vcHMsCi0JCQkJICAgICAgTlVMTCwKKwkJCQkgICAgICBOVUxMLCBxdWV1ZS0+cGFnZV9jYiwK IAkJCQkgICAgICBxdWV1ZS0+cGFnZXNfdG9fbWFwLAogCQkJCSAgICAgIG5yX21vcHMpOwogCQlC VUdfT04ocmV0KTsKQEAgLTEzOTQsNyArMTQwOSw3IEBAIHZvaWQgeGVudmlmX2lkeF91bm1hcChz dHJ1Y3QgeGVudmlmX3F1ZXVlICpxdWV1ZSwgdTE2CnBlbmRpbmdfaWR4KQoKIAlnbnR0YWJfc2V0 X3VubWFwX29wKCZ0eF91bm1hcF9vcCwKIAkJCSAgICBpZHhfdG9fa2FkZHIocXVldWUsIHBlbmRp bmdfaWR4KSwKLQkJCSAgICBHTlRNQVBfaG9zdF9tYXAsCisJCQkgICAgR05UVEFCX2hvc3RfbWFw LAogCQkJICAgIHF1ZXVlLT5ncmFudF90eF9oYW5kbGVbcGVuZGluZ19pZHhdKTsKIAl4ZW52aWZf Z3JhbnRfaGFuZGxlX3Jlc2V0KHF1ZXVlLCBwZW5kaW5nX2lkeCk7CgpAQCAtMTYyMiw3ICsxNjM3 LDcgQEAgc3RhdGljIGludCBfX2luaXQgbmV0YmFja19pbml0KHZvaWQpCiB7CiAJaW50IHJjID0g MDsKCi0JaWYgKCF4ZW5fZG9tYWluKCkpCisJaWYgKCF4ZW5fZG9tYWluKCkgJiYgIXhlbl9zaGlt X2RvbWFpbl9nZXQoKSkKIAkJcmV0dXJuIC1FTk9ERVY7CgogCS8qIEFsbG93IGFzIG1hbnkgcXVl dWVzIGFzIHRoZXJlIGFyZSBDUFVzIGJ1dCBtYXguIDggaWYgdXNlciBoYXMgbm90CkBAIC0xNjYz LDYgKzE2NzgsNyBAQCBzdGF0aWMgdm9pZCBfX2V4aXQgbmV0YmFja19maW5pKHZvaWQpCiAJZGVi dWdmc19yZW1vdmVfcmVjdXJzaXZlKHhlbl9uZXRiYWNrX2RiZ19yb290KTsKICNlbmRpZiAvKiBD T05GSUdfREVCVUdfRlMgKi8KIAl4ZW52aWZfeGVuYnVzX2ZpbmkoKTsKKwl4ZW5fc2hpbV9kb21h aW5fcHV0KCk7CiB9CiBtb2R1bGVfZXhpdChuZXRiYWNrX2ZpbmkpOwoKZGlmZiAtLWdpdCBhL2Ry aXZlcnMveGVuL2dyYW50LXRhYmxlLmMgYi9kcml2ZXJzL3hlbi9ncmFudC10YWJsZS5jCmluZGV4 IDdlYTZmYjZhMmU1ZC4uYjRjOWQ3ZmY1MzFmIDEwMDY0NAotLS0gYS9kcml2ZXJzL3hlbi9ncmFu dC10YWJsZS5jCisrKyBiL2RyaXZlcnMveGVuL2dyYW50LXRhYmxlLmMKQEAgLTEwMzEsNiArMTAz MSw3IEBAIHZvaWQgZ250dGFiX2ZvcmVhY2hfZ3JhbnQoc3RydWN0IHBhZ2UgKipwYWdlcywKCiBp bnQgZ250dGFiX21hcF9yZWZzKHN0cnVjdCBnbnR0YWJfbWFwX2dyYW50X3JlZiAqbWFwX29wcywK IAkJICAgIHN0cnVjdCBnbnR0YWJfbWFwX2dyYW50X3JlZiAqa21hcF9vcHMsCisJCSAgICBzdHJ1 Y3QgZ250dGFiX3BhZ2VfY2hhbmdlZCAqcGFnZV9jYiwKIAkJICAgIHN0cnVjdCBwYWdlICoqcGFn ZXMsIHVuc2lnbmVkIGludCBjb3VudCkKIHsKIAlpbnQgaSwgcmV0OwpAQCAtMTA0NSw2ICsxMDQ2 LDEyIEBAIGludCBnbnR0YWJfbWFwX3JlZnMoc3RydWN0IGdudHRhYl9tYXBfZ3JhbnRfcmVmICpt YXBfb3BzLAogCQl7CiAJCQlzdHJ1Y3QgeGVuX3BhZ2VfZm9yZWlnbiAqZm9yZWlnbjsKCisJCQlp ZiAoeGVuX3NoaW1fZG9tYWluKCkgJiYgcGFnZV9jYikgeworCQkJCXBhZ2VfY2JbaV0uY2IobWFw X29wc1tpXS5ob3N0X2FkZHIsCisJCQkJCSAgICAgIHBhZ2VfY2JbaV0uY3R4KTsKKwkJCQljb250 aW51ZTsKKwkJCX0KKwogCQkJU2V0UGFnZUZvcmVpZ24ocGFnZXNbaV0pOwogCQkJZm9yZWlnbiA9 IHhlbl9wYWdlX2ZvcmVpZ24ocGFnZXNbaV0pOwogCQkJZm9yZWlnbi0+ZG9taWQgPSBtYXBfb3Bz W2ldLmRvbTsKZGlmZiAtLWdpdCBhL2luY2x1ZGUveGVuL2dyYW50X3RhYmxlLmggYi9pbmNsdWRl L3hlbi9ncmFudF90YWJsZS5oCmluZGV4IDliYzViYzA3ZDRkMy4uNWUxN2ZhMDhlNzc5IDEwMDY0 NAotLS0gYS9pbmNsdWRlL3hlbi9ncmFudF90YWJsZS5oCisrKyBiL2luY2x1ZGUveGVuL2dyYW50 X3RhYmxlLmgKQEAgLTU1LDYgKzU1LDkgQEAKIC8qIE5SX0dSQU5UX0ZSQU1FUyBtdXN0IGJlIGxl c3MgdGhhbiBvciBlcXVhbCB0byB0aGF0IGNvbmZpZ3VyZWQgaW4gWGVuICovCiAjZGVmaW5lIE5S X0dSQU5UX0ZSQU1FUyA0CgorLyogU2VsZWN0cyBob3N0IG1hcCBvbmx5IGlmIG9uIG5hdGl2ZSBY ZW4gKi8KKyNkZWZpbmUgR05UVEFCX2hvc3RfbWFwICh4ZW5fc2hpbV9kb21haW4oKSA/IDAgOiBH TlRNQVBfaG9zdF9tYXApCisKIHN0cnVjdCBnbnR0YWJfZnJlZV9jYWxsYmFjayB7CiAJc3RydWN0 IGdudHRhYl9mcmVlX2NhbGxiYWNrICpuZXh0OwogCXZvaWQgKCpmbikodm9pZCAqKTsKQEAgLTc4 LDYgKzgxLDEyIEBAIHN0cnVjdCBnbnRhYl91bm1hcF9xdWV1ZV9kYXRhCiAJdW5zaWduZWQgaW50 IGFnZTsKIH07Cgorc3RydWN0IGdudHRhYl9wYWdlX2NoYW5nZWQKK3sKKwl2b2lkICgqY2IpKHBo eXNfYWRkcl90IGFkZHIsIHZvaWQgKm9wYXF1ZSk7CisJdm9pZCAqY3R4OworfTsKKwogaW50IGdu dHRhYl9pbml0KHZvaWQpOwogaW50IGdudHRhYl9zdXNwZW5kKHZvaWQpOwogaW50IGdudHRhYl9y ZXN1bWUodm9pZCk7CkBAIC0yMjEsNiArMjMwLDcgQEAgdm9pZCBnbnR0YWJfcGFnZXNfY2xlYXJf cHJpdmF0ZShpbnQgbnJfcGFnZXMsIHN0cnVjdCBwYWdlCioqcGFnZXMpOwoKIGludCBnbnR0YWJf bWFwX3JlZnMoc3RydWN0IGdudHRhYl9tYXBfZ3JhbnRfcmVmICptYXBfb3BzLAogCQkgICAgc3Ry dWN0IGdudHRhYl9tYXBfZ3JhbnRfcmVmICprbWFwX29wcywKKwkJICAgIHN0cnVjdCBnbnR0YWJf cGFnZV9jaGFuZ2VkICpjYiwKIAkJICAgIHN0cnVjdCBwYWdlICoqcGFnZXMsIHVuc2lnbmVkIGlu dCBjb3VudCk7CiBpbnQgZ250dGFiX3VubWFwX3JlZnMoc3RydWN0IGdudHRhYl91bm1hcF9ncmFu dF9yZWYgKnVubWFwX29wcywKIAkJICAgICAgc3RydWN0IGdudHRhYl91bm1hcF9ncmFudF9yZWYg Kmt1bm1hcF9vcHMsCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fXwpYZW4tZGV2ZWwgbWFpbGluZyBsaXN0Clhlbi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9y ZwpodHRwczovL2xpc3RzLnhlbnByb2plY3Qub3JnL21haWxtYW4vbGlzdGluZm8veGVuLWRldmVs