From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA64CC433EF for ; Tue, 12 Apr 2022 02:24:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245741AbiDLC1K (ORCPT ); Mon, 11 Apr 2022 22:27:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60006 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238083AbiDLC1H (ORCPT ); Mon, 11 Apr 2022 22:27:07 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 4934DBC83 for ; Mon, 11 Apr 2022 19:24:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1649730290; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=U91PiURNFV2fKxTIEP7GO5tX2GZNd7Bfp7ekztFiEcU=; b=HiOB0yGOniOKHBcsXmsQmYknMIF9hga060rCgwWOiLCCvVot8iz4r5W8LuHMkN3xB0sLmR feqsjc0EaQWsHsv6y4nJ56acaIJz7IPBy2nOey1H9xaQUahVBtKztYBDsIgx9cfmRqTOt7 hqjo+n39u/MYWFvw1re3XW1Ophm+y68= Received: from mail-lj1-f197.google.com (mail-lj1-f197.google.com [209.85.208.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-272-McGLCUiePOmc6y9X3Qh3eQ-1; Mon, 11 Apr 2022 22:24:48 -0400 X-MC-Unique: McGLCUiePOmc6y9X3Qh3eQ-1 Received: by mail-lj1-f197.google.com with SMTP id p9-20020a2ea4c9000000b0024b62f1ddebso1582194ljm.23 for ; Mon, 11 Apr 2022 19:24:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=U91PiURNFV2fKxTIEP7GO5tX2GZNd7Bfp7ekztFiEcU=; b=bSJxXqaraeWhw4R6cAz4gcFKCJFrNxr9Cml72S4Xyr/NCPahsZGh6EJD/PeJldXR68 hF7fiDR6KveKwEHkwAjlsnhX2RKozTk7Cxqxy9wwsMM74dzBSBuCN/tMdyywhVMcyAsz /bTryyYLvT/3W6YJ/N1FGtSg8mT3MeoBvk8lUcSEaXq2KWVGWz4oD5cmaabW+KKr1guC CSDd2lMSOrmIRl75VgUj0prE18gSFccFThQHGf8LlUu3Qu6McaCtS044xbsP2j2nJ4Yv qeRhdhXI1o9e/HTlP1WBlZsfWY8XlXG2ZrHnlDV2/4t1hAxhcvQ1GzIYN6UvdoVXvZuc 3csA== X-Gm-Message-State: AOAM532FpaUVX9ogHAWAZmomuwCSZ5lql1x6yoAhTysqitQnGf3Wv921 swTquAGDqd3MEstIKd6hSuWRRoUUIKWQ+7/xpvnnhKu9b/ijygPGf4NivBFqiXasQlB9S+JOa0r 3AjHqBlYUz5sizX1N4GdJwIuWGPJCv0hyOUNJjfTp X-Received: by 2002:a05:6512:b81:b0:448:b342:513c with SMTP id b1-20020a0565120b8100b00448b342513cmr23533665lfv.257.1649730287279; Mon, 11 Apr 2022 19:24:47 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyriilgsCa0fnjLvRc9+m4ztpQqX137F7IBRBNtJ2Ku+BVjGMKTZvwHHTDT7U+xGoryLTsKpDLdOUKsJnHhYd8= X-Received: by 2002:a05:6512:b81:b0:448:b342:513c with SMTP id b1-20020a0565120b8100b00448b342513cmr23533653lfv.257.1649730287091; Mon, 11 Apr 2022 19:24:47 -0700 (PDT) MIME-Version: 1.0 References: <20220406083538.16274-1-jasowang@redhat.com> <20220406083538.16274-5-jasowang@redhat.com> <20220406075952-mutt-send-email-mst@kernel.org> <87wng2e527.fsf@redhat.com> <20220408150307.24b6b99f.pasic@linux.ibm.com> <20220410034556-mutt-send-email-mst@kernel.org> <877d7vbspu.fsf@redhat.com> <20220412020145.32e26e5a.pasic@linux.ibm.com> In-Reply-To: <20220412020145.32e26e5a.pasic@linux.ibm.com> From: Jason Wang Date: Tue, 12 Apr 2022 10:24:35 +0800 Message-ID: Subject: Re: [PATCH V2 4/5] virtio-pci: implement synchronize_vqs() To: Halil Pasic Cc: Cornelia Huck , "Michael S. Tsirkin" , "Paul E. McKenney" , Peter Zijlstra , Marc Zyngier , linux-kernel , virtualization , Thomas Gleixner Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Apr 12, 2022 at 8:02 AM Halil Pasic wrote: > > On Mon, 11 Apr 2022 16:27:41 +0200 > Cornelia Huck wrote: > > > On Mon, Apr 11 2022, Jason Wang wrote: > > > > > On Sun, Apr 10, 2022 at 3:51 PM Michael S. Tsirkin w= rote: > > >> > > >> On Fri, Apr 08, 2022 at 03:03:07PM +0200, Halil Pasic wrote: > > >> > On Wed, 06 Apr 2022 15:04:32 +0200 > > >> > Cornelia Huck wrote: > > >> > > > >> > > On Wed, Apr 06 2022, "Michael S. Tsirkin" wrote= : > > >> > > > > >> > > > On Wed, Apr 06, 2022 at 04:35:37PM +0800, Jason Wang wrote: > > >> > > >> This patch implements PCI version of synchronize_vqs(). > > >> > > >> > > >> > > >> Cc: Thomas Gleixner > > >> > > >> Cc: Peter Zijlstra > > >> > > >> Cc: "Paul E. McKenney" > > >> > > >> Cc: Marc Zyngier > > >> > > >> Signed-off-by: Jason Wang > > >> > > > > > >> > > > Please add implementations at least for ccw and mmio. > > >> > > > > >> > > I'm not sure what (if anything) can/should be done for ccw... > > >> > > > >> > If nothing needs to be done I would like to have at least a commen= t in > > >> > the code that explains why. So that somebody who reads the code > > >> > doesn't wonder: why is virtio-ccw not implementing that callback. > > >> > > >> Right. > > >> > > >> I am currently thinking instead of making this optional in the > > >> core we should make it mandatory, and have transports which do not > > >> need to sync have an empty stub with documentation explaining why. > > > > Yes, that makes sense to me. If we can explain why we don't need to do > > anything, we should keep that explanation easily accessible. > > > > >> > > >> Also, do we want to document this sync is explicitly for irq enable/= disable? > > >> synchronize_irq_enable_disable? > > > > > > I would not since the transport is not guaranteed to use an interrupt > > > for callbacks. > > > > > >> > > >> > > >> > > > > >> > > > > > >> > > >> --- > > >> > > >> drivers/virtio/virtio_pci_common.c | 14 ++++++++++++++ > > >> > > >> drivers/virtio/virtio_pci_common.h | 2 ++ > > >> > > >> drivers/virtio/virtio_pci_legacy.c | 1 + > > >> > > >> drivers/virtio/virtio_pci_modern.c | 2 ++ > > >> > > >> 4 files changed, 19 insertions(+) > > >> > > >> > > >> > > >> diff --git a/drivers/virtio/virtio_pci_common.c b/drivers/vir= tio/virtio_pci_common.c > > >> > > >> index d724f676608b..b78c8bc93a97 100644 > > >> > > >> --- a/drivers/virtio/virtio_pci_common.c > > >> > > >> +++ b/drivers/virtio/virtio_pci_common.c > > >> > > >> @@ -37,6 +37,20 @@ void vp_synchronize_vectors(struct virtio_= device *vdev) > > >> > > >> synchronize_irq(pci_irq_vector(vp_dev->pci_d= ev, i)); > > >> > > >> } > > >> > > >> > > >> > > >> +void vp_synchronize_vqs(struct virtio_device *vdev) > > >> > > >> +{ > > >> > > >> + struct virtio_pci_device *vp_dev =3D to_vp_device(vd= ev); > > >> > > >> + int i; > > >> > > >> + > > >> > > >> + if (vp_dev->intx_enabled) { > > >> > > >> + synchronize_irq(vp_dev->pci_dev->irq); > > >> > > >> + return; > > >> > > >> + } > > >> > > >> + > > >> > > >> + for (i =3D 0; i < vp_dev->msix_vectors; ++i) > > >> > > >> + synchronize_irq(pci_irq_vector(vp_dev->pci_d= ev, i)); > > >> > > >> +} > > >> > > >> + > > >> > > > > >> > > ...given that this seems to synchronize threaded interrupt handl= ers? > > >> > > Halil, do you think ccw needs to do anything? (AFAICS, we only h= ave one > > >> > > 'irq' for channel devices anyway, and the handler just calls the > > >> > > relevant callbacks directly.) > > >> > > > >> > Sorry I don't understand enough yet. A more verbose documentation = on > > >> > "virtio_synchronize_vqs - synchronize with virtqueue callbacks" wo= uld > > >> > surely benefit me. It may be more than enough for a back-belt but = it > > >> > ain't enough for me to tell what is the callback supposed to accom= plish. > > > > +1 for more explanations. > > > > >> > > > >> > I will have to study this discussion and the code more thoroughly. > > >> > Tentatively I side with Jason and Michael in a sense, that I don't > > >> > believe virtio-ccw is safe against rough interrupts. > > > > > > That's my feeling as well. > > > > I'd say ccw is safe against "notification interrupts before indicators > > have been registered". > > I believe Jason's scope is broader than that. Let me try to explain. A > quote form the standard: > """ > 3.1.1 Driver Requirements: Device Initialization > The driver MUST follow this sequence to initialize a device: > 1. Reset the device. > 2. Set the ACKNOWLEDGE status bit: the guest OS has noticed the devic= e. > 3. Set the DRIVER status bit: the guest OS knows how to drive the dev= ice. > 4. Read device feature bits, and write the subset of feature bits und= erstood by the OS and driver to the device. During this step the driver MAY= read (but MUST NOT write) the device-specific configuration fields to chec= k that it can support the device before accepting it. > 5. Set the FEATURES_OK status bit. The driver MUST NOT accept new fea= ture bits after this step. > 6. Re-read device status to ensure the FEATURES_OK bit is still set: = otherwise, the device does not support our subset of features and the devic= e is unusable. > 7. Perform device-specific setup, including discovery of virtqueues f= or the device, optional per-bus setup, reading and possibly writing the dev= ice=E2=80=99s virtio configuration space, and population of virtqueues. > 8. Set the DRIVER_OK status bit. At this point the device is =E2=80= =9Clive=E2=80=9D. > """ > > That means stuff may happen between "discovery of virtqueues" and "DRIVER= _OK". So it > is not sufficient to be "safe against notifications before indicators > have been registered", but we want to be also safe between "indicators ha= ve > been registered" and "DRIVER_OK status has been set". > > @Jason: can you confirm? Exactly. > > Regarding the question "are we safe against notifications before > indicators have been registered" I think we really need to think about > something like Secure Execution. We don't have, and we are unlikely > to have in hardware virtio-ccw implementations, and for a malicious hyper= visor > that has full access to the guest memory hardening makes no sense. Does s390 have something like memory encryption? (I guess yes). In the case of x86 VM encryption, the I/O buffers were now done via software IOTLB, that's why hardening of the virtio driver is needed to prevent the hypervisor to poke the swiotlb etc. Thanks > > But if we assume that an attacker can inject adapter interrupts for an ar= bitrary > ISC, and can poke any shared memory (notifier bits are shared)... Things = might > become critical already when register_adapter_interrupt() does it's magic= . > > > > For the reverse case, maybe we should always > > invalidate the indicators in the reset case? More information regarding > > the attack vector would help here :) > > > > My main concern is that we would need to synchronize against a single > > interrupt that covers all kinds of I/O interrupts, not just a single > > device... > > > > Could we synchronize on struct airq_info's lock member? If we were > to grab all of these that might be involved... > > AFAIU for the synchronize implementation we need a lock or a set of locks > that contain all the possible vring_interrupt() calls with the queuues > that belong to the given device as a critical section. That way, one > has the acquire's and release's in place so that the vrign_interrupt() > either guaranteed to finish before the change of driver_ready is > guaranteed to be complete, or it is guaranteed to see the change. > > In any case, I guess we should first get clear on the first part. I.e. > when do we want to allow host->guest notifications. > > Regards, > Halil > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5910FC433F5 for ; Tue, 12 Apr 2022 02:24:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id F0A2660EA3; Tue, 12 Apr 2022 02:24:55 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RGJBc1zfL-Cx; Tue, 12 Apr 2022 02:24:54 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 44C2360E43; Tue, 12 Apr 2022 02:24:54 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 2276FC0082; Tue, 12 Apr 2022 02:24:54 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id ED8FFC002F for ; Tue, 12 Apr 2022 02:24:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id CBF93405E2 for ; Tue, 12 Apr 2022 02:24:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wTjXAbJ6LC1E for ; Tue, 12 Apr 2022 02:24:52 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id DAAFE40122 for ; Tue, 12 Apr 2022 02:24:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1649730290; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=U91PiURNFV2fKxTIEP7GO5tX2GZNd7Bfp7ekztFiEcU=; b=HiOB0yGOniOKHBcsXmsQmYknMIF9hga060rCgwWOiLCCvVot8iz4r5W8LuHMkN3xB0sLmR feqsjc0EaQWsHsv6y4nJ56acaIJz7IPBy2nOey1H9xaQUahVBtKztYBDsIgx9cfmRqTOt7 hqjo+n39u/MYWFvw1re3XW1Ophm+y68= Received: from mail-lf1-f70.google.com (mail-lf1-f70.google.com [209.85.167.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-154-hOpSWZ3DPzKEanzqaWc8jA-1; Mon, 11 Apr 2022 22:24:48 -0400 X-MC-Unique: hOpSWZ3DPzKEanzqaWc8jA-1 Received: by mail-lf1-f70.google.com with SMTP id j12-20020a05651231cc00b0045dd5f543eeso5244707lfe.15 for ; Mon, 11 Apr 2022 19:24:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=U91PiURNFV2fKxTIEP7GO5tX2GZNd7Bfp7ekztFiEcU=; b=35ghd/g+O7qj42+/5Jx2G0RPgAN3xorOuavpZYsfNqMdEATQD4hxtF2PzNSWOXMNJp 5W/UEHApgu+4C89Qm+MGyrh+EVf3kO0O/ZdfpRE2S+gZ8u/UYSuUYGl30qjllS0m1hH8 aKTrhrIFG/NgLFY58lzgu8DzKvkqDmYyHOT01wHzXobcRNhyS7yhEWcN7lKV9WKyoxN7 c4VdXbNko9KWubQe4FItwX9RJ+g02SA/MKWwEzhkFK6Q48/ixR1CJiaOtjz8PpYgDogE 0ZeJNyF5JSxmzN2m2z+bFf0CQ3vT6D+LqS1qPGMFPm0fC/H7dJ0EWWGKlQYeNd0csMuD MWCA== X-Gm-Message-State: AOAM533t5jf+7Hyy6s3+PLBEGeVxiGa60AXHUyr2TVFX/ETaKzIFUsAm yVweHHhJ6omsz+cpNGHEh/H5YYdEHTjWAd/CTEfnY41CqnHuY2Qu8llt2PhaPG5+Kpzu9dR0ygF FEd/Vstei16aK2Sgxe7R0WDy+is+7v6P/JR14YGhXXgIoNbLjjW1IJyQNbA== X-Received: by 2002:a05:6512:b81:b0:448:b342:513c with SMTP id b1-20020a0565120b8100b00448b342513cmr23533666lfv.257.1649730287279; Mon, 11 Apr 2022 19:24:47 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyriilgsCa0fnjLvRc9+m4ztpQqX137F7IBRBNtJ2Ku+BVjGMKTZvwHHTDT7U+xGoryLTsKpDLdOUKsJnHhYd8= X-Received: by 2002:a05:6512:b81:b0:448:b342:513c with SMTP id b1-20020a0565120b8100b00448b342513cmr23533653lfv.257.1649730287091; Mon, 11 Apr 2022 19:24:47 -0700 (PDT) MIME-Version: 1.0 References: <20220406083538.16274-1-jasowang@redhat.com> <20220406083538.16274-5-jasowang@redhat.com> <20220406075952-mutt-send-email-mst@kernel.org> <87wng2e527.fsf@redhat.com> <20220408150307.24b6b99f.pasic@linux.ibm.com> <20220410034556-mutt-send-email-mst@kernel.org> <877d7vbspu.fsf@redhat.com> <20220412020145.32e26e5a.pasic@linux.ibm.com> In-Reply-To: <20220412020145.32e26e5a.pasic@linux.ibm.com> From: Jason Wang Date: Tue, 12 Apr 2022 10:24:35 +0800 Message-ID: Subject: Re: [PATCH V2 4/5] virtio-pci: implement synchronize_vqs() To: Halil Pasic Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: "Paul E. McKenney" , "Michael S. Tsirkin" , Peter Zijlstra , Marc Zyngier , Cornelia Huck , linux-kernel , virtualization , Thomas Gleixner X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" T24gVHVlLCBBcHIgMTIsIDIwMjIgYXQgODowMiBBTSBIYWxpbCBQYXNpYyA8cGFzaWNAbGludXgu aWJtLmNvbT4gd3JvdGU6Cj4KPiBPbiBNb24sIDExIEFwciAyMDIyIDE2OjI3OjQxICswMjAwCj4g Q29ybmVsaWEgSHVjayA8Y29odWNrQHJlZGhhdC5jb20+IHdyb3RlOgo+Cj4gPiBPbiBNb24sIEFw ciAxMSAyMDIyLCBKYXNvbiBXYW5nIDxqYXNvd2FuZ0ByZWRoYXQuY29tPiB3cm90ZToKPiA+Cj4g PiA+IE9uIFN1biwgQXByIDEwLCAyMDIyIGF0IDM6NTEgUE0gTWljaGFlbCBTLiBUc2lya2luIDxt c3RAcmVkaGF0LmNvbT4gd3JvdGU6Cj4gPiA+Pgo+ID4gPj4gT24gRnJpLCBBcHIgMDgsIDIwMjIg YXQgMDM6MDM6MDdQTSArMDIwMCwgSGFsaWwgUGFzaWMgd3JvdGU6Cj4gPiA+PiA+IE9uIFdlZCwg MDYgQXByIDIwMjIgMTU6MDQ6MzIgKzAyMDAKPiA+ID4+ID4gQ29ybmVsaWEgSHVjayA8Y29odWNr QHJlZGhhdC5jb20+IHdyb3RlOgo+ID4gPj4gPgo+ID4gPj4gPiA+IE9uIFdlZCwgQXByIDA2IDIw MjIsICJNaWNoYWVsIFMuIFRzaXJraW4iIDxtc3RAcmVkaGF0LmNvbT4gd3JvdGU6Cj4gPiA+PiA+ ID4KPiA+ID4+ID4gPiA+IE9uIFdlZCwgQXByIDA2LCAyMDIyIGF0IDA0OjM1OjM3UE0gKzA4MDAs IEphc29uIFdhbmcgd3JvdGU6Cj4gPiA+PiA+ID4gPj4gVGhpcyBwYXRjaCBpbXBsZW1lbnRzIFBD SSB2ZXJzaW9uIG9mIHN5bmNocm9uaXplX3ZxcygpLgo+ID4gPj4gPiA+ID4+Cj4gPiA+PiA+ID4g Pj4gQ2M6IFRob21hcyBHbGVpeG5lciA8dGdseEBsaW51dHJvbml4LmRlPgo+ID4gPj4gPiA+ID4+ IENjOiBQZXRlciBaaWpsc3RyYSA8cGV0ZXJ6QGluZnJhZGVhZC5vcmc+Cj4gPiA+PiA+ID4gPj4g Q2M6ICJQYXVsIEUuIE1jS2VubmV5IiA8cGF1bG1ja0BrZXJuZWwub3JnPgo+ID4gPj4gPiA+ID4+ IENjOiBNYXJjIFp5bmdpZXIgPG1hekBrZXJuZWwub3JnPgo+ID4gPj4gPiA+ID4+IFNpZ25lZC1v ZmYtYnk6IEphc29uIFdhbmcgPGphc293YW5nQHJlZGhhdC5jb20+Cj4gPiA+PiA+ID4gPgo+ID4g Pj4gPiA+ID4gUGxlYXNlIGFkZCBpbXBsZW1lbnRhdGlvbnMgYXQgbGVhc3QgZm9yIGNjdyBhbmQg bW1pby4KPiA+ID4+ID4gPgo+ID4gPj4gPiA+IEknbSBub3Qgc3VyZSB3aGF0IChpZiBhbnl0aGlu ZykgY2FuL3Nob3VsZCBiZSBkb25lIGZvciBjY3cuLi4KPiA+ID4+ID4KPiA+ID4+ID4gSWYgbm90 aGluZyBuZWVkcyB0byBiZSBkb25lIEkgd291bGQgbGlrZSB0byBoYXZlIGF0IGxlYXN0IGEgY29t bWVudCBpbgo+ID4gPj4gPiB0aGUgY29kZSB0aGF0IGV4cGxhaW5zIHdoeS4gU28gdGhhdCBzb21l Ym9keSB3aG8gcmVhZHMgdGhlIGNvZGUKPiA+ID4+ID4gZG9lc24ndCB3b25kZXI6IHdoeSBpcyB2 aXJ0aW8tY2N3IG5vdCBpbXBsZW1lbnRpbmcgdGhhdCBjYWxsYmFjay4KPiA+ID4+Cj4gPiA+PiBS aWdodC4KPiA+ID4+Cj4gPiA+PiBJIGFtIGN1cnJlbnRseSB0aGlua2luZyBpbnN0ZWFkIG9mIG1h a2luZyB0aGlzIG9wdGlvbmFsIGluIHRoZQo+ID4gPj4gY29yZSB3ZSBzaG91bGQgbWFrZSBpdCBt YW5kYXRvcnksIGFuZCBoYXZlIHRyYW5zcG9ydHMgd2hpY2ggZG8gbm90Cj4gPiA+PiBuZWVkIHRv IHN5bmMgaGF2ZSBhbiBlbXB0eSBzdHViIHdpdGggZG9jdW1lbnRhdGlvbiBleHBsYWluaW5nIHdo eS4KPiA+Cj4gPiBZZXMsIHRoYXQgbWFrZXMgc2Vuc2UgdG8gbWUuIElmIHdlIGNhbiBleHBsYWlu IHdoeSB3ZSBkb24ndCBuZWVkIHRvIGRvCj4gPiBhbnl0aGluZywgd2Ugc2hvdWxkIGtlZXAgdGhh dCBleHBsYW5hdGlvbiBlYXNpbHkgYWNjZXNzaWJsZS4KPiA+Cj4gPiA+Pgo+ID4gPj4gQWxzbywg ZG8gd2Ugd2FudCB0byBkb2N1bWVudCB0aGlzIHN5bmMgaXMgZXhwbGljaXRseSBmb3IgaXJxIGVu YWJsZS9kaXNhYmxlPwo+ID4gPj4gc3luY2hyb25pemVfaXJxX2VuYWJsZV9kaXNhYmxlPwo+ID4g Pgo+ID4gPiBJIHdvdWxkIG5vdCBzaW5jZSB0aGUgdHJhbnNwb3J0IGlzIG5vdCBndWFyYW50ZWVk IHRvIHVzZSBhbiBpbnRlcnJ1cHQKPiA+ID4gZm9yIGNhbGxiYWNrcy4KPiA+ID4KPiA+ID4+Cj4g PiA+Pgo+ID4gPj4gPiA+Cj4gPiA+PiA+ID4gPgo+ID4gPj4gPiA+ID4+IC0tLQo+ID4gPj4gPiA+ ID4+ICBkcml2ZXJzL3ZpcnRpby92aXJ0aW9fcGNpX2NvbW1vbi5jIHwgMTQgKysrKysrKysrKysr KysKPiA+ID4+ID4gPiA+PiAgZHJpdmVycy92aXJ0aW8vdmlydGlvX3BjaV9jb21tb24uaCB8ICAy ICsrCj4gPiA+PiA+ID4gPj4gIGRyaXZlcnMvdmlydGlvL3ZpcnRpb19wY2lfbGVnYWN5LmMgfCAg MSArCj4gPiA+PiA+ID4gPj4gIGRyaXZlcnMvdmlydGlvL3ZpcnRpb19wY2lfbW9kZXJuLmMgfCAg MiArKwo+ID4gPj4gPiA+ID4+ICA0IGZpbGVzIGNoYW5nZWQsIDE5IGluc2VydGlvbnMoKykKPiA+ ID4+ID4gPiA+Pgo+ID4gPj4gPiA+ID4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL3ZpcnRpby92aXJ0 aW9fcGNpX2NvbW1vbi5jIGIvZHJpdmVycy92aXJ0aW8vdmlydGlvX3BjaV9jb21tb24uYwo+ID4g Pj4gPiA+ID4+IGluZGV4IGQ3MjRmNjc2NjA4Yi4uYjc4YzhiYzkzYTk3IDEwMDY0NAo+ID4gPj4g PiA+ID4+IC0tLSBhL2RyaXZlcnMvdmlydGlvL3ZpcnRpb19wY2lfY29tbW9uLmMKPiA+ID4+ID4g PiA+PiArKysgYi9kcml2ZXJzL3ZpcnRpby92aXJ0aW9fcGNpX2NvbW1vbi5jCj4gPiA+PiA+ID4g Pj4gQEAgLTM3LDYgKzM3LDIwIEBAIHZvaWQgdnBfc3luY2hyb25pemVfdmVjdG9ycyhzdHJ1Y3Qg dmlydGlvX2RldmljZSAqdmRldikKPiA+ID4+ID4gPiA+PiAgICAgICAgICAgICAgICAgIHN5bmNo cm9uaXplX2lycShwY2lfaXJxX3ZlY3Rvcih2cF9kZXYtPnBjaV9kZXYsIGkpKTsKPiA+ID4+ID4g PiA+PiAgfQo+ID4gPj4gPiA+ID4+Cj4gPiA+PiA+ID4gPj4gK3ZvaWQgdnBfc3luY2hyb25pemVf dnFzKHN0cnVjdCB2aXJ0aW9fZGV2aWNlICp2ZGV2KQo+ID4gPj4gPiA+ID4+ICt7Cj4gPiA+PiA+ ID4gPj4gKyAgICAgICAgc3RydWN0IHZpcnRpb19wY2lfZGV2aWNlICp2cF9kZXYgPSB0b192cF9k ZXZpY2UodmRldik7Cj4gPiA+PiA+ID4gPj4gKyAgICAgICAgaW50IGk7Cj4gPiA+PiA+ID4gPj4g Kwo+ID4gPj4gPiA+ID4+ICsgICAgICAgIGlmICh2cF9kZXYtPmludHhfZW5hYmxlZCkgewo+ID4g Pj4gPiA+ID4+ICsgICAgICAgICAgICAgICAgc3luY2hyb25pemVfaXJxKHZwX2Rldi0+cGNpX2Rl di0+aXJxKTsKPiA+ID4+ID4gPiA+PiArICAgICAgICAgICAgICAgIHJldHVybjsKPiA+ID4+ID4g PiA+PiArICAgICAgICB9Cj4gPiA+PiA+ID4gPj4gKwo+ID4gPj4gPiA+ID4+ICsgICAgICAgIGZv ciAoaSA9IDA7IGkgPCB2cF9kZXYtPm1zaXhfdmVjdG9yczsgKytpKQo+ID4gPj4gPiA+ID4+ICsg ICAgICAgICAgICAgICAgc3luY2hyb25pemVfaXJxKHBjaV9pcnFfdmVjdG9yKHZwX2Rldi0+cGNp X2RldiwgaSkpOwo+ID4gPj4gPiA+ID4+ICt9Cj4gPiA+PiA+ID4gPj4gKwo+ID4gPj4gPiA+Cj4g PiA+PiA+ID4gLi4uZ2l2ZW4gdGhhdCB0aGlzIHNlZW1zIHRvIHN5bmNocm9uaXplIHRocmVhZGVk IGludGVycnVwdCBoYW5kbGVycz8KPiA+ID4+ID4gPiBIYWxpbCwgZG8geW91IHRoaW5rIGNjdyBu ZWVkcyB0byBkbyBhbnl0aGluZz8gKEFGQUlDUywgd2Ugb25seSBoYXZlIG9uZQo+ID4gPj4gPiA+ ICdpcnEnIGZvciBjaGFubmVsIGRldmljZXMgYW55d2F5LCBhbmQgdGhlIGhhbmRsZXIganVzdCBj YWxscyB0aGUKPiA+ID4+ID4gPiByZWxldmFudCBjYWxsYmFja3MgZGlyZWN0bHkuKQo+ID4gPj4g Pgo+ID4gPj4gPiBTb3JyeSBJIGRvbid0IHVuZGVyc3RhbmQgZW5vdWdoIHlldC4gQSBtb3JlIHZl cmJvc2UgZG9jdW1lbnRhdGlvbiBvbgo+ID4gPj4gPiAidmlydGlvX3N5bmNocm9uaXplX3ZxcyAt IHN5bmNocm9uaXplIHdpdGggdmlydHF1ZXVlIGNhbGxiYWNrcyIgd291bGQKPiA+ID4+ID4gc3Vy ZWx5IGJlbmVmaXQgbWUuIEl0IG1heSBiZSBtb3JlIHRoYW4gZW5vdWdoIGZvciBhIGJhY2stYmVs dCBidXQgaXQKPiA+ID4+ID4gYWluJ3QgZW5vdWdoIGZvciBtZSB0byB0ZWxsIHdoYXQgaXMgdGhl IGNhbGxiYWNrIHN1cHBvc2VkIHRvIGFjY29tcGxpc2guCj4gPgo+ID4gKzEgZm9yIG1vcmUgZXhw bGFuYXRpb25zLgo+ID4KPiA+ID4+ID4KPiA+ID4+ID4gSSB3aWxsIGhhdmUgdG8gc3R1ZHkgdGhp cyBkaXNjdXNzaW9uIGFuZCB0aGUgY29kZSBtb3JlIHRob3JvdWdobHkuCj4gPiA+PiA+IFRlbnRh dGl2ZWx5IEkgc2lkZSB3aXRoIEphc29uIGFuZCBNaWNoYWVsIGluIGEgc2Vuc2UsIHRoYXQgSSBk b24ndAo+ID4gPj4gPiBiZWxpZXZlIHZpcnRpby1jY3cgaXMgc2FmZSBhZ2FpbnN0IHJvdWdoIGlu dGVycnVwdHMuCj4gPiA+Cj4gPiA+IFRoYXQncyBteSBmZWVsaW5nIGFzIHdlbGwuCj4gPgo+ID4g SSdkIHNheSBjY3cgaXMgc2FmZSBhZ2FpbnN0ICJub3RpZmljYXRpb24gaW50ZXJydXB0cyBiZWZv cmUgaW5kaWNhdG9ycwo+ID4gaGF2ZSBiZWVuIHJlZ2lzdGVyZWQiLgo+Cj4gSSBiZWxpZXZlIEph c29uJ3Mgc2NvcGUgaXMgYnJvYWRlciB0aGFuIHRoYXQuIExldCBtZSB0cnkgdG8gZXhwbGFpbi4g QQo+IHF1b3RlIGZvcm0gdGhlIHN0YW5kYXJkOgo+ICIiIgo+IDMuMS4xIERyaXZlciBSZXF1aXJl bWVudHM6IERldmljZSBJbml0aWFsaXphdGlvbgo+IFRoZSBkcml2ZXIgTVVTVCBmb2xsb3cgdGhp cyBzZXF1ZW5jZSB0byBpbml0aWFsaXplIGEgZGV2aWNlOgo+ICAgICAxLiBSZXNldCB0aGUgZGV2 aWNlLgo+ICAgICAyLiBTZXQgdGhlIEFDS05PV0xFREdFIHN0YXR1cyBiaXQ6IHRoZSBndWVzdCBP UyBoYXMgbm90aWNlZCB0aGUgZGV2aWNlLgo+ICAgICAzLiBTZXQgdGhlIERSSVZFUiBzdGF0dXMg Yml0OiB0aGUgZ3Vlc3QgT1Mga25vd3MgaG93IHRvIGRyaXZlIHRoZSBkZXZpY2UuCj4gICAgIDQu IFJlYWQgZGV2aWNlIGZlYXR1cmUgYml0cywgYW5kIHdyaXRlIHRoZSBzdWJzZXQgb2YgZmVhdHVy ZSBiaXRzIHVuZGVyc3Rvb2QgYnkgdGhlIE9TIGFuZCBkcml2ZXIgdG8gdGhlIGRldmljZS4gRHVy aW5nIHRoaXMgc3RlcCB0aGUgZHJpdmVyIE1BWSByZWFkIChidXQgTVVTVCBOT1Qgd3JpdGUpIHRo ZSBkZXZpY2Utc3BlY2lmaWMgY29uZmlndXJhdGlvbiBmaWVsZHMgdG8gY2hlY2sgdGhhdCBpdCBj YW4gc3VwcG9ydCB0aGUgZGV2aWNlIGJlZm9yZSBhY2NlcHRpbmcgaXQuCj4gICAgIDUuIFNldCB0 aGUgRkVBVFVSRVNfT0sgc3RhdHVzIGJpdC4gVGhlIGRyaXZlciBNVVNUIE5PVCBhY2NlcHQgbmV3 IGZlYXR1cmUgYml0cyBhZnRlciB0aGlzIHN0ZXAuCj4gICAgIDYuIFJlLXJlYWQgZGV2aWNlIHN0 YXR1cyB0byBlbnN1cmUgdGhlIEZFQVRVUkVTX09LIGJpdCBpcyBzdGlsbCBzZXQ6IG90aGVyd2lz ZSwgdGhlIGRldmljZSBkb2VzIG5vdCBzdXBwb3J0IG91ciBzdWJzZXQgb2YgZmVhdHVyZXMgYW5k IHRoZSBkZXZpY2UgaXMgdW51c2FibGUuCj4gICAgIDcuIFBlcmZvcm0gZGV2aWNlLXNwZWNpZmlj IHNldHVwLCBpbmNsdWRpbmcgZGlzY292ZXJ5IG9mIHZpcnRxdWV1ZXMgZm9yIHRoZSBkZXZpY2Us IG9wdGlvbmFsIHBlci1idXMgc2V0dXAsIHJlYWRpbmcgYW5kIHBvc3NpYmx5IHdyaXRpbmcgdGhl IGRldmljZeKAmXMgdmlydGlvIGNvbmZpZ3VyYXRpb24gc3BhY2UsIGFuZCBwb3B1bGF0aW9uIG9m IHZpcnRxdWV1ZXMuCj4gICAgIDguIFNldCB0aGUgRFJJVkVSX09LIHN0YXR1cyBiaXQuIEF0IHRo aXMgcG9pbnQgdGhlIGRldmljZSBpcyDigJxsaXZl4oCdLgo+ICIiIgo+Cj4gVGhhdCBtZWFucyBz dHVmZiBtYXkgaGFwcGVuIGJldHdlZW4gImRpc2NvdmVyeSBvZiB2aXJ0cXVldWVzIiBhbmQgIkRS SVZFUl9PSyIuIFNvIGl0Cj4gaXMgbm90IHN1ZmZpY2llbnQgdG8gYmUgInNhZmUgYWdhaW5zdCBu b3RpZmljYXRpb25zIGJlZm9yZSBpbmRpY2F0b3JzCj4gaGF2ZSBiZWVuIHJlZ2lzdGVyZWQiLCBi dXQgd2Ugd2FudCB0byBiZSBhbHNvIHNhZmUgYmV0d2VlbiAiaW5kaWNhdG9ycyBoYXZlCj4gYmVl biByZWdpc3RlcmVkIiBhbmQgIkRSSVZFUl9PSyBzdGF0dXMgaGFzIGJlZW4gc2V0Ii4KPgo+IEBK YXNvbjogY2FuIHlvdSBjb25maXJtPwoKRXhhY3RseS4KCj4KPiBSZWdhcmRpbmcgdGhlIHF1ZXN0 aW9uICJhcmUgd2Ugc2FmZSBhZ2FpbnN0IG5vdGlmaWNhdGlvbnMgYmVmb3JlCj4gaW5kaWNhdG9y cyBoYXZlIGJlZW4gcmVnaXN0ZXJlZCIgSSB0aGluayB3ZSByZWFsbHkgbmVlZCB0byB0aGluayBh Ym91dAo+IHNvbWV0aGluZyBsaWtlIFNlY3VyZSBFeGVjdXRpb24uIFdlIGRvbid0IGhhdmUsIGFu ZCB3ZSBhcmUgdW5saWtlbHkKPiB0byBoYXZlIGluIGhhcmR3YXJlIHZpcnRpby1jY3cgaW1wbGVt ZW50YXRpb25zLCBhbmQgZm9yIGEgbWFsaWNpb3VzIGh5cGVydmlzb3IKPiB0aGF0IGhhcyBmdWxs IGFjY2VzcyB0byB0aGUgZ3Vlc3QgbWVtb3J5IGhhcmRlbmluZyBtYWtlcyBubyBzZW5zZS4KCkRv ZXMgczM5MCBoYXZlIHNvbWV0aGluZyBsaWtlIG1lbW9yeSBlbmNyeXB0aW9uPyAoSSBndWVzcyB5 ZXMpLiBJbiB0aGUKY2FzZSBvZiB4ODYgVk0gZW5jcnlwdGlvbiwgdGhlIEkvTyBidWZmZXJzIHdl cmUgbm93IGRvbmUgdmlhIHNvZnR3YXJlCklPVExCLCB0aGF0J3Mgd2h5IGhhcmRlbmluZyBvZiB0 aGUgdmlydGlvIGRyaXZlciBpcyBuZWVkZWQgdG8gcHJldmVudAp0aGUgaHlwZXJ2aXNvciB0byBw b2tlIHRoZSBzd2lvdGxiIGV0Yy4KClRoYW5rcwoKPgo+IEJ1dCBpZiB3ZSBhc3N1bWUgdGhhdCBh biBhdHRhY2tlciBjYW4gaW5qZWN0IGFkYXB0ZXIgaW50ZXJydXB0cyBmb3IgYW4gYXJiaXRyYXJ5 Cj4gSVNDLCBhbmQgY2FuIHBva2UgYW55IHNoYXJlZCBtZW1vcnkgKG5vdGlmaWVyIGJpdHMgYXJl IHNoYXJlZCkuLi4gVGhpbmdzIG1pZ2h0Cj4gYmVjb21lIGNyaXRpY2FsIGFscmVhZHkgd2hlbiBy ZWdpc3Rlcl9hZGFwdGVyX2ludGVycnVwdCgpIGRvZXMgaXQncyBtYWdpYy4KPgo+Cj4gPiBGb3Ig dGhlIHJldmVyc2UgY2FzZSwgbWF5YmUgd2Ugc2hvdWxkIGFsd2F5cwo+ID4gaW52YWxpZGF0ZSB0 aGUgaW5kaWNhdG9ycyBpbiB0aGUgcmVzZXQgY2FzZT8gTW9yZSBpbmZvcm1hdGlvbiByZWdhcmRp bmcKPiA+IHRoZSBhdHRhY2sgdmVjdG9yIHdvdWxkIGhlbHAgaGVyZSA6KQo+ID4KPiA+IE15IG1h aW4gY29uY2VybiBpcyB0aGF0IHdlIHdvdWxkIG5lZWQgdG8gc3luY2hyb25pemUgYWdhaW5zdCBh IHNpbmdsZQo+ID4gaW50ZXJydXB0IHRoYXQgY292ZXJzIGFsbCBraW5kcyBvZiBJL08gaW50ZXJy dXB0cywgbm90IGp1c3QgYSBzaW5nbGUKPiA+IGRldmljZS4uLgo+ID4KPgo+IENvdWxkIHdlIHN5 bmNocm9uaXplIG9uIHN0cnVjdCBhaXJxX2luZm8ncyBsb2NrIG1lbWJlcj8gSWYgd2Ugd2VyZQo+ IHRvIGdyYWIgYWxsIG9mIHRoZXNlIHRoYXQgbWlnaHQgYmUgaW52b2x2ZWQuLi4KPgo+IEFGQUlV IGZvciB0aGUgc3luY2hyb25pemUgaW1wbGVtZW50YXRpb24gd2UgbmVlZCBhIGxvY2sgb3IgYSBz ZXQgb2YgbG9ja3MKPiB0aGF0IGNvbnRhaW4gYWxsIHRoZSBwb3NzaWJsZSB2cmluZ19pbnRlcnJ1 cHQoKSBjYWxscyB3aXRoIHRoZSBxdWV1dWVzCj4gdGhhdCBiZWxvbmcgdG8gdGhlIGdpdmVuIGRl dmljZSBhcyBhIGNyaXRpY2FsIHNlY3Rpb24uIFRoYXQgd2F5LCBvbmUKPiBoYXMgdGhlIGFjcXVp cmUncyBhbmQgcmVsZWFzZSdzIGluIHBsYWNlIHNvIHRoYXQgdGhlIHZyaWduX2ludGVycnVwdCgp Cj4gZWl0aGVyIGd1YXJhbnRlZWQgdG8gZmluaXNoIGJlZm9yZSB0aGUgY2hhbmdlIG9mIGRyaXZl cl9yZWFkeSBpcwo+IGd1YXJhbnRlZWQgdG8gYmUgY29tcGxldGUsIG9yIGl0IGlzIGd1YXJhbnRl ZWQgdG8gc2VlIHRoZSBjaGFuZ2UuCj4KPiBJbiBhbnkgY2FzZSwgSSBndWVzcyB3ZSBzaG91bGQg Zmlyc3QgZ2V0IGNsZWFyIG9uIHRoZSBmaXJzdCBwYXJ0LiBJLmUuCj4gd2hlbiBkbyB3ZSB3YW50 IHRvIGFsbG93IGhvc3QtPmd1ZXN0IG5vdGlmaWNhdGlvbnMuCj4KPiBSZWdhcmRzLAo+IEhhbGls Cj4KCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fClZpcnR1 YWxpemF0aW9uIG1haWxpbmcgbGlzdApWaXJ0dWFsaXphdGlvbkBsaXN0cy5saW51eC1mb3VuZGF0 aW9uLm9yZwpodHRwczovL2xpc3RzLmxpbnV4Zm91bmRhdGlvbi5vcmcvbWFpbG1hbi9saXN0aW5m by92aXJ0dWFsaXphdGlvbg==