From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4BB9C433EF for ; Wed, 27 Apr 2022 15:16:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239567AbiD0PTb (ORCPT ); Wed, 27 Apr 2022 11:19:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239712AbiD0PS6 (ORCPT ); Wed, 27 Apr 2022 11:18:58 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id D51533A5DA for ; Wed, 27 Apr 2022 08:15:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1651072543; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XWOAdhImrq2/TYQ+L3686e0RCpgtyiJg1rlmg+GyPGc=; b=c/M/YfWWi/Z19b3Rx6FbQkMLsqxiq1e3kus5CMdijGZT7uAFShWEQya667Fgy8X+ZCt577 B/CPUee5Uj/pi/eyY1vDxowbdd6azsUrsij8Ij35Pyxa2rV+qFhbCJ9eMqBc5/QZFmMFvw eg+4p18VG24NbtD+CsHvnCCIAqSSnoM= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-61-EnTZz21LO02aNnV-nJJGFw-1; Wed, 27 Apr 2022 11:15:41 -0400 X-MC-Unique: EnTZz21LO02aNnV-nJJGFw-1 Received: by mail-wm1-f72.google.com with SMTP id az19-20020a05600c601300b003914ac8efb8so689764wmb.2 for ; Wed, 27 Apr 2022 08:15:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=XWOAdhImrq2/TYQ+L3686e0RCpgtyiJg1rlmg+GyPGc=; b=3YGujOYhklVI+48UNksolZBfbQuldH/ElYgao/ijNE7pPb1pvinsGH9AHlRBCJj2JF kOwg+SCXE1jlaQkZ9UQrtpahq+zH6pSkSBgcjbfOsSyIazPIYW/Eq6ibD15C0pjm9WCi /HqhklSJBXiAwGDf4VMD97TGoq3PeXWHm3bIKTSmC13ymMDndh9V4HbxOjFtBxScZpOL AVXnHLxHDJdGMNDphBmZeaD1CJCi/Ehdpkh0nz4GQ26Ph5tB/du5leBi/2fP+0Xe/Ba1 VOuffjmGAOJ+YRfq01Zr3P6ViOHll7queoXD2GNTtiVeGvEarCCpaAkAyWpBZkv7zoK4 UAVw== X-Gm-Message-State: AOAM530Ko233E5WGRSS19h23Bs96XvCwZXgvlRJdyg58jyJkCEVptpbT 7RI8mcIaCjbmTp8APnaIBC9z56RWSoSmMo5Io+4xSiGkC4L9fBdmtXrp7yFIOKbR432oNpLAiy2 Vfwb00bzfnOKUIGQdcYEDUeLf X-Received: by 2002:a05:6000:1a8b:b0:20a:aaf7:75e8 with SMTP id f11-20020a0560001a8b00b0020aaaf775e8mr22115522wry.66.1651072540653; Wed, 27 Apr 2022 08:15:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwE2AlpbHYOLX4zY56a2kJT8noUAH/+2Yr1v5MnPXpsBur3d6mwrrg1tCJ6Nr9H5Fv/c4lYyw== X-Received: by 2002:a05:6000:1a8b:b0:20a:aaf7:75e8 with SMTP id f11-20020a0560001a8b00b0020aaaf775e8mr22115488wry.66.1651072540308; Wed, 27 Apr 2022 08:15:40 -0700 (PDT) Received: from redhat.com ([191.96.168.103]) by smtp.gmail.com with ESMTPSA id o26-20020adfa11a000000b0020ade8b8fc5sm7088993wro.107.2022.04.27.08.15.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Apr 2022 08:15:39 -0700 (PDT) Date: Wed, 27 Apr 2022 11:15:33 -0400 From: "Michael S. Tsirkin" To: Jason Wang Cc: Halil Pasic , Cornelia Huck , linux-kernel , virtualization , Stefano Garzarella , eperezma , Cindy Lu , Thomas Gleixner , Peter Zijlstra , "Paul E. McKenney" , Marc Zyngier Subject: Re: [PATCH V3 6/9] virtio-ccw: implement synchronize_cbs() Message-ID: <20220427111228-mutt-send-email-mst@kernel.org> References: <20220425233434-mutt-send-email-mst@kernel.org> <20220425233604-mutt-send-email-mst@kernel.org> <20220425235134-mutt-send-email-mst@kernel.org> <20220425235415-mutt-send-email-mst@kernel.org> <20220426022420-mutt-send-email-mst@kernel.org> <20220427022907-mutt-send-email-mst@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Apr 27, 2022 at 03:57:57PM +0800, Jason Wang wrote: > On Wed, Apr 27, 2022 at 2:30 PM Michael S. Tsirkin wrote: > > > > On Wed, Apr 27, 2022 at 11:53:25AM +0800, Jason Wang wrote: > > > On Tue, Apr 26, 2022 at 2:30 PM Michael S. Tsirkin wrote: > > > > > > > > On Tue, Apr 26, 2022 at 12:07:39PM +0800, Jason Wang wrote: > > > > > On Tue, Apr 26, 2022 at 11:55 AM Michael S. Tsirkin wrote: > > > > > > > > > > > > On Mon, Apr 25, 2022 at 11:53:24PM -0400, Michael S. Tsirkin wrote: > > > > > > > On Tue, Apr 26, 2022 at 11:42:45AM +0800, Jason Wang wrote: > > > > > > > > > > > > > > > > 在 2022/4/26 11:38, Michael S. Tsirkin 写道: > > > > > > > > > On Mon, Apr 25, 2022 at 11:35:41PM -0400, Michael S. Tsirkin wrote: > > > > > > > > > > On Tue, Apr 26, 2022 at 04:29:11AM +0200, Halil Pasic wrote: > > > > > > > > > > > On Mon, 25 Apr 2022 09:59:55 -0400 > > > > > > > > > > > "Michael S. Tsirkin" wrote: > > > > > > > > > > > > > > > > > > > > > > > On Mon, Apr 25, 2022 at 10:54:24AM +0200, Cornelia Huck wrote: > > > > > > > > > > > > > On Mon, Apr 25 2022, "Michael S. Tsirkin" wrote: > > > > > > > > > > > > > > On Mon, Apr 25, 2022 at 10:44:15AM +0800, Jason Wang wrote: > > > > > > > > > > > > > > > This patch tries to implement the synchronize_cbs() for ccw. For the > > > > > > > > > > > > > > > vring_interrupt() that is called via virtio_airq_handler(), the > > > > > > > > > > > > > > > synchronization is simply done via the airq_info's lock. For the > > > > > > > > > > > > > > > vring_interrupt() that is called via virtio_ccw_int_handler(), a per > > > > > > > > > > > > > > > device spinlock for irq is introduced ans used in the synchronization > > > > > > > > > > > > > > > method. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Cc: Thomas Gleixner > > > > > > > > > > > > > > > Cc: Peter Zijlstra > > > > > > > > > > > > > > > Cc: "Paul E. McKenney" > > > > > > > > > > > > > > > Cc: Marc Zyngier > > > > > > > > > > > > > > > Cc: Halil Pasic > > > > > > > > > > > > > > > Cc: Cornelia Huck > > > > > > > > > > > > > > > Signed-off-by: Jason Wang > > > > > > > > > > > > > > > > > > > > > > > > > > > > This is the only one that is giving me pause. Halil, Cornelia, > > > > > > > > > > > > > > should we be concerned about the performance impact here? > > > > > > > > > > > > > > Any chance it can be tested? > > > > > > > > > > > > > We can have a bunch of devices using the same airq structure, and the > > > > > > > > > > > > > sync cb creates a choke point, same as registering/unregistering. > > > > > > > > > > > > BTW can callbacks for multiple VQs run on multiple CPUs at the moment? > > > > > > > > > > > I'm not sure I understand the question. > > > > > > > > > > > > > > > > > > > > > > I do think we can have multiple CPUs that are executing some portion of > > > > > > > > > > > virtio_ccw_int_handler(). So I guess the answer is yes. Connie what do you think? > > > > > > > > > > > > > > > > > > > > > > On the other hand we could also end up serializing synchronize_cbs() > > > > > > > > > > > calls for different devices if they happen to use the same airq_info. But > > > > > > > > > > > this probably was not your question > > > > > > > > > > > > > > > > > > > > I am less concerned about synchronize_cbs being slow and more about > > > > > > > > > > the slowdown in interrupt processing itself. > > > > > > > > > > > > > > > > > > > > > > this patch serializes them on a spinlock. > > > > > > > > > > > > > > > > > > > > > > > Those could then pile up on the newly introduced spinlock. > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > Halil > > > > > > > > > > Hmm yea ... not good. > > > > > > > > > Is there any other way to synchronize with all callbacks? > > > > > > > > > > > > > > > > > > > > > > > > Maybe using rwlock as airq handler? > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > rwlock is still a shared cacheline bouncing between CPUs and > > > > > > > a bunch of ordering instructions. > > > > > > > > > > Yes, but it should be faster than spinlocks anyhow. > > > > > > > > > > > > Maybe something per-cpu + some IPIs to run things on all CPUs instead? > > > > > > > > > > Is this something like a customized version of synchronzie_rcu_expedited()? > > > > > > > > With interrupts running in an RCU read size critical section? > > > > > > For vring_interrupt(), yes. > > > > > > > > > > Quite possibly that is also an option. > > > > This will need a bunch of documentation since this is not > > > > a standard use of RCU, > > > > > > According to Documentation/RCU/requirements.rst, it looks like a legal case: > > > > > > " > > > The Linux kernel has interrupts, and RCU read-side critical sections are > > > legal within interrupt handlers and within interrupt-disabled regions of > > > code, as are invocations of call_rcu(). > > > " > > > > My problem is it is not clear what data is protected by rcu here. > > Nothing is tagged with __rcu or uses rcu_dereference. > > It should be the vq->broken here. We can rcuify it. I don't exactly exactly see how - rcu_dereference works on pointers. Feel free to send a patch. > > > We need at least an ack from rcu maintainers that this is > > a valid use of rcu and not an undocumented side effect. > > Yes. > > Thanks > > > > > > > And as discussed, synchronize_rcu_expedited() is not friendly to real > > > time workload. I think we can simply > > > > > > 1) protect vring_interrupt() with rcu_read_lock() > > > 2) use synchronize_rcu() in synchronize_cbs for ccw > > > > > > And if we care about the long delay we can use per device srcu to reduce that? > > > > > > Thanks > > > > > > > and probably get a confirmation > > > > from RCU maintainers that whatever assumptions we make > > > > are guaranteed to hold down the road. > > > > > > > > > > > > > > > > ... and I think classic and device interrupts are different enough > > > > > > here ... > > > > > > > > > > Yes. > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > MST > > > > > > > > > > > > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9D4E4C433F5 for ; Wed, 27 Apr 2022 15:15:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 2B34641936; Wed, 27 Apr 2022 15:15:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CxJnsu2wwSQp; Wed, 27 Apr 2022 15:15:54 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTPS id 0F1E741930; Wed, 27 Apr 2022 15:15:53 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D830DC0032; Wed, 27 Apr 2022 15:15:53 +0000 (UTC) Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by lists.linuxfoundation.org (Postfix) with ESMTP id 093D8C002D for ; Wed, 27 Apr 2022 15:15:53 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id EA70640ABB for ; Wed, 27 Apr 2022 15:15:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3Cfj7F1CmZsx for ; Wed, 27 Apr 2022 15:15:51 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp2.osuosl.org (Postfix) with ESMTPS id BDD3A4014C for ; Wed, 27 Apr 2022 15:15:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1651072550; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XWOAdhImrq2/TYQ+L3686e0RCpgtyiJg1rlmg+GyPGc=; b=PwuaW4XwlmAjCWoIX3iW4ZSvMMhL8B5IARL5MBYzJc5NjQjJHU+fTIuRAkmOz4P7e8GYq7 CJl9biVr8gedxfWR+MotzbIphZd/JcHcGcF+RQnBfoPNBS02lJRzOXbHKVXWx2N+cypUem OzXUYNztb8hvrgCGNJYpmCI78lO9TEA= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-115-m3ql83ZvPkSEmsnzV9IwwQ-1; Wed, 27 Apr 2022 11:15:41 -0400 X-MC-Unique: m3ql83ZvPkSEmsnzV9IwwQ-1 Received: by mail-wm1-f71.google.com with SMTP id d6-20020a05600c34c600b0039296a2ac7cso691397wmq.1 for ; Wed, 27 Apr 2022 08:15:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=XWOAdhImrq2/TYQ+L3686e0RCpgtyiJg1rlmg+GyPGc=; b=MNySHOIIIJr3onVG+jiw0DPn5WYts9GEjP8p8osVLDErkXdcUcW1o4CEEEA18DUUAv TinT719hQ6SiYK4/5nilz/F+OQCAIm5F7Yw3FkwgSqNDYV3/NNPLWCZgWyimy8fYW7pY yBN1VKSgQZa9VRKqZnXmgOhyuzQvmMkVJJl6mODWyNB6o8dRm8Y1KI7kP3M9p5VK5wI3 7UexLuVn8HZxhpgBVLcKsErMr5pcvc+GhTKll3hsvoo+hqvoCP+G9OqK+bpEjsVpRcmX fR0qLmVaDnY8cY+B6/Lbv9EBf9HbeQXtp69vZ/MkoOqHGHaDIgyHt/bbGA+3yhfvcHha TQlw== X-Gm-Message-State: AOAM531CkYasnFgcRpq6dlu5PP742E1bfM3b2hDzhyn5l4AMkbouC3u/ j7vYhu2W00qUagysmS6DMXzbiel5QYumOBfDYBvulVsUZsmVtHqluQo6n8uzs5//M/0xGKPWCL4 Ne3F5GLkapFQBrYZhssfB3LbuCIQtFUWWpG6X39wnwg== X-Received: by 2002:a05:6000:1a8b:b0:20a:aaf7:75e8 with SMTP id f11-20020a0560001a8b00b0020aaaf775e8mr22115524wry.66.1651072540654; Wed, 27 Apr 2022 08:15:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwE2AlpbHYOLX4zY56a2kJT8noUAH/+2Yr1v5MnPXpsBur3d6mwrrg1tCJ6Nr9H5Fv/c4lYyw== X-Received: by 2002:a05:6000:1a8b:b0:20a:aaf7:75e8 with SMTP id f11-20020a0560001a8b00b0020aaaf775e8mr22115488wry.66.1651072540308; Wed, 27 Apr 2022 08:15:40 -0700 (PDT) Received: from redhat.com ([191.96.168.103]) by smtp.gmail.com with ESMTPSA id o26-20020adfa11a000000b0020ade8b8fc5sm7088993wro.107.2022.04.27.08.15.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 27 Apr 2022 08:15:39 -0700 (PDT) Date: Wed, 27 Apr 2022 11:15:33 -0400 From: "Michael S. Tsirkin" To: Jason Wang Subject: Re: [PATCH V3 6/9] virtio-ccw: implement synchronize_cbs() Message-ID: <20220427111228-mutt-send-email-mst@kernel.org> References: <20220425233434-mutt-send-email-mst@kernel.org> <20220425233604-mutt-send-email-mst@kernel.org> <20220425235134-mutt-send-email-mst@kernel.org> <20220425235415-mutt-send-email-mst@kernel.org> <20220426022420-mutt-send-email-mst@kernel.org> <20220427022907-mutt-send-email-mst@kernel.org> MIME-Version: 1.0 In-Reply-To: Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mst@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Disposition: inline Cc: "Paul E. McKenney" , Cindy Lu , Peter Zijlstra , Marc Zyngier , Cornelia Huck , linux-kernel , virtualization , Halil Pasic , eperezma , Thomas Gleixner X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" T24gV2VkLCBBcHIgMjcsIDIwMjIgYXQgMDM6NTc6NTdQTSArMDgwMCwgSmFzb24gV2FuZyB3cm90 ZToKPiBPbiBXZWQsIEFwciAyNywgMjAyMiBhdCAyOjMwIFBNIE1pY2hhZWwgUy4gVHNpcmtpbiA8 bXN0QHJlZGhhdC5jb20+IHdyb3RlOgo+ID4KPiA+IE9uIFdlZCwgQXByIDI3LCAyMDIyIGF0IDEx OjUzOjI1QU0gKzA4MDAsIEphc29uIFdhbmcgd3JvdGU6Cj4gPiA+IE9uIFR1ZSwgQXByIDI2LCAy MDIyIGF0IDI6MzAgUE0gTWljaGFlbCBTLiBUc2lya2luIDxtc3RAcmVkaGF0LmNvbT4gd3JvdGU6 Cj4gPiA+ID4KPiA+ID4gPiBPbiBUdWUsIEFwciAyNiwgMjAyMiBhdCAxMjowNzozOVBNICswODAw LCBKYXNvbiBXYW5nIHdyb3RlOgo+ID4gPiA+ID4gT24gVHVlLCBBcHIgMjYsIDIwMjIgYXQgMTE6 NTUgQU0gTWljaGFlbCBTLiBUc2lya2luIDxtc3RAcmVkaGF0LmNvbT4gd3JvdGU6Cj4gPiA+ID4g PiA+Cj4gPiA+ID4gPiA+IE9uIE1vbiwgQXByIDI1LCAyMDIyIGF0IDExOjUzOjI0UE0gLTA0MDAs IE1pY2hhZWwgUy4gVHNpcmtpbiB3cm90ZToKPiA+ID4gPiA+ID4gPiBPbiBUdWUsIEFwciAyNiwg MjAyMiBhdCAxMTo0Mjo0NUFNICswODAwLCBKYXNvbiBXYW5nIHdyb3RlOgo+ID4gPiA+ID4gPiA+ ID4KPiA+ID4gPiA+ID4gPiA+IOWcqCAyMDIyLzQvMjYgMTE6MzgsIE1pY2hhZWwgUy4gVHNpcmtp biDlhpnpgZM6Cj4gPiA+ID4gPiA+ID4gPiA+IE9uIE1vbiwgQXByIDI1LCAyMDIyIGF0IDExOjM1 OjQxUE0gLTA0MDAsIE1pY2hhZWwgUy4gVHNpcmtpbiB3cm90ZToKPiA+ID4gPiA+ID4gPiA+ID4g PiBPbiBUdWUsIEFwciAyNiwgMjAyMiBhdCAwNDoyOToxMUFNICswMjAwLCBIYWxpbCBQYXNpYyB3 cm90ZToKPiA+ID4gPiA+ID4gPiA+ID4gPiA+IE9uIE1vbiwgMjUgQXByIDIwMjIgMDk6NTk6NTUg LTA0MDAKPiA+ID4gPiA+ID4gPiA+ID4gPiA+ICJNaWNoYWVsIFMuIFRzaXJraW4iIDxtc3RAcmVk aGF0LmNvbT4gd3JvdGU6Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gPiA+ ID4gPiBPbiBNb24sIEFwciAyNSwgMjAyMiBhdCAxMDo1NDoyNEFNICswMjAwLCBDb3JuZWxpYSBI dWNrIHdyb3RlOgo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IE9uIE1vbiwgQXByIDI1IDIwMjIs ICJNaWNoYWVsIFMuIFRzaXJraW4iIDxtc3RAcmVkaGF0LmNvbT4gd3JvdGU6Cj4gPiA+ID4gPiA+ ID4gPiA+ID4gPiA+ID4gPiBPbiBNb24sIEFwciAyNSwgMjAyMiBhdCAxMDo0NDoxNUFNICswODAw LCBKYXNvbiBXYW5nIHdyb3RlOgo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBUaGlzIHBh dGNoIHRyaWVzIHRvIGltcGxlbWVudCB0aGUgc3luY2hyb25pemVfY2JzKCkgZm9yIGNjdy4gRm9y IHRoZQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB2cmluZ19pbnRlcnJ1cHQoKSB0aGF0 IGlzIGNhbGxlZCB2aWEgdmlydGlvX2FpcnFfaGFuZGxlcigpLCB0aGUKPiA+ID4gPiA+ID4gPiA+ ID4gPiA+ID4gPiA+ID4gc3luY2hyb25pemF0aW9uIGlzIHNpbXBseSBkb25lIHZpYSB0aGUgYWly cV9pbmZvJ3MgbG9jay4gRm9yIHRoZQo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiB2cmlu Z19pbnRlcnJ1cHQoKSB0aGF0IGlzIGNhbGxlZCB2aWEgdmlydGlvX2Njd19pbnRfaGFuZGxlcigp LCBhIHBlcgo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBkZXZpY2Ugc3BpbmxvY2sgZm9y IGlycSBpcyBpbnRyb2R1Y2VkIGFucyB1c2VkIGluIHRoZSBzeW5jaHJvbml6YXRpb24KPiA+ID4g PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gbWV0aG9kLgo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ ID4gPgo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBDYzogVGhvbWFzIEdsZWl4bmVyIDx0 Z2x4QGxpbnV0cm9uaXguZGU+Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IENjOiBQZXRl ciBaaWpsc3RyYSA8cGV0ZXJ6QGluZnJhZGVhZC5vcmc+Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ ID4gPiA+IENjOiAiUGF1bCBFLiBNY0tlbm5leSIgPHBhdWxtY2tAa2VybmVsLm9yZz4KPiA+ID4g PiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gQ2M6IE1hcmMgWnluZ2llciA8bWF6QGtlcm5lbC5vcmc+ Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IENjOiBIYWxpbCBQYXNpYyA8cGFzaWNAbGlu dXguaWJtLmNvbT4KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gQ2M6IENvcm5lbGlhIEh1 Y2sgPGNvaHVja0ByZWRoYXQuY29tPgo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBTaWdu ZWQtb2ZmLWJ5OiBKYXNvbiBXYW5nIDxqYXNvd2FuZ0ByZWRoYXQuY29tPgo+ID4gPiA+ID4gPiA+ ID4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IFRoaXMgaXMgdGhlIG9u bHkgb25lIHRoYXQgaXMgZ2l2aW5nIG1lIHBhdXNlLiBIYWxpbCwgQ29ybmVsaWEsCj4gPiA+ID4g PiA+ID4gPiA+ID4gPiA+ID4gPiBzaG91bGQgd2UgYmUgY29uY2VybmVkIGFib3V0IHRoZSBwZXJm b3JtYW5jZSBpbXBhY3QgaGVyZT8KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiA+IEFueSBjaGFu Y2UgaXQgY2FuIGJlIHRlc3RlZD8KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBXZSBjYW4gaGF2 ZSBhIGJ1bmNoIG9mIGRldmljZXMgdXNpbmcgdGhlIHNhbWUgYWlycSBzdHJ1Y3R1cmUsIGFuZCB0 aGUKPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gPiBzeW5jIGNiIGNyZWF0ZXMgYSBjaG9rZSBwb2lu dCwgc2FtZSBhcyByZWdpc3RlcmluZy91bnJlZ2lzdGVyaW5nLgo+ID4gPiA+ID4gPiA+ID4gPiA+ ID4gPiBCVFcgY2FuIGNhbGxiYWNrcyBmb3IgbXVsdGlwbGUgVlFzIHJ1biBvbiBtdWx0aXBsZSBD UFVzIGF0IHRoZSBtb21lbnQ/Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBJJ20gbm90IHN1cmUgSSB1 bmRlcnN0YW5kIHRoZSBxdWVzdGlvbi4KPiA+ID4gPiA+ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ ID4gPiA+ID4gPiBJIGRvIHRoaW5rIHdlIGNhbiBoYXZlIG11bHRpcGxlIENQVXMgdGhhdCBhcmUg ZXhlY3V0aW5nIHNvbWUgcG9ydGlvbiBvZgo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdmlydGlvX2Nj d19pbnRfaGFuZGxlcigpLiBTbyBJIGd1ZXNzIHRoZSBhbnN3ZXIgaXMgeWVzLiBDb25uaWUgd2hh dCBkbyB5b3UgdGhpbms/Cj4gPiA+ID4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gPiA+ ID4gT24gdGhlIG90aGVyIGhhbmQgd2UgY291bGQgYWxzbyBlbmQgdXAgc2VyaWFsaXppbmcgc3lu Y2hyb25pemVfY2JzKCkKPiA+ID4gPiA+ID4gPiA+ID4gPiA+IGNhbGxzIGZvciBkaWZmZXJlbnQg ZGV2aWNlcyBpZiB0aGV5IGhhcHBlbiB0byB1c2UgdGhlIHNhbWUgYWlycV9pbmZvLiBCdXQKPiA+ ID4gPiA+ID4gPiA+ID4gPiA+IHRoaXMgcHJvYmFibHkgd2FzIG5vdCB5b3VyIHF1ZXN0aW9uCj4g PiA+ID4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+ID4gPiBJIGFtIGxlc3MgY29uY2VybmVk IGFib3V0ICBzeW5jaHJvbml6ZV9jYnMgYmVpbmcgc2xvdyBhbmQgbW9yZSBhYm91dAo+ID4gPiA+ ID4gPiA+ID4gPiA+IHRoZSBzbG93ZG93biBpbiBpbnRlcnJ1cHQgcHJvY2Vzc2luZyBpdHNlbGYu Cj4gPiA+ID4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gdGhpcyBwYXRjaCBz ZXJpYWxpemVzIHRoZW0gb24gYSBzcGlubG9jay4KPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4KPiA+ ID4gPiA+ID4gPiA+ID4gPiA+IFRob3NlIGNvdWxkIHRoZW4gcGlsZSB1cCBvbiB0aGUgbmV3bHkg aW50cm9kdWNlZCBzcGlubG9jay4KPiA+ID4gPiA+ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4g PiA+ID4gPiBSZWdhcmRzLAo+ID4gPiA+ID4gPiA+ID4gPiA+ID4gSGFsaWwKPiA+ID4gPiA+ID4g PiA+ID4gPiBIbW0geWVhIC4uLiBub3QgZ29vZC4KPiA+ID4gPiA+ID4gPiA+ID4gSXMgdGhlcmUg YW55IG90aGVyIHdheSB0byBzeW5jaHJvbml6ZSB3aXRoIGFsbCBjYWxsYmFja3M/Cj4gPiA+ID4g PiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+IE1heWJlIHVzaW5nIHJ3bG9j ayBhcyBhaXJxIGhhbmRsZXI/Cj4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gVGhhbmtz Cj4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4gcndsb2NrIGlzIHN0 aWxsIGEgc2hhcmVkIGNhY2hlbGluZSBib3VuY2luZyBiZXR3ZWVuIENQVXMgYW5kCj4gPiA+ID4g PiA+ID4gYSBidW5jaCBvZiBvcmRlcmluZyBpbnN0cnVjdGlvbnMuCj4gPiA+ID4gPgo+ID4gPiA+ ID4gWWVzLCBidXQgaXQgc2hvdWxkIGJlIGZhc3RlciB0aGFuIHNwaW5sb2NrcyBhbnlob3cuCj4g PiA+ID4gPgo+ID4gPiA+ID4gPiA+IE1heWJlIHNvbWV0aGluZyBwZXItY3B1ICsgc29tZSBJUElz IHRvIHJ1biB0aGluZ3Mgb24gYWxsIENQVXMgaW5zdGVhZD8KPiA+ID4gPiA+Cj4gPiA+ID4gPiBJ cyB0aGlzIHNvbWV0aGluZyBsaWtlIGEgY3VzdG9taXplZCB2ZXJzaW9uIG9mIHN5bmNocm9uemll X3JjdV9leHBlZGl0ZWQoKT8KPiA+ID4gPgo+ID4gPiA+IFdpdGggaW50ZXJydXB0cyBydW5uaW5n IGluIGFuIFJDVSByZWFkIHNpemUgY3JpdGljYWwgc2VjdGlvbj8KPiA+ID4KPiA+ID4gRm9yIHZy aW5nX2ludGVycnVwdCgpLCB5ZXMuCj4gPiA+Cj4gPiA+Cj4gPiA+ID4gUXVpdGUgcG9zc2libHkg dGhhdCBpcyBhbHNvIGFuIG9wdGlvbi4KPiA+ID4gPiBUaGlzIHdpbGwgbmVlZCBhIGJ1bmNoIG9m IGRvY3VtZW50YXRpb24gc2luY2UgdGhpcyBpcyBub3QKPiA+ID4gPiBhIHN0YW5kYXJkIHVzZSBv ZiBSQ1UsCj4gPiA+Cj4gPiA+IEFjY29yZGluZyB0byBEb2N1bWVudGF0aW9uL1JDVS9yZXF1aXJl bWVudHMucnN0LCBpdCBsb29rcyBsaWtlIGEgbGVnYWwgY2FzZToKPiA+ID4KPiA+ID4gIgo+ID4g PiBUaGUgTGludXgga2VybmVsIGhhcyBpbnRlcnJ1cHRzLCBhbmQgUkNVIHJlYWQtc2lkZSBjcml0 aWNhbCBzZWN0aW9ucyBhcmUKPiA+ID4gbGVnYWwgd2l0aGluIGludGVycnVwdCBoYW5kbGVycyBh bmQgd2l0aGluIGludGVycnVwdC1kaXNhYmxlZCByZWdpb25zIG9mCj4gPiA+IGNvZGUsIGFzIGFy ZSBpbnZvY2F0aW9ucyBvZiBjYWxsX3JjdSgpLgo+ID4gPiAiCj4gPgo+ID4gTXkgcHJvYmxlbSBp cyBpdCBpcyBub3QgY2xlYXIgd2hhdCBkYXRhIGlzIHByb3RlY3RlZCBieSByY3UgaGVyZS4KPiA+ IE5vdGhpbmcgaXMgdGFnZ2VkIHdpdGggX19yY3Ugb3IgdXNlcyByY3VfZGVyZWZlcmVuY2UuCj4g Cj4gSXQgc2hvdWxkIGJlIHRoZSB2cS0+YnJva2VuIGhlcmUuIFdlIGNhbiByY3VpZnkgaXQuCgoK SSBkb24ndCBleGFjdGx5IGV4YWN0bHkgc2VlIGhvdyAtIHJjdV9kZXJlZmVyZW5jZSB3b3JrcyBv biBwb2ludGVycy4KRmVlbCBmcmVlIHRvIHNlbmQgYSBwYXRjaC4KCgo+IAo+ID4gV2UgbmVlZCBh dCBsZWFzdCBhbiBhY2sgZnJvbSByY3UgbWFpbnRhaW5lcnMgdGhhdCB0aGlzIGlzCj4gPiBhIHZh bGlkIHVzZSBvZiByY3UgYW5kIG5vdCBhbiB1bmRvY3VtZW50ZWQgc2lkZSBlZmZlY3QuCj4gCj4g WWVzLgo+IAo+IFRoYW5rcwo+IAo+IAo+ID4KPiA+ID4gQW5kIGFzIGRpc2N1c3NlZCwgc3luY2hy b25pemVfcmN1X2V4cGVkaXRlZCgpIGlzIG5vdCBmcmllbmRseSB0byByZWFsCj4gPiA+IHRpbWUg d29ya2xvYWQuIEkgdGhpbmsgd2UgY2FuIHNpbXBseQo+ID4gPgo+ID4gPiAxKSBwcm90ZWN0IHZy aW5nX2ludGVycnVwdCgpIHdpdGggcmN1X3JlYWRfbG9jaygpCj4gPiA+IDIpIHVzZSBzeW5jaHJv bml6ZV9yY3UoKSBpbiBzeW5jaHJvbml6ZV9jYnMgZm9yIGNjdwo+ID4gPgo+ID4gPiBBbmQgaWYg d2UgY2FyZSBhYm91dCB0aGUgbG9uZyBkZWxheSB3ZSBjYW4gdXNlIHBlciBkZXZpY2Ugc3JjdSB0 byByZWR1Y2UgdGhhdD8KPiA+ID4KPiA+ID4gVGhhbmtzCj4gPiA+Cj4gPiA+ID4gYW5kIHByb2Jh Ymx5IGdldCBhIGNvbmZpcm1hdGlvbgo+ID4gPiA+IGZyb20gUkNVIG1haW50YWluZXJzIHRoYXQg d2hhdGV2ZXIgYXNzdW1wdGlvbnMgd2UgbWFrZQo+ID4gPiA+IGFyZSBndWFyYW50ZWVkIHRvIGhv bGQgZG93biB0aGUgcm9hZC4KPiA+ID4gPgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiAuLi4gYW5k IEkgdGhpbmsgY2xhc3NpYyBhbmQgZGV2aWNlIGludGVycnVwdHMgYXJlIGRpZmZlcmVudCBlbm91 Z2gKPiA+ID4gPiA+ID4gaGVyZSAuLi4KPiA+ID4gPiA+Cj4gPiA+ID4gPiBZZXMuCj4gPiA+ID4g Pgo+ID4gPiA+ID4gVGhhbmtzCj4gPiA+ID4gPgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4g Pgo+ID4gPiA+ID4gPiA+ID4gPiA+IC0tCj4gPiA+ID4gPiA+ID4gPiA+ID4gTVNUCj4gPiA+ID4g PiA+Cj4gPiA+ID4KPiA+CgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fXwpWaXJ0dWFsaXphdGlvbiBtYWlsaW5nIGxpc3QKVmlydHVhbGl6YXRpb25AbGlzdHMu bGludXgtZm91bmRhdGlvbi5vcmcKaHR0cHM6Ly9saXN0cy5saW51eGZvdW5kYXRpb24ub3JnL21h aWxtYW4vbGlzdGluZm8vdmlydHVhbGl6YXRpb24=