From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.4 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EDF5C4CECE for ; Fri, 13 Mar 2020 17:21:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E4964206B7 for ; Fri, 13 Mar 2020 17:21:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="jmMDqFmz" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726956AbgCMRVz (ORCPT ); Fri, 13 Mar 2020 13:21:55 -0400 Received: from smtp-fw-9102.amazon.com ([207.171.184.29]:44690 "EHLO smtp-fw-9102.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726446AbgCMRVy (ORCPT ); Fri, 13 Mar 2020 13:21:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1584120112; x=1615656112; h=date:from:to:cc:message-id:references:mime-version: content-transfer-encoding:in-reply-to:subject; bh=bTuPsH7AF4MczqLCOh4ZEWM4t7cj+3gohQ8SR0U6ZXw=; b=jmMDqFmzm+9YbQBvvRNiBZyu7RwggOAuhrmoh+aZWV0rXlndnI5aMUfl +n/csb4gO+gAR45HAG9h836XaPznUFmUwRTcqOcH4jrWweGQPsSqu8c2/ ryrA45btubCNtL2ALgh7MzoBYkme4oULhm8eTDwqGPikxenntFIS9jhvN U=; IronPort-SDR: JW0cZ6rA9C70JpjTu+PtridNwuuUyQx4+YAlmckpbYjwnpZvKgcj3PB0wM5zdrP6sQS5NG8AAW DPJzO9ciY+lw== X-IronPort-AV: E=Sophos;i="5.70,549,1574121600"; d="scan'208";a="31080865" Subject: Re: [RFC PATCH v3 06/12] xen-blkfront: add callbacks for PM suspend and hibernation Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-1a-67b371d8.us-east-1.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP; 13 Mar 2020 17:21:48 +0000 Received: from EX13MTAUEE002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan3.iad.amazon.com [10.40.159.166]) by email-inbound-relay-1a-67b371d8.us-east-1.amazon.com (Postfix) with ESMTPS id 6BED5A24C5; Fri, 13 Mar 2020 17:21:41 +0000 (UTC) Received: from EX13D08UEE004.ant.amazon.com (10.43.62.182) by EX13MTAUEE002.ant.amazon.com (10.43.62.24) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 13 Mar 2020 17:21:25 +0000 Received: from EX13MTAUEA001.ant.amazon.com (10.43.61.82) by EX13D08UEE004.ant.amazon.com (10.43.62.182) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 13 Mar 2020 17:21:25 +0000 Received: from dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com (172.22.96.68) by mail-relay.amazon.com (10.43.61.243) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Fri, 13 Mar 2020 17:21:24 +0000 Received: by dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com (Postfix, from userid 4335130) id 9ECD140348; Fri, 13 Mar 2020 17:21:24 +0000 (UTC) Date: Fri, 13 Mar 2020 17:21:24 +0000 From: Anchal Agarwal To: Roger Pau =?iso-8859-1?Q?Monn=E9?= CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , Message-ID: <20200313172124.GB8513@dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com> References: <890c404c585d7790514527f0c021056a7be6e748.1581721799.git.anchalag@amazon.com> <20200221142445.GZ4679@Air-de-Roger> <20200306184033.GA25358@dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com> <20200309095245.GY24458@Air-de-Roger.citrite.net> <20200312090435.GK24449@Air-de-Roger.citrite.net> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20200312090435.GK24449@Air-de-Roger.citrite.net> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Mar 12, 2020 at 10:04:35AM +0100, Roger Pau Monné wrote: > CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. > > > > On Wed, Mar 11, 2020 at 10:25:15PM +0000, Agarwal, Anchal wrote: > > Hi Roger, > > I am trying to understand your comments on indirect descriptors specially without polluting the mailing list hence emailing you personally. > > IMO it's better to send to the mailing list. The issues or questions > you have about indirect descriptors can be helpful to others in the > future. If there's no confidential information please send to the > list next time. > > Feel free to forward this reply to the list also. > Sure no problem at all. > > Hope that's ok by you. Please see my response inline. > > > > On Fri, Mar 06, 2020 at 06:40:33PM +0000, Anchal Agarwal wrote: > > > On Fri, Feb 21, 2020 at 03:24:45PM +0100, Roger Pau Monné wrote: > > > > On Fri, Feb 14, 2020 at 11:25:34PM +0000, Anchal Agarwal wrote: > > > > > blkfront_gather_backend_features(info); > > > > > /* Reset limits changed by blk_mq_update_nr_hw_queues(). */ > > > > > blkif_set_queue_limits(info); > > > > > @@ -2046,6 +2063,9 @@ static int blkif_recover(struct blkfront_info *info) > > > > > kick_pending_request_queues(rinfo); > > > > > } > > > > > > > > > > + if (frozen) > > > > > + return 0; > > > > > > > > I have to admit my memory is fuzzy here, but don't you need to > > > > re-queue requests in case the backend has different limits of indirect > > > > descriptors per request for example? > > > > > > > > Or do we expect that the frontend is always going to be resumed on the > > > > same backend, and thus features won't change? > > > > > > > So to understand your question better here, AFAIU the maximum number of indirect > > > grefs is fixed by the backend, but the frontend can issue requests with any > > > number of indirect segments as long as it's less than the number provided by > > > the backend. So by your question you mean this max number of MAX_INDIRECT_SEGMENTS > > > 256 on backend can change ? > > > > Yes, number of indirect descriptors supported by the backend can > > change, because you moved to a different backend, or because the > > maximum supported by the backend has changed. It's also possible to > > resume on a backend that has no indirect descriptors support at all. > > > > AFAIU, the code for requeuing the requests is only for xen suspend/resume. These request in the queue are > > same that gets added to queuelist in blkfront_resume. Also, even if indirect descriptors change on resume, > > they just need to be broadcasted to frontend and which means we could just mean that a request can process > > more data. > > Or less data. You could legitimately migrate from a host that has > indirect descriptors to one without, in which case requests would need > to be smaller to fit the ring slots. > > > We do setup indirect descriptors on front end on blkif_recover before returning and queue limits are > > setup accordingly. > > Am I missing anything here? > > Calling blkif_recover should take care of it AFAICT. As it resets the > queue limits according to the data announced on xenstore. > > I think I got confused, using blkif_recover should be fine, sorry. > Ok. Thanks for confirming. I will fixup other suggestions in the patch and send out a v4. > > > > > > > @@ -2625,6 +2671,62 @@ static void blkif_release(struct gendisk *disk, fmode_t mode) > > > > > mutex_unlock(&blkfront_mutex); > > > > > } > > > > > > > > > > +static int blkfront_freeze(struct xenbus_device *dev) > > > > > +{ > > > > > + unsigned int i; > > > > > + struct blkfront_info *info = dev_get_drvdata(&dev->dev); > > > > > + struct blkfront_ring_info *rinfo; > > > > > + /* This would be reasonable timeout as used in xenbus_dev_shutdown() */ > > > > > + unsigned int timeout = 5 * HZ; > > > > > + int err = 0; > > > > > + > > > > > + info->connected = BLKIF_STATE_FREEZING; > > > > > + > > > > > + blk_mq_freeze_queue(info->rq); > > > > > + blk_mq_quiesce_queue(info->rq); > > > > > > > > Don't you need to also drain the queue and make sure it's empty? > > > > > > > blk_mq_freeze_queue and blk_mq_quiesce_queue should take care of running HW queues synchronously > > > and making sure all the ongoing dispatches have finished. Did I understand your question right? > > > > Can you please add some check to that end? (ie: that there are no > > pending requests on any queue?) > > > > Well a check to see if there are any unconsumed responses could be done. > > I haven't come across use case in my testing where this failed but maybe there are other > > setups that may cause issue here. > > Thanks! It's mostly to be on the safe side if we expect the queues and > rings to be fully drained. > ACK. > Roger. Thanks, Anchal From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIM_ADSP_ALL,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5D42C10DCE for ; Fri, 13 Mar 2020 17:22:40 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B7F0B206B1 for ; Fri, 13 Mar 2020 17:22:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="jmMDqFmz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B7F0B206B1 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jCo0Q-0007la-O9; Fri, 13 Mar 2020 17:21:54 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jCo0O-0007lV-PA for xen-devel@lists.xenproject.org; Fri, 13 Mar 2020 17:21:52 +0000 X-Inumbo-ID: 20bcd661-654f-11ea-b35c-12813bfff9fa Received: from smtp-fw-9102.amazon.com (unknown [207.171.184.29]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 20bcd661-654f-11ea-b35c-12813bfff9fa; Fri, 13 Mar 2020 17:21:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1584120112; x=1615656112; h=date:from:to:cc:message-id:references:mime-version: content-transfer-encoding:in-reply-to:subject; bh=bTuPsH7AF4MczqLCOh4ZEWM4t7cj+3gohQ8SR0U6ZXw=; b=jmMDqFmzm+9YbQBvvRNiBZyu7RwggOAuhrmoh+aZWV0rXlndnI5aMUfl +n/csb4gO+gAR45HAG9h836XaPznUFmUwRTcqOcH4jrWweGQPsSqu8c2/ ryrA45btubCNtL2ALgh7MzoBYkme4oULhm8eTDwqGPikxenntFIS9jhvN U=; IronPort-SDR: JW0cZ6rA9C70JpjTu+PtridNwuuUyQx4+YAlmckpbYjwnpZvKgcj3PB0wM5zdrP6sQS5NG8AAW DPJzO9ciY+lw== X-IronPort-AV: E=Sophos;i="5.70,549,1574121600"; d="scan'208";a="31080865" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-1a-67b371d8.us-east-1.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP; 13 Mar 2020 17:21:48 +0000 Received: from EX13MTAUEE002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan3.iad.amazon.com [10.40.159.166]) by email-inbound-relay-1a-67b371d8.us-east-1.amazon.com (Postfix) with ESMTPS id 6BED5A24C5; Fri, 13 Mar 2020 17:21:41 +0000 (UTC) Received: from EX13D08UEE004.ant.amazon.com (10.43.62.182) by EX13MTAUEE002.ant.amazon.com (10.43.62.24) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 13 Mar 2020 17:21:25 +0000 Received: from EX13MTAUEA001.ant.amazon.com (10.43.61.82) by EX13D08UEE004.ant.amazon.com (10.43.62.182) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 13 Mar 2020 17:21:25 +0000 Received: from dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com (172.22.96.68) by mail-relay.amazon.com (10.43.61.243) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Fri, 13 Mar 2020 17:21:24 +0000 Received: by dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com (Postfix, from userid 4335130) id 9ECD140348; Fri, 13 Mar 2020 17:21:24 +0000 (UTC) Date: Fri, 13 Mar 2020 17:21:24 +0000 From: Anchal Agarwal To: Roger Pau =?iso-8859-1?Q?Monn=E9?= Message-ID: <20200313172124.GB8513@dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com> References: <890c404c585d7790514527f0c021056a7be6e748.1581721799.git.anchalag@amazon.com> <20200221142445.GZ4679@Air-de-Roger> <20200306184033.GA25358@dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com> <20200309095245.GY24458@Air-de-Roger.citrite.net> <20200312090435.GK24449@Air-de-Roger.citrite.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200312090435.GK24449@Air-de-Roger.citrite.net> User-Agent: Mutt/1.5.21 (2010-09-15) Precedence: Bulk Subject: Re: [Xen-devel] [RFC PATCH v3 06/12] xen-blkfront: add callbacks for PM suspend and hibernation X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: eduval@amazon.com, len.brown@intel.com, peterz@infradead.org, benh@kernel.crashing.org, x86@kernel.org, linux-mm@kvack.org, pavel@ucw.cz, hpa@zytor.com, tglx@linutronix.de, sstabellini@kernel.org, fllinden@amaozn.com, kamatam@amazon.com, mingo@redhat.com, xen-devel@lists.xenproject.org, sblbir@amazon.com, axboe@kernel.dk, konrad.wilk@oracle.com, anchalag@amazon.com, bp@alien8.de, boris.ostrovsky@oracle.com, jgross@suse.com, netdev@vger.kernel.org, linux-pm@vger.kernel.org, rjw@rjwysocki.net, linux-kernel@vger.kernel.org, vkuznets@redhat.com, davem@davemloft.net, dwmw@amazon.co.uk, roger.pau@citrix.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" T24gVGh1LCBNYXIgMTIsIDIwMjAgYXQgMTA6MDQ6MzVBTSArMDEwMCwgUm9nZXIgUGF1IE1vbm7D qSB3cm90ZToKPiBDQVVUSU9OOiBUaGlzIGVtYWlsIG9yaWdpbmF0ZWQgZnJvbSBvdXRzaWRlIG9m IHRoZSBvcmdhbml6YXRpb24uIERvIG5vdCBjbGljayBsaW5rcyBvciBvcGVuIGF0dGFjaG1lbnRz IHVubGVzcyB5b3UgY2FuIGNvbmZpcm0gdGhlIHNlbmRlciBhbmQga25vdyB0aGUgY29udGVudCBp cyBzYWZlLgo+IAo+IAo+IAo+IE9uIFdlZCwgTWFyIDExLCAyMDIwIGF0IDEwOjI1OjE1UE0gKzAw MDAsIEFnYXJ3YWwsIEFuY2hhbCB3cm90ZToKPiA+IEhpIFJvZ2VyLAo+ID4gSSBhbSB0cnlpbmcg dG8gdW5kZXJzdGFuZCB5b3VyIGNvbW1lbnRzIG9uIGluZGlyZWN0IGRlc2NyaXB0b3JzIHNwZWNp YWxseSB3aXRob3V0IHBvbGx1dGluZyB0aGUgbWFpbGluZyBsaXN0IGhlbmNlIGVtYWlsaW5nIHlv dSBwZXJzb25hbGx5Lgo+IAo+IElNTyBpdCdzIGJldHRlciB0byBzZW5kIHRvIHRoZSBtYWlsaW5n IGxpc3QuIFRoZSBpc3N1ZXMgb3IgcXVlc3Rpb25zCj4geW91IGhhdmUgYWJvdXQgaW5kaXJlY3Qg ZGVzY3JpcHRvcnMgY2FuIGJlIGhlbHBmdWwgdG8gb3RoZXJzIGluIHRoZQo+IGZ1dHVyZS4gSWYg dGhlcmUncyBubyBjb25maWRlbnRpYWwgaW5mb3JtYXRpb24gcGxlYXNlIHNlbmQgdG8gdGhlCj4g bGlzdCBuZXh0IHRpbWUuCj4gCj4gRmVlbCBmcmVlIHRvIGZvcndhcmQgdGhpcyByZXBseSB0byB0 aGUgbGlzdCBhbHNvLgo+ClN1cmUgbm8gcHJvYmxlbSBhdCBhbGwuCj4gPiBIb3BlIHRoYXQncyBv ayBieSB5b3UuICBQbGVhc2Ugc2VlIG15IHJlc3BvbnNlIGlubGluZS4KPiA+Cj4gPiAgICAgT24g RnJpLCBNYXIgMDYsIDIwMjAgYXQgMDY6NDA6MzNQTSArMDAwMCwgQW5jaGFsIEFnYXJ3YWwgd3Jv dGU6Cj4gPiAgICAgPiBPbiBGcmksIEZlYiAyMSwgMjAyMCBhdCAwMzoyNDo0NVBNICswMTAwLCBS b2dlciBQYXUgTW9ubsOpIHdyb3RlOgo+ID4gICAgID4gPiBPbiBGcmksIEZlYiAxNCwgMjAyMCBh dCAxMToyNTozNFBNICswMDAwLCBBbmNoYWwgQWdhcndhbCB3cm90ZToKPiA+ICAgICA+ID4gPiAg IGJsa2Zyb250X2dhdGhlcl9iYWNrZW5kX2ZlYXR1cmVzKGluZm8pOwo+ID4gICAgID4gPiA+ICAg LyogUmVzZXQgbGltaXRzIGNoYW5nZWQgYnkgYmxrX21xX3VwZGF0ZV9ucl9od19xdWV1ZXMoKS4g Ki8KPiA+ICAgICA+ID4gPiAgIGJsa2lmX3NldF9xdWV1ZV9saW1pdHMoaW5mbyk7Cj4gPiAgICAg PiA+ID4gQEAgLTIwNDYsNiArMjA2Myw5IEBAIHN0YXRpYyBpbnQgYmxraWZfcmVjb3ZlcihzdHJ1 Y3QgYmxrZnJvbnRfaW5mbyAqaW5mbykKPiA+ICAgICA+ID4gPiAgICAgICAgICAga2lja19wZW5k aW5nX3JlcXVlc3RfcXVldWVzKHJpbmZvKTsKPiA+ICAgICA+ID4gPiAgIH0KPiA+ICAgICA+ID4g Pgo+ID4gICAgID4gPiA+ICsgaWYgKGZyb3plbikKPiA+ICAgICA+ID4gPiArICAgICAgICAgcmV0 dXJuIDA7Cj4gPiAgICAgPiA+Cj4gPiAgICAgPiA+IEkgaGF2ZSB0byBhZG1pdCBteSBtZW1vcnkg aXMgZnV6enkgaGVyZSwgYnV0IGRvbid0IHlvdSBuZWVkIHRvCj4gPiAgICAgPiA+IHJlLXF1ZXVl IHJlcXVlc3RzIGluIGNhc2UgdGhlIGJhY2tlbmQgaGFzIGRpZmZlcmVudCBsaW1pdHMgb2YgaW5k aXJlY3QKPiA+ICAgICA+ID4gZGVzY3JpcHRvcnMgcGVyIHJlcXVlc3QgZm9yIGV4YW1wbGU/Cj4g PiAgICAgPiA+Cj4gPiAgICAgPiA+IE9yIGRvIHdlIGV4cGVjdCB0aGF0IHRoZSBmcm9udGVuZCBp cyBhbHdheXMgZ29pbmcgdG8gYmUgcmVzdW1lZCBvbiB0aGUKPiA+ICAgICA+ID4gc2FtZSBiYWNr ZW5kLCBhbmQgdGh1cyBmZWF0dXJlcyB3b24ndCBjaGFuZ2U/Cj4gPiAgICAgPiA+Cj4gPiAgICAg PiBTbyB0byB1bmRlcnN0YW5kIHlvdXIgcXVlc3Rpb24gYmV0dGVyIGhlcmUsIEFGQUlVIHRoZSAg bWF4aW11bSBudW1iZXIgb2YgaW5kaXJlY3QKPiA+ICAgICA+IGdyZWZzIGlzIGZpeGVkIGJ5IHRo ZSBiYWNrZW5kLCBidXQgdGhlIGZyb250ZW5kIGNhbiBpc3N1ZSByZXF1ZXN0cyB3aXRoIGFueQo+ ID4gICAgID4gbnVtYmVyIG9mIGluZGlyZWN0IHNlZ21lbnRzIGFzIGxvbmcgYXMgaXQncyBsZXNz IHRoYW4gdGhlIG51bWJlciBwcm92aWRlZCBieQo+ID4gICAgID4gdGhlIGJhY2tlbmQuIFNvIGJ5 IHlvdXIgcXVlc3Rpb24geW91IG1lYW4gdGhpcyBtYXggbnVtYmVyIG9mIE1BWF9JTkRJUkVDVF9T RUdNRU5UUwo+ID4gICAgID4gMjU2IG9uIGJhY2tlbmQgY2FuIGNoYW5nZSA/Cj4gPgo+ID4gICAg IFllcywgbnVtYmVyIG9mIGluZGlyZWN0IGRlc2NyaXB0b3JzIHN1cHBvcnRlZCBieSB0aGUgYmFj a2VuZCBjYW4KPiA+ICAgICBjaGFuZ2UsIGJlY2F1c2UgeW91IG1vdmVkIHRvIGEgZGlmZmVyZW50 IGJhY2tlbmQsIG9yIGJlY2F1c2UgdGhlCj4gPiAgICAgbWF4aW11bSBzdXBwb3J0ZWQgYnkgdGhl IGJhY2tlbmQgaGFzIGNoYW5nZWQuIEl0J3MgYWxzbyBwb3NzaWJsZSB0bwo+ID4gICAgIHJlc3Vt ZSBvbiBhIGJhY2tlbmQgdGhhdCBoYXMgbm8gaW5kaXJlY3QgZGVzY3JpcHRvcnMgc3VwcG9ydCBh dCBhbGwuCj4gPgo+ID4gQUZBSVUsIHRoZSBjb2RlIGZvciByZXF1ZXVpbmcgdGhlIHJlcXVlc3Rz IGlzIG9ubHkgZm9yIHhlbiBzdXNwZW5kL3Jlc3VtZS4gVGhlc2UgcmVxdWVzdCBpbiB0aGUgcXVl dWUgYXJlCj4gPiBzYW1lIHRoYXQgZ2V0cyBhZGRlZCB0byBxdWV1ZWxpc3QgaW4gYmxrZnJvbnRf cmVzdW1lLiBBbHNvLCBldmVuIGlmIGluZGlyZWN0IGRlc2NyaXB0b3JzIGNoYW5nZSBvbiByZXN1 bWUsCj4gPiB0aGV5IGp1c3QgbmVlZCB0byBiZSBicm9hZGNhc3RlZCB0byBmcm9udGVuZCBhbmQg d2hpY2ggbWVhbnMgd2UgY291bGQganVzdCBtZWFuIHRoYXQgYSByZXF1ZXN0IGNhbiBwcm9jZXNz Cj4gPiBtb3JlIGRhdGEuCj4gCj4gT3IgbGVzcyBkYXRhLiBZb3UgY291bGQgbGVnaXRpbWF0ZWx5 IG1pZ3JhdGUgZnJvbSBhIGhvc3QgdGhhdCBoYXMKPiBpbmRpcmVjdCBkZXNjcmlwdG9ycyB0byBv bmUgd2l0aG91dCwgaW4gd2hpY2ggY2FzZSByZXF1ZXN0cyB3b3VsZCBuZWVkCj4gdG8gYmUgc21h bGxlciB0byBmaXQgdGhlIHJpbmcgc2xvdHMuCj4gCj4gPiBXZSBkbyBzZXR1cCBpbmRpcmVjdCBk ZXNjcmlwdG9ycyBvbiBmcm9udCBlbmQgb24gYmxraWZfcmVjb3ZlciBiZWZvcmUgcmV0dXJuaW5n IGFuZCBxdWV1ZSBsaW1pdHMgYXJlCj4gPiBzZXR1cCBhY2NvcmRpbmdseS4KPiA+IEFtIEkgbWlz c2luZyBhbnl0aGluZyBoZXJlPwo+IAo+IENhbGxpbmcgYmxraWZfcmVjb3ZlciBzaG91bGQgdGFr ZSBjYXJlIG9mIGl0IEFGQUlDVC4gQXMgaXQgcmVzZXRzIHRoZQo+IHF1ZXVlIGxpbWl0cyBhY2Nv cmRpbmcgdG8gdGhlIGRhdGEgYW5ub3VuY2VkIG9uIHhlbnN0b3JlLgo+IAo+IEkgdGhpbmsgSSBn b3QgY29uZnVzZWQsIHVzaW5nIGJsa2lmX3JlY292ZXIgc2hvdWxkIGJlIGZpbmUsIHNvcnJ5Lgo+ IApPay4gVGhhbmtzIGZvciBjb25maXJtaW5nLiBJIHdpbGwgZml4dXAgb3RoZXIgc3VnZ2VzdGlv bnMgaW4gdGhlIHBhdGNoIGFuZCBzZW5kCm91dCBhIHY0Lgo+ID4KPiA+ICAgICA+ID4gPiBAQCAt MjYyNSw2ICsyNjcxLDYyIEBAIHN0YXRpYyB2b2lkIGJsa2lmX3JlbGVhc2Uoc3RydWN0IGdlbmRp c2sgKmRpc2ssIGZtb2RlX3QgbW9kZSkKPiA+ICAgICA+ID4gPiAgIG11dGV4X3VubG9jaygmYmxr ZnJvbnRfbXV0ZXgpOwo+ID4gICAgID4gPiA+ICB9Cj4gPiAgICAgPiA+ID4KPiA+ICAgICA+ID4g PiArc3RhdGljIGludCBibGtmcm9udF9mcmVlemUoc3RydWN0IHhlbmJ1c19kZXZpY2UgKmRldikK PiA+ICAgICA+ID4gPiArewo+ID4gICAgID4gPiA+ICsgdW5zaWduZWQgaW50IGk7Cj4gPiAgICAg PiA+ID4gKyBzdHJ1Y3QgYmxrZnJvbnRfaW5mbyAqaW5mbyA9IGRldl9nZXRfZHJ2ZGF0YSgmZGV2 LT5kZXYpOwo+ID4gICAgID4gPiA+ICsgc3RydWN0IGJsa2Zyb250X3JpbmdfaW5mbyAqcmluZm87 Cj4gPiAgICAgPiA+ID4gKyAvKiBUaGlzIHdvdWxkIGJlIHJlYXNvbmFibGUgdGltZW91dCBhcyB1 c2VkIGluIHhlbmJ1c19kZXZfc2h1dGRvd24oKSAqLwo+ID4gICAgID4gPiA+ICsgdW5zaWduZWQg aW50IHRpbWVvdXQgPSA1ICogSFo7Cj4gPiAgICAgPiA+ID4gKyBpbnQgZXJyID0gMDsKPiA+ICAg ICA+ID4gPiArCj4gPiAgICAgPiA+ID4gKyBpbmZvLT5jb25uZWN0ZWQgPSBCTEtJRl9TVEFURV9G UkVFWklORzsKPiA+ICAgICA+ID4gPiArCj4gPiAgICAgPiA+ID4gKyBibGtfbXFfZnJlZXplX3F1 ZXVlKGluZm8tPnJxKTsKPiA+ICAgICA+ID4gPiArIGJsa19tcV9xdWllc2NlX3F1ZXVlKGluZm8t PnJxKTsKPiA+ICAgICA+ID4KPiA+ICAgICA+ID4gRG9uJ3QgeW91IG5lZWQgdG8gYWxzbyBkcmFp biB0aGUgcXVldWUgYW5kIG1ha2Ugc3VyZSBpdCdzIGVtcHR5Pwo+ID4gICAgID4gPgo+ID4gICAg ID4gYmxrX21xX2ZyZWV6ZV9xdWV1ZSBhbmQgYmxrX21xX3F1aWVzY2VfcXVldWUgc2hvdWxkIHRh a2UgY2FyZSBvZiBydW5uaW5nIEhXIHF1ZXVlcyBzeW5jaHJvbm91c2x5Cj4gPiAgICAgPiBhbmQg bWFraW5nIHN1cmUgYWxsIHRoZSBvbmdvaW5nIGRpc3BhdGNoZXMgaGF2ZSBmaW5pc2hlZC4gRGlk IEkgdW5kZXJzdGFuZCB5b3VyIHF1ZXN0aW9uIHJpZ2h0Pwo+ID4KPiA+ICAgICBDYW4geW91IHBs ZWFzZSBhZGQgc29tZSBjaGVjayB0byB0aGF0IGVuZD8gKGllOiB0aGF0IHRoZXJlIGFyZSBubwo+ ID4gICAgIHBlbmRpbmcgcmVxdWVzdHMgb24gYW55IHF1ZXVlPykKPiA+Cj4gPiBXZWxsIGEgY2hl Y2sgdG8gc2VlIGlmIHRoZXJlIGFyZSBhbnkgdW5jb25zdW1lZCByZXNwb25zZXMgY291bGQgYmUg ZG9uZS4KPiA+IEkgaGF2ZW4ndCBjb21lIGFjcm9zcyB1c2UgY2FzZSBpbiBteSB0ZXN0aW5nIHdo ZXJlIHRoaXMgZmFpbGVkIGJ1dCBtYXliZSB0aGVyZSBhcmUgb3RoZXIKPiA+IHNldHVwcyB0aGF0 IG1heSBjYXVzZSBpc3N1ZSBoZXJlLgo+IAo+IFRoYW5rcyEgSXQncyBtb3N0bHkgdG8gYmUgb24g dGhlIHNhZmUgc2lkZSBpZiB3ZSBleHBlY3QgdGhlIHF1ZXVlcyBhbmQKPiByaW5ncyB0byBiZSBm dWxseSBkcmFpbmVkLgo+IApBQ0suCj4gUm9nZXIuClRoYW5rcywKQW5jaGFsCgpfX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpYZW4tZGV2ZWwgbWFpbGluZyBs aXN0Clhlbi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9yZwpodHRwczovL2xpc3RzLnhlbnByb2pl Y3Qub3JnL21haWxtYW4vbGlzdGluZm8veGVuLWRldmVs