From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga05.intel.com ([192.55.52.43]:21481 "EHLO mga05.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752037AbdC0Loz (ORCPT ); Mon, 27 Mar 2017 07:44:55 -0400 Subject: Re: [Intel-gfx] [PATCH] drm/i915: Keep all engine locks across scheduling To: Chris Wilson , intel-gfx@lists.freedesktop.org, "# v4 . 10+" References: <20170326084420.3231-1-chris@chris-wilson.co.uk> <20170326084637.13394-1-chris@chris-wilson.co.uk> <05a2f5b7-f004-573d-6f22-50ce59d2e62c@linux.intel.com> <20170327103123.GC10606@nuc-i3427.alporthouse.com> From: Tvrtko Ursulin Message-ID: <26f4763c-d603-d0c4-f12f-f6337aecb2f9@linux.intel.com> Date: Mon, 27 Mar 2017 12:39:38 +0100 MIME-Version: 1.0 In-Reply-To: <20170327103123.GC10606@nuc-i3427.alporthouse.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Sender: stable-owner@vger.kernel.org List-ID: On 27/03/2017 11:31, Chris Wilson wrote: > On Mon, Mar 27, 2017 at 11:11:47AM +0100, Tvrtko Ursulin wrote: >> >> On 26/03/2017 09:46, Chris Wilson wrote: >>> Unlocking is dangerous. In this case we combine an early update to the >>> out-of-queue request, because we know that it will be inserted into the >>> correct FIFO priority-ordered slot when it becomes ready in the future. >>> However, given sufficient enthusiasm, it may become ready as we are >>> continuing to reschedule, and so may gazump the FIFO if we have since >>> dropped its spinlock. The result is that it may be executed too early, >>> before its dependees. >>> >>> Fixes: 20311bd35060 ("drm/i915/scheduler: Execute requests in order of priorities") >>> Testcase: igt/gem_exec_whisper >>> Signed-off-by: Chris Wilson >>> Cc: Tvrtko Ursulin >>> Cc: # v4.10+ >>> --- >>> drivers/gpu/drm/i915/intel_lrc.c | 54 +++++++++++++++++++++++++++------------- >>> 1 file changed, 37 insertions(+), 17 deletions(-) >>> >>> diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c >>> index dd0e9d587852..3fdabba0a32d 100644 >>> --- a/drivers/gpu/drm/i915/intel_lrc.c >>> +++ b/drivers/gpu/drm/i915/intel_lrc.c >>> @@ -658,30 +658,47 @@ static void execlists_submit_request(struct drm_i915_gem_request *request) >>> spin_unlock_irqrestore(&engine->timeline->lock, flags); >>> } >>> >>> -static struct intel_engine_cs * >>> -pt_lock_engine(struct i915_priotree *pt, struct intel_engine_cs *locked) >>> +static inline struct intel_engine_cs * >>> +pt_lock_engine(struct i915_priotree *pt, unsigned long *locked) >>> { >>> - struct intel_engine_cs *engine; >>> - >>> - engine = container_of(pt, >>> - struct drm_i915_gem_request, >>> - priotree)->engine; >>> - if (engine != locked) { >>> - if (locked) >>> - spin_unlock_irq(&locked->timeline->lock); >>> - spin_lock_irq(&engine->timeline->lock); >>> - } >>> + struct intel_engine_cs *engine = >>> + container_of(pt, struct drm_i915_gem_request, priotree)->engine; >>> + >>> + /* Locking the engines in a random order will rightfully trigger a >>> + * spasm in lockdep. However, we can ignore lockdep (by marking each >>> + * as a seperate nesting) so long as we never nest the >>> + * engine->timeline->lock elsewhere. Also the number of nesting >>> + * subclasses is severely limited (7) which is going to cause an >>> + * issue at some point. >>> + * BUILD_BUG_ON(I915_NUM_ENGINES >= MAX_LOCKDEP_SUBCLASSES); >> >> Lets bite the bullet and not hide this BUILD_BUG_ON in a comment. :I > > The code would continue to work nevertheless, just lockdep would > eventually give up. I like it slightly better than taking either a > global spinlock for engine->execlists_queue insertion, or taking the > spinlock on every engine for scheduling. How often will we reschedule > across engines? Not sure. I think counting on "doesn't happen often" and "it still works" falls short of your high standards! ;) So a global execlist_queue lock if it must be.. >>> + */ >>> + if (!__test_and_set_bit(engine->id, locked)) >>> + spin_lock_nested(&engine->timeline->lock, >>> + hweight_long(*locked)); >>> >>> return engine; >>> } >>> >>> +static void >>> +unlock_engines(struct drm_i915_private *i915, unsigned long locked) >>> +{ >>> + struct intel_engine_cs *engine; >>> + unsigned long tmp; >>> + >>> + for_each_engine_masked(engine, i915, locked, tmp) >>> + spin_unlock(&engine->timeline->lock); >>> +} >>> + >>> static void execlists_schedule(struct drm_i915_gem_request *request, int prio) >>> { >>> - struct intel_engine_cs *engine = NULL; >>> + struct intel_engine_cs *engine; >>> struct i915_dependency *dep, *p; >>> struct i915_dependency stack; >>> + unsigned long locked = 0; >>> LIST_HEAD(dfs); >>> >>> + BUILD_BUG_ON(I915_NUM_ENGINES > BITS_PER_LONG); >>> + >>> if (prio <= READ_ONCE(request->priotree.priority)) >>> return; >>> >>> @@ -691,6 +708,9 @@ static void execlists_schedule(struct drm_i915_gem_request *request, int prio) >>> stack.signaler = &request->priotree; >>> list_add(&stack.dfs_link, &dfs); >>> >>> + GEM_BUG_ON(irqs_disabled()); >>> + local_irq_disable(); >>> + >> >> Why not just irqsave/restore? Sounds like too low level for this >> position in the flow. If just optimisation it would need a comment I >> think. > > It was because we are not taking the spin lock/unlock inside the same > block, so it felt dangerous. Who holds the irqflags? Hm yes, it cannot be made elegant. >>> /* Recursively bump all dependent priorities to match the new request. >>> * >>> * A naive approach would be to use recursion: >>> @@ -719,7 +739,7 @@ static void execlists_schedule(struct drm_i915_gem_request *request, int prio) >>> if (!RB_EMPTY_NODE(&pt->node)) >>> continue; >>> >>> - engine = pt_lock_engine(pt, engine); >>> + engine = pt_lock_engine(pt, &locked); >>> >>> /* If it is not already in the rbtree, we can update the >>> * priority inplace and skip over it (and its dependencies) >>> @@ -737,7 +757,7 @@ static void execlists_schedule(struct drm_i915_gem_request *request, int prio) >>> >>> INIT_LIST_HEAD(&dep->dfs_link); >>> >>> - engine = pt_lock_engine(pt, engine); >>> + engine = pt_lock_engine(pt, &locked); >>> >>> if (prio <= pt->priority) >>> continue; >>> @@ -750,8 +770,8 @@ static void execlists_schedule(struct drm_i915_gem_request *request, int prio) >>> engine->execlist_first = &pt->node; >>> } >>> >>> - if (engine) >>> - spin_unlock_irq(&engine->timeline->lock); >>> + unlock_engines(request->i915, locked); >>> + local_irq_enable(); >>> >>> /* XXX Do we need to preempt to make room for us and our deps? */ >>> } >>> >> >> I am trying to think whether removing the skip on requests not in >> the execution tree would work and help any. > > It's dangerous due to the duplicate branches in the dependency graph that > we are resolving to generate the topological ordering. We need a way to > do a mark-and-sweep whilst also ensuring that we end up with the correct > order. I'm open to (better :) suggestions. > >> Or if the above scheme >> is completely safe or we would need to lock atomically all engines >> requests on which will be touched. Especially since the code is only >> dealing with adjusting the priorities so I don't immediately see how >> it can cause out of order execution. > > interrupt leading to submit_request, which wants to then insert a > request into the execlist_queue rbtree vs ->schedule() also trying to > manipulate the rbtree (and in this case elements currently outside of the > rbtree). Our insertion into the rbtree ensures fifo so that we don't > reorder the equivalent priority dependencies during ->schedule(), hence > if we mark an out-of-rbtree request as a higher priority before > inserting all of its dependencies into the tree, if the submit_notify > occurs, it will insert the request into the tree before we get to insert > its dependencies, hence reordering. Ok I get the general idea. I don't have any better suggestions at the moment than trying the global lock. Luckily you have just removed one atomic from the irq handler so one step forward, two steps back. :) Regards, Tvrtko From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tvrtko Ursulin Subject: Re: [PATCH] drm/i915: Keep all engine locks across scheduling Date: Mon, 27 Mar 2017 12:39:38 +0100 Message-ID: <26f4763c-d603-d0c4-f12f-f6337aecb2f9@linux.intel.com> References: <20170326084420.3231-1-chris@chris-wilson.co.uk> <20170326084637.13394-1-chris@chris-wilson.co.uk> <05a2f5b7-f004-573d-6f22-50ce59d2e62c@linux.intel.com> <20170327103123.GC10606@nuc-i3427.alporthouse.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Return-path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by gabe.freedesktop.org (Postfix) with ESMTPS id B87AA6E1C8 for ; Mon, 27 Mar 2017 11:39:40 +0000 (UTC) In-Reply-To: <20170327103123.GC10606@nuc-i3427.alporthouse.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" To: Chris Wilson , intel-gfx@lists.freedesktop.org, "# v4 . 10+" List-Id: intel-gfx@lists.freedesktop.org Ck9uIDI3LzAzLzIwMTcgMTE6MzEsIENocmlzIFdpbHNvbiB3cm90ZToKPiBPbiBNb24sIE1hciAy NywgMjAxNyBhdCAxMToxMTo0N0FNICswMTAwLCBUdnJ0a28gVXJzdWxpbiB3cm90ZToKPj4KPj4g T24gMjYvMDMvMjAxNyAwOTo0NiwgQ2hyaXMgV2lsc29uIHdyb3RlOgo+Pj4gVW5sb2NraW5nIGlz IGRhbmdlcm91cy4gSW4gdGhpcyBjYXNlIHdlIGNvbWJpbmUgYW4gZWFybHkgdXBkYXRlIHRvIHRo ZQo+Pj4gb3V0LW9mLXF1ZXVlIHJlcXVlc3QsIGJlY2F1c2Ugd2Uga25vdyB0aGF0IGl0IHdpbGwg YmUgaW5zZXJ0ZWQgaW50byB0aGUKPj4+IGNvcnJlY3QgRklGTyBwcmlvcml0eS1vcmRlcmVkIHNs b3Qgd2hlbiBpdCBiZWNvbWVzIHJlYWR5IGluIHRoZSBmdXR1cmUuCj4+PiBIb3dldmVyLCBnaXZl biBzdWZmaWNpZW50IGVudGh1c2lhc20sIGl0IG1heSBiZWNvbWUgcmVhZHkgYXMgd2UgYXJlCj4+ PiBjb250aW51aW5nIHRvIHJlc2NoZWR1bGUsIGFuZCBzbyBtYXkgZ2F6dW1wIHRoZSBGSUZPIGlm IHdlIGhhdmUgc2luY2UKPj4+IGRyb3BwZWQgaXRzIHNwaW5sb2NrLiBUaGUgcmVzdWx0IGlzIHRo YXQgaXQgbWF5IGJlIGV4ZWN1dGVkIHRvbyBlYXJseSwKPj4+IGJlZm9yZSBpdHMgZGVwZW5kZWVz Lgo+Pj4KPj4+IEZpeGVzOiAyMDMxMWJkMzUwNjAgKCJkcm0vaTkxNS9zY2hlZHVsZXI6IEV4ZWN1 dGUgcmVxdWVzdHMgaW4gb3JkZXIgb2YgcHJpb3JpdGllcyIpCj4+PiBUZXN0Y2FzZTogaWd0L2dl bV9leGVjX3doaXNwZXIKPj4+IFNpZ25lZC1vZmYtYnk6IENocmlzIFdpbHNvbiA8Y2hyaXNAY2hy aXMtd2lsc29uLmNvLnVrPgo+Pj4gQ2M6IFR2cnRrbyBVcnN1bGluIDx0dnJ0a28udXJzdWxpbkBp bnRlbC5jb20+Cj4+PiBDYzogPHN0YWJsZUB2Z2VyLmtlcm5lbC5vcmc+ICMgdjQuMTArCj4+PiAt LS0KPj4+IGRyaXZlcnMvZ3B1L2RybS9pOTE1L2ludGVsX2xyYy5jIHwgNTQgKysrKysrKysrKysr KysrKysrKysrKysrKysrLS0tLS0tLS0tLS0tLQo+Pj4gMSBmaWxlIGNoYW5nZWQsIDM3IGluc2Vy dGlvbnMoKyksIDE3IGRlbGV0aW9ucygtKQo+Pj4KPj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL2dw dS9kcm0vaTkxNS9pbnRlbF9scmMuYyBiL2RyaXZlcnMvZ3B1L2RybS9pOTE1L2ludGVsX2xyYy5j Cj4+PiBpbmRleCBkZDBlOWQ1ODc4NTIuLjNmZGFiYmEwYTMyZCAxMDA2NDQKPj4+IC0tLSBhL2Ry aXZlcnMvZ3B1L2RybS9pOTE1L2ludGVsX2xyYy5jCj4+PiArKysgYi9kcml2ZXJzL2dwdS9kcm0v aTkxNS9pbnRlbF9scmMuYwo+Pj4gQEAgLTY1OCwzMCArNjU4LDQ3IEBAIHN0YXRpYyB2b2lkIGV4 ZWNsaXN0c19zdWJtaXRfcmVxdWVzdChzdHJ1Y3QgZHJtX2k5MTVfZ2VtX3JlcXVlc3QgKnJlcXVl c3QpCj4+PiAJc3Bpbl91bmxvY2tfaXJxcmVzdG9yZSgmZW5naW5lLT50aW1lbGluZS0+bG9jaywg ZmxhZ3MpOwo+Pj4gfQo+Pj4KPj4+IC1zdGF0aWMgc3RydWN0IGludGVsX2VuZ2luZV9jcyAqCj4+ PiAtcHRfbG9ja19lbmdpbmUoc3RydWN0IGk5MTVfcHJpb3RyZWUgKnB0LCBzdHJ1Y3QgaW50ZWxf ZW5naW5lX2NzICpsb2NrZWQpCj4+PiArc3RhdGljIGlubGluZSBzdHJ1Y3QgaW50ZWxfZW5naW5l X2NzICoKPj4+ICtwdF9sb2NrX2VuZ2luZShzdHJ1Y3QgaTkxNV9wcmlvdHJlZSAqcHQsIHVuc2ln bmVkIGxvbmcgKmxvY2tlZCkKPj4+IHsKPj4+IC0Jc3RydWN0IGludGVsX2VuZ2luZV9jcyAqZW5n aW5lOwo+Pj4gLQo+Pj4gLQllbmdpbmUgPSBjb250YWluZXJfb2YocHQsCj4+PiAtCQkJICAgICAg c3RydWN0IGRybV9pOTE1X2dlbV9yZXF1ZXN0LAo+Pj4gLQkJCSAgICAgIHByaW90cmVlKS0+ZW5n aW5lOwo+Pj4gLQlpZiAoZW5naW5lICE9IGxvY2tlZCkgewo+Pj4gLQkJaWYgKGxvY2tlZCkKPj4+ IC0JCQlzcGluX3VubG9ja19pcnEoJmxvY2tlZC0+dGltZWxpbmUtPmxvY2spOwo+Pj4gLQkJc3Bp bl9sb2NrX2lycSgmZW5naW5lLT50aW1lbGluZS0+bG9jayk7Cj4+PiAtCX0KPj4+ICsJc3RydWN0 IGludGVsX2VuZ2luZV9jcyAqZW5naW5lID0KPj4+ICsJCWNvbnRhaW5lcl9vZihwdCwgc3RydWN0 IGRybV9pOTE1X2dlbV9yZXF1ZXN0LCBwcmlvdHJlZSktPmVuZ2luZTsKPj4+ICsKPj4+ICsJLyog TG9ja2luZyB0aGUgZW5naW5lcyBpbiBhIHJhbmRvbSBvcmRlciB3aWxsIHJpZ2h0ZnVsbHkgdHJp Z2dlciBhCj4+PiArCSAqIHNwYXNtIGluIGxvY2tkZXAuIEhvd2V2ZXIsIHdlIGNhbiBpZ25vcmUg bG9ja2RlcCAoYnkgbWFya2luZyBlYWNoCj4+PiArCSAqIGFzIGEgc2VwZXJhdGUgbmVzdGluZykg c28gbG9uZyBhcyB3ZSBuZXZlciBuZXN0IHRoZQo+Pj4gKwkgKiBlbmdpbmUtPnRpbWVsaW5lLT5s b2NrIGVsc2V3aGVyZS4gQWxzbyB0aGUgbnVtYmVyIG9mIG5lc3RpbmcKPj4+ICsJICogc3ViY2xh c3NlcyBpcyBzZXZlcmVseSBsaW1pdGVkICg3KSB3aGljaCBpcyBnb2luZyB0byBjYXVzZSBhbgo+ Pj4gKwkgKiBpc3N1ZSBhdCBzb21lIHBvaW50Lgo+Pj4gKwkgKiBCVUlMRF9CVUdfT04oSTkxNV9O VU1fRU5HSU5FUyA+PSBNQVhfTE9DS0RFUF9TVUJDTEFTU0VTKTsKPj4KPj4gTGV0cyBiaXRlIHRo ZSBidWxsZXQgYW5kIG5vdCBoaWRlIHRoaXMgQlVJTERfQlVHX09OIGluIGEgY29tbWVudC4gOkkK Pgo+IFRoZSBjb2RlIHdvdWxkIGNvbnRpbnVlIHRvIHdvcmsgbmV2ZXJ0aGVsZXNzLCBqdXN0IGxv Y2tkZXAgd291bGQKPiBldmVudHVhbGx5IGdpdmUgdXAuIEkgbGlrZSBpdCBzbGlnaHRseSBiZXR0 ZXIgdGhhbiB0YWtpbmcgZWl0aGVyIGEKPiBnbG9iYWwgc3BpbmxvY2sgZm9yIGVuZ2luZS0+ZXhl Y2xpc3RzX3F1ZXVlIGluc2VydGlvbiwgb3IgdGFraW5nIHRoZQo+IHNwaW5sb2NrIG9uIGV2ZXJ5 IGVuZ2luZSBmb3Igc2NoZWR1bGluZy4gSG93IG9mdGVuIHdpbGwgd2UgcmVzY2hlZHVsZQo+IGFj cm9zcyBlbmdpbmVzPyBOb3Qgc3VyZS4KCkkgdGhpbmsgY291bnRpbmcgb24gImRvZXNuJ3QgaGFw cGVuIG9mdGVuIiBhbmQgIml0IHN0aWxsIHdvcmtzIiBmYWxscyAKc2hvcnQgb2YgeW91ciBoaWdo IHN0YW5kYXJkcyEgOykgU28gYSBnbG9iYWwgZXhlY2xpc3RfcXVldWUgbG9jayBpZiBpdCAKbXVz dCBiZS4uCgo+Pj4gKwkgKi8KPj4+ICsJaWYgKCFfX3Rlc3RfYW5kX3NldF9iaXQoZW5naW5lLT5p ZCwgbG9ja2VkKSkKPj4+ICsJCXNwaW5fbG9ja19uZXN0ZWQoJmVuZ2luZS0+dGltZWxpbmUtPmxv Y2ssCj4+PiArCQkJCSBod2VpZ2h0X2xvbmcoKmxvY2tlZCkpOwo+Pj4KPj4+IAlyZXR1cm4gZW5n aW5lOwo+Pj4gfQo+Pj4KPj4+ICtzdGF0aWMgdm9pZAo+Pj4gK3VubG9ja19lbmdpbmVzKHN0cnVj dCBkcm1faTkxNV9wcml2YXRlICppOTE1LCB1bnNpZ25lZCBsb25nIGxvY2tlZCkKPj4+ICt7Cj4+ PiArCXN0cnVjdCBpbnRlbF9lbmdpbmVfY3MgKmVuZ2luZTsKPj4+ICsJdW5zaWduZWQgbG9uZyB0 bXA7Cj4+PiArCj4+PiArCWZvcl9lYWNoX2VuZ2luZV9tYXNrZWQoZW5naW5lLCBpOTE1LCBsb2Nr ZWQsIHRtcCkKPj4+ICsJCXNwaW5fdW5sb2NrKCZlbmdpbmUtPnRpbWVsaW5lLT5sb2NrKTsKPj4+ ICt9Cj4+PiArCj4+PiBzdGF0aWMgdm9pZCBleGVjbGlzdHNfc2NoZWR1bGUoc3RydWN0IGRybV9p OTE1X2dlbV9yZXF1ZXN0ICpyZXF1ZXN0LCBpbnQgcHJpbykKPj4+IHsKPj4+IC0Jc3RydWN0IGlu dGVsX2VuZ2luZV9jcyAqZW5naW5lID0gTlVMTDsKPj4+ICsJc3RydWN0IGludGVsX2VuZ2luZV9j cyAqZW5naW5lOwo+Pj4gCXN0cnVjdCBpOTE1X2RlcGVuZGVuY3kgKmRlcCwgKnA7Cj4+PiAJc3Ry dWN0IGk5MTVfZGVwZW5kZW5jeSBzdGFjazsKPj4+ICsJdW5zaWduZWQgbG9uZyBsb2NrZWQgPSAw Owo+Pj4gCUxJU1RfSEVBRChkZnMpOwo+Pj4KPj4+ICsJQlVJTERfQlVHX09OKEk5MTVfTlVNX0VO R0lORVMgPiBCSVRTX1BFUl9MT05HKTsKPj4+ICsKPj4+IAlpZiAocHJpbyA8PSBSRUFEX09OQ0Uo cmVxdWVzdC0+cHJpb3RyZWUucHJpb3JpdHkpKQo+Pj4gCQlyZXR1cm47Cj4+Pgo+Pj4gQEAgLTY5 MSw2ICs3MDgsOSBAQCBzdGF0aWMgdm9pZCBleGVjbGlzdHNfc2NoZWR1bGUoc3RydWN0IGRybV9p OTE1X2dlbV9yZXF1ZXN0ICpyZXF1ZXN0LCBpbnQgcHJpbykKPj4+IAlzdGFjay5zaWduYWxlciA9 ICZyZXF1ZXN0LT5wcmlvdHJlZTsKPj4+IAlsaXN0X2FkZCgmc3RhY2suZGZzX2xpbmssICZkZnMp Owo+Pj4KPj4+ICsJR0VNX0JVR19PTihpcnFzX2Rpc2FibGVkKCkpOwo+Pj4gKwlsb2NhbF9pcnFf ZGlzYWJsZSgpOwo+Pj4gKwo+Pgo+PiBXaHkgbm90IGp1c3QgaXJxc2F2ZS9yZXN0b3JlPyBTb3Vu ZHMgbGlrZSB0b28gbG93IGxldmVsIGZvciB0aGlzCj4+IHBvc2l0aW9uIGluIHRoZSBmbG93LiBJ ZiBqdXN0IG9wdGltaXNhdGlvbiBpdCB3b3VsZCBuZWVkIGEgY29tbWVudCBJCj4+IHRoaW5rLgo+ Cj4gSXQgd2FzIGJlY2F1c2Ugd2UgYXJlIG5vdCB0YWtpbmcgdGhlIHNwaW4gbG9jay91bmxvY2sg aW5zaWRlIHRoZSBzYW1lCj4gYmxvY2ssIHNvIGl0IGZlbHQgZGFuZ2Vyb3VzLiBXaG8gaG9sZHMg dGhlIGlycWZsYWdzPwoKSG0geWVzLCBpdCBjYW5ub3QgYmUgbWFkZSBlbGVnYW50LgoKPj4+IAkv KiBSZWN1cnNpdmVseSBidW1wIGFsbCBkZXBlbmRlbnQgcHJpb3JpdGllcyB0byBtYXRjaCB0aGUg bmV3IHJlcXVlc3QuCj4+PiAJICoKPj4+IAkgKiBBIG5haXZlIGFwcHJvYWNoIHdvdWxkIGJlIHRv IHVzZSByZWN1cnNpb246Cj4+PiBAQCAtNzE5LDcgKzczOSw3IEBAIHN0YXRpYyB2b2lkIGV4ZWNs aXN0c19zY2hlZHVsZShzdHJ1Y3QgZHJtX2k5MTVfZ2VtX3JlcXVlc3QgKnJlcXVlc3QsIGludCBw cmlvKQo+Pj4gCQlpZiAoIVJCX0VNUFRZX05PREUoJnB0LT5ub2RlKSkKPj4+IAkJCWNvbnRpbnVl Owo+Pj4KPj4+IC0JCWVuZ2luZSA9IHB0X2xvY2tfZW5naW5lKHB0LCBlbmdpbmUpOwo+Pj4gKwkJ ZW5naW5lID0gcHRfbG9ja19lbmdpbmUocHQsICZsb2NrZWQpOwo+Pj4KPj4+IAkJLyogSWYgaXQg aXMgbm90IGFscmVhZHkgaW4gdGhlIHJidHJlZSwgd2UgY2FuIHVwZGF0ZSB0aGUKPj4+IAkJICog cHJpb3JpdHkgaW5wbGFjZSBhbmQgc2tpcCBvdmVyIGl0IChhbmQgaXRzIGRlcGVuZGVuY2llcykK Pj4+IEBAIC03MzcsNyArNzU3LDcgQEAgc3RhdGljIHZvaWQgZXhlY2xpc3RzX3NjaGVkdWxlKHN0 cnVjdCBkcm1faTkxNV9nZW1fcmVxdWVzdCAqcmVxdWVzdCwgaW50IHByaW8pCj4+Pgo+Pj4gCQlJ TklUX0xJU1RfSEVBRCgmZGVwLT5kZnNfbGluayk7Cj4+Pgo+Pj4gLQkJZW5naW5lID0gcHRfbG9j a19lbmdpbmUocHQsIGVuZ2luZSk7Cj4+PiArCQllbmdpbmUgPSBwdF9sb2NrX2VuZ2luZShwdCwg JmxvY2tlZCk7Cj4+Pgo+Pj4gCQlpZiAocHJpbyA8PSBwdC0+cHJpb3JpdHkpCj4+PiAJCQljb250 aW51ZTsKPj4+IEBAIC03NTAsOCArNzcwLDggQEAgc3RhdGljIHZvaWQgZXhlY2xpc3RzX3NjaGVk dWxlKHN0cnVjdCBkcm1faTkxNV9nZW1fcmVxdWVzdCAqcmVxdWVzdCwgaW50IHByaW8pCj4+PiAJ CQllbmdpbmUtPmV4ZWNsaXN0X2ZpcnN0ID0gJnB0LT5ub2RlOwo+Pj4gCX0KPj4+Cj4+PiAtCWlm IChlbmdpbmUpCj4+PiAtCQlzcGluX3VubG9ja19pcnEoJmVuZ2luZS0+dGltZWxpbmUtPmxvY2sp Owo+Pj4gKwl1bmxvY2tfZW5naW5lcyhyZXF1ZXN0LT5pOTE1LCBsb2NrZWQpOwo+Pj4gKwlsb2Nh bF9pcnFfZW5hYmxlKCk7Cj4+Pgo+Pj4gCS8qIFhYWCBEbyB3ZSBuZWVkIHRvIHByZWVtcHQgdG8g bWFrZSByb29tIGZvciB1cyBhbmQgb3VyIGRlcHM/ICovCj4+PiB9Cj4+Pgo+Pgo+PiBJIGFtIHRy eWluZyB0byB0aGluayB3aGV0aGVyIHJlbW92aW5nIHRoZSBza2lwIG9uIHJlcXVlc3RzIG5vdCBp bgo+PiB0aGUgZXhlY3V0aW9uIHRyZWUgd291bGQgd29yayBhbmQgaGVscCBhbnkuCj4KPiBJdCdz IGRhbmdlcm91cyBkdWUgdG8gdGhlIGR1cGxpY2F0ZSBicmFuY2hlcyBpbiB0aGUgZGVwZW5kZW5j eSBncmFwaCB0aGF0Cj4gd2UgYXJlIHJlc29sdmluZyB0byBnZW5lcmF0ZSB0aGUgdG9wb2xvZ2lj YWwgb3JkZXJpbmcuIFdlIG5lZWQgYSB3YXkgdG8KPiBkbyBhIG1hcmstYW5kLXN3ZWVwIHdoaWxz dCBhbHNvIGVuc3VyaW5nIHRoYXQgd2UgZW5kIHVwIHdpdGggdGhlIGNvcnJlY3QKPiBvcmRlci4g SSdtIG9wZW4gdG8gKGJldHRlciA6KSBzdWdnZXN0aW9ucy4KPgo+PiBPciBpZiB0aGUgYWJvdmUg c2NoZW1lCj4+IGlzIGNvbXBsZXRlbHkgc2FmZSBvciB3ZSB3b3VsZCBuZWVkIHRvIGxvY2sgYXRv bWljYWxseSBhbGwgZW5naW5lcwo+PiByZXF1ZXN0cyBvbiB3aGljaCB3aWxsIGJlIHRvdWNoZWQu IEVzcGVjaWFsbHkgc2luY2UgdGhlIGNvZGUgaXMgb25seQo+PiBkZWFsaW5nIHdpdGggYWRqdXN0 aW5nIHRoZSBwcmlvcml0aWVzIHNvIEkgZG9uJ3QgaW1tZWRpYXRlbHkgc2VlIGhvdwo+PiBpdCBj YW4gY2F1c2Ugb3V0IG9mIG9yZGVyIGV4ZWN1dGlvbi4KPgo+IGludGVycnVwdCBsZWFkaW5nIHRv IHN1Ym1pdF9yZXF1ZXN0LCB3aGljaCB3YW50cyB0byB0aGVuIGluc2VydCBhCj4gcmVxdWVzdCBp bnRvIHRoZSBleGVjbGlzdF9xdWV1ZSByYnRyZWUgdnMgLT5zY2hlZHVsZSgpIGFsc28gdHJ5aW5n IHRvCj4gbWFuaXB1bGF0ZSB0aGUgcmJ0cmVlIChhbmQgaW4gdGhpcyBjYXNlIGVsZW1lbnRzIGN1 cnJlbnRseSBvdXRzaWRlIG9mIHRoZQo+IHJidHJlZSkuIE91ciBpbnNlcnRpb24gaW50byB0aGUg cmJ0cmVlIGVuc3VyZXMgZmlmbyBzbyB0aGF0IHdlIGRvbid0Cj4gcmVvcmRlciB0aGUgZXF1aXZh bGVudCBwcmlvcml0eSBkZXBlbmRlbmNpZXMgZHVyaW5nIC0+c2NoZWR1bGUoKSwgaGVuY2UKPiBp ZiB3ZSBtYXJrIGFuIG91dC1vZi1yYnRyZWUgcmVxdWVzdCBhcyBhIGhpZ2hlciBwcmlvcml0eSBi ZWZvcmUKPiBpbnNlcnRpbmcgYWxsIG9mIGl0cyBkZXBlbmRlbmNpZXMgaW50byB0aGUgdHJlZSwg aWYgdGhlIHN1Ym1pdF9ub3RpZnkKPiBvY2N1cnMsIGl0IHdpbGwgaW5zZXJ0IHRoZSByZXF1ZXN0 IGludG8gdGhlIHRyZWUgYmVmb3JlIHdlIGdldCB0byBpbnNlcnQKPiBpdHMgZGVwZW5kZW5jaWVz LCBoZW5jZSByZW9yZGVyaW5nLgoKT2sgSSBnZXQgdGhlIGdlbmVyYWwgaWRlYS4gSSBkb24ndCBo YXZlIGFueSBiZXR0ZXIgc3VnZ2VzdGlvbnMgYXQgdGhlIAptb21lbnQgdGhhbiB0cnlpbmcgdGhl IGdsb2JhbCBsb2NrLiBMdWNraWx5IHlvdSBoYXZlIGp1c3QgcmVtb3ZlZCBvbmUgCmF0b21pYyBm cm9tIHRoZSBpcnEgaGFuZGxlciBzbyBvbmUgc3RlcCBmb3J3YXJkLCB0d28gc3RlcHMgYmFjay4g OikKClJlZ2FyZHMsCgpUdnJ0a28KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX18KSW50ZWwtZ2Z4IG1haWxpbmcgbGlzdApJbnRlbC1nZnhAbGlzdHMuZnJlZWRl c2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8v aW50ZWwtZ2Z4Cg==