From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5088EC07E9C for ; Fri, 9 Jul 2021 07:40:54 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 104BC61396 for ; Fri, 9 Jul 2021 07:40:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 104BC61396 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 23F766E9C7; Fri, 9 Jul 2021 07:40:53 +0000 (UTC) Received: from mail-oi1-x233.google.com (mail-oi1-x233.google.com [IPv6:2607:f8b0:4864:20::233]) by gabe.freedesktop.org (Postfix) with ESMTPS id AC83C6E9C7 for ; Fri, 9 Jul 2021 07:40:51 +0000 (UTC) Received: by mail-oi1-x233.google.com with SMTP id u11so11222959oiv.1 for ; Fri, 09 Jul 2021 00:40:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=8bhsFUO69iuV07cQJ2ssyZKt/x0LWfOEI39FZQi0ksY=; b=D8pReIZyfO8pTn+Pey0ICsRwKXfdp/h1AOulR3xe/b8/UA2fzA2ijirenNtLAPSRXp aTIs8vN3VULqkDApGE0/+oZDyLPUOjvMTbJSsgL4KOKBAOSf3C9jFggo5IBIN41pvdpH H8uY+1LhU2UchmPcEa440AMge0uqQEMYFArwg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=8bhsFUO69iuV07cQJ2ssyZKt/x0LWfOEI39FZQi0ksY=; b=h4yzTbIzuArtYoBVKwtiCB7Ub9SkpPac/7gnMX/xcWu7KQhAY2x1jbbsyKkVtSOx69 49tU5+C71DE8vcqdFeRkg28yV+qvNT8ZNs250UDmrceix3oSV0ZMsXVf1WGXGi9U8NHF 2yaEPqVGfEcaewLQojC8QnLvSH09oUHvOHMuguAZV2nAFy1qO6BsIkqsQfkkbTFN3i27 /SyJX8o5HuLhFsnizpkc7CiuWAuUlyOGcp7XsZRe2klk5sJ8AZ8PXd7fNIakk2BjYU42 EPGB9aE5Gr3FMill7Yh3TjaLQlPQ2E6ar6I/ZmfloSztk8lUKxYXJMGezfbKdBRQquH8 kuUg== X-Gm-Message-State: AOAM532zYNxkPmOef1MWJkhk+hLCueXQ5aPC1jWFJljPu46mSauYX/d7 ZjJosGNOZckn3olOqAPb9SX1c+AzBbab4Wt/OgXNBQ== X-Google-Smtp-Source: ABdhPJzJg+Hi4VukkWJZSO5MNKrkvmAUDLoAeabDpqwzzKeq0/uV9lDGQXbEvkn9yOwzZcDaqdpWH0iqu3ULnUUDDcY= X-Received: by 2002:aca:4b55:: with SMTP id y82mr2361204oia.14.1625816451045; Fri, 09 Jul 2021 00:40:51 -0700 (PDT) MIME-Version: 1.0 References: <20210708173754.3877540-4-daniel.vetter@ffwll.ch> <20210708215439.4093557-1-daniel.vetter@ffwll.ch> <14149638-6cc7-5281-c6b6-d6d08d13713f@amd.com> In-Reply-To: <14149638-6cc7-5281-c6b6-d6d08d13713f@amd.com> From: Daniel Vetter Date: Fri, 9 Jul 2021 09:40:39 +0200 Message-ID: Subject: Re: [PATCH] drm/sched: Barriers are needed for entity->last_scheduled To: =?UTF-8?Q?Christian_K=C3=B6nig?= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Intel Graphics Development , DRI Development , Steven Price , Boris Brezillon , Daniel Vetter , Lee Jones Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Fri, Jul 9, 2021 at 8:58 AM Christian K=C3=B6nig wrote: > > > > Am 08.07.21 um 23:54 schrieb Daniel Vetter: > > It might be good enough on x86 with just READ_ONCE, but the write side > > should then at least be WRITE_ONCE because x86 has total store order. > > > > It's definitely not enough on arm. > > > > Fix this proplery, which means > > - explain the need for the barrier in both places > > - point at the other side in each comment > > > > Also pull out the !sched_list case as the first check, so that the > > code flow is clearer. > > > > While at it sprinkle some comments around because it was very > > non-obvious to me what's actually going on here and why. > > > > Note that we really need full barriers here, at first I thought > > store-release and load-acquire on ->last_scheduled would be enough, > > but we actually requiring ordering between that and the queue state. > > > > v2: Put smp_rmp() in the right place and fix up comment (Andrey) > > > > Signed-off-by: Daniel Vetter > > Cc: "Christian K=C3=B6nig" > > Cc: Steven Price > > Cc: Daniel Vetter > > Cc: Andrey Grodzovsky > > Cc: Lee Jones > > Cc: Boris Brezillon > > --- > > drivers/gpu/drm/scheduler/sched_entity.c | 27 ++++++++++++++++++++++-= - > > 1 file changed, 25 insertions(+), 2 deletions(-) > > > > diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm= /scheduler/sched_entity.c > > index 64d398166644..6366006c0fcf 100644 > > --- a/drivers/gpu/drm/scheduler/sched_entity.c > > +++ b/drivers/gpu/drm/scheduler/sched_entity.c > > @@ -439,8 +439,16 @@ struct drm_sched_job *drm_sched_entity_pop_job(str= uct drm_sched_entity *entity) > > dma_fence_set_error(&sched_job->s_fence->finished, -ECANC= ELED); > > > > dma_fence_put(entity->last_scheduled); > > + > > entity->last_scheduled =3D dma_fence_get(&sched_job->s_fence->fin= ished); > > > > + /* > > + * If the queue is empty we allow drm_sched_entity_select_rq() to > > + * locklessly access ->last_scheduled. This only works if we set = the > > + * pointer before we dequeue and if we a write barrier here. > > + */ > > + smp_wmb(); > > + > > That whole stuff needs to be inside the spsc queue, not outside. > > Otherwise drm_sched_entity_is_idle() won't work either and cause a lot > of trouble during process tear down. Nah, that just means you need another 2 comments with their barrier to explain how things are serialized there against entity->stopped. The queue only needs to provide store-release and load-acquire barriers from a functional pov, if you assumie more then that's very strange. We need barriers in the other direction here (I haven't looked at what entity_is_idle) needs. This is why the first rule of lockless algorithms is "don't", and the second one is to very painstaikingly document every barrier necessary with - explanation what it synchronizes and why - and a pointer to where the other side of the barrier is in the code (there always has to be one, barrier on one side does nothing) -Daniel > Christian. > > > spsc_queue_pop(&entity->job_queue); > > return sched_job; > > } > > @@ -459,10 +467,25 @@ void drm_sched_entity_select_rq(struct drm_sched_= entity *entity) > > struct drm_gpu_scheduler *sched; > > struct drm_sched_rq *rq; > > > > - if (spsc_queue_count(&entity->job_queue) || !entity->sched_list) > > + /* single possible engine and already selected */ > > + if (!entity->sched_list) > > + return; > > + > > + /* queue non-empty, stay on the same engine */ > > + if (spsc_queue_count(&entity->job_queue)) > > return; > > > > - fence =3D READ_ONCE(entity->last_scheduled); > > + /* > > + * Only when the queue is empty are we guaranteed that the schedu= ler > > + * thread cannot change ->last_scheduled. To enforce ordering we = need > > + * a read barrier here. See drm_sched_entity_pop_job() for the ot= her > > + * side. > > + */ > > + smp_rmb(); > > + > > + fence =3D entity->last_scheduled; > > + > > + /* stay on the same engine if the previous job hasn't finished */ > > if (fence && !dma_fence_is_signaled(fence)) > > return; > > > --=20 Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E248C07E99 for ; Fri, 9 Jul 2021 07:40:58 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 622A161370 for ; Fri, 9 Jul 2021 07:40:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 622A161370 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 95D446E9C8; Fri, 9 Jul 2021 07:40:53 +0000 (UTC) Received: from mail-oi1-x22e.google.com (mail-oi1-x22e.google.com [IPv6:2607:f8b0:4864:20::22e]) by gabe.freedesktop.org (Postfix) with ESMTPS id BEC1C6E9C8 for ; Fri, 9 Jul 2021 07:40:51 +0000 (UTC) Received: by mail-oi1-x22e.google.com with SMTP id w127so11188112oig.12 for ; Fri, 09 Jul 2021 00:40:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=8bhsFUO69iuV07cQJ2ssyZKt/x0LWfOEI39FZQi0ksY=; b=D8pReIZyfO8pTn+Pey0ICsRwKXfdp/h1AOulR3xe/b8/UA2fzA2ijirenNtLAPSRXp aTIs8vN3VULqkDApGE0/+oZDyLPUOjvMTbJSsgL4KOKBAOSf3C9jFggo5IBIN41pvdpH H8uY+1LhU2UchmPcEa440AMge0uqQEMYFArwg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=8bhsFUO69iuV07cQJ2ssyZKt/x0LWfOEI39FZQi0ksY=; b=t4v95btU2OO+xoY5nwxV9NsQbjsAW9q8AZPsMTv6DUgUuQsurVeWK9CzhrmmINIPJR zWRlvNiKClDMTYeyhYTiJBleuKUormnd4cXkDxk89yWWt53WYeWqmO3RMjwVF51WKuWJ 2MwDZT3CIxgQEQ0TSA4/NVGUft/kW75koVLFMxGPictDyVC7TXpg0p4rxhh24EeqGvL2 m13rFneJBjOJXvcSvZTJFA8Gm+DhFcue/7aq/Qeq1XeP6YzU0iTdL3b/gsbmyf4F2TvT rSrh1NUXZD8lCScuVBq2E8unQGmTWJMt+Fd83K6HhbA9xnmo2n6DgyxAARBW35m1LjdM /7+g== X-Gm-Message-State: AOAM530Rj5xkyl7rlje0GpW7SNyzfcIg6xie700irn83Jpuf/6wk+qU+ bP+8iN4BCVMnYjG9W8eL7rrezpzgHGnix12EAkqKjw== X-Google-Smtp-Source: ABdhPJzJg+Hi4VukkWJZSO5MNKrkvmAUDLoAeabDpqwzzKeq0/uV9lDGQXbEvkn9yOwzZcDaqdpWH0iqu3ULnUUDDcY= X-Received: by 2002:aca:4b55:: with SMTP id y82mr2361204oia.14.1625816451045; Fri, 09 Jul 2021 00:40:51 -0700 (PDT) MIME-Version: 1.0 References: <20210708173754.3877540-4-daniel.vetter@ffwll.ch> <20210708215439.4093557-1-daniel.vetter@ffwll.ch> <14149638-6cc7-5281-c6b6-d6d08d13713f@amd.com> In-Reply-To: <14149638-6cc7-5281-c6b6-d6d08d13713f@amd.com> From: Daniel Vetter Date: Fri, 9 Jul 2021 09:40:39 +0200 Message-ID: To: =?UTF-8?Q?Christian_K=C3=B6nig?= Subject: Re: [Intel-gfx] [PATCH] drm/sched: Barriers are needed for entity->last_scheduled X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrey Grodzovsky , Intel Graphics Development , DRI Development , Steven Price , Daniel Vetter , Lee Jones Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" T24gRnJpLCBKdWwgOSwgMjAyMSBhdCA4OjU4IEFNIENocmlzdGlhbiBLw7ZuaWcgPGNocmlzdGlh bi5rb2VuaWdAYW1kLmNvbT4gd3JvdGU6Cj4KPgo+Cj4gQW0gMDguMDcuMjEgdW0gMjM6NTQgc2No cmllYiBEYW5pZWwgVmV0dGVyOgo+ID4gSXQgbWlnaHQgYmUgZ29vZCBlbm91Z2ggb24geDg2IHdp dGgganVzdCBSRUFEX09OQ0UsIGJ1dCB0aGUgd3JpdGUgc2lkZQo+ID4gc2hvdWxkIHRoZW4gYXQg bGVhc3QgYmUgV1JJVEVfT05DRSBiZWNhdXNlIHg4NiBoYXMgdG90YWwgc3RvcmUgb3JkZXIuCj4g Pgo+ID4gSXQncyBkZWZpbml0ZWx5IG5vdCBlbm91Z2ggb24gYXJtLgo+ID4KPiA+IEZpeCB0aGlz IHByb3BsZXJ5LCB3aGljaCBtZWFucwo+ID4gLSBleHBsYWluIHRoZSBuZWVkIGZvciB0aGUgYmFy cmllciBpbiBib3RoIHBsYWNlcwo+ID4gLSBwb2ludCBhdCB0aGUgb3RoZXIgc2lkZSBpbiBlYWNo IGNvbW1lbnQKPiA+Cj4gPiBBbHNvIHB1bGwgb3V0IHRoZSAhc2NoZWRfbGlzdCBjYXNlIGFzIHRo ZSBmaXJzdCBjaGVjaywgc28gdGhhdCB0aGUKPiA+IGNvZGUgZmxvdyBpcyBjbGVhcmVyLgo+ID4K PiA+IFdoaWxlIGF0IGl0IHNwcmlua2xlIHNvbWUgY29tbWVudHMgYXJvdW5kIGJlY2F1c2UgaXQg d2FzIHZlcnkKPiA+IG5vbi1vYnZpb3VzIHRvIG1lIHdoYXQncyBhY3R1YWxseSBnb2luZyBvbiBo ZXJlIGFuZCB3aHkuCj4gPgo+ID4gTm90ZSB0aGF0IHdlIHJlYWxseSBuZWVkIGZ1bGwgYmFycmll cnMgaGVyZSwgYXQgZmlyc3QgSSB0aG91Z2h0Cj4gPiBzdG9yZS1yZWxlYXNlIGFuZCBsb2FkLWFj cXVpcmUgb24gLT5sYXN0X3NjaGVkdWxlZCB3b3VsZCBiZSBlbm91Z2gsCj4gPiBidXQgd2UgYWN0 dWFsbHkgcmVxdWlyaW5nIG9yZGVyaW5nIGJldHdlZW4gdGhhdCBhbmQgdGhlIHF1ZXVlIHN0YXRl Lgo+ID4KPiA+IHYyOiBQdXQgc21wX3JtcCgpIGluIHRoZSByaWdodCBwbGFjZSBhbmQgZml4IHVw IGNvbW1lbnQgKEFuZHJleSkKPiA+Cj4gPiBTaWduZWQtb2ZmLWJ5OiBEYW5pZWwgVmV0dGVyIDxk YW5pZWwudmV0dGVyQGludGVsLmNvbT4KPiA+IENjOiAiQ2hyaXN0aWFuIEvDtm5pZyIgPGNocmlz dGlhbi5rb2VuaWdAYW1kLmNvbT4KPiA+IENjOiBTdGV2ZW4gUHJpY2UgPHN0ZXZlbi5wcmljZUBh cm0uY29tPgo+ID4gQ2M6IERhbmllbCBWZXR0ZXIgPGRhbmllbC52ZXR0ZXJAZmZ3bGwuY2g+Cj4g PiBDYzogQW5kcmV5IEdyb2R6b3Zza3kgPGFuZHJleS5ncm9kem92c2t5QGFtZC5jb20+Cj4gPiBD YzogTGVlIEpvbmVzIDxsZWUuam9uZXNAbGluYXJvLm9yZz4KPiA+IENjOiBCb3JpcyBCcmV6aWxs b24gPGJvcmlzLmJyZXppbGxvbkBjb2xsYWJvcmEuY29tPgo+ID4gLS0tCj4gPiAgIGRyaXZlcnMv Z3B1L2RybS9zY2hlZHVsZXIvc2NoZWRfZW50aXR5LmMgfCAyNyArKysrKysrKysrKysrKysrKysr KysrLS0KPiA+ICAgMSBmaWxlIGNoYW5nZWQsIDI1IGluc2VydGlvbnMoKyksIDIgZGVsZXRpb25z KC0pCj4gPgo+ID4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvZ3B1L2RybS9zY2hlZHVsZXIvc2NoZWRf ZW50aXR5LmMgYi9kcml2ZXJzL2dwdS9kcm0vc2NoZWR1bGVyL3NjaGVkX2VudGl0eS5jCj4gPiBp bmRleCA2NGQzOTgxNjY2NDQuLjYzNjYwMDZjMGZjZiAxMDA2NDQKPiA+IC0tLSBhL2RyaXZlcnMv Z3B1L2RybS9zY2hlZHVsZXIvc2NoZWRfZW50aXR5LmMKPiA+ICsrKyBiL2RyaXZlcnMvZ3B1L2Ry bS9zY2hlZHVsZXIvc2NoZWRfZW50aXR5LmMKPiA+IEBAIC00MzksOCArNDM5LDE2IEBAIHN0cnVj dCBkcm1fc2NoZWRfam9iICpkcm1fc2NoZWRfZW50aXR5X3BvcF9qb2Ioc3RydWN0IGRybV9zY2hl ZF9lbnRpdHkgKmVudGl0eSkKPiA+ICAgICAgICAgICAgICAgZG1hX2ZlbmNlX3NldF9lcnJvcigm c2NoZWRfam9iLT5zX2ZlbmNlLT5maW5pc2hlZCwgLUVDQU5DRUxFRCk7Cj4gPgo+ID4gICAgICAg ZG1hX2ZlbmNlX3B1dChlbnRpdHktPmxhc3Rfc2NoZWR1bGVkKTsKPiA+ICsKPiA+ICAgICAgIGVu dGl0eS0+bGFzdF9zY2hlZHVsZWQgPSBkbWFfZmVuY2VfZ2V0KCZzY2hlZF9qb2ItPnNfZmVuY2Ut PmZpbmlzaGVkKTsKPiA+Cj4gPiArICAgICAvKgo+ID4gKyAgICAgICogSWYgdGhlIHF1ZXVlIGlz IGVtcHR5IHdlIGFsbG93IGRybV9zY2hlZF9lbnRpdHlfc2VsZWN0X3JxKCkgdG8KPiA+ICsgICAg ICAqIGxvY2tsZXNzbHkgYWNjZXNzIC0+bGFzdF9zY2hlZHVsZWQuIFRoaXMgb25seSB3b3JrcyBp ZiB3ZSBzZXQgdGhlCj4gPiArICAgICAgKiBwb2ludGVyIGJlZm9yZSB3ZSBkZXF1ZXVlIGFuZCBp ZiB3ZSBhIHdyaXRlIGJhcnJpZXIgaGVyZS4KPiA+ICsgICAgICAqLwo+ID4gKyAgICAgc21wX3dt YigpOwo+ID4gKwo+Cj4gVGhhdCB3aG9sZSBzdHVmZiBuZWVkcyB0byBiZSBpbnNpZGUgdGhlIHNw c2MgcXVldWUsIG5vdCBvdXRzaWRlLgo+Cj4gT3RoZXJ3aXNlIGRybV9zY2hlZF9lbnRpdHlfaXNf aWRsZSgpIHdvbid0IHdvcmsgZWl0aGVyIGFuZCBjYXVzZSBhIGxvdAo+IG9mIHRyb3VibGUgZHVy aW5nIHByb2Nlc3MgdGVhciBkb3duLgoKTmFoLCB0aGF0IGp1c3QgbWVhbnMgeW91IG5lZWQgYW5v dGhlciAyIGNvbW1lbnRzIHdpdGggdGhlaXIgYmFycmllciB0bwpleHBsYWluIGhvdyB0aGluZ3Mg YXJlIHNlcmlhbGl6ZWQgdGhlcmUgYWdhaW5zdCBlbnRpdHktPnN0b3BwZWQuIFRoZQpxdWV1ZSBv bmx5IG5lZWRzIHRvIHByb3ZpZGUgc3RvcmUtcmVsZWFzZSBhbmQgbG9hZC1hY3F1aXJlIGJhcnJp ZXJzCmZyb20gYSBmdW5jdGlvbmFsIHBvdiwgaWYgeW91IGFzc3VtaWUgbW9yZSB0aGVuIHRoYXQn cyB2ZXJ5IHN0cmFuZ2UuCldlIG5lZWQgYmFycmllcnMgaW4gdGhlIG90aGVyIGRpcmVjdGlvbiBo ZXJlIChJIGhhdmVuJ3QgbG9va2VkIGF0IHdoYXQKZW50aXR5X2lzX2lkbGUpIG5lZWRzLgoKVGhp cyBpcyB3aHkgdGhlIGZpcnN0IHJ1bGUgb2YgbG9ja2xlc3MgYWxnb3JpdGhtcyBpcyAiZG9uJ3Qi LCBhbmQgdGhlCnNlY29uZCBvbmUgaXMgdG8gdmVyeSBwYWluc3RhaWtpbmdseSBkb2N1bWVudCBl dmVyeSBiYXJyaWVyIG5lY2Vzc2FyeQp3aXRoCi0gZXhwbGFuYXRpb24gd2hhdCBpdCBzeW5jaHJv bml6ZXMgYW5kIHdoeQotIGFuZCBhIHBvaW50ZXIgdG8gd2hlcmUgdGhlIG90aGVyIHNpZGUgb2Yg dGhlIGJhcnJpZXIgaXMgaW4gdGhlIGNvZGUKKHRoZXJlIGFsd2F5cyBoYXMgdG8gYmUgb25lLCBi YXJyaWVyIG9uIG9uZSBzaWRlIGRvZXMgbm90aGluZykKLURhbmllbAoKPiBDaHJpc3RpYW4uCj4K PiA+ICAgICAgIHNwc2NfcXVldWVfcG9wKCZlbnRpdHktPmpvYl9xdWV1ZSk7Cj4gPiAgICAgICBy ZXR1cm4gc2NoZWRfam9iOwo+ID4gICB9Cj4gPiBAQCAtNDU5LDEwICs0NjcsMjUgQEAgdm9pZCBk cm1fc2NoZWRfZW50aXR5X3NlbGVjdF9ycShzdHJ1Y3QgZHJtX3NjaGVkX2VudGl0eSAqZW50aXR5 KQo+ID4gICAgICAgc3RydWN0IGRybV9ncHVfc2NoZWR1bGVyICpzY2hlZDsKPiA+ICAgICAgIHN0 cnVjdCBkcm1fc2NoZWRfcnEgKnJxOwo+ID4KPiA+IC0gICAgIGlmIChzcHNjX3F1ZXVlX2NvdW50 KCZlbnRpdHktPmpvYl9xdWV1ZSkgfHwgIWVudGl0eS0+c2NoZWRfbGlzdCkKPiA+ICsgICAgIC8q IHNpbmdsZSBwb3NzaWJsZSBlbmdpbmUgYW5kIGFscmVhZHkgc2VsZWN0ZWQgKi8KPiA+ICsgICAg IGlmICghZW50aXR5LT5zY2hlZF9saXN0KQo+ID4gKyAgICAgICAgICAgICByZXR1cm47Cj4gPiAr Cj4gPiArICAgICAvKiBxdWV1ZSBub24tZW1wdHksIHN0YXkgb24gdGhlIHNhbWUgZW5naW5lICov Cj4gPiArICAgICBpZiAoc3BzY19xdWV1ZV9jb3VudCgmZW50aXR5LT5qb2JfcXVldWUpKQo+ID4g ICAgICAgICAgICAgICByZXR1cm47Cj4gPgo+ID4gLSAgICAgZmVuY2UgPSBSRUFEX09OQ0UoZW50 aXR5LT5sYXN0X3NjaGVkdWxlZCk7Cj4gPiArICAgICAvKgo+ID4gKyAgICAgICogT25seSB3aGVu IHRoZSBxdWV1ZSBpcyBlbXB0eSBhcmUgd2UgZ3VhcmFudGVlZCB0aGF0IHRoZSBzY2hlZHVsZXIK PiA+ICsgICAgICAqIHRocmVhZCBjYW5ub3QgY2hhbmdlIC0+bGFzdF9zY2hlZHVsZWQuIFRvIGVu Zm9yY2Ugb3JkZXJpbmcgd2UgbmVlZAo+ID4gKyAgICAgICogYSByZWFkIGJhcnJpZXIgaGVyZS4g U2VlIGRybV9zY2hlZF9lbnRpdHlfcG9wX2pvYigpIGZvciB0aGUgb3RoZXIKPiA+ICsgICAgICAq IHNpZGUuCj4gPiArICAgICAgKi8KPiA+ICsgICAgIHNtcF9ybWIoKTsKPiA+ICsKPiA+ICsgICAg IGZlbmNlID0gZW50aXR5LT5sYXN0X3NjaGVkdWxlZDsKPiA+ICsKPiA+ICsgICAgIC8qIHN0YXkg b24gdGhlIHNhbWUgZW5naW5lIGlmIHRoZSBwcmV2aW91cyBqb2IgaGFzbid0IGZpbmlzaGVkICov Cj4gPiAgICAgICBpZiAoZmVuY2UgJiYgIWRtYV9mZW5jZV9pc19zaWduYWxlZChmZW5jZSkpCj4g PiAgICAgICAgICAgICAgIHJldHVybjsKPiA+Cj4KCgotLSAKRGFuaWVsIFZldHRlcgpTb2Z0d2Fy ZSBFbmdpbmVlciwgSW50ZWwgQ29ycG9yYXRpb24KaHR0cDovL2Jsb2cuZmZ3bGwuY2gKX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KSW50ZWwtZ2Z4IG1haWxp bmcgbGlzdApJbnRlbC1nZnhAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJl ZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vaW50ZWwtZ2Z4Cg==