From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 362C5C07E96 for ; Thu, 8 Jul 2021 19:53:59 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 85FDA6162D for ; Thu, 8 Jul 2021 19:53:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 85FDA6162D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 18D416E990; Thu, 8 Jul 2021 19:53:58 +0000 (UTC) Received: from mail-oi1-x235.google.com (mail-oi1-x235.google.com [IPv6:2607:f8b0:4864:20::235]) by gabe.freedesktop.org (Postfix) with ESMTPS id B89E56E990 for ; Thu, 8 Jul 2021 19:53:57 +0000 (UTC) Received: by mail-oi1-x235.google.com with SMTP id j65so3112346oih.5 for ; Thu, 08 Jul 2021 12:53:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=6gVz732TeLzAh3u/w29NhT4HzKDPkmcXR6VsrHgctQw=; b=HM1pSqrII40fXe/t97xpI2ResegZ8KFIMT4k7/EVAY3AtMh6CpCINeKZhiKRLMGCcE Y1FeRygYFvIe+dMqjdzEO0qlyRiVC/wpY/yWYIxvT5777EWn1VwoQ75FgA3xpfgWfdr/ qrjPpfGggHTcTpJMhd0rDr28DVXaf7ywnRnxM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=6gVz732TeLzAh3u/w29NhT4HzKDPkmcXR6VsrHgctQw=; b=B+7ddCHWsN+UzfELudJ6Bx3U4oDPnRuY4Ti/fVumXufVmcQwQO/ZlFK2zx4v3sLjJ5 yKoqZVMZe+dgH2P4S8GQ8dZvkh4ctkdVpXCDiHZhgXKZVeFND2ipOTDCuM3O0k+PyDe4 Hxm0Kd2f+9A1zE6YRpWiJOdJ846XaRoZF5rGP4dPkOKKLD/aR+7EZdecmdG3E9OPHPJr m95VRlQxupXiuOlyvSoiIk3KAcIQ8Ym80vZtvvW1cdeKTBQLHT082oGmZhDhWh+XwmrQ YhMSe+JzNVKMech5YgKpsrZ7J7QeSIxv0tjyLQgtnLJfCh9S+6xwWzxx04+0tgRq9y98 2chg== X-Gm-Message-State: AOAM533B+87e958HSWN+VqM0aDE9+kdjYwP635OqO/sOlg25UR43Tgn0 kW9FkIfGTTWkK7BfFRd5ID7IjR3oc9ijSx2U+JvUqQ== X-Google-Smtp-Source: ABdhPJy7Hpe4TqC16iOiipZ9J62IOW7zBuQatN2fRr0jBxp0Ne4X62deA1glG6KaqWDKH8oTGiAwYYm3AkoihTtHFMM= X-Received: by 2002:aca:f491:: with SMTP id s139mr18925339oih.128.1625774036971; Thu, 08 Jul 2021 12:53:56 -0700 (PDT) MIME-Version: 1.0 References: <20210708173754.3877540-1-daniel.vetter@ffwll.ch> <20210708173754.3877540-4-daniel.vetter@ffwll.ch> <9cc93adc-0a93-a0d5-422b-a57fac9bd450@amd.com> In-Reply-To: <9cc93adc-0a93-a0d5-422b-a57fac9bd450@amd.com> From: Daniel Vetter Date: Thu, 8 Jul 2021 21:53:45 +0200 Message-ID: Subject: Re: [PATCH v3 03/20] drm/sched: Barriers are needed for entity->last_scheduled To: Andrey Grodzovsky Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Intel Graphics Development , DRI Development , Steven Price , Boris Brezillon , Daniel Vetter , Lee Jones , =?UTF-8?Q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Thu, Jul 8, 2021 at 8:56 PM Andrey Grodzovsky wrote: > On 2021-07-08 1:37 p.m., Daniel Vetter wrote: > > It might be good enough on x86 with just READ_ONCE, but the write side > > should then at least be WRITE_ONCE because x86 has total store order. > > > > It's definitely not enough on arm. > > > > Fix this proplery, which means > > - explain the need for the barrier in both places > > - point at the other side in each comment > > > > Also pull out the !sched_list case as the first check, so that the > > code flow is clearer. > > > > While at it sprinkle some comments around because it was very > > non-obvious to me what's actually going on here and why. > > > > Note that we really need full barriers here, at first I thought > > store-release and load-acquire on ->last_scheduled would be enough, > > but we actually requiring ordering between that and the queue state. > > > > Signed-off-by: Daniel Vetter > > Cc: "Christian K=C3=B6nig" > > Cc: Steven Price > > Cc: Daniel Vetter > > Cc: Andrey Grodzovsky > > Cc: Lee Jones > > Cc: Boris Brezillon > > --- > > drivers/gpu/drm/scheduler/sched_entity.c | 27 ++++++++++++++++++++++-= - > > 1 file changed, 25 insertions(+), 2 deletions(-) > > > > diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm= /scheduler/sched_entity.c > > index 64d398166644..4e1124ed80e0 100644 > > --- a/drivers/gpu/drm/scheduler/sched_entity.c > > +++ b/drivers/gpu/drm/scheduler/sched_entity.c > > @@ -439,8 +439,16 @@ struct drm_sched_job *drm_sched_entity_pop_job(str= uct drm_sched_entity *entity) > > dma_fence_set_error(&sched_job->s_fence->finished, -ECANC= ELED); > > > > dma_fence_put(entity->last_scheduled); > > + > > entity->last_scheduled =3D dma_fence_get(&sched_job->s_fence->fin= ished); > > > > + /* > > + * if the queue is empty we allow drm_sched_job_arm() to lockless= ly > > > Probably meant drm_sched_entity_select_rq here Which is called from drm_sched_job_arm but yes. I'll switch it around. > > + * access ->last_scheduled. This only works if we set the pointer= before > > + * we dequeue and if we a write barrier here. > > + */ > > + smp_wmb(); > > + > > spsc_queue_pop(&entity->job_queue); > > return sched_job; > > } > > @@ -459,10 +467,25 @@ void drm_sched_entity_select_rq(struct drm_sched_= entity *entity) > > struct drm_gpu_scheduler *sched; > > struct drm_sched_rq *rq; > > > > - if (spsc_queue_count(&entity->job_queue) || !entity->sched_list) > > + /* single possible engine and already selected */ > > + if (!entity->sched_list) > > + return; > > + > > + /* queue non-empty, stay on the same engine */ > > + if (spsc_queue_count(&entity->job_queue)) > > return; > > > Shouldn't smp_rmb be here in between ? Given the queue is empty we want t= o > be certain we are reading the most recent value of entity->last_scheduled Yeah I had a load_acquire barrier here earlier and then put the smp_rmb() on the wrong side. Will fix. > > Andrey > > > > > > > - fence =3D READ_ONCE(entity->last_scheduled); > > + fence =3D entity->last_scheduled; > > + > > + /* > > + * Only when the queue is empty are we guaranteed the the schedul= er > > + * thread cannot change ->last_scheduled. To enforce ordering we = need > > + * a read barrier here. See drm_sched_entity_pop_job() for the ot= her > > + * side. > > + */ > > + smp_rmb(); > > + > > + /* stay on the same engine if the previous job hasn't finished */ > > if (fence && !dma_fence_is_signaled(fence)) > > return; > > --=20 Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 355E2C07E96 for ; Thu, 8 Jul 2021 19:54:03 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E986A6162D for ; Thu, 8 Jul 2021 19:54:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E986A6162D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id A76DE6E992; Thu, 8 Jul 2021 19:53:58 +0000 (UTC) Received: from mail-oi1-x236.google.com (mail-oi1-x236.google.com [IPv6:2607:f8b0:4864:20::236]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9AED06E990 for ; Thu, 8 Jul 2021 19:53:57 +0000 (UTC) Received: by mail-oi1-x236.google.com with SMTP id s24so9286773oiw.2 for ; Thu, 08 Jul 2021 12:53:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=6gVz732TeLzAh3u/w29NhT4HzKDPkmcXR6VsrHgctQw=; b=HM1pSqrII40fXe/t97xpI2ResegZ8KFIMT4k7/EVAY3AtMh6CpCINeKZhiKRLMGCcE Y1FeRygYFvIe+dMqjdzEO0qlyRiVC/wpY/yWYIxvT5777EWn1VwoQ75FgA3xpfgWfdr/ qrjPpfGggHTcTpJMhd0rDr28DVXaf7ywnRnxM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=6gVz732TeLzAh3u/w29NhT4HzKDPkmcXR6VsrHgctQw=; b=X9VBPmtCN9MnbxY35mgzVBqxCLMczVGy4Cz4j22jWYe6PVXqyzITC4PANhi3M8Ccej YPeil8l7IGpak8kiJwVHBCg/5R0TODN/otHPwz3ODWvrIuOX1BA4LTLqu5RW70jXiCB5 OlA3brOG5r6dWA7tIGbWy3xmoqZw6GejI+XHjyOOkUwiD/U+UNLS3G1ldyoRkTyO4Mru BX2LN7a9O6sFAV/9vgiujBqhtDMULqO7BZaEag+qQ7SpvlVseOqlLHHuhwDEL68hOA2/ 3U9TqzZXb3T7UaB4KsPoKPhdfYJVnHOtY46mFdfhlf344mwC9GhE4c2abosGAx2LmKZG NU+g== X-Gm-Message-State: AOAM533xVbrAi1gFVJggwoDSlH0xguWPpevjaOOiyk9dpHQ0XE8fT9HK h0SYIrMt8JCUT8WnyXY6yxOfjlsdh0m4d3BFXpvHAQ== X-Google-Smtp-Source: ABdhPJy7Hpe4TqC16iOiipZ9J62IOW7zBuQatN2fRr0jBxp0Ne4X62deA1glG6KaqWDKH8oTGiAwYYm3AkoihTtHFMM= X-Received: by 2002:aca:f491:: with SMTP id s139mr18925339oih.128.1625774036971; Thu, 08 Jul 2021 12:53:56 -0700 (PDT) MIME-Version: 1.0 References: <20210708173754.3877540-1-daniel.vetter@ffwll.ch> <20210708173754.3877540-4-daniel.vetter@ffwll.ch> <9cc93adc-0a93-a0d5-422b-a57fac9bd450@amd.com> In-Reply-To: <9cc93adc-0a93-a0d5-422b-a57fac9bd450@amd.com> From: Daniel Vetter Date: Thu, 8 Jul 2021 21:53:45 +0200 Message-ID: To: Andrey Grodzovsky Subject: Re: [Intel-gfx] [PATCH v3 03/20] drm/sched: Barriers are needed for entity->last_scheduled X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Intel Graphics Development , DRI Development , Steven Price , Daniel Vetter , Lee Jones , =?UTF-8?Q?Christian_K=C3=B6nig?= Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" T24gVGh1LCBKdWwgOCwgMjAyMSBhdCA4OjU2IFBNIEFuZHJleSBHcm9kem92c2t5CjxhbmRyZXku Z3JvZHpvdnNreUBhbWQuY29tPiB3cm90ZToKPiBPbiAyMDIxLTA3LTA4IDE6MzcgcC5tLiwgRGFu aWVsIFZldHRlciB3cm90ZToKPiA+IEl0IG1pZ2h0IGJlIGdvb2QgZW5vdWdoIG9uIHg4NiB3aXRo IGp1c3QgUkVBRF9PTkNFLCBidXQgdGhlIHdyaXRlIHNpZGUKPiA+IHNob3VsZCB0aGVuIGF0IGxl YXN0IGJlIFdSSVRFX09OQ0UgYmVjYXVzZSB4ODYgaGFzIHRvdGFsIHN0b3JlIG9yZGVyLgo+ID4K PiA+IEl0J3MgZGVmaW5pdGVseSBub3QgZW5vdWdoIG9uIGFybS4KPiA+Cj4gPiBGaXggdGhpcyBw cm9wbGVyeSwgd2hpY2ggbWVhbnMKPiA+IC0gZXhwbGFpbiB0aGUgbmVlZCBmb3IgdGhlIGJhcnJp ZXIgaW4gYm90aCBwbGFjZXMKPiA+IC0gcG9pbnQgYXQgdGhlIG90aGVyIHNpZGUgaW4gZWFjaCBj b21tZW50Cj4gPgo+ID4gQWxzbyBwdWxsIG91dCB0aGUgIXNjaGVkX2xpc3QgY2FzZSBhcyB0aGUg Zmlyc3QgY2hlY2ssIHNvIHRoYXQgdGhlCj4gPiBjb2RlIGZsb3cgaXMgY2xlYXJlci4KPiA+Cj4g PiBXaGlsZSBhdCBpdCBzcHJpbmtsZSBzb21lIGNvbW1lbnRzIGFyb3VuZCBiZWNhdXNlIGl0IHdh cyB2ZXJ5Cj4gPiBub24tb2J2aW91cyB0byBtZSB3aGF0J3MgYWN0dWFsbHkgZ29pbmcgb24gaGVy ZSBhbmQgd2h5Lgo+ID4KPiA+IE5vdGUgdGhhdCB3ZSByZWFsbHkgbmVlZCBmdWxsIGJhcnJpZXJz IGhlcmUsIGF0IGZpcnN0IEkgdGhvdWdodAo+ID4gc3RvcmUtcmVsZWFzZSBhbmQgbG9hZC1hY3F1 aXJlIG9uIC0+bGFzdF9zY2hlZHVsZWQgd291bGQgYmUgZW5vdWdoLAo+ID4gYnV0IHdlIGFjdHVh bGx5IHJlcXVpcmluZyBvcmRlcmluZyBiZXR3ZWVuIHRoYXQgYW5kIHRoZSBxdWV1ZSBzdGF0ZS4K PiA+Cj4gPiBTaWduZWQtb2ZmLWJ5OiBEYW5pZWwgVmV0dGVyIDxkYW5pZWwudmV0dGVyQGludGVs LmNvbT4KPiA+IENjOiAiQ2hyaXN0aWFuIEvDtm5pZyIgPGNocmlzdGlhbi5rb2VuaWdAYW1kLmNv bT4KPiA+IENjOiBTdGV2ZW4gUHJpY2UgPHN0ZXZlbi5wcmljZUBhcm0uY29tPgo+ID4gQ2M6IERh bmllbCBWZXR0ZXIgPGRhbmllbC52ZXR0ZXJAZmZ3bGwuY2g+Cj4gPiBDYzogQW5kcmV5IEdyb2R6 b3Zza3kgPGFuZHJleS5ncm9kem92c2t5QGFtZC5jb20+Cj4gPiBDYzogTGVlIEpvbmVzIDxsZWUu am9uZXNAbGluYXJvLm9yZz4KPiA+IENjOiBCb3JpcyBCcmV6aWxsb24gPGJvcmlzLmJyZXppbGxv bkBjb2xsYWJvcmEuY29tPgo+ID4gLS0tCj4gPiAgIGRyaXZlcnMvZ3B1L2RybS9zY2hlZHVsZXIv c2NoZWRfZW50aXR5LmMgfCAyNyArKysrKysrKysrKysrKysrKysrKysrLS0KPiA+ICAgMSBmaWxl IGNoYW5nZWQsIDI1IGluc2VydGlvbnMoKyksIDIgZGVsZXRpb25zKC0pCj4gPgo+ID4gZGlmZiAt LWdpdCBhL2RyaXZlcnMvZ3B1L2RybS9zY2hlZHVsZXIvc2NoZWRfZW50aXR5LmMgYi9kcml2ZXJz L2dwdS9kcm0vc2NoZWR1bGVyL3NjaGVkX2VudGl0eS5jCj4gPiBpbmRleCA2NGQzOTgxNjY2NDQu LjRlMTEyNGVkODBlMCAxMDA2NDQKPiA+IC0tLSBhL2RyaXZlcnMvZ3B1L2RybS9zY2hlZHVsZXIv c2NoZWRfZW50aXR5LmMKPiA+ICsrKyBiL2RyaXZlcnMvZ3B1L2RybS9zY2hlZHVsZXIvc2NoZWRf ZW50aXR5LmMKPiA+IEBAIC00MzksOCArNDM5LDE2IEBAIHN0cnVjdCBkcm1fc2NoZWRfam9iICpk cm1fc2NoZWRfZW50aXR5X3BvcF9qb2Ioc3RydWN0IGRybV9zY2hlZF9lbnRpdHkgKmVudGl0eSkK PiA+ICAgICAgICAgICAgICAgZG1hX2ZlbmNlX3NldF9lcnJvcigmc2NoZWRfam9iLT5zX2ZlbmNl LT5maW5pc2hlZCwgLUVDQU5DRUxFRCk7Cj4gPgo+ID4gICAgICAgZG1hX2ZlbmNlX3B1dChlbnRp dHktPmxhc3Rfc2NoZWR1bGVkKTsKPiA+ICsKPiA+ICAgICAgIGVudGl0eS0+bGFzdF9zY2hlZHVs ZWQgPSBkbWFfZmVuY2VfZ2V0KCZzY2hlZF9qb2ItPnNfZmVuY2UtPmZpbmlzaGVkKTsKPiA+Cj4g PiArICAgICAvKgo+ID4gKyAgICAgICogaWYgdGhlIHF1ZXVlIGlzIGVtcHR5IHdlIGFsbG93IGRy bV9zY2hlZF9qb2JfYXJtKCkgdG8gbG9ja2xlc3NseQo+Cj4KPiBQcm9iYWJseSBtZWFudCBkcm1f c2NoZWRfZW50aXR5X3NlbGVjdF9ycSBoZXJlCgpXaGljaCBpcyBjYWxsZWQgZnJvbSBkcm1fc2No ZWRfam9iX2FybSBidXQgeWVzLiBJJ2xsIHN3aXRjaCBpdCBhcm91bmQuCgo+ID4gKyAgICAgICog YWNjZXNzIC0+bGFzdF9zY2hlZHVsZWQuIFRoaXMgb25seSB3b3JrcyBpZiB3ZSBzZXQgdGhlIHBv aW50ZXIgYmVmb3JlCj4gPiArICAgICAgKiB3ZSBkZXF1ZXVlIGFuZCBpZiB3ZSBhIHdyaXRlIGJh cnJpZXIgaGVyZS4KPiA+ICsgICAgICAqLwo+ID4gKyAgICAgc21wX3dtYigpOwo+ID4gKwo+ID4g ICAgICAgc3BzY19xdWV1ZV9wb3AoJmVudGl0eS0+am9iX3F1ZXVlKTsKPiA+ICAgICAgIHJldHVy biBzY2hlZF9qb2I7Cj4gPiAgIH0KPiA+IEBAIC00NTksMTAgKzQ2NywyNSBAQCB2b2lkIGRybV9z Y2hlZF9lbnRpdHlfc2VsZWN0X3JxKHN0cnVjdCBkcm1fc2NoZWRfZW50aXR5ICplbnRpdHkpCj4g PiAgICAgICBzdHJ1Y3QgZHJtX2dwdV9zY2hlZHVsZXIgKnNjaGVkOwo+ID4gICAgICAgc3RydWN0 IGRybV9zY2hlZF9ycSAqcnE7Cj4gPgo+ID4gLSAgICAgaWYgKHNwc2NfcXVldWVfY291bnQoJmVu dGl0eS0+am9iX3F1ZXVlKSB8fCAhZW50aXR5LT5zY2hlZF9saXN0KQo+ID4gKyAgICAgLyogc2lu Z2xlIHBvc3NpYmxlIGVuZ2luZSBhbmQgYWxyZWFkeSBzZWxlY3RlZCAqLwo+ID4gKyAgICAgaWYg KCFlbnRpdHktPnNjaGVkX2xpc3QpCj4gPiArICAgICAgICAgICAgIHJldHVybjsKPiA+ICsKPiA+ ICsgICAgIC8qIHF1ZXVlIG5vbi1lbXB0eSwgc3RheSBvbiB0aGUgc2FtZSBlbmdpbmUgKi8KPiA+ ICsgICAgIGlmIChzcHNjX3F1ZXVlX2NvdW50KCZlbnRpdHktPmpvYl9xdWV1ZSkpCj4gPiAgICAg ICAgICAgICAgIHJldHVybjsKPgo+Cj4gU2hvdWxkbid0IHNtcF9ybWIgYmUgaGVyZSBpbiBiZXR3 ZWVuID8gR2l2ZW4gdGhlIHF1ZXVlIGlzIGVtcHR5IHdlIHdhbnQgdG8KPiBiZSBjZXJ0YWluIHdl IGFyZSByZWFkaW5nIHRoZSBtb3N0IHJlY2VudCB2YWx1ZSBvZiBlbnRpdHktPmxhc3Rfc2NoZWR1 bGVkCgpZZWFoIEkgaGFkIGEgbG9hZF9hY3F1aXJlIGJhcnJpZXIgaGVyZSBlYXJsaWVyIGFuZCB0 aGVuIHB1dCB0aGUKc21wX3JtYigpIG9uIHRoZSB3cm9uZyBzaWRlLiBXaWxsIGZpeC4KPgo+IEFu ZHJleQo+Cj4KPgo+ID4KPiA+IC0gICAgIGZlbmNlID0gUkVBRF9PTkNFKGVudGl0eS0+bGFzdF9z Y2hlZHVsZWQpOwo+ID4gKyAgICAgZmVuY2UgPSBlbnRpdHktPmxhc3Rfc2NoZWR1bGVkOwo+ID4g Kwo+ID4gKyAgICAgLyoKPiA+ICsgICAgICAqIE9ubHkgd2hlbiB0aGUgcXVldWUgaXMgZW1wdHkg YXJlIHdlIGd1YXJhbnRlZWQgdGhlIHRoZSBzY2hlZHVsZXIKPiA+ICsgICAgICAqIHRocmVhZCBj YW5ub3QgY2hhbmdlIC0+bGFzdF9zY2hlZHVsZWQuIFRvIGVuZm9yY2Ugb3JkZXJpbmcgd2UgbmVl ZAo+ID4gKyAgICAgICogYSByZWFkIGJhcnJpZXIgaGVyZS4gU2VlIGRybV9zY2hlZF9lbnRpdHlf cG9wX2pvYigpIGZvciB0aGUgb3RoZXIKPiA+ICsgICAgICAqIHNpZGUuCj4gPiArICAgICAgKi8K PiA+ICsgICAgIHNtcF9ybWIoKTsKPiA+ICsKPiA+ICsgICAgIC8qIHN0YXkgb24gdGhlIHNhbWUg ZW5naW5lIGlmIHRoZSBwcmV2aW91cyBqb2IgaGFzbid0IGZpbmlzaGVkICovCj4gPiAgICAgICBp ZiAoZmVuY2UgJiYgIWRtYV9mZW5jZV9pc19zaWduYWxlZChmZW5jZSkpCj4gPiAgICAgICAgICAg ICAgIHJldHVybjsKPiA+CgoKCi0tIApEYW5pZWwgVmV0dGVyClNvZnR3YXJlIEVuZ2luZWVyLCBJ bnRlbCBDb3Jwb3JhdGlvbgpodHRwOi8vYmxvZy5mZndsbC5jaApfX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fXwpJbnRlbC1nZnggbWFpbGluZyBsaXN0CkludGVs LWdmeEBsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcv bWFpbG1hbi9saXN0aW5mby9pbnRlbC1nZngK