From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C68BC433E0 for ; Fri, 19 Jun 2020 18:08:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F256121527 for ; Fri, 19 Jun 2020 18:08:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ucsc.edu header.i=@ucsc.edu header.b="ChZ+VoMR" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2389713AbgFSSIk (ORCPT ); Fri, 19 Jun 2020 14:08:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730338AbgFSSIj (ORCPT ); Fri, 19 Jun 2020 14:08:39 -0400 Received: from mail-oi1-x241.google.com (mail-oi1-x241.google.com [IPv6:2607:f8b0:4864:20::241]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2205CC06174E for ; Fri, 19 Jun 2020 11:08:38 -0700 (PDT) Received: by mail-oi1-x241.google.com with SMTP id a3so9235250oid.4 for ; Fri, 19 Jun 2020 11:08:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google-2018; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=D3DT/dvyeplutPzx9MyRCxvTfOCv1fH0L/e2zdrU1Ts=; b=ChZ+VoMRW2BGAarmRuTatpQt+A2gEkWMBVqgXJQbdxvS/j/qrK6g25/SiVVsKmXji/ +bMSnPiyyTYYOQ0ggSaV7+oDGigp9Ml1dXNICrLE9+rx+oh4USC6aACXJ/Sv7u2eCFFh kZclDbM2oLieqMEwbtoW7QLCkhhN40mapJV4kGCuWSZhfjXBCAwUaZXh3ubbmooiB+yQ GVCuUfEd7ZhVrf3WQd5s+QlsNsLZhMf5kHugGn/67c9r7Xza2Wk9L3FEiEwy/our4Pqe 9cl83b50tBF/HBiba/3ldJLV/8qqrq4EZWio+jjwLwaePR3d9UEi21HZDd2DzsbDsPDF teog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=D3DT/dvyeplutPzx9MyRCxvTfOCv1fH0L/e2zdrU1Ts=; b=UpumzrLUxm2uS0CQ8FPDf1kuXvEk4wl8mkCabHsqN40A2/3UjcBhstR6RHy94Nk2sA kkm75jHAxiFmdYW1UcRIiVZEl2RPar8kZ8iAf7lcIPPWIx11VNw5p6LJpbf8opnDQLw9 6U/l1toYBQHigQRysqPcQ12VewuRV+X1SOdwuMuXtpiA5b5e7JOy/gO2VLEnEaI7Brk7 EzhmYYS4tkgqk/OwCaB5y/1AXon1PX/77OGpD+1DnG5BVvKVqSB0+OJhsKMCkLEbbhah u/13wOyaNF2xDA9NPF5114A1+fB6BQ0YUG4FRqQssP1VFSfN4TQ6cXLL7TKZigqnPiIV Du8Q== X-Gm-Message-State: AOAM531hEwDI/2qFRA9vtXrLCH+nkQyeMqI/askhv74803hFjE2Wi/Lg 28RL7YY9hwmktAgG6rsc5jJa1Z+b6jKx3mGdDPOlsw== X-Google-Smtp-Source: ABdhPJyw4j4Q+v1uB8Pg2InuO2lrwXJPlBJJQTkw1GKIuJ75wkL37Vt92Kh2YqsyukJqYxXP2jT4LnMzUhmplwQ2634= X-Received: by 2002:aca:d15:: with SMTP id 21mr3826252oin.41.1592590117336; Fri, 19 Jun 2020 11:08:37 -0700 (PDT) MIME-Version: 1.0 References: <20200617182841.jnbxgshi7bawfzls@mpHalley.localdomain> <20200617190901.zpss2lsh6qsu5zuf@mpHalley.local> <1ab101ef-7b74-060f-c2bc-d4c36dec91f0@lightnvm.io> <20200617194013.3wlz2ajnb6iopd4k@mpHalley.local> <20200618015526.GA1138429@dhcp-10-100-145-180.wdl.wdc.com> <20200618211945.GA2347@C02WT3WMHTD6> In-Reply-To: From: Heiner Litz Date: Fri, 19 Jun 2020 11:08:26 -0700 Message-ID: Subject: Re: [PATCH 5/5] nvme: support for zoned namespaces To: Matias Bjorling Cc: Keith Busch , Damien Le Moal , =?UTF-8?Q?Javier_Gonz=C3=A1lez?= , =?UTF-8?Q?Matias_Bj=C3=B8rling?= , Christoph Hellwig , Keith Busch , "linux-nvme@lists.infradead.org" , "linux-block@vger.kernel.org" , Sagi Grimberg , Jens Axboe , Hans Holmberg , Dmitry Fomichev , Ajay Joshi , Aravind Ramesh , Niklas Cassel , Judy Brock Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Hi Matias, no, I am rather saying that the Linux kernel has a deficit or at least is not a good fit for ZNS because it cannot enforce in-order delivery. The requirement of sequential writes basically imposes this requirement. Append essentially a Linux specific fix on the ZNS level and that enforcing ordering would be a cleaner way to enable QD>1. On Fri, Jun 19, 2020 at 3:29 AM Matias Bjorling w= rote: > > > -----Original Message----- > > From: Heiner Litz > > Sent: Friday, 19 June 2020 00.05 > > To: Keith Busch > > Cc: Damien Le Moal ; Javier Gonz=C3=A1lez > > ; Matias Bj=C3=B8rling ; Matias Bjo= rling > > ; Christoph Hellwig ; Keith Busch > > ; linux-nvme@lists.infradead.org; linux- > > block@vger.kernel.org; Sagi Grimberg ; Jens Axboe > > ; Hans Holmberg ; Dmitry > > Fomichev ; Ajay Joshi ; > > Aravind Ramesh ; Niklas Cassel > > ; Judy Brock > > Subject: Re: [PATCH 5/5] nvme: support for zoned namespaces > > > > Matias, Keith, > > thanks, this all sounds good and it makes total sense to hide striping = from the > > user. > > > > In the end, the real problem really seems to be that ZNS effectively re= quires in- > > order IO delivery which the kernel cannot guarantee. I think fixing thi= s problem > > in the ZNS specification instead of in the communication substrate (ker= nel) is > > problematic, especially as out-of-order delivery absolutely has no bene= fit in the > > case of ZNS. > > But I guess this has been discussed before.. > > I'm a bit dense, by the above, is your conclusion that ZNS has a deficit/= feature, which OCSSD didn't already have? They both had the same requiremen= t that a chunk/zone must be written sequentially. It's the name of the game= when deploying NAND-based media, I am not sure how ZNS should be able to h= elp with this. The goal of ZNS is to align with the media (and OCSSD), whic= h makes writes required to be sequential, and one thereby gets a bunch of b= enefits. > > If there was an understanding that ZNS would allow one to write randomly,= I must probably disappoint. For random writes, typical implementations eit= her use a write-back scheme, that stores data in random write media first, = and then later write it out sequentially, or write a host-side FTL (with it= s usual overheads). From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77713C433E0 for ; Fri, 19 Jun 2020 18:08:49 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4B34920DD4 for ; Fri, 19 Jun 2020 18:08:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="emS0Ci21"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ucsc.edu header.i=@ucsc.edu header.b="ChZ+VoMR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4B34920DD4 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=ucsc.edu Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gFoE17oA8ptVy3e2406gNiAqubuF4yC/G/7/Ltc24Ns=; b=emS0Ci21Ta0gI0 9fFeD0bG4EpmbfLf/QccdUu4jPayfiGkpJSCSbRdURoy1xty7hwPgfnc6SOy1SxEGxTbgH1iF0i/w QwpO+VjM2EdNiS4Z0Cqp3LQYwStH5RTaBxFRJkbxdqZ1eEehreBuXj/+9+V0AfbkZ78LBi4e1nFiL GU717Qd3so0GGsMb+Ob5QyyHGAnSYp9vsw2tUM2bk5tzs/sKC1Y242mY4C0lzU5rOW6C3fgBabhjY 3MusTD+IlJ2nrfKmuFqRtf2efkfG84xGttMgBtTl4EbOAsJ/8bsalI+N4+ALIZvtheBkhNV+He4V5 eHd7b8Q9xAq1Xl5Zri6g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1jmLRT-0002Pc-9T; Fri, 19 Jun 2020 18:08:43 +0000 Received: from mail-oi1-x242.google.com ([2607:f8b0:4864:20::242]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1jmLRQ-0002OQ-Vl for linux-nvme@lists.infradead.org; Fri, 19 Jun 2020 18:08:42 +0000 Received: by mail-oi1-x242.google.com with SMTP id a21so9212475oic.8 for ; Fri, 19 Jun 2020 11:08:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google-2018; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=D3DT/dvyeplutPzx9MyRCxvTfOCv1fH0L/e2zdrU1Ts=; b=ChZ+VoMRW2BGAarmRuTatpQt+A2gEkWMBVqgXJQbdxvS/j/qrK6g25/SiVVsKmXji/ +bMSnPiyyTYYOQ0ggSaV7+oDGigp9Ml1dXNICrLE9+rx+oh4USC6aACXJ/Sv7u2eCFFh kZclDbM2oLieqMEwbtoW7QLCkhhN40mapJV4kGCuWSZhfjXBCAwUaZXh3ubbmooiB+yQ GVCuUfEd7ZhVrf3WQd5s+QlsNsLZhMf5kHugGn/67c9r7Xza2Wk9L3FEiEwy/our4Pqe 9cl83b50tBF/HBiba/3ldJLV/8qqrq4EZWio+jjwLwaePR3d9UEi21HZDd2DzsbDsPDF teog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=D3DT/dvyeplutPzx9MyRCxvTfOCv1fH0L/e2zdrU1Ts=; b=U8Cnk1ABhLMfWp15zg+yu8YO+rDqGiNumMPk3ZmIqEdIz9xyIjPMphq/xaQxmkM7zB pUCnE+mGGnWmKPOiCrO7CSNkS72q7mqcPihBNU25yul2nSapXfAPokL7LzN16axBPQZj 2RMJkJf0aVmV+s8Dg6Yx2zHfjaexKYtVTCdlSUxPBz4ASJnkmY8NeyqzUyYe0c3XqIgL GcoZRJShZdRLN9/ljka4V4d19HFU+iaG+tIX/7WAr2WYMEQ5ry1zDbrhSrU62gbUaIqj 0R+rpYIbiPUCg67RO1jFDscI8j4FVWHXNVDVCXVpR/h6TFvjc11HYLhPTekgop6uuzlK DWQw== X-Gm-Message-State: AOAM533lYxEtlTvLp++bkOxC0Q2F8QbaWl6vgN4i8ly6ecNzmystRZvE mQXFzIbeQAIAtw68lv0kSTD6PaM5dBa58smHSnIGpQ== X-Google-Smtp-Source: ABdhPJyw4j4Q+v1uB8Pg2InuO2lrwXJPlBJJQTkw1GKIuJ75wkL37Vt92Kh2YqsyukJqYxXP2jT4LnMzUhmplwQ2634= X-Received: by 2002:aca:d15:: with SMTP id 21mr3826252oin.41.1592590117336; Fri, 19 Jun 2020 11:08:37 -0700 (PDT) MIME-Version: 1.0 References: <20200617182841.jnbxgshi7bawfzls@mpHalley.localdomain> <20200617190901.zpss2lsh6qsu5zuf@mpHalley.local> <1ab101ef-7b74-060f-c2bc-d4c36dec91f0@lightnvm.io> <20200617194013.3wlz2ajnb6iopd4k@mpHalley.local> <20200618015526.GA1138429@dhcp-10-100-145-180.wdl.wdc.com> <20200618211945.GA2347@C02WT3WMHTD6> In-Reply-To: From: Heiner Litz Date: Fri, 19 Jun 2020 11:08:26 -0700 Message-ID: Subject: Re: [PATCH 5/5] nvme: support for zoned namespaces To: Matias Bjorling X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200619_110841_055569_57D05FD8 X-CRM114-Status: GOOD ( 17.90 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jens Axboe , Niklas Cassel , Damien Le Moal , Ajay Joshi , Sagi Grimberg , Keith Busch , Dmitry Fomichev , Aravind Ramesh , =?UTF-8?Q?Javier_Gonz=C3=A1lez?= , "linux-nvme@lists.infradead.org" , "linux-block@vger.kernel.org" , Hans Holmberg , Keith Busch , =?UTF-8?Q?Matias_Bj=C3=B8rling?= , Judy Brock , Christoph Hellwig Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org SGkgTWF0aWFzLApubywgSSBhbSByYXRoZXIgc2F5aW5nIHRoYXQgdGhlIExpbnV4IGtlcm5lbCBo YXMgYSBkZWZpY2l0IG9yIGF0IGxlYXN0CmlzIG5vdCBhIGdvb2QgZml0IGZvciBaTlMgYmVjYXVz ZSBpdCBjYW5ub3QgZW5mb3JjZSBpbi1vcmRlciBkZWxpdmVyeS4KVGhlIHJlcXVpcmVtZW50IG9m IHNlcXVlbnRpYWwgd3JpdGVzIGJhc2ljYWxseSBpbXBvc2VzIHRoaXMKcmVxdWlyZW1lbnQuIEFw cGVuZCBlc3NlbnRpYWxseSBhIExpbnV4IHNwZWNpZmljIGZpeCBvbiB0aGUgWk5TIGxldmVsCmFu ZCB0aGF0IGVuZm9yY2luZyBvcmRlcmluZyB3b3VsZCBiZSBhIGNsZWFuZXIgd2F5IHRvIGVuYWJs ZSBRRD4xLgoKT24gRnJpLCBKdW4gMTksIDIwMjAgYXQgMzoyOSBBTSBNYXRpYXMgQmpvcmxpbmcg PE1hdGlhcy5Cam9ybGluZ0B3ZGMuY29tPiB3cm90ZToKPgo+ID4gLS0tLS1PcmlnaW5hbCBNZXNz YWdlLS0tLS0KPiA+IEZyb206IEhlaW5lciBMaXR6IDxobGl0ekB1Y3NjLmVkdT4KPiA+IFNlbnQ6 IEZyaWRheSwgMTkgSnVuZSAyMDIwIDAwLjA1Cj4gPiBUbzogS2VpdGggQnVzY2ggPGtidXNjaEBr ZXJuZWwub3JnPgo+ID4gQ2M6IERhbWllbiBMZSBNb2FsIDxEYW1pZW4uTGVNb2FsQHdkYy5jb20+ OyBKYXZpZXIgR29uesOhbGV6Cj4gPiA8amF2aWVyQGphdmlnb24uY29tPjsgTWF0aWFzIEJqw7hy bGluZyA8bWJAbGlnaHRudm0uaW8+OyBNYXRpYXMgQmpvcmxpbmcKPiA+IDxNYXRpYXMuQmpvcmxp bmdAd2RjLmNvbT47IENocmlzdG9waCBIZWxsd2lnIDxoY2hAbHN0LmRlPjsgS2VpdGggQnVzY2gK PiA+IDxLZWl0aC5CdXNjaEB3ZGMuY29tPjsgbGludXgtbnZtZUBsaXN0cy5pbmZyYWRlYWQub3Jn OyBsaW51eC0KPiA+IGJsb2NrQHZnZXIua2VybmVsLm9yZzsgU2FnaSBHcmltYmVyZyA8c2FnaUBn cmltYmVyZy5tZT47IEplbnMgQXhib2UKPiA+IDxheGJvZUBrZXJuZWwuZGs+OyBIYW5zIEhvbG1i ZXJnIDxIYW5zLkhvbG1iZXJnQHdkYy5jb20+OyBEbWl0cnkKPiA+IEZvbWljaGV2IDxEbWl0cnku Rm9taWNoZXZAd2RjLmNvbT47IEFqYXkgSm9zaGkgPEFqYXkuSm9zaGlAd2RjLmNvbT47Cj4gPiBB cmF2aW5kIFJhbWVzaCA8QXJhdmluZC5SYW1lc2hAd2RjLmNvbT47IE5pa2xhcyBDYXNzZWwKPiA+ IDxOaWtsYXMuQ2Fzc2VsQHdkYy5jb20+OyBKdWR5IEJyb2NrIDxqdWR5LmJyb2NrQHNhbXN1bmcu Y29tPgo+ID4gU3ViamVjdDogUmU6IFtQQVRDSCA1LzVdIG52bWU6IHN1cHBvcnQgZm9yIHpvbmVk IG5hbWVzcGFjZXMKPiA+Cj4gPiBNYXRpYXMsIEtlaXRoLAo+ID4gdGhhbmtzLCB0aGlzIGFsbCBz b3VuZHMgZ29vZCBhbmQgaXQgbWFrZXMgdG90YWwgc2Vuc2UgdG8gaGlkZSBzdHJpcGluZyBmcm9t IHRoZQo+ID4gdXNlci4KPiA+Cj4gPiBJbiB0aGUgZW5kLCB0aGUgcmVhbCBwcm9ibGVtIHJlYWxs eSBzZWVtcyB0byBiZSB0aGF0IFpOUyBlZmZlY3RpdmVseSByZXF1aXJlcyBpbi0KPiA+IG9yZGVy IElPIGRlbGl2ZXJ5IHdoaWNoIHRoZSBrZXJuZWwgY2Fubm90IGd1YXJhbnRlZS4gSSB0aGluayBm aXhpbmcgdGhpcyBwcm9ibGVtCj4gPiBpbiB0aGUgWk5TIHNwZWNpZmljYXRpb24gaW5zdGVhZCBv ZiBpbiB0aGUgY29tbXVuaWNhdGlvbiBzdWJzdHJhdGUgKGtlcm5lbCkgaXMKPiA+IHByb2JsZW1h dGljLCBlc3BlY2lhbGx5IGFzIG91dC1vZi1vcmRlciBkZWxpdmVyeSBhYnNvbHV0ZWx5IGhhcyBu byBiZW5lZml0IGluIHRoZQo+ID4gY2FzZSBvZiBaTlMuCj4gPiBCdXQgSSBndWVzcyB0aGlzIGhh cyBiZWVuIGRpc2N1c3NlZCBiZWZvcmUuLgo+Cj4gSSdtIGEgYml0IGRlbnNlLCBieSB0aGUgYWJv dmUsIGlzIHlvdXIgY29uY2x1c2lvbiB0aGF0IFpOUyBoYXMgYSBkZWZpY2l0L2ZlYXR1cmUsIHdo aWNoIE9DU1NEIGRpZG4ndCBhbHJlYWR5IGhhdmU/IFRoZXkgYm90aCBoYWQgdGhlIHNhbWUgcmVx dWlyZW1lbnQgdGhhdCBhIGNodW5rL3pvbmUgbXVzdCBiZSB3cml0dGVuIHNlcXVlbnRpYWxseS4g SXQncyB0aGUgbmFtZSBvZiB0aGUgZ2FtZSB3aGVuIGRlcGxveWluZyBOQU5ELWJhc2VkIG1lZGlh LCBJIGFtIG5vdCBzdXJlIGhvdyBaTlMgc2hvdWxkIGJlIGFibGUgdG8gaGVscCB3aXRoIHRoaXMu IFRoZSBnb2FsIG9mIFpOUyBpcyB0byBhbGlnbiB3aXRoIHRoZSBtZWRpYSAoYW5kIE9DU1NEKSwg d2hpY2ggbWFrZXMgd3JpdGVzIHJlcXVpcmVkIHRvIGJlIHNlcXVlbnRpYWwsIGFuZCBvbmUgdGhl cmVieSBnZXRzIGEgYnVuY2ggb2YgYmVuZWZpdHMuCj4KPiBJZiB0aGVyZSB3YXMgYW4gdW5kZXJz dGFuZGluZyB0aGF0IFpOUyB3b3VsZCBhbGxvdyBvbmUgdG8gd3JpdGUgcmFuZG9tbHksIEkgbXVz dCBwcm9iYWJseSBkaXNhcHBvaW50LiBGb3IgcmFuZG9tIHdyaXRlcywgdHlwaWNhbCBpbXBsZW1l bnRhdGlvbnMgZWl0aGVyIHVzZSBhIHdyaXRlLWJhY2sgc2NoZW1lLCB0aGF0IHN0b3JlcyBkYXRh IGluIHJhbmRvbSB3cml0ZSBtZWRpYSBmaXJzdCwgYW5kIHRoZW4gbGF0ZXIgd3JpdGUgaXQgb3V0 IHNlcXVlbnRpYWxseSwgb3Igd3JpdGUgYSBob3N0LXNpZGUgRlRMICh3aXRoIGl0cyB1c3VhbCBv dmVyaGVhZHMpLgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X18KbGludXgtbnZtZSBtYWlsaW5nIGxpc3QKbGludXgtbnZtZUBsaXN0cy5pbmZyYWRlYWQub3Jn Cmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtbnZtZQo=