From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C23D9C388F7 for ; Tue, 3 Nov 2020 14:32:35 +0000 (UTC) Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3FE90223AB for ; Tue, 3 Nov 2020 14:32:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kinvolk.io header.i=@kinvolk.io header.b="jNSrYWNM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3FE90223AB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kinvolk.io Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=containers-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id E153D861C8; Tue, 3 Nov 2020 14:32:34 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hsnlbC8f8Gz3; Tue, 3 Nov 2020 14:32:33 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by fraxinus.osuosl.org (Postfix) with ESMTP id A586D861A3; Tue, 3 Nov 2020 14:32:33 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8274CC0889; Tue, 3 Nov 2020 14:32:32 +0000 (UTC) Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 6F31FC0051 for ; Tue, 3 Nov 2020 14:32:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 3ADC92051F for ; Tue, 3 Nov 2020 14:32:31 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tgC63CaIxkTT for ; Tue, 3 Nov 2020 14:32:29 +0000 (UTC) X-Greylist: delayed 00:21:40 by SQLgrey-1.7.6 Received: from mail-lj1-f195.google.com (mail-lj1-f195.google.com [209.85.208.195]) by silver.osuosl.org (Postfix) with ESMTPS id 0B095203E5 for ; Tue, 3 Nov 2020 14:32:28 +0000 (UTC) Received: by mail-lj1-f195.google.com with SMTP id m16so19275796ljo.6 for ; Tue, 03 Nov 2020 06:32:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kinvolk.io; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=y8WQjpT9rqVrgeh99s8m7L7lW+jgLfIjyGz2M8+krP0=; b=jNSrYWNMri4UzTfQTwvUoeI3d9PbxAntszBtbkOHWPbsmFNLXoekIJ8TcfDFJbJf7v 6+eYLKBEUBAXwADA9DJI10J7imuIzSuwLQhtXLJY1nc7fmB3dMavlYA1Xmy9wmU6aG0F m9nefJssTHLgeDNMLEKBtF2hV/CbFCCq5QOzc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=y8WQjpT9rqVrgeh99s8m7L7lW+jgLfIjyGz2M8+krP0=; b=YnZREH4wgGYkfpn/Y8q4uOLt2oEJNk4h2XBt5nv0Ie+Zvx3oHdGw27FuQGCUhPEYH9 7+xZuzXrqlHiXidvJObnxgqRc1vA/Dh4cUwZ3k4cJ+2Cq1jyLvmKyxvv2dwDX8ZImn8m 1UjLNiS1J8BXqFaYW9joedIuBql0p1Owvj76GUJ0D+mqzY5PjFlkqX7AT8d84au8ejXS Fgs26j3cw5MRy7G6WgZqvH27s7fjiVF9CYmEYUhQKPkW96RD8gXg+ODf8CXYHtvsbJhd eVYmTFQrjP8Fl5BG+0k2Spf5pzyU1231UdJ74MeVIZtIZHmWrlMnBE9pifx7QhZR0LLI mu1Q== X-Gm-Message-State: AOAM531YB5mRcDh++rSHZ9IdoO3VvZwqylWrxix+BrL5F976IlPd8y36 nKptZ9a9cQn+4uj65e+7s0UGfE4DGIQ+zxjr8odoBiG8zBskk3qQ X-Google-Smtp-Source: ABdhPJwp4QL8RwmQmcT2CoPcUplHSU9HiByLsXAEntwM01IPEN+3MiJMYkcJQ6Od81V70aDSuFwsTxILAnA/OMfjXrI= X-Received: by 2002:a05:6402:a57:: with SMTP id bt23mr10741907edb.62.1604412647178; Tue, 03 Nov 2020 06:10:47 -0800 (PST) MIME-Version: 1.0 References: <20201029003252.2128653-1-christian.brauner@ubuntu.com> <87pn51ghju.fsf@x220.int.ebiederm.org> <20201029155148.5odu4j2kt62ahcxq@yavin.dot.cyphar.com> <87361xdm4c.fsf@x220.int.ebiederm.org> In-Reply-To: <87361xdm4c.fsf@x220.int.ebiederm.org> From: Alban Crequy Date: Tue, 3 Nov 2020 15:10:35 +0100 Message-ID: Subject: Re: [PATCH 00/34] fs: idmapped mounts To: "Eric W. Biederman" Cc: Lennart Poettering , Mimi Zohar , David Howells , Andreas Dilger , Linux Containers , Tycho Andersen , Miklos Szeredi , smbarber@chromium.org, Christoph Hellwig , linux-ext4@vger.kernel.org, Mrunal Patel , Kees Cook , Arnd Bergmann , Jann Horn , selinux@vger.kernel.org, Josh Triplett , linux-fsdevel , Alexander Viro , Andy Lutomirski , OGAWA Hirofumi , Geoffrey Thomas , James Bottomley , John Johansen , Theodore Tso , Seth Forshee , Dmitry Kasatkin , Stephen Smalley , Jonathan Corbet , linux-unionfs@vger.kernel.org, LSM , linux-audit@redhat.com, linux-api@vger.kernel.org, Casey Schaufler , linux-integrity , Todd Kjos X-BeenThere: containers@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux Containers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: containers-bounces@lists.linux-foundation.org Sender: "Containers" T24gVGh1LCBPY3QgMjksIDIwMjAgYXQgNTozNyBQTSBFcmljIFcuIEJpZWRlcm1hbiA8ZWJpZWRl cm1AeG1pc3Npb24uY29tPiB3cm90ZToKPgo+IEFsZWtzYSBTYXJhaSA8Y3lwaGFyQGN5cGhhci5j b20+IHdyaXRlczoKPgo+ID4gT24gMjAyMC0xMC0yOSwgRXJpYyBXLiBCaWVkZXJtYW4gPGViaWVk ZXJtQHhtaXNzaW9uLmNvbT4gd3JvdGU6Cj4gPj4gQ2hyaXN0aWFuIEJyYXVuZXIgPGNocmlzdGlh bi5icmF1bmVyQHVidW50dS5jb20+IHdyaXRlczoKPiA+Pgo+ID4+ID4gSGV5IGV2ZXJ5b25lLAo+ ID4+ID4KPiA+PiA+IEkgdmFuaXNoZWQgZm9yIGEgbGl0dGxlIHdoaWxlIHRvIGZvY3VzIG9uIHRo aXMgd29yayBoZXJlIHNvIHNvcnJ5IGZvcgo+ID4+ID4gbm90IGJlaW5nIGF2YWlsYWJsZSBieSBt YWlsIGZvciBhIHdoaWxlLgo+ID4+ID4KPiA+PiA+IFNpbmNlIHF1aXRlIGEgbG9uZyB0aW1lIHdl IGhhdmUgaXNzdWVzIHdpdGggc2hhcmluZyBtb3VudHMgYmV0d2Vlbgo+ID4+ID4gbXVsdGlwbGUg dW5wcml2aWxlZ2VkIGNvbnRhaW5lcnMgd2l0aCBkaWZmZXJlbnQgaWQgbWFwcGluZ3MsIHNoYXJp bmcgYQo+ID4+ID4gcm9vdGZzIGJldHdlZW4gbXVsdGlwbGUgY29udGFpbmVycyB3aXRoIGRpZmZl cmVudCBpZCBtYXBwaW5ncywgYW5kIGFsc28KPiA+PiA+IHNoYXJpbmcgcmVndWxhciBkaXJlY3Rv cmllcyBhbmQgZmlsZXN5c3RlbXMgYmV0d2VlbiB1c2VycyB3aXRoIGRpZmZlcmVudAo+ID4+ID4g dWlkcyBhbmQgZ2lkcy4gVGhlIGxhdHRlciB1c2UtY2FzZXMgaGF2ZSBiZWNvbWUgZXZlbiBtb3Jl IGltcG9ydGFudCB3aXRoCj4gPj4gPiB0aGUgYXZhaWxhYmlsaXR5IGFuZCBhZG9wdGlvbiBvZiBz eXN0ZW1kLWhvbWVkIChjZi4gWzFdKSB0byBpbXBsZW1lbnQKPiA+PiA+IHBvcnRhYmxlIGhvbWUg ZGlyZWN0b3JpZXMuCj4gPj4KPiA+PiBDYW4geW91IHdhbGsgdXMgdGhyb3VnaCB0aGUgbW90aXZh dGluZyB1c2UgY2FzZT8KPiA+Pgo+ID4+IEFzIG9mIHRoaXMgeWVhcidzIExQQyBJIGhhZCB0aGUg ZGlzdGluY3QgaW1wcmVzc2lvbiB0aGF0IHRoZSBwcmltYXJ5IHVzZQo+ID4+IGNhc2UgZm9yIHN1 Y2ggYSBmZWF0dXJlIHdhcyBkdWUgdG8gdGhlIFJMSU1JVF9OUFJPQyBwcm9ibGVtIHdoZXJlIHR3 bwo+ID4+IGNvbnRhaW5lcnMgd2l0aCB0aGUgc2FtZSB1c2VycyBzdGlsbCB3YW50ZWQgZGlmZmVy ZW50IHVpZCBtYXBwaW5ncyB0bwo+ID4+IHRoZSBkaXNrIGJlY2F1c2UgdGhlIHVzZXJzIHdlcmUg Y29uZmxpY3Rpbmcgd2l0aCBlYWNoIG90aGVyIGJlY2F1c2Ugb2YKPiA+PiB0aGUgcGVyIHVzZXIg cmxpbWl0cy4KPiA+Pgo+ID4+IEZpeGluZyBybGltaXRzIGlzIHN0cmFpZ2h0IGZvcndhcmQgdG8g aW1wbGVtZW50LCBhbmQgZWFzaWVyIHRvIG1hbmFnZQo+ID4+IGZvciBpbXBsZW1lbnRhdGlvbnMg YW5kIGFkbWluaXN0cmF0b3JzLgo+ID4KPiA+IFRoaXMgaXMgc2VwYXJhdGUgdG8gdGhlIHF1ZXN0 aW9uIG9mICJpc29sYXRlZCB1c2VyIG5hbWVzcGFjZXMiIGFuZAo+ID4gbWFuYWdpbmcgZGlmZmVy ZW50IG1hcHBpbmdzIGJldHdlZW4gY29udGFpbmVycy4gVGhpcyBwYXRjaHNldCBpcyBzb2x2aW5n Cj4gPiB0aGUgc2FtZSBwcm9ibGVtIHRoYXQgc2hpZnRmcyBzb2x2ZWQgLS0gc2hhcmluZyBhIHNp bmdsZSBkaXJlY3RvcnkgdHJlZQo+ID4gYmV0d2VlbiBjb250YWluZXJzIHRoYXQgaGF2ZSBkaWZm ZXJlbnQgSUQgbWFwcGluZ3MuIHJsaW1pdHMgKG5vciBhbnkgb2YKPiA+IHRoZSBvdGhlciBwcm9w b3NhbHMgd2UgZGlzY3Vzc2VkIGF0IExQQykgd2lsbCBoZWxwIHdpdGggdGhpcyBwcm9ibGVtLgo+ Cj4gRmlyc3QgYW5kIGZvcmVtb3N0OiBBIHVpZCBzaGlmdCBvbiB3cml0ZSB0byBhIGZpbGVzeXN0 ZW0gaXMgYSBzZWN1cml0eQo+IGJ1ZyB3YWl0aW5nIHRvIGhhcHBlbi4gIFRoaXMgaXMgZXNwZWNp YWxseSBpbiB0aGUgY29udGV4dCBvZiBmYWNpbGl0aWVzCj4gbGlrZSBpb3VyaW5nLCB0aGF0IHBs YXkgdmVyeSBhZ3Jlc3NpdmUgZ2FtZXMgd2l0aCBob3cgcHJvY2VzcyBjb250ZXh0Cj4gbWFrZXMg aXQgdG8gIHN5c3RlbSBjYWxscy4KPgo+IFRoZSBvbmx5IHJlYXNvbiBjb250YWluZXJzIHdlcmUg bm90IGltbWVkaWF0ZWx5IGV4cGxvaXRhYmxlIHdoZW4gaW91cmluZwo+IHdhcyBpbnRyb2R1Y2Vk IGlzIGJlY2F1c2UgdGhlIG1lY2hhbmlzbXMgYXJlIGJ1aWx0IHNvIHRoYXQgZXZlbiBpZgo+IHNv bWV0aGluZyBlc2NhcGVzIGNvbnRhaW5tZW50IHRoZSBzZWN1cml0eSBwcm9wZXJ0aWVzIHN0aWxs IGFwcGx5Lgo+IENoYW5nZXMgdG8gdGhlIHVpZCB3aGVuIHdyaXRpbmcgdG8gdGhlIGZpbGVzeXN0 ZW0gZG9lcyBub3QgaGF2ZSB0aGF0Cj4gcHJvcGVydHkuICBUaGUgdGluaWVzdCBzbGlwIGluIGNv bnRhaW5tZW50IHdpbGwgYmUgYSBzZWN1cml0eSBpc3N1ZS4KPgo+IFRoaXMgaXMgbm90IGV2ZW4g dGhlIGxlYXN0IGJpdCB0aGVvcmV0aWNhbC4gIEkgaGF2ZSBzZWVtIHJlcG9ydHMgb2YgaG93Cj4g c2hpdGZzK292ZXJsYXlmcyBjcmVhdGVkIGEgc2l0dWF0aW9uIHdoZXJlIGFueW9uZSBjb3VsZCBy ZWFkCj4gL2V0Yy9zaGFkb3cuCj4KPiBJZiB5b3UgYXJlIGdvaW5nIHRvIHdyaXRlIHVzaW5nIHRo ZSBzYW1lIHVpZCB0byBkaXNrIGZyb20gZGlmZmVyZW50Cj4gY29udGFpbmVycyB0aGUgcXVlc3Rp b24gYmVjb21lcyB3aHkgY2FuJ3QgdGhvc2UgY29udGFpbmVycyBjb25maWd1cmUKPiB0aG9zZSB1 c2VycyB0byB1c2UgdGhlIHNhbWUga3VpZD8KPgo+IFdoYXQgZml4aW5nIHJsaW1pdHMgZG9lcyBp cyBpdCBmaXhlcyBvbmUgb2YgdGhlIHJlYXNvbnMgdGhhdCBkaWZmZXJlbnQKPiBjb250YWluZXJz IGNvdWxkIG5vdCBzaGFyZSB0aGUgc2FtZSBrdWlkIGZvciB1c2VycyB0aGF0IHdhbnQgdG8gd3Jp dGUgdG8KPiBkaXNrIHdpdGggdGhlIHNhbWUgdWlkLgo+Cj4KPiBJIGh1bWJseSBzdWdnZXN0IHRo YXQgaXQgd2lsbCBiZSBtb3JlIHNlY3VyZSwgYW5kIGVhc2llciB0byBtYWludGFpbiBmb3IKPiBi b3RoIGRldmVsb3BlcnMgYW5kIHVzZXJzIGlmIHdlIGZpeCB0aGUgcmVhc29ucyBwZW9wbGUgd2Fu dCBkaWZmZXJlbnQKPiBjb250YWluZXJzIHRvIGhhdmUgdGhlIHNhbWUgdXNlciBydW5uaW5nIHdp dGggZGlmZmVyZW50IGt1aWRzLgo+Cj4gSWYgbm90IHdoYXQgYXJlIHRoZSByZWFzb25zIHdlIGZ1 bmRhbWVudGFsbHkgbmVlZCB0aGUgc2FtZSBvbi1kaXNrIHVzZXIKPiB1c2luZyBtdWx0aXBsZSBr dWlkcyBpbiB0aGUga2VybmVsPwoKSSB3b3VsZCBsaWtlIHRvIHVzZSB0aGlzIHBhdGNoIHNldCBp biB0aGUgY29udGV4dCBvZiBLdWJlcm5ldGVzLiBJCmRlc2NyaWJlZCBteSB0d28gcG9zc2libGUg c2V0dXBzIGluCmh0dHBzOi8vd3d3LnNwaW5pY3MubmV0L2xpc3RzL2xpbnV4LWNvbnRhaW5lcnMv bXNnMzY1MzcuaHRtbDoKCjEuIEVhY2ggS3ViZXJuZXRlcyBwb2QgaGFzIGl0cyBvd24gdXNlcm5z IGJ1dCB3aXRoIHRoZSBzYW1lIHVzZXIgaWQgbWFwcGluZwoyLiBFYWNoIEt1YmVybmV0ZXMgcG9k IGhhcyBpdHMgb3duIHVzZXJucyB3aXRoIG5vbi1vdmVybGFwcGluZyB1c2VyIGlkCm1hcHBpbmcg KHByb3ZpZGluZyBhZGRpdGlvbmFsIGlzb2xhdGlvbiBiZXR3ZWVuIHBvZHMpCgpCdXQgZXZlbiBp biB0aGUgc2V0dXAgd2hlcmUgYWxsIHBvZHMgcnVuIHdpdGggdGhlIHNhbWUgaWQgbWFwcGluZ3Ms CnRoaXMgcGF0Y2ggc2V0IGlzIHN0aWxsIHVzZWZ1bCB0byBtZSBmb3IgMiByZWFzb25zOgoKMS4g VG8gYXZvaWQgdGhlIGV4cGVuc2l2ZSByZWN1cnNpdmUgY2hvd24gb2YgdGhlIHJvb3Rmcy4gV2Ug Y2Fubm90Cm5lY2Vzc2FyaWx5IGV4dHJhY3QgdGhlIHRhcmJhbGwgZGlyZWN0bHkgd2l0aCB0aGUg cmlnaHQgdWlkcyBiZWNhdXNlCndlIG1pZ2h0IHVzZSB0aGUgc2FtZSBjb250YWluZXIgaW1hZ2Ug Zm9yIHByaXZpbGVnZWQgY29udGFpbmVycyAod2l0aAp0aGUgaG9zdCB1c2VybnMpIGFuZCB1bnBy aXZpbGVnZWQgY29udGFpbmVycyAod2l0aCBhIG5ldyB1c2VybnMpLCBzbwp3ZSBoYXZlIGF0IGxl YXN0IDIg4oCcbWFwcGluZ3PigJ0gKHRha2luZyBtb3JlIHRpbWUgYW5kIHJlc3VsdGluZyBpbiBt b3JlCnN0b3JhZ2Ugc3BhY2UpLiBBbHRob3VnaCB0aGUg4oCcbWV0YWNvcHnigJ0gbW91bnQgb3B0 aW9uIGluIG92ZXJsYXlmcwpoZWxwcyB0byBtYWtlIHRoZSByZWN1cnNpdmUgY2hvd24gZmFzdGVy LCBpdCBjYW4gc3RpbGwgdGFrZSB0aW1lIHdpdGgKbGFyZ2UgY29udGFpbmVyIGltYWdlcyB3aXRo IGxvdHMgb2YgZmlsZXMuIEnigJlkIGxpa2UgdG8gdXNlIHRoaXMgcGF0Y2gKc2V0IHRvIHNldCB1 cCB0aGUgcm9vdCBmcyBpbiBjb25zdGFudCB0aW1lLgoKMi4gVG8gbWFuYWdlIGxhcmdlIGV4dGVy bmFsIHZvbHVtZXMgKE5GUyBvciBvdGhlciBmaWxlc3lzdGVtcykuIEV2ZW4KaWYgYWRtaW5zIGNh biBkZWNpZGUgdG8gdXNlIHRoZSBzYW1lIGt1aWQgb24gYWxsIHRoZSBub2RlcyBvZiB0aGUKS3Vi ZXJuZXRlcyBjbHVzdGVyLCB0aGlzIGlzIGltcHJhY3RpY2FsIGZvciBtaWdyYXRpb24uIFBlb3Bs ZSBjYW4gaGF2ZQpleGlzdGluZyBLdWJlcm5ldGVzIGNsdXN0ZXJzIChjdXJyZW50bHkgd2l0aG91 dCB1c2luZyB1c2VyIG5hbWVzcGFjZXMpCmFuZCBsYXJnZSBORlMgdm9sdW1lcy4gSWYgdGhleSB3 YW50IHRvIHN3aXRjaCB0byBhIG5ldyB2ZXJzaW9uIG9mCkt1YmVybmV0ZXMgd2l0aCB0aGUgdXNl ciBuYW1lc3BhY2UgZmVhdHVyZSBlbmFibGVkLCB0aGV5IHdvdWxkIG5lZWQgdG8KcmVjdXJzaXZl bHkgY2hvd24gYWxsIHRoZSBmaWxlcyBvbiB0aGUgTkZTIHNoYXJlcy4gVGhpcyBjb3VsZCB0YWtl CnRpbWUgb24gbGFyZ2UgZmlsZXN5c3RlbXMgYW5kIHJlYWxpc3RpY2FsbHksIHdlIHdhbnQgdG8g c3VwcG9ydApyb2xsaW5nIHVwZGF0ZXMgd2hlcmUgc29tZSBub2RlcyB1c2UgdGhlIHByZXZpb3Vz IHZlcnNpb24gd2l0aG91dCB1c2VyCm5hbWVzcGFjZXMgYW5kIG5ldyBub2RlcyBhcmUgcHJvZ3Jl c3NpdmVseSBtaWdyYXRlZCB0byB0aGUgbmV3IHVzZXJucwp3aXRoIHRoZSBuZXcgaWQgbWFwcGlu Zy4gSWYgYm90aCBzZXRzIG9mIG5vZGVzIHVzZSB0aGUgc2FtZSBORlMgc2hhcmUsCnRoYXQgY2Fu 4oCZdCB3b3JrLgoKQWxiYW4KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX18KQ29udGFpbmVycyBtYWlsaW5nIGxpc3QKQ29udGFpbmVyc0BsaXN0cy5saW51eC1m b3VuZGF0aW9uLm9yZwpodHRwczovL2xpc3RzLmxpbnV4Zm91bmRhdGlvbi5vcmcvbWFpbG1hbi9s aXN0aW5mby9jb250YWluZXJz From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31348C55179 for ; Tue, 3 Nov 2020 14:10:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C095422226 for ; Tue, 3 Nov 2020 14:10:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kinvolk.io header.i=@kinvolk.io header.b="jNSrYWNM" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729295AbgKCOKx (ORCPT ); Tue, 3 Nov 2020 09:10:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729561AbgKCOKt (ORCPT ); Tue, 3 Nov 2020 09:10:49 -0500 Received: from mail-ed1-x542.google.com (mail-ed1-x542.google.com [IPv6:2a00:1450:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9C1A6C061A4A for ; Tue, 3 Nov 2020 06:10:48 -0800 (PST) Received: by mail-ed1-x542.google.com with SMTP id k9so18418079edo.5 for ; Tue, 03 Nov 2020 06:10:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kinvolk.io; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=y8WQjpT9rqVrgeh99s8m7L7lW+jgLfIjyGz2M8+krP0=; b=jNSrYWNMri4UzTfQTwvUoeI3d9PbxAntszBtbkOHWPbsmFNLXoekIJ8TcfDFJbJf7v 6+eYLKBEUBAXwADA9DJI10J7imuIzSuwLQhtXLJY1nc7fmB3dMavlYA1Xmy9wmU6aG0F m9nefJssTHLgeDNMLEKBtF2hV/CbFCCq5QOzc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=y8WQjpT9rqVrgeh99s8m7L7lW+jgLfIjyGz2M8+krP0=; b=jogLaJKduvF+00PDLnr+13S4azMAWA3p7CkmFAhc2n1S/TJNTHWCBY7C3EJCU7eNmv 8yHbvCWKLs33dC10GEMkFggla5a00Q/GLjk562VPIOsNXNlwjuzZC2ZIsdP67Hx5KJ4m fby3w02p0d6elQQe7RDSlytwSVKuq72lL2cGgpCGZ5ZEiesn25MV096qtrc3cyy85uMd AZ7ea1XDkP1Og2DxFdFoolpbq3u8qV2yNfPMH1ZZN81Zmd2V0iJEwbIilIF9c1j6aOcR 7RtBiomxD2BF37kbp9VVK0vpn3eJryPufkWstACXOJop73AbxUcNSZojaPfXVILeXjc2 5RhQ== X-Gm-Message-State: AOAM530/VZstfhV9Wl33ybbjdnmDf2GcfpdS66WysBFPm/3PiGoZxZ3n pJmydQ0meA4pG68vSI1u1Ot/LmH5DfKhIt0KtjJ8gA== X-Google-Smtp-Source: ABdhPJwp4QL8RwmQmcT2CoPcUplHSU9HiByLsXAEntwM01IPEN+3MiJMYkcJQ6Od81V70aDSuFwsTxILAnA/OMfjXrI= X-Received: by 2002:a05:6402:a57:: with SMTP id bt23mr10741907edb.62.1604412647178; Tue, 03 Nov 2020 06:10:47 -0800 (PST) MIME-Version: 1.0 References: <20201029003252.2128653-1-christian.brauner@ubuntu.com> <87pn51ghju.fsf@x220.int.ebiederm.org> <20201029155148.5odu4j2kt62ahcxq@yavin.dot.cyphar.com> <87361xdm4c.fsf@x220.int.ebiederm.org> In-Reply-To: <87361xdm4c.fsf@x220.int.ebiederm.org> From: Alban Crequy Date: Tue, 3 Nov 2020 15:10:35 +0100 Message-ID: Subject: Re: [PATCH 00/34] fs: idmapped mounts To: "Eric W. Biederman" Cc: Aleksa Sarai , Christian Brauner , Alexander Viro , Christoph Hellwig , linux-fsdevel , John Johansen , James Morris , Mimi Zohar , Dmitry Kasatkin , Stephen Smalley , Casey Schaufler , Arnd Bergmann , Andreas Dilger , OGAWA Hirofumi , Geoffrey Thomas , Mrunal Patel , Josh Triplett , Andy Lutomirski , Amir Goldstein , Miklos Szeredi , Theodore Tso , Tycho Andersen , David Howells , James Bottomley , Jann Horn , Seth Forshee , =?UTF-8?Q?St=C3=A9phane_Graber?= , Lennart Poettering , smbarber@chromium.org, Phil Estes , Serge Hallyn , Kees Cook , Todd Kjos , Jonathan Corbet , Linux Containers , LSM , linux-api@vger.kernel.org, linux-ext4@vger.kernel.org, linux-unionfs@vger.kernel.org, linux-audit@redhat.com, linux-integrity , selinux@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-unionfs@vger.kernel.org On Thu, Oct 29, 2020 at 5:37 PM Eric W. Biederman w= rote: > > Aleksa Sarai writes: > > > On 2020-10-29, Eric W. Biederman wrote: > >> Christian Brauner writes: > >> > >> > Hey everyone, > >> > > >> > I vanished for a little while to focus on this work here so sorry fo= r > >> > not being available by mail for a while. > >> > > >> > Since quite a long time we have issues with sharing mounts between > >> > multiple unprivileged containers with different id mappings, sharing= a > >> > rootfs between multiple containers with different id mappings, and a= lso > >> > sharing regular directories and filesystems between users with diffe= rent > >> > uids and gids. The latter use-cases have become even more important = with > >> > the availability and adoption of systemd-homed (cf. [1]) to implemen= t > >> > portable home directories. > >> > >> Can you walk us through the motivating use case? > >> > >> As of this year's LPC I had the distinct impression that the primary u= se > >> case for such a feature was due to the RLIMIT_NPROC problem where two > >> containers with the same users still wanted different uid mappings to > >> the disk because the users were conflicting with each other because of > >> the per user rlimits. > >> > >> Fixing rlimits is straight forward to implement, and easier to manage > >> for implementations and administrators. > > > > This is separate to the question of "isolated user namespaces" and > > managing different mappings between containers. This patchset is solvin= g > > the same problem that shiftfs solved -- sharing a single directory tree > > between containers that have different ID mappings. rlimits (nor any of > > the other proposals we discussed at LPC) will help with this problem. > > First and foremost: A uid shift on write to a filesystem is a security > bug waiting to happen. This is especially in the context of facilities > like iouring, that play very agressive games with how process context > makes it to system calls. > > The only reason containers were not immediately exploitable when iouring > was introduced is because the mechanisms are built so that even if > something escapes containment the security properties still apply. > Changes to the uid when writing to the filesystem does not have that > property. The tiniest slip in containment will be a security issue. > > This is not even the least bit theoretical. I have seem reports of how > shitfs+overlayfs created a situation where anyone could read > /etc/shadow. > > If you are going to write using the same uid to disk from different > containers the question becomes why can't those containers configure > those users to use the same kuid? > > What fixing rlimits does is it fixes one of the reasons that different > containers could not share the same kuid for users that want to write to > disk with the same uid. > > > I humbly suggest that it will be more secure, and easier to maintain for > both developers and users if we fix the reasons people want different > containers to have the same user running with different kuids. > > If not what are the reasons we fundamentally need the same on-disk user > using multiple kuids in the kernel? I would like to use this patch set in the context of Kubernetes. I described my two possible setups in https://www.spinics.net/lists/linux-containers/msg36537.html: 1. Each Kubernetes pod has its own userns but with the same user id mapping 2. Each Kubernetes pod has its own userns with non-overlapping user id mapping (providing additional isolation between pods) But even in the setup where all pods run with the same id mappings, this patch set is still useful to me for 2 reasons: 1. To avoid the expensive recursive chown of the rootfs. We cannot necessarily extract the tarball directly with the right uids because we might use the same container image for privileged containers (with the host userns) and unprivileged containers (with a new userns), so we have at least 2 =E2=80=9Cmappings=E2=80=9D (taking more time and resulti= ng in more storage space). Although the =E2=80=9Cmetacopy=E2=80=9D mount option in ove= rlayfs helps to make the recursive chown faster, it can still take time with large container images with lots of files. I=E2=80=99d like to use this pat= ch set to set up the root fs in constant time. 2. To manage large external volumes (NFS or other filesystems). Even if admins can decide to use the same kuid on all the nodes of the Kubernetes cluster, this is impractical for migration. People can have existing Kubernetes clusters (currently without using user namespaces) and large NFS volumes. If they want to switch to a new version of Kubernetes with the user namespace feature enabled, they would need to recursively chown all the files on the NFS shares. This could take time on large filesystems and realistically, we want to support rolling updates where some nodes use the previous version without user namespaces and new nodes are progressively migrated to the new userns with the new id mapping. If both sets of nodes use the same NFS share, that can=E2=80=99t work. Alban From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7DC3DC2D0A3 for ; Mon, 9 Nov 2020 22:53:57 +0000 (UTC) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C0B0C206CB for ; Mon, 9 Nov 2020 22:53:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C0B0C206CB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kinvolk.io Authentication-Results: mail.kernel.org; spf=tempfail smtp.mailfrom=linux-audit-bounces@redhat.com Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-234-5uxibessO7-w1t1cvAiFxg-1; Mon, 09 Nov 2020 17:53:52 -0500 X-MC-Unique: 5uxibessO7-w1t1cvAiFxg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 81642186DD2C; Mon, 9 Nov 2020 22:53:49 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 31B393C04; Mon, 9 Nov 2020 22:53:49 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id B1BF1180B658; Mon, 9 Nov 2020 22:53:48 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id 0A3EAsn5027533 for ; Tue, 3 Nov 2020 09:10:55 -0500 Received: by smtp.corp.redhat.com (Postfix) id D1E7794646; Tue, 3 Nov 2020 14:10:54 +0000 (UTC) Received: from mimecast-mx02.redhat.com (mimecast02.extmail.prod.ext.rdu2.redhat.com [10.11.55.18]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CC5642EF97 for ; Tue, 3 Nov 2020 14:10:52 +0000 (UTC) Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A57C48007A4 for ; Tue, 3 Nov 2020 14:10:52 +0000 (UTC) Received: from mail-ed1-f67.google.com (mail-ed1-f67.google.com [209.85.208.67]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-492-k13TeLBqMfqrvn5VJYbTnw-1; Tue, 03 Nov 2020 09:10:48 -0500 X-MC-Unique: k13TeLBqMfqrvn5VJYbTnw-1 Received: by mail-ed1-f67.google.com with SMTP id b9so8183733edu.10 for ; Tue, 03 Nov 2020 06:10:48 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=y8WQjpT9rqVrgeh99s8m7L7lW+jgLfIjyGz2M8+krP0=; b=V4VQ9r1bV7pK2VYA6vb8+srl4/9f/dN/SYlHdgNz3ryTiKm2MZyuNxckw3aISR3sec k331F17GBZyw2buEcA1azDVxwyMu9FfNOr4iEBqRtFt8Cw5Y9cEaUjoo1GWWsHRSaRmp 2Kl69M65Dw5nJkkZhNDwhSiQmHSIP4VSqpN+Evfw6V2m1gbh1RcmBEsiAtu1rihRWkBY /PrneawWzj+s9axKx2+JiqK1NImiDMt5mCeVchJxA1Os065vxMcs3O0XxG2OA/OZfBF1 YW1wRWIFJ7ygNsTkTP29TAFx5rej9EDSzOLDNMsQIIoMesuTgo18o9LMPHkjdOaVbM6J 32ow== X-Gm-Message-State: AOAM530lZuA8Ca7giMT491Rk6OF70OauPa/nGYlpPeU91aGRP+D5Fr9C fdztQkW67wXR2eGa8rMcycHIom17z4/j28gS/zHGPQ== X-Google-Smtp-Source: ABdhPJwp4QL8RwmQmcT2CoPcUplHSU9HiByLsXAEntwM01IPEN+3MiJMYkcJQ6Od81V70aDSuFwsTxILAnA/OMfjXrI= X-Received: by 2002:a05:6402:a57:: with SMTP id bt23mr10741907edb.62.1604412647178; Tue, 03 Nov 2020 06:10:47 -0800 (PST) MIME-Version: 1.0 References: <20201029003252.2128653-1-christian.brauner@ubuntu.com> <87pn51ghju.fsf@x220.int.ebiederm.org> <20201029155148.5odu4j2kt62ahcxq@yavin.dot.cyphar.com> <87361xdm4c.fsf@x220.int.ebiederm.org> In-Reply-To: <87361xdm4c.fsf@x220.int.ebiederm.org> From: Alban Crequy Date: Tue, 3 Nov 2020 15:10:35 +0100 Message-ID: Subject: Re: [PATCH 00/34] fs: idmapped mounts To: "Eric W. Biederman" X-Mimecast-Impersonation-Protect: Policy=CLT - Impersonation Protection Definition; Similar Internal Domain=false; Similar Monitored External Domain=false; Custom External Domain=false; Mimecast External Domain=false; Newly Observed Domain=false; Internal User Name=false; Custom Display Name List=false; Reply-to Address Mismatch=false; Targeted Threat Dictionary=false; Mimecast Threat Dictionary=false; Custom Threat Dictionary=false X-Mimecast-Bulk-Signature: yes X-Mimecast-Spam-Signature: bulk X-Scanned-By: MIMEDefang 2.79 on 10.11.54.5 X-MIME-Autoconverted: from quoted-printable to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id 0A3EAsn5027533 X-loop: linux-audit@redhat.com X-Mailman-Approved-At: Mon, 09 Nov 2020 17:53:46 -0500 Cc: Phil Estes , Lennart Poettering , Amir Goldstein , Mimi Zohar , David Howells , Andreas Dilger , Linux Containers , Christian Brauner , Tycho Andersen , Miklos Szeredi , James Morris , smbarber@chromium.org, Christoph Hellwig , linux-ext4@vger.kernel.org, Mrunal Patel , Serge Hallyn , Arnd Bergmann , Jann Horn , selinux@vger.kernel.org, Josh Triplett , linux-fsdevel , Aleksa Sarai , Alexander Viro , Andy Lutomirski , OGAWA Hirofumi , Geoffrey Thomas , James Bottomley , John Johansen , Theodore Tso , Seth Forshee , Dmitry Kasatkin , Jonathan Corbet , linux-unionfs@vger.kernel.org, LSM , linux-audit@redhat.com, linux-api@vger.kernel.org, linux-integrity , =?UTF-8?Q?St=C3=A9phane_Graber?= , Todd Kjos X-BeenThere: linux-audit@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Linux Audit Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: linux-audit-bounces@redhat.com Errors-To: linux-audit-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=linux-audit-bounces@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 T24gVGh1LCBPY3QgMjksIDIwMjAgYXQgNTozNyBQTSBFcmljIFcuIEJpZWRlcm1hbiA8ZWJpZWRl cm1AeG1pc3Npb24uY29tPiB3cm90ZToKPgo+IEFsZWtzYSBTYXJhaSA8Y3lwaGFyQGN5cGhhci5j b20+IHdyaXRlczoKPgo+ID4gT24gMjAyMC0xMC0yOSwgRXJpYyBXLiBCaWVkZXJtYW4gPGViaWVk ZXJtQHhtaXNzaW9uLmNvbT4gd3JvdGU6Cj4gPj4gQ2hyaXN0aWFuIEJyYXVuZXIgPGNocmlzdGlh bi5icmF1bmVyQHVidW50dS5jb20+IHdyaXRlczoKPiA+Pgo+ID4+ID4gSGV5IGV2ZXJ5b25lLAo+ ID4+ID4KPiA+PiA+IEkgdmFuaXNoZWQgZm9yIGEgbGl0dGxlIHdoaWxlIHRvIGZvY3VzIG9uIHRo aXMgd29yayBoZXJlIHNvIHNvcnJ5IGZvcgo+ID4+ID4gbm90IGJlaW5nIGF2YWlsYWJsZSBieSBt YWlsIGZvciBhIHdoaWxlLgo+ID4+ID4KPiA+PiA+IFNpbmNlIHF1aXRlIGEgbG9uZyB0aW1lIHdl IGhhdmUgaXNzdWVzIHdpdGggc2hhcmluZyBtb3VudHMgYmV0d2Vlbgo+ID4+ID4gbXVsdGlwbGUg dW5wcml2aWxlZ2VkIGNvbnRhaW5lcnMgd2l0aCBkaWZmZXJlbnQgaWQgbWFwcGluZ3MsIHNoYXJp bmcgYQo+ID4+ID4gcm9vdGZzIGJldHdlZW4gbXVsdGlwbGUgY29udGFpbmVycyB3aXRoIGRpZmZl cmVudCBpZCBtYXBwaW5ncywgYW5kIGFsc28KPiA+PiA+IHNoYXJpbmcgcmVndWxhciBkaXJlY3Rv cmllcyBhbmQgZmlsZXN5c3RlbXMgYmV0d2VlbiB1c2VycyB3aXRoIGRpZmZlcmVudAo+ID4+ID4g dWlkcyBhbmQgZ2lkcy4gVGhlIGxhdHRlciB1c2UtY2FzZXMgaGF2ZSBiZWNvbWUgZXZlbiBtb3Jl IGltcG9ydGFudCB3aXRoCj4gPj4gPiB0aGUgYXZhaWxhYmlsaXR5IGFuZCBhZG9wdGlvbiBvZiBz eXN0ZW1kLWhvbWVkIChjZi4gWzFdKSB0byBpbXBsZW1lbnQKPiA+PiA+IHBvcnRhYmxlIGhvbWUg ZGlyZWN0b3JpZXMuCj4gPj4KPiA+PiBDYW4geW91IHdhbGsgdXMgdGhyb3VnaCB0aGUgbW90aXZh dGluZyB1c2UgY2FzZT8KPiA+Pgo+ID4+IEFzIG9mIHRoaXMgeWVhcidzIExQQyBJIGhhZCB0aGUg ZGlzdGluY3QgaW1wcmVzc2lvbiB0aGF0IHRoZSBwcmltYXJ5IHVzZQo+ID4+IGNhc2UgZm9yIHN1 Y2ggYSBmZWF0dXJlIHdhcyBkdWUgdG8gdGhlIFJMSU1JVF9OUFJPQyBwcm9ibGVtIHdoZXJlIHR3 bwo+ID4+IGNvbnRhaW5lcnMgd2l0aCB0aGUgc2FtZSB1c2VycyBzdGlsbCB3YW50ZWQgZGlmZmVy ZW50IHVpZCBtYXBwaW5ncyB0bwo+ID4+IHRoZSBkaXNrIGJlY2F1c2UgdGhlIHVzZXJzIHdlcmUg Y29uZmxpY3Rpbmcgd2l0aCBlYWNoIG90aGVyIGJlY2F1c2Ugb2YKPiA+PiB0aGUgcGVyIHVzZXIg cmxpbWl0cy4KPiA+Pgo+ID4+IEZpeGluZyBybGltaXRzIGlzIHN0cmFpZ2h0IGZvcndhcmQgdG8g aW1wbGVtZW50LCBhbmQgZWFzaWVyIHRvIG1hbmFnZQo+ID4+IGZvciBpbXBsZW1lbnRhdGlvbnMg YW5kIGFkbWluaXN0cmF0b3JzLgo+ID4KPiA+IFRoaXMgaXMgc2VwYXJhdGUgdG8gdGhlIHF1ZXN0 aW9uIG9mICJpc29sYXRlZCB1c2VyIG5hbWVzcGFjZXMiIGFuZAo+ID4gbWFuYWdpbmcgZGlmZmVy ZW50IG1hcHBpbmdzIGJldHdlZW4gY29udGFpbmVycy4gVGhpcyBwYXRjaHNldCBpcyBzb2x2aW5n Cj4gPiB0aGUgc2FtZSBwcm9ibGVtIHRoYXQgc2hpZnRmcyBzb2x2ZWQgLS0gc2hhcmluZyBhIHNp bmdsZSBkaXJlY3RvcnkgdHJlZQo+ID4gYmV0d2VlbiBjb250YWluZXJzIHRoYXQgaGF2ZSBkaWZm ZXJlbnQgSUQgbWFwcGluZ3MuIHJsaW1pdHMgKG5vciBhbnkgb2YKPiA+IHRoZSBvdGhlciBwcm9w b3NhbHMgd2UgZGlzY3Vzc2VkIGF0IExQQykgd2lsbCBoZWxwIHdpdGggdGhpcyBwcm9ibGVtLgo+ Cj4gRmlyc3QgYW5kIGZvcmVtb3N0OiBBIHVpZCBzaGlmdCBvbiB3cml0ZSB0byBhIGZpbGVzeXN0 ZW0gaXMgYSBzZWN1cml0eQo+IGJ1ZyB3YWl0aW5nIHRvIGhhcHBlbi4gIFRoaXMgaXMgZXNwZWNp YWxseSBpbiB0aGUgY29udGV4dCBvZiBmYWNpbGl0aWVzCj4gbGlrZSBpb3VyaW5nLCB0aGF0IHBs YXkgdmVyeSBhZ3Jlc3NpdmUgZ2FtZXMgd2l0aCBob3cgcHJvY2VzcyBjb250ZXh0Cj4gbWFrZXMg aXQgdG8gIHN5c3RlbSBjYWxscy4KPgo+IFRoZSBvbmx5IHJlYXNvbiBjb250YWluZXJzIHdlcmUg bm90IGltbWVkaWF0ZWx5IGV4cGxvaXRhYmxlIHdoZW4gaW91cmluZwo+IHdhcyBpbnRyb2R1Y2Vk IGlzIGJlY2F1c2UgdGhlIG1lY2hhbmlzbXMgYXJlIGJ1aWx0IHNvIHRoYXQgZXZlbiBpZgo+IHNv bWV0aGluZyBlc2NhcGVzIGNvbnRhaW5tZW50IHRoZSBzZWN1cml0eSBwcm9wZXJ0aWVzIHN0aWxs IGFwcGx5Lgo+IENoYW5nZXMgdG8gdGhlIHVpZCB3aGVuIHdyaXRpbmcgdG8gdGhlIGZpbGVzeXN0 ZW0gZG9lcyBub3QgaGF2ZSB0aGF0Cj4gcHJvcGVydHkuICBUaGUgdGluaWVzdCBzbGlwIGluIGNv bnRhaW5tZW50IHdpbGwgYmUgYSBzZWN1cml0eSBpc3N1ZS4KPgo+IFRoaXMgaXMgbm90IGV2ZW4g dGhlIGxlYXN0IGJpdCB0aGVvcmV0aWNhbC4gIEkgaGF2ZSBzZWVtIHJlcG9ydHMgb2YgaG93Cj4g c2hpdGZzK292ZXJsYXlmcyBjcmVhdGVkIGEgc2l0dWF0aW9uIHdoZXJlIGFueW9uZSBjb3VsZCBy ZWFkCj4gL2V0Yy9zaGFkb3cuCj4KPiBJZiB5b3UgYXJlIGdvaW5nIHRvIHdyaXRlIHVzaW5nIHRo ZSBzYW1lIHVpZCB0byBkaXNrIGZyb20gZGlmZmVyZW50Cj4gY29udGFpbmVycyB0aGUgcXVlc3Rp b24gYmVjb21lcyB3aHkgY2FuJ3QgdGhvc2UgY29udGFpbmVycyBjb25maWd1cmUKPiB0aG9zZSB1 c2VycyB0byB1c2UgdGhlIHNhbWUga3VpZD8KPgo+IFdoYXQgZml4aW5nIHJsaW1pdHMgZG9lcyBp cyBpdCBmaXhlcyBvbmUgb2YgdGhlIHJlYXNvbnMgdGhhdCBkaWZmZXJlbnQKPiBjb250YWluZXJz IGNvdWxkIG5vdCBzaGFyZSB0aGUgc2FtZSBrdWlkIGZvciB1c2VycyB0aGF0IHdhbnQgdG8gd3Jp dGUgdG8KPiBkaXNrIHdpdGggdGhlIHNhbWUgdWlkLgo+Cj4KPiBJIGh1bWJseSBzdWdnZXN0IHRo YXQgaXQgd2lsbCBiZSBtb3JlIHNlY3VyZSwgYW5kIGVhc2llciB0byBtYWludGFpbiBmb3IKPiBi b3RoIGRldmVsb3BlcnMgYW5kIHVzZXJzIGlmIHdlIGZpeCB0aGUgcmVhc29ucyBwZW9wbGUgd2Fu dCBkaWZmZXJlbnQKPiBjb250YWluZXJzIHRvIGhhdmUgdGhlIHNhbWUgdXNlciBydW5uaW5nIHdp dGggZGlmZmVyZW50IGt1aWRzLgo+Cj4gSWYgbm90IHdoYXQgYXJlIHRoZSByZWFzb25zIHdlIGZ1 bmRhbWVudGFsbHkgbmVlZCB0aGUgc2FtZSBvbi1kaXNrIHVzZXIKPiB1c2luZyBtdWx0aXBsZSBr dWlkcyBpbiB0aGUga2VybmVsPwoKSSB3b3VsZCBsaWtlIHRvIHVzZSB0aGlzIHBhdGNoIHNldCBp biB0aGUgY29udGV4dCBvZiBLdWJlcm5ldGVzLiBJCmRlc2NyaWJlZCBteSB0d28gcG9zc2libGUg c2V0dXBzIGluCmh0dHBzOi8vd3d3LnNwaW5pY3MubmV0L2xpc3RzL2xpbnV4LWNvbnRhaW5lcnMv bXNnMzY1MzcuaHRtbDoKCjEuIEVhY2ggS3ViZXJuZXRlcyBwb2QgaGFzIGl0cyBvd24gdXNlcm5z IGJ1dCB3aXRoIHRoZSBzYW1lIHVzZXIgaWQgbWFwcGluZwoyLiBFYWNoIEt1YmVybmV0ZXMgcG9k IGhhcyBpdHMgb3duIHVzZXJucyB3aXRoIG5vbi1vdmVybGFwcGluZyB1c2VyIGlkCm1hcHBpbmcg KHByb3ZpZGluZyBhZGRpdGlvbmFsIGlzb2xhdGlvbiBiZXR3ZWVuIHBvZHMpCgpCdXQgZXZlbiBp biB0aGUgc2V0dXAgd2hlcmUgYWxsIHBvZHMgcnVuIHdpdGggdGhlIHNhbWUgaWQgbWFwcGluZ3Ms CnRoaXMgcGF0Y2ggc2V0IGlzIHN0aWxsIHVzZWZ1bCB0byBtZSBmb3IgMiByZWFzb25zOgoKMS4g VG8gYXZvaWQgdGhlIGV4cGVuc2l2ZSByZWN1cnNpdmUgY2hvd24gb2YgdGhlIHJvb3Rmcy4gV2Ug Y2Fubm90Cm5lY2Vzc2FyaWx5IGV4dHJhY3QgdGhlIHRhcmJhbGwgZGlyZWN0bHkgd2l0aCB0aGUg cmlnaHQgdWlkcyBiZWNhdXNlCndlIG1pZ2h0IHVzZSB0aGUgc2FtZSBjb250YWluZXIgaW1hZ2Ug Zm9yIHByaXZpbGVnZWQgY29udGFpbmVycyAod2l0aAp0aGUgaG9zdCB1c2VybnMpIGFuZCB1bnBy aXZpbGVnZWQgY29udGFpbmVycyAod2l0aCBhIG5ldyB1c2VybnMpLCBzbwp3ZSBoYXZlIGF0IGxl YXN0IDIg4oCcbWFwcGluZ3PigJ0gKHRha2luZyBtb3JlIHRpbWUgYW5kIHJlc3VsdGluZyBpbiBt b3JlCnN0b3JhZ2Ugc3BhY2UpLiBBbHRob3VnaCB0aGUg4oCcbWV0YWNvcHnigJ0gbW91bnQgb3B0 aW9uIGluIG92ZXJsYXlmcwpoZWxwcyB0byBtYWtlIHRoZSByZWN1cnNpdmUgY2hvd24gZmFzdGVy LCBpdCBjYW4gc3RpbGwgdGFrZSB0aW1lIHdpdGgKbGFyZ2UgY29udGFpbmVyIGltYWdlcyB3aXRo IGxvdHMgb2YgZmlsZXMuIEnigJlkIGxpa2UgdG8gdXNlIHRoaXMgcGF0Y2gKc2V0IHRvIHNldCB1 cCB0aGUgcm9vdCBmcyBpbiBjb25zdGFudCB0aW1lLgoKMi4gVG8gbWFuYWdlIGxhcmdlIGV4dGVy bmFsIHZvbHVtZXMgKE5GUyBvciBvdGhlciBmaWxlc3lzdGVtcykuIEV2ZW4KaWYgYWRtaW5zIGNh biBkZWNpZGUgdG8gdXNlIHRoZSBzYW1lIGt1aWQgb24gYWxsIHRoZSBub2RlcyBvZiB0aGUKS3Vi ZXJuZXRlcyBjbHVzdGVyLCB0aGlzIGlzIGltcHJhY3RpY2FsIGZvciBtaWdyYXRpb24uIFBlb3Bs ZSBjYW4gaGF2ZQpleGlzdGluZyBLdWJlcm5ldGVzIGNsdXN0ZXJzIChjdXJyZW50bHkgd2l0aG91 dCB1c2luZyB1c2VyIG5hbWVzcGFjZXMpCmFuZCBsYXJnZSBORlMgdm9sdW1lcy4gSWYgdGhleSB3 YW50IHRvIHN3aXRjaCB0byBhIG5ldyB2ZXJzaW9uIG9mCkt1YmVybmV0ZXMgd2l0aCB0aGUgdXNl ciBuYW1lc3BhY2UgZmVhdHVyZSBlbmFibGVkLCB0aGV5IHdvdWxkIG5lZWQgdG8KcmVjdXJzaXZl bHkgY2hvd24gYWxsIHRoZSBmaWxlcyBvbiB0aGUgTkZTIHNoYXJlcy4gVGhpcyBjb3VsZCB0YWtl CnRpbWUgb24gbGFyZ2UgZmlsZXN5c3RlbXMgYW5kIHJlYWxpc3RpY2FsbHksIHdlIHdhbnQgdG8g c3VwcG9ydApyb2xsaW5nIHVwZGF0ZXMgd2hlcmUgc29tZSBub2RlcyB1c2UgdGhlIHByZXZpb3Vz IHZlcnNpb24gd2l0aG91dCB1c2VyCm5hbWVzcGFjZXMgYW5kIG5ldyBub2RlcyBhcmUgcHJvZ3Jl c3NpdmVseSBtaWdyYXRlZCB0byB0aGUgbmV3IHVzZXJucwp3aXRoIHRoZSBuZXcgaWQgbWFwcGlu Zy4gSWYgYm90aCBzZXRzIG9mIG5vZGVzIHVzZSB0aGUgc2FtZSBORlMgc2hhcmUsCnRoYXQgY2Fu 4oCZdCB3b3JrLgoKQWxiYW4KCgotLQpMaW51eC1hdWRpdCBtYWlsaW5nIGxpc3QKTGludXgtYXVk aXRAcmVkaGF0LmNvbQpodHRwczovL3d3dy5yZWRoYXQuY29tL21haWxtYW4vbGlzdGluZm8vbGlu dXgtYXVkaXQ=