From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf1-f194.google.com ([209.85.210.194]:37186 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728632AbeJAMUo (ORCPT ); Mon, 1 Oct 2018 08:20:44 -0400 Received: by mail-pf1-f194.google.com with SMTP id x26-v6so8353644pfn.4 for ; Sun, 30 Sep 2018 22:44:42 -0700 (PDT) Date: Mon, 1 Oct 2018 15:44:28 +1000 From: Aleksa Sarai Subject: Re: [PATCH 2/3] namei: implement AT_THIS_ROOT chroot-like path resolution Message-ID: <20181001054246.gfinmx3api7kjhmc@ryuk> References: <20180929103453.12025-1-cyphar@cyphar.com> <20180929131534.24472-1-cyphar@cyphar.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="t73bxnc3vrj4rh47" Content-Disposition: inline In-Reply-To: Sender: linux-arch-owner@vger.kernel.org List-ID: To: Jann Horn Cc: "Eric W. Biederman" , jlayton@kernel.org, Bruce Fields , Al Viro , Arnd Bergmann , shuah@kernel.org, David Howells , Andy Lutomirski , christian@brauner.io, Tycho Andersen , kernel list , linux-fsdevel@vger.kernel.org, linux-arch , linux-kselftest@vger.kernel.org, dev@opencontainers.org, containers@lists.linux-foundation.org, Linux API Message-ID: <20181001054428.Ch5hdC4Vnt8payOY6Olsjr7LlM3jh0-OBxgH3IU9o8E@z> --t73bxnc3vrj4rh47 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2018-09-29, Jann Horn wrote: > The problem is what happens if a folder you are walking through is > concurrently moved out of the chroot. Consider the following scenario: >=20 > You attempt to open "C/../../etc/passwd" under the root "/A/B". > Something else concurrently moves /A/B/C to /A/C. This can result in > the following: >=20 > 1. You start the path walk and reach /A/B/C. > 2. The other process moves /A/B/C to /A/C. Your path walk is now at /A/C. > 3. Your path walk follows the first ".." up into /A. This is outside > the process root, but you never actually encountered the process root, > so you don't notice. > 4. Your path walk follows the second ".." up to /. Again, this is > outside the process root, but you don't notice. > 5. Your path walk walks down to /etc/passwd, and the open completes > successfully. You now have an fd pointing outside your chroot. >=20 > If the root of your walk is below an attacker-controlled directory, > this of course means that you lose instantly. If you point the root of > the walk at a directory out of which a process in the container > wouldn't be able to move the file, you're probably kinda mostly fine - > as long as you know, for certain, that nothing else on the system > would ever do that. But I still wouldn't feel good about that. Please correct me if I'm wrong here (this is the first patch I've written for VFS). Isn't the retry/LOOKUP_REVAL code meant to handle this -- or does that only handle if a particular path component changes *while* it's being walked through? Is it possible for a path walk to succeed after a path component was unmounted (obviously you can't delete a directory path component since you'd get -ENOTEMPTY)? If this is an issue for AT_THIS_ROOT, I believe this might also be an issue for AT_BENEATH since they are effectively both using the same nd->root trick (so you could similarly trick AT_BENEATH to not error out). So we'd need to figure out how to solve this problem in order for AT_BENEATH to be safe. Speaking naively, doesn't it make sense to invalidate the walk if a path component was modified? Or is this something that would be far too costly with little benefit? What if we do more aggressive nd->root checks when resolving with AT_BENEATH or AT_THIS_ROOT (or if nd->root !=3D current->mnt_ns->root)? Regarding chroot attacks, I was aware of the trivial chroot-open-chroot-fchdir attack but I was not aware that there was a rename attack for chroot. Thanks for bringing this up! > I believe that the only way to robustly use this would be to point the > dirfd at a mount point, such that you know that being moved out of the > chroot is impossible because the mount point limits movement of > directories under it. (Well, technically, it doesn't, but it ensures > that if a directory does dangerously move away, the syscall fails.) It > might make sense to hardcode this constraint in the implementation of > AT_THIS_ROOT, to keep people from shooting themselves in the foot. Unless I'm missing something, would this not also affect using a mountpoint as a dirfd-root (with MS_MOVE of an already-walked-through path component) -- or does MS_MOVE cause a rewalk in a way that rename does not? I wouldn't mind tying AT_THIS_ROOT to only work on mountpoints (I thought that bind-mounts would be an issue but you also get -EXDEV when trying to rename across bind-mounts even if they are on the same underlying filesystem). But AT_BENEATH might be a more bitter pill to swallow. I'm not sure. In the usecase of container runtimes, we wouldn't generally be doing resolution of attacker-controlled paths but it still definitely doesn't hurt to consider this part of the threat model -- to avoid foot-gunning as you've said. (There also might be some nested-container cases where you might want to do that.) > > Currently most container runtimes try to do this resolution in > > userspace[1], causing many potential race conditions. In addition, the > > "obvious" alternative (actually performing a {ch,pivot_}root(2)) > > requires a fork+exec which is *very* costly if necessary for every > > filesystem operation involving a container. >=20 > Wait. fork() I understand, but why exec? And actually, you don't need > a full fork() either, clone() lets you do this with some process parts > shared. And then you also shouldn't need to use SCM_RIGHTS, just keep > the file descriptor table shared. And why chroot()/pivot_root(), > wouldn't you want to use setns()? You're right about this -- for C runtimes. In Go we cannot do a raw clone() or fork() (if you do it manually with RawSyscall you'll end with broken runtime state). So you're forced to do fork+exec (which then means that you can't use CLONE_FILES and must use SCM_RIGHTS). Same goes for CLONE_VFORK. (It should be noted that multi-threaded C runtimes have somewhat similar issues -- AFAIK you can technically only use AS-Safe glibc functions after a fork() but that's more of a theoretical concern here. If you just use raw syscalls there isn't an issue.) As for why use setns() rather than pivot_root(), there are cases where you're operating on a container's image without a running container (think image extraction or snapshotting tools). In those cases, you would need to set up a dummy container process in order to setns() into its namespaces. You are right that setns() would be a better option if you want the truthful state of what mounts the container sees. [I also don't like the idea of joining the user namespace of a malicious container unless it's necessary but that's probably just needless paranoia more than anything -- since you're not joining the pidns you aren't trivially addressable by a malicious container.] > // Ensure that we are non-dumpable. Together with > // commit bfedb589252c, this ensures that container root > // can't trace our child once it enters the container. > // My patch > // https://lore.kernel.org/lkml/1451098351-8917-1-git-send-email-jann@the= jh.net/ > // would make this unnecessary, but that patch didn't > // land because Eric nacked it (for political reasons, > // because people incorrectly claimed that this was a > // security fix): Unless I'm very much mistaken this was fixed by bfedb589252c ("mm: Add a user_ns owner to mm_struct and fix ptrace permission checks"). If you join a user namespace then processes within that user namespace won't have ptrace_may_access() permissions because your mm is owned by an ancestor user namespace -- only after exec() will you be traceable. We still use PR_SET_DUMPABLE in runc but that's because we support older kernels (and people don't use user namespaces under Docker) but with user namespaces this should not be required anymore. --=20 Aleksa Sarai Senior Software Engineer (Containers) SUSE Linux GmbH --t73bxnc3vrj4rh47 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEXzbGxhtUYBJKdfWmnhiqJn3bjbQFAluxtDwACgkQnhiqJn3b jbTZIxAArmxLc+mBBcfWYPIpM74Ucomwg3OuI7QevIPkqP91a/NNcaaOp8yfs+yx dDKMde7i8F2tfmteGGx0FRKqzeSzfQuDcP2gDi32lfQAzltGjtdFcpdIEWBUn5CN 24ZrLhaNDNV4SGK8xySiGFXTmDRTdhoNPwyE4keC2Ty6wQhoNnko4XAs1kKr9fwT MyhNbX176+mVPGipanmhpGlWixTnkubypkHZOAmIEBSMSsQGOHQ/8qLXmQl9Hao7 JVNb6HdbrwgwKUbVJWwjNDOVmxZZL/Do4FU6bg75AmhyJu7dYs/WRkT8ykgabkQn dkp7jcaXrDIVX2YXaLwO+0azthUjidbBZ5pdjQRns3ZQ4Sq+1Y9kPjcBQbecb+KH g/v44GDFXYh+pRhF+ATnJaYvdj2Z85VkcWMBjqAy4YljXZ3INDOi5z69+e+4MTl7 +Wn3tYO1cTTqPp3yfTaRFsj+SvM2HvBgLQOXoUhkB0f75sdA3QnWIdRUNqRwPdXU wOrEncCybyga2uyJylmNnNCXYvD2F9kuSheCCkj7QFt4wwnwj+VK6WFW48c7EXMf /oR9zYZB/1VWlSHltpFnqgc+FQLHWZSGG0RsPY7awPEtzwo7cefEPue+Vjz9JjNu M/DND5OZgujKRMsz5/nkz2WWSp9MxVeV1QWrd2EXpV4K94EdnHY= =0lfj -----END PGP SIGNATURE----- --t73bxnc3vrj4rh47--