Re: [PATCH v1 0/7] Remove in-tree usage of MAP_DENYWRITE

[ebiederm@xmission.com (Eric W. Biederman)]

David Laight <David.Laight@ACULAB.COM> writes:

> From: Eric W. Biederman
>> Sent: 12 August 2021 19:47
> ...
>> So today the best advice I can give to userspace is to mark their
>> executables and shared libraries as read-only and immutable.  Otherwise
>> a change to the executable file can change what is mapped into memory.
>> MAP_PRIVATE does not help.
>
> While 'immutable' might be ok for files installed by distributions
> it would be a PITA in development.

For development simply making the files read-only should be sufficient.

What I think should happen is when a new binary is installed it should
always be placed in a new file and renamed to the old name, rather than
copied over the old file.  That can be added to a makefile by writing to
a temporary file name and then using "mv" to the final name.

I tried to look at which options I would need to give to install to
implement that pattern but I don't see it.  Perhaps -b for backup.

I thought I could overcome the feature of CAP_DAC_OVERRIDE where
read-only files can be over-written by adding the immutable attribute.
I just tested and reread the code and I see that using immutable has
a couple of problems.  Only root is allowed to set immutable, and not
even root is allowed to rename or delete immutable files while the
immutable attribute is set.

> ETXTBUSY is a useful reminder that the file you are copying from
> machine A to machine B (etc) is still running and probably ought
> to be killed/stopped before you get confused.

That is true.

> I've never really understood why it doesn't stop shared libraries
> being overwritten - but they do tend to be updated less often.

The problem is that MAP_DENYWRITE can be applied to any file.  Which
makes it another kind of mandatory file lock, and it allows blocking
preventing all writes to any file you can read/mmap.  Which creates all
kinds of denial-of service opportunities.

A nasty example would be using mmap MAP_DENYWRITE on /var/log/messages.
Which a hostile actor could use to hide traces of their presence on a
machine.

So far no one has pointed out how to abuse denying writes to
/proc/self/exe so we can keep the denywrite behavior there for now.

> Overwriting an in-use shared library could be really confusing.
> It is likely that all the code is actually in memory.
> So everything carries on running as normal.
> Until the kernel gets under memory pressure and discards a page.
> Then a page from the new version is faulted in and random
> programs start getting SEGVs.
> This could be days after the borked update.

This should actually happen quite quickly after a borked update.  The
pages in the page cache are cache coherent so as soon as you get a cpu
cache flush the new contents of the overwritten page will be visible.



Which gets to half of the confusion with this.  Long ago and far away
rtld in glibc was written with the assumption that something called
MAP_COPY existed (I think it exists on hurd?).  On Linux at one point it
was emulated with MAP_PRIVATE | MAP_DENYWRITE.  Then we made
MAP_DENYWRITE a noop.  This was probably 20 years ago now.

The glibc rtld implementation is still written using MAP_COPY with
MAP_COPY defined to MAP_PRIVATE | MAP_DENYWRITE on linux.

Even with all of the improvements to the linux mm subsystem since the
year 2000.  Linux the linux mm code does not have the infrastructure to
support MAP_COPY.  Semantically MAP_COPY sounds nice.  An implementation
unfortunately would require that anyone who performs a write(2) to a
file mapped MAP_COPY would require walking the file mapping data
structures and creating one copy of the page for each place that page is
mapped MAP_COPY.  Which in the case of someone overwriting rtld would
require one copy of ld.so in memory for each program running on the
system.  A 457 * 162KiB = 74MiB increase on my little system that has
been up for a while.  Where libc would require something like 457 *
1.8MiB = 822.6MiB.  The performance of creating those copies would
likely also be atrocious.

Anything short of performing copy-on-write when a file is written using
write(2) would not provide the protection for programs.



Florian Weimer, would it be possible to get glibc's ld.so implementation to use
MAP_SHARED?  Just so people reading the code know what to expect of the
kernel?  As far as I can tell there is not a practical difference
between a read-only MAP_PRIVATE and a read-only MAP_SHARED.


Michael Kerrisk, is there any change we can document that in linux for
MAP_PRIVATE mappings the mapped pages will match the underlying file
unless a value is written to a page through the mapping?

Eric