Why does Python want to read /proc/meminfo

Why does Python want to read /proc/meminfo

[Ian Pilcher]

I am trying to write an SELinux policy to confine a simple service that
I have written in Python, and I'm trying to decide whether to allow or
dontaudit various denials.

To start, I've reduced my service to the simplest case:

   #!/usr/bin/python

   import sys

   sys.exit()

Running this program in a confined domain generated the following
denial:

avc:  denied  { read } for  pid=2024 comm="denatc" name="meminfo" 
dev="proc" ino=4026532028 scontext=system_u:system_r:denatc_t:s0 
tcontext=system_u:object_r:proc_t:s0 tclass=file

The program does continue on and exit cleanly, so it doesn't seem to
strictly require the access.

Does anyone know why Python is trying to access this file, or what
functionality I might be missing if I don't allow the access?

-- 
========================================================================
Ian Pilcher                                         arequipeno@gmail.com
-------- "I grew up before Mark Zuckerberg invented friendship" --------
========================================================================

Re: Why does Python want to read /proc/meminfo

[Ian Pilcher]

On 05/06/2017 12:51 AM, dieter wrote:
> Personally, I doubt that you will find a reference.
> Instead, I assume that the reference comes from the C runtime library.
> It might hepl optimize memory management to know about "meminfo" details.

You're right.  Seems that it's glibc's qsort().

So it seems that any service written in Python (or any other program
that uses qsort) needs to be given read access to most of /proc or deal
with the (unspecified) consequences of not allowing qsort() to determine
the amount of memory in the system.

Delightful.

-- 
========================================================================
Ian Pilcher                                         arequipeno@gmail.com
-------- "I grew up before Mark Zuckerberg invented friendship" --------
========================================================================

Re: Why does Python want to read /proc/meminfo

[Dominick Grift]

[-- Attachment #1: Type: text/plain, Size: 2512 bytes --]

On Sat, May 06, 2017 at 11:07:34AM -0500, Ian Pilcher wrote:
> On 05/06/2017 12:51 AM, dieter wrote:
> > Personally, I doubt that you will find a reference.
> > Instead, I assume that the reference comes from the C runtime library.
> > It might hepl optimize memory management to know about "meminfo" details.
> 
> You're right.  Seems that it's glibc's qsort().
> 
> So it seems that any service written in Python (or any other program
> that uses qsort) needs to be given read access to most of /proc or deal
> with the (unspecified) consequences of not allowing qsort() to determine
> the amount of memory in the system.
> 
> Delightful.

For SELinux policy writers the above is a minor issue compared to pythons' shutil.copy2() implementation
shutil.copy2 behaves like `cp -a` and copies all of the security attriutes with the file.

This often causes us to give consumers of shutil.copy2() much broader access than strictly needed

This is also a problem with `cp -a` (so python is not alone in this). I am not sure how python handles shutil.copy2() but it would be much better for us if it would exclude the MAC security attributes when it copies

Sort of like shutil.copy2_except_MAC_security_attributes or `cp -a-minus-MAC-security-attributes`

Consider this:

some consumer of shutil.copy2 copies /etc/shadow to /tmp/myapp-XXXXX and edits it there.

Because it copies the security context with /etc/shadow, it later needs to be granted to edit files with the security context for shadow files. Effectively forcing us to allow the process to edit files with the shadow security context

Whereas if it would have excluded the MAC security contexts when it copied the file, we would have just been forced to allow the process to read file with the shadow file context, and not to write to it

The above is a little surrealistic example but it is a very common issue in general

for example pip3 does this all the time and so pip3 processes need much broader access than strictly necessary

> 
> -- 
> ========================================================================
> Ian Pilcher                                         arequipeno@gmail.com
> -------- "I grew up before Mark Zuckerberg invented friendship" --------
> ========================================================================

-- 
Key fingerprint = 5F4D 3CDB D3F8 3652 FBD8  02D5 3B6C 5F1D 2C7B 6B02
https://sks-keyservers.net/pks/lookup?op=get&search=0x3B6C5F1D2C7B6B02
Dominick Grift

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: Why does Python want to read /proc/meminfo

[Dan Stromberg]

On Sat, May 6, 2017 at 9:07 AM, Ian Pilcher <arequipeno@gmail.com> wrote:
> On 05/06/2017 12:51 AM, dieter wrote:
> You're right.  Seems that it's glibc's qsort().
>
> So it seems that any service written in Python (or any other program
> that uses qsort) needs to be given read access to most of /proc or deal
> with the (unspecified) consequences of not allowing qsort() to determine
> the amount of memory in the system.

Actually, this appears to be a false conclusion.  Supporting detail
for CPython 2.x and 3.x:

$ strace -f python -c 'import sys' 2>&1 | egrep /proc
below cmd output started 2017 Sat May 06 09:37:36 PM PDT
above cmd output done    2017 Sat May 06 09:37:36 PM PDT
dstromberg@dell-inspiron:~ x86_64-unknown-linux-gnu 14674

$ strace -f python3 -c 'import sys' 2>&1 | egrep /proc
below cmd output started 2017 Sat May 06 09:37:53 PM PDT
above cmd output done    2017 Sat May 06 09:37:53 PM PDT
dstromberg@dell-inspiron:~ x86_64-unknown-linux-gnu 14674

Also, keep in mind that "python" is just the reference implementation
of the language.  There are several implementations of python the
language now.

Don't be too hard on glibc. It seems to work pretty well, and a good
sort algorithm probably benefits significantly from knowing how much
RAM is available when sorting very large lists.

Also, don't be overly hard on SELinux.  It's a relatively young
technology and may still adapt to such needs better in the future.

HTH

Re: Why does Python want to read /proc/meminfo

[Stephen Smalley]

On Sat, 2017-05-06 at 11:07 -0500, Ian Pilcher wrote:
> On 05/06/2017 12:51 AM, dieter wrote:
> > Personally, I doubt that you will find a reference.
> > Instead, I assume that the reference comes from the C runtime
> > library.
> > It might hepl optimize memory management to know about "meminfo"
> > details.
> 
> You're right.  Seems that it's glibc's qsort().
> 
> So it seems that any service written in Python (or any other program
> that uses qsort) needs to be given read access to most of /proc or
> deal
> with the (unspecified) consequences of not allowing qsort() to
> determine
> the amount of memory in the system.
> 
> Delightful.

Alternatively, assign a specific type to /proc/meminfo and allow read
to it without opening up access to most of /proc.  That is what Android
has done, for example.

Re: Why does Python want to read /proc/meminfo

[Dominick Grift]

[-- Attachment #1: Type: text/plain, Size: 3774 bytes --]

On Mon, May 08, 2017 at 09:42:09AM -0400, Stephen Smalley wrote:
> On Sat, 2017-05-06 at 20:00 +0200, Dominick Grift wrote:
> > On Sat, May 06, 2017 at 11:07:34AM -0500, Ian Pilcher wrote:
> > > On 05/06/2017 12:51 AM, dieter wrote:
> > > > Personally, I doubt that you will find a reference.
> > > > Instead, I assume that the reference comes from the C runtime
> > > > library.
> > > > It might hepl optimize memory management to know about "meminfo"
> > > > details.
> > > 
> > > You're right.  Seems that it's glibc's qsort().
> > > 
> > > So it seems that any service written in Python (or any other
> > > program
> > > that uses qsort) needs to be given read access to most of /proc or
> > > deal
> > > with the (unspecified) consequences of not allowing qsort() to
> > > determine
> > > the amount of memory in the system.
> > > 
> > > Delightful.
> > 
> > For SELinux policy writers the above is a minor issue compared to
> > pythons' shutil.copy2() implementation
> > shutil.copy2 behaves like `cp -a` and copies all of the security
> > attriutes with the file.
> > 
> > This often causes us to give consumers of shutil.copy2() much broader
> > access than strictly needed
> > 
> > This is also a problem with `cp -a` (so python is not alone in this).
> > I am not sure how python handles shutil.copy2() but it would be much
> > better for us if it would exclude the MAC security attributes when it
> > copies
> > 
> > Sort of like shutil.copy2_except_MAC_security_attributes or `cp -a-
> > minus-MAC-security-attributes`
> > 
> > Consider this:
> > 
> > some consumer of shutil.copy2 copies /etc/shadow to /tmp/myapp-XXXXX
> > and edits it there.
> > 
> > Because it copies the security context with /etc/shadow, it later
> > needs to be granted to edit files with the security context for
> > shadow files. Effectively forcing us to allow the process to edit
> > files with the shadow security context
> > 
> > Whereas if it would have excluded the MAC security contexts when it
> > copied the file, we would have just been forced to allow the process
> > to read file with the shadow file context, and not to write to it
> > 
> > The above is a little surrealistic example but it is a very common
> > issue in general
> > 
> > for example pip3 does this all the time and so pip3 processes need
> > much broader access than strictly necessary
> 
> shutil.copy2 behavior wrt SELinux was previously discussed here:
> http://selinux.tycho.nsa.narkive.com/euegPAIr/copying-setting-security-selinux-xattr-explicitly
> 
> As a counterexample for your proposal, suppose that shutil.copy2()
> defaults to not copying the MAC attributes.  Now a process copies
> /etc/shadow to /tmp/myapp-XXXXX and edits it there.  Since we don't
> copy MAC attributes, the file ends up being labeled with whatever type
> and range transition is defined for the process on /tmp, or defaults to
> that of /tmp.  As a result, the /etc/shadow data is now readable under
> whatever type/range that happens to be, and potentially leaks to
> processes not authorized to read /etc/shadow.
> 
> Given that shutil.copy2 says that it copies metadata, it makes sense to
> at least try to copy SELinux attributes.  I think the only question is
> what to do in case of failure; the earlier thread proposed ignoring
> EACCES in the same way it already ignores several other errors.
> 
> 

Yes that is true, but yes then at least don't fail hard if copying the contexts fails as youve suggested so that we can prevent the ralabel in policy and get away with it

-- 
Key fingerprint = 5F4D 3CDB D3F8 3652 FBD8  02D5 3B6C 5F1D 2C7B 6B02
https://sks-keyservers.net/pks/lookup?op=get&search=0x3B6C5F1D2C7B6B02
Dominick Grift

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: Why does Python want to read /proc/meminfo

[Stephen Smalley]

On Sat, 2017-05-06 at 20:00 +0200, Dominick Grift wrote:
> On Sat, May 06, 2017 at 11:07:34AM -0500, Ian Pilcher wrote:
> > On 05/06/2017 12:51 AM, dieter wrote:
> > > Personally, I doubt that you will find a reference.
> > > Instead, I assume that the reference comes from the C runtime
> > > library.
> > > It might hepl optimize memory management to know about "meminfo"
> > > details.
> > 
> > You're right.  Seems that it's glibc's qsort().
> > 
> > So it seems that any service written in Python (or any other
> > program
> > that uses qsort) needs to be given read access to most of /proc or
> > deal
> > with the (unspecified) consequences of not allowing qsort() to
> > determine
> > the amount of memory in the system.
> > 
> > Delightful.
> 
> For SELinux policy writers the above is a minor issue compared to
> pythons' shutil.copy2() implementation
> shutil.copy2 behaves like `cp -a` and copies all of the security
> attriutes with the file.
> 
> This often causes us to give consumers of shutil.copy2() much broader
> access than strictly needed
> 
> This is also a problem with `cp -a` (so python is not alone in this).
> I am not sure how python handles shutil.copy2() but it would be much
> better for us if it would exclude the MAC security attributes when it
> copies
> 
> Sort of like shutil.copy2_except_MAC_security_attributes or `cp -a-
> minus-MAC-security-attributes`
> 
> Consider this:
> 
> some consumer of shutil.copy2 copies /etc/shadow to /tmp/myapp-XXXXX
> and edits it there.
> 
> Because it copies the security context with /etc/shadow, it later
> needs to be granted to edit files with the security context for
> shadow files. Effectively forcing us to allow the process to edit
> files with the shadow security context
> 
> Whereas if it would have excluded the MAC security contexts when it
> copied the file, we would have just been forced to allow the process
> to read file with the shadow file context, and not to write to it
> 
> The above is a little surrealistic example but it is a very common
> issue in general
> 
> for example pip3 does this all the time and so pip3 processes need
> much broader access than strictly necessary

shutil.copy2 behavior wrt SELinux was previously discussed here:
http://selinux.tycho.nsa.narkive.com/euegPAIr/copying-setting-security-selinux-xattr-explicitly

As a counterexample for your proposal, suppose that shutil.copy2()
defaults to not copying the MAC attributes.  Now a process copies
/etc/shadow to /tmp/myapp-XXXXX and edits it there.  Since we don't
copy MAC attributes, the file ends up being labeled with whatever type
and range transition is defined for the process on /tmp, or defaults to
that of /tmp.  As a result, the /etc/shadow data is now readable under
whatever type/range that happens to be, and potentially leaks to
processes not authorized to read /etc/shadow.

Given that shutil.copy2 says that it copies metadata, it makes sense to
at least try to copy SELinux attributes.  I think the only question is
what to do in case of failure; the earlier thread proposed ignoring
EACCES in the same way it already ignores several other errors.

Re: Why does Python want to read /proc/meminfo

[Daniel Walsh]

On 05/06/2017 12:54 AM, Ian Pilcher wrote:
> I am trying to write an SELinux policy to confine a simple service that
> I have written in Python, and I'm trying to decide whether to allow or
> dontaudit various denials.
>
> To start, I've reduced my service to the simplest case:
>
>   #!/usr/bin/python
>
>   import sys
>
>   sys.exit()
>
> Running this program in a confined domain generated the following
> denial:
>
> avc:  denied  { read } for  pid=2024 comm="denatc" name="meminfo" 
> dev="proc" ino=4026532028 scontext=system_u:system_r:denatc_t:s0 
> tcontext=system_u:object_r:proc_t:s0 tclass=file
>
> The program does continue on and exit cleanly, so it doesn't seem to
> strictly require the access.
>
> Does anyone know why Python is trying to access this file, or what
> functionality I might be missing if I don't allow the access?
>
Usually tools read /proc/meminfo to figure out how much memory is 
available on the system and then to make some assumption about how much 
memory they can use.  A tool might allocate a memory buffer as X% of 
total memory on the system. (This is a bad assumption, since cgroups 
could alter the total amount of memory availabel to the process, but 
/proc/meminfo does not reflect the amount of memory available in the 
cgroup).  The code that looks at /proc/meminfo might be builtin to 
libc,  I would figure that whatever is trying to read /proc/meminfo 
expects to fail in certain situations, so it falls back to an alternate 
code path. This is most likely what you are seeing.


In most situations reading /proc/meminfo would not be considered a 
security risk, especially considering in the case of Cgroups the kernel 
will LIE.  :^)