Creating executable device nodes in /dev?

Creating executable device nodes in /dev?

[Andy Lutomirski]

Hi udev people-

The upcoming Linux SGX driver has a device node /dev/sgx.  User code
opens it, does various setup things, mmaps it, and needs to be able to
create PROT_EXEC mappings.  This gets quite awkward if /dev is mounted
noexec.

Can udev arrange to make a device node executable on distros that make
/dev noexec?  This could be done by bind-mounting from an exec tmpfs.
Alternatively, the kernel could probably learn to ignore noexec on
/dev/sgx, but that seems a little bit evil.

Thanks,
Andy

Creating executable device nodes in /dev?

[Andy Lutomirski]

Hi udev people-

The upcoming Linux SGX driver has a device node /dev/sgx.  User code
opens it, does various setup things, mmaps it, and needs to be able to
create PROT_EXEC mappings.  This gets quite awkward if /dev is mounted
noexec.

Can udev arrange to make a device node executable on distros that make
/dev noexec?  This could be done by bind-mounting from an exec tmpfs.
Alternatively, the kernel could probably learn to ignore noexec on
/dev/sgx, but that seems a little bit evil.

Thanks,
Andy

Re: Creating executable device nodes in /dev?

[Zbigniew Jędrzejewski-Szmek]

On Thu, Nov 19, 2020 at 08:17:08AM -0800, Andy Lutomirski wrote:
> Hi udev people-
> 
> The upcoming Linux SGX driver has a device node /dev/sgx.  User code
> opens it, does various setup things, mmaps it, and needs to be able to
> create PROT_EXEC mappings.  This gets quite awkward if /dev is mounted
> noexec.
> 
> Can udev arrange to make a device node executable on distros that make
> /dev noexec?  This could be done by bind-mounting from an exec tmpfs.
> Alternatively, the kernel could probably learn to ignore noexec on
> /dev/sgx, but that seems a little bit evil.

I'd be inclined to simply drop noexec from /dev by default.
We don't do noexec on either /tmp or /dev/shm (because that causes immediate
problems with stuff like Java and cffi). And if you have those two at your
disposal anyway, having noexec on /dev doesn't seem important.

Afaik, the kernel would refuse execve() on a character or block device
anyway. Thus noexec on /dev matters only for actual binaries copied to
/dev, which requires root privileges in the first place.

Zbyszek

Re: Creating executable device nodes in /dev?

[Topi Miettinen]

On 19.11.2020 18.32, Zbigniew Jędrzejewski-Szmek wrote:
> On Thu, Nov 19, 2020 at 08:17:08AM -0800, Andy Lutomirski wrote:
>> Hi udev people-
>>
>> The upcoming Linux SGX driver has a device node /dev/sgx.  User code
>> opens it, does various setup things, mmaps it, and needs to be able to
>> create PROT_EXEC mappings.  This gets quite awkward if /dev is mounted
>> noexec.
>>
>> Can udev arrange to make a device node executable on distros that make
>> /dev noexec?  This could be done by bind-mounting from an exec tmpfs.
>> Alternatively, the kernel could probably learn to ignore noexec on
>> /dev/sgx, but that seems a little bit evil.
> 
> I'd be inclined to simply drop noexec from /dev by default.
> We don't do noexec on either /tmp or /dev/shm (because that causes immediate
> problems with stuff like Java and cffi). And if you have those two at your
> disposal anyway, having noexec on /dev doesn't seem important.

I'd propose to not enable exec globally, but if a service needs SGX, it 
could use something like MountOptions=/dev:exec only in those cases 
where it's needed. That way it's possible to disallow writable and 
executable file systems for most services (which typically don't need 
/tmp or /dev/shm either). Of course the opposite 
(MountOptions=/dev:noexec) would be also possible, but I'd expect that 
this would be needed to be used more often.

-Topi

Re: Creating executable device nodes in /dev?

[Topi Miettinen]

On 19.11.2020 18.32, Zbigniew Jędrzejewski-Szmek wrote:
> On Thu, Nov 19, 2020 at 08:17:08AM -0800, Andy Lutomirski wrote:
>> Hi udev people-
>>
>> The upcoming Linux SGX driver has a device node /dev/sgx.  User code
>> opens it, does various setup things, mmaps it, and needs to be able to
>> create PROT_EXEC mappings.  This gets quite awkward if /dev is mounted
>> noexec.
>>
>> Can udev arrange to make a device node executable on distros that make
>> /dev noexec?  This could be done by bind-mounting from an exec tmpfs.
>> Alternatively, the kernel could probably learn to ignore noexec on
>> /dev/sgx, but that seems a little bit evil.
> 
> I'd be inclined to simply drop noexec from /dev by default.
> We don't do noexec on either /tmp or /dev/shm (because that causes immediate
> problems with stuff like Java and cffi). And if you have those two at your
> disposal anyway, having noexec on /dev doesn't seem important.

I'd propose to not enable exec globally, but if a service needs SGX, it 
could use something like MountOptions=/dev:exec only in those cases 
where it's needed. That way it's possible to disallow writable and 
executable file systems for most services (which typically don't need 
/tmp or /dev/shm either). Of course the opposite 
(MountOptions=/dev:noexec) would be also possible, but I'd expect that 
this would be needed to be used more often.

-Topi

Re: Creating executable device nodes in /dev?

[Andy Lutomirski]

On Thu, Nov 19, 2020 at 10:05 AM Topi Miettinen <toiwoton@gmail.com> wrote:
>
> On 19.11.2020 18.32, Zbigniew Jędrzejewski-Szmek wrote:
> > On Thu, Nov 19, 2020 at 08:17:08AM -0800, Andy Lutomirski wrote:
> >> Hi udev people-
> >>
> >> The upcoming Linux SGX driver has a device node /dev/sgx.  User code
> >> opens it, does various setup things, mmaps it, and needs to be able to
> >> create PROT_EXEC mappings.  This gets quite awkward if /dev is mounted
> >> noexec.
> >>
> >> Can udev arrange to make a device node executable on distros that make
> >> /dev noexec?  This could be done by bind-mounting from an exec tmpfs.
> >> Alternatively, the kernel could probably learn to ignore noexec on
> >> /dev/sgx, but that seems a little bit evil.
> >
> > I'd be inclined to simply drop noexec from /dev by default.
> > We don't do noexec on either /tmp or /dev/shm (because that causes immediate
> > problems with stuff like Java and cffi). And if you have those two at your
> > disposal anyway, having noexec on /dev doesn't seem important.
>
> I'd propose to not enable exec globally, but if a service needs SGX, it
> could use something like MountOptions=/dev:exec only in those cases
> where it's needed. That way it's possible to disallow writable and
> executable file systems for most services (which typically don't need
> /tmp or /dev/shm either). Of course the opposite
> (MountOptions=/dev:noexec) would be also possible, but I'd expect that
> this would be needed to be used more often.
>

I imagine the opposite would be more sensible.  It seems odd to me
that we would want any SGX-using service to require both special mount
options and regular ACL permissions.

As  a further argument, I just did this on a Fedora system:

$ find /dev -perm /ugo+x -a \! -type d -a \! -type l

No results.  So making /dev noexec doesn't seem to have any benefit.

Re: Creating executable device nodes in /dev?

[Andy Lutomirski]

On Thu, Nov 19, 2020 at 10:05 AM Topi Miettinen <toiwoton@gmail.com> wrote:
>
> On 19.11.2020 18.32, Zbigniew Jędrzejewski-Szmek wrote:
> > On Thu, Nov 19, 2020 at 08:17:08AM -0800, Andy Lutomirski wrote:
> >> Hi udev people-
> >>
> >> The upcoming Linux SGX driver has a device node /dev/sgx.  User code
> >> opens it, does various setup things, mmaps it, and needs to be able to
> >> create PROT_EXEC mappings.  This gets quite awkward if /dev is mounted
> >> noexec.
> >>
> >> Can udev arrange to make a device node executable on distros that make
> >> /dev noexec?  This could be done by bind-mounting from an exec tmpfs.
> >> Alternatively, the kernel could probably learn to ignore noexec on
> >> /dev/sgx, but that seems a little bit evil.
> >
> > I'd be inclined to simply drop noexec from /dev by default.
> > We don't do noexec on either /tmp or /dev/shm (because that causes immediate
> > problems with stuff like Java and cffi). And if you have those two at your
> > disposal anyway, having noexec on /dev doesn't seem important.
>
> I'd propose to not enable exec globally, but if a service needs SGX, it
> could use something like MountOptions=/dev:exec only in those cases
> where it's needed. That way it's possible to disallow writable and
> executable file systems for most services (which typically don't need
> /tmp or /dev/shm either). Of course the opposite
> (MountOptions=/dev:noexec) would be also possible, but I'd expect that
> this would be needed to be used more often.
>

I imagine the opposite would be more sensible.  It seems odd to me
that we would want any SGX-using service to require both special mount
options and regular ACL permissions.

As  a further argument, I just did this on a Fedora system:

$ find /dev -perm /ugo+x -a \! -type d -a \! -type l

No results.  So making /dev noexec doesn't seem to have any benefit.

Re: Creating executable device nodes in /dev?

[Topi Miettinen]

On 8.12.2020 20.07, Andy Lutomirski wrote:
> On Thu, Nov 19, 2020 at 10:05 AM Topi Miettinen <toiwoton@gmail.com> wrote:
>>
>> On 19.11.2020 18.32, Zbigniew Jędrzejewski-Szmek wrote:
>>> On Thu, Nov 19, 2020 at 08:17:08AM -0800, Andy Lutomirski wrote:
>>>> Hi udev people-
>>>>
>>>> The upcoming Linux SGX driver has a device node /dev/sgx.  User code
>>>> opens it, does various setup things, mmaps it, and needs to be able to
>>>> create PROT_EXEC mappings.  This gets quite awkward if /dev is mounted
>>>> noexec.
>>>>
>>>> Can udev arrange to make a device node executable on distros that make
>>>> /dev noexec?  This could be done by bind-mounting from an exec tmpfs.
>>>> Alternatively, the kernel could probably learn to ignore noexec on
>>>> /dev/sgx, but that seems a little bit evil.
>>>
>>> I'd be inclined to simply drop noexec from /dev by default.
>>> We don't do noexec on either /tmp or /dev/shm (because that causes immediate
>>> problems with stuff like Java and cffi). And if you have those two at your
>>> disposal anyway, having noexec on /dev doesn't seem important.
>>
>> I'd propose to not enable exec globally, but if a service needs SGX, it
>> could use something like MountOptions=/dev:exec only in those cases
>> where it's needed. That way it's possible to disallow writable and
>> executable file systems for most services (which typically don't need
>> /tmp or /dev/shm either). Of course the opposite
>> (MountOptions=/dev:noexec) would be also possible, but I'd expect that
>> this would be needed to be used more often.
>>
> 
> I imagine the opposite would be more sensible.  It seems odd to me
> that we would want any SGX-using service to require both special mount
> options and regular ACL permissions.

How common are thes SGX-using services? Will every service start using 
it without any special measures taken on it's behalf, or perhaps only a 
special SGX control tool needs access? What about unprivileged user 
applications, do they ever want to access SGX? Could something like 
Widevine deep in a browser need to talk to SGX in a DRM scheme?

> As  a further argument, I just did this on a Fedora system:
> 
> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
> 
> No results.  So making /dev noexec doesn't seem to have any benefit.

It's no surprise that there aren't any executables in /dev since 
removing MAKEDEV ages ago. That's not the issue, which is that /dev is a 
writable directory (for UID=0 but no capabilities are needed) and thus a 
potential location for constructing unapproved executables if it is also 
mounted exec (W^X).

-Topi

Re: Creating executable device nodes in /dev?

[Topi Miettinen]

On 8.12.2020 20.07, Andy Lutomirski wrote:
> On Thu, Nov 19, 2020 at 10:05 AM Topi Miettinen <toiwoton@gmail.com> wrote:
>>
>> On 19.11.2020 18.32, Zbigniew Jędrzejewski-Szmek wrote:
>>> On Thu, Nov 19, 2020 at 08:17:08AM -0800, Andy Lutomirski wrote:
>>>> Hi udev people-
>>>>
>>>> The upcoming Linux SGX driver has a device node /dev/sgx.  User code
>>>> opens it, does various setup things, mmaps it, and needs to be able to
>>>> create PROT_EXEC mappings.  This gets quite awkward if /dev is mounted
>>>> noexec.
>>>>
>>>> Can udev arrange to make a device node executable on distros that make
>>>> /dev noexec?  This could be done by bind-mounting from an exec tmpfs.
>>>> Alternatively, the kernel could probably learn to ignore noexec on
>>>> /dev/sgx, but that seems a little bit evil.
>>>
>>> I'd be inclined to simply drop noexec from /dev by default.
>>> We don't do noexec on either /tmp or /dev/shm (because that causes immediate
>>> problems with stuff like Java and cffi). And if you have those two at your
>>> disposal anyway, having noexec on /dev doesn't seem important.
>>
>> I'd propose to not enable exec globally, but if a service needs SGX, it
>> could use something like MountOptions=/dev:exec only in those cases
>> where it's needed. That way it's possible to disallow writable and
>> executable file systems for most services (which typically don't need
>> /tmp or /dev/shm either). Of course the opposite
>> (MountOptions=/dev:noexec) would be also possible, but I'd expect that
>> this would be needed to be used more often.
>>
> 
> I imagine the opposite would be more sensible.  It seems odd to me
> that we would want any SGX-using service to require both special mount
> options and regular ACL permissions.

How common are thes SGX-using services? Will every service start using 
it without any special measures taken on it's behalf, or perhaps only a 
special SGX control tool needs access? What about unprivileged user 
applications, do they ever want to access SGX? Could something like 
Widevine deep in a browser need to talk to SGX in a DRM scheme?

> As  a further argument, I just did this on a Fedora system:
> 
> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
> 
> No results.  So making /dev noexec doesn't seem to have any benefit.

It's no surprise that there aren't any executables in /dev since 
removing MAKEDEV ages ago. That's not the issue, which is that /dev is a 
writable directory (for UID=0 but no capabilities are needed) and thus a 
potential location for constructing unapproved executables if it is also 
mounted exec (W^X).

-Topi

Re: Creating executable device nodes in /dev?

[Andy Lutomirski]

> On Dec 8, 2020, at 12:45 PM, Topi Miettinen <toiwoton@gmail.com> wrote:
> 
> On 8.12.2020 20.07, Andy Lutomirski wrote:
>>> On Thu, Nov 19, 2020 at 10:05 AM Topi Miettinen <toiwoton@gmail.com> wrote:
>>> 
>>> On 19.11.2020 18.32, Zbigniew Jędrzejewski-Szmek wrote:
>>>> On Thu, Nov 19, 2020 at 08:17:08AM -0800, Andy Lutomirski wrote:
>>>>> Hi udev people-
>>>>> 
>>>>> The upcoming Linux SGX driver has a device node /dev/sgx.  User code
>>>>> opens it, does various setup things, mmaps it, and needs to be able to
>>>>> create PROT_EXEC mappings.  This gets quite awkward if /dev is mounted
>>>>> noexec.
>>>>> 
>>>>> Can udev arrange to make a device node executable on distros that make
>>>>> /dev noexec?  This could be done by bind-mounting from an exec tmpfs.
>>>>> Alternatively, the kernel could probably learn to ignore noexec on
>>>>> /dev/sgx, but that seems a little bit evil.
>>>> 
>>>> I'd be inclined to simply drop noexec from /dev by default.
>>>> We don't do noexec on either /tmp or /dev/shm (because that causes immediate
>>>> problems with stuff like Java and cffi). And if you have those two at your
>>>> disposal anyway, having noexec on /dev doesn't seem important.
>>> 
>>> I'd propose to not enable exec globally, but if a service needs SGX, it
>>> could use something like MountOptions=/dev:exec only in those cases
>>> where it's needed. That way it's possible to disallow writable and
>>> executable file systems for most services (which typically don't need
>>> /tmp or /dev/shm either). Of course the opposite
>>> (MountOptions=/dev:noexec) would be also possible, but I'd expect that
>>> this would be needed to be used more often.
>>> 
>> I imagine the opposite would be more sensible.  It seems odd to me
>> that we would want any SGX-using service to require both special mount
>> options and regular ACL permissions.
> 
> How common are thes SGX-using services? Will every service start using it without any special measures taken on it's behalf, or perhaps only a special SGX control tool needs access? What about unprivileged user applications, do they ever want to access SGX? Could something like Widevine deep in a browser need to talk to SGX in a DRM scheme?

I honestly don’t know. Widevine is probably some unholy mess of SGX and ME crud. But regular user programs may well end up using SGX for little non-evil enclaves, e.g. storing their keys securely.  It would be nice if unprivileged enclaves just work as long as the use has appropriate permissions on the device nodes.

SGX adoption has been severely hampered by the massive series of recent vulnerabilities and by Intel’s silly licensing scheme. The latter won’t be supported upstream.

> 
>> As  a further argument, I just did this on a Fedora system:
>> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
>> No results.  So making /dev noexec doesn't seem to have any benefit.
> 
> It's no surprise that there aren't any executables in /dev since removing MAKEDEV ages ago. That's not the issue, which is that /dev is a writable directory (for UID=0 but no capabilities are needed) and thus a potential location for constructing unapproved executables if it is also mounted exec (W^X).

UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.

> 
> -Topi

Re: Creating executable device nodes in /dev?

[Andy Lutomirski]

> On Dec 8, 2020, at 12:45 PM, Topi Miettinen <toiwoton@gmail.com> wrote:
> 
> On 8.12.2020 20.07, Andy Lutomirski wrote:
>>> On Thu, Nov 19, 2020 at 10:05 AM Topi Miettinen <toiwoton@gmail.com> wrote:
>>> 
>>> On 19.11.2020 18.32, Zbigniew Jędrzejewski-Szmek wrote:
>>>> On Thu, Nov 19, 2020 at 08:17:08AM -0800, Andy Lutomirski wrote:
>>>>> Hi udev people-
>>>>> 
>>>>> The upcoming Linux SGX driver has a device node /dev/sgx.  User code
>>>>> opens it, does various setup things, mmaps it, and needs to be able to
>>>>> create PROT_EXEC mappings.  This gets quite awkward if /dev is mounted
>>>>> noexec.
>>>>> 
>>>>> Can udev arrange to make a device node executable on distros that make
>>>>> /dev noexec?  This could be done by bind-mounting from an exec tmpfs.
>>>>> Alternatively, the kernel could probably learn to ignore noexec on
>>>>> /dev/sgx, but that seems a little bit evil.
>>>> 
>>>> I'd be inclined to simply drop noexec from /dev by default.
>>>> We don't do noexec on either /tmp or /dev/shm (because that causes immediate
>>>> problems with stuff like Java and cffi). And if you have those two at your
>>>> disposal anyway, having noexec on /dev doesn't seem important.
>>> 
>>> I'd propose to not enable exec globally, but if a service needs SGX, it
>>> could use something like MountOptions=/dev:exec only in those cases
>>> where it's needed. That way it's possible to disallow writable and
>>> executable file systems for most services (which typically don't need
>>> /tmp or /dev/shm either). Of course the opposite
>>> (MountOptions=/dev:noexec) would be also possible, but I'd expect that
>>> this would be needed to be used more often.
>>> 
>> I imagine the opposite would be more sensible.  It seems odd to me
>> that we would want any SGX-using service to require both special mount
>> options and regular ACL permissions.
> 
> How common are thes SGX-using services? Will every service start using it without any special measures taken on it's behalf, or perhaps only a special SGX control tool needs access? What about unprivileged user applications, do they ever want to access SGX? Could something like Widevine deep in a browser need to talk to SGX in a DRM scheme?

I honestly don’t know. Widevine is probably some unholy mess of SGX and ME crud. But regular user programs may well end up using SGX for little non-evil enclaves, e.g. storing their keys securely.  It would be nice if unprivileged enclaves just work as long as the use has appropriate permissions on the device nodes.

SGX adoption has been severely hampered by the massive series of recent vulnerabilities and by Intel’s silly licensing scheme. The latter won’t be supported upstream.

> 
>> As  a further argument, I just did this on a Fedora system:
>> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
>> No results.  So making /dev noexec doesn't seem to have any benefit.
> 
> It's no surprise that there aren't any executables in /dev since removing MAKEDEV ages ago. That's not the issue, which is that /dev is a writable directory (for UID=0 but no capabilities are needed) and thus a potential location for constructing unapproved executables if it is also mounted exec (W^X).

UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.

> 
> -Topi

Re: Creating executable device nodes in /dev?

[Topi Miettinen]

On 8.12.2020 23.30, Andy Lutomirski wrote:
> 
>> On Dec 8, 2020, at 12:45 PM, Topi Miettinen <toiwoton@gmail.com> wrote:
>>
>> On 8.12.2020 20.07, Andy Lutomirski wrote:
>>>> On Thu, Nov 19, 2020 at 10:05 AM Topi Miettinen <toiwoton@gmail.com> wrote:
>>>>
>>>> On 19.11.2020 18.32, Zbigniew Jędrzejewski-Szmek wrote:
>>>>> On Thu, Nov 19, 2020 at 08:17:08AM -0800, Andy Lutomirski wrote:
>>>>>> Hi udev people-
>>>>>>
>>>>>> The upcoming Linux SGX driver has a device node /dev/sgx.  User code
>>>>>> opens it, does various setup things, mmaps it, and needs to be able to
>>>>>> create PROT_EXEC mappings.  This gets quite awkward if /dev is mounted
>>>>>> noexec.
>>>>>>
>>>>>> Can udev arrange to make a device node executable on distros that make
>>>>>> /dev noexec?  This could be done by bind-mounting from an exec tmpfs.
>>>>>> Alternatively, the kernel could probably learn to ignore noexec on
>>>>>> /dev/sgx, but that seems a little bit evil.
>>>>>
>>>>> I'd be inclined to simply drop noexec from /dev by default.
>>>>> We don't do noexec on either /tmp or /dev/shm (because that causes immediate
>>>>> problems with stuff like Java and cffi). And if you have those two at your
>>>>> disposal anyway, having noexec on /dev doesn't seem important.
>>>>
>>>> I'd propose to not enable exec globally, but if a service needs SGX, it
>>>> could use something like MountOptions=/dev:exec only in those cases
>>>> where it's needed. That way it's possible to disallow writable and
>>>> executable file systems for most services (which typically don't need
>>>> /tmp or /dev/shm either). Of course the opposite
>>>> (MountOptions=/dev:noexec) would be also possible, but I'd expect that
>>>> this would be needed to be used more often.
>>>>
>>> I imagine the opposite would be more sensible.  It seems odd to me
>>> that we would want any SGX-using service to require both special mount
>>> options and regular ACL permissions.
>>
>> How common are thes SGX-using services? Will every service start using it without any special measures taken on it's behalf, or perhaps only a special SGX control tool needs access? What about unprivileged user applications, do they ever want to access SGX? Could something like Widevine deep in a browser need to talk to SGX in a DRM scheme?
> 
> I honestly don’t know. Widevine is probably some unholy mess of SGX and ME crud. But regular user programs may well end up using SGX for little non-evil enclaves, e.g. storing their keys securely.  It would be nice if unprivileged enclaves just work as long as the use has appropriate permissions on the device nodes.

Maybe, it would be also great if the access could be limited to those 
users or services which actually need it, by principle of least privilege.

> SGX adoption has been severely hampered by the massive series of recent vulnerabilities and by Intel’s silly licensing scheme. The latter won’t be supported upstream.
> 
>>
>>> As  a further argument, I just did this on a Fedora system:
>>> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
>>> No results.  So making /dev noexec doesn't seem to have any benefit.
>>
>> It's no surprise that there aren't any executables in /dev since removing MAKEDEV ages ago. That's not the issue, which is that /dev is a writable directory (for UID=0 but no capabilities are needed) and thus a potential location for constructing unapproved executables if it is also mounted exec (W^X).
> 
> UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.

Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also 
SELinux is not universal and the policies might not contain all users or 
services.

-Topi

Re: Creating executable device nodes in /dev?

[Topi Miettinen]

On 8.12.2020 23.30, Andy Lutomirski wrote:
> 
>> On Dec 8, 2020, at 12:45 PM, Topi Miettinen <toiwoton@gmail.com> wrote:
>>
>> On 8.12.2020 20.07, Andy Lutomirski wrote:
>>>> On Thu, Nov 19, 2020 at 10:05 AM Topi Miettinen <toiwoton@gmail.com> wrote:
>>>>
>>>> On 19.11.2020 18.32, Zbigniew Jędrzejewski-Szmek wrote:
>>>>> On Thu, Nov 19, 2020 at 08:17:08AM -0800, Andy Lutomirski wrote:
>>>>>> Hi udev people-
>>>>>>
>>>>>> The upcoming Linux SGX driver has a device node /dev/sgx.  User code
>>>>>> opens it, does various setup things, mmaps it, and needs to be able to
>>>>>> create PROT_EXEC mappings.  This gets quite awkward if /dev is mounted
>>>>>> noexec.
>>>>>>
>>>>>> Can udev arrange to make a device node executable on distros that make
>>>>>> /dev noexec?  This could be done by bind-mounting from an exec tmpfs.
>>>>>> Alternatively, the kernel could probably learn to ignore noexec on
>>>>>> /dev/sgx, but that seems a little bit evil.
>>>>>
>>>>> I'd be inclined to simply drop noexec from /dev by default.
>>>>> We don't do noexec on either /tmp or /dev/shm (because that causes immediate
>>>>> problems with stuff like Java and cffi). And if you have those two at your
>>>>> disposal anyway, having noexec on /dev doesn't seem important.
>>>>
>>>> I'd propose to not enable exec globally, but if a service needs SGX, it
>>>> could use something like MountOptions=/dev:exec only in those cases
>>>> where it's needed. That way it's possible to disallow writable and
>>>> executable file systems for most services (which typically don't need
>>>> /tmp or /dev/shm either). Of course the opposite
>>>> (MountOptions=/dev:noexec) would be also possible, but I'd expect that
>>>> this would be needed to be used more often.
>>>>
>>> I imagine the opposite would be more sensible.  It seems odd to me
>>> that we would want any SGX-using service to require both special mount
>>> options and regular ACL permissions.
>>
>> How common are thes SGX-using services? Will every service start using it without any special measures taken on it's behalf, or perhaps only a special SGX control tool needs access? What about unprivileged user applications, do they ever want to access SGX? Could something like Widevine deep in a browser need to talk to SGX in a DRM scheme?
> 
> I honestly don’t know. Widevine is probably some unholy mess of SGX and ME crud. But regular user programs may well end up using SGX for little non-evil enclaves, e.g. storing their keys securely.  It would be nice if unprivileged enclaves just work as long as the use has appropriate permissions on the device nodes.

Maybe, it would be also great if the access could be limited to those 
users or services which actually need it, by principle of least privilege.

> SGX adoption has been severely hampered by the massive series of recent vulnerabilities and by Intel’s silly licensing scheme. The latter won’t be supported upstream.
> 
>>
>>> As  a further argument, I just did this on a Fedora system:
>>> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
>>> No results.  So making /dev noexec doesn't seem to have any benefit.
>>
>> It's no surprise that there aren't any executables in /dev since removing MAKEDEV ages ago. That's not the issue, which is that /dev is a writable directory (for UID=0 but no capabilities are needed) and thus a potential location for constructing unapproved executables if it is also mounted exec (W^X).
> 
> UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.

Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also 
SELinux is not universal and the policies might not contain all users or 
services.

-Topi

Re: Creating executable device nodes in /dev?

[Jarkko Sakkinen]

On Tue, Dec 08, 2020 at 10:07:17AM -0800, Andy Lutomirski wrote:
> On Thu, Nov 19, 2020 at 10:05 AM Topi Miettinen <toiwoton@gmail.com> wrote:
> >
> > On 19.11.2020 18.32, Zbigniew Jędrzejewski-Szmek wrote:
> > > On Thu, Nov 19, 2020 at 08:17:08AM -0800, Andy Lutomirski wrote:
> > >> Hi udev people-
> > >>
> > >> The upcoming Linux SGX driver has a device node /dev/sgx.  User code
> > >> opens it, does various setup things, mmaps it, and needs to be able to
> > >> create PROT_EXEC mappings.  This gets quite awkward if /dev is mounted
> > >> noexec.
> > >>
> > >> Can udev arrange to make a device node executable on distros that make
> > >> /dev noexec?  This could be done by bind-mounting from an exec tmpfs.
> > >> Alternatively, the kernel could probably learn to ignore noexec on
> > >> /dev/sgx, but that seems a little bit evil.
> > >
> > > I'd be inclined to simply drop noexec from /dev by default.
> > > We don't do noexec on either /tmp or /dev/shm (because that causes immediate
> > > problems with stuff like Java and cffi). And if you have those two at your
> > > disposal anyway, having noexec on /dev doesn't seem important.
> >
> > I'd propose to not enable exec globally, but if a service needs SGX, it
> > could use something like MountOptions=/dev:exec only in those cases
> > where it's needed. That way it's possible to disallow writable and
> > executable file systems for most services (which typically don't need
> > /tmp or /dev/shm either). Of course the opposite
> > (MountOptions=/dev:noexec) would be also possible, but I'd expect that
> > this would be needed to be used more often.
> >
> 
> I imagine the opposite would be more sensible.  It seems odd to me
> that we would want any SGX-using service to require both special mount
> options and regular ACL permissions.
> 
> As  a further argument, I just did this on a Fedora system:
> 
> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
> 
> No results.  So making /dev noexec doesn't seem to have any benefit.

Neither does my Ubuntu installation with '-xdev' added (because of
/dev/shm mount).

find /dev -xdev -perm /ugo+x -a \! -type d -a \! -type l

/Jarkko

Re: Creating executable device nodes in /dev?

[Jarkko Sakkinen]

On Tue, Dec 08, 2020 at 10:07:17AM -0800, Andy Lutomirski wrote:
> On Thu, Nov 19, 2020 at 10:05 AM Topi Miettinen <toiwoton@gmail.com> wrote:
> >
> > On 19.11.2020 18.32, Zbigniew Jędrzejewski-Szmek wrote:
> > > On Thu, Nov 19, 2020 at 08:17:08AM -0800, Andy Lutomirski wrote:
> > >> Hi udev people-
> > >>
> > >> The upcoming Linux SGX driver has a device node /dev/sgx.  User code
> > >> opens it, does various setup things, mmaps it, and needs to be able to
> > >> create PROT_EXEC mappings.  This gets quite awkward if /dev is mounted
> > >> noexec.
> > >>
> > >> Can udev arrange to make a device node executable on distros that make
> > >> /dev noexec?  This could be done by bind-mounting from an exec tmpfs.
> > >> Alternatively, the kernel could probably learn to ignore noexec on
> > >> /dev/sgx, but that seems a little bit evil.
> > >
> > > I'd be inclined to simply drop noexec from /dev by default.
> > > We don't do noexec on either /tmp or /dev/shm (because that causes immediate
> > > problems with stuff like Java and cffi). And if you have those two at your
> > > disposal anyway, having noexec on /dev doesn't seem important.
> >
> > I'd propose to not enable exec globally, but if a service needs SGX, it
> > could use something like MountOptions=/dev:exec only in those cases
> > where it's needed. That way it's possible to disallow writable and
> > executable file systems for most services (which typically don't need
> > /tmp or /dev/shm either). Of course the opposite
> > (MountOptions=/dev:noexec) would be also possible, but I'd expect that
> > this would be needed to be used more often.
> >
> 
> I imagine the opposite would be more sensible.  It seems odd to me
> that we would want any SGX-using service to require both special mount
> options and regular ACL permissions.
> 
> As  a further argument, I just did this on a Fedora system:
> 
> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
> 
> No results.  So making /dev noexec doesn't seem to have any benefit.

Neither does my Ubuntu installation with '-xdev' added (because of
/dev/shm mount).

find /dev -xdev -perm /ugo+x -a \! -type d -a \! -type l

/Jarkko

Re: Creating executable device nodes in /dev?

[Jarkko Sakkinen]

On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> > > > As  a further argument, I just did this on a Fedora system:
> > > > $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
> > > > No results.  So making /dev noexec doesn't seem to have any benefit.
> > > 
> > > It's no surprise that there aren't any executables in /dev since
> > > removing MAKEDEV ages ago. That's not the issue, which is that
> > > /dev is a writable directory (for UID=0 but no capabilities are
> > > needed) and thus a potential location for constructing unapproved
> > > executables if it is also mounted exec (W^X).
> > 
> > UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
> 
> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
> is not universal and the policies might not contain all users or services.
> 
> -Topi

What's the data that supports having noexec /dev anyway? With root
access I can then just use something else like /dev/shm mount.

Has there been out in the wild real world cases that noexec mount
of would have prevented?

For me this sounds a lot just something that "feels more secure"
without any measurable benefit. Can you prove me wrong?

/Jarkko

Re: Creating executable device nodes in /dev?

[Jarkko Sakkinen]

On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> > > > As  a further argument, I just did this on a Fedora system:
> > > > $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
> > > > No results.  So making /dev noexec doesn't seem to have any benefit.
> > > 
> > > It's no surprise that there aren't any executables in /dev since
> > > removing MAKEDEV ages ago. That's not the issue, which is that
> > > /dev is a writable directory (for UID=0 but no capabilities are
> > > needed) and thus a potential location for constructing unapproved
> > > executables if it is also mounted exec (W^X).
> > 
> > UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
> 
> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
> is not universal and the policies might not contain all users or services.
> 
> -Topi

What's the data that supports having noexec /dev anyway? With root
access I can then just use something else like /dev/shm mount.

Has there been out in the wild real world cases that noexec mount
of would have prevented?

For me this sounds a lot just something that "feels more secure"
without any measurable benefit. Can you prove me wrong?

/Jarkko

Re: Creating executable device nodes in /dev?

[Jarkko Sakkinen]

On Wed, Dec 09, 2020 at 02:15:28AM +0200, Jarkko Sakkinen wrote:
> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> > > > > As  a further argument, I just did this on a Fedora system:
> > > > > $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
> > > > > No results.  So making /dev noexec doesn't seem to have any benefit.
> > > > 
> > > > It's no surprise that there aren't any executables in /dev since
> > > > removing MAKEDEV ages ago. That's not the issue, which is that
> > > > /dev is a writable directory (for UID=0 but no capabilities are
> > > > needed) and thus a potential location for constructing unapproved
> > > > executables if it is also mounted exec (W^X).
> > > 
> > > UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
> > 
> > Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
> > is not universal and the policies might not contain all users or services.
> > 
> > -Topi
> 
> What's the data that supports having noexec /dev anyway? With root
> access I can then just use something else like /dev/shm mount.
> 
> Has there been out in the wild real world cases that noexec mount
> of would have prevented?

Typo: "of" = "of /dev"

> For me this sounds a lot just something that "feels more secure"
> without any measurable benefit. Can you prove me wrong?

The debate is circled around something not well defined. Of course you
get theoretically more safe system when you decrease priviliges anywhere
in the system. Like you could start do grazy things with stuff that
unprivilged user has access, in order to prevent malware to elevate to
UID 0 in the first place.

I think where this go intellectually wrong is that we are talking about
*default installation* of a distribution. That should have somewhat sane
common sense access control settings. For like a normal desktop user
noexec /dev will not do any possible favor.

Then there is the case when you want to harden installation for an
application, let's' say some server. In that case you will anyway
fine-tune the security settings and go grazy enough with hardening.
When you tailor a server, it's a standard practice to enumerate and
adjust the mount points if needed.

To summarize, I neither understand the intended target audience.

/Jarkko

Re: Creating executable device nodes in /dev?

[Jarkko Sakkinen]

On Wed, Dec 09, 2020 at 02:15:28AM +0200, Jarkko Sakkinen wrote:
> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> > > > > As  a further argument, I just did this on a Fedora system:
> > > > > $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
> > > > > No results.  So making /dev noexec doesn't seem to have any benefit.
> > > > 
> > > > It's no surprise that there aren't any executables in /dev since
> > > > removing MAKEDEV ages ago. That's not the issue, which is that
> > > > /dev is a writable directory (for UID=0 but no capabilities are
> > > > needed) and thus a potential location for constructing unapproved
> > > > executables if it is also mounted exec (W^X).
> > > 
> > > UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
> > 
> > Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
> > is not universal and the policies might not contain all users or services.
> > 
> > -Topi
> 
> What's the data that supports having noexec /dev anyway? With root
> access I can then just use something else like /dev/shm mount.
> 
> Has there been out in the wild real world cases that noexec mount
> of would have prevented?

Typo: "of" = "of /dev"

> For me this sounds a lot just something that "feels more secure"
> without any measurable benefit. Can you prove me wrong?

The debate is circled around something not well defined. Of course you
get theoretically more safe system when you decrease priviliges anywhere
in the system. Like you could start do grazy things with stuff that
unprivilged user has access, in order to prevent malware to elevate to
UID 0 in the first place.

I think where this go intellectually wrong is that we are talking about
*default installation* of a distribution. That should have somewhat sane
common sense access control settings. For like a normal desktop user
noexec /dev will not do any possible favor.

Then there is the case when you want to harden installation for an
application, let's' say some server. In that case you will anyway
fine-tune the security settings and go grazy enough with hardening.
When you tailor a server, it's a standard practice to enumerate and
adjust the mount points if needed.

To summarize, I neither understand the intended target audience.

/Jarkko

Antw: [EXT] Re: [systemd-devel] Creating executable device nodes in /dev?

[Ulrich Windl]

>>> Jarkko Sakkinen <jarkko@kernel.org> schrieb am 09.12.2020 um 01:15 in Nachricht
<20201209001521.GA64007@kernel.org>:

...
> 
> What's the data that supports having noexec /dev anyway? With root
> access I can then just use something else like /dev/shm mount.
> 
> Has there been out in the wild real world cases that noexec mount
> of would have prevented?
> 
> For me this sounds a lot just something that "feels more secure"
> without any measurable benefit. Can you prove me wrong?

I think the better question is: Why not allow it? I.e.: Why do you want to forbid it?

Event though I wouldn't like it myself, I could even think of noexec /tmp.

Regards,
Ulrich

Antw: [EXT] Re: [systemd-devel] Creating executable device nodes in /dev?

[Ulrich Windl]

>>> Jarkko Sakkinen <jarkko@kernel.org> schrieb am 09.12.2020 um 01:15 in Nachricht
<20201209001521.GA64007@kernel.org>:

...
> 
> What's the data that supports having noexec /dev anyway? With root
> access I can then just use something else like /dev/shm mount.
> 
> Has there been out in the wild real world cases that noexec mount
> of would have prevented?
> 
> For me this sounds a lot just something that "feels more secure"
> without any measurable benefit. Can you prove me wrong?

I think the better question is: Why not allow it? I.e.: Why do you want to forbid it?

Event though I wouldn't like it myself, I could even think of noexec /tmp.

Regards,
Ulrich

Re: Creating executable device nodes in /dev?

[Topi Miettinen]

On 9.12.2020 2.15, Jarkko Sakkinen wrote:
> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>>>>> As  a further argument, I just did this on a Fedora system:
>>>>> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
>>>>
>>>> It's no surprise that there aren't any executables in /dev since
>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>>>> /dev is a writable directory (for UID=0 but no capabilities are
>>>> needed) and thus a potential location for constructing unapproved
>>>> executables if it is also mounted exec (W^X).
>>>
>>> UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
>>
>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
>> is not universal and the policies might not contain all users or services.
>>
>> -Topi
> 
> What's the data that supports having noexec /dev anyway? With root
> access I can then just use something else like /dev/shm mount.
> 
> Has there been out in the wild real world cases that noexec mount
> of would have prevented?
> 
> For me this sounds a lot just something that "feels more secure"
> without any measurable benefit. Can you prove me wrong?

I don't think security works that way. An attacker has various methods 
to choose from, some are more interesting than others. The case where 
rw,exec /dev would be interesting would imply that easier or more common 
avenues would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, 
or /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP 
approach with no need for any file system access is getting more common. 
It does not mean that it would not be prudent to block the relatively 
easy approaches too, including /dev.

-Topi

Re: Creating executable device nodes in /dev?

[Topi Miettinen]

On 9.12.2020 2.15, Jarkko Sakkinen wrote:
> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>>>>> As  a further argument, I just did this on a Fedora system:
>>>>> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
>>>>
>>>> It's no surprise that there aren't any executables in /dev since
>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>>>> /dev is a writable directory (for UID=0 but no capabilities are
>>>> needed) and thus a potential location for constructing unapproved
>>>> executables if it is also mounted exec (W^X).
>>>
>>> UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
>>
>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
>> is not universal and the policies might not contain all users or services.
>>
>> -Topi
> 
> What's the data that supports having noexec /dev anyway? With root
> access I can then just use something else like /dev/shm mount.
> 
> Has there been out in the wild real world cases that noexec mount
> of would have prevented?
> 
> For me this sounds a lot just something that "feels more secure"
> without any measurable benefit. Can you prove me wrong?

I don't think security works that way. An attacker has various methods 
to choose from, some are more interesting than others. The case where 
rw,exec /dev would be interesting would imply that easier or more common 
avenues would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, 
or /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP 
approach with no need for any file system access is getting more common. 
It does not mean that it would not be prudent to block the relatively 
easy approaches too, including /dev.

-Topi

Re: Creating executable device nodes in /dev?

[Topi Miettinen]

On 9.12.2020 2.42, Jarkko Sakkinen wrote:
> On Wed, Dec 09, 2020 at 02:15:28AM +0200, Jarkko Sakkinen wrote:
>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>>>>>> As  a further argument, I just did this on a Fedora system:
>>>>>> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
>>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
>>>>>
>>>>> It's no surprise that there aren't any executables in /dev since
>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>>>>> /dev is a writable directory (for UID=0 but no capabilities are
>>>>> needed) and thus a potential location for constructing unapproved
>>>>> executables if it is also mounted exec (W^X).
>>>>
>>>> UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
>>>
>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
>>> is not universal and the policies might not contain all users or services.
>>>
>>> -Topi
>>
>> What's the data that supports having noexec /dev anyway? With root
>> access I can then just use something else like /dev/shm mount.
>>
>> Has there been out in the wild real world cases that noexec mount
>> of would have prevented?
> 
> Typo: "of" = "of /dev"
> 
>> For me this sounds a lot just something that "feels more secure"
>> without any measurable benefit. Can you prove me wrong?
> 
> The debate is circled around something not well defined. Of course you
> get theoretically more safe system when you decrease priviliges anywhere
> in the system. Like you could start do grazy things with stuff that
> unprivilged user has access, in order to prevent malware to elevate to
> UID 0 in the first place.
> 
> I think where this go intellectually wrong is that we are talking about
> *default installation* of a distribution. That should have somewhat sane
> common sense access control settings. For like a normal desktop user
> noexec /dev will not do any possible favor.
> 
> Then there is the case when you want to harden installation for an
> application, let's' say some server. In that case you will anyway
> fine-tune the security settings and go grazy enough with hardening.
> When you tailor a server, it's a standard practice to enumerate and
> adjust the mount points if needed.

I think we agree that there's a need for either way to allow SGX (if 
default is hardened) or a hardening option (in the opposite case). For 
systemd I see two approaches:

1. Default noexec /dev, override with something like
- ExecPaths=/dev
- MountOptions=/dev:rw,exec,dev,nosuid
- or even MountOptions=/dev/sgx:rw,exec,dev,nosuid
- ProtectDev=no
- AllowSGX=yes

2. Default exec /dev, override with
- NoExecPaths=/dev
- MountOptions=/dev:rw,noexec,dev,nosuid
- ProtectDev=yes
- DenySGX=yes

I'd prefer 1. but of course 2. would be reasonable.

> To summarize, I neither understand the intended target audience.

We have something in common: me neither. What's the target audience for 
SGX? What's the use case? What are the users: browsers, system services? 
How would applications use SGX? Should udev rules for /dev/sgx make it 
available to any logged in users with uaccess tags?

-Topi

Re: Creating executable device nodes in /dev?

[Topi Miettinen]

On 9.12.2020 2.42, Jarkko Sakkinen wrote:
> On Wed, Dec 09, 2020 at 02:15:28AM +0200, Jarkko Sakkinen wrote:
>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>>>>>> As  a further argument, I just did this on a Fedora system:
>>>>>> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
>>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
>>>>>
>>>>> It's no surprise that there aren't any executables in /dev since
>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>>>>> /dev is a writable directory (for UID=0 but no capabilities are
>>>>> needed) and thus a potential location for constructing unapproved
>>>>> executables if it is also mounted exec (W^X).
>>>>
>>>> UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
>>>
>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
>>> is not universal and the policies might not contain all users or services.
>>>
>>> -Topi
>>
>> What's the data that supports having noexec /dev anyway? With root
>> access I can then just use something else like /dev/shm mount.
>>
>> Has there been out in the wild real world cases that noexec mount
>> of would have prevented?
> 
> Typo: "of" = "of /dev"
> 
>> For me this sounds a lot just something that "feels more secure"
>> without any measurable benefit. Can you prove me wrong?
> 
> The debate is circled around something not well defined. Of course you
> get theoretically more safe system when you decrease priviliges anywhere
> in the system. Like you could start do grazy things with stuff that
> unprivilged user has access, in order to prevent malware to elevate to
> UID 0 in the first place.
> 
> I think where this go intellectually wrong is that we are talking about
> *default installation* of a distribution. That should have somewhat sane
> common sense access control settings. For like a normal desktop user
> noexec /dev will not do any possible favor.
> 
> Then there is the case when you want to harden installation for an
> application, let's' say some server. In that case you will anyway
> fine-tune the security settings and go grazy enough with hardening.
> When you tailor a server, it's a standard practice to enumerate and
> adjust the mount points if needed.

I think we agree that there's a need for either way to allow SGX (if 
default is hardened) or a hardening option (in the opposite case). For 
systemd I see two approaches:

1. Default noexec /dev, override with something like
- ExecPaths=/dev
- MountOptions=/dev:rw,exec,dev,nosuid
- or even MountOptions=/dev/sgx:rw,exec,dev,nosuid
- ProtectDev=no
- AllowSGX=yes

2. Default exec /dev, override with
- NoExecPaths=/dev
- MountOptions=/dev:rw,noexec,dev,nosuid
- ProtectDev=yes
- DenySGX=yes

I'd prefer 1. but of course 2. would be reasonable.

> To summarize, I neither understand the intended target audience.

We have something in common: me neither. What's the target audience for 
SGX? What's the use case? What are the users: browsers, system services? 
How would applications use SGX? Should udev rules for /dev/sgx make it 
available to any logged in users with uaccess tags?

-Topi

Re: Creating executable device nodes in /dev?

[Jethro Beekman]

[-- Attachment #1: Type: text/plain, Size: 4323 bytes --]

On 2020-12-09 09:58, Topi Miettinen wrote:
> On 9.12.2020 2.42, Jarkko Sakkinen wrote:
>> On Wed, Dec 09, 2020 at 02:15:28AM +0200, Jarkko Sakkinen wrote:
>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>>>>>>> As  a further argument, I just did this on a Fedora system:
>>>>>>> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
>>>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
>>>>>>
>>>>>> It's no surprise that there aren't any executables in /dev since
>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
>>>>>> needed) and thus a potential location for constructing unapproved
>>>>>> executables if it is also mounted exec (W^X).
>>>>>
>>>>> UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
>>>>
>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
>>>> is not universal and the policies might not contain all users or services.
>>>>
>>>> -Topi
>>>
>>> What's the data that supports having noexec /dev anyway? With root
>>> access I can then just use something else like /dev/shm mount.
>>>
>>> Has there been out in the wild real world cases that noexec mount
>>> of would have prevented?
>>
>> Typo: "of" = "of /dev"
>>
>>> For me this sounds a lot just something that "feels more secure"
>>> without any measurable benefit. Can you prove me wrong?
>>
>> The debate is circled around something not well defined. Of course you
>> get theoretically more safe system when you decrease priviliges anywhere
>> in the system. Like you could start do grazy things with stuff that
>> unprivilged user has access, in order to prevent malware to elevate to
>> UID 0 in the first place.
>>
>> I think where this go intellectually wrong is that we are talking about
>> *default installation* of a distribution. That should have somewhat sane
>> common sense access control settings. For like a normal desktop user
>> noexec /dev will not do any possible favor.
>>
>> Then there is the case when you want to harden installation for an
>> application, let's' say some server. In that case you will anyway
>> fine-tune the security settings and go grazy enough with hardening.
>> When you tailor a server, it's a standard practice to enumerate and
>> adjust the mount points if needed.
> 
> I think we agree that there's a need for either way to allow SGX (if default is hardened) or a hardening option (in the opposite case). For systemd I see two approaches:
> 
> 1. Default noexec /dev, override with something like
> - ExecPaths=/dev
> - MountOptions=/dev:rw,exec,dev,nosuid
> - or even MountOptions=/dev/sgx:rw,exec,dev,nosuid
> - ProtectDev=no
> - AllowSGX=yes
> 
> 2. Default exec /dev, override with
> - NoExecPaths=/dev
> - MountOptions=/dev:rw,noexec,dev,nosuid
> - ProtectDev=yes
> - DenySGX=yes
> 
> I'd prefer 1. but of course 2. would be reasonable.
> 
>> To summarize, I neither understand the intended target audience.
> 
> We have something in common: me neither. What's the target audience for SGX? What's the use case? What are the users: browsers, system services? How would applications use SGX? Should udev rules for /dev/sgx make it available to any logged in users with uaccess tags?

Unless the system administrator/distribution has a specific reason to disable SGX, /dev/sgx_enclave it should be generally available to any user process (mode 0666). SGX just provides a different way for an application to execute code by using the ENCLU x86 instruction. Ultimately, any application author could decide to use SGX in its design, including browsers, system services, network services, etc.

The device doesn't really provide any I/O or other hardware access. Even when the device is in use by a process, the OS has full control over scheduling and resource management. The only reason it's a device in /dev is to be able to provide an fd for use with mmap so that it can setup memory ranges for use with the ENCLU instruction.

/dev/sgx_provision provides access to CPU hardware identifier and should be root/root 0600 to preserve privacy.

--
Jethro Beekman | Fortanix



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4490 bytes --]

Re: Creating executable device nodes in /dev?

[Jethro Beekman]

[-- Attachment #1: Type: text/plain, Size: 4323 bytes --]

On 2020-12-09 09:58, Topi Miettinen wrote:
> On 9.12.2020 2.42, Jarkko Sakkinen wrote:
>> On Wed, Dec 09, 2020 at 02:15:28AM +0200, Jarkko Sakkinen wrote:
>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>>>>>>> As  a further argument, I just did this on a Fedora system:
>>>>>>> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
>>>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
>>>>>>
>>>>>> It's no surprise that there aren't any executables in /dev since
>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
>>>>>> needed) and thus a potential location for constructing unapproved
>>>>>> executables if it is also mounted exec (W^X).
>>>>>
>>>>> UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
>>>>
>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
>>>> is not universal and the policies might not contain all users or services.
>>>>
>>>> -Topi
>>>
>>> What's the data that supports having noexec /dev anyway? With root
>>> access I can then just use something else like /dev/shm mount.
>>>
>>> Has there been out in the wild real world cases that noexec mount
>>> of would have prevented?
>>
>> Typo: "of" = "of /dev"
>>
>>> For me this sounds a lot just something that "feels more secure"
>>> without any measurable benefit. Can you prove me wrong?
>>
>> The debate is circled around something not well defined. Of course you
>> get theoretically more safe system when you decrease priviliges anywhere
>> in the system. Like you could start do grazy things with stuff that
>> unprivilged user has access, in order to prevent malware to elevate to
>> UID 0 in the first place.
>>
>> I think where this go intellectually wrong is that we are talking about
>> *default installation* of a distribution. That should have somewhat sane
>> common sense access control settings. For like a normal desktop user
>> noexec /dev will not do any possible favor.
>>
>> Then there is the case when you want to harden installation for an
>> application, let's' say some server. In that case you will anyway
>> fine-tune the security settings and go grazy enough with hardening.
>> When you tailor a server, it's a standard practice to enumerate and
>> adjust the mount points if needed.
> 
> I think we agree that there's a need for either way to allow SGX (if default is hardened) or a hardening option (in the opposite case). For systemd I see two approaches:
> 
> 1. Default noexec /dev, override with something like
> - ExecPaths=/dev
> - MountOptions=/dev:rw,exec,dev,nosuid
> - or even MountOptions=/dev/sgx:rw,exec,dev,nosuid
> - ProtectDev=no
> - AllowSGX=yes
> 
> 2. Default exec /dev, override with
> - NoExecPaths=/dev
> - MountOptions=/dev:rw,noexec,dev,nosuid
> - ProtectDev=yes
> - DenySGX=yes
> 
> I'd prefer 1. but of course 2. would be reasonable.
> 
>> To summarize, I neither understand the intended target audience.
> 
> We have something in common: me neither. What's the target audience for SGX? What's the use case? What are the users: browsers, system services? How would applications use SGX? Should udev rules for /dev/sgx make it available to any logged in users with uaccess tags?

Unless the system administrator/distribution has a specific reason to disable SGX, /dev/sgx_enclave it should be generally available to any user process (mode 0666). SGX just provides a different way for an application to execute code by using the ENCLU x86 instruction. Ultimately, any application author could decide to use SGX in its design, including browsers, system services, network services, etc.

The device doesn't really provide any I/O or other hardware access. Even when the device is in use by a process, the OS has full control over scheduling and resource management. The only reason it's a device in /dev is to be able to provide an fd for use with mmap so that it can setup memory ranges for use with the ENCLU instruction.

/dev/sgx_provision provides access to CPU hardware identifier and should be root/root 0600 to preserve privacy.

--
Jethro Beekman | Fortanix



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4490 bytes --]

Re: Creating executable device nodes in /dev?

[Andy Lutomirski]

> On Dec 9, 2020, at 12:58 AM, Topi Miettinen <toiwoton@gmail.com> wrote:
> 
> On 9.12.2020 2.42, Jarkko Sakkinen wrote:
>>> On Wed, Dec 09, 2020 at 02:15:28AM +0200, Jarkko Sakkinen wrote:
>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>>>>>>> As  a further argument, I just did this on a Fedora system:
>>>>>>> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
>>>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
>>>>>> 
>>>>>> It's no surprise that there aren't any executables in /dev since
>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
>>>>>> needed) and thus a potential location for constructing unapproved
>>>>>> executables if it is also mounted exec (W^X).
>>>>> 
>>>>> UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
>>>> 
>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
>>>> is not universal and the policies might not contain all users or services.
>>>> 
>>>> -Topi
>>> 
>>> What's the data that supports having noexec /dev anyway? With root
>>> access I can then just use something else like /dev/shm mount.
>>> 
>>> Has there been out in the wild real world cases that noexec mount
>>> of would have prevented?
>> Typo: "of" = "of /dev"
>>> For me this sounds a lot just something that "feels more secure"
>>> without any measurable benefit. Can you prove me wrong?
>> The debate is circled around something not well defined. Of course you
>> get theoretically more safe system when you decrease priviliges anywhere
>> in the system. Like you could start do grazy things with stuff that
>> unprivilged user has access, in order to prevent malware to elevate to
>> UID 0 in the first place.
>> I think where this go intellectually wrong is that we are talking about
>> *default installation* of a distribution. That should have somewhat sane
>> common sense access control settings. For like a normal desktop user
>> noexec /dev will not do any possible favor.
>> Then there is the case when you want to harden installation for an
>> application, let's' say some server. In that case you will anyway
>> fine-tune the security settings and go grazy enough with hardening.
>> When you tailor a server, it's a standard practice to enumerate and
>> adjust the mount points if needed.
> 
> I think we agree that there's a need for either way to allow SGX (if default is hardened) or a hardening option (in the opposite case). For systemd I see two approaches:
> 
> 1. Default noexec /dev, override with something like
> - ExecPaths=/dev
> - MountOptions=/dev:rw,exec,dev,nosuid
> - or even MountOptions=/dev/sgx:rw,exec,dev,nosuid
> - ProtectDev=no
> - AllowSGX=yes
> 
> 2. Default exec /dev, override with
> - NoExecPaths=/dev
> - MountOptions=/dev:rw,noexec,dev,nosuid
> - ProtectDev=yes
> - DenySGX=yes
> 
> I'd prefer 1. but of course 2. would be reasonable.

I would argue for 2, for the following reason.  I absolutely agree that hardening a system by making it impossible to create executable code dynamically is valuable, but I don’t think it’s a good default. By default, programs like gcc and clang should work, but so should JITs, and JITs are getting more popular and powerful all the time.  In a default setting that allows JITs, etc, I see no benefit at all to making /dev noexec. To the contrary, making /dev noexec seems like plugging a little restricted corner of code creation (because it requires UID=0) while allowing the easy ways (/tmp, /home, /dev/shm, unshare(2), mmap(), etc).  By all means let admins harden this, but I see no reason to apply some of the hardening when the rest is disabled.

> 
>> To summarize, I neither understand the intended target audience.
> 
> We have something in common: me neither. What's the target audience for SGX? What's the use case? What are the users: browsers, system services? How would applications use SGX? Should udev rules for /dev/sgx make it available to any logged in users with uaccess tags?
> 
> 

I would certainly like it to be available to all software, with the possible exception of extra-hardened systems. Using SGX is not really an interesting attack surface. The main threat is that malware might use SGX to make itself hard to reverse engineer.

In Intel’s original vision, only specially licensed vendors could create SGX software, but Linux pushed back against this quite hard, and new CPUs allow unlicensed enclaves. So your Skylake CPUs support SGX, but not on Linux.

Re: Creating executable device nodes in /dev?

[Andy Lutomirski]

> On Dec 9, 2020, at 12:58 AM, Topi Miettinen <toiwoton@gmail.com> wrote:
> 
> On 9.12.2020 2.42, Jarkko Sakkinen wrote:
>>> On Wed, Dec 09, 2020 at 02:15:28AM +0200, Jarkko Sakkinen wrote:
>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>>>>>>> As  a further argument, I just did this on a Fedora system:
>>>>>>> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
>>>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
>>>>>> 
>>>>>> It's no surprise that there aren't any executables in /dev since
>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
>>>>>> needed) and thus a potential location for constructing unapproved
>>>>>> executables if it is also mounted exec (W^X).
>>>>> 
>>>>> UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
>>>> 
>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
>>>> is not universal and the policies might not contain all users or services.
>>>> 
>>>> -Topi
>>> 
>>> What's the data that supports having noexec /dev anyway? With root
>>> access I can then just use something else like /dev/shm mount.
>>> 
>>> Has there been out in the wild real world cases that noexec mount
>>> of would have prevented?
>> Typo: "of" = "of /dev"
>>> For me this sounds a lot just something that "feels more secure"
>>> without any measurable benefit. Can you prove me wrong?
>> The debate is circled around something not well defined. Of course you
>> get theoretically more safe system when you decrease priviliges anywhere
>> in the system. Like you could start do grazy things with stuff that
>> unprivilged user has access, in order to prevent malware to elevate to
>> UID 0 in the first place.
>> I think where this go intellectually wrong is that we are talking about
>> *default installation* of a distribution. That should have somewhat sane
>> common sense access control settings. For like a normal desktop user
>> noexec /dev will not do any possible favor.
>> Then there is the case when you want to harden installation for an
>> application, let's' say some server. In that case you will anyway
>> fine-tune the security settings and go grazy enough with hardening.
>> When you tailor a server, it's a standard practice to enumerate and
>> adjust the mount points if needed.
> 
> I think we agree that there's a need for either way to allow SGX (if default is hardened) or a hardening option (in the opposite case). For systemd I see two approaches:
> 
> 1. Default noexec /dev, override with something like
> - ExecPaths=/dev
> - MountOptions=/dev:rw,exec,dev,nosuid
> - or even MountOptions=/dev/sgx:rw,exec,dev,nosuid
> - ProtectDev=no
> - AllowSGX=yes
> 
> 2. Default exec /dev, override with
> - NoExecPaths=/dev
> - MountOptions=/dev:rw,noexec,dev,nosuid
> - ProtectDev=yes
> - DenySGX=yes
> 
> I'd prefer 1. but of course 2. would be reasonable.

I would argue for 2, for the following reason.  I absolutely agree that hardening a system by making it impossible to create executable code dynamically is valuable, but I don’t think it’s a good default. By default, programs like gcc and clang should work, but so should JITs, and JITs are getting more popular and powerful all the time.  In a default setting that allows JITs, etc, I see no benefit at all to making /dev noexec. To the contrary, making /dev noexec seems like plugging a little restricted corner of code creation (because it requires UID=0) while allowing the easy ways (/tmp, /home, /dev/shm, unshare(2), mmap(), etc).  By all means let admins harden this, but I see no reason to apply some of the hardening when the rest is disabled.

> 
>> To summarize, I neither understand the intended target audience.
> 
> We have something in common: me neither. What's the target audience for SGX? What's the use case? What are the users: browsers, system services? How would applications use SGX? Should udev rules for /dev/sgx make it available to any logged in users with uaccess tags?
> 
> 

I would certainly like it to be available to all software, with the possible exception of extra-hardened systems. Using SGX is not really an interesting attack surface. The main threat is that malware might use SGX to make itself hard to reverse engineer.

In Intel’s original vision, only specially licensed vendors could create SGX software, but Linux pushed back against this quite hard, and new CPUs allow unlicensed enclaves. So your Skylake CPUs support SGX, but not on Linux.

Re: Creating executable device nodes in /dev?

[Topi Miettinen]

On 9.12.2020 17.14, Andy Lutomirski wrote:
> 
>> On Dec 9, 2020, at 12:58 AM, Topi Miettinen <toiwoton@gmail.com> wrote:
>>
>> On 9.12.2020 2.42, Jarkko Sakkinen wrote:
>>>> On Wed, Dec 09, 2020 at 02:15:28AM +0200, Jarkko Sakkinen wrote:
>>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>>>>>>>> As  a further argument, I just did this on a Fedora system:
>>>>>>>> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
>>>>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
>>>>>>>
>>>>>>> It's no surprise that there aren't any executables in /dev since
>>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
>>>>>>> needed) and thus a potential location for constructing unapproved
>>>>>>> executables if it is also mounted exec (W^X).
>>>>>>
>>>>>> UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
>>>>>
>>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
>>>>> is not universal and the policies might not contain all users or services.
>>>>>
>>>>> -Topi
>>>>
>>>> What's the data that supports having noexec /dev anyway? With root
>>>> access I can then just use something else like /dev/shm mount.
>>>>
>>>> Has there been out in the wild real world cases that noexec mount
>>>> of would have prevented?
>>> Typo: "of" = "of /dev"
>>>> For me this sounds a lot just something that "feels more secure"
>>>> without any measurable benefit. Can you prove me wrong?
>>> The debate is circled around something not well defined. Of course you
>>> get theoretically more safe system when you decrease priviliges anywhere
>>> in the system. Like you could start do grazy things with stuff that
>>> unprivilged user has access, in order to prevent malware to elevate to
>>> UID 0 in the first place.
>>> I think where this go intellectually wrong is that we are talking about
>>> *default installation* of a distribution. That should have somewhat sane
>>> common sense access control settings. For like a normal desktop user
>>> noexec /dev will not do any possible favor.
>>> Then there is the case when you want to harden installation for an
>>> application, let's' say some server. In that case you will anyway
>>> fine-tune the security settings and go grazy enough with hardening.
>>> When you tailor a server, it's a standard practice to enumerate and
>>> adjust the mount points if needed.
>>
>> I think we agree that there's a need for either way to allow SGX (if default is hardened) or a hardening option (in the opposite case). For systemd I see two approaches:
>>
>> 1. Default noexec /dev, override with something like
>> - ExecPaths=/dev
>> - MountOptions=/dev:rw,exec,dev,nosuid
>> - or even MountOptions=/dev/sgx:rw,exec,dev,nosuid
>> - ProtectDev=no
>> - AllowSGX=yes
>>
>> 2. Default exec /dev, override with
>> - NoExecPaths=/dev
>> - MountOptions=/dev:rw,noexec,dev,nosuid
>> - ProtectDev=yes
>> - DenySGX=yes
>>
>> I'd prefer 1. but of course 2. would be reasonable.
> 
> I would argue for 2, for the following reason.  I absolutely agree that hardening a system by making it impossible to create executable code dynamically is valuable, but I don’t think it’s a good default. By default, programs like gcc and clang should work, but so should JITs, and JITs are getting more popular and powerful all the time.  In a default setting that allows JITs, etc, I see no benefit at all to making /dev noexec. To the contrary, making /dev noexec seems like plugging a little restricted corner of code creation (because it requires UID=0) while allowing the easy ways (/tmp, /home, /dev/shm, unshare(2), mmap(), etc).  By all means let admins harden this, but I see no reason to apply some of the hardening when the rest is disabled.

Makes sense, especially if anything in theory could be expected to use 
SGX. In practice, probably no system services will at least initially, 
so hardening knobs make also sense.

> 
>>
>>> To summarize, I neither understand the intended target audience.
>>
>> We have something in common: me neither. What's the target audience for SGX? What's the use case? What are the users: browsers, system services? How would applications use SGX? Should udev rules for /dev/sgx make it available to any logged in users with uaccess tags?
>>
>>
> 
> I would certainly like it to be available to all software, with the possible exception of extra-hardened systems. Using SGX is not really an interesting attack surface. The main threat is that malware might use SGX to make itself hard to reverse engineer.

Maybe also malware which can escape all means of detection, enforced by 
the CPU? Though I don't know if any malware scanners for Linux work can 
check for fileless, memory only malware.

> 
> In Intel’s original vision, only specially licensed vendors could create SGX software, but Linux pushed back against this quite hard, and new CPUs allow unlicensed enclaves. So your Skylake CPUs support SGX, but not on Linux.

Kudos to Linux for the push.

-Topi

Re: Creating executable device nodes in /dev?

[Topi Miettinen]

On 9.12.2020 17.14, Andy Lutomirski wrote:
> 
>> On Dec 9, 2020, at 12:58 AM, Topi Miettinen <toiwoton@gmail.com> wrote:
>>
>> On 9.12.2020 2.42, Jarkko Sakkinen wrote:
>>>> On Wed, Dec 09, 2020 at 02:15:28AM +0200, Jarkko Sakkinen wrote:
>>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>>>>>>>> As  a further argument, I just did this on a Fedora system:
>>>>>>>> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
>>>>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
>>>>>>>
>>>>>>> It's no surprise that there aren't any executables in /dev since
>>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
>>>>>>> needed) and thus a potential location for constructing unapproved
>>>>>>> executables if it is also mounted exec (W^X).
>>>>>>
>>>>>> UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
>>>>>
>>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
>>>>> is not universal and the policies might not contain all users or services.
>>>>>
>>>>> -Topi
>>>>
>>>> What's the data that supports having noexec /dev anyway? With root
>>>> access I can then just use something else like /dev/shm mount.
>>>>
>>>> Has there been out in the wild real world cases that noexec mount
>>>> of would have prevented?
>>> Typo: "of" = "of /dev"
>>>> For me this sounds a lot just something that "feels more secure"
>>>> without any measurable benefit. Can you prove me wrong?
>>> The debate is circled around something not well defined. Of course you
>>> get theoretically more safe system when you decrease priviliges anywhere
>>> in the system. Like you could start do grazy things with stuff that
>>> unprivilged user has access, in order to prevent malware to elevate to
>>> UID 0 in the first place.
>>> I think where this go intellectually wrong is that we are talking about
>>> *default installation* of a distribution. That should have somewhat sane
>>> common sense access control settings. For like a normal desktop user
>>> noexec /dev will not do any possible favor.
>>> Then there is the case when you want to harden installation for an
>>> application, let's' say some server. In that case you will anyway
>>> fine-tune the security settings and go grazy enough with hardening.
>>> When you tailor a server, it's a standard practice to enumerate and
>>> adjust the mount points if needed.
>>
>> I think we agree that there's a need for either way to allow SGX (if default is hardened) or a hardening option (in the opposite case). For systemd I see two approaches:
>>
>> 1. Default noexec /dev, override with something like
>> - ExecPaths=/dev
>> - MountOptions=/dev:rw,exec,dev,nosuid
>> - or even MountOptions=/dev/sgx:rw,exec,dev,nosuid
>> - ProtectDev=no
>> - AllowSGX=yes
>>
>> 2. Default exec /dev, override with
>> - NoExecPaths=/dev
>> - MountOptions=/dev:rw,noexec,dev,nosuid
>> - ProtectDev=yes
>> - DenySGX=yes
>>
>> I'd prefer 1. but of course 2. would be reasonable.
> 
> I would argue for 2, for the following reason.  I absolutely agree that hardening a system by making it impossible to create executable code dynamically is valuable, but I don’t think it’s a good default. By default, programs like gcc and clang should work, but so should JITs, and JITs are getting more popular and powerful all the time.  In a default setting that allows JITs, etc, I see no benefit at all to making /dev noexec. To the contrary, making /dev noexec seems like plugging a little restricted corner of code creation (because it requires UID=0) while allowing the easy ways (/tmp, /home, /dev/shm, unshare(2), mmap(), etc).  By all means let admins harden this, but I see no reason to apply some of the hardening when the rest is disabled.

Makes sense, especially if anything in theory could be expected to use 
SGX. In practice, probably no system services will at least initially, 
so hardening knobs make also sense.

> 
>>
>>> To summarize, I neither understand the intended target audience.
>>
>> We have something in common: me neither. What's the target audience for SGX? What's the use case? What are the users: browsers, system services? How would applications use SGX? Should udev rules for /dev/sgx make it available to any logged in users with uaccess tags?
>>
>>
> 
> I would certainly like it to be available to all software, with the possible exception of extra-hardened systems. Using SGX is not really an interesting attack surface. The main threat is that malware might use SGX to make itself hard to reverse engineer.

Maybe also malware which can escape all means of detection, enforced by 
the CPU? Though I don't know if any malware scanners for Linux work can 
check for fileless, memory only malware.

> 
> In Intel’s original vision, only specially licensed vendors could create SGX software, but Linux pushed back against this quite hard, and new CPUs allow unlicensed enclaves. So your Skylake CPUs support SGX, but not on Linux.

Kudos to Linux for the push.

-Topi

Re: Creating executable device nodes in /dev?

[Andy Lutomirski]

On Wed, Dec 9, 2020 at 11:22 AM Topi Miettinen <toiwoton@gmail.com> wrote:
>
> On 9.12.2020 17.14, Andy Lutomirski wrote:
> >

> Maybe also malware which can escape all means of detection, enforced by
> the CPU? Though I don't know if any malware scanners for Linux work can
> check for fileless, memory only malware.

I don't think this is really relevant to malware detection.  You can't
do syscalls from SGX code, for example, and, even if you could,
malware behavior analysis would be unaffected.  The concern seems to
be more that, once someone has discovered some malware, if it's
protected by SGX then it's plausible that you can't disassemble it.

>
> >
> > In Intel’s original vision, only specially licensed vendors could create SGX software, but Linux pushed back against this quite hard, and new CPUs allow unlicensed enclaves. So your Skylake CPUs support SGX, but not on Linux.
>
> Kudos to Linux for the push.

:)

I don't know if Linux gets full credit for this, but I think we at
least had some impact.

Re: Creating executable device nodes in /dev?

[Andy Lutomirski]

On Wed, Dec 9, 2020 at 11:22 AM Topi Miettinen <toiwoton@gmail.com> wrote:
>
> On 9.12.2020 17.14, Andy Lutomirski wrote:
> >

> Maybe also malware which can escape all means of detection, enforced by
> the CPU? Though I don't know if any malware scanners for Linux work can
> check for fileless, memory only malware.

I don't think this is really relevant to malware detection.  You can't
do syscalls from SGX code, for example, and, even if you could,
malware behavior analysis would be unaffected.  The concern seems to
be more that, once someone has discovered some malware, if it's
protected by SGX then it's plausible that you can't disassemble it.

>
> >
> > In Intel’s original vision, only specially licensed vendors could create SGX software, but Linux pushed back against this quite hard, and new CPUs allow unlicensed enclaves. So your Skylake CPUs support SGX, but not on Linux.
>
> Kudos to Linux for the push.

:)

I don't know if Linux gets full credit for this, but I think we at
least had some impact.

Re: Creating executable device nodes in /dev?

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 1851 bytes --]

On Wed, 2020-12-09 at 07:14 -0800, Andy Lutomirski wrote:
> On Dec 9, 2020, at 12:58 AM, Topi Miettinen <toiwoton@gmail.com> wrote:
[...]
> > I think we agree that there's a need for either way to allow SGX
> > (if default is hardened) or a hardening option (in the opposite
> > case). For systemd I see two approaches:
> > 
> > 1. Default noexec /dev, override with something like
> > - ExecPaths=/dev
> > - MountOptions=/dev:rw,exec,dev,nosuid
> > - or even MountOptions=/dev/sgx:rw,exec,dev,nosuid
> > - ProtectDev=no
> > - AllowSGX=yes
> > 
> > 2. Default exec /dev, override with
> > - NoExecPaths=/dev
> > - MountOptions=/dev:rw,noexec,dev,nosuid
> > - ProtectDev=yes
> > - DenySGX=yes
> > 
> > I'd prefer 1. but of course 2. would be reasonable.
> 
> I would argue for 2, for the following reason.  I absolutely agree
> that hardening a system by making it impossible to create executable
> code dynamically is valuable, but I don’t think it’s a good default.
> By default, programs like gcc and clang should work, but so should
> JITs, and JITs are getting more popular and powerful all the time. 
> In a default setting that allows JITs, etc, I see no benefit at all
> to making /dev noexec. To the contrary, making /dev noexec seems like
> plugging a little restricted corner of code creation (because it
> requires UID=0) while allowing the easy ways (/tmp, /home, /dev/shm,
> unshare(2), mmap(), etc).  By all means let admins harden this, but I
> see no reason to apply some of the hardening when the rest is
> disabled.
[...]

I'm convinced.  I've committed a change to initramfs-tools that removes
the noexec mount option again.

Ben.

-- 
Ben Hutchings
It is a miracle that curiosity survives formal education.
                                                      - Albert Einstein

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Creating executable device nodes in /dev?

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 1851 bytes --]

On Wed, 2020-12-09 at 07:14 -0800, Andy Lutomirski wrote:
> On Dec 9, 2020, at 12:58 AM, Topi Miettinen <toiwoton@gmail.com> wrote:
[...]
> > I think we agree that there's a need for either way to allow SGX
> > (if default is hardened) or a hardening option (in the opposite
> > case). For systemd I see two approaches:
> > 
> > 1. Default noexec /dev, override with something like
> > - ExecPaths=/dev
> > - MountOptions=/dev:rw,exec,dev,nosuid
> > - or even MountOptions=/dev/sgx:rw,exec,dev,nosuid
> > - ProtectDev=no
> > - AllowSGX=yes
> > 
> > 2. Default exec /dev, override with
> > - NoExecPaths=/dev
> > - MountOptions=/dev:rw,noexec,dev,nosuid
> > - ProtectDev=yes
> > - DenySGX=yes
> > 
> > I'd prefer 1. but of course 2. would be reasonable.
> 
> I would argue for 2, for the following reason.  I absolutely agree
> that hardening a system by making it impossible to create executable
> code dynamically is valuable, but I don’t think it’s a good default.
> By default, programs like gcc and clang should work, but so should
> JITs, and JITs are getting more popular and powerful all the time. 
> In a default setting that allows JITs, etc, I see no benefit at all
> to making /dev noexec. To the contrary, making /dev noexec seems like
> plugging a little restricted corner of code creation (because it
> requires UID=0) while allowing the easy ways (/tmp, /home, /dev/shm,
> unshare(2), mmap(), etc).  By all means let admins harden this, but I
> see no reason to apply some of the hardening when the rest is
> disabled.
[...]

I'm convinced.  I've committed a change to initramfs-tools that removes
the noexec mount option again.

Ben.

-- 
Ben Hutchings
It is a miracle that curiosity survives formal education.
                                                      - Albert Einstein

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Antw: [EXT] Re: [systemd-devel] Creating executable device nodes in /dev?

[Jarkko Sakkinen]

On Wed, Dec 09, 2020 at 08:58:52AM +0100, Ulrich Windl wrote:
> >>> Jarkko Sakkinen <jarkko@kernel.org> schrieb am 09.12.2020 um 01:15 in Nachricht
> <20201209001521.GA64007@kernel.org>:
> 
> ...
> > 
> > What's the data that supports having noexec /dev anyway? With root
> > access I can then just use something else like /dev/shm mount.
> > 
> > Has there been out in the wild real world cases that noexec mount
> > of would have prevented?
> > 
> > For me this sounds a lot just something that "feels more secure"
> > without any measurable benefit. Can you prove me wrong?
> 
> I think the better question is: Why not allow it? I.e.: Why do you want to forbid it?
> 
> Event though I wouldn't like it myself, I could even think of noexec /tmp.

On an instance of an OS you should limit whatever is appropriate for
your use case. The debate is about sane defaults.

My argument is essentially that noexec /dev is not a sane default.
For anyone to who this makes sense, does such thing anyway. For
others, noexec /dev is only artificially useful.

> Regards,
> Ulrich

/Jarkko

Re: Antw: [EXT] Re: [systemd-devel] Creating executable device nodes in /dev?

[Jarkko Sakkinen]

On Wed, Dec 09, 2020 at 08:58:52AM +0100, Ulrich Windl wrote:
> >>> Jarkko Sakkinen <jarkko@kernel.org> schrieb am 09.12.2020 um 01:15 in Nachricht
> <20201209001521.GA64007@kernel.org>:
> 
> ...
> > 
> > What's the data that supports having noexec /dev anyway? With root
> > access I can then just use something else like /dev/shm mount.
> > 
> > Has there been out in the wild real world cases that noexec mount
> > of would have prevented?
> > 
> > For me this sounds a lot just something that "feels more secure"
> > without any measurable benefit. Can you prove me wrong?
> 
> I think the better question is: Why not allow it? I.e.: Why do you want to forbid it?
> 
> Event though I wouldn't like it myself, I could even think of noexec /tmp.

On an instance of an OS you should limit whatever is appropriate for
your use case. The debate is about sane defaults.

My argument is essentially that noexec /dev is not a sane default.
For anyone to who this makes sense, does such thing anyway. For
others, noexec /dev is only artificially useful.

> Regards,
> Ulrich

/Jarkko

Re: Creating executable device nodes in /dev?

[Jarkko Sakkinen]

On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
> On 9.12.2020 2.15, Jarkko Sakkinen wrote:
> > On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> > > > > > As  a further argument, I just did this on a Fedora system:
> > > > > > $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
> > > > > > No results.  So making /dev noexec doesn't seem to have any benefit.
> > > > > 
> > > > > It's no surprise that there aren't any executables in /dev since
> > > > > removing MAKEDEV ages ago. That's not the issue, which is that
> > > > > /dev is a writable directory (for UID=0 but no capabilities are
> > > > > needed) and thus a potential location for constructing unapproved
> > > > > executables if it is also mounted exec (W^X).
> > > > 
> > > > UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
> > > 
> > > Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
> > > is not universal and the policies might not contain all users or services.
> > > 
> > > -Topi
> > 
> > What's the data that supports having noexec /dev anyway? With root
> > access I can then just use something else like /dev/shm mount.
> > 
> > Has there been out in the wild real world cases that noexec mount
> > of would have prevented?
> > 
> > For me this sounds a lot just something that "feels more secure"
> > without any measurable benefit. Can you prove me wrong?
> 
> I don't think security works that way. An attacker has various methods to
> choose from, some are more interesting than others. The case where rw,exec
> /dev would be interesting would imply that easier or more common avenues
> would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
> /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP approach
> with no need for any file system access is getting more common. It does not
> mean that it would not be prudent to block the relatively easy approaches
> too, including /dev.

What if we add a new mount option "chrexec", which allows exec
for character devices (S_IFCHR).

> -Topi


/Jarkko

Re: Creating executable device nodes in /dev?

[Jarkko Sakkinen]

On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
> On 9.12.2020 2.15, Jarkko Sakkinen wrote:
> > On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> > > > > > As  a further argument, I just did this on a Fedora system:
> > > > > > $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
> > > > > > No results.  So making /dev noexec doesn't seem to have any benefit.
> > > > > 
> > > > > It's no surprise that there aren't any executables in /dev since
> > > > > removing MAKEDEV ages ago. That's not the issue, which is that
> > > > > /dev is a writable directory (for UID=0 but no capabilities are
> > > > > needed) and thus a potential location for constructing unapproved
> > > > > executables if it is also mounted exec (W^X).
> > > > 
> > > > UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
> > > 
> > > Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
> > > is not universal and the policies might not contain all users or services.
> > > 
> > > -Topi
> > 
> > What's the data that supports having noexec /dev anyway? With root
> > access I can then just use something else like /dev/shm mount.
> > 
> > Has there been out in the wild real world cases that noexec mount
> > of would have prevented?
> > 
> > For me this sounds a lot just something that "feels more secure"
> > without any measurable benefit. Can you prove me wrong?
> 
> I don't think security works that way. An attacker has various methods to
> choose from, some are more interesting than others. The case where rw,exec
> /dev would be interesting would imply that easier or more common avenues
> would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
> /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP approach
> with no need for any file system access is getting more common. It does not
> mean that it would not be prudent to block the relatively easy approaches
> too, including /dev.

What if we add a new mount option "chrexec", which allows exec
for character devices (S_IFCHR).

> -Topi


/Jarkko

Re: Creating executable device nodes in /dev?

[Greg KH]

On Fri, Dec 11, 2020 at 12:46:35PM +0200, Jarkko Sakkinen wrote:
> On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
> > On 9.12.2020 2.15, Jarkko Sakkinen wrote:
> > > On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> > > > > > > As  a further argument, I just did this on a Fedora system:
> > > > > > > $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
> > > > > > > No results.  So making /dev noexec doesn't seem to have any benefit.
> > > > > > 
> > > > > > It's no surprise that there aren't any executables in /dev since
> > > > > > removing MAKEDEV ages ago. That's not the issue, which is that
> > > > > > /dev is a writable directory (for UID=0 but no capabilities are
> > > > > > needed) and thus a potential location for constructing unapproved
> > > > > > executables if it is also mounted exec (W^X).
> > > > > 
> > > > > UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
> > > > 
> > > > Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
> > > > is not universal and the policies might not contain all users or services.
> > > > 
> > > > -Topi
> > > 
> > > What's the data that supports having noexec /dev anyway? With root
> > > access I can then just use something else like /dev/shm mount.
> > > 
> > > Has there been out in the wild real world cases that noexec mount
> > > of would have prevented?
> > > 
> > > For me this sounds a lot just something that "feels more secure"
> > > without any measurable benefit. Can you prove me wrong?
> > 
> > I don't think security works that way. An attacker has various methods to
> > choose from, some are more interesting than others. The case where rw,exec
> > /dev would be interesting would imply that easier or more common avenues
> > would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
> > /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP approach
> > with no need for any file system access is getting more common. It does not
> > mean that it would not be prudent to block the relatively easy approaches
> > too, including /dev.
> 
> What if we add a new mount option "chrexec", which allows exec
> for character devices (S_IFCHR).

Oh please no.

Re: Creating executable device nodes in /dev?

[Greg KH]

On Fri, Dec 11, 2020 at 12:46:35PM +0200, Jarkko Sakkinen wrote:
> On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
> > On 9.12.2020 2.15, Jarkko Sakkinen wrote:
> > > On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> > > > > > > As  a further argument, I just did this on a Fedora system:
> > > > > > > $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
> > > > > > > No results.  So making /dev noexec doesn't seem to have any benefit.
> > > > > > 
> > > > > > It's no surprise that there aren't any executables in /dev since
> > > > > > removing MAKEDEV ages ago. That's not the issue, which is that
> > > > > > /dev is a writable directory (for UID=0 but no capabilities are
> > > > > > needed) and thus a potential location for constructing unapproved
> > > > > > executables if it is also mounted exec (W^X).
> > > > > 
> > > > > UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
> > > > 
> > > > Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
> > > > is not universal and the policies might not contain all users or services.
> > > > 
> > > > -Topi
> > > 
> > > What's the data that supports having noexec /dev anyway? With root
> > > access I can then just use something else like /dev/shm mount.
> > > 
> > > Has there been out in the wild real world cases that noexec mount
> > > of would have prevented?
> > > 
> > > For me this sounds a lot just something that "feels more secure"
> > > without any measurable benefit. Can you prove me wrong?
> > 
> > I don't think security works that way. An attacker has various methods to
> > choose from, some are more interesting than others. The case where rw,exec
> > /dev would be interesting would imply that easier or more common avenues
> > would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
> > /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP approach
> > with no need for any file system access is getting more common. It does not
> > mean that it would not be prudent to block the relatively easy approaches
> > too, including /dev.
> 
> What if we add a new mount option "chrexec", which allows exec
> for character devices (S_IFCHR).

Oh please no.

Re: Creating executable device nodes in /dev?

[Zbigniew Jędrzejewski-Szmek]

On Wed, Dec 09, 2020 at 10:58:59PM +0100, Ben Hutchings wrote:
> I'm convinced.  I've committed a change to initramfs-tools that removes
> the noexec mount option again.

Systemd counterpart: https://github.com/systemd/systemd/pull/17940.

Zbyszek

Re: Creating executable device nodes in /dev?

[Topi Miettinen]

On 11.12.2020 12.46, Jarkko Sakkinen wrote:
> On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
>> On 9.12.2020 2.15, Jarkko Sakkinen wrote:
>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>>>>>>> As  a further argument, I just did this on a Fedora system:
>>>>>>> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
>>>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
>>>>>>
>>>>>> It's no surprise that there aren't any executables in /dev since
>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
>>>>>> needed) and thus a potential location for constructing unapproved
>>>>>> executables if it is also mounted exec (W^X).
>>>>>
>>>>> UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
>>>>
>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
>>>> is not universal and the policies might not contain all users or services.
>>>>
>>>> -Topi
>>>
>>> What's the data that supports having noexec /dev anyway? With root
>>> access I can then just use something else like /dev/shm mount.
>>>
>>> Has there been out in the wild real world cases that noexec mount
>>> of would have prevented?
>>>
>>> For me this sounds a lot just something that "feels more secure"
>>> without any measurable benefit. Can you prove me wrong?
>>
>> I don't think security works that way. An attacker has various methods to
>> choose from, some are more interesting than others. The case where rw,exec
>> /dev would be interesting would imply that easier or more common avenues
>> would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
>> /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP approach
>> with no need for any file system access is getting more common. It does not
>> mean that it would not be prudent to block the relatively easy approaches
>> too, including /dev.
> 
> What if we add a new mount option "chrexec", which allows exec
> for character devices (S_IFCHR).

I think devices are a bad match for SGX because devices haven't been 
executable and SGX is actually an operation for memory. So something 
like memfd_create(, MFD_SGX) or mmap(,, PROT_READ|PROT_EXEC|PROT_SGX) 
would be much more natural. Even better would be something that 
conceptully also works for AMD version (either with the same flags or 
MFD_SGX / MFD_whatever_the_AMD_version_is).

-Topi

Re: Creating executable device nodes in /dev?

[Topi Miettinen]

On 11.12.2020 12.46, Jarkko Sakkinen wrote:
> On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
>> On 9.12.2020 2.15, Jarkko Sakkinen wrote:
>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>>>>>>> As  a further argument, I just did this on a Fedora system:
>>>>>>> $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
>>>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
>>>>>>
>>>>>> It's no surprise that there aren't any executables in /dev since
>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
>>>>>> needed) and thus a potential location for constructing unapproved
>>>>>> executables if it is also mounted exec (W^X).
>>>>>
>>>>> UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
>>>>
>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
>>>> is not universal and the policies might not contain all users or services.
>>>>
>>>> -Topi
>>>
>>> What's the data that supports having noexec /dev anyway? With root
>>> access I can then just use something else like /dev/shm mount.
>>>
>>> Has there been out in the wild real world cases that noexec mount
>>> of would have prevented?
>>>
>>> For me this sounds a lot just something that "feels more secure"
>>> without any measurable benefit. Can you prove me wrong?
>>
>> I don't think security works that way. An attacker has various methods to
>> choose from, some are more interesting than others. The case where rw,exec
>> /dev would be interesting would imply that easier or more common avenues
>> would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
>> /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP approach
>> with no need for any file system access is getting more common. It does not
>> mean that it would not be prudent to block the relatively easy approaches
>> too, including /dev.
> 
> What if we add a new mount option "chrexec", which allows exec
> for character devices (S_IFCHR).

I think devices are a bad match for SGX because devices haven't been 
executable and SGX is actually an operation for memory. So something 
like memfd_create(, MFD_SGX) or mmap(,, PROT_READ|PROT_EXEC|PROT_SGX) 
would be much more natural. Even better would be something that 
conceptully also works for AMD version (either with the same flags or 
MFD_SGX / MFD_whatever_the_AMD_version_is).

-Topi

Re: [systemd-devel] Creating executable device nodes in /dev?

[Christian Brauner]

On Fri, Dec 11, 2020 at 12:29:31PM +0100, Greg KH wrote:
> On Fri, Dec 11, 2020 at 12:46:35PM +0200, Jarkko Sakkinen wrote:
> > On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
> > > On 9.12.2020 2.15, Jarkko Sakkinen wrote:
> > > > On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> > > > > > > > As  a further argument, I just did this on a Fedora system:
> > > > > > > > $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
> > > > > > > > No results.  So making /dev noexec doesn't seem to have any benefit.
> > > > > > > 
> > > > > > > It's no surprise that there aren't any executables in /dev since
> > > > > > > removing MAKEDEV ages ago. That's not the issue, which is that
> > > > > > > /dev is a writable directory (for UID=0 but no capabilities are
> > > > > > > needed) and thus a potential location for constructing unapproved
> > > > > > > executables if it is also mounted exec (W^X).
> > > > > > 
> > > > > > UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
> > > > > 
> > > > > Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
> > > > > is not universal and the policies might not contain all users or services.
> > > > > 
> > > > > -Topi
> > > > 
> > > > What's the data that supports having noexec /dev anyway? With root
> > > > access I can then just use something else like /dev/shm mount.
> > > > 
> > > > Has there been out in the wild real world cases that noexec mount
> > > > of would have prevented?
> > > > 
> > > > For me this sounds a lot just something that "feels more secure"
> > > > without any measurable benefit. Can you prove me wrong?
> > > 
> > > I don't think security works that way. An attacker has various methods to
> > > choose from, some are more interesting than others. The case where rw,exec
> > > /dev would be interesting would imply that easier or more common avenues
> > > would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
> > > /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP approach
> > > with no need for any file system access is getting more common. It does not
> > > mean that it would not be prudent to block the relatively easy approaches
> > > too, including /dev.
> > 
> > What if we add a new mount option "chrexec", which allows exec
> > for character devices (S_IFCHR).
> 
> Oh please no.

Greg's right. That's very obviously a horrible hack so this is an
instant nak from my side.

Christian

Re: [systemd-devel] Creating executable device nodes in /dev?

[Christian Brauner]

On Fri, Dec 11, 2020 at 12:29:31PM +0100, Greg KH wrote:
> On Fri, Dec 11, 2020 at 12:46:35PM +0200, Jarkko Sakkinen wrote:
> > On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
> > > On 9.12.2020 2.15, Jarkko Sakkinen wrote:
> > > > On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> > > > > > > > As  a further argument, I just did this on a Fedora system:
> > > > > > > > $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
> > > > > > > > No results.  So making /dev noexec doesn't seem to have any benefit.
> > > > > > > 
> > > > > > > It's no surprise that there aren't any executables in /dev since
> > > > > > > removing MAKEDEV ages ago. That's not the issue, which is that
> > > > > > > /dev is a writable directory (for UID=0 but no capabilities are
> > > > > > > needed) and thus a potential location for constructing unapproved
> > > > > > > executables if it is also mounted exec (W^X).
> > > > > > 
> > > > > > UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
> > > > > 
> > > > > Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
> > > > > is not universal and the policies might not contain all users or services.
> > > > > 
> > > > > -Topi
> > > > 
> > > > What's the data that supports having noexec /dev anyway? With root
> > > > access I can then just use something else like /dev/shm mount.
> > > > 
> > > > Has there been out in the wild real world cases that noexec mount
> > > > of would have prevented?
> > > > 
> > > > For me this sounds a lot just something that "feels more secure"
> > > > without any measurable benefit. Can you prove me wrong?
> > > 
> > > I don't think security works that way. An attacker has various methods to
> > > choose from, some are more interesting than others. The case where rw,exec
> > > /dev would be interesting would imply that easier or more common avenues
> > > would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
> > > /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP approach
> > > with no need for any file system access is getting more common. It does not
> > > mean that it would not be prudent to block the relatively easy approaches
> > > too, including /dev.
> > 
> > What if we add a new mount option "chrexec", which allows exec
> > for character devices (S_IFCHR).
> 
> Oh please no.

Greg's right. That's very obviously a horrible hack so this is an
instant nak from my side.

Christian

Re: [systemd-devel] Creating executable device nodes in /dev?

[Christian Brauner]

On Fri, Dec 11, 2020 at 12:29:31PM +0100, Greg KH wrote:
> On Fri, Dec 11, 2020 at 12:46:35PM +0200, Jarkko Sakkinen wrote:
> > On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
> > > On 9.12.2020 2.15, Jarkko Sakkinen wrote:
> > > > On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> > > > > > > > As  a further argument, I just did this on a Fedora system:
> > > > > > > > $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
> > > > > > > > No results.  So making /dev noexec doesn't seem to have any benefit.
> > > > > > > 
> > > > > > > It's no surprise that there aren't any executables in /dev since
> > > > > > > removing MAKEDEV ages ago. That's not the issue, which is that
> > > > > > > /dev is a writable directory (for UID=0 but no capabilities are
> > > > > > > needed) and thus a potential location for constructing unapproved
> > > > > > > executables if it is also mounted exec (W^X).
> > > > > > 
> > > > > > UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
> > > > > 
> > > > > Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
> > > > > is not universal and the policies might not contain all users or services.
> > > > > 
> > > > > -Topi
> > > > 
> > > > What's the data that supports having noexec /dev anyway? With root
> > > > access I can then just use something else like /dev/shm mount.
> > > > 
> > > > Has there been out in the wild real world cases that noexec mount
> > > > of would have prevented?
> > > > 
> > > > For me this sounds a lot just something that "feels more secure"
> > > > without any measurable benefit. Can you prove me wrong?
> > > 
> > > I don't think security works that way. An attacker has various methods to
> > > choose from, some are more interesting than others. The case where rw,exec
> > > /dev would be interesting would imply that easier or more common avenues
> > > would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
> > > /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP approach
> > > with no need for any file system access is getting more common. It does not
> > > mean that it would not be prudent to block the relatively easy approaches
> > > too, including /dev.
> > 
> > What if we add a new mount option "chrexec", which allows exec
> > for character devices (S_IFCHR).
> 
> Oh please no.

(Once more because my I was subscribed with an old email address.)

Greg's right. That's very obviously a horrible hack so this is an
instant nak from my side.

Christian

Re: [systemd-devel] Creating executable device nodes in /dev?

[Christian Brauner]

On Fri, Dec 11, 2020 at 12:29:31PM +0100, Greg KH wrote:
> On Fri, Dec 11, 2020 at 12:46:35PM +0200, Jarkko Sakkinen wrote:
> > On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
> > > On 9.12.2020 2.15, Jarkko Sakkinen wrote:
> > > > On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> > > > > > > > As  a further argument, I just did this on a Fedora system:
> > > > > > > > $ find /dev -perm /ugo+x -a \! -type d -a \! -type l
> > > > > > > > No results.  So making /dev noexec doesn't seem to have any benefit.
> > > > > > > 
> > > > > > > It's no surprise that there aren't any executables in /dev since
> > > > > > > removing MAKEDEV ages ago. That's not the issue, which is that
> > > > > > > /dev is a writable directory (for UID=0 but no capabilities are
> > > > > > > needed) and thus a potential location for constructing unapproved
> > > > > > > executables if it is also mounted exec (W^X).
> > > > > > 
> > > > > > UID 0 can just change mount options, though, unless SELinux or similar is used. And SELinux can protect /dev just fine without noexec.
> > > > > 
> > > > > Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also SELinux
> > > > > is not universal and the policies might not contain all users or services.
> > > > > 
> > > > > -Topi
> > > > 
> > > > What's the data that supports having noexec /dev anyway? With root
> > > > access I can then just use something else like /dev/shm mount.
> > > > 
> > > > Has there been out in the wild real world cases that noexec mount
> > > > of would have prevented?
> > > > 
> > > > For me this sounds a lot just something that "feels more secure"
> > > > without any measurable benefit. Can you prove me wrong?
> > > 
> > > I don't think security works that way. An attacker has various methods to
> > > choose from, some are more interesting than others. The case where rw,exec
> > > /dev would be interesting would imply that easier or more common avenues
> > > would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
> > > /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP approach
> > > with no need for any file system access is getting more common. It does not
> > > mean that it would not be prudent to block the relatively easy approaches
> > > too, including /dev.
> > 
> > What if we add a new mount option "chrexec", which allows exec
> > for character devices (S_IFCHR).
> 
> Oh please no.

(Once more because my I was subscribed with an old email address.)

Greg's right. That's very obviously a horrible hack so this is an
instant nak from my side.

Christian

Antw: [EXT] Re: [systemd-devel] Creating executable device nodes in /dev?

[Ulrich Windl]

>>> Topi Miettinen <toiwoton@gmail.com> schrieb am 11.12.2020 um 12:46 in
Nachricht
<27796c04-249e-6cf0-c3e1-0fd657a82f9c@gmail.com>:
> On 11.12.2020 12.46, Jarkko Sakkinen wrote:
>> On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
>>> On 9.12.2020 2.15, Jarkko Sakkinen wrote:
>>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>>>>>>>> As  a further argument, I just did this on a Fedora system:
>>>>>>>> $ find /dev ‑perm /ugo+x ‑a \! ‑type d ‑a \! ‑type l
>>>>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
>>>>>>>
>>>>>>> It's no surprise that there aren't any executables in /dev since
>>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
>>>>>>> needed) and thus a potential location for constructing unapproved
>>>>>>> executables if it is also mounted exec (W^X).
>>>>>>
>>>>>> UID 0 can just change mount options, though, unless SELinux or similar
is 
> used. And SELinux can protect /dev just fine without noexec.
>>>>>
>>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also
SELinux
>>>>> is not universal and the policies might not contain all users or
services.
>>>>>
>>>>> ‑Topi
>>>>
>>>> What's the data that supports having noexec /dev anyway? With root
>>>> access I can then just use something else like /dev/shm mount.
>>>>
>>>> Has there been out in the wild real world cases that noexec mount
>>>> of would have prevented?
>>>>
>>>> For me this sounds a lot just something that "feels more secure"
>>>> without any measurable benefit. Can you prove me wrong?
>>>
>>> I don't think security works that way. An attacker has various methods to
>>> choose from, some are more interesting than others. The case where
rw,exec
>>> /dev would be interesting would imply that easier or more common avenues
>>> would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
>>> /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP
approach
>>> with no need for any file system access is getting more common. It does
not
>>> mean that it would not be prudent to block the relatively easy approaches
>>> too, including /dev.
>> 
>> What if we add a new mount option "chrexec", which allows exec
>> for character devices (S_IFCHR).
> 
> I think devices are a bad match for SGX because devices haven't been 
> executable and SGX is actually an operation for memory. So something 
> like memfd_create(, MFD_SGX) or mmap(,, PROT_READ|PROT_EXEC|PROT_SGX) 
> would be much more natural. Even better would be something that 
> conceptully also works for AMD version (either with the same flags or 
> MFD_SGX / MFD_whatever_the_AMD_version_is).

+1

Antw: [EXT] Re: [systemd-devel] Creating executable device nodes in /dev?

[Ulrich Windl]

>>> Topi Miettinen <toiwoton@gmail.com> schrieb am 11.12.2020 um 12:46 in
Nachricht
<27796c04-249e-6cf0-c3e1-0fd657a82f9c@gmail.com>:
> On 11.12.2020 12.46, Jarkko Sakkinen wrote:
>> On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
>>> On 9.12.2020 2.15, Jarkko Sakkinen wrote:
>>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>>>>>>>> As  a further argument, I just did this on a Fedora system:
>>>>>>>> $ find /dev ‑perm /ugo+x ‑a \! ‑type d ‑a \! ‑type l
>>>>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
>>>>>>>
>>>>>>> It's no surprise that there aren't any executables in /dev since
>>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
>>>>>>> needed) and thus a potential location for constructing unapproved
>>>>>>> executables if it is also mounted exec (W^X).
>>>>>>
>>>>>> UID 0 can just change mount options, though, unless SELinux or similar
is 
> used. And SELinux can protect /dev just fine without noexec.
>>>>>
>>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also
SELinux
>>>>> is not universal and the policies might not contain all users or
services.
>>>>>
>>>>> ‑Topi
>>>>
>>>> What's the data that supports having noexec /dev anyway? With root
>>>> access I can then just use something else like /dev/shm mount.
>>>>
>>>> Has there been out in the wild real world cases that noexec mount
>>>> of would have prevented?
>>>>
>>>> For me this sounds a lot just something that "feels more secure"
>>>> without any measurable benefit. Can you prove me wrong?
>>>
>>> I don't think security works that way. An attacker has various methods to
>>> choose from, some are more interesting than others. The case where
rw,exec
>>> /dev would be interesting would imply that easier or more common avenues
>>> would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
>>> /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP
approach
>>> with no need for any file system access is getting more common. It does
not
>>> mean that it would not be prudent to block the relatively easy approaches
>>> too, including /dev.
>> 
>> What if we add a new mount option "chrexec", which allows exec
>> for character devices (S_IFCHR).
> 
> I think devices are a bad match for SGX because devices haven't been 
> executable and SGX is actually an operation for memory. So something 
> like memfd_create(, MFD_SGX) or mmap(,, PROT_READ|PROT_EXEC|PROT_SGX) 
> would be much more natural. Even better would be something that 
> conceptully also works for AMD version (either with the same flags or 
> MFD_SGX / MFD_whatever_the_AMD_version_is).

+1

Re: Antw: [EXT] Re: [systemd-devel] Creating executable device nodes in /dev?

[Jarkko Sakkinen]

On Mon, Dec 14, 2020 at 08:25:50AM +0100, Ulrich Windl wrote:
> >>> Topi Miettinen <toiwoton@gmail.com> schrieb am 11.12.2020 um 12:46 in
> Nachricht
> <27796c04-249e-6cf0-c3e1-0fd657a82f9c@gmail.com>:
> > On 11.12.2020 12.46, Jarkko Sakkinen wrote:
> >> On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
> >>> On 9.12.2020 2.15, Jarkko Sakkinen wrote:
> >>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> >>>>>>>> As  a further argument, I just did this on a Fedora system:
> >>>>>>>> $ find /dev ‑perm /ugo+x ‑a \! ‑type d ‑a \! ‑type l
> >>>>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
> >>>>>>>
> >>>>>>> It's no surprise that there aren't any executables in /dev since
> >>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
> >>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
> >>>>>>> needed) and thus a potential location for constructing unapproved
> >>>>>>> executables if it is also mounted exec (W^X).
> >>>>>>
> >>>>>> UID 0 can just change mount options, though, unless SELinux or similar
> is 
> > used. And SELinux can protect /dev just fine without noexec.
> >>>>>
> >>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also
> SELinux
> >>>>> is not universal and the policies might not contain all users or
> services.
> >>>>>
> >>>>> ‑Topi
> >>>>
> >>>> What's the data that supports having noexec /dev anyway? With root
> >>>> access I can then just use something else like /dev/shm mount.
> >>>>
> >>>> Has there been out in the wild real world cases that noexec mount
> >>>> of would have prevented?
> >>>>
> >>>> For me this sounds a lot just something that "feels more secure"
> >>>> without any measurable benefit. Can you prove me wrong?
> >>>
> >>> I don't think security works that way. An attacker has various methods to
> >>> choose from, some are more interesting than others. The case where
> rw,exec
> >>> /dev would be interesting would imply that easier or more common avenues
> >>> would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
> >>> /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP
> approach
> >>> with no need for any file system access is getting more common. It does
> not
> >>> mean that it would not be prudent to block the relatively easy approaches
> >>> too, including /dev.
> >> 
> >> What if we add a new mount option "chrexec", which allows exec
> >> for character devices (S_IFCHR).
> > 
> > I think devices are a bad match for SGX because devices haven't been 
> > executable and SGX is actually an operation for memory. So something 
> > like memfd_create(, MFD_SGX) or mmap(,, PROT_READ|PROT_EXEC|PROT_SGX) 
> > would be much more natural. Even better would be something that 
> > conceptully also works for AMD version (either with the same flags or 
> > MFD_SGX / MFD_whatever_the_AMD_version_is).
> 
> +1

SGX reserved memory from kernel's point of view is IO memory.

Mapping SGX to memfd would not be a great idea, as it does not map
into concept of anonymous file backed by regular memory.

A device file is very natural match actually. We have ioctl API for
uploading enclave pages during the build procedure to the enclave and
custom #PF handler. Conceptually it's a lot like video memory or such
special device specific memory area.

There's no AMD equivalent of this technology.

/Jarkko

Re: Antw: [EXT] Re: [systemd-devel] Creating executable device nodes in /dev?

[Jarkko Sakkinen]

On Mon, Dec 14, 2020 at 08:25:50AM +0100, Ulrich Windl wrote:
> >>> Topi Miettinen <toiwoton@gmail.com> schrieb am 11.12.2020 um 12:46 in
> Nachricht
> <27796c04-249e-6cf0-c3e1-0fd657a82f9c@gmail.com>:
> > On 11.12.2020 12.46, Jarkko Sakkinen wrote:
> >> On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
> >>> On 9.12.2020 2.15, Jarkko Sakkinen wrote:
> >>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> >>>>>>>> As  a further argument, I just did this on a Fedora system:
> >>>>>>>> $ find /dev ‑perm /ugo+x ‑a \! ‑type d ‑a \! ‑type l
> >>>>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
> >>>>>>>
> >>>>>>> It's no surprise that there aren't any executables in /dev since
> >>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
> >>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
> >>>>>>> needed) and thus a potential location for constructing unapproved
> >>>>>>> executables if it is also mounted exec (W^X).
> >>>>>>
> >>>>>> UID 0 can just change mount options, though, unless SELinux or similar
> is 
> > used. And SELinux can protect /dev just fine without noexec.
> >>>>>
> >>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also
> SELinux
> >>>>> is not universal and the policies might not contain all users or
> services.
> >>>>>
> >>>>> ‑Topi
> >>>>
> >>>> What's the data that supports having noexec /dev anyway? With root
> >>>> access I can then just use something else like /dev/shm mount.
> >>>>
> >>>> Has there been out in the wild real world cases that noexec mount
> >>>> of would have prevented?
> >>>>
> >>>> For me this sounds a lot just something that "feels more secure"
> >>>> without any measurable benefit. Can you prove me wrong?
> >>>
> >>> I don't think security works that way. An attacker has various methods to
> >>> choose from, some are more interesting than others. The case where
> rw,exec
> >>> /dev would be interesting would imply that easier or more common avenues
> >>> would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
> >>> /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP
> approach
> >>> with no need for any file system access is getting more common. It does
> not
> >>> mean that it would not be prudent to block the relatively easy approaches
> >>> too, including /dev.
> >> 
> >> What if we add a new mount option "chrexec", which allows exec
> >> for character devices (S_IFCHR).
> > 
> > I think devices are a bad match for SGX because devices haven't been 
> > executable and SGX is actually an operation for memory. So something 
> > like memfd_create(, MFD_SGX) or mmap(,, PROT_READ|PROT_EXEC|PROT_SGX) 
> > would be much more natural. Even better would be something that 
> > conceptully also works for AMD version (either with the same flags or 
> > MFD_SGX / MFD_whatever_the_AMD_version_is).
> 
> +1

SGX reserved memory from kernel's point of view is IO memory.

Mapping SGX to memfd would not be a great idea, as it does not map
into concept of anonymous file backed by regular memory.

A device file is very natural match actually. We have ioctl API for
uploading enclave pages during the build procedure to the enclave and
custom #PF handler. Conceptually it's a lot like video memory or such
special device specific memory area.

There's no AMD equivalent of this technology.

/Jarkko

Re: Antw: [EXT] Re: [systemd-devel] Creating executable device nodes in /dev?

[Jarkko Sakkinen]

On Tue, Dec 15, 2020 at 06:19:09AM +0200, Jarkko Sakkinen wrote:
> On Mon, Dec 14, 2020 at 08:25:50AM +0100, Ulrich Windl wrote:
> > >>> Topi Miettinen <toiwoton@gmail.com> schrieb am 11.12.2020 um 12:46 in
> > Nachricht
> > <27796c04-249e-6cf0-c3e1-0fd657a82f9c@gmail.com>:
> > > On 11.12.2020 12.46, Jarkko Sakkinen wrote:
> > >> On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
> > >>> On 9.12.2020 2.15, Jarkko Sakkinen wrote:
> > >>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> > >>>>>>>> As  a further argument, I just did this on a Fedora system:
> > >>>>>>>> $ find /dev ‑perm /ugo+x ‑a \! ‑type d ‑a \! ‑type l
> > >>>>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
> > >>>>>>>
> > >>>>>>> It's no surprise that there aren't any executables in /dev since
> > >>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
> > >>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
> > >>>>>>> needed) and thus a potential location for constructing unapproved
> > >>>>>>> executables if it is also mounted exec (W^X).
> > >>>>>>
> > >>>>>> UID 0 can just change mount options, though, unless SELinux or similar
> > is 
> > > used. And SELinux can protect /dev just fine without noexec.
> > >>>>>
> > >>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also
> > SELinux
> > >>>>> is not universal and the policies might not contain all users or
> > services.
> > >>>>>
> > >>>>> ‑Topi
> > >>>>
> > >>>> What's the data that supports having noexec /dev anyway? With root
> > >>>> access I can then just use something else like /dev/shm mount.
> > >>>>
> > >>>> Has there been out in the wild real world cases that noexec mount
> > >>>> of would have prevented?
> > >>>>
> > >>>> For me this sounds a lot just something that "feels more secure"
> > >>>> without any measurable benefit. Can you prove me wrong?
> > >>>
> > >>> I don't think security works that way. An attacker has various methods to
> > >>> choose from, some are more interesting than others. The case where
> > rw,exec
> > >>> /dev would be interesting would imply that easier or more common avenues
> > >>> would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
> > >>> /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP
> > approach
> > >>> with no need for any file system access is getting more common. It does
> > not
> > >>> mean that it would not be prudent to block the relatively easy approaches
> > >>> too, including /dev.
> > >> 
> > >> What if we add a new mount option "chrexec", which allows exec
> > >> for character devices (S_IFCHR).
> > > 
> > > I think devices are a bad match for SGX because devices haven't been 
> > > executable and SGX is actually an operation for memory. So something 
> > > like memfd_create(, MFD_SGX) or mmap(,, PROT_READ|PROT_EXEC|PROT_SGX) 
> > > would be much more natural. Even better would be something that 
> > > conceptully also works for AMD version (either with the same flags or 
> > > MFD_SGX / MFD_whatever_the_AMD_version_is).
> > 
> > +1
> 
> SGX reserved memory from kernel's point of view is IO memory.
> 
> Mapping SGX to memfd would not be a great idea, as it does not map
> into concept of anonymous file backed by regular memory.
> 
> A device file is very natural match actually. We have ioctl API for
> uploading enclave pages during the build procedure to the enclave and
> custom #PF handler. Conceptually it's a lot like video memory or such
> special device specific memory area.
> 
> There's no AMD equivalent of this technology.

Anyway, I take a not on "PROT_SGX" as one of the ways sort this out in
the future. That would at least fit what we have. Thanks for all the
feedback.

/Jarkko

Re: Antw: [EXT] Re: [systemd-devel] Creating executable device nodes in /dev?

[Jarkko Sakkinen]

On Tue, Dec 15, 2020 at 06:19:09AM +0200, Jarkko Sakkinen wrote:
> On Mon, Dec 14, 2020 at 08:25:50AM +0100, Ulrich Windl wrote:
> > >>> Topi Miettinen <toiwoton@gmail.com> schrieb am 11.12.2020 um 12:46 in
> > Nachricht
> > <27796c04-249e-6cf0-c3e1-0fd657a82f9c@gmail.com>:
> > > On 11.12.2020 12.46, Jarkko Sakkinen wrote:
> > >> On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
> > >>> On 9.12.2020 2.15, Jarkko Sakkinen wrote:
> > >>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> > >>>>>>>> As  a further argument, I just did this on a Fedora system:
> > >>>>>>>> $ find /dev ‑perm /ugo+x ‑a \! ‑type d ‑a \! ‑type l
> > >>>>>>>> No results.  So making /dev noexec doesn't seem to have any benefit.
> > >>>>>>>
> > >>>>>>> It's no surprise that there aren't any executables in /dev since
> > >>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
> > >>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
> > >>>>>>> needed) and thus a potential location for constructing unapproved
> > >>>>>>> executables if it is also mounted exec (W^X).
> > >>>>>>
> > >>>>>> UID 0 can just change mount options, though, unless SELinux or similar
> > is 
> > > used. And SELinux can protect /dev just fine without noexec.
> > >>>>>
> > >>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also
> > SELinux
> > >>>>> is not universal and the policies might not contain all users or
> > services.
> > >>>>>
> > >>>>> ‑Topi
> > >>>>
> > >>>> What's the data that supports having noexec /dev anyway? With root
> > >>>> access I can then just use something else like /dev/shm mount.
> > >>>>
> > >>>> Has there been out in the wild real world cases that noexec mount
> > >>>> of would have prevented?
> > >>>>
> > >>>> For me this sounds a lot just something that "feels more secure"
> > >>>> without any measurable benefit. Can you prove me wrong?
> > >>>
> > >>> I don't think security works that way. An attacker has various methods to
> > >>> choose from, some are more interesting than others. The case where
> > rw,exec
> > >>> /dev would be interesting would imply that easier or more common avenues
> > >>> would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
> > >>> /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP
> > approach
> > >>> with no need for any file system access is getting more common. It does
> > not
> > >>> mean that it would not be prudent to block the relatively easy approaches
> > >>> too, including /dev.
> > >> 
> > >> What if we add a new mount option "chrexec", which allows exec
> > >> for character devices (S_IFCHR).
> > > 
> > > I think devices are a bad match for SGX because devices haven't been 
> > > executable and SGX is actually an operation for memory. So something 
> > > like memfd_create(, MFD_SGX) or mmap(,, PROT_READ|PROT_EXEC|PROT_SGX) 
> > > would be much more natural. Even better would be something that 
> > > conceptully also works for AMD version (either with the same flags or 
> > > MFD_SGX / MFD_whatever_the_AMD_version_is).
> > 
> > +1
> 
> SGX reserved memory from kernel's point of view is IO memory.
> 
> Mapping SGX to memfd would not be a great idea, as it does not map
> into concept of anonymous file backed by regular memory.
> 
> A device file is very natural match actually. We have ioctl API for
> uploading enclave pages during the build procedure to the enclave and
> custom #PF handler. Conceptually it's a lot like video memory or such
> special device specific memory area.
> 
> There's no AMD equivalent of this technology.

Anyway, I take a not on "PROT_SGX" as one of the ways sort this out in
the future. That would at least fit what we have. Thanks for all the
feedback.

/Jarkko

Re: Antw: [EXT] Re: [systemd-devel] Creating executable device nodes in /dev?

[Ulrich Windl]

>>> Jarkko Sakkinen <jarkko@kernel.org> schrieb am 15.12.2020 um 05:19 in
Nachricht
<20201215041903.GA21875@kernel.org>:
> On Mon, Dec 14, 2020 at 08:25:50AM +0100, Ulrich Windl wrote:
>> >>> Topi Miettinen <toiwoton@gmail.com> schrieb am 11.12.2020 um 12:46 in
>> Nachricht
>> <27796c04-249e-6cf0-c3e1-0fd657a82f9c@gmail.com>:
>> > On 11.12.2020 12.46, Jarkko Sakkinen wrote:
>> >> On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
>> >>> On 9.12.2020 2.15, Jarkko Sakkinen wrote:
>> >>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>> >>>>>>>> As  a further argument, I just did this on a Fedora system:
>> >>>>>>>> $ find /dev ‑perm /ugo+x ‑a \! ‑type d ‑a \! ‑type l
>> >>>>>>>> No results.  So making /dev noexec doesn't seem to have any
benefit.
>> >>>>>>>
>> >>>>>>> It's no surprise that there aren't any executables in /dev since
>> >>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>> >>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
>> >>>>>>> needed) and thus a potential location for constructing unapproved
>> >>>>>>> executables if it is also mounted exec (W^X).
>> >>>>>>
>> >>>>>> UID 0 can just change mount options, though, unless SELinux or
similar
>> is 
>> > used. And SELinux can protect /dev just fine without noexec.
>> >>>>>
>> >>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also
>> SELinux
>> >>>>> is not universal and the policies might not contain all users or
>> services.
>> >>>>>
>> >>>>> ‑Topi
>> >>>>
>> >>>> What's the data that supports having noexec /dev anyway? With root
>> >>>> access I can then just use something else like /dev/shm mount.
>> >>>>
>> >>>> Has there been out in the wild real world cases that noexec mount
>> >>>> of would have prevented?
>> >>>>
>> >>>> For me this sounds a lot just something that "feels more secure"
>> >>>> without any measurable benefit. Can you prove me wrong?
>> >>>
>> >>> I don't think security works that way. An attacker has various methods
to
>> >>> choose from, some are more interesting than others. The case where
>> rw,exec
>> >>> /dev would be interesting would imply that easier or more common
avenues
>> >>> would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
>> >>> /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP
>> approach
>> >>> with no need for any file system access is getting more common. It
does
>> not
>> >>> mean that it would not be prudent to block the relatively easy
approaches
>> >>> too, including /dev.
>> >> 
>> >> What if we add a new mount option "chrexec", which allows exec
>> >> for character devices (S_IFCHR).
>> > 
>> > I think devices are a bad match for SGX because devices haven't been 
>> > executable and SGX is actually an operation for memory. So something 
>> > like memfd_create(, MFD_SGX) or mmap(,, PROT_READ|PROT_EXEC|PROT_SGX) 
>> > would be much more natural. Even better would be something that 
>> > conceptully also works for AMD version (either with the same flags or 
>> > MFD_SGX / MFD_whatever_the_AMD_version_is).
>> 
>> +1
> 
> SGX reserved memory from kernel's point of view is IO memory.
> 
> Mapping SGX to memfd would not be a great idea, as it does not map
> into concept of anonymous file backed by regular memory.
> 
> A device file is very natural match actually. We have ioctl API for
> uploading enclave pages during the build procedure to the enclave and
> custom #PF handler. Conceptually it's a lot like video memory or such
> special device specific memory area.
> 
> There's no AMD equivalent of this technology.

Hi!

Back to "noexec": AFAIR the execute bit does not make sense for device files,
and the purpose probably was to avoid execution of non-device files (e.g.
regular executables) from inside /dev (where they should not be). So in this
view "noexec" makes sense.
There were similar arguments for not allowing device files in user
directories.

Regards,
Ulrich

> 
> /Jarkko

Re: Antw: [EXT] Re: [systemd-devel] Creating executable device nodes in /dev?

[Ulrich Windl]

>>> Jarkko Sakkinen <jarkko@kernel.org> schrieb am 15.12.2020 um 05:19 in
Nachricht
<20201215041903.GA21875@kernel.org>:
> On Mon, Dec 14, 2020 at 08:25:50AM +0100, Ulrich Windl wrote:
>> >>> Topi Miettinen <toiwoton@gmail.com> schrieb am 11.12.2020 um 12:46 in
>> Nachricht
>> <27796c04-249e-6cf0-c3e1-0fd657a82f9c@gmail.com>:
>> > On 11.12.2020 12.46, Jarkko Sakkinen wrote:
>> >> On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
>> >>> On 9.12.2020 2.15, Jarkko Sakkinen wrote:
>> >>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>> >>>>>>>> As  a further argument, I just did this on a Fedora system:
>> >>>>>>>> $ find /dev ‑perm /ugo+x ‑a \! ‑type d ‑a \! ‑type l
>> >>>>>>>> No results.  So making /dev noexec doesn't seem to have any
benefit.
>> >>>>>>>
>> >>>>>>> It's no surprise that there aren't any executables in /dev since
>> >>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>> >>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
>> >>>>>>> needed) and thus a potential location for constructing unapproved
>> >>>>>>> executables if it is also mounted exec (W^X).
>> >>>>>>
>> >>>>>> UID 0 can just change mount options, though, unless SELinux or
similar
>> is 
>> > used. And SELinux can protect /dev just fine without noexec.
>> >>>>>
>> >>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also
>> SELinux
>> >>>>> is not universal and the policies might not contain all users or
>> services.
>> >>>>>
>> >>>>> ‑Topi
>> >>>>
>> >>>> What's the data that supports having noexec /dev anyway? With root
>> >>>> access I can then just use something else like /dev/shm mount.
>> >>>>
>> >>>> Has there been out in the wild real world cases that noexec mount
>> >>>> of would have prevented?
>> >>>>
>> >>>> For me this sounds a lot just something that "feels more secure"
>> >>>> without any measurable benefit. Can you prove me wrong?
>> >>>
>> >>> I don't think security works that way. An attacker has various methods
to
>> >>> choose from, some are more interesting than others. The case where
>> rw,exec
>> >>> /dev would be interesting would imply that easier or more common
avenues
>> >>> would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
>> >>> /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP
>> approach
>> >>> with no need for any file system access is getting more common. It
does
>> not
>> >>> mean that it would not be prudent to block the relatively easy
approaches
>> >>> too, including /dev.
>> >> 
>> >> What if we add a new mount option "chrexec", which allows exec
>> >> for character devices (S_IFCHR).
>> > 
>> > I think devices are a bad match for SGX because devices haven't been 
>> > executable and SGX is actually an operation for memory. So something 
>> > like memfd_create(, MFD_SGX) or mmap(,, PROT_READ|PROT_EXEC|PROT_SGX) 
>> > would be much more natural. Even better would be something that 
>> > conceptully also works for AMD version (either with the same flags or 
>> > MFD_SGX / MFD_whatever_the_AMD_version_is).
>> 
>> +1
> 
> SGX reserved memory from kernel's point of view is IO memory.
> 
> Mapping SGX to memfd would not be a great idea, as it does not map
> into concept of anonymous file backed by regular memory.
> 
> A device file is very natural match actually. We have ioctl API for
> uploading enclave pages during the build procedure to the enclave and
> custom #PF handler. Conceptually it's a lot like video memory or such
> special device specific memory area.
> 
> There's no AMD equivalent of this technology.

Hi!

Back to "noexec": AFAIR the execute bit does not make sense for device files,
and the purpose probably was to avoid execution of non-device files (e.g.
regular executables) from inside /dev (where they should not be). So in this
view "noexec" makes sense.
There were similar arguments for not allowing device files in user
directories.

Regards,
Ulrich

> 
> /Jarkko

Re: Antw: [EXT] Re: [systemd-devel] Creating executable device nodes in /dev?

[Topi Miettinen]

On 16.12.2020 12.03, Ulrich Windl wrote:
>>>> Jarkko Sakkinen <jarkko@kernel.org> schrieb am 15.12.2020 um 05:19 in
> Nachricht
> <20201215041903.GA21875@kernel.org>:
>> On Mon, Dec 14, 2020 at 08:25:50AM +0100, Ulrich Windl wrote:
>>>>>> Topi Miettinen <toiwoton@gmail.com> schrieb am 11.12.2020 um 12:46 in
>>> Nachricht
>>> <27796c04-249e-6cf0-c3e1-0fd657a82f9c@gmail.com>:
>>>> On 11.12.2020 12.46, Jarkko Sakkinen wrote:
>>>>> On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
>>>>>> On 9.12.2020 2.15, Jarkko Sakkinen wrote:
>>>>>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>>>>>>>>>>> As  a further argument, I just did this on a Fedora system:
>>>>>>>>>>> $ find /dev ‑perm /ugo+x ‑a \! ‑type d ‑a \! ‑type l
>>>>>>>>>>> No results.  So making /dev noexec doesn't seem to have any
> benefit.
>>>>>>>>>>
>>>>>>>>>> It's no surprise that there aren't any executables in /dev since
>>>>>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>>>>>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
>>>>>>>>>> needed) and thus a potential location for constructing unapproved
>>>>>>>>>> executables if it is also mounted exec (W^X).
>>>>>>>>>
>>>>>>>>> UID 0 can just change mount options, though, unless SELinux or
> similar
>>> is
>>>> used. And SELinux can protect /dev just fine without noexec.
>>>>>>>>
>>>>>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also
>>> SELinux
>>>>>>>> is not universal and the policies might not contain all users or
>>> services.
>>>>>>>>
>>>>>>>> ‑Topi
>>>>>>>
>>>>>>> What's the data that supports having noexec /dev anyway? With root
>>>>>>> access I can then just use something else like /dev/shm mount.
>>>>>>>
>>>>>>> Has there been out in the wild real world cases that noexec mount
>>>>>>> of would have prevented?
>>>>>>>
>>>>>>> For me this sounds a lot just something that "feels more secure"
>>>>>>> without any measurable benefit. Can you prove me wrong?
>>>>>>
>>>>>> I don't think security works that way. An attacker has various methods
> to
>>>>>> choose from, some are more interesting than others. The case where
>>> rw,exec
>>>>>> /dev would be interesting would imply that easier or more common
> avenues
>>>>>> would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
>>>>>> /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP
>>> approach
>>>>>> with no need for any file system access is getting more common. It
> does
>>> not
>>>>>> mean that it would not be prudent to block the relatively easy
> approaches
>>>>>> too, including /dev.
>>>>>
>>>>> What if we add a new mount option "chrexec", which allows exec
>>>>> for character devices (S_IFCHR).
>>>>
>>>> I think devices are a bad match for SGX because devices haven't been
>>>> executable and SGX is actually an operation for memory. So something
>>>> like memfd_create(, MFD_SGX) or mmap(,, PROT_READ|PROT_EXEC|PROT_SGX)
>>>> would be much more natural. Even better would be something that
>>>> conceptully also works for AMD version (either with the same flags or
>>>> MFD_SGX / MFD_whatever_the_AMD_version_is).
>>>
>>> +1
>>
>> SGX reserved memory from kernel's point of view is IO memory.
>>
>> Mapping SGX to memfd would not be a great idea, as it does not map
>> into concept of anonymous file backed by regular memory.
>>
>> A device file is very natural match actually. We have ioctl API for
>> uploading enclave pages during the build procedure to the enclave and
>> custom #PF handler. Conceptually it's a lot like video memory or such
>> special device specific memory area.
>>
>> There's no AMD equivalent of this technology.
> 
> Hi!
> 
> Back to "noexec": AFAIR the execute bit does not make sense for device files,
> and the purpose probably was to avoid execution of non-device files (e.g.
> regular executables) from inside /dev (where they should not be). So in this
> view "noexec" makes sense.
> There were similar arguments for not allowing device files in user
> directories.

PR#17940 (https://github.com/systemd/systemd/pull/17940) was merged, so 
/dev will now on be mounted with "exec" by systemd.

I made issue #17942 (https://github.com/systemd/systemd/issues/17942) to 
discuss related hardening options. I'm leaning towards 
NoExecPaths=/ExecPaths= as it would enable nice hardening by 
allow-listing of all executable content for system services with simple 
directives like:

[Service]
NoExecPaths=/
ExecPaths=/usr/sbin/daemon /lib64/ld-linux-x86-64.so.2 /usr/lib

Then a service infected with malware would not be able to execute a 
shell present in the system or downloaded later, if that was not 
explicitly allowed. /dev would also not have "exec" flag by default, but 
SGX could be allowed with "ExecPaths=/dev/sgx" when needed.

-Topi

Re: Antw: [EXT] Re: [systemd-devel] Creating executable device nodes in /dev?

[Topi Miettinen]

On 16.12.2020 12.03, Ulrich Windl wrote:
>>>> Jarkko Sakkinen <jarkko@kernel.org> schrieb am 15.12.2020 um 05:19 in
> Nachricht
> <20201215041903.GA21875@kernel.org>:
>> On Mon, Dec 14, 2020 at 08:25:50AM +0100, Ulrich Windl wrote:
>>>>>> Topi Miettinen <toiwoton@gmail.com> schrieb am 11.12.2020 um 12:46 in
>>> Nachricht
>>> <27796c04-249e-6cf0-c3e1-0fd657a82f9c@gmail.com>:
>>>> On 11.12.2020 12.46, Jarkko Sakkinen wrote:
>>>>> On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
>>>>>> On 9.12.2020 2.15, Jarkko Sakkinen wrote:
>>>>>>> On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
>>>>>>>>>>> As  a further argument, I just did this on a Fedora system:
>>>>>>>>>>> $ find /dev ‑perm /ugo+x ‑a \! ‑type d ‑a \! ‑type l
>>>>>>>>>>> No results.  So making /dev noexec doesn't seem to have any
> benefit.
>>>>>>>>>>
>>>>>>>>>> It's no surprise that there aren't any executables in /dev since
>>>>>>>>>> removing MAKEDEV ages ago. That's not the issue, which is that
>>>>>>>>>> /dev is a writable directory (for UID=0 but no capabilities are
>>>>>>>>>> needed) and thus a potential location for constructing unapproved
>>>>>>>>>> executables if it is also mounted exec (W^X).
>>>>>>>>>
>>>>>>>>> UID 0 can just change mount options, though, unless SELinux or
> similar
>>> is
>>>> used. And SELinux can protect /dev just fine without noexec.
>>>>>>>>
>>>>>>>> Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also
>>> SELinux
>>>>>>>> is not universal and the policies might not contain all users or
>>> services.
>>>>>>>>
>>>>>>>> ‑Topi
>>>>>>>
>>>>>>> What's the data that supports having noexec /dev anyway? With root
>>>>>>> access I can then just use something else like /dev/shm mount.
>>>>>>>
>>>>>>> Has there been out in the wild real world cases that noexec mount
>>>>>>> of would have prevented?
>>>>>>>
>>>>>>> For me this sounds a lot just something that "feels more secure"
>>>>>>> without any measurable benefit. Can you prove me wrong?
>>>>>>
>>>>>> I don't think security works that way. An attacker has various methods
> to
>>>>>> choose from, some are more interesting than others. The case where
>>> rw,exec
>>>>>> /dev would be interesting would imply that easier or more common
> avenues
>>>>>> would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
>>>>>> /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP
>>> approach
>>>>>> with no need for any file system access is getting more common. It
> does
>>> not
>>>>>> mean that it would not be prudent to block the relatively easy
> approaches
>>>>>> too, including /dev.
>>>>>
>>>>> What if we add a new mount option "chrexec", which allows exec
>>>>> for character devices (S_IFCHR).
>>>>
>>>> I think devices are a bad match for SGX because devices haven't been
>>>> executable and SGX is actually an operation for memory. So something
>>>> like memfd_create(, MFD_SGX) or mmap(,, PROT_READ|PROT_EXEC|PROT_SGX)
>>>> would be much more natural. Even better would be something that
>>>> conceptully also works for AMD version (either with the same flags or
>>>> MFD_SGX / MFD_whatever_the_AMD_version_is).
>>>
>>> +1
>>
>> SGX reserved memory from kernel's point of view is IO memory.
>>
>> Mapping SGX to memfd would not be a great idea, as it does not map
>> into concept of anonymous file backed by regular memory.
>>
>> A device file is very natural match actually. We have ioctl API for
>> uploading enclave pages during the build procedure to the enclave and
>> custom #PF handler. Conceptually it's a lot like video memory or such
>> special device specific memory area.
>>
>> There's no AMD equivalent of this technology.
> 
> Hi!
> 
> Back to "noexec": AFAIR the execute bit does not make sense for device files,
> and the purpose probably was to avoid execution of non-device files (e.g.
> regular executables) from inside /dev (where they should not be). So in this
> view "noexec" makes sense.
> There were similar arguments for not allowing device files in user
> directories.

PR#17940 (https://github.com/systemd/systemd/pull/17940) was merged, so 
/dev will now on be mounted with "exec" by systemd.

I made issue #17942 (https://github.com/systemd/systemd/issues/17942) to 
discuss related hardening options. I'm leaning towards 
NoExecPaths=/ExecPaths= as it would enable nice hardening by 
allow-listing of all executable content for system services with simple 
directives like:

[Service]
NoExecPaths=/
ExecPaths=/usr/sbin/daemon /lib64/ld-linux-x86-64.so.2 /usr/lib

Then a service infected with malware would not be able to execute a 
shell present in the system or downloaded later, if that was not 
explicitly allowed. /dev would also not have "exec" flag by default, but 
SGX could be allowed with "ExecPaths=/dev/sgx" when needed.

-Topi

Re: Antw: [EXT] Re: [systemd-devel] Creating executable device nodes in /dev?

[Jarkko Sakkinen]

On Wed, Dec 16, 2020 at 03:05:05PM +0200, Topi Miettinen wrote:
> On 16.12.2020 12.03, Ulrich Windl wrote:
> > > > > Jarkko Sakkinen <jarkko@kernel.org> schrieb am 15.12.2020 um 05:19 in
> > Nachricht
> > <20201215041903.GA21875@kernel.org>:
> > > On Mon, Dec 14, 2020 at 08:25:50AM +0100, Ulrich Windl wrote:
> > > > > > > Topi Miettinen <toiwoton@gmail.com> schrieb am 11.12.2020 um 12:46 in
> > > > Nachricht
> > > > <27796c04-249e-6cf0-c3e1-0fd657a82f9c@gmail.com>:
> > > > > On 11.12.2020 12.46, Jarkko Sakkinen wrote:
> > > > > > On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
> > > > > > > On 9.12.2020 2.15, Jarkko Sakkinen wrote:
> > > > > > > > On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> > > > > > > > > > > > As  a further argument, I just did this on a Fedora system:
> > > > > > > > > > > > $ find /dev ‑perm /ugo+x ‑a \! ‑type d ‑a \! ‑type l
> > > > > > > > > > > > No results.  So making /dev noexec doesn't seem to have any
> > benefit.
> > > > > > > > > > > 
> > > > > > > > > > > It's no surprise that there aren't any executables in /dev since
> > > > > > > > > > > removing MAKEDEV ages ago. That's not the issue, which is that
> > > > > > > > > > > /dev is a writable directory (for UID=0 but no capabilities are
> > > > > > > > > > > needed) and thus a potential location for constructing unapproved
> > > > > > > > > > > executables if it is also mounted exec (W^X).
> > > > > > > > > > 
> > > > > > > > > > UID 0 can just change mount options, though, unless SELinux or
> > similar
> > > > is
> > > > > used. And SELinux can protect /dev just fine without noexec.
> > > > > > > > > 
> > > > > > > > > Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also
> > > > SELinux
> > > > > > > > > is not universal and the policies might not contain all users or
> > > > services.
> > > > > > > > > 
> > > > > > > > > ‑Topi
> > > > > > > > 
> > > > > > > > What's the data that supports having noexec /dev anyway? With root
> > > > > > > > access I can then just use something else like /dev/shm mount.
> > > > > > > > 
> > > > > > > > Has there been out in the wild real world cases that noexec mount
> > > > > > > > of would have prevented?
> > > > > > > > 
> > > > > > > > For me this sounds a lot just something that "feels more secure"
> > > > > > > > without any measurable benefit. Can you prove me wrong?
> > > > > > > 
> > > > > > > I don't think security works that way. An attacker has various methods
> > to
> > > > > > > choose from, some are more interesting than others. The case where
> > > > rw,exec
> > > > > > > /dev would be interesting would imply that easier or more common
> > avenues
> > > > > > > would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
> > > > > > > /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP
> > > > approach
> > > > > > > with no need for any file system access is getting more common. It
> > does
> > > > not
> > > > > > > mean that it would not be prudent to block the relatively easy
> > approaches
> > > > > > > too, including /dev.
> > > > > > 
> > > > > > What if we add a new mount option "chrexec", which allows exec
> > > > > > for character devices (S_IFCHR).
> > > > > 
> > > > > I think devices are a bad match for SGX because devices haven't been
> > > > > executable and SGX is actually an operation for memory. So something
> > > > > like memfd_create(, MFD_SGX) or mmap(,, PROT_READ|PROT_EXEC|PROT_SGX)
> > > > > would be much more natural. Even better would be something that
> > > > > conceptully also works for AMD version (either with the same flags or
> > > > > MFD_SGX / MFD_whatever_the_AMD_version_is).
> > > > 
> > > > +1
> > > 
> > > SGX reserved memory from kernel's point of view is IO memory.
> > > 
> > > Mapping SGX to memfd would not be a great idea, as it does not map
> > > into concept of anonymous file backed by regular memory.
> > > 
> > > A device file is very natural match actually. We have ioctl API for
> > > uploading enclave pages during the build procedure to the enclave and
> > > custom #PF handler. Conceptually it's a lot like video memory or such
> > > special device specific memory area.
> > > 
> > > There's no AMD equivalent of this technology.
> > 
> > Hi!
> > 
> > Back to "noexec": AFAIR the execute bit does not make sense for device files,
> > and the purpose probably was to avoid execution of non-device files (e.g.
> > regular executables) from inside /dev (where they should not be). So in this
> > view "noexec" makes sense.
> > There were similar arguments for not allowing device files in user
> > directories.
> 
> PR#17940 (https://github.com/systemd/systemd/pull/17940) was merged, so /dev
> will now on be mounted with "exec" by systemd.
> 
> I made issue #17942 (https://github.com/systemd/systemd/issues/17942) to
> discuss related hardening options. I'm leaning towards
> NoExecPaths=/ExecPaths= as it would enable nice hardening by allow-listing
> of all executable content for system services with simple directives like:
> 
> [Service]
> NoExecPaths=/
> ExecPaths=/usr/sbin/daemon /lib64/ld-linux-x86-64.so.2 /usr/lib
> 
> Then a service infected with malware would not be able to execute a shell
> present in the system or downloaded later, if that was not explicitly
> allowed. /dev would also not have "exec" flag by default, but SGX could be
> allowed with "ExecPaths=/dev/sgx" when needed.

OK, this sounds relieving, thank you (late resp, I was vacation last
week).


> -Topi


/Jarkko

Re: Antw: [EXT] Re: [systemd-devel] Creating executable device nodes in /dev?

[Jarkko Sakkinen]

On Wed, Dec 16, 2020 at 03:05:05PM +0200, Topi Miettinen wrote:
> On 16.12.2020 12.03, Ulrich Windl wrote:
> > > > > Jarkko Sakkinen <jarkko@kernel.org> schrieb am 15.12.2020 um 05:19 in
> > Nachricht
> > <20201215041903.GA21875@kernel.org>:
> > > On Mon, Dec 14, 2020 at 08:25:50AM +0100, Ulrich Windl wrote:
> > > > > > > Topi Miettinen <toiwoton@gmail.com> schrieb am 11.12.2020 um 12:46 in
> > > > Nachricht
> > > > <27796c04-249e-6cf0-c3e1-0fd657a82f9c@gmail.com>:
> > > > > On 11.12.2020 12.46, Jarkko Sakkinen wrote:
> > > > > > On Wed, Dec 09, 2020 at 10:35:21AM +0200, Topi Miettinen wrote:
> > > > > > > On 9.12.2020 2.15, Jarkko Sakkinen wrote:
> > > > > > > > On Wed, Dec 09, 2020 at 01:15:27AM +0200, Topi Miettinen wrote:
> > > > > > > > > > > > As  a further argument, I just did this on a Fedora system:
> > > > > > > > > > > > $ find /dev ‑perm /ugo+x ‑a \! ‑type d ‑a \! ‑type l
> > > > > > > > > > > > No results.  So making /dev noexec doesn't seem to have any
> > benefit.
> > > > > > > > > > > 
> > > > > > > > > > > It's no surprise that there aren't any executables in /dev since
> > > > > > > > > > > removing MAKEDEV ages ago. That's not the issue, which is that
> > > > > > > > > > > /dev is a writable directory (for UID=0 but no capabilities are
> > > > > > > > > > > needed) and thus a potential location for constructing unapproved
> > > > > > > > > > > executables if it is also mounted exec (W^X).
> > > > > > > > > > 
> > > > > > > > > > UID 0 can just change mount options, though, unless SELinux or
> > similar
> > > > is
> > > > > used. And SELinux can protect /dev just fine without noexec.
> > > > > > > > > 
> > > > > > > > > Well, mounting would need CAP_SYS_ADMIN in addition to UID 0. Also
> > > > SELinux
> > > > > > > > > is not universal and the policies might not contain all users or
> > > > services.
> > > > > > > > > 
> > > > > > > > > ‑Topi
> > > > > > > > 
> > > > > > > > What's the data that supports having noexec /dev anyway? With root
> > > > > > > > access I can then just use something else like /dev/shm mount.
> > > > > > > > 
> > > > > > > > Has there been out in the wild real world cases that noexec mount
> > > > > > > > of would have prevented?
> > > > > > > > 
> > > > > > > > For me this sounds a lot just something that "feels more secure"
> > > > > > > > without any measurable benefit. Can you prove me wrong?
> > > > > > > 
> > > > > > > I don't think security works that way. An attacker has various methods
> > to
> > > > > > > choose from, some are more interesting than others. The case where
> > > > rw,exec
> > > > > > > /dev would be interesting would imply that easier or more common
> > avenues
> > > > > > > would be blocked, for example rw,exec /dev/shm, /tmp, /var/tmp, or
> > > > > > > /run/user/$UID/ for user. Also fileless malware with pure ROP/JOP
> > > > approach
> > > > > > > with no need for any file system access is getting more common. It
> > does
> > > > not
> > > > > > > mean that it would not be prudent to block the relatively easy
> > approaches
> > > > > > > too, including /dev.
> > > > > > 
> > > > > > What if we add a new mount option "chrexec", which allows exec
> > > > > > for character devices (S_IFCHR).
> > > > > 
> > > > > I think devices are a bad match for SGX because devices haven't been
> > > > > executable and SGX is actually an operation for memory. So something
> > > > > like memfd_create(, MFD_SGX) or mmap(,, PROT_READ|PROT_EXEC|PROT_SGX)
> > > > > would be much more natural. Even better would be something that
> > > > > conceptully also works for AMD version (either with the same flags or
> > > > > MFD_SGX / MFD_whatever_the_AMD_version_is).
> > > > 
> > > > +1
> > > 
> > > SGX reserved memory from kernel's point of view is IO memory.
> > > 
> > > Mapping SGX to memfd would not be a great idea, as it does not map
> > > into concept of anonymous file backed by regular memory.
> > > 
> > > A device file is very natural match actually. We have ioctl API for
> > > uploading enclave pages during the build procedure to the enclave and
> > > custom #PF handler. Conceptually it's a lot like video memory or such
> > > special device specific memory area.
> > > 
> > > There's no AMD equivalent of this technology.
> > 
> > Hi!
> > 
> > Back to "noexec": AFAIR the execute bit does not make sense for device files,
> > and the purpose probably was to avoid execution of non-device files (e.g.
> > regular executables) from inside /dev (where they should not be). So in this
> > view "noexec" makes sense.
> > There were similar arguments for not allowing device files in user
> > directories.
> 
> PR#17940 (https://github.com/systemd/systemd/pull/17940) was merged, so /dev
> will now on be mounted with "exec" by systemd.
> 
> I made issue #17942 (https://github.com/systemd/systemd/issues/17942) to
> discuss related hardening options. I'm leaning towards
> NoExecPaths=/ExecPaths= as it would enable nice hardening by allow-listing
> of all executable content for system services with simple directives like:
> 
> [Service]
> NoExecPaths=/
> ExecPaths=/usr/sbin/daemon /lib64/ld-linux-x86-64.so.2 /usr/lib
> 
> Then a service infected with malware would not be able to execute a shell
> present in the system or downloaded later, if that was not explicitly
> allowed. /dev would also not have "exec" flag by default, but SGX could be
> allowed with "ExecPaths=/dev/sgx" when needed.

OK, this sounds relieving, thank you (late resp, I was vacation last
week).


> -Topi


/Jarkko