[PATCH 0/7] introduce __ro_mostly_after_init section marker

[PATCH 0/7] introduce __ro_mostly_after_init section marker

[Hoeun Ryu]

 After `__ro_after_init` marker is included in kernel, many kernel data
objects can be read-only-after-init. But there are many other places that
would be good to read-only-after-init but `__ro_after_init` can not be simply
applicable to them because they should be writable at some points, which are
during module_init/exit or dynamic de/registration for a specific subsystem.
 `__ro_mostly_after_init` is basically the same to `__ro_after_init`. The
section is mapped as read-only after kernel init. The different thing is
this section is temporarily mapped as read-write during module_init/exit and
de/registration of a subsystem using set_ro_mostly_after_init_rw/ro pair.

- Tested only on arm64.

Description:
  0001 patch is `__ro_mostly_after_init` itself.
  0002 patch is to add set_ro_mostly_after_init_rw/ro pair using
    set_memory_rw/ro.
  0003 patch is to make the section read-write in module_init/exit.
  0004 patch is an example for dynamic init/deinit of a subsystem.
  0005 patch is an example for __ro_mostly_after_init section modified during
    module_init/exit.
  0006/0007 patches are fixes for arm64 kernel mapping.

Hoeun Ryu (7):
  arch: add __ro_mostly_after_init section marker
  init: add set_ro_mostly_after_init_rw/ro function
  module: modify memory attrs for __ro_mostly_after_init during
    module_init/exit
  selinux: mark __ro_mostly_after_init for selinux_hooks/selinux_nf_ops
  cpu: mark ro_mostly_after_init for cpuhp_ap/bp_states
  arm64: add __map_kernel_segment to accept additional vm flags
  arm64: map seperately rodata sections for __ro_mostly_after_init
    section

 arch/arm64/mm/mmu.c               | 44 ++++++++++++++++++++++++++++++++-------
 include/asm-generic/sections.h    |  1 +
 include/asm-generic/vmlinux.lds.h | 10 +++++++++
 include/linux/cache.h             | 11 ++++++++++
 include/linux/init.h              |  6 ++++++
 init/main.c                       | 24 +++++++++++++++++++++
 kernel/cpu.c                      |  4 ++--
 kernel/module.c                   | 10 +++++++--
 security/selinux/hooks.c          |  8 +++++--
 9 files changed, 105 insertions(+), 13 deletions(-)

-- 
2.7.4

[kernel-hardening] [PATCH 0/7] introduce __ro_mostly_after_init section marker

[Hoeun Ryu]

 After `__ro_after_init` marker is included in kernel, many kernel data
objects can be read-only-after-init. But there are many other places that
would be good to read-only-after-init but `__ro_after_init` can not be simply
applicable to them because they should be writable at some points, which are
during module_init/exit or dynamic de/registration for a specific subsystem.
 `__ro_mostly_after_init` is basically the same to `__ro_after_init`. The
section is mapped as read-only after kernel init. The different thing is
this section is temporarily mapped as read-write during module_init/exit and
de/registration of a subsystem using set_ro_mostly_after_init_rw/ro pair.

- Tested only on arm64.

Description:
  0001 patch is `__ro_mostly_after_init` itself.
  0002 patch is to add set_ro_mostly_after_init_rw/ro pair using
    set_memory_rw/ro.
  0003 patch is to make the section read-write in module_init/exit.
  0004 patch is an example for dynamic init/deinit of a subsystem.
  0005 patch is an example for __ro_mostly_after_init section modified during
    module_init/exit.
  0006/0007 patches are fixes for arm64 kernel mapping.

Hoeun Ryu (7):
  arch: add __ro_mostly_after_init section marker
  init: add set_ro_mostly_after_init_rw/ro function
  module: modify memory attrs for __ro_mostly_after_init during
    module_init/exit
  selinux: mark __ro_mostly_after_init for selinux_hooks/selinux_nf_ops
  cpu: mark ro_mostly_after_init for cpuhp_ap/bp_states
  arm64: add __map_kernel_segment to accept additional vm flags
  arm64: map seperately rodata sections for __ro_mostly_after_init
    section

 arch/arm64/mm/mmu.c               | 44 ++++++++++++++++++++++++++++++++-------
 include/asm-generic/sections.h    |  1 +
 include/asm-generic/vmlinux.lds.h | 10 +++++++++
 include/linux/cache.h             | 11 ++++++++++
 include/linux/init.h              |  6 ++++++
 init/main.c                       | 24 +++++++++++++++++++++
 kernel/cpu.c                      |  4 ++--
 kernel/module.c                   | 10 +++++++--
 security/selinux/hooks.c          |  8 +++++--
 9 files changed, 105 insertions(+), 13 deletions(-)

-- 
2.7.4

Re: [kernel-hardening] [PATCH 0/7] introduce __ro_mostly_after_init section marker

[Ard Biesheuvel]

Hi Hoeun,

On 19 February 2017 at 10:03, Hoeun Ryu <hoeun.ryu@gmail.com> wrote:
>  After `__ro_after_init` marker is included in kernel, many kernel data
> objects can be read-only-after-init. But there are many other places that
> would be good to read-only-after-init but `__ro_after_init` can not be simply
> applicable to them because they should be writable at some points, which are
> during module_init/exit or dynamic de/registration for a specific subsystem.

The argument sounds reasonable, but it would really help if you could
describe some use cases in more detail.

>  `__ro_mostly_after_init` is basically the same to `__ro_after_init`. The
> section is mapped as read-only after kernel init. The different thing is
> this section is temporarily mapped as read-write during module_init/exit and
> de/registration of a subsystem using set_ro_mostly_after_init_rw/ro pair.
>
> - Tested only on arm64.
>
> Description:
>   0001 patch is `__ro_mostly_after_init` itself.
>   0002 patch is to add set_ro_mostly_after_init_rw/ro pair using
>     set_memory_rw/ro.
>   0003 patch is to make the section read-write in module_init/exit.
>   0004 patch is an example for dynamic init/deinit of a subsystem.
>   0005 patch is an example for __ro_mostly_after_init section modified during
>     module_init/exit.
>   0006/0007 patches are fixes for arm64 kernel mapping.
>
> Hoeun Ryu (7):
>   arch: add __ro_mostly_after_init section marker
>   init: add set_ro_mostly_after_init_rw/ro function
>   module: modify memory attrs for __ro_mostly_after_init during
>     module_init/exit
>   selinux: mark __ro_mostly_after_init for selinux_hooks/selinux_nf_ops
>   cpu: mark ro_mostly_after_init for cpuhp_ap/bp_states
>   arm64: add __map_kernel_segment to accept additional vm flags
>   arm64: map seperately rodata sections for __ro_mostly_after_init
>     section
>
>  arch/arm64/mm/mmu.c               | 44 ++++++++++++++++++++++++++++++++-------
>  include/asm-generic/sections.h    |  1 +
>  include/asm-generic/vmlinux.lds.h | 10 +++++++++
>  include/linux/cache.h             | 11 ++++++++++
>  include/linux/init.h              |  6 ++++++
>  init/main.c                       | 24 +++++++++++++++++++++
>  kernel/cpu.c                      |  4 ++--
>  kernel/module.c                   | 10 +++++++--
>  security/selinux/hooks.c          |  8 +++++--
>  9 files changed, 105 insertions(+), 13 deletions(-)
>
> --
> 2.7.4
>

Re: [kernel-hardening] [PATCH 0/7] introduce __ro_mostly_after_init section marker

[Mark Rutland]

On Sun, Feb 19, 2017 at 07:03:38PM +0900, Hoeun Ryu wrote:
>  After `__ro_after_init` marker is included in kernel, many kernel data
> objects can be read-only-after-init. But there are many other places that
> would be good to read-only-after-init but `__ro_after_init` can not be simply
> applicable to them because they should be writable at some points, which are
> during module_init/exit or dynamic de/registration for a specific subsystem.

Could you elaborate on this?

For modules, I assume that the __ro_after_init data structures are part
of the module, and not part of the "real" kernel image. Is that the case?

Which specific subsystems whish to modify data structures that are
__ro_after_init?

This sounds like the proposed mostly-ro/rarely-rw stuff would be a
better fit for that case.

Thanks,
Mark.

>  `__ro_mostly_after_init` is basically the same to `__ro_after_init`. The
> section is mapped as read-only after kernel init. The different thing is
> this section is temporarily mapped as read-write during module_init/exit and
> de/registration of a subsystem using set_ro_mostly_after_init_rw/ro pair.
> 
> - Tested only on arm64.
> 
> Description:
>   0001 patch is `__ro_mostly_after_init` itself.
>   0002 patch is to add set_ro_mostly_after_init_rw/ro pair using
>     set_memory_rw/ro.
>   0003 patch is to make the section read-write in module_init/exit.
>   0004 patch is an example for dynamic init/deinit of a subsystem.
>   0005 patch is an example for __ro_mostly_after_init section modified during
>     module_init/exit.
>   0006/0007 patches are fixes for arm64 kernel mapping.
> 
> Hoeun Ryu (7):
>   arch: add __ro_mostly_after_init section marker
>   init: add set_ro_mostly_after_init_rw/ro function
>   module: modify memory attrs for __ro_mostly_after_init during
>     module_init/exit
>   selinux: mark __ro_mostly_after_init for selinux_hooks/selinux_nf_ops
>   cpu: mark ro_mostly_after_init for cpuhp_ap/bp_states
>   arm64: add __map_kernel_segment to accept additional vm flags
>   arm64: map seperately rodata sections for __ro_mostly_after_init
>     section
> 
>  arch/arm64/mm/mmu.c               | 44 ++++++++++++++++++++++++++++++++-------
>  include/asm-generic/sections.h    |  1 +
>  include/asm-generic/vmlinux.lds.h | 10 +++++++++
>  include/linux/cache.h             | 11 ++++++++++
>  include/linux/init.h              |  6 ++++++
>  init/main.c                       | 24 +++++++++++++++++++++
>  kernel/cpu.c                      |  4 ++--
>  kernel/module.c                   | 10 +++++++--
>  security/selinux/hooks.c          |  8 +++++--
>  9 files changed, 105 insertions(+), 13 deletions(-)
> 
> -- 
> 2.7.4
> 

Re: [kernel-hardening] [PATCH 0/7] introduce __ro_mostly_after_init section marker

[Ho-Eun Ryu]

> On 20 Feb 2017, at 7:02 PM, Mark Rutland <mark.rutland@arm.com> wrote:
> 
> On Sun, Feb 19, 2017 at 07:03:38PM +0900, Hoeun Ryu wrote:
>> After `__ro_after_init` marker is included in kernel, many kernel data
>> objects can be read-only-after-init. But there are many other places that
>> would be good to read-only-after-init but `__ro_after_init` can not be simply
>> applicable to them because they should be writable at some points, which are
>> during module_init/exit or dynamic de/registration for a specific subsystem.
> 
> Could you elaborate on this?
> 
> For modules, I assume that the __ro_after_init data structures are part
> of the module, and not part of the "real" kernel image. Is that the case?
> 

__ro_mostly_after_init is for kernel builtin core subsystems, not for modules themselves.
The section can be writable only during kernel init and module_init/exit.
Some hooks (or array of hooks) of a core subsystem can be marked as __ro_mostly_after_init
similar to that way of __ro_after_init. After that some modules that may write to those hooks of
the subsystem to register/deregister something to the subsystem can safely access those section.
Please see RFC 3/7 that makes this section writable.

In addition, some subsystems may use this marker for their (array of) hooks and make them writable
only at some point of time via set_ro_mostly_after_init_rw/ro pair.
please read RFC 4/7 for selinux.

> Which specific subsystems whish to modify data structures that are
> __ro_after_init?

I’m not intending to make writable __ro_after_init section but introducing new section marker
that works mostly like __ro_after_init but can be written to at some points.
please see RFC 5/7 for cpuhotplug.

> 
> This sounds like the proposed mostly-ro/rarely-rw stuff would be a
> better fit for that case.
> 
> Thanks,
> Mark.
> 
>> `__ro_mostly_after_init` is basically the same to `__ro_after_init`. The
>> section is mapped as read-only after kernel init. The different thing is
>> this section is temporarily mapped as read-write during module_init/exit and
>> de/registration of a subsystem using set_ro_mostly_after_init_rw/ro pair.
>> 
>> - Tested only on arm64.
>> 
>> Description:
>>  0001 patch is `__ro_mostly_after_init` itself.
>>  0002 patch is to add set_ro_mostly_after_init_rw/ro pair using
>>    set_memory_rw/ro.
>>  0003 patch is to make the section read-write in module_init/exit.
>>  0004 patch is an example for dynamic init/deinit of a subsystem.
>>  0005 patch is an example for __ro_mostly_after_init section modified during
>>    module_init/exit.
>>  0006/0007 patches are fixes for arm64 kernel mapping.
>> 
>> Hoeun Ryu (7):
>>  arch: add __ro_mostly_after_init section marker
>>  init: add set_ro_mostly_after_init_rw/ro function
>>  module: modify memory attrs for __ro_mostly_after_init during
>>    module_init/exit
>>  selinux: mark __ro_mostly_after_init for selinux_hooks/selinux_nf_ops
>>  cpu: mark ro_mostly_after_init for cpuhp_ap/bp_states
>>  arm64: add __map_kernel_segment to accept additional vm flags
>>  arm64: map seperately rodata sections for __ro_mostly_after_init
>>    section
>> 
>> arch/arm64/mm/mmu.c               | 44 ++++++++++++++++++++++++++++++++-------
>> include/asm-generic/sections.h    |  1 +
>> include/asm-generic/vmlinux.lds.h | 10 +++++++++
>> include/linux/cache.h             | 11 ++++++++++
>> include/linux/init.h              |  6 ++++++
>> init/main.c                       | 24 +++++++++++++++++++++
>> kernel/cpu.c                      |  4 ++--
>> kernel/module.c                   | 10 +++++++--
>> security/selinux/hooks.c          |  8 +++++--
>> 9 files changed, 105 insertions(+), 13 deletions(-)
>> 
>> -- 
>> 2.7.4
>> 

Re: [kernel-hardening] [PATCH 0/7] introduce __ro_mostly_after_init section marker

[Ho-Eun Ryu]

> On 19 Feb 2017, at 8:14 PM, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> 
> Hi Hoeun,
> 
> On 19 February 2017 at 10:03, Hoeun Ryu <hoeun.ryu@gmail.com> wrote:
>> After `__ro_after_init` marker is included in kernel, many kernel data
>> objects can be read-only-after-init. But there are many other places that
>> would be good to read-only-after-init but `__ro_after_init` can not be simply
>> applicable to them because they should be writable at some points, which are
>> during module_init/exit or dynamic de/registration for a specific subsystem.
> 
> The argument sounds reasonable, but it would really help if you could
> describe some use cases in more detail.

Please see RFC 4/7 [1] and RFC 5/7 [2].

RFC 4 shows that cpu_ap/bp_states are marked as __ro_mostly_after_init.
cpu_ap/bp_states can not be marked as __ro_after_init because some modules
can write to those objects using cpuhp_setup/remove_state api during module_init/exit.

RFC 5 shows that selinux_hooks and selinux_nf_hooks are marked as __ro_mostly_after_init.
those hooks can not be marked as __ro_after_init because selinux_disable() function.
Those hooks should be writable during selinux_disable to deregister the hooks.

[1] https://lkml.org/lkml/2017/2/19/37
[2] https://lkml.org/lkml/2017/2/19/35

> 
>> `__ro_mostly_after_init` is basically the same to `__ro_after_init`. The
>> section is mapped as read-only after kernel init. The different thing is
>> this section is temporarily mapped as read-write during module_init/exit and
>> de/registration of a subsystem using set_ro_mostly_after_init_rw/ro pair.
>> 
>> - Tested only on arm64.
>> 
>> Description:
>>  0001 patch is `__ro_mostly_after_init` itself.
>>  0002 patch is to add set_ro_mostly_after_init_rw/ro pair using
>>    set_memory_rw/ro.
>>  0003 patch is to make the section read-write in module_init/exit.
>>  0004 patch is an example for dynamic init/deinit of a subsystem.
>>  0005 patch is an example for __ro_mostly_after_init section modified during
>>    module_init/exit.
>>  0006/0007 patches are fixes for arm64 kernel mapping.
>> 
>> Hoeun Ryu (7):
>>  arch: add __ro_mostly_after_init section marker
>>  init: add set_ro_mostly_after_init_rw/ro function
>>  module: modify memory attrs for __ro_mostly_after_init during
>>    module_init/exit
>>  selinux: mark __ro_mostly_after_init for selinux_hooks/selinux_nf_ops
>>  cpu: mark ro_mostly_after_init for cpuhp_ap/bp_states
>>  arm64: add __map_kernel_segment to accept additional vm flags
>>  arm64: map seperately rodata sections for __ro_mostly_after_init
>>    section
>> 
>> arch/arm64/mm/mmu.c               | 44 ++++++++++++++++++++++++++++++++-------
>> include/asm-generic/sections.h    |  1 +
>> include/asm-generic/vmlinux.lds.h | 10 +++++++++
>> include/linux/cache.h             | 11 ++++++++++
>> include/linux/init.h              |  6 ++++++
>> init/main.c                       | 24 +++++++++++++++++++++
>> kernel/cpu.c                      |  4 ++--
>> kernel/module.c                   | 10 +++++++--
>> security/selinux/hooks.c          |  8 +++++--
>> 9 files changed, 105 insertions(+), 13 deletions(-)
>> 
>> --
>> 2.7.4
>> 

Re: [kernel-hardening] [PATCH 0/7] introduce __ro_mostly_after_init section marker

[Mark Rutland]

On Tue, Feb 21, 2017 at 03:10:45PM +0900, Ho-Eun Ryu wrote:
> > On 20 Feb 2017, at 7:02 PM, Mark Rutland <mark.rutland@arm.com> wrote:
> > On Sun, Feb 19, 2017 at 07:03:38PM +0900, Hoeun Ryu wrote:
> >> After `__ro_after_init` marker is included in kernel, many kernel data
> >> objects can be read-only-after-init. But there are many other places that
> >> would be good to read-only-after-init but `__ro_after_init` can not be simply
> >> applicable to them because they should be writable at some points, which are
> >> during module_init/exit or dynamic de/registration for a specific subsystem.
> > 
> > Could you elaborate on this?
> > 
> > For modules, I assume that the __ro_after_init data structures are part
> > of the module, and not part of the "real" kernel image. Is that the case?
> 
> __ro_mostly_after_init is for kernel builtin core subsystems, not for
> modules themselves.  The section can be writable only during kernel
> init and module_init/exit.  Some hooks (or array of hooks) of a core
> subsystem can be marked as __ro_mostly_after_init similar to that way
> of __ro_after_init. After that some modules that may write to those
> hooks of the subsystem to register/deregister something to the
> subsystem can safely access those section.  Please see RFC 3/7 that
> makes this section writable.
> 
> In addition, some subsystems may use this marker for their (array of)
> hooks and make them writable only at some point of time via
> set_ro_mostly_after_init_rw/ro pair.  please read RFC 4/7 for selinux.

Ok.

This sounds like a limited case of __write_rarely, where we only expect
to perform writes in specific time windows.

I'd prefer if we could attack __write_rarely, and use the appraoch to
cater for this case, but I am not necessarily opposed to this approach
if other architectures are happy with it.

> > Which specific subsystems whish to modify data structures that are
> > __ro_after_init?
> 
> I’m not intending to make writable __ro_after_init section but
> introducing new section marker that works mostly like __ro_after_init
> but can be written to at some points.  please see RFC 5/7 for
> cpuhotplug.

Ah. I clearly had not read this sufficiently thoroughly; sorry about
that.

Thanks,
Mark.