What will happen if 2 processes map same physical page

What will happen if 2 processes map same physical page

[Lev Olshvang]

Hi all,

The question is it ipossiblle in Linux/MMU/TLB  that 2 processes map to the same physical address?
Will CPU or  TLB discover that second process tries to reach occupied physical page?

What if first process set page permission to read and second whats to write to this page ?
Perhaps during context switch all page access permissions of first process is flashed out from MMU ?


I confess I do not know what exactly happens in TLB and between Linux and hardware view of pages.

I am particularly intereste in ARMv7, perhaps its architecture have some solutions to these question.

Regards,
Lev



_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: What will happen if 2 processes map same physical page

[Valdis Klētnieks]

On Wed, 20 Mar 2019 16:42:39 +0300, Lev Olshvang said:
> The question is it ipossiblle in Linux/MMU/TLB  that 2 processes map to
> the same physical address?

Totally possible.  That's how mmap shared memory works, and why shared
libraries are possible.

> Will CPU or  TLB discover that second process tries to reach occupied physical page?

Well, the hardware won't discover it as a "second" process, it only knows it's
processing *this* memory access.

> What if first process set page permission to read and second whats to write to this page ?

Perfectly OK - the two processes have separate page table mappings, with
separate permission bits. So (for example) physical page 0x17F000 is mapped to
virtual address 0x2034D000 with read-only permission n process 1's page tables,
and to virtual address 0x98FF3000 with read-write permission in process 2's
page tables. No problem.

(And before you ask, yes it's possible for process 2 to running on one core
doing a write to the page at the exact same time that process 1 is doing a read
on another core.  Depending on the hardware cache design, this may or may not
get process 1 updated data.  This is why locking and memory barriers are
important. See Documentation/memory-barriers.txt for more details)

"And then there's the Alpha" - a processor design that got much of its speed by
being weird about this stuff. :)

> Perhaps during context switch all page access permissions of first process is
> flashed out from MMU ?

Actually, the kernel just points the MMU at a new set of page table entries and lets
the TLB reload as needed. In particular, on most architectures, the kernel tries really
hard to ensure that all processes share at least part of their page table mappings so
the kernel is always mapped at the same place, meaning that there's a better chance
that on a syscall, the TLB already has hot entries for large parts of the kernel so no
TLB reloads are needed.


_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: What will happen if 2 processes map same physical page

[Lev Olshvang]

Hi Vaaldis,

Thanks for answer,
I still wondering whether the kernel will allow write to a read-only page of shared library while it has mapped to several processes? Kernel knows that page's reference count >1,  will it allow mmap/mprotect to change page protection ? Or will it allow direct right by physical address?
I suppose that CPU should raise page fault when write is made to read only page, 

What is the sequence  CPU raises page faul before write to page of after data is written
Will  CPU wait until kernel will consider what to do , whether agree and change PTE  "writable " bit to 1 ?
Or kernel may disagree and raise SEGFAULT?

I checked in the handle_mm_fault()  calls for arch_vma_access_permitted() which just returns true on most architectures which is very strange and  contradicts my prediction of SEFFAULT.
arch_vma_access_permitted() retutus true when is sees that access is made from foreign process?
https://elixir.bootlin.com/linux/latest/ident/arch_vma_access_permitted

I am totally confused.

What do you think ?

Regards,
Lev



20.03.2019, 20:08, "Valdis Klētnieks" <valdis.kletnieks@vt.edu>:
> On Wed, 20 Mar 2019 16:42:39 +0300, Lev Olshvang said:
>>  The question is it ipossiblle in Linux/MMU/TLB that 2 processes map to
>>  the same physical address?
>
> Totally possible. That's how mmap shared memory works, and why shared
> libraries are possible.
>
>>  Will CPU or TLB discover that second process tries to reach occupied physical page?
>
> Well, the hardware won't discover it as a "second" process, it only knows it's
> processing *this* memory access.
>
>>  What if first process set page permission to read and second whats to write to this page ?
>
> Perfectly OK - the two processes have separate page table mappings, with
> separate permission bits. So (for example) physical page 0x17F000 is mapped to
> virtual address 0x2034D000 with read-only permission n process 1's page tables,
> and to virtual address 0x98FF3000 with read-write permission in process 2's
> page tables. No problem.
>
> (And before you ask, yes it's possible for process 2 to running on one core
> doing a write to the page at the exact same time that process 1 is doing a read
> on another core. Depending on the hardware cache design, this may or may not
> get process 1 updated data. This is why locking and memory barriers are
> important. See Documentation/memory-barriers.txt for more details)
>
> "And then there's the Alpha" - a processor design that got much of its speed by
> being weird about this stuff. :)
>
>>  Perhaps during context switch all page access permissions of first process is
>>  flashed out from MMU ?
>
> Actually, the kernel just points the MMU at a new set of page table entries and lets
> the TLB reload as needed. In particular, on most architectures, the kernel tries really
> hard to ensure that all processes share at least part of their page table mappings so
> the kernel is always mapped at the same place, meaning that there's a better chance
> that on a syscall, the TLB already has hot entries for large parts of the kernel so no
> TLB reloads are needed.

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: What will happen if 2 processes map same physical page

[Okash Khawaja]

On Thu, 21 Mar 2019 12:56:17 +0300
Lev Olshvang <levonshe@yandex.com> wrote:

> Hi Vaaldis,
> 
> Thanks for answer,
> I still wondering whether the kernel will allow write to a read-only
> page of shared library while it has mapped to several processes?
> Kernel knows that page's reference count >1,  will it allow
> mmap/mprotect to change page protection ? Or will it allow direct
> right by physical address? I suppose that CPU should raise page fault
> when write is made to read only page, 
> 
> What is the sequence  CPU raises page faul before write to page of
> after data is written Will  CPU wait until kernel will consider what
> to do , whether agree and change PTE  "writable " bit to 1 ? Or
> kernel may disagree and raise SEGFAULT?

Note that each process has its own PTE. So PTE in one process may say
the page is writable and PTE in another process may say it's read-only.

> 
> I checked in the handle_mm_fault()  calls for
> arch_vma_access_permitted() which just returns true on most
> architectures which is very strange and  contradicts my prediction of
> SEFFAULT. arch_vma_access_permitted() retutus true when is sees that
> access is made from foreign process?
> https://elixir.bootlin.com/linux/latest/ident/arch_vma_access_permitted
> 
> I am totally confused.
> 
> What do you think ?
> 
> Regards,
> Lev

It looks like there are two separate questions in the email.

1) Will kernel allow the same physical page to be mapped as read-only
in one process and as read-write in another process?

2) How page fault is generated?

Answer for first is yes. Same physical page can be mapped with
different permissions in two different processes. It means read-only
process will ultimately (hopefully very soon) notice changes made by
read-write process.

Answer for second question is a bit complicated. However there is a
trick to it. Once we know that, rest will become clear automaticaly.
The trick (at least for x86 systems) is that permissions are maintained
at two different levels:

- VMA level
- PTE level (or PUD level for larger page size but that is not relevant
  here)

When a page in memory is accessed, permission on corresponding VMA is
checked first. If the access is allowed by VMA then PTE permissions are
checked. Otherwise segfault is generated. If permissions at PTE level
don't match the access type then a page fault is generated. That's when
page fault hander kicks in and tries to resolve the problem by faulting
the page into RAM, copying the page in RAM (for copy-on-write) etc.

> 
> 
> 
> 20.03.2019, 20:08, "Valdis Klētnieks" <valdis.kletnieks@vt.edu>:
> > On Wed, 20 Mar 2019 16:42:39 +0300, Lev Olshvang said:  
> >>  The question is it ipossiblle in Linux/MMU/TLB that 2 processes
> >> map to the same physical address?  
> >
> > Totally possible. That's how mmap shared memory works, and why
> > shared libraries are possible.
> >  
> >>  Will CPU or TLB discover that second process tries to reach
> >> occupied physical page?  
> >
> > Well, the hardware won't discover it as a "second" process, it only
> > knows it's processing *this* memory access.
> >  
> >>  What if first process set page permission to read and second
> >> whats to write to this page ?  
> >
> > Perfectly OK - the two processes have separate page table mappings,
> > with separate permission bits. So (for example) physical page
> > 0x17F000 is mapped to virtual address 0x2034D000 with read-only
> > permission n process 1's page tables, and to virtual address
> > 0x98FF3000 with read-write permission in process 2's page tables.
> > No problem.
> >
> > (And before you ask, yes it's possible for process 2 to running on
> > one core doing a write to the page at the exact same time that
> > process 1 is doing a read on another core. Depending on the
> > hardware cache design, this may or may not get process 1 updated
> > data. This is why locking and memory barriers are important. See
> > Documentation/memory-barriers.txt for more details)
> >
> > "And then there's the Alpha" - a processor design that got much of
> > its speed by being weird about this stuff. :)
> >  
> >>  Perhaps during context switch all page access permissions of
> >> first process is flashed out from MMU ?  
> >
> > Actually, the kernel just points the MMU at a new set of page table
> > entries and lets the TLB reload as needed. In particular, on most
> > architectures, the kernel tries really hard to ensure that all
> > processes share at least part of their page table mappings so the
> > kernel is always mapped at the same place, meaning that there's a
> > better chance that on a syscall, the TLB already has hot entries
> > for large parts of the kernel so no TLB reloads are needed.  
> 
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies@kernelnewbies.org
> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies


_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: What will happen if 2 processes map same physical page

[Lev Olshvang]

Hello Okash,

Actually there were 3 question:

Third question is :
Is there any way to tell the kernel that PTE of same physical page should be equal in all processes?

For example, shared lib mapped from different processes to same physical page must have same PTE, isn it?

And  the presence of SELinux feature SELINUX_CHECKREQPROT_VALUE  indicates for me that kernel somehow knows the correct page protections. (although I do not see in code how it is done)

But the question might be rephrased :  IMHO Kernel should mandate same PTE flags no matter how many virtual mapping were made to the same physical page.

What do you think?






21.03.2019, 13:45, "Okash Khawaja" <okash.khawaja@gmail.com>:
> On Thu, 21 Mar 2019 12:56:17 +0300
> Lev Olshvang <levonshe@yandex.com> wrote:
>
>>  Hi Vaaldis,
>>
>>  Thanks for answer,
>>  I still wondering whether the kernel will allow write to a read-only
>>  page of shared library while it has mapped to several processes?
>>  Kernel knows that page's reference count >1, will it allow
>>  mmap/mprotect to change page protection ? Or will it allow direct
>>  right by physical address? I suppose that CPU should raise page fault
>>  when write is made to read only page,
>>
>>  What is the sequence CPU raises page faul before write to page of
>>  after data is written Will CPU wait until kernel will consider what
>>  to do , whether agree and change PTE "writable " bit to 1 ? Or
>>  kernel may disagree and raise SEGFAULT?
>
> Note that each process has its own PTE. So PTE in one process may say
> the page is writable and PTE in another process may say it's read-only.
>
>>  I checked in the handle_mm_fault() calls for
>>  arch_vma_access_permitted() which just returns true on most
>>  architectures which is very strange and contradicts my prediction of
>>  SEFFAULT. arch_vma_access_permitted() retutus true when is sees that
>>  access is made from foreign process?
>>  https://elixir.bootlin.com/linux/latest/ident/arch_vma_access_permitted
>>
>>  I am totally confused.
>>
>>  What do you think ?
>>
>>  Regards,
>>  Lev
>
> It looks like there are two separate questions in the email.
>
> 1) Will kernel allow the same physical page to be mapped as read-only
> in one process and as read-write in another process?
>
> 2) How page fault is generated?
>
> Answer for first is yes. Same physical page can be mapped with
> different permissions in two different processes. It means read-only
> process will ultimately (hopefully very soon) notice changes made by
> read-write process.
>
> Answer for second question is a bit complicated. However there is a
> trick to it. Once we know that, rest will become clear automaticaly.
> The trick (at least for x86 systems) is that permissions are maintained
> at two different levels:
>
> - VMA level
> - PTE level (or PUD level for larger page size but that is not relevant
>   here)
>
> When a page in memory is accessed, permission on corresponding VMA is
> checked first. If the access is allowed by VMA then PTE permissions are
> checked. Otherwise segfault is generated. If permissions at PTE level
> don't match the access type then a page fault is generated. That's when
> page fault hander kicks in and tries to resolve the problem by faulting
> the page into RAM, copying the page in RAM (for copy-on-write) etc.
>
>>  20.03.2019, 20:08, "Valdis Klētnieks" <valdis.kletnieks@vt.edu>:
>>  > On Wed, 20 Mar 2019 16:42:39 +0300, Lev Olshvang said:
>>  >>  The question is it ipossiblle in Linux/MMU/TLB that 2 processes
>>  >> map to the same physical address?
>>  >
>>  > Totally possible. That's how mmap shared memory works, and why
>>  > shared libraries are possible.
>>  >
>>  >>  Will CPU or TLB discover that second process tries to reach
>>  >> occupied physical page?
>>  >
>>  > Well, the hardware won't discover it as a "second" process, it only
>>  > knows it's processing *this* memory access.
>>  >
>>  >>  What if first process set page permission to read and second
>>  >> whats to write to this page ?
>>  >
>>  > Perfectly OK - the two processes have separate page table mappings,
>>  > with separate permission bits. So (for example) physical page
>>  > 0x17F000 is mapped to virtual address 0x2034D000 with read-only
>>  > permission n process 1's page tables, and to virtual address
>>  > 0x98FF3000 with read-write permission in process 2's page tables.
>>  > No problem.
>>  >
>>  > (And before you ask, yes it's possible for process 2 to running on
>>  > one core doing a write to the page at the exact same time that
>>  > process 1 is doing a read on another core. Depending on the
>>  > hardware cache design, this may or may not get process 1 updated
>>  > data. This is why locking and memory barriers are important. See
>>  > Documentation/memory-barriers.txt for more details)
>>  >
>>  > "And then there's the Alpha" - a processor design that got much of
>>  > its speed by being weird about this stuff. :)
>>  >
>>  >>  Perhaps during context switch all page access permissions of
>>  >> first process is flashed out from MMU ?
>>  >
>>  > Actually, the kernel just points the MMU at a new set of page table
>>  > entries and lets the TLB reload as needed. In particular, on most
>>  > architectures, the kernel tries really hard to ensure that all
>>  > processes share at least part of their page table mappings so the
>>  > kernel is always mapped at the same place, meaning that there's a
>>  > better chance that on a syscall, the TLB already has hot entries
>>  > for large parts of the kernel so no TLB reloads are needed.
>>
>>  _______________________________________________
>>  Kernelnewbies mailing list
>>  Kernelnewbies@kernelnewbies.org
>>  https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: What will happen if 2 processes map same physical page

[Okash Khawaja]

Hi Lev,

On Fri, 22 Mar 2019 12:15:49 +0300
Lev Olshvang <levonshe@yandex.com> wrote:

> Hello Okash,
> 
> Actually there were 3 question:
> 
> Third question is :
> Is there any way to tell the kernel that PTE of same physical page
> should be equal in all processes?
Not that I am aware of. You mention SELinux feature which might be a
way but I don't know about it.

> 
> For example, shared lib mapped from different processes to same
> physical page must have same PTE, isn it?
This is policy vs. mechanism question. I think kernel - wherever
possible - provides mechanism, leaving policy to be determined by user
space. So in case of shared lib mapping, it's the dynamic linker which
ensures that all mappings are similar across different processes.

> 
> And  the presence of SELinux feature SELINUX_CHECKREQPROT_VALUE
> indicates for me that kernel somehow knows the correct page
> protections. (although I do not see in code how it is done)
> 
> But the question might be rephrased :  IMHO Kernel should mandate
> same PTE flags no matter how many virtual mapping were made to the
> same physical page.
> 
> What do you think?

I'm not sure if that would be the best default option. One reason is
that PTE flags aren't a property of physical page itself. Instead, PTE
flags are a property of a process's view of the physical page. Physical
page itself is a series of bytes. Meaning is given to those bytes by
the process which accesses them.

This is a bit more subjective opinion rather than an objective answer
so I'm ot sure if this answers your query.

Thanks,
Okash

> 
> 
> 
> 
> 
> 
> 21.03.2019, 13:45, "Okash Khawaja" <okash.khawaja@gmail.com>:
> > On Thu, 21 Mar 2019 12:56:17 +0300
> > Lev Olshvang <levonshe@yandex.com> wrote:
> >  
> >>  Hi Vaaldis,
> >>
> >>  Thanks for answer,
> >>  I still wondering whether the kernel will allow write to a
> >> read-only page of shared library while it has mapped to several
> >> processes? Kernel knows that page's reference count >1, will it
> >> allow mmap/mprotect to change page protection ? Or will it allow
> >> direct right by physical address? I suppose that CPU should raise
> >> page fault when write is made to read only page,
> >>
> >>  What is the sequence CPU raises page faul before write to page of
> >>  after data is written Will CPU wait until kernel will consider
> >> what to do , whether agree and change PTE "writable " bit to 1 ? Or
> >>  kernel may disagree and raise SEGFAULT?  
> >
> > Note that each process has its own PTE. So PTE in one process may
> > say the page is writable and PTE in another process may say it's
> > read-only. 
> >>  I checked in the handle_mm_fault() calls for
> >>  arch_vma_access_permitted() which just returns true on most
> >>  architectures which is very strange and contradicts my prediction
> >> of SEFFAULT. arch_vma_access_permitted() retutus true when is sees
> >> that access is made from foreign process?
> >>  https://elixir.bootlin.com/linux/latest/ident/arch_vma_access_permitted
> >>
> >>  I am totally confused.
> >>
> >>  What do you think ?
> >>
> >>  Regards,
> >>  Lev  
> >
> > It looks like there are two separate questions in the email.
> >
> > 1) Will kernel allow the same physical page to be mapped as
> > read-only in one process and as read-write in another process?
> >
> > 2) How page fault is generated?
> >
> > Answer for first is yes. Same physical page can be mapped with
> > different permissions in two different processes. It means read-only
> > process will ultimately (hopefully very soon) notice changes made by
> > read-write process.
> >
> > Answer for second question is a bit complicated. However there is a
> > trick to it. Once we know that, rest will become clear automaticaly.
> > The trick (at least for x86 systems) is that permissions are
> > maintained at two different levels:
> >
> > - VMA level
> > - PTE level (or PUD level for larger page size but that is not
> > relevant here)
> >
> > When a page in memory is accessed, permission on corresponding VMA
> > is checked first. If the access is allowed by VMA then PTE
> > permissions are checked. Otherwise segfault is generated. If
> > permissions at PTE level don't match the access type then a page
> > fault is generated. That's when page fault hander kicks in and
> > tries to resolve the problem by faulting the page into RAM, copying
> > the page in RAM (for copy-on-write) etc. 
> >>  20.03.2019, 20:08, "Valdis Klētnieks" <valdis.kletnieks@vt.edu>:  
> >>  > On Wed, 20 Mar 2019 16:42:39 +0300, Lev Olshvang said:  
> >>  >>  The question is it ipossiblle in Linux/MMU/TLB that 2
> >> processes >> map to the same physical address?  
> >>  >
> >>  > Totally possible. That's how mmap shared memory works, and why
> >>  > shared libraries are possible.
> >>  >  
> >>  >>  Will CPU or TLB discover that second process tries to reach
> >>  >> occupied physical page?  
> >>  >
> >>  > Well, the hardware won't discover it as a "second" process, it
> >> only > knows it's processing *this* memory access.
> >>  >  
> >>  >>  What if first process set page permission to read and second
> >>  >> whats to write to this page ?  
> >>  >
> >>  > Perfectly OK - the two processes have separate page table
> >> mappings, > with separate permission bits. So (for example)
> >> physical page > 0x17F000 is mapped to virtual address 0x2034D000
> >> with read-only > permission n process 1's page tables, and to
> >> virtual address > 0x98FF3000 with read-write permission in process
> >> 2's page tables. > No problem.
> >>  >
> >>  > (And before you ask, yes it's possible for process 2 to running
> >> on > one core doing a write to the page at the exact same time that
> >>  > process 1 is doing a read on another core. Depending on the
> >>  > hardware cache design, this may or may not get process 1 updated
> >>  > data. This is why locking and memory barriers are important. See
> >>  > Documentation/memory-barriers.txt for more details)
> >>  >
> >>  > "And then there's the Alpha" - a processor design that got much
> >> of > its speed by being weird about this stuff. :)
> >>  >  
> >>  >>  Perhaps during context switch all page access permissions of
> >>  >> first process is flashed out from MMU ?  
> >>  >
> >>  > Actually, the kernel just points the MMU at a new set of page
> >> table > entries and lets the TLB reload as needed. In particular,
> >> on most > architectures, the kernel tries really hard to ensure
> >> that all > processes share at least part of their page table
> >> mappings so the > kernel is always mapped at the same place,
> >> meaning that there's a > better chance that on a syscall, the TLB
> >> already has hot entries > for large parts of the kernel so no TLB
> >> reloads are needed.  
> >>
> >>  _______________________________________________
> >>  Kernelnewbies mailing list
> >>  Kernelnewbies@kernelnewbies.org
> >>  https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies  


_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: What will happen if 2 processes map same physical page

[Valdis Klētnieks]

On Fri, 22 Mar 2019 12:15:49 +0300, Lev Olshvang said:

> But the question might be rephrased :  IMHO Kernel should mandate same PTE
> flags no matter how many virtual mapping were made to the same physical page.

And exactly *why* should it be "mandated"?  Certainly, for many classes of objects,
such as shared libraries, it's a desirable feature (maybe - but see below).

However, there's plenty of *other* use cases where the programmer may want to
have one control process having read/write access to a memory segment, while
a bunch of worker processes are merely reading the data.

For instance, if you're serving out complicated computations to sub-processes that
involve a lot of parameters and input data, the control process already *has* all this
data (potentially megabytes of it) in memory. Using shared memory to transfer it to
the worker process is a lot more efficient than having to stuff it all through a socket.

And even for shared libraries, you may want one process to be able to write to the
space while others are reading it, for live patching and similar functions.  (Yes, there's
a security trade-off there - and yes, there are sites that will accept the risk, and no,
that sort of trade-off belongs in userspace, not in the kernel).

The kernel does mechanism, not policy.  So it's totally reasonable to have a
defined way for userspace to say "this page can only be shared with these
permissions" - that's mechanism. Having the kernel force a specific value
without a good architectural reason is policy.

(Sometimes the kernel does force things to work a specific way if it's required
to guarantee system stability.  That's why you can't use the  write() system
call on a directory even if you have write permissions - you can only use stuff
like link() and open(). Permissions on shared memory pages don't involve that
sort of kernel self-defense issue.


_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies