How to change page permission from inside the kernel?

How to change page permission from inside the kernel?

[Ahmed Soliman]

I have a memory page allocated with mmap() from user space, This
address is passed to some kernel module (kvm_intel to be specific) and
i want to know how can I change the page permission from inside there
My goal is to achieve something like this
              mprotect(mem, PAGE_SIZE, PROT_READ)
except for mprotect can't be called from the kernel, and I couldn't
find the right way to do it.

How to change page permission from inside the kernel?

[valdis.kletnieks at vt.edu]

On Fri, 06 Jul 2018 20:06:29 +0200, Ahmed Soliman said:
> I have a memory page allocated with mmap() from user space, This
> address is passed to some kernel module (kvm_intel to be specific) and
> i want to know how can I change the page permission from inside there
> My goal is to achieve something like this
>               mprotect(mem, PAGE_SIZE, PROT_READ)
> except for mprotect can't be called from the kernel, and I couldn't
> find the right way to do it.

You better have a *really* good reason for wanting to do that from inside
the kernel, when userspace is perfectly able to do it for itself.

So there's two questions here:

1) Why does the page's protection need to be changed?
2) And why from inside the kernel?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 486 bytes
Desc: not available
URL: <http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20180706/190ef547/attachment.sig>

How to change page permission from inside the kernel?

[Ahmed Soliman]

> So there's two questions here:
>
from inside KVM lkm (/virt/kvm and arch/x86/kvm )
> 1) Why does the page's protection need to be changed?

Implementing some kernel protection against subset of rootkits that
manipulates kernel static data (memory pages as well as their
mappings) by having them enforced by hypervisor which is KVM in our
case (it is one way enforcement only reset by reboot).  For the sake
of the question what is going here is guest virtual address -> guest
frame number -> host virtual address and then something that behaves
like mprotect but inside a loadable kernel module the protection
request is done via hypercall, so KVM should handle that internally.
The point is all memory used by KVM for virtualization is  mmapped at
userspace and then passed to KVM using an IOCTL and kvm assumed to be
free to do whatever it wants with that memory area.

> 2) And why from inside the kernel?
Because this needs to be done from inside KVM.

Note: I am aware that this won't be effective against rootkits that
live in userspace, rootkits that target kernel dynamic data, files on
disk, as well as VMM escapes, but I believe if the attack surface is
reduced by just a little bit, it is not that bad, so please lets not
discuss that :P.

Thanks.

How to change page permission from inside the kernel?

[valdis.kletnieks at vt.edu]

On Fri, 06 Jul 2018 21:29:40 +0200, you said:

> Implementing some kernel protection against subset of rootkits that
> manipulates kernel static data (memory pages as well as their
> mappings) by having them enforced by hypervisor which is KVM in our

Can you give an actual example of a case where *all* of the following are true?

1) It's a page that's safe to make R/O out from under the code that uses that page.
2) It's a kernel static data that's R/W  (Hint: stuff known to be R/O is already set to R/O
  at boot or module load time, so if it's R/W it probably *needs* to be that...)
3) the rootkit *is* able to screw with kernel pages, but somehow *is not* able to
   disable your protection (remember, all it takes is one NOP or BRANCH opcode in
    the right place).
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 486 bytes
Desc: not available
URL: <http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20180706/91855bc4/attachment.sig>

Fwd: How to change page permission from inside the kernel?

[Ahmed Soliman]

---------- Forwarded message ----------
From: Ahmed Soliman <ahmedsoliman0x666@gmail.com>
Date: 6 July 2018 at 23:56
Subject: Re: How to change page permission from inside the kernel?
To: Valdis Kletnieks <valdis.kletnieks@vt.edu>


>> Implementing some kernel protection against subset of rootkits that
>> manipulates kernel static data (memory pages as well as their
>> mappings) by having them enforced by hypervisor which is KVM in our
>
> Can you give an actual example of a case where *all* of the following are true?
>
I think there is a little bit confusion between the current R/O
protection and what are we doing. so I will need to make some
distinctions than I will answer all the questions.
First here is a bunch of accronyms that I will use for convenience
(pulled out KVM  mmu.txt)

pfn   host page frame number
hpa   host physical address
hva   host virtual address
gfn   guest frame number
gpa   guest physical address
gva   guest virtual address

To aviod confusion I will refer to the thing we are doing as ROE (read
only enforcement) and it is alittle bit different from typical R/O
protection in 2 aspects:
- R/O is enforced by hardware but ROE is enforced by Hypervisor(still
using Hardware protection but with hypervisor on top of it)
- R/O can be enabled and disabled by the kernel at its own will but
ROE can be enabled by the guest kernel and once enabled the hypervisor
will make sure it never gets disabled again, so if even if the kernel
decided to modify a paged that has ROE, it can't without a reboot.

Here is a very simple description of how ROE is supposed to works (I
am missing out some tiny details like remapping pages):
 The kernel passes a gva of a page to its hypervisor.
 The hypervisor converts gva to hva then places R/O on that page
note that is 2 levels of paging we have the host page table with the
appropriate R/O protection (including guest's own ROE requests) and
the guest page table its own R/O protections that the guest can
modify.

There is another part that isn't designed yet to handle DMAs and
registers with static content

> 1) It's a page that's safe to make R/O out from under the code that uses that page.
Yes because the guest kernel and hypervisor will coordinate together
which pages of the _guest kernel_ memory should have this protections.
of course the userspace process running the hypervisor (let it be qemu
or any other software) do own the page and can do whatever it wishes
with it.
> 2) It's a kernel static data that's R/W  (Hint: stuff known to be R/O is already set to R/O
>   at boot or module load time, so if it's R/W it probably *needs* to be that...)
I think this question is from the confusion between R/W and ROE what
we are simply going to do is that have every R/O page that will never
be R/W again to also be ROE if the kernel is compiled with the right
options (we will add that too) to be a kvm guest.
> 3) the rootkit *is* able to screw with kernel pages, but somehow *is not* able to
>    disable your protection (remember, all it takes is one NOP or BRANCH opcode in
>     the right place).

I can't think of a way to do it. having a rootkit being able to
manipulate kernel pages can only be done though one of these ways none
of which is considered a real threat (except for the DMA part thats
why we are stil learning about current DMA memory protections and
limitations):
- VMM escape where it can then modify the paging protections in the
host allocated memory that was assigned to the guest.
- Screwing the page table itself which is not static data. but parts
of the details I am missing is that hypervisor will keep track of all
guest page number, guest frame number of all ROE protected pages and
make sure that it is never the wrong mapping when updating the TLB.
- DMA and we are planning to do similar protection for DMA memory but
still design in progress.

other than these described above, I can't think of a way to manipulate
a ROE protected page without a reboot.

Fwd: How to change page permission from inside the kernel?

[valdis.kletnieks at vt.edu]

On Fri, 06 Jul 2018 23:59:30 +0200, Ahmed Soliman said:

> ROE can be enabled by the guest kernel and once enabled the hypervisor
> will make sure it never gets disabled again, so if even if the kernel
> decided to modify a paged that has ROE, it can't without a reboot.

So in essence, you're willing to *permanently* remove pages from the
usable set in the guest? (Usable as in "able to write to it").

What happens after you've been up for 3 weeks and you're running out of
usable pages?

How does this interact with ballooning?

What ways can malware in the guest use this to DoS or otherwise break
the kernel? (Setting the page that contains the 'struct proc' for PID 1 would
be amusing, and I'm sure there's plenty of amusement with race conditions to
make other kernel threads fail when they encounter a page they were expecting
to be R/W)

Bonus points for explaining the hypervisor/guest interface needed to make this work.

>> Can you give an actual example of a case where *all* of the following are true?
>> 1) It's a page that's safe to make R/O out from under the code that uses that page.
>> 2) It's a kernel static data that's R/W  (Hint: stuff known to be R/O is already set to R/O
>>   at boot or module load time, so if it's R/W it probably *needs* to be that...)
>> 3) the rootkit *is* able to screw with kernel pages, but somehow *is not* able to
>>    disable your protection (remember, all it takes is one NOP or BRANCH opcode in
>>     the right place).

Well actually, what I was looking for was a description of the rootkit that you're
defending against - where it lives, what kernel data it attacks, and so on....

> I can't think of a way to do it. having a rootkit being able to manipulate
> kernel pages can only be done though one of these ways none of which is
> considered a real threat

If you're unable to explain the threat you're trying to guard against,
you're going to be unable to guard against it properly.

Hint:  Your protection is trivially bypassed by simply *replacing* the
protected page.  The attacker snarfs up a R/W 4K page, copies the
protected page to the new page, injects whatever malware bits they
find amusing, and change the virtual memory pointer tables so this
new page frame is referenced by the virtual address of the old page.
If the hypervisor even notices the swap, all it will probably do is make
the new page R/O - but that matters not because it's already compromised.

> other than these described above, I can't think of a way to manipulate a ROE
> protected page without a reboot.

You missed the point - your protection can be bypassed without manipulating
a ROE page.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 486 bytes
Desc: not available
URL: <http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20180706/68713872/attachment-0001.sig>

Fwd: How to change page permission from inside the kernel?

[Ahmed Soliman]

> What happens after you've been up for 3 weeks and you're running out of
> usable pages?
That can't happen, it is my mistake missing some details, this is for
only protecting Kernel Pages,
Pages that are hold code or static data that is created once and
assumed to be there for ever, like kernel
code section as well as any static data. However this isn't to be used
on any user space memory
(from the guest's perspective) process because processes start and end.

> How does this interact with ballooning?
I haven't thought about this _yet_. Ballooning is complex, I  can
start simple and expand later to support all
of the current KVM features, Also I am not even sure if ballooning is
considered related or not because If
Kernel pages can be part of the Balloon then I would that when they
are copied they are copied with the right
permissions and this is done completely independent of gva and gpa.

> What ways can malware in the guest use this to DoS or otherwise break
> the kernel? (Setting the page that contains the 'struct proc' for PID 1 would
> be amusing, and I'm sure there's plenty of amusement with race conditions to
> make other kernel threads fail when they encounter a page they were expecting
> to be R/W)

I should state that hypercalls are done only from kernel mode (I am
not sure about KVM but
t is the case in Xen and if KVM allows user mode hypercall we can
check for this specific one to
be created from kernel mode it is easy)
In this case, worrying about a if kernel code can Dos the kernel via
ROE is relatively meaningless
because there are already many other ways to break the system once one
have kernel mode access.

However If you mean that this way guest kernel can break host kernel,
then this is a different story, it will
not be possible because of kvm MMU is designed to make guest not being
able to access memory that host
never allocated to it.

So we are replacing some way to hide a rootkit in a kernel by a way to
crash the kernel
(given that there are many ways to do both). I think in most cases it
would be rather better if the  guest OS crashes
(thus indicating that something is wrong) rather than have a rootkit
working silently.
> Well actually, what I was looking for was a description of the rootkit that you're
> defending against - where it lives, what kernel data it attacks, and so on....
What about some real example?
I just found this one from google
https://github.com/nurupo/rootkit/blob/master/rootkit.c
While I aggree that the rootkit will make its way into the kernel but
it _will not_ be able to achieve all of its functionalities.
anything that uses "_asm_hook_patch" function should fail because the
system call tables is protected. I am not sure if ROE will prevent
other parts of this rootkit from working or not but here is one :).
But I am pretty sure that there are other static data that can be
abused by rootkits inside the kernel that should rather be protected.

> If you're unable to explain the threat you're trying to guard against,
> you're going to be unable to guard against it properly.
Well I kinda had one, but you never asked for it. We assume that the
attacker is running in kernel mode and his goal is to alter static
data/ code section in the kernel for whatever reason
with out a reboot. if reboot is needed well it can be trivially done
by replacing the whole kernel with a new one including any malicious
code and then faulting the running kernel thus forcing a reboot or
even.
Also attacks that involve dynamic data (except for the page table) is
out of scope including attacks that involve any block device.
We plan to handle memory either that is accessed either via CPU or DMA
as well as privileged registers that are set once and never modify
again.
We haven't agreed on how an unauthorized write to ROE protected memory
would be handled but worst case it can lead to crashing the guest OS,
Crashing the Host OS is considered a big breach and it shouldn't be
possible. Although this thing isn't really part of threat model but it
is kinda of a motto that crash is better than rootkit. because once it
happens we will know it is there and the indecent can be thoroughly
inspected.
> Hint:  Your protection is trivially bypassed by simply *replacing* the
> protected page.  The attacker snarfs up a R/W 4K page, copies the
> protected page to the new page, injects whatever malware bits they
> find amusing, and change the virtual memory pointer tables so this
> new page frame is referenced by the virtual address of the old page.
> If the hypervisor even notices the swap, all it will probably do is make
> the new page R/O - but that matters not because it's already compromised.

I think I did explain how we handled this:
>>- Screwing the page table itself which is not static data. but parts
>>of the details I am missing is that hypervisor will keep track of all
>>guest page number, guest frame number of all ROE protected pages and
>>make sure that it is never the wrong mapping when updating the TLB.

> You missed the point - your protection can be bypassed without manipulating
> a ROE page.
Changing the virtual memory pointer table is ok but again these memory
mappings will never
make it to the TLB and will be caught during by KVM MMU because it
will explicitly check both gva and gpa when resolving a gva so perhaps
this will put the guest kernel in some inconsistent state (again
crashing the guest is considered much much better than a rootkit
running in silence).

Fwd: How to change page permission from inside the kernel?

[valdis.kletnieks at vt.edu]

On Sat, 07 Jul 2018 01:31:45 +0200, Ahmed Soliman said:

> > You missed the point - your protection can be bypassed without manipulating
> > a ROE page.
> Changing the virtual memory pointer table is ok but again these memory
> mappings will never
> make it to the TLB and will be caught during by KVM MMU because it
> will explicitly check both gva and gpa when resolving a gva so perhaps

Did you actually *check* that this will happen, or will KVM just say "Oh,
the VM changed their mapping" and go with the new mapping?

In other words, *prove* that they won't make it to the TLB - and while
you're at it, deal with the case where the TLB isn't the source of the
mapping (which is the case on every single TLB miss, where it has to
go consult the page tables for the mapping.  And given Intel chipset's
over-exuberant approach to speculative execution and pipelining, there's
a good chance that the instant the proper mapping is found from the page
tables, it's used to make the actual memory reference, while the TLB gets
updated after the fact.  Plus, a lot of that happens in hardware, without
KVM getting a chance to inspect things.

(I don't actually know how the TLB works - but the point is you'll have to go
reading the Intel and AMD processor manuals to get that part right...)

While you're reading it, look for the implications of the "load/store bypassing
cache" instructions - can you force a surprise by storing directly to main memory
for a page table entry that is in L3 cache on the other socket?  (Note that the
kernel has memory barriers all over the place to avoid such surprises - but
examine the converse, where malware intentionally doesn't use a barrier so
there's inconsistency between cache, memory, and registers)

I take it you assuming more special-case checking which will require patches
to the KVM side (at a minimum, the KVM needs to keep a list of pages that
should not be changed, and check that at every required point?

Bonus points for properly handling the case where KVM pages an ROE page
out of real memory, and then pages it back into a different page frame (unless
you want to claim it works like mlock() - at which point you need to consider
the implications of a guest permanently mlock()ing up to 16M or 32M or even more).

You might want to stop and ask yourself why the people who did all the W^X
code for the kernel didn't extend it another step and add your ROE scheme.. ;)




-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 486 bytes
Desc: not available
URL: <http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20180706/79016ee9/attachment.sig>

How to change page permission from inside the kernel?

[Rik van Riel]

On Fri, 2018-07-06 at 21:29 +0200, Ahmed Soliman wrote:

> > 2) And why from inside the kernel?
> 
> Because this needs to be done from inside KVM.
> 
> Note: I am aware that this won't be effective against rootkits that
> live in userspace, rootkits that target kernel dynamic data, files on
> disk, as well as VMM escapes, but I believe if the attack surface is
> reduced by just a little bit, it is not that bad, so please lets not
> discuss that :P.

It sounds like the only permission you care about is
the permission of the _guest_ writing to that memory,
not the permission of the qemu-kvm userspace program
writing to that memory.

You may be looking at the wrong page mapping to
manipulate.

-- 
All Rights Reversed.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: This is a digitally signed message part
URL: <http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20180706/6087b7ee/attachment.sig>

How to change page permission from inside the kernel?

[Ahmed Soliman]

> It sounds like the only permission you care about is
> the permission of the _guest_ writing to that memory,
> not the permission of the qemu-kvm userspace program
> writing to that memory.
Yes that is perhaps what I meant.
>
> You may be looking at the wrong page mapping to
> manipulate.

I think that it is still the same mapping because hva created by
qemu-kvm is the one used by the guest (after another layer of paging
?).
I will check TLB specs and verify valdis worries about problems with
interfering data moving to TLB.
But I think I moved away from the original question. So I will re-ask
it again, How can i change set R/O protection to a page from inside a
kernel module ? That was the current problem I was facing.

> --
> All Rights Reversed.

How to change page permission from inside the kernel?

[Ruben Safir]

On 07/06/2018 02:06 PM, Ahmed Soliman wrote:
> I have a memory page allocated with mmap() from user space, This
> address is passed to some kernel module (kvm_intel to be specific) and
> i want to know how can I change the page permission from inside there
> My goal is to achieve something like this
>               mprotect(mem, PAGE_SIZE, PROT_READ)
> except for mprotect can't be called from the kernel, and I couldn't
> find the right way to do it.
> 
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> 

sounds like a good way to make a virus

-- 
So many immigrant groups have swept through our town
that Brooklyn, like Atlantis, reaches mythological
proportions in the mind of the world - RI Safir 1998
http://www.mrbrklyn.com

DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002
http://www.nylxs.com - Leadership Development in Free Software
http://www2.mrbrklyn.com/resources - Unpublished Archive
http://www.coinhangout.com - coins!
http://www.brooklyn-living.com

Being so tracked is for FARM ANIMALS and and extermination camps,
but incompatible with living as a free human being. -RI Safir 2013