Regarding high mem

Regarding high mem

[Kshemendra KP]

   On x86 kernel is normally split into 3GB (user) : 1 GB (Kernel) spaces.
Kernel can only directly
   manipulate 1 GB (around 889 MB) from the PAGE_OFFSET (0xC0000000). The
user space
   memofy below PAGE_OFFSET kernel can't directly access, it considers this
memory as
   high memory. Kernel needs to kmap()/kmap_atomic() map user page and
access that region.

   If the memory is above 4 GB with page extension, it is clear that kernel
can't access it as
   kernel uses "void *" and/or  "unsigned long" to hold the address. But it
is not clear for me
   why kernel's can't directly access memory below PAGE_OFFSET ( 0-3GB)
directly.


Regards

Kshemendra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20121016/64397843/attachment.html 

Regarding high mem

[Mulyadi Santosa]

Hi....

On Tue, Oct 16, 2012 at 7:17 PM, Kshemendra KP <kshemendra@suphalaam.com> wrote:
>
>    On x86 kernel is normally split into 3GB (user) : 1 GB (Kernel) spaces.
> Kernel can only directly
>    manipulate 1 GB (around 889 MB) from the PAGE_OFFSET (0xC0000000). The
> user space
>    memofy below PAGE_OFFSET kernel can't directly access, it considers this
> memory as
>    high memory. Kernel needs to kmap()/kmap_atomic() map user page and
> access that region.

I think there is misunderstanding here.

User space memory are mapped between 0 up to 3 GiB in x86 32 bit.
Therefore it is within the reach of kernel.

Of course, we must be within that process address space. Or in other
word, those user space memory are directly reachable if we load the
page global directory (PGD) used by the related process/task.

If not, you need some ways to access the target's process address
space. I couldn't recall the functions's name though



-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

Regarding high mem

[Chetan Nanda]

On Tue, Oct 16, 2012 at 5:47 PM, Kshemendra KP <kshemendra@suphalaam.com>wrote:

>
>    On x86 kernel is normally split into 3GB (user) : 1 GB (Kernel) spaces.
> Kernel can only directly
>    manipulate 1 GB (around 889 MB) from the PAGE_OFFSET (0xC0000000). The
> user space
>    memofy below PAGE_OFFSET kernel can't directly access, it considers
> this memory as
>    high memory. Kernel needs to kmap()/kmap_atomic() map user page and
> access that region.
>

As per my understanding, if kernel code is running under a process context
then it can access lower 3GB address space, provided address in 0-3GB is in
process address space.  And its not a high memory.
High memory is a virtual address space with 1GB kernel space to map RAM
pages beyond 896MB.



>    If the memory is above 4 GB with page extension, it is clear that
> kernel can't access it as
>    kernel uses "void *" and/or  "unsigned long" to hold the address. But
> it is not clear for me
>    why kernel's can't directly access memory below PAGE_OFFSET ( 0-3GB)
> directly.
>

Kernel can access 0-3GB memory, take an example of application sending data
to kernel via system call e.g. write.
in Write system call pointer to the buffer will be in 0-3Gb address space
of calling application. And kernel will access this address to write data
to drivers/etc ..


>
> Regards
>
> Kshemendra
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
> Chetan Nanda
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20121017/e008baa4/attachment.html 

Regarding high mem

[Kshemendra KP]

On Wed, Oct 17, 2012 at 2:04 PM, Chetan Nanda <chetannanda@gmail.com> wrote:

>
>
> On Tue, Oct 16, 2012 at 5:47 PM, Kshemendra KP <kshemendra@suphalaam.com>wrote:
>
>>
>>    On x86 kernel is normally split into 3GB (user) : 1 GB (Kernel)
>> spaces. Kernel can only directly
>>    manipulate 1 GB (around 889 MB) from the PAGE_OFFSET (0xC0000000). The
>> user space
>>    memofy below PAGE_OFFSET kernel can't directly access, it considers
>> this memory as
>>    high memory. Kernel needs to kmap()/kmap_atomic() map user page and
>> access that region.
>>
>
> As per my understanding, if kernel code is running under a process context
> then it can access lower 3GB address space, provided address in 0-3GB is in
> process address space.  And its not a high memory.
> High memory is a virtual address space with 1GB kernel space to map RAM
> pages beyond 896MB.
>

One thing not clear to me is, during boot,  kernel is copied to 1st MB of
the RAM in x86 architecture as this  architecture
has ISA mem map hole (640k -to 1MB). From 1st BM till 896MB is occupied by
the kernel. Then user space will be
made available beyond 896 MB.  With this reasoning, it is not clear to me,
whether user space which is present
beyond 896 MB is in high memory.

>
>
>
>>    If the memory is above 4 GB with page extension, it is clear that
>> kernel can't access it as
>>    kernel uses "void *" and/or  "unsigned long" to hold the address. But
>> it is not clear for me
>>    why kernel's can't directly access memory below PAGE_OFFSET ( 0-3GB)
>> directly.
>>
>
> Kernel can access 0-3GB memory, take an example of application sending
> data to kernel via system call e.g. write.
> in Write system call pointer to the buffer will be in 0-3Gb address space
> of calling application. And kernel will access this address to write data
> to drivers/etc ..
>
>
>>
>> Regards
>>
>> Kshemendra
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>
>> Chetan Nanda
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20121017/2bcebe46/attachment-0001.html 

Regarding high mem

[Chetan Nanda]

On Oct 17, 2012 2:22 PM, "Kshemendra KP" <kshemendra@suphalaam.com> wrote:
>
>
>
>
> On Wed, Oct 17, 2012 at 2:04 PM, Chetan Nanda <chetannanda@gmail.com>
wrote:
>>
>>
>>
>> On Tue, Oct 16, 2012 at 5:47 PM, Kshemendra KP <kshemendra@suphalaam.com>
wrote:
>>>
>>>
>>>    On x86 kernel is normally split into 3GB (user) : 1 GB (Kernel)
spaces. Kernel can only directly
>>>    manipulate 1 GB (around 889 MB) from the PAGE_OFFSET (0xC0000000).
The user space
>>>    memofy below PAGE_OFFSET kernel can't directly access, it considers
this memory as
>>>    high memory. Kernel needs to kmap()/kmap_atomic() map user page and
access that region.
>>
>>
>> As per my understanding, if kernel code is running under a process
context then it can access lower 3GB address space, provided address in
0-3GB is in process address space.  And its not a high memory.
>> High memory is a virtual address space with 1GB kernel space to map RAM
pages beyond 896MB.
>
>
> One thing not clear to me is, during boot,  kernel is copied to 1st MB of
the RAM in x86 architecture as this  architecture
> has ISA mem map hole (640k -to 1MB). From 1st BM till 896MB is occupied
by the kernel. Then user space will be
> made available beyond 896 MB.  With this reasoning, it is not clear to
me, whether user space which is present
> beyond 896 MB is in high memory.

On x86 kernel will be loaded in RAM from 1st MB. But kernel code and data
will not go upto 896MB.
Kernel code + data will be loaded in ram from 1MB - end_data (IIRC). and
address space between end_data to 896Mb is used for dynamic allocation
(kmalloc ) etc...

I would recomend you to read 2nd chapter of understanding linux kernel.
This chapter contain detailed info on memory addressing used by kernel on
x86
>>
>>
>>
>>>
>>>    If the memory is above 4 GB with page extension, it is clear that
kernel can't access it as
>>>    kernel uses "void *" and/or  "unsigned long" to hold the address.
But it is not clear for me
>>>    why kernel's can't directly access memory below PAGE_OFFSET ( 0-3GB)
directly.
>>
>>
>> Kernel can access 0-3GB memory, take an example of application sending
data to kernel via system call e.g. write.
>> in Write system call pointer to the buffer will be in 0-3Gb address
space of calling application. And kernel will access this address to write
data to drivers/etc ..
>>
>>>
>>>
>>> Regards
>>>
>>> Kshemendra
>>>
>>> _______________________________________________
>>> Kernelnewbies mailing list
>>> Kernelnewbies at kernelnewbies.org
>>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>>
Chetan Nanda
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20121017/e23523b0/attachment.html 

Regarding high mem

[Rik van Riel]

On 10/17/2012 04:52 AM, Kshemendra KP wrote:

> One thing not clear to me is, during boot,  kernel is copied to 1st MB
> of the RAM in x86 architecture as this  architecture
> has ISA mem map hole (640k -to 1MB). From 1st BM till 896MB is occupied
> by the kernel. Then user space will be
> made available beyond 896 MB.  With this reasoning, it is not clear to
> me, whether user space which is present
> beyond 896 MB is in high memory.

You appear to be confusing virtual and physical memory.

Physical addresses 0 through 896MB are mapped at
virtual addresses 3GB through 3GB+896MB.

The 128MB above that are used for vmalloc, and a
few other miscellaneous things.  This 1GB of
kernel virtual memory is the same in every process.

Virtual addresses 0-3GB are used for userspace,
which each process getting its own private 3GB
sized virtual memory area.

Ranges of process virtual memory can be used,
or unused. The used ranges could be backed by
any physical memory in the system (highmem, normal &
dma zones), or not by any memory at all (backed by
swap, or on-disk file pages).

Does that make sense?