Paging of Kernel Memory

Paging of Kernel Memory

[ashish anand]

Hi
on wed 7th sep  Christopher Harvey wrote

>>It means that it can't be swapped to your swap partition, even if
>>you're not using it.

 this thing I understood it pretty well but what about the line
"Therefore, every byte of
 memory you consume is one less byte of available physical memory".What is
the meaning of this line and why it is so.


On Tue, Sep 6, 2011 at 9:30 PM, <kernelnewbies-request@kernelnewbies.org>wrote:

> Send Kernelnewbies mailing list submissions to
>        kernelnewbies at kernelnewbies.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>        http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> or, via email, send a message with subject or body 'help' to
>        kernelnewbies-request at kernelnewbies.org
>
> You can reach the person managing the list at
>        kernelnewbies-owner at kernelnewbies.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Kernelnewbies digest..."
>
>
> Today's Topics:
>
>   1. Re: Re: Regarding threaded irq and normal irq (anish singh)
>   2. CRC calculation in Module.symvers (Shakthi Kannan)
>   3. Re: Finding physical and logical core number in a kernel
>      module ? (Alexandre Courbot)
>   4. How to identify Connected USB Storage device (Harddisk, flash
>      Memory    or SSD) (mani)
>   5. Paging of Kernel Memory (Vijay Chauhan)
>   6. Re: Paging of Kernel Memory (Christopher Harvey)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 6 Sep 2011 13:57:22 +0530
> From: anish singh <anish198519851985@gmail.com>
> Subject: Re: Re: Regarding threaded irq and normal irq
> To: Mulyadi Santosa <mulyadi.santosa@gmail.com>
> Cc: Greg KH <greg@kroah.com>, tglx at linutronix.de,       sandeep kumar
>        <coolsandyforyou@gmail.com>, kernelnewbies at kernelnewbies.org
> Message-ID:
>        <CAK7N6voUN+b8nj=nQNoU1wGVr=b9jCm+0Q43eHg7bPLcgB4Nzw@mail.gmail.com
> >
> Content-Type: text/plain; charset=UTF-8
>
> On Tue, Sep 6, 2011 at 10:59 AM, anish singh
> <anish198519851985@gmail.com> wrote:
> > On Tue, Sep 6, 2011 at 10:51 AM, Mulyadi Santosa
> > <mulyadi.santosa@gmail.com> wrote:
> >> Hi :)
> >>
> >> On 06/09/2011, sandeep kumar <coolsandyforyou@gmail.com> wrote:
> >>> Hmm..But when i tried to take a mutex lock in threaded_irq, it is
> throwing a
> >>> warning message
> >>> "BUG: sleeping function called from invalid context"... So i was
> wondering
> >>> which way it is..
> >>
> >> Please don't top post :)
> >>
> >> Anyway, regarding that "BUG:" message, could it be that mutex function
> >> mistakenly assume it's called inside top half, thus bails out since
> > How can mutex assume? IMHO the message would have printed
> > because sleeping would be forbidden there but again AFAIK in the threaded
> > handler we can do I2C transactions which can sleep.So kind of
> > paradox there.
>
> >>something like using in_irq() inside the threaded irq handler?
> I think after going thorough the code of mutex_lock we can find out the
> reason why it is printing the warning eventhough we are calling it in a
> threaded handler i.e. kernel thread.
>
> >From the code:
> 126 /**
> 127  * might_sleep - annotation for functions that can sleep
> 128  *
> 129  * this macro will print a stack trace if it is executed in an atomic
> 130  * context (spinlock, irq-handler, ...).
>
> 129 line no is important.
> This might_sleep is called by mutex_lock function and in the defintion
> of __might_sleep  i found below code.I think this below if condition is
> false because irqs_disabled is true.This causes the warning.
>
>         if ((preempt_count_equals(preempt_offset) && !irqs_disabled()) ||
>             system_state != SYSTEM_RUNNING || oops_in_progress)
>               return;
>
> Hope i am right?
>
> > ps:I have seen the code where sandeep is doing the taking the mutex lock.
> > He is taking mutex lock at the beginging of the handler and releasing
> > at the end.
> >> sleeping is forbidden there? which mutex function do you use? (sorry I
> >> don't follow every locking functions exist now)
> >>
> >> --
> >> regards,
> >>
> >> Mulyadi Santosa
> >> Freelance Linux trainer and consultant
> >>
> >> blog: the-hydra.blogspot.com
> >> training: mulyaditraining.blogspot.com
> >>
> >> _______________________________________________
> >> Kernelnewbies mailing list
> >> Kernelnewbies at kernelnewbies.org
> >> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> >>
> >
>
>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 6 Sep 2011 12:41:24 +0530
> From: Shakthi Kannan <shakthimaan@gmail.com>
> Subject: CRC calculation in Module.symvers
> To: kernelnewbies at kernelnewbies.org
> Message-ID:
>        <CABG-yt2W02ToqJJ=-n773Lg-JCRnDYc7qzA0HsCw+morgnSx8Q@mail.gmail.com
> >
> Content-Type: text/plain; charset=UTF-8
>
> Greetings!
>
> I would like to know as to:
>
> 1. How the CRC is computed for an exported symbol in the Linux kernel, and
> 2. Given a kernel .c file, how can I obtain the CRC checksums for a symbol.
>
> For linux-3.0.2 (example), I compiled with CONFIG_MODVERSIONS=y, and
> running make built vmlinux, and generated the Module.symvers file. I
> believe genksyms is used to generate the CRC information.
>
> I had referred the "Kernel Symbols and CONFIG_MODVERSIONS"
> documentation [1] written by Mark McLoughlin. I tried the following in
> the top-level kernel sources:
>
>  $ gcc -E -D__GENKSYMS__ -DCONFIG_MODVERSIONS -DEXPORT_SYMTAB
> crypto/api.c -I/tmp/linux-3.0.2/include
> -I/tmp/linux-3.0.2/arch/x86/include | ./scripts/genksyms/genksyms
>
> and I get the following error output:
>
> === Output ===
>
> /tmp/linux-3.0.2/include/linux/jiffies.h:43:42: error: division by zero in
> #if
> In file included from /tmp/linux-3.0.2/include/linux/tracepoint.h:19:0,
>                 from /tmp/linux-3.0.2/include/linux/module.h:18,
>                 from crypto/api.c:22:
> /tmp/linux-3.0.2/include/linux/rcupdate.h:160:2: error: #error
> "Unknown RCU implementation specified to kernel configuration"
> In file included from /tmp/linux-3.0.2/include/linux/module.h:21:0,
>                 from crypto/api.c:22:
> /tmp/linux-3.0.2/arch/x86/include/asm/module.h:59:2: error: #error
> unknown processor family
>
> === End ===
>
> The documentation for genksyms is old, and now there is no -k option?
>
> Appreciate any inputs in this regard,
>
> Thanks!
>
> SK
>
> [1] Kernel Symbols and CONFIG_MODVERSIONS.
> http://www.skynet.ie/~mark/pub/symbols.txt
>
> --
> Shakthi Kannan
> http://www.shakthimaan.com
>
>
>
> ------------------------------
>
> Message: 3
> Date: Tue, 6 Sep 2011 16:14:19 +0900
> From: Alexandre Courbot <gnurou@gmail.com>
> Subject: Re: Finding physical and logical core number in a kernel
>        module ?
> To: Vaibhav Jain <vjoss197@gmail.com>
> Cc: kernelnewbies at kernelnewbies.org
> Message-ID:
>        <CAAVeFu+h9=-roQcJaoh_d4XNbm8BtqtbQRR7qrWgx0B-k27dHA@mail.gmail.com
> >
> Content-Type: text/plain; charset=UTF-8
>
> > I agree that the kernel module is not restricted to any core but is it
> > possible that whenever it is executed
> > it gives the core number whichever is executing it ?
>
> Have a look at the end of include/linux/smp.h: smp_processor_id()
> gives you the number of the current executing CPU. get_cpu() will do
> the same and will also disable preemption so that you will stay on
> that CPU until put_cpu() is called.
>
> >From user-space, you can use sched_getcpu() to obtain the same
> information.
>
> Hope this helps,
> Alex.
>
>
>
> ------------------------------
>
> Message: 4
> Date: Tue, 6 Sep 2011 16:02:33 +0530
> From: mani <manishrma@gmail.com>
> Subject: How to identify Connected USB Storage device (Harddisk, flash
>        Memory  or SSD)
> To: kernelnewbies at kernelnewbies.org
> Message-ID:
>        <CAB+TZU93_edNTfbNVKvQdgWiJxDTNHWWunNT6VNbHP+=RTixfw@mail.gmail.com
> >
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi all,
>
> Is there any way to find out the type of the connected USB disk (Hard
> disk/SDD)?
>
> AFAIK there is a significant performance difference in Harddisk/SDD
> devices.
>
> Previously i though that the rotational flag in the sysfs will distinguish
> them but it is same for both the types
>
> #cat /sys/block/sda/queue/rotational
> 1 for both flash & HDD
>
> Is there any way to distinguish the USB Harddisk or SDD drives ?
>
> Thanks & Regards,
> Manish
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110906/f2bd2875/attachment-0001.html
>
> ------------------------------
>
> Message: 5
> Date: Tue, 6 Sep 2011 17:48:12 +0530
> From: Vijay Chauhan <kernel.vijay@gmail.com>
> Subject: Paging of Kernel Memory
> To: kernelnewbies at kernelnewbies.org
> Message-ID:
>        <CAJ61zBDd4tH7jn2YEpDSmbaTXODUqpjOKanPsk0sqgHbKyek8w@mail.gmail.com
> >
> Content-Type: text/plain; charset=ISO-8859-1
>
> Hi,
>
> i was going through the Linux Kernel Development book. It mention the
> following:
>
> "Additionally, kernel memory is not pageable.Therefore, every byte of
> memory you consume is one less byte of available physical memory."
>
> What is the meaning of 'Kernel memory is not pageable'? Anyone can
> give details would help me in understanding.
>
> Thanks,
> Vijay
>
>
>
> ------------------------------
>
> Message: 6
> Date: Tue, 06 Sep 2011 09:38:26 -0500
> From: Christopher Harvey <chris@basementcode.com>
> Subject: Re: Paging of Kernel Memory
> To: <kernelnewbies@kernelnewbies.org>
> Message-ID: <1d7f4b0da950c823cb12c54a4d555234@basementcode.com>
> Content-Type: text/plain; charset=UTF-8; format=flowed
>
> On Tue, 6 Sep 2011 17:48:12 +0530, Vijay Chauhan wrote:
> > Hi,
> >
> > i was going through the Linux Kernel Development book. It mention the
> > following:
> >
> > "Additionally, kernel memory is not pageable.Therefore, every byte of
> > memory you consume is one less byte of available physical memory."
> >
> > What is the meaning of 'Kernel memory is not pageable'? Anyone can
> > give details would help me in understanding.
> >
> > Thanks,
> > Vijay
> >
> It means that it can't be swapped to your swap partition, even if
> you're not using it.
>
>
>
> ------------------------------
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>
>
> End of Kernelnewbies Digest, Vol 10, Issue 8
> ********************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110907/c930c259/attachment-0001.html 

Paging of Kernel Memory

[Mulyadi Santosa]

Hi :)

On Wed, Sep 7, 2011 at 09:44, ashish anand <ashishanand26cs@gmail.com> wrote:
> Hi
> on wed 7th sep??Christopher Harvey wrote
>>>It means that it can't be swapped to your swap partition, even if
>>>you're not using it.
>
> ?this thing I understood it pretty well but what about the line
> "Therefore, every byte of
> ?memory you consume is one less byte of available physical memory".What is
> the meaning of this line and why it is so.

could you please next time cut out the unrelated message? :)

Anyway, that passage means that the bigger your kernel image is, the
lesser your free memory are. This is simply because your kernel image
is entirely loaded and locked in RAM.

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

Paging of Kernel Memory

[Rik van Riel]

On 09/07/2011 12:41 AM, Mulyadi Santosa wrote:
> Hi :)
>
> On Wed, Sep 7, 2011 at 09:44, ashish anand<ashishanand26cs@gmail.com>  wrote:
>> Hi
>> on wed 7th sep  Christopher Harvey wrote
>>>> It means that it can't be swapped to your swap partition, even if
>>>> you're not using it.
>>
>>   this thing I understood it pretty well but what about the line
>> "Therefore, every byte of
>>   memory you consume is one less byte of available physical memory".What is
>> the meaning of this line and why it is so.
>
> could you please next time cut out the unrelated message? :)
>
> Anyway, that passage means that the bigger your kernel image is, the
> lesser your free memory are. This is simply because your kernel image
> is entirely loaded and locked in RAM.

This was a concern back in the day where systems had 4MB of
RAM and the kernel took up 1MB.

However, nowadays the kernel takes up a few MB in a system
with several GB of memory, so it really no longer matters...

-- 
All rights reversed.

Paging of Kernel Memory

[Asutosh Das]

Hi
I think it also can mean that there is a 1-1 mapping in the kernel(except
for high mem) as there cannot be any page-faults in the kernel. So for
memory allocated in kernel, a physical memory mapping exists, thus lessening
the available physical memory.


On 6 September 2011 17:48, Vijay Chauhan <kernel.vijay@gmail.com> wrote:

> Hi,
>
> i was going through the Linux Kernel Development book. It mention the
> following:
>
> "Additionally, kernel memory is not pageable.Therefore, every byte of
> memory you consume is one less byte of available physical memory."
>
> What is the meaning of 'Kernel memory is not pageable'? Anyone can
> give details would help me in understanding.
>
> Thanks,
> Vijay
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>



-- 
~/asd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110908/4d827fa9/attachment.html 

Paging of Kernel Memory

[Christopher Harvey]

On Tue, 6 Sep 2011 17:48:12 +0530, Vijay Chauhan wrote:
> Hi,
>
> i was going through the Linux Kernel Development book. It mention the
> following:
>
> "Additionally, kernel memory is not pageable.Therefore, every byte of
> memory you consume is one less byte of available physical memory."
>
> What is the meaning of 'Kernel memory is not pageable'? Anyone can
> give details would help me in understanding.
>
> Thanks,
> Vijay
>
It means that it can't be swapped to your swap partition, even if 
you're not using it.

Paging of Kernel Memory

[Vijay Chauhan]

Hi,

i was going through the Linux Kernel Development book. It mention the following:

"Additionally, kernel memory is not pageable.Therefore, every byte of
memory you consume is one less byte of available physical memory."

What is the meaning of 'Kernel memory is not pageable'? Anyone can
give details would help me in understanding.

Thanks,
Vijay