[RFC][PATCH] Re: 4Gb ram not showing up

[RFC][PATCH] Re: 4Gb ram not showing up

[Bodo Eggert]

Change the description of CONFIG_*HIGHMEM* to reflect "lost" memory due to 
PCI space and the existence of the NX flag.

Signed-Off-By: Bodo Eggert <7eggert@gmx.de>
---
I made this quick patch using the information from LKML as I remembered 
it. Please verify.

--- 2.6.21/arch/i386/Kconfig.ori	2007-06-06 13:41:09.000000000 +0200
+++ 2.6.21/arch/i386/Kconfig	2007-06-06 14:07:40.000000000 +0200
@@ -495,8 +495,8 @@ config NOHIGHMEM
 	bool "off"
 	depends on !X86_NUMAQ
 	---help---
-	  Linux can use up to 64 Gigabytes of physical memory on x86 systems.
-	  However, the address space of 32-bit x86 processors is only 4
+	  Linux can use up to 64 Gigabytes of physical address space on x86
+	  systems. However, the address space of 32-bit x86 processors is only 4
 	  Gigabytes large. That means that, if you have a large amount of
 	  physical memory, not all of it can be "permanently mapped" by the
 	  kernel. The physical memory that's not permanently mapped is called
@@ -510,8 +510,15 @@ config NOHIGHMEM
 	  by the kernel to permanently map as much physical memory as
 	  possible.
 
-	  If the machine has between 1 and 4 Gigabytes physical RAM, then
+
+	  If the machine has between 1 and 3.5 Gigabytes physical RAM, then
 	  answer "4GB" here.
+	  
+	  The PCI address space will usurally take 512 MB or 1 GB of address
+	  space. This address space is unavailable to RAM, but depending on the
+	  chipset (and BIOS settings), memory overlapping the PCI address space
+	  may be mapped beyond the 4 GB limit and be available using "64GB".
+
 
 	  If more than 4 Gigabytes is used then answer "64GB" here. This
 	  selection turns Intel PAE (Physical Address Extension) mode on.
@@ -520,6 +527,10 @@ config NOHIGHMEM
 	  processors (Pentium Pro and better). NOTE: If you say "64GB" here,
 	  then the kernel will not boot on CPUs that don't support PAE!
 
+	  An additional benefit of the 64GB-Mode is the availability of the
+	  no-execute-pageflag, which can be used to prevent some attacks from
+	  injecting malicious code into applications.
+
 	  The actual amount of total physical memory will either be
 	  auto detected or can be forced by using a kernel command line option
 	  such as "mem=256M". (Try "man bootparam" or see the documentation of
@@ -532,14 +543,14 @@ config HIGHMEM4G
 	bool "4GB"
 	depends on !X86_NUMAQ
 	help
-	  Select this if you have a 32-bit processor and between 1 and 4
+	  Select this if you have a 32-bit processor and between 1 and 3.5
 	  gigabytes of physical RAM.
 
 config HIGHMEM64G
-	bool "64GB"
+	bool "64GB (enables no-execute memory protection if available)"
 	depends on X86_CMPXCHG64
 	help
-	  Select this if you have a 32-bit processor and more than 4
+	  Select this if you have a 32-bit processor and more than 3.5
 	  gigabytes of physical RAM.
 
 endchoice
-- 
RAM DISK is not an installation procedure! 

Re: [RFC][PATCH] Re: 4Gb ram not showing up

[Lennart Sorensen]

On Wed, Jun 06, 2007 at 02:12:22PM +0200, Bodo Eggert wrote:
> Change the description of CONFIG_*HIGHMEM* to reflect "lost" memory due to 
> PCI space and the existence of the NX flag.
> 
> Signed-Off-By: Bodo Eggert <7eggert@gmx.de>
> ---
> I made this quick patch using the information from LKML as I remembered 
> it. Please verify.
> 
> --- 2.6.21/arch/i386/Kconfig.ori	2007-06-06 13:41:09.000000000 +0200
> +++ 2.6.21/arch/i386/Kconfig	2007-06-06 14:07:40.000000000 +0200
> @@ -495,8 +495,8 @@ config NOHIGHMEM
>  	bool "off"
>  	depends on !X86_NUMAQ
>  	---help---
> -	  Linux can use up to 64 Gigabytes of physical memory on x86 systems.
> -	  However, the address space of 32-bit x86 processors is only 4
> +	  Linux can use up to 64 Gigabytes of physical address space on x86
> +	  systems. However, the address space of 32-bit x86 processors is only 4
>  	  Gigabytes large. That means that, if you have a large amount of
>  	  physical memory, not all of it can be "permanently mapped" by the
>  	  kernel. The physical memory that's not permanently mapped is called
> @@ -510,8 +510,15 @@ config NOHIGHMEM
>  	  by the kernel to permanently map as much physical memory as
>  	  possible.
>  
> -	  If the machine has between 1 and 4 Gigabytes physical RAM, then
> +
> +	  If the machine has between 1 and 3.5 Gigabytes physical RAM, then
>  	  answer "4GB" here.
> +	  
> +	  The PCI address space will usurally take 512 MB or 1 GB of address
                                     usually

> +	  space. This address space is unavailable to RAM, but depending on the
> +	  chipset (and BIOS settings), memory overlapping the PCI address space
> +	  may be mapped beyond the 4 GB limit and be available using "64GB".
> +
>  
>  	  If more than 4 Gigabytes is used then answer "64GB" here. This
>  	  selection turns Intel PAE (Physical Address Extension) mode on.
> @@ -520,6 +527,10 @@ config NOHIGHMEM
>  	  processors (Pentium Pro and better). NOTE: If you say "64GB" here,
>  	  then the kernel will not boot on CPUs that don't support PAE!
>  
> +	  An additional benefit of the 64GB-Mode is the availability of the
> +	  no-execute-pageflag, which can be used to prevent some attacks from
> +	  injecting malicious code into applications.
> +
>  	  The actual amount of total physical memory will either be
>  	  auto detected or can be forced by using a kernel command line option
>  	  such as "mem=256M". (Try "man bootparam" or see the documentation of
> @@ -532,14 +543,14 @@ config HIGHMEM4G
>  	bool "4GB"
>  	depends on !X86_NUMAQ
>  	help
> -	  Select this if you have a 32-bit processor and between 1 and 4
> +	  Select this if you have a 32-bit processor and between 1 and 3.5
>  	  gigabytes of physical RAM.
>  
>  config HIGHMEM64G
> -	bool "64GB"
> +	bool "64GB (enables no-execute memory protection if available)"
>  	depends on X86_CMPXCHG64
>  	help
> -	  Select this if you have a 32-bit processor and more than 4
> +	  Select this if you have a 32-bit processor and more than 3.5
>  	  gigabytes of physical RAM.
>  
>  endchoice

Seems like an improvement to me.  To fully explain how it could be 3 or
3.5 or 3.25 or who knows how many GB you can actually use without PAE
would probably require writing a small novel.  Certainly talking about
address space instead of amounts of physical memory is more correct.

--
Len Sorensen

Re: [RFC][PATCH] Re: 4Gb ram not showing up

[Wakko Warner]

Lennart Sorensen wrote:
> On Wed, Jun 06, 2007 at 02:12:22PM +0200, Bodo Eggert wrote:
> > Change the description of CONFIG_*HIGHMEM* to reflect "lost" memory due to 
> > PCI space and the existence of the NX flag.
> > 
> > Signed-Off-By: Bodo Eggert <7eggert@gmx.de>
> > ---
> > I made this quick patch using the information from LKML as I remembered 
> > it. Please verify.
> > 
> > @@ -510,8 +510,15 @@ config NOHIGHMEM
> >  	  by the kernel to permanently map as much physical memory as
> >  	  possible.
> >  
> > -	  If the machine has between 1 and 4 Gigabytes physical RAM, then
> > +
> > +	  If the machine has between 1 and 3.5 Gigabytes physical RAM, then
> >  	  answer "4GB" here.
> > +	  

Might be better to say 3 Gigabytes instead of 3.5.  I booted a 4gb kernel on
a machine with 4gb at work and only saw 3gb of memory.

Also, I've noticed that there's ~128mb of memory lost if you select 1gb when
you have 1gb of memory installed.

> >  	  If more than 4 Gigabytes is used then answer "64GB" here. This
> >  	  selection turns Intel PAE (Physical Address Extension) mode on.
> > @@ -520,6 +527,10 @@ config NOHIGHMEM
> >  	  processors (Pentium Pro and better). NOTE: If you say "64GB" here,
> >  	  then the kernel will not boot on CPUs that don't support PAE!
> >  
> > +	  An additional benefit of the 64GB-Mode is the availability of the
> > +	  no-execute-pageflag, which can be used to prevent some attacks from
> > +	  injecting malicious code into applications.
> > +
> >  	  The actual amount of total physical memory will either be
> >  	  auto detected or can be forced by using a kernel command line option
> >  	  such as "mem=256M". (Try "man bootparam" or see the documentation of
> > @@ -532,14 +543,14 @@ config HIGHMEM4G
> >  	bool "4GB"
> >  	depends on !X86_NUMAQ
> >  	help
> > -	  Select this if you have a 32-bit processor and between 1 and 4
> > +	  Select this if you have a 32-bit processor and between 1 and 3.5
> >  	  gigabytes of physical RAM.
> >  
> >  config HIGHMEM64G
> > -	bool "64GB"
> > +	bool "64GB (enables no-execute memory protection if available)"
> >  	depends on X86_CMPXCHG64
> >  	help
> > -	  Select this if you have a 32-bit processor and more than 4
> > +	  Select this if you have a 32-bit processor and more than 3.5
> >  	  gigabytes of physical RAM.

Same here

> Seems like an improvement to me.  To fully explain how it could be 3 or
> 3.5 or 3.25 or who knows how many GB you can actually use without PAE
> would probably require writing a small novel.  Certainly talking about
> address space instead of amounts of physical memory is more correct.

It's probably worth it to just say 3gb.

I've looked at the cpuinfo for several machines, all of which (pIII and
better, not all physically support even 2gb of memory) show pae in the
flags, so is there really a reason not to just specify 64gb always?

The oldest machine I have access to is a pII 333mhz machine, cpuinfo also
lists pae in the cpu flags.

-- 
 Lab tests show that use of micro$oft causes cancer in lab animals
 Got Gas???

Re: [RFC][PATCH] Re: 4Gb ram not showing up

[H. Peter Anvin]

Lennart Sorensen wrote:
> 
> Seems like an improvement to me.  To fully explain how it could be 3 or
> 3.5 or 3.25 or who knows how many GB you can actually use without PAE
> would probably require writing a small novel.  Certainly talking about
> address space instead of amounts of physical memory is more correct.
> 

On some machines it's even just 2 GB.

	-hpa

Re: [RFC][PATCH] Re: 4Gb ram not showing up

[Andrew Lyon]

On 6/6/07, H. Peter Anvin <hpa@zytor.com> wrote:
> Lennart Sorensen wrote:
> >
> > Seems like an improvement to me.  To fully explain how it could be 3 or
> > 3.5 or 3.25 or who knows how many GB you can actually use without PAE
> > would probably require writing a small novel.  Certainly talking about
> > address space instead of amounts of physical memory is more correct.
> >
>
> On some machines it's even just 2 GB.
>
>         -hpa
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Could this also cause a system to be unstable? my abit athlon64 at
work will not run x64 with more than 1gb ram, and i have a colo server
with supermicro & 2 x dual core xeons that will not run with more than
2gb.

Both systems have long uptimes but if i add ram they crash within
minutes of booting.

Tried several kernels up to 2.6.21 and gave up, I can send dmesg
output but the crashes are completely random.

andy

Re: [RFC][PATCH] Re: 4Gb ram not showing up

[Tom Moore]

This looks like it would have probably given me the hint I needed.  Can 
I also suggest that you make a change to arch/i386/kernel/setup.c, line 296:

                if (max_pfn > MAX_NONPAE_PFN) {
                        max_pfn = MAX_NONPAE_PFN;
                        printk(KERN_WARNING "Warning only 4GB will be 
used.\n");
-                        printk(KERN_WARNING "Use a PAE enabled kernel.\n");
+                        printk(KERN_WARNING "Use a PAE enabled kernel 
by selecting the HIGHMEM64G configuration option.\n");
                }

It does not seem that PAE is a configuration option that is exposed 
through make menuconfig.  How else are people supposed to know that PAE 
is a side effect of CONFIG_HIGHMEM64G unless they know the kbuild logic?

Tom

Bodo Eggert wrote:
> Change the description of CONFIG_*HIGHMEM* to reflect "lost" memory due to 
> PCI space and the existence of the NX flag.
>
> Signed-Off-By: Bodo Eggert <7eggert@gmx.de>
> ---
> I made this quick patch using the information from LKML as I remembered 
> it. Please verify.
>
> --- 2.6.21/arch/i386/Kconfig.ori	2007-06-06 13:41:09.000000000 +0200
> +++ 2.6.21/arch/i386/Kconfig	2007-06-06 14:07:40.000000000 +0200
> @@ -495,8 +495,8 @@ config NOHIGHMEM
>  	bool "off"
>  	depends on !X86_NUMAQ
>  	---help---
> -	  Linux can use up to 64 Gigabytes of physical memory on x86 systems.
> -	  However, the address space of 32-bit x86 processors is only 4
> +	  Linux can use up to 64 Gigabytes of physical address space on x86
> +	  systems. However, the address space of 32-bit x86 processors is only 4
>  	  Gigabytes large. That means that, if you have a large amount of
>  	  physical memory, not all of it can be "permanently mapped" by the
>  	  kernel. The physical memory that's not permanently mapped is called
> @@ -510,8 +510,15 @@ config NOHIGHMEM
>  	  by the kernel to permanently map as much physical memory as
>  	  possible.
>  
> -	  If the machine has between 1 and 4 Gigabytes physical RAM, then
> +
> +	  If the machine has between 1 and 3.5 Gigabytes physical RAM, then
>  	  answer "4GB" here.
> +	  
> +	  The PCI address space will usurally take 512 MB or 1 GB of address
> +	  space. This address space is unavailable to RAM, but depending on the
> +	  chipset (and BIOS settings), memory overlapping the PCI address space
> +	  may be mapped beyond the 4 GB limit and be available using "64GB".
> +
>  
>  	  If more than 4 Gigabytes is used then answer "64GB" here. This
>  	  selection turns Intel PAE (Physical Address Extension) mode on.
> @@ -520,6 +527,10 @@ config NOHIGHMEM
>  	  processors (Pentium Pro and better). NOTE: If you say "64GB" here,
>  	  then the kernel will not boot on CPUs that don't support PAE!
>  
> +	  An additional benefit of the 64GB-Mode is the availability of the
> +	  no-execute-pageflag, which can be used to prevent some attacks from
> +	  injecting malicious code into applications.
> +
>  	  The actual amount of total physical memory will either be
>  	  auto detected or can be forced by using a kernel command line option
>  	  such as "mem=256M". (Try "man bootparam" or see the documentation of
> @@ -532,14 +543,14 @@ config HIGHMEM4G
>  	bool "4GB"
>  	depends on !X86_NUMAQ
>  	help
> -	  Select this if you have a 32-bit processor and between 1 and 4
> +	  Select this if you have a 32-bit processor and between 1 and 3.5
>  	  gigabytes of physical RAM.
>  
>  config HIGHMEM64G
> -	bool "64GB"
> +	bool "64GB (enables no-execute memory protection if available)"
>  	depends on X86_CMPXCHG64
>  	help
> -	  Select this if you have a 32-bit processor and more than 4
> +	  Select this if you have a 32-bit processor and more than 3.5
>  	  gigabytes of physical RAM.
>  
>  endchoice
>   

Re: [RFC][PATCH] Re: 4Gb ram not showing up

[H. Peter Anvin]

Andrew Lyon wrote:
> 
> Could this also cause a system to be unstable? my abit athlon64 at
> work will not run x64 with more than 1gb ram, and i have a colo server
> with supermicro & 2 x dual core xeons that will not run with more than
> 2gb.
> 
> Both systems have long uptimes but if i add ram they crash within
> minutes of booting.
> 

That's much more likely to be caused by bad memory bus layout so that it
doesn't handle the higher loading.

Run memtest86+ on the expanded memory for some time.

	-hpa

Re: [RFC][PATCH] Re: 4Gb ram not showing up

[Andrew Lyon]

On 6/7/07, H. Peter Anvin <hpa@zytor.com> wrote:
> Andrew Lyon wrote:
> >
> > Could this also cause a system to be unstable? my abit athlon64 at
> > work will not run x64 with more than 1gb ram, and i have a colo server
> > with supermicro & 2 x dual core xeons that will not run with more than
> > 2gb.
> >
> > Both systems have long uptimes but if i add ram they crash within
> > minutes of booting.
> >
>
> That's much more likely to be caused by bad memory bus layout so that it
> doesn't handle the higher loading.
>
> Run memtest86+ on the expanded memory for some time.
>
>         -hpa
>

I have run memtest86+ for days at a time on both systems with no errors at all.

broken hardware i guess, i am not surprised about the abit board, ive
had nothing but trouble with abit motherboards and do not use them any
more, but the supermicro is a xeon server board and i was using
kingston ram, i wouldnt expect that to have issues.

latest bios on both.

Andy

Re: [RFC][PATCH] Re: 4Gb ram not showing up

[H. Peter Anvin]

Andrew Lyon wrote:
> 
> I have run memtest86+ for days at a time on both systems with no errors
> at all.
> 
> broken hardware i guess, i am not surprised about the abit board, ive
> had nothing but trouble with abit motherboards and do not use them any
> more, but the supermicro is a xeon server board and i was using
> kingston ram, i wouldnt expect that to have issues.
> 
> latest bios on both.
> 

And have you verified that it's specced for the memory load you're
putting on it?  Not all platforms can use every slot with certain kinds
of DIMMs, depending on chip and rank count.

	-hpa

Re: [RFC][PATCH] Re: 4Gb ram not showing up

[Matt Keenan]

Andrew Lyon wrote:
> On 6/7/07, H. Peter Anvin <hpa@zytor.com> wrote:
>> Andrew Lyon wrote:
>> >
>> > Could this also cause a system to be unstable? my abit athlon64 at
>> > work will not run x64 with more than 1gb ram, and i have a colo server
>> > with supermicro & 2 x dual core xeons that will not run with more than
>> > 2gb.
>> >
>> > Both systems have long uptimes but if i add ram they crash within
>> > minutes of booting.
>> >
>>
>> That's much more likely to be caused by bad memory bus layout so that it
>> doesn't handle the higher loading.
>>
>> Run memtest86+ on the expanded memory for some time.
>>
>>         -hpa
>>
>
> I have run memtest86+ for days at a time on both systems with no
> errors at all.
>
> broken hardware i guess, i am not surprised about the abit board, ive
> had nothing but trouble with abit motherboards and do not use them any
> more, but the supermicro is a xeon server board and i was using
> kingston ram, i wouldnt expect that to have issues.
I have had problems with kingston ram and heavily populated boards
(being ASUS boards didn't help either).  I have found through trial and
error that gigabyte and/or msi boards with crucial memory seems to work
the best, and for servers i always buy low latency ecc ram, even if it
is a slower frequency.  I also had problems with machines that passed
memtest but failed in real life.. you may also want to check your power
supply and make sure you buy boards with multiple (6 or more) voltage
regulators.

Matt

Re: [RFC][PATCH] Re: 4Gb ram not showing up

[Robert Hancock]

Andrew Lyon wrote:
> Could this also cause a system to be unstable? my abit athlon64 at
> work will not run x64 with more than 1gb ram, and i have a colo server
> with supermicro & 2 x dual core xeons that will not run with more than
> 2gb.
> 
> Both systems have long uptimes but if i add ram they crash within
> minutes of booting.
> 
> Tried several kernels up to 2.6.21 and gave up, I can send dmesg
> output but the crashes are completely random.

That is likely some other problem, like bad RAM or memory timing issues.

-- 
Robert Hancock      Saskatoon, SK, Canada
To email, remove "nospam" from hancockr@nospamshaw.ca
Home Page: http://www.roberthancock.com/