[OT??] HELP: Getting lousy memory throughput from Abit KD7

[OT??] HELP: Getting lousy memory throughput from Abit KD7

[Timothy Miller]

I recently bought an Abit KD7, and I'm having a few problems with it.  I
hope someone can please help me with them.

According to the BIOS screen, the board model I have is
KT400-8235-6A6LYA1AC-DN.  I have an Athlon XP 2800+ (barton core,
333mhz fsb) with 1 GIG of PC2700 DDR.  The board has AwardBIOS.

I'm running Red Hat 9 with their latest kernel (something like 2.4.20-18).

The Corsair memory I'm using turns out to be from SpecTek, which uses
Micron silicon.  I called SpecTek to get a datasheet, and they
directed me to the data sheet for Micron MT46V64M4TG-6T.

I use the following settings for memory:

CAS latency: 2.5
Bank Interleave: 4
Trp: 3T
Tras: 7T
Trcd: 3T
Drive strength and controls are all Auto
DRAM Access: 3T
Enhance DRAM Performance: Disabled
DRAM Command Rate: 1T
Write Recovery Time: 3T
tWTR: 1T

The problem that I encounter is that I seem to be getting unusually low 
memory throughput. Using "memtest86", I get 665mb/sec. Using STREAM 
under Linux, I get closer to 1040 mb/sec peak. Doing some searches on 
google has revealed that most people get closer to their peak bandwidth, 
so, I should be expecting at least 2400 from STREAM, judging from the 
numbers others are getting. I know in the real world, you never get 
peak, but in synthetic tests, I should not be getting half the rated 
bandwidth.

I have already tried flashing the latest BIOS but the only result was 
that "Fast CPU command decode" option now causes the system to not POST 
(It didn't before).

If anyone could please help me to figure out what is slowing things
down, I would appreciate it very much.

I have a feeling this isn't a Linux-related issue, but you never know. 
Does STREAM report very low throughput compares to, say, SiSOFT SANDRA? 
  I haven't tried SANDRA because I don't want to install Windows.


Thank you.

[ON TOPIC] HELP: Getting lousy memory throughput from Abit KD7

[Timothy Miller]

Below is a message I wrote earlier to the list, but I didn't see any 
reply.

Since that time, I installed Windows and ran SiSoft SANDRA to test my 
memory throughput.  The result I got was a little over 2300 MiB/sec, 
which is pretty much what I'd expect.  On the other hand, STREAM shows 
me getting about half that while I'm running Linux.

This suggests to me that there is one of these possibilities which is 
contributing to the poor performance under Linux:

1) Linux is not programming the KT400 chipset properly and is thus not 
getting the throughput it could.
2) SANDRA uses instructions (SSE, etc) which are able to access memory 
more efficiently than whatever STREAM is using.
3) SANDRA inflates its scores to make people feel better.
4) Linux has not properly set up the CPU caches.


Given the speed of the processor and the fact that usually, most memory 
access hits the cache, I would expect the memory bus to be saturated 
during the test.  The KT400 does some amount of read-ahead, and since 
the data can be read out of a cache line much faster than it can be 
loaded into the cache, I wouldn't expect much time delay between when 
one line is read and the next one has a miss.  Is the latency so great? 
  What about the out-of-order execution of the CPU?


What I want to know is this:

Am I getting realistic throughput for what STREAM does?

What is SANDRA doing that lets it get such high scores when STREAM does not?


Thanks.




Timothy Miller wrote:
> I recently bought an Abit KD7, and I'm having a few problems with it.  I
> hope someone can please help me with them.
> 
> According to the BIOS screen, the board model I have is
> KT400-8235-6A6LYA1AC-DN.  I have an Athlon XP 2800+ (barton core,
> 333mhz fsb) with 1 GIG of PC2700 DDR.  The board has AwardBIOS.
> 
> I'm running Red Hat 9 with their latest kernel (something like 2.4.20-18).
> 
> The Corsair memory I'm using turns out to be from SpecTek, which uses
> Micron silicon.  I called SpecTek to get a datasheet, and they
> directed me to the data sheet for Micron MT46V64M4TG-6T.
> 
> I use the following settings for memory:
> 
> CAS latency: 2.5
> Bank Interleave: 4
> Trp: 3T
> Tras: 7T
> Trcd: 3T
> Drive strength and controls are all Auto
> DRAM Access: 3T
> Enhance DRAM Performance: Disabled
> DRAM Command Rate: 1T
> Write Recovery Time: 3T
> tWTR: 1T
> 
> The problem that I encounter is that I seem to be getting unusually low 
> memory throughput. Using "memtest86", I get 665mb/sec. Using STREAM 
> under Linux, I get closer to 1040 mb/sec peak. Doing some searches on 
> google has revealed that most people get closer to their peak bandwidth, 
> so, I should be expecting at least 2400 from STREAM, judging from the 
> numbers others are getting. I know in the real world, you never get 
> peak, but in synthetic tests, I should not be getting half the rated 
> bandwidth.
> 
> I have already tried flashing the latest BIOS but the only result was 
> that "Fast CPU command decode" option now causes the system to not POST 
> (It didn't before).
> 
> If anyone could please help me to figure out what is slowing things
> down, I would appreciate it very much.
> 
> I have a feeling this isn't a Linux-related issue, but you never know. 
> Does STREAM report very low throughput compares to, say, SiSOFT SANDRA? 
>  I haven't tried SANDRA because I don't want to install Windows.
> 
> 
> Thank you.
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
> 

Re: [ON TOPIC] HELP: Getting lousy memory throughput from Abit KD7

[Richard B. Johnson]

On Tue, 22 Jul 2003, Timothy Miller wrote:

> Below is a message I wrote earlier to the list, but I didn't see any
> reply.
>
> Since that time, I installed Windows and ran SiSoft SANDRA to test my
> memory throughput.  The result I got was a little over 2300 MiB/sec,
> which is pretty much what I'd expect.  On the other hand, STREAM shows
> me getting about half that while I'm running Linux.
>
> This suggests to me that there is one of these possibilities which is
> contributing to the poor performance under Linux:
>
> 1) Linux is not programming the KT400 chipset properly and is thus not
> getting the throughput it could.
> 2) SANDRA uses instructions (SSE, etc) which are able to access memory
> more efficiently than whatever STREAM is using.
> 3) SANDRA inflates its scores to make people feel better.
> 4) Linux has not properly set up the CPU caches.
>
>
> Given the speed of the processor and the fact that usually, most memory
> access hits the cache, I would expect the memory bus to be saturated
> during the test.  The KT400 does some amount of read-ahead, and since
> the data can be read out of a cache line much faster than it can be
> loaded into the cache, I wouldn't expect much time delay between when
> one line is read and the next one has a miss.  Is the latency so great?
>   What about the out-of-order execution of the CPU?
>
[SNIPPED...]

First, Linux doesn't muck with the memory controller so whatever
the BIOS has set up, you get when Linux runs. Some programs that
test RAM speed do not use 'Wall clock', i.e., actual, time. They
use some other invention. This makes form comparison problems.

If I needed to really find the memory access time, I would write
a program to test it.

(1) Use getttimeofday() for timing.
(2) Record minimum, maximum, and average times. That tells you a lot.
(3) Disconnect the "!#^@$!@*_$" network when you are running the test.
    The broadcast junk on a typical LAN, especially when there are
    M$ machines on it, will eat up so may CPU cycles, you will think
    you are testing an i386!
(4) Warm the cache first by reading everything in the buffer you
    are going to test.
(5) Use large buffers to minimize loop-count overhead.

Properly perform the math, i.e., do the timing in the milliseconds
you get from gettimeofday(), like:

   double_start_time = (double)tb.tv_usecs + ((double)tb.tv_secs * 1e6);

(you put the casts where they will preserve the values correctly)....

You will probably be amazed at how well your system performs. This
dual CPU 400 MHz thing, with 100 MHz memory does 1,900++ MiB/sec.

Basically trust nobody. Make your own test.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.20 on an i686 machine (797.90 BogoMips).
            Note 96.31% of all statistics are fiction.

Re: [ON TOPIC] HELP: Getting lousy memory throughput from Abit KD7

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 730 bytes --]

On Tue, 22 Jul 2003 15:34:51 EDT, "Richard B. Johnson" said:

> If I needed to really find the memory access time, I would write
> a program to test it.

> (4) Warm the cache first by reading everything in the buffer you
>     are going to test.

At which point you're measuring the cache speed not the memory speed.

> You will probably be amazed at how well your system performs. This
> dual CPU 400 MHz thing, with 100 MHz memory does 1,900++ MiB/sec.

Which probably explains this result.  Do a quick sanity check - this number
seems to indicate 20 bytes per memory clock, EVERY clock - either there's
some VERY creative use of prefetching to make sure that you never hit a
cache line miss, or you're measuring the cache.. ;)

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: [ON TOPIC] HELP: Getting lousy memory throughput from Abit KD7

[Dave Jones]

On Tue, Jul 22, 2003 at 03:21:15PM -0400, Timothy Miller wrote:
 > This suggests to me that there is one of these possibilities which is 
 > contributing to the poor performance under Linux:
 > 
 > 1) Linux is not programming the KT400 chipset properly and is thus not 
 > getting the throughput it could.

Linux doesn't do any chipset specific tuning.
 
 > 2) SANDRA uses instructions (SSE, etc) which are able to access memory 
 > more efficiently than whatever STREAM is using.

very likely. its the only way to really saturate the bus.

 > 3) SANDRA inflates its scores to make people feel better.

unlikely.

 > 4) Linux has not properly set up the CPU caches.

Linux doesn't do anything with CPU caches.

 > What I want to know is this:
 > Am I getting realistic throughput for what STREAM does?

I'm unfamiliar with STREAM, so cannot comment

 > What is SANDRA doing that lets it get such high scores when STREAM does not?

Is this even an apples to apples comparison? If not, it's irrelevant.

		Dave

Re: [ON TOPIC] HELP: Getting lousy memory throughput from Abit KD7

[Jamie Lokier]

Dave Jones wrote:
>  > 4) Linux has not properly set up the CPU caches.
> 
> Linux doesn't do anything with CPU caches.

Unless you count the MTRR adjustments that it does.

-- Jamie

Re: [ON TOPIC] HELP: Getting lousy memory throughput from Abit KD7

[Mike Fedyk]

On Tue, Jul 22, 2003 at 10:04:01PM +0100, Dave Jones wrote:
> Is this even an apples to apples comparison? If not, it's irrelevant.

It is comparing a benchmark on Windows to Streams (STREAMS?) on Linux.

One is an apple, and I don't know what the other is.

Re: [OT??] HELP: Getting lousy memory throughput from Abit KD7

[Marc Wilson]

On Thu, Jul 17, 2003 at 05:14:29PM -0400, Timothy Miller wrote:
> I have already tried flashing the latest BIOS but the only result was 
> that "Fast CPU command decode" option now causes the system to not POST 
> (It didn't before).

A 333 mhz FSB CPU will not post on this motherboard if this is enabled with
the latest BIOS.  I have no idea why... I'm not a guru.  I just had a bunch
of CPU's around to test on my KD7-RAID. :)

-- 
 Marc Wilson |     Lavish spending can be disastrous.  Don't buy any
 msw@cox.net |     lavishes for a while.