Re: horrible disk thorughput on itanium

Re: horrible disk thorughput on itanium

[Andi Kleen]

Andrew Morton <akpm@zip.com.au> writes:
> 
> The fact that you get the same throughput on each platform with
> the block I/O part of the test indicates that the hardware and
> kernel are OK, but the C library is broken.

The usual difference is if you have a pthreads capable C library
or not. For newer glibc bonnie++ should definitely use 
putc_unlocked(); otherwise it'll eat lock overhead for each character
to take the FILE lock. 

As far as I can see bonnie++ doesn't use putc_unlocked, but putc.

With libc5 it likely would magically get a lot faster @)

-Andi

Re: horrible disk thorughput on itanium

[Mike Galbraith]

On 6 Dec 2001, Andi Kleen wrote:

> Andrew Morton <akpm@zip.com.au> writes:
> >
> > The fact that you get the same throughput on each platform with
> > the block I/O part of the test indicates that the hardware and
> > kernel are OK, but the C library is broken.
>
> The usual difference is if you have a pthreads capable C library
> or not. For newer glibc bonnie++ should definitely use
> putc_unlocked(); otherwise it'll eat lock overhead for each character
> to take the FILE lock.
>
> As far as I can see bonnie++ doesn't use putc_unlocked, but putc.

Plain old Bonnie suffered from the same thing.  I long ago made it
use putc_unlocked() here because throughput was horrible otherwise.

	-Mike

Re: horrible disk thorughput on itanium

[Linus Torvalds]

In article <Pine.LNX.4.33.0112070710120.747-100000@mikeg.weiden.de>,
Mike Galbraith  <mikeg@wen-online.de> wrote:
>
>> Andrew Morton <akpm@zip.com.au> writes:
>>
>> As far as I can see bonnie++ doesn't use putc_unlocked, but putc.
>
>Plain old Bonnie suffered from the same thing.  I long ago made it
>use putc_unlocked() here because throughput was horrible otherwise.

Oh, yeah, blame it on bonnie.

	"Our C library 'putc' is horribly sucky"

	"Well, then, use something else then".

Isn't somebody ashamed of glibc and willing to try to fix it? It might
be as simple as just testing a static flag "have I used pthread_create"
or even a function pointer that gets switched around at pthread_create..

"putc()" is a standard function.  If it sucks, let's get it fixed.  And
instead of changing bonnie, how about pinging the _real_ people who
write sucky code?

		Linus

Re: horrible disk thorughput on itanium

[Greg Hennessy]

In article <9upmqm$7p4$1@penguin.transmeta.com>,
Linus Torvalds <torvalds@transmeta.com> wrote:
> Isn't somebody ashamed of glibc and willing to try to fix it? It might
> be as simple as just testing a static flag "have I used pthread_create"
> or even a function pointer that gets switched around at pthread_create..

As the person who started this thread, I'll say that I'm willing to
test new alternatives, Redhat engineers gave me a newer kernel to see
if it helped (it didn't) and if someone can give me (or point me to) a
glibc with better io I'm glad.

Right now I have to explain to my boss why my $4K pentium computers do
io faster than my $20K itanium computer. And since our major software
code is 3rd party, we can't rewrite the appilcation.

Re: horrible disk thorughput on itanium

[Marco Colombo]

On Fri, 7 Dec 2001, Greg Hennessy wrote:

> In article <9upmqm$7p4$1@penguin.transmeta.com>,
> Linus Torvalds <torvalds@transmeta.com> wrote:
> > Isn't somebody ashamed of glibc and willing to try to fix it? It might
> > be as simple as just testing a static flag "have I used pthread_create"
> > or even a function pointer that gets switched around at pthread_create..
> 
> As the person who started this thread, I'll say that I'm willing to
> test new alternatives, Redhat engineers gave me a newer kernel to see
> if it helped (it didn't) and if someone can give me (or point me to) a
> glibc with better io I'm glad.
> 
> Right now I have to explain to my boss why my $4K pentium computers do
> io faster than my $20K itanium computer. And since our major software
> code is 3rd party, we can't rewrite the appilcation.

As I understand it, they just told you that the benchmark you use gives
uncorrect data on the itanium box. IIRC, you said that users were
complaing about poor performance, so I think you should investigate on
the application(s) you need instead of bonnie. Try and collect more real
world data (if possible) or at least use some other benchmarks.

If it turns out that the (hi-end, I presume) application you're using
it's both disk I/O bound and it actually uses putc(), then maybe the
problem it's just using 3rd party applications without sources... all you
have to explain to your boss is that you know what the problem is, and how
to fix it, and that the "fix" is really easy (... if only you had the
source). Add those 20K to the TCO of your closed source application.

.TM.
-- 
      ____/  ____/   /
     /      /       /			Marco Colombo
    ___/  ___  /   /		      Technical Manager
   /          /   /			 ESI s.r.l.
 _____/ _____/  _/		       Colombo@ESI.it

Re: horrible disk thorughput on itanium

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 1075 bytes --]

Hi,

On Thu, Dec 06, 2001 at 11:07:14AM -0500, Greg Hennessy wrote:
> [root@hydra bonnie]# cat bonnie.hydra bonnie.leo 
[...]
> 	      --Seeks---
> Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU
> /sec %CPU
>           100  1765 100.0 282891 100.1 377295 100.0  2058 100.0 592709
> 	  99.5 51920.4 196.5

I bet you have more than 100MB RAM, so you measure memory performance
instead of disk performance in the block reads and writes. Same for seeks.
Don't do that. Always use at least twice you ram size if you're interested
in seeing your disk speed.
Newer bonnies warn you about it. The current release is 1.2.
http://www.garloff.de/kurt/linux/bonnie/

The char reads/writes seem to suffer glibc overhead.
You can use the _unlocked variants with option -u and see whether this makes
a difference.

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE GmbH, Nuernberg, DE                                SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: horrible disk thorughput on itanium

[Michael Poole]

Padraig Brady <padraig@antefacto.com> writes:

> Well you have to deal with the general case. A single threaded
> app linking against a multithreaded lib. It mightn't be just
> shared FILE*'s that could cause problems.

The question was about putc(), and presumably other stdio functions.
They deal with FILE*'s.  They are also commonly used and small
operations, so forcing locking on an app that doesn't ask for it can
(and has in the past) cause a major performance degredation.  I bet
you could find very similar results for memory allocation in other
applications.

> > Linus's suggestion to add hooks to pthread_create() gets around that
> > problem, anyway.
> 
> I don't think this will work as I said before current apps that
> use _unlocked() functions directly manipulate the stdio structures,
> hence a "new smarter locking stdio" would never get used by existing
> compiled apps.

As Linus pointed out, you just need another test in those macros, so
they can be forced to call functions rather than using inline code.
When you call a function, it can use whatever new smarter locking
library you want.

> > Alternatively, the multi-threaded library could
> > require any application linking to it to define _REENTRANT.
> 
> It could, but what if an existing interface (lib) is changed
> from signle to multithreaded. You can't preclude this.

I certainly can preclude that.  I'd consider adding threads and their
locking considerations to the library a change in the API and ABI --
and quite a big one at that.  If you change the ABI (rather than just
extending it), you must increase the major version number, which
prevents linking with those applications that expect the semantics of
earlier versions.

(BSD variants have in the past had both libc and libc_r, where only
libc_r makes you pay locking overhead.  This is yet another way to
enforce the ABI separation between single- and multi-threaded
applications, with different tradeoffs than the others mentioned.)

-- Michael

Re: horrible disk thorughput on itanium

[Marco Colombo]

On Fri, 7 Dec 2001, Greg Hennessy wrote:

> In article <Pine.LNX.4.33.0112070941330.8465-100000@penguin.transmeta.com>,
> Linus Torvalds <torvalds@transmeta.com> wrote:
> > bonnie is a _benchmark_. It's meant for finding bad performance. Changing
> > it to make it work better when performance is bad is _pointless_. You've
> > now made the whole point of bonnie go away.
> 
> It isn't just bonnie showing bad performance. My application shows it,
> bonnie shows it, and tiobench shows it. I think the focus on putc
> may be too intense. It isn't suprizing to me that either the kernel
> or glibc may not be optimised on ia64, I'm just trying to figure out
> how to get better io rates out of my itanium machine.

Does a simple 'dd' show the problem? I mean,

time dd if=/dev/zero of=/somelargefile count=somelargenumber bs=8k

is it much slower on the itanium, too? dd doesn't use putc(), I hope.

Just for comparison, I've run the following command here:

# time dd if=/dev/zero of=/u2/test count=250000 bs=8k
250000+0 records in
250000+0 records out
0.14user 12.95system 1:23.15elapsed 15%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (117major+18minor)pagefaults 0swaps

(Almost idle box, 256MB RAM, UDMA disk, 2.2.x kernel)

You may try with a bigger file, expecially if you have more RAM.

.TM.
-- 
      ____/  ____/   /
     /      /       /			Marco Colombo
    ___/  ___  /   /		      Technical Manager
   /          /   /			 ESI s.r.l.
 _____/ _____/  _/		       Colombo@ESI.it

Re: horrible disk thorughput on itanium

[Padraig Brady]

Michael Poole wrote:

> Padraig Brady <padraig@antefacto.com> writes:
> 
> 
>>This breaks for the case discussed @
>>http://sources.redhat.com/ml/bug-glibc/2001-11/msg00079.html
>>I.E. if you have a multithreaded lib being linked by
>>single threaded apps (Note multithreaded lib, not just a
>>threadsafe lib (I.E. the lib calls pthread_create())).
>>
> 
> That's an interesting, but very contrived, example.  Can you find a
> single multi-threaded lib that uses FILE*'s shared with the
> application using it?


Well you have to deal with the general case. A single threaded

app linking against a multithreaded lib. It mightn't be just
shared FILE*'s that could cause problems.


> Linus's suggestion to add hooks to pthread_create() gets around that
> problem, anyway.


I don't think this will work as I said before current apps that
use _unlocked() functions directly manipulate the stdio structures,
hence a "new smarter locking stdio" would never get used by existing
compiled apps.

> Alternatively, the multi-threaded library could
> require any application linking to it to define _REENTRANT.


It could, but what if an existing interface (lib) is changed

from signle to multithreaded. You can't preclude this.


> After all, it's silly to talk about a 'multi-threaded' library linked
> to a 'single-threaded' application -- the application plus any
> libraries, as a whole, are either multithreaded or not.  They have to
> be on the same page to deal with *any* locking issues.
> 
> -- Michael

Padraig.

Re: horrible disk thorughput on itanium

[Michael Poole]

Padraig Brady <padraig@antefacto.com> writes:

> This breaks for the case discussed @
> http://sources.redhat.com/ml/bug-glibc/2001-11/msg00079.html
> I.E. if you have a multithreaded lib being linked by
> single threaded apps (Note multithreaded lib, not just a
> threadsafe lib (I.E. the lib calls pthread_create())).

That's an interesting, but very contrived, example.  Can you find a
single multi-threaded lib that uses FILE*'s shared with the
application using it?

Linus's suggestion to add hooks to pthread_create() gets around that
problem, anyway.  Alternatively, the multi-threaded library could
require any application linking to it to define _REENTRANT.

After all, it's silly to talk about a 'multi-threaded' library linked
to a 'single-threaded' application -- the application plus any
libraries, as a whole, are either multithreaded or not.  They have to
be on the same page to deal with *any* locking issues.

-- Michael

Re: horrible disk thorughput on itanium

[Greg Hennessy]

In article <Pine.LNX.4.33.0112070941330.8465-100000@penguin.transmeta.com>,
Linus Torvalds <torvalds@transmeta.com> wrote:
> bonnie is a _benchmark_. It's meant for finding bad performance. Changing
> it to make it work better when performance is bad is _pointless_. You've
> now made the whole point of bonnie go away.

It isn't just bonnie showing bad performance. My application shows it,
bonnie shows it, and tiobench shows it. I think the focus on putc
may be too intense. It isn't suprizing to me that either the kernel
or glibc may not be optimised on ia64, I'm just trying to figure out
how to get better io rates out of my itanium machine.

Re: horrible disk thorughput on itanium

[David S. Miller]

   From: Andi Kleen <ak@suse.de>
   Date: 07 Dec 2001 14:54:49 +0100
   
   It is a common problem on all OS that eventually got threadsafe stdio. 
   I bet putc sucks on Solaris too.

Nope, they even inline the full implementation it in their header
files so they beat even our putc_unlocked() to the point it is
embarassing.

Re: horrible disk thorughput on itanium

[Padraig Brady]

Linus Torvalds wrote:

> On Fri, 7 Dec 2001, Andi Kleen wrote:
> 
>>Your proposals sound rather dangerous. They would silently break recompiled
>>threaded programs that need the locking and don't use -D__REENTRANT
>>
> 
> No it wouldn't.
> 
> Once you do a pthread_create(), the locking is there.
> 
> Before you do a pthread_create(), it doesn't lock.
> 
> What's the problem? Before you do a pthread_create(), you don't _NEED_
> locking, because there is only one thread that accesses the stdio data
> structures.
> 
> And there are no races - if there is only one thread, then another thread
> couldn't be suddenly doing a pthread_create() during a stdio operations.
> 
> Safe, and efficient. Yes, it adds a flag test or a indirect branch, but
> considering that you avoid a serialized locking instruction, the
> optimization sounds obvious.
> 
> 		Linus

Re: horrible disk thorughput on itanium

[Padraig Brady]

Michael Poole wrote:

> Andi Kleen <ak@suse.de> writes:
> 
> 
>>>You can be thread-safe without sucking dead baby donkeys through a straw.
>>>I already mentioned two possible ways to fix it so that you have locking
>>>when you need to, and no locking when you don't.
>>>
>>Your proposals sound rather dangerous. They would silently break recompiled
>>threaded programs that need the locking and don't use -D__REENTRANT (most
>>people do not seem to use it). I doubt the possible pain from that is 
>>worth it for speeding up an basically obsolete interface like putc. 
>>
>>i.e. if someone wants speed they definitely shouldn't use putc()
>>
> 
> Threaded programs that need locking and don't define _THREAD_SAFE or
> _REENTRANT or whatever is appropriate are already broken -- they just
> don't know it yet.
> 
> FreeBSD #defines putc and getc to their unlocked versions unless
> _THREAD_SAFE is defined, and people don't seem to think its libc is
> broken.  Many lightly threaded programs, in fact, wouldn't need or
> even want the locked variants to be the default.  One app I've worked
> with only reads and writes any given FILE* from one thread, and I saw
> an 4x speedup by switching to the unlocked variants.

This breaks for the case discussed @
http://sources.redhat.com/ml/bug-glibc/2001-11/msg00079.html
I.E. if you have a multithreaded lib being linked by
single threaded apps (Note multithreaded lib, not just a
threadsafe lib (I.E. the lib calls pthread_create())).

Padraig.

Re: horrible disk thorughput on itanium

[Padraig Brady]

Andi Kleen wrote:

>>You can be thread-safe without sucking dead baby donkeys through a straw.
>>I already mentioned two possible ways to fix it so that you have locking
>>when you need to, and no locking when you don't.
>>
> 
> Your proposals sound rather dangerous. They would silently break recompiled
> threaded programs that need the locking and don't use -D__REENTRANT (most
> people do not seem to use it).


I would worry about threaded progs that don't -D_REENTRANT as
they are broken.

> I doubt the possible pain from that is 
> worth it for speeding up an basically obsolete interface like putc. 
> 
> i.e. if someone wants speed they definitely shouldn't use putc()

It's not just putc, it's all of stdio.

Padraig.

Re: horrible disk thorughput on itanium

[Linus Torvalds]

On Fri, 7 Dec 2001, Andi Kleen wrote:
>
> Your proposals sound rather dangerous. They would silently break recompiled
> threaded programs that need the locking and don't use -D__REENTRANT

No it wouldn't.

Once you do a pthread_create(), the locking is there.

Before you do a pthread_create(), it doesn't lock.

What's the problem? Before you do a pthread_create(), you don't _NEED_
locking, because there is only one thread that accesses the stdio data
structures.

And there are no races - if there is only one thread, then another thread
couldn't be suddenly doing a pthread_create() during a stdio operations.

Safe, and efficient. Yes, it adds a flag test or a indirect branch, but
considering that you avoid a serialized locking instruction, the
optimization sounds obvious.

		Linus

Re: horrible disk thorughput on itanium

[Michael Poole]

Andi Kleen <ak@suse.de> writes:

> > You can be thread-safe without sucking dead baby donkeys through a straw.
> > I already mentioned two possible ways to fix it so that you have locking
> > when you need to, and no locking when you don't.
> 
> Your proposals sound rather dangerous. They would silently break recompiled
> threaded programs that need the locking and don't use -D__REENTRANT (most
> people do not seem to use it). I doubt the possible pain from that is 
> worth it for speeding up an basically obsolete interface like putc. 
> 
> i.e. if someone wants speed they definitely shouldn't use putc()

Threaded programs that need locking and don't define _THREAD_SAFE or
_REENTRANT or whatever is appropriate are already broken -- they just
don't know it yet.

FreeBSD #defines putc and getc to their unlocked versions unless
_THREAD_SAFE is defined, and people don't seem to think its libc is
broken.  Many lightly threaded programs, in fact, wouldn't need or
even want the locked variants to be the default.  One app I've worked
with only reads and writes any given FILE* from one thread, and I saw
an 4x speedup by switching to the unlocked variants.

It's generally a bad idea to make people pay for a feature they don't
ask for.  FreeBSD's libc understands this; glibc apaprently doesn't.

-- Michael Poole

Re: horrible disk thorughput on itanium

[Andi Kleen]

> You can be thread-safe without sucking dead baby donkeys through a straw.
> I already mentioned two possible ways to fix it so that you have locking
> when you need to, and no locking when you don't.

Your proposals sound rather dangerous. They would silently break recompiled
threaded programs that need the locking and don't use -D__REENTRANT (most
people do not seem to use it). I doubt the possible pain from that is 
worth it for speeding up an basically obsolete interface like putc. 

i.e. if someone wants speed they definitely shouldn't use putc()

-Andi

Re: horrible disk thorughput on itanium

[Robert Love]

On Fri, 2001-12-07 at 12:40, Richard Gooch wrote:

> > The link to the mailing list post from bug-glibc says otherwise,
> > that is the problem.  Using the unlocked version isn't implied by
> > not setting __REENTRANT.
> 
> The bug is in glibc. An application shouldn't need to be changed to
> work around that bug. putc() is a well-known interface, and people
> shouldn't have to code around a change in that interface.

Right.  That's why I referenced a post on bug-glibc and called the issue
a problem.  I'm not defending the heaping mass known as glibc ... it
should be fixed.

	Robert Love

Re: horrible disk thorughput on itanium

[Linus Torvalds]

On 7 Dec 2001, Andi Kleen wrote:
> torvalds@transmeta.com (Linus Torvalds) writes:
> >
> > "putc()" is a standard function.  If it sucks, let's get it fixed.  And
> > instead of changing bonnie, how about pinging the _real_ people who
> > write sucky code?
>
> It is easy to fix. Just do #define putc putc_unlocked

Sure. And why don't you also do

	#define sin(x) (1)
	#define sqrt(x) (1)
	#define strlen(x) (1)
	...

to make other benchmarks happier?

bonnie is a _benchmark_. It's meant for finding bad performance. Changing
it to make it work better when performance is bad is _pointless_. You've
now made the whole point of bonnie go away.

> There is just a slight problem: it'll fail if your application is threaded
> and wants to use the same FILE from multiple threads.
>
> It is a common problem on all OS that eventually got threadsafe stdio.

It's a common problem with bad programming.

You can be thread-safe without sucking dead baby donkeys through a straw.
I already mentioned two possible ways to fix it so that you have locking
when you need to, and no locking when you don't.

		Linus

Re: horrible disk thorughput on itanium

[Richard Gooch]

Robert Love writes:
> On Fri, 2001-12-07 at 11:14, Richard Gooch wrote:
> 
> > This kind of thing should be covered by _REENTRANT. If you don't
> > compile with _REENTRANT (the default), then putc() should be the
> > unlocked version.
> 
> The link to the mailing list post from bug-glibc says otherwise,
> that is the problem.  Using the unlocked version isn't implied by
> not setting __REENTRANT.

The bug is in glibc. An application shouldn't need to be changed to
work around that bug. putc() is a well-known interface, and people
shouldn't have to code around a change in that interface.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: horrible disk thorughput on itanium

[Robert Love]

On Fri, 2001-12-07 at 11:14, Richard Gooch wrote:

> This kind of thing should be covered by _REENTRANT. If you don't
> compile with _REENTRANT (the default), then putc() should be the
> unlocked version.

The link to the mailing list post from bug-glibc says otherwise, that is
the problem.  Using the unlocked version isn't implied by not setting
__REENTRANT.

	Robert Love

Re: horrible disk thorughput on itanium

[Richard Gooch]

Andi Kleen writes:
> torvalds@transmeta.com (Linus Torvalds) writes:
> > 
> > "putc()" is a standard function.  If it sucks, let's get it fixed.  And
> > instead of changing bonnie, how about pinging the _real_ people who
> > write sucky code?
> 
> It is easy to fix. Just do #define putc putc_unlocked
> There is just a slight problem: it'll fail if your application is threaded
> and wants to use the same FILE from multiple threads.
> 
> It is a common problem on all OS that eventually got threadsafe stdio. 
> I bet putc sucks on Solaris too.

This kind of thing should be covered by _REENTRANT. If you don't
compile with _REENTRANT (the default), then putc() should be the
unlocked version.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: horrible disk thorughput on itanium

[Padraig Brady]

Andi Kleen wrote:

> torvalds@transmeta.com (Linus Torvalds) writes:
> 
>>"putc()" is a standard function.  If it sucks, let's get it fixed.  And
>>instead of changing bonnie, how about pinging the _real_ people who
>>write sucky code?
>>
> 
> It is easy to fix. Just do #define putc putc_unlocked
> There is just a slight problem: it'll fail if your application is threaded
> and wants to use the same FILE from multiple threads.
> 
> It is a common problem on all OS that eventually got threadsafe stdio. 
> I bet putc sucks on Solaris too.
> 
> -Andi


Interesting thread on this:
http://sources.redhat.com/ml/bug-glibc/2001-11/msg00079.html

for glibc 2.2.4 with the following program with input file
of 354371 lines of text(/usr/share/doc/*), where the average line
length was 37 chars.

getc/putc
real 
2.181s
user 
2.150s
sys 
0.030s
getc_unlocked/putc_unlocked
real 
0.326s
user 
0.280s
sys 
0.040s

I.E. 669% faster!

Padraig.

-------------------
#include <stdio.h>

#ifndef  _REENTRANT
#    undef getc
#    define getc(x)   getc_unlocked(x)
#    undef putc
#    define putc(x,y) putc_unlocked(x,y)
#endif //_REENTRANT

void main(void)
{
     int c;
     while((c=getc(stdin))!=EOF) putc(c,stdout);
}

Re: horrible disk thorughput on itanium

[Andi Kleen]

torvalds@transmeta.com (Linus Torvalds) writes:
> 
> "putc()" is a standard function.  If it sucks, let's get it fixed.  And
> instead of changing bonnie, how about pinging the _real_ people who
> write sucky code?

It is easy to fix. Just do #define putc putc_unlocked
There is just a slight problem: it'll fail if your application is threaded
and wants to use the same FILE from multiple threads.

It is a common problem on all OS that eventually got threadsafe stdio. 
I bet putc sucks on Solaris too.

-Andi

Re: horrible disk thorughput on itanium

[Dan Kegel]

Linus wrote:
> >> As far as I can see bonnie++ doesn't use putc_unlocked, but putc.
> >
> >Plain old Bonnie suffered from the same thing.  I long ago made it
> >use putc_unlocked() here because throughput was horrible otherwise.
> 
> Oh, yeah, blame it on bonnie.
> 
>         "Our C library 'putc' is horribly sucky"
> 
>         "Well, then, use something else then".
> 
> Isn't somebody ashamed of glibc and willing to try to fix it? It might
> be as simple as just testing a static flag "have I used pthread_create"
> or even a function pointer that gets switched around at pthread_create..

That sounds racy.  Better to make the change at compile time, maybe?
Say, 

#ifdef __USE_REENTRANT 
#define putc(_ch, _fp) _IO_putc (_ch, _fp)
#else
#define putc(_ch, _fp) _IO_putc_unlocked (_ch, _fp)
#endif

That's pedantically safe, I think.  

- Dan

Re: horrible disk thorughput on itanium

[Andrew Morton]

Greg Hennessy wrote:
> 
> ...
> Hydra is the itanium, leo is the 32 bit machine. The character io of
> hydra is a factor of 10 slower than that of leo. Is this more likely a
> kernel issue, or a glibc issue? Both machiness run standard redhat
> 7.1, and 2.4.9-12smp kernels.
> 

The character I/O part of bonnie++ writes a single character at a time,
via stdio.  It's more a test of your C library than of the kernel.

The fact that you get the same throughput on each platform with
the block I/O part of the test indicates that the hardware and
kernel are OK, but the C library is broken.

Not sure how to diagnose this.  Probably you should write a
simple five-line stdio-based test program, see if that exhibits
the same behaviour, then fiddle with setvbuf().

-

Re: horrible disk thorughput on itanium

[Arjan van de Ven]

Greg Hennessy wrote:
> 
> I recently installed a  both a Dell dual cpu 2500 server (dual 1.6 ghz
> ia32 chips) and a dell 7150 (dual IA64 chips). My users complained
> that the disk io speed on the itanium seemed very slow, even though
> both servers have a megaraid controller with seagate cheetah
> disks. Bonnie also shows the ia64 machine having worse throughput than
> the ia32 machine.

How much RAM do you have ?

horrible disk thorughput on itanium

[Greg Hennessy]

I recently installed a  both a Dell dual cpu 2500 server (dual 1.6 ghz
ia32 chips) and a dell 7150 (dual IA64 chips). My users complained
that the disk io speed on the itanium seemed very slow, even though
both servers have a megaraid controller with seagate cheetah
disks. Bonnie also shows the ia64 machine having worse throughput than
the ia32 machine. 

[root@hydra bonnie]# cat bonnie.hydra bonnie.leo 
              -------Sequential Output-------- ---Sequential Input--
	      --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block---
	      --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU
/sec %CPU
          100  1765 100.0 282891 100.1 377295 100.0  2058 100.0 592709
	  99.5 51920.4 196.5
              -------Sequential Output-------- ---Sequential Input--
	      --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block---
	      --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU
/sec %CPU
          100 17049 100.1 265197 101.0 197094 98.2 16631 100.4 675831
	  99.0 40400.0 191.9

Hydra is the itanium, leo is the 32 bit machine. The character io of
hydra is a factor of 10 slower than that of leo. Is this more likely a
kernel issue, or a glibc issue? Both machiness run standard redhat
7.1, and 2.4.9-12smp kernels.