Re: kernel.org mirroring (Re: [GIT PULL] MMC update)

[Linus Torvalds <torvalds@osdl.org>]

On Thu, 7 Dec 2006, H. Peter Anvin wrote:
> 
> What it could do better is it could prevent multiple identical queries from
> being launched in parallel.  That's the real problem we see; under high load,
> Apache times out so the git query never gets into the cache; but in the
> meantime, the common queries might easily have been launched 20 times in
> parallel.  Unfortunately, the most common queries are also extremely
> expensive.

Ahh. I'd have expected that apache itself had some serialization facility, 
that would kind of go hand-in-hand with any caching.

It really would make more sense to have anything that does caching 
serialize the address that gets cached (think "page cache" layer in the 
kernel: the _cache_ is also the serialization point, and is what 
guarantees that we don't do stupid multiple reads to the same address).

I'm surprised that Apache can't do that. Or maybe it can, and it just 
needs some configuration entry? I don't know apache.. I realize that 
because Apache doesn't know before-hand whether something is cacheable or 
not, it must probably _default_ to running the CGI scripts to the same 
address in parallel, but it would be stupid to not have the option to 
serialize.

That said, from some of the other horrors I've heard about, "stupid" may 
be just scratching at the surface.