archive mirror
 help / color / mirror / Atom feed
From: Alan Mayer <>
To: Milton Miller <>
Cc: Alan Mayer <>,
Subject: Re: irqs on large machines (patch)
Date: Wed, 25 Sep 2002 09:35:59 -0500 (CDT)	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

On Wed, 25 Sep 2002, Milton Miller wrote:

> PowerPC 64 (p690 in particular) had a similar but different
> problem with NR_IRQS.
> The hardware gives direct vectors, but the number space is 9 bits
> to identify the pci host bridge in the drawers, and yet 4 more to
> identify the source on the pci bus.  All of the 9 bits are used at
> various levels of the hardware for routing, so the global number
> space is a bit sparse.
> Each interrupt can be sent to the global queue or a specific
> processor's queue.

Ours can only be sent to a specific processor.

> Rather than size NR_IRQS for the native index, the hardware numbers
> are mapped with a simple mapping table that directly maps dense
> linux irqs to hardware numbers.  Thus the linux NR_IRQs is set based on
> the total number of hardware sources.  (And yes, we found we needed
> /proc/interrupts to be seq_file based, but that is long merged).

And that's our problem.  The total number of hardware sources of interrupts
is N IO nodes * 6 buses per node * 2 slots per bus = 12N sources of
IO interrupts.  N can be anywhere between 1 and 256.  How big N is depends
entirely on how much money the customer wants to spend.  Some of our
customers have LOTS of money.  So, the IRQ space is sparse for small values
of N, with the density increasing with N.  It doesn't take long before
an interrupt vector is overloaded, so it only makes sense to try and
differentiate between interrupt vector x sent to processor y and 
interrupt vector x sent to processor z.

> Perhaps this will give you ideas for other alternatives?  This approach
> could also allow you to assign IO interrupts to a node if your hardware
> allows.

There are lots of alternatives.  However, most of them require deviating
significantly from what David has in the IA64 Linux patch.
However, we made the design decision that
we would try to co-exist peacefully with the IA64 Linux patch.  So far,
we have been able to do that.  We are THIS close (imagine finger and thumb
nearly touching) to living within the IA64 Linux code.  We just need
this small patch.  We want ALL of the code we develop for Linux and IA64
to live in some "official" (meaning, non-SGI) release.  Right now, I think
we're something under 100 lines of code different.  We want that to be
> milton
> -- 
> [I'll look for any replys on the list]

Still crazy, after all these years.
Alan J. Mayer
WORK: 651-683-3131
HOME: 651-407-0134

      reply	other threads:[~2002-09-25 14:30 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2002-09-24 19:22 irqs on large machines (patch) Alan Mayer
2002-09-25  9:11 ` Milton Miller
2002-09-25 14:35   ` Alan Mayer [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).