Re: [ofa-general] Further 2.6.23 merge plans...

[Sean Hefty <mshefty@ichips.intel.com>]

>  - Take a look at Sean's local SA caching patches.  I merged
>    everything else from Sean's tree, but I'm still undecided about
>    these.  I haven't read them carefully yet, but even aside from that
>    I don't have a good feeling about whether there's consensus about
>    this yet.  Any opinions about merging, for or against, would be
>    appreciated here.

Obviously I'm biased here, but we've definitely seen local caching of 
path records (PR) greatly improve performance for large MPI job runs. 
(Our largest jobs wouldn't run without it.)  The development of the 
feature was requested and paid for by the US national labs. 
Infinicon/Silverstorm/QLogic also had this feature in their IB stack for 
scalability reasons as well.  PR caching is done in the stack today by 
IPoIB.

The implementation is hidden under the current kernel ib_sa interface, 
is disabled by default, and automatically fails over to standard PR 
queries if needed.  Removing the cache later should be fairly easy.

But to be fair, it will be difficult to enable both QoS and local PR 
caching.  To me, this would be the strongest reason against using it. 
However, QoS places additional burden on the SA, which will make scaling 
even more challenging.

- Sean