Re: [RFC] XRC upstream merge reboot

[Jack Morgenstein <jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>]

On Wednesday 20 July 2011 21:51, Hefty, Sean wrote:
> I've tried to come up with a clean way to determine the lifetime of an xrc tgt qp,\
> and I think the best approach is still: 
> 
> 1. Allow the creating process to destroy it at any time, and
> 
> 2a. If not explicitly destroyed, the tgt qp is bound to the lifetime of the xrc domain
> or
> 2b. The creating process specifies during the creation of the tgt qp
> whether the qp should be destroyed on exit. 
> 
> The MPIs associate an xrc domain with a job, so this should work.
> Everything else significantly complicates the usage model and implementation,
> both for verbs and the CM.  An application can maintain a reference count
> out of band with a persistent server and use explicit destruction
> if they want to share the xrcd across jobs.
I assume that you intend the persistent server to replace the reg_xrc_rcv_qp/
unreg_xrc_rcv_qp verbs.  Correct?
> 
> Option 2a is the current implementation, but 2b should be a minor change.
> I'd like to reach a consensus on the right approach here, since there doesn't
> appear to be issues elsewhere.  
> 
> - Sean

I have no opinion either way (with regard to tgt qp registration and reference counting).
The OFED xrc implementation was driven by the requirements of the MPI community.

If MPI can use a different XRC domain per job (and deallocate the domain
at the job's end), this would solve the tgt qp lifetime problem (-- by
destroying all the tgt qp's when the xrc domain is deallocated).

Regarding option 2b: do you mean that in this case the tgt qp is NOT bound to the
XRC domain lifetime? who destroys the tgt qp in this case when the creator indicates
that the tgt qp should not be destroyed on exit?

I am concerned with backwards compatibility, here.  It seems that XRC users will need to
change their source-code, not just recompile.  I am assuming that OFED will take the
mainstream kernel implementation at some point.  Since this is **userspace** code, there could be a problem
if OFED users upgrade their OFED installation to one which supports the new interface.
This could be especially difficult if, for example, the customer is using 3rd-party packages
which utilize the current OFED xrc interface.  We could start seeing customers not take
new OFED releases solely because of the XRC incompatibility (or worse, customers upgrading
and then finding out that their 3rd-party XRC apps no longer work).

Having a new OFED support BOTH interfaces is a nightmare I don't even want to think about!

-Jack
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html