From: "Hefty, Sean" <sean.hefty-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
To: Jack Morgenstein
<jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
Cc: "linux-rdma
(linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org)"
<linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>,
"tziporet-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org"
<tziporet-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>,
"dotanb-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org"
<dotanb-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
Subject: RE: [RFC] XRC upstream merge reboot
Date: Thu, 23 Jun 2011 18:03:50 +0000 [thread overview]
Message-ID: <1828884A29C6694DAF28B7E6B8A82373029F48@ORSMSX101.amr.corp.intel.com> (raw)
In-Reply-To: <201106230935.07425.jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
> If you are thinking of using this for tracking, there is no way that I can
> see except reference counting to know when all users of the TGT have
> finished with it. Indicating usage by "tapping" the TGT QP whenever an
> SRQ receives a packet via the TGT is not a good idea (this is data
> path!), not to mention that quiescence is not a good indicator of whether
> an app is done using the target (the amount of time that things
> are quiet depends on the app).
I mentioned the event as a standard way for communicating the tgt qp to other processes using the xrcd. Currently, we have something like this:
Process 1:
create tgt qp
for each relevant process
send out of band message with tgt qpn
Process 2..n:
receive tgt qpn
kernel call to register with tgt qp
kernel searches for tgt qpn, bumps ref count
<later>
kernel call to unregister with tgt qp
kernel searches for tgt qpn, decrements ref count
The use of an event is simply an (incomplete) idea to standardize what is currently done by sending/receiving the out of band messages with the tgt qpn. On this note, I can't imagine that this usage model is better performing that simply creating a new xrcd for each job...
I was thinking about this more last night, in more generic terms, what we want is a way for an application to allocate a QP in the kernel and share it with other apps. I think it would be best to reuse the existing ibverbs API and kern-abi as much as possible. (Eventually someone may find a use for this other than just TGT QPs.) I just don't know how we handle sharing or transferring ownership of the CM data.
- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2011-06-23 18:03 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2011-05-16 21:13 [RFC] XRC upstream merge reboot Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373F7AB-P5GAC/sN6hmkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2011-05-18 14:54 ` Jack Morgenstein
[not found] ` <201105181754.33759.jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2011-05-18 15:27 ` Hefty, Sean
2011-06-22 7:17 ` Jack Morgenstein
[not found] ` <201106221017.06212.jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2011-06-22 16:14 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373029A95-P5GAC/sN6hmkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2011-06-22 17:03 ` Jack Morgenstein
[not found] ` <201106222003.50214.jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2011-06-22 18:19 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373029B3F-P5GAC/sN6hmkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2011-06-22 19:21 ` Jack Morgenstein
[not found] ` <201106222221.05993.jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2011-06-22 19:32 ` Tziporet Koren
2011-06-22 19:57 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373029BDE-P5GAC/sN6hmkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2011-06-23 6:11 ` Jack Morgenstein
2011-06-23 6:35 ` Jack Morgenstein
[not found] ` <201106230935.07425.jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2011-06-23 18:03 ` Hefty, Sean [this message]
2011-07-20 18:51 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373136F63B9-P5GAC/sN6hmkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2011-07-21 7:38 ` Jack Morgenstein
[not found] ` <201107211038.23000.jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2011-07-21 7:57 ` Jack Morgenstein
2011-07-21 11:58 ` Jeff Squyres
[not found] ` <D8276D45-5FE8-464C-B3A4-14404DE8C760-FYB4Gu1CFyUAvxtiuMwx3w@public.gmane.org>
2011-07-21 12:47 ` Jack Morgenstein
[not found] ` <201107211547.31850.jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2011-07-21 12:46 ` Jeff Squyres
2011-07-21 16:06 ` Hefty, Sean
2011-07-21 17:53 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373136F6691-P5GAC/sN6hmkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2011-07-26 20:04 ` Shamis, Pavel
[not found] ` <26AE60A9-D055-4D40-A830-5AADDBA20ED8-1Heg1YXhbW8@public.gmane.org>
2011-08-01 15:03 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373136F9075-P5GAC/sN6hmkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2011-08-01 16:20 ` Shamis, Pavel
[not found] ` <AE625966-FD97-4DBF-A024-22B83B5F3E39-1Heg1YXhbW8@public.gmane.org>
2011-08-01 18:28 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373136F9194-P5GAC/sN6hmkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2011-08-02 10:44 ` Jack Morgenstein
[not found] ` <201108021344.25284.jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2011-08-02 16:29 ` Shamis, Pavel
[not found] ` <32D25205-3E9C-4757-B0AB-7117BDF3F2F7-1Heg1YXhbW8@public.gmane.org>
2011-08-03 10:37 ` Jack Morgenstein
[not found] ` <201108031337.24527.jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2011-08-10 22:20 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A8237316E3E55C-Q3cL8pyY+6ukrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2011-08-11 14:12 ` Shamis, Pavel
2011-08-21 14:42 ` Jack Morgenstein
[not found] ` <201108211742.18803.jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2011-08-22 15:46 ` Hefty, Sean
2011-08-02 19:08 ` Shamis, Pavel
[not found] ` <EABE213A-448A-45F8-B131-AE1EE3F9547F-1Heg1YXhbW8@public.gmane.org>
2011-08-02 21:25 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373136F962C-P5GAC/sN6hmkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2011-08-02 23:00 ` Shamis, Pavel
[not found] ` <DE779D97-F54F-45E4-B3D4-DBEB10F9302D-1Heg1YXhbW8@public.gmane.org>
2011-08-02 23:53 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373136F967E-P5GAC/sN6hmkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2011-08-03 13:47 ` Shamis, Pavel
[not found] ` <5C691E518F345F4882FAB9E9839E60BA0BCA4622F4-vxojlfkN5A++qDdrU24kdQ@public.gmane.org>
2011-08-03 15:52 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373136F97AC-P5GAC/sN6hmkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2011-08-03 20:22 ` Shamis, Pavel
[not found] ` <5C691E518F345F4882FAB9E9839E60BA0BCA462300-vxojlfkN5A++qDdrU24kdQ@public.gmane.org>
2011-08-03 20:49 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373136F9A13-P5GAC/sN6hmkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2011-08-03 21:16 ` Shamis, Pavel
[not found] ` <5C691E518F345F4882FAB9E9839E60BA0BCA462301-vxojlfkN5A++qDdrU24kdQ@public.gmane.org>
2011-08-03 21:36 ` Jason Gunthorpe
[not found] ` <20110803213642.GE28465-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2011-08-04 0:06 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373136F9A7E-P5GAC/sN6hmkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2011-08-04 4:05 ` Jason Gunthorpe
[not found] ` <20110804040503.GA13935-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2011-08-04 5:53 ` Hefty, Sean
2011-05-18 16:44 ` Roland Dreier
[not found] ` <BANLkTimWMU9ohSQGYEEnFR0HbBaypFR51A-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2011-05-18 17:02 ` Jason Gunthorpe
[not found] ` <20110518170226.GA2595-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2011-05-18 17:30 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373FBC7-P5GAC/sN6hmkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2011-05-18 18:05 ` Jason Gunthorpe
[not found] ` <20110518180519.GA11860-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
2011-05-18 18:13 ` Hefty, Sean
[not found] ` <1828884A29C6694DAF28B7E6B8A82373FC13-P5GAC/sN6hmkrb+BlOpmy7fspsVTdybXVpNB7YpNyf8@public.gmane.org>
2011-05-18 18:22 ` Jason Gunthorpe
2011-05-18 19:22 ` Roland Dreier
[not found] ` <BANLkTi=cLjErM3pKzihyFtGWZ0kSu9BiPA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2011-05-19 5:29 ` Hefty, Sean
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1828884A29C6694DAF28B7E6B8A82373029F48@ORSMSX101.amr.corp.intel.com \
--to=sean.hefty-ral2jqcrhueavxtiumwx3w@public.gmane.org \
--cc=dotanb-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org \
--cc=jackm-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org \
--cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
--cc=tziporet-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.