RE: ibv_req_notify_cq clarification

["Hefty, Sean" <sean.hefty@intel.com>]

> I agree that this behavior is most likely better, but it contradicts the api
> agreement.
> It seems like you should write your app according to the provider you're using
> in order to get it right.

The app should code for edge triggered behavior for correctness.  I'm saying if the NIC actually uses level triggering (I don't know if they do or not), the app can't tell and will just work.

> > The app can do:
> >
> > Wakeup
> > While read cqe succeeds
> > 	Process cqe
> > Read cq event
> > Arm cq
> > /* cqe's may have been written between the last read and arming */
> > While read cqe succeeds
> > 	Process cqe
> > Sleep
> >
> > Or shorten this to:
> >
> > Wakeup
> > Read cq event
> > Arm cq
> > While read cqe succeeds
> > 	Process cqe
> > Sleep
> >
> > In both cases, all cqe's must be processed after calling arm, and it's possible to
> read a cq event only to find the cq empty.  One could argue which is more efficient,
> but we're talking about a sleeping thread in either case.
> 
> Yea, this seems correct.
> But as I said in my reply to Jason, libibverbs examples, pyverbs tests and
> perftest all go from "Read cq event" to "Arm cq" immediately.

Both options provide correct behavior, regardless of how signaling works.  The issue is that perftest does:

For I = 1 to 16
	If read cqe succeeds
		Process cqe

Instead of

While read cqe succeeds
	Process cqe

Perftest will only work if it's lucky or with level signaling.  I would not assume perftest is correct just because no one has reported the problem before.  :)

- Sean