Re: [Lsf-pc] [LSF/MM TOPIC] iSCSI MQ adoption via MCS discussion

[James Bottomley <James.Bottomley@HansenPartnership.com>]

On Thu, 2015-01-08 at 21:03 -0800, Nicholas A. Bellinger wrote:
> On Thu, 2015-01-08 at 15:22 -0800, James Bottomley wrote:
> > On Thu, 2015-01-08 at 14:57 -0800, Nicholas A. Bellinger wrote:
> > > On Thu, 2015-01-08 at 14:29 -0800, James Bottomley wrote:
> > > > On Thu, 2015-01-08 at 14:16 -0800, Nicholas A. Bellinger wrote:
> 
> <SNIP>
> 
> > > The point is that a simple session wide counter for command sequence
> > > number assignment is significantly less overhead than all of the
> > > overhead associated with running a full multipath stack atop multiple
> > > sessions.
> > 
> > I don't see how that's relevant to issue speed, which was the measure we
> > were using: The layers above are just a hopper.  As long as they're
> > loaded, the MQ lower layer can issue at full speed.  So as long as the
> > multipath hopper is efficient enough to keep the queues loaded there's
> > no speed degradation.
> > 
> > The problem with a sequence point inside the MQ issue layer is that it
> > can cause a stall that reduces the issue speed. so the counter sequence
> > point causes a degraded issue speed over the multipath hopper approach
> > above even if the multipath approach has a higher CPU overhead.
> > 
> > Now, if the system is close to 100% cpu already, *then* the multipath
> > overhead will try to take CPU power we don't have and cause a stall, but
> > it's only in the flat out CPU case.
> > 
> > > Not to mention that our iSCSI/iSER initiator is already taking a session
> > > wide lock when sending outgoing PDUs, so adding a session wide counter
> > > isn't adding any additional synchronization overhead vs. what's already
> > > in place.
> > 
> > I'll leave it up to the iSER people to decide whether they're redoing
> > this as part of the MQ work.
> > 
> 
> Session wide command sequence number synchronization isn't something to
> be removed as part of the MQ work.  It's a iSCSI/iSER protocol
> requirement.

The sequence number is a requirement of the session.  Multiple separate
sessions means no SN correlation between the different connections, so
no global requirement for a SN counter across the queues ... that's what
Mike was saying about implementing multipath not using MCS.  With MCS we
have a single session for all the queues and thus have to correlate the
sequence number across all the connections and hence all the queues;
without it we don't.  That's why the sequence number becomes a potential
stall point in MQ implementation of MCS which can be obviated if we use
a separate session per queue.

James