Re: [Lsf-pc] [LSF/MM TOPIC] iSCSI MQ adoption via MCS discussion

[Bart Van Assche <bart.vanassche@sandisk.com>]

On 01/11/15 10:40, Sagi Grimberg wrote:
> I would say there is no need for specific coordination from iSCSI PoV.
> This is exactly what flow steering is designed for. As I see it, in
> order to get the TX/RX to match rings, the user can attach 5-tuple rules
> (using standard ethtool) to steer packets to the right rings.

Hello Sagi,

Can the 5-tuple rules be chosen such that it is guaranteed that the
sockets used to implement per-CPU queues are spread evenly over MSI-X
completion vectors ? If not, would it help to add a socket option to the
Linux network stack that allows to select the TX ring explicitly, just
like ib_create_cq() in the Linux RDMA stack allows to select a
completion vector explicitly ? My concerns are as follows:
- If the number of queues exceeds the number of MSI-X vectors then I
  expect that it will be much easier to guarantee even spreading by
  selecting tx queues explicitly instead of relying on a hashing scheme.
- On multi-socket systems it is important to process completion
  interrupts on the CPU socket from where the I/O was initiated. I'm
  not sure it is possible to guarantee this when using a hashing
  algorithm to select the TX ring.

Bart.