RE: msgr2 protocol

[Avner Ben Hanoch <avnerb@mellanox.com>]

> -----Original Message-----
> From: Yehuda Sadeh-Weinraub 
> Sent: Wednesday, June 29, 2016 19:53
> 
> On Wed, Jun 29, 2016 at 4:59 AM, Avner Ben Hanoch
> <avnerb@mellanox.com> wrote:
> >
> > On Sat, 28 May 2016 11:19 AM, Yehuda Sadeh-Weinraub
> <yehuda@xxxxxxxxxx> wrote:
> >>On Fri, May 27, 2016 at 10:37 AM, Sage Weil <sweil@xxxxxxxxxx> wrote:
> >>>
> >>> If you do a connect and immediately write a few bytes to teh TCP
> >>> stream, does that actaully translate to fewer packets?  I was
> >>> guessing that the server writing the first bytes of the exchange
> >>> would be fine but if it speeds things up for the client to
> >>> optimistically start the exchange too we may as well...
> >>>
> >>
> >>While haven't really looked at it recently, I don't think it'd be possible to
> embed data with the SYN packet using the plain >vanilla tcp implementation.
> However, I believe that doing connect() and sending data immediately
> following it should improve >things, specifically if doing async connect (as
> with the async messenger), but this still needs to be proven.
> >>
> >>Yehuda
> >
> > I am using TCP with network sniffers like Wireshark and this is always the
> case that I see in Linux  - *sending data soon after connect will always save
> packet by combining the ACK from the last step of TCP 3-way handshake with
> the 1st data packet* .
> > This is the case even when I did "short" activity between connect and send.
> >
> > Sniffer will show you 3 packets on the stream:
> > 1.      Client sends SYN packet
> > 2.      Server replies with SYN-ACK packet
> > 3.      Client send *data packet* that have the ACK flag set in it (this ACK
> completes the TCP 3-way handshake and makes 'accept' return on the server
> side)
> >
> > synchronous or asynchronous socket isn't relevant here because 'connect'
> returns with success upon receiving SYN-ACK from the server regardless of the
> actual client send of the TCP 3-way completing ACK (i.e., the client application
> doesn't need this ACK for relying on the socket as connected - only the server
> side need it).
> >
> > From my experience, even disabling nagle (TCP_NODELAY) doesn't affect
> > this behavior (probably because TCP_NODELAY only affect sending *data*
> > faster but does not change TCP handshake behavior)
> 
> Right. However, I was aiming at sending the data with the SYN, not with the
> ACK that follows it (trying to avoid the first roundtrip). I assume it's not really
> possible with vanilla tcp.
> 
> Yehuda
> 
I think that Sage's idea/question was to have fewer packets and to speed things up by having the client starting the exchange instead of having the server starting the exchange.
Hence, I am saying - YES.  This idea is correct.  It will save one packet and it will save time of half round trip (i.e., one leg) because the client is allowed to send data after 2 legs of TCP 3-way handshake; while the server must wait for completion of all 3 legs of TCP 3-way handshake before sending any data.
And I am saying that this is the behavior of vanilla TCP without any special settings.

If you want to send traffic with the 1st SYN packet than it sounds a bit strange even for non-vanilla TCP.  Because the client doesn't even know that the server is alive at this phase, and the server doesn't know that the client is legitimate at this phase.  This will also probably expose your server to security issue that is called SYN attack, because servers are afraid of allocating connection resources just for SYN packets (because the src ip/port can be fictitious at this phase before the client used the server's reply)

Avner