RE: Bug in short splice to socket?

From: David Laight <David.Laight@ACULAB.COM>
To: 'Linus Torvalds' <torvalds@linux-foundation.org>,
	David Howells <dhowells@redhat.com>
Cc: Jakub Kicinski <kuba@kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"David S. Miller" <davem@davemloft.net>,
	"Eric Dumazet" <edumazet@google.com>,
	Paolo Abeni <pabeni@redhat.com>,
	"Willem de Bruijn" <willemdebruijn.kernel@gmail.com>,
	David Ahern <dsahern@kernel.org>,
	Matthew Wilcox <willy@infradead.org>,
	Jens Axboe <axboe@kernel.dk>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	Chuck Lever <chuck.lever@oracle.com>,
	"Boris Pismenny" <borisp@nvidia.com>,
	John Fastabend <john.fastabend@gmail.com>,
	Christoph Hellwig <hch@infradead.org>
Subject: RE: Bug in short splice to socket?
Date: Mon, 5 Jun 2023 11:03:21 +0000	[thread overview]
Message-ID: <e94820ba53924e96b31ac983c84269f8@AcuMS.aculab.com> (raw)
In-Reply-To: <CAHk-=wji_2UwFMkUYkygsYRek05NwaQkH-vA=yKQtQS9Js+urQ@mail.gmail.com>

From: Linus Torvalds
> Sent: 01 June 2023 16:12
> 
> On Thu, Jun 1, 2023 at 10:34 AM David Howells <dhowells@redhat.com> wrote:
> >
> > At the moment, it transcribes 16 pages at a time.  I could make it set
> > MSG_MORE only if (a) SPLICE_F_MORE was passed into the splice() syscall or (b)
> > there's yet more data in the buffer.
> 
> That would at least be a good first step.
> 
> > However, this might well cause a malfunction in UDP, for example.  MSG_MORE
> > corks the current packet, so if I ask sendfile() say shove 32K into a packet,
> > if, say, 16K is read from the source and entirely transcribed into the packet,
> 
> If you use splice() for UDP, I don't think you would normally expect
> to get all that well-defined packet boundaries.

Especially since (assuming I've understood other bits of this thread)
the splice() can get split into multiple sendmsg() calls for other
reasons.

What semantics are you trying to implement for AF_TLS?
MSG_MORE has different effects on different protocols.

For UDP the next data is appended to the datagram being built.
(This is really pretty pointless, doing it in the caller will be faster!)

For TCP it stops the pending data being sent immediately.
And more data is appended.
I'm pretty sure it gets sent on timeout.

For SCTP the data chunk created for the sendmsg() isn't sent immediately.
Any more sendmsg(MSG_MORE) get queued until a full ethernet packet
is buffered.
The pending data is sent on timeout.
This is pretty much the only way to get two (or more) DATA chunks
into an ethernet frame when Nagle is disabled.

But I get the impression AF_TLS is deciding not to encode/send
the data because 'there isn't enough'.
That seems wrong.

Note that you can't use a zero length sendmsg() to flush pending
data - if there is no pending data some protocols will send a 
zero length data message.
A socket option/ioctl (eg UNCORK) could be (ab)used to force
queued data be sent.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)