Re: [PATCH net] tcp: make sure EPOLLOUT wont be missed

From: Jason Baron <jbaron@akamai.com>
To: Eric Dumazet <eric.dumazet@gmail.com>,
	Eric Dumazet <edumazet@google.com>,
	"David S . Miller" <davem@davemloft.net>
Cc: netdev <netdev@vger.kernel.org>,
	Soheil Hassas Yeganeh <soheil@google.com>,
	Neal Cardwell <ncardwell@google.com>,
	Vladimir Rutsky <rutsky@google.com>
Subject: Re: [PATCH net] tcp: make sure EPOLLOUT wont be missed
Date: Mon, 19 Aug 2019 14:40:59 -0400	[thread overview]
Message-ID: <21a28c54-1f13-4941-4374-ac67a326c6ab@akamai.com> (raw)
In-Reply-To: <c31d7c04-9d6f-aefb-500e-5b9b635ff221@gmail.com>

On 8/17/19 12:26 PM, Eric Dumazet wrote:
> 
> 
> On 8/17/19 4:19 PM, Jason Baron wrote:
>>
>>
>> On 8/17/19 12:26 AM, Eric Dumazet wrote:
>>> As Jason Baron explained in commit 790ba4566c1a ("tcp: set SOCK_NOSPACE
>>> under memory pressure"), it is crucial we properly set SOCK_NOSPACE
>>> when needed.
>>>
>>> However, Jason patch had a bug, because the 'nonblocking' status
>>> as far as sk_stream_wait_memory() is concerned is governed
>>> by MSG_DONTWAIT flag passed at sendmsg() time :
>>>
>>>     long timeo = sock_sndtimeo(sk, flags & MSG_DONTWAIT);
>>>
>>> So it is very possible that tcp sendmsg() calls sk_stream_wait_memory(),
>>> and that sk_stream_wait_memory() returns -EAGAIN with SOCK_NOSPACE
>>> cleared, if sk->sk_sndtimeo has been set to a small (but not zero)
>>> value.
>>
>> Is MSG_DONTWAIT not set in this case? The original patch was intended
>> only for the explicit non-blocking case. The epoll manpage says:
>> "EPOLLET flag should use nonblocking file descriptors". So the original
>> intention was not to impact the blocking case. This seems to me like
>> a different use-case.
>>
> 
> I guess the problem is how we define 'non-blocking' ...
> 
> SO_SNDTIMEO can be used by application to implement a variation of non-blocking,
> by waiting for a socket event with a short timeout, to maybe recover
> from memory pressure conditions in a more efficient way than simply looping.
> 
> Note that the man page for epoll() only _suggests_ to use nonblocking file descriptors.
> 
> <quote>
>        The  suggested  way  to use epoll as an edge-triggered (EPOLLET)
>        interface is as follows:
> 
>               i   with nonblocking file descriptors; and
> 
>               ii  by  waiting  for  an  event  only  after  read(2)  or
>                   write(2) return EAGAIN.
> </quote>
> 
> 

Ok, seems reasonable:
Acked-by: Jason Baron <jbaron@akamai.com>

I found a similar pattern in net/smc/smc_tx.c, which I also just sent a
patch for.

Thanks,

-Jason