[RFC net-next af_unix v1 0/1] Allow delivery of SIGURG on AF_UNIX streams socket

[RFC net-next af_unix v1 0/1] Allow delivery of SIGURG on AF_UNIX streams socket

[rao.shoaib]

From: Rao Shoaib <rao.shoaib@oracle.com>

We have a use case where thousands of processes connect locally to a
database and issue queries that are serviced by a pool of threads.
Communication is done over AF_UNIX sockets. Currently, there is no way
for the submitter to signal the servicing thread about an urgent
condition such as abandoning the query. This patch addresses that
requirement by adding support for MSG_OOB flag for AF_UNIX sockets.
On receipt of such a flag, the kernel sends a SIGURG to the peer.

Rao Shoaib (1):
  af_unix: Allow delivery of SIGURG on AF_UNIX streams socket

 net/unix/af_unix.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

-- 
1.8.3.1

[RFC net-next af_unix v1 1/1] af_unix: Allow delivery of SIGURG on AF_UNIX streams socket

[rao.shoaib]

From: Rao Shoaib <rao.shoaib@oracle.com>

For AF_UNIX stream socket send SIGURG to the peer if
the msg has MSG_OOB flag set.

Signed-off-by: Rao Shoaib <rao.shoaib@oracle.com>
---
 net/unix/af_unix.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c
index 92784e5..4f01d74 100644
--- a/net/unix/af_unix.c
+++ b/net/unix/af_unix.c
@@ -1840,8 +1840,6 @@ static int unix_stream_sendmsg(struct socket *sock, struct msghdr *msg,
 		return err;
 
 	err = -EOPNOTSUPP;
-	if (msg->msg_flags&MSG_OOB)
-		goto out_err;
 
 	if (msg->msg_namelen) {
 		err = sk->sk_state == TCP_ESTABLISHED ? -EISCONN : -EOPNOTSUPP;
@@ -1906,6 +1904,9 @@ static int unix_stream_sendmsg(struct socket *sock, struct msghdr *msg,
 		sent += size;
 	}
 
+	if (msg->msg_flags & MSG_OOB)
+		sk_send_sigurg(other);
+
 	scm_destroy(&scm);
 
 	return sent;
-- 
1.8.3.1

Re: [RFC net-next af_unix v1 0/1] Allow delivery of SIGURG on AF_UNIX streams socket

[Andy Lutomirski]

On Sat, Nov 28, 2020 at 7:07 PM <rao.shoaib@oracle.com> wrote:
>
> From: Rao Shoaib <rao.shoaib@oracle.com>
>
> We have a use case where thousands of processes connect locally to a
> database and issue queries that are serviced by a pool of threads.
> Communication is done over AF_UNIX sockets. Currently, there is no way
> for the submitter to signal the servicing thread about an urgent
> condition such as abandoning the query.

Sure there is.  You could close() the socket.  You could send() a
message saying that your query wants to be canceled.  You could use a
second socket for this if it's somehow inconvenient to do it with a
single socket.

> This patch addresses that
> requirement by adding support for MSG_OOB flag for AF_UNIX sockets.
> On receipt of such a flag, the kernel sends a SIGURG to the peer.

SIGURG is a horrible interface.  It doesn't scale -- signals are slow,
and they give no indication of *which* socket has urgent data.  Aside
from this, the "urgent" interface itself is nuts -- even the TCP RFCs
and implementations seem unable to agree on exactly how to tell
*which* data is urgent.  At least epoll might be a scalable way to
tell which socket has urgent data, but that's not what your patch
description seems to be talking about.

Oh yeah, SIGURG simply doesn't work across privilege boundaries.

I'm also a bit nervous that there might be some program out there that
expects SIGIO but not SIGURG on unix sockets, and you're breaking
them.

--Andy

Re: [RFC net-next af_unix v1 0/1] Allow delivery of SIGURG on AF_UNIX streams socket

[Rao Shoaib]

Hi Andy,

Thanks for your comments. The RFC proposes implementing missing features 
of the current socket API for AF_UNIX sockets. It does not change any 
aspect of the current API[1]. Your comments are about issues with the 
current socket API and hence are not in the scope of this RFC. We can 
work on a different RFC to address issues with the current socket API.

[1] The only feature missing in the RFC is delivery of urgent data 
marker, RFC proposes to implement only the generation of SIGURG. 
Delivery of the urgent marker was dropped based on internal feedback. I 
am open to adding that to make the API comparable to TCP. Unlike TCP, 
there is no ambiguity regarding the position of urgent byte for AF_UNIX 
sockets, as the communication is between two sockets on the same system. 
Currently the behavior is controlled via a configuration option, and 
that option will apply to both sockets.

Regards,

Shoaib

On 11/28/20 8:02 PM, Andy Lutomirski wrote:
> On Sat, Nov 28, 2020 at 7:07 PM <rao.shoaib@oracle.com> wrote:
>> From: Rao Shoaib <rao.shoaib@oracle.com>
>>
>> We have a use case where thousands of processes connect locally to a
>> database and issue queries that are serviced by a pool of threads.
>> Communication is done over AF_UNIX sockets. Currently, there is no way
>> for the submitter to signal the servicing thread about an urgent
>> condition such as abandoning the query.
> Sure there is.  You could close() the socket.  You could send() a
> message saying that your query wants to be canceled.  You could use a
> second socket for this if it's somehow inconvenient to do it with a
> single socket.
>
>> This patch addresses that
>> requirement by adding support for MSG_OOB flag for AF_UNIX sockets.
>> On receipt of such a flag, the kernel sends a SIGURG to the peer.
> SIGURG is a horrible interface.  It doesn't scale -- signals are slow,
> and they give no indication of *which* socket has urgent data.  Aside
> from this, the "urgent" interface itself is nuts -- even the TCP RFCs
> and implementations seem unable to agree on exactly how to tell
> *which* data is urgent.  At least epoll might be a scalable way to
> tell which socket has urgent data, but that's not what your patch
> description seems to be talking about.
>
> Oh yeah, SIGURG simply doesn't work across privilege boundaries.
>
> I'm also a bit nervous that there might be some program out there that
> expects SIGIO but not SIGURG on unix sockets, and you're breaking
> them.
>
> --Andy

Re: [RFC net-next af_unix v1 0/1] Allow delivery of SIGURG on AF_UNIX streams socket

[Matthew Wilcox]

On Sat, Nov 28, 2020 at 08:02:37PM -0800, Andy Lutomirski wrote:
> On Sat, Nov 28, 2020 at 7:07 PM <rao.shoaib@oracle.com> wrote:
> > We have a use case where thousands of processes connect locally to a
> > database and issue queries that are serviced by a pool of threads.
> > Communication is done over AF_UNIX sockets. Currently, there is no way
> > for the submitter to signal the servicing thread about an urgent
> > condition such as abandoning the query.
> 
> Sure there is.  You could close() the socket.  You could send() a
> message saying that your query wants to be canceled.  You could use a
> second socket for this if it's somehow inconvenient to do it with a
> single socket.

The application already works with SIGURG (and TCP, obviously).  Yes,
it could be changed to use one of these other methods, but they're all
more inconvenient to use than SIGURG.

> > This patch addresses that
> > requirement by adding support for MSG_OOB flag for AF_UNIX sockets.
> > On receipt of such a flag, the kernel sends a SIGURG to the peer.
> 
> SIGURG is a horrible interface.  It doesn't scale -- signals are slow,
> and they give no indication of *which* socket has urgent data.

It doesn't need to scale.  This is "client wishes to abort the operation",
just like telnet.  Each thread only has one socket (and, thanks to
F_SETOWN_EX, signals for that socket get directed to that thread).

> Aside
> from this, the "urgent" interface itself is nuts -- even the TCP RFCs
> and implementations seem unable to agree on exactly how to tell
> *which* data is urgent.

That doesn't matter.  Unix sockets with MSG_OOB behave like they
have had SO_OOBINLINE which is the recommendation from
https://tools.ietf.org/html/rfc6093

> At least epoll might be a scalable way to
> tell which socket has urgent data, but that's not what your patch
> description seems to be talking about.

epoll is the wrong interface for this application, as far as I can tell.

> Oh yeah, SIGURG simply doesn't work across privilege boundaries.

Please elaborate.  SIGURG is default-ignore, so you have to enable a
handler for it.  And you have to call SETOWN to receive a signal.

> I'm also a bit nervous that there might be some program out there that
> expects SIGIO but not SIGURG on unix sockets, and you're breaking
> them.

The only incompatible scenario I've been able to think of is an
application which has both TCP and Unix sockets will suddenly get its
SIGURG handler called for Unix sockets when it's only expecting to get
it called for TCP sockets.  That feels like a very unlikely scenario.

Re: [RFC net-next af_unix v1 0/1] Allow delivery of SIGURG on AF_UNIX streams socket

[Andy Lutomirski]

> On Dec 7, 2020, at 8:35 AM, Matthew Wilcox <willy@infradead.org> wrote:
> 
> On Sat, Nov 28, 2020 at 08:02:37PM -0800, Andy Lutomirski wrote:
>>> On Sat, Nov 28, 2020 at 7:07 PM <rao.shoaib@oracle.com> wrote:
>>> We have a use case where thousands of processes connect locally to a
>>> database and issue queries that are serviced by a pool of threads.
>>> Communication is done over AF_UNIX sockets. Currently, there is no way
>>> for the submitter to signal the servicing thread about an urgent
>>> condition such as abandoning the query.
>> 
>> Sure there is.  You could close() the socket.  You could send() a
>> message saying that your query wants to be canceled.  You could use a
>> second socket for this if it's somehow inconvenient to do it with a
>> single socket.
> 
> The application already works with SIGURG (and TCP, obviously).  Yes,
> it could be changed to use one of these other methods, but they're all
> more inconvenient to use than SIGURG.
> 
>>> This patch addresses that
>>> requirement by adding support for MSG_OOB flag for AF_UNIX sockets.
>>> On receipt of such a flag, the kernel sends a SIGURG to the peer.
>> 
>> SIGURG is a horrible interface.  It doesn't scale -- signals are slow,
>> and they give no indication of *which* socket has urgent data.
> 
> It doesn't need to scale.  This is "client wishes to abort the operation",
> just like telnet.  Each thread only has one socket (and, thanks to
> F_SETOWN_EX, signals for that socket get directed to that thread).
> 

Oh, you have thousands of clients and each client has its own server thread. Lovely.

>> Aside
>> from this, the "urgent" interface itself is nuts -- even the TCP RFCs
>> and implementations seem unable to agree on exactly how to tell
>> *which* data is urgent.
> 
> That doesn't matter.  Unix sockets with MSG_OOB behave like they
> have had SO_OOBINLINE which is the recommendation from
> https://tools.ietf.org/html/rfc6093

Fair enough.

> 
>> At least epoll might be a scalable way to
>> tell which socket has urgent data, but that's not what your patch
>> description seems to be talking about.
> 
> epoll is the wrong interface for this application, as far as I can tell.
> 
>> Oh yeah, SIGURG simply doesn't work across privilege boundaries.
> 
> Please elaborate.  SIGURG is default-ignore, so you have to enable a
> handler for it.  And you have to call SETOWN to receive a signal.
> 
>> I'm also a bit nervous that there might be some program out there that
>> expects SIGIO but not SIGURG on unix sockets, and you're breaking
>> them.
> 
> The only incompatible scenario I've been able to think of is an
> application which has both TCP and Unix sockets will suddenly get its
> SIGURG handler called for Unix sockets when it's only expecting to get
> it called for TCP sockets.  That feels like a very unlikely scenario.

So maybe this is okay. It does seem like a hack, but maybe it’s a hack that’s not too bad to support in the kernel.

Re: [RFC net-next af_unix v1 0/1] Allow delivery of SIGURG on AF_UNIX streams socket

[Matthew Wilcox]

On Mon, Dec 07, 2020 at 09:00:12AM -0800, Andy Lutomirski wrote:
> >> I'm also a bit nervous that there might be some program out there that
> >> expects SIGIO but not SIGURG on unix sockets, and you're breaking
> >> them.
> > 
> > The only incompatible scenario I've been able to think of is an
> > application which has both TCP and Unix sockets will suddenly get its
> > SIGURG handler called for Unix sockets when it's only expecting to get
> > it called for TCP sockets.  That feels like a very unlikely scenario.
> 
> So maybe this is okay. It does seem like a hack, but maybe it’s a hack that’s not too bad to support in the kernel.

I see it as extending a (somewhat bogus) TCP socket API to Unix sockets.
The mechanism is badly named.  If it were called MSG_URG instead of
MSG_OOB, I think it'd be a lot less objectionable.  Apparently the
sockets people were enamoured with the idea of having multiple streams
of data inside a single socket, which is madness.

I'd really like a way to generate SIGURG on pipes too, but that's harder
to wire up.

Re: [RFC net-next af_unix v1 0/1] Allow delivery of SIGURG on AF_UNIX streams socket

[Jakub Kicinski]

On Sat, 14 Nov 2020 18:58:04 -0800 rao.shoaib@oracle.com wrote:
> From: Rao Shoaib <rao.shoaib@oracle.com>
> 
> The use of AF_UNIX sockets is on the rise. We have a case where thousands
> of processes connect locally to a database and issue queries that are
> serviced by a pool of threads. Communication is done over AF_UNIX
> sockets. Currently, there is no way for the submitter to signal the
> servicing thread about an urgent condition such as abandoning
> the query. This patch addresses that requirement by adding support for
> MSG_OOB flag for AF_UNIX sockets. On receipt of such a flag,
> the kernel sends a SIGURG to the peer.

You need to widen the CC list on this, I doubt anyone paid much
attention.

[RFC net-next af_unix v1 0/1] Allow delivery of SIGURG on AF_UNIX streams socket

[rao.shoaib]

From: Rao Shoaib <rao.shoaib@oracle.com>

The use of AF_UNIX sockets is on the rise. We have a case where thousands
of processes connect locally to a database and issue queries that are
serviced by a pool of threads. Communication is done over AF_UNIX
sockets. Currently, there is no way for the submitter to signal the
servicing thread about an urgent condition such as abandoning
the query. This patch addresses that requirement by adding support for
MSG_OOB flag for AF_UNIX sockets. On receipt of such a flag,
the kernel sends a SIGURG to the peer.

Rao Shoaib (1):
  af_unix: Allow delivery of SIGURG on AF_UNIX streams socket

 net/unix/af_unix.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

-- 
1.8.3.1

[RFC net-next af_unix v1 0/1] Allow delivery of SIGURG on AF_UNIX streams socket

[rao.shoaib]

From: Rao Shoaib <rao.shoaib@oracle.com>

The use of AF_UNIX sockets is on the rise. We have a case where thousands
of processes connect locally to a database and issue queries that are
serviced by a pool of threads. Communication is done over AF_UNIX
sockets. Currently, there is no way for the submitter to signal the
servicing thread about an urgent condition such as abandoning
the query. This patch addresses that requirement by adding support for
MSG_OOB flag for AF_UNIX sockets. On receipt of such a flag,
the kernel sends a SIGURG to the peer.

Rao Shoaib (1):
  af_unix: Allow delivery of SIGURG on AF_UNIX streams socket

 net/unix/af_unix.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

-- 
1.8.3.1