* [patch] socket.7: Document SO_INCOMING_CPU
@ 2017-02-18 4:06 Francois Saint-Jacques
[not found] ` <CABNn7+rHfUTjMtm3Biqfx83G6Rr7pY-LuHXY=JqA6N3H_SstZg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
0 siblings, 1 reply; 8+ messages in thread
From: Francois Saint-Jacques @ 2017-02-18 4:06 UTC (permalink / raw)
To: mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w
Cc: linux-man-u79uwXL29TY76Z2rM5mHXA, netdev-u79uwXL29TY76Z2rM5mHXA
This socket option is undocumented. Applies on the latest version
(man-pages-4.09-511).
diff --git a/man7/socket.7 b/man7/socket.7
index 3efd7a5d8..1a3ffa253 100644
--- a/man7/socket.7
+++ b/man7/socket.7
@@ -490,6 +490,26 @@ flag on a socket
operation.
Expects an integer boolean flag.
.TP
+.BR SO_INCOMING_CPU " (getsockopt since Linux 3.19, setsockopt since
Linux 4.4)"
+.\" getsocktop 2c8c56e15df3d4c2af3d656e44feb18789f75837
+.\" setsocktop 70da268b569d32a9fddeea85dc18043de9d89f89
+Sets or gets the cpu affinity of a socket. Expects an integer flag.
+.sp
+.in +4n
+.nf
+int cpu = 1;
+socklen_t len = sizeof(cpu);
+setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
+.fi
+.in
+.sp
+The typical use case is one listener per RX queue, as the associated listener
+should only accept flows handled in softirq by the same cpu. This provides
+optimal NUMA behavior and keep cpu caches hot.
+.TP
.B SO_KEEPALIVE
Enable sending of keep-alive messages on connection-oriented sockets.
Expects an integer boolean flag.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [patch] socket.7: Document SO_INCOMING_CPU
[not found] ` <CABNn7+rHfUTjMtm3Biqfx83G6Rr7pY-LuHXY=JqA6N3H_SstZg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2017-02-19 20:55 ` Michael Kerrisk (man-pages)
2017-02-20 15:26 ` Francois Saint-Jacques
[not found] ` <63815aac-9c8f-c599-9422-5c312cefc9e8-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
0 siblings, 2 replies; 8+ messages in thread
From: Michael Kerrisk (man-pages) @ 2017-02-19 20:55 UTC (permalink / raw)
To: Francois Saint-Jacques
Cc: mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w,
linux-man-u79uwXL29TY76Z2rM5mHXA, netdev-u79uwXL29TY76Z2rM5mHXA,
Eric Dumazet
[CC += Eric, so that he might review]
Hello Francois,
On 02/18/2017 05:06 AM, Francois Saint-Jacques wrote:
> This socket option is undocumented. Applies on the latest version
> (man-pages-4.09-511).
>
> diff --git a/man7/socket.7 b/man7/socket.7
> index 3efd7a5d8..1a3ffa253 100644
> --- a/man7/socket.7
> +++ b/man7/socket.7
> @@ -490,6 +490,26 @@ flag on a socket
> operation.
> Expects an integer boolean flag.
> .TP
> +.BR SO_INCOMING_CPU " (getsockopt since Linux 3.19, setsockopt since
> Linux 4.4)"
> +.\" getsocktop 2c8c56e15df3d4c2af3d656e44feb18789f75837
> +.\" setsocktop 70da268b569d32a9fddeea85dc18043de9d89f89
> +Sets or gets the cpu affinity of a socket. Expects an integer flag.
> +.sp
> +.in +4n
> +.nf
> +int cpu = 1;
> +socklen_t len = sizeof(cpu);
> +setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
> +.fi
> +.in
> +.sp
> +The typical use case is one listener per RX queue, as the associated listener
> +should only accept flows handled in softirq by the same cpu. This provides
> +optimal NUMA behavior and keep cpu caches hot.
> +.TP
> .B SO_KEEPALIVE
> Enable sending of keep-alive messages on connection-oriented sockets.
> Expects an integer boolean flag.
Thank you! Patch applied.
I have tried to enhance the description somewhat. I'm not sure whether
what I've written is quite correct (or whether it should be further
extended). Eric, could you please take a look at the following, and let
me know if anything needs fixing:
SO_INCOMING_CPU (gettable since Linux 3.19, settable since Linux
4.4)
Sets or gets the CPU affinity of a socket. Expects an
integer flag.
int cpu = 1;
socklen_t len = sizeof(cpu);
setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
Because all of the packets for a single stream (i.e., all
packets for the same 4-tuple) arrive on the single RX queue
that is associated with a particular CPU, the typical use
case is to employ one listening process per RX queue, with
the incoming flow being handled by a listener on the same
CPU that is handling the RX queue. This provides optimal
NUMA behavior and keeps CPU caches hot.
Cheers,
Michael
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [patch] socket.7: Document SO_INCOMING_CPU
2017-02-19 20:55 ` Michael Kerrisk (man-pages)
@ 2017-02-20 15:26 ` Francois Saint-Jacques
[not found] ` <63815aac-9c8f-c599-9422-5c312cefc9e8-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
1 sibling, 0 replies; 8+ messages in thread
From: Francois Saint-Jacques @ 2017-02-20 15:26 UTC (permalink / raw)
To: Michael Kerrisk (man-pages); +Cc: linux-man, netdev, Eric Dumazet
> Because all of the packets for a single stream (i.e., all
> packets for the same 4-tuple) arrive on the single RX queue
This is a configuration dependent behaviour and not always the
default. Maybe put a reference to `ethtool(8)` for more information
instead of possibly misleading the user?
François
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [patch] socket.7: Document SO_INCOMING_CPU
[not found] ` <63815aac-9c8f-c599-9422-5c312cefc9e8-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
@ 2017-04-19 13:20 ` Michael Kerrisk (man-pages)
2017-04-19 17:05 ` Eric Dumazet
0 siblings, 1 reply; 8+ messages in thread
From: Michael Kerrisk (man-pages) @ 2017-04-19 13:20 UTC (permalink / raw)
To: Francois Saint-Jacques
Cc: mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w,
linux-man-u79uwXL29TY76Z2rM5mHXA, netdev-u79uwXL29TY76Z2rM5mHXA,
Eric Dumazet
Ping Eric!
Would you have a chance to review the proposed text below, please.
Thanks,
Michael
On 02/19/2017 09:55 PM, Michael Kerrisk (man-pages) wrote:
> [CC += Eric, so that he might review]
>
> Hello Francois,
>
> On 02/18/2017 05:06 AM, Francois Saint-Jacques wrote:
>> This socket option is undocumented. Applies on the latest version
>> (man-pages-4.09-511).
>>
>> diff --git a/man7/socket.7 b/man7/socket.7
>> index 3efd7a5d8..1a3ffa253 100644
>> --- a/man7/socket.7
>> +++ b/man7/socket.7
>> @@ -490,6 +490,26 @@ flag on a socket
>> operation.
>> Expects an integer boolean flag.
>> .TP
>> +.BR SO_INCOMING_CPU " (getsockopt since Linux 3.19, setsockopt since
>> Linux 4.4)"
>> +.\" getsocktop 2c8c56e15df3d4c2af3d656e44feb18789f75837
>> +.\" setsocktop 70da268b569d32a9fddeea85dc18043de9d89f89
>> +Sets or gets the cpu affinity of a socket. Expects an integer flag.
>> +.sp
>> +.in +4n
>> +.nf
>> +int cpu = 1;
>> +socklen_t len = sizeof(cpu);
>> +setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
>> +.fi
>> +.in
>> +.sp
>> +The typical use case is one listener per RX queue, as the associated listener
>> +should only accept flows handled in softirq by the same cpu. This provides
>> +optimal NUMA behavior and keep cpu caches hot.
>> +.TP
>> .B SO_KEEPALIVE
>> Enable sending of keep-alive messages on connection-oriented sockets.
>> Expects an integer boolean flag.
>
> Thank you! Patch applied.
>
> I have tried to enhance the description somewhat. I'm not sure whether
> what I've written is quite correct (or whether it should be further
> extended). Eric, could you please take a look at the following, and let
> me know if anything needs fixing:
>
> SO_INCOMING_CPU (gettable since Linux 3.19, settable since Linux
> 4.4)
> Sets or gets the CPU affinity of a socket. Expects an
> integer flag.
>
> int cpu = 1;
> socklen_t len = sizeof(cpu);
> setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
>
> Because all of the packets for a single stream (i.e., all
> packets for the same 4-tuple) arrive on the single RX queue
> that is associated with a particular CPU, the typical use
> case is to employ one listening process per RX queue, with
> the incoming flow being handled by a listener on the same
> CPU that is handling the RX queue. This provides optimal
> NUMA behavior and keeps CPU caches hot.
>
> Cheers,
>
> Michael
>
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [patch] socket.7: Document SO_INCOMING_CPU
2017-04-19 13:20 ` Michael Kerrisk (man-pages)
@ 2017-04-19 17:05 ` Eric Dumazet
[not found] ` <1492621535.22296.8.camel-XN9IlZ5yJG9HTL0Zs8A6p+yfmBU6pStAUsxypvmhUTTZJqsBc5GL+g@public.gmane.org>
0 siblings, 1 reply; 8+ messages in thread
From: Eric Dumazet @ 2017-04-19 17:05 UTC (permalink / raw)
To: Michael Kerrisk (man-pages)
Cc: Francois Saint-Jacques, linux-man, netdev, Eric Dumazet
On Wed, 2017-04-19 at 15:20 +0200, Michael Kerrisk (man-pages) wrote:
> Ping Eric!
>
> Would you have a chance to review the proposed text below, please.
>
> Thanks,
>
Hi Michael
Sorry for the delay.
Note that setting the option is not supported if SO_REUSEPORT is used.
Socket will be selected from an array, either by a hash or BPF program
that has no access to this information.
Thanks !
> Michael
>
> On 02/19/2017 09:55 PM, Michael Kerrisk (man-pages) wrote:
> > [CC += Eric, so that he might review]
> >
> > Hello Francois,
> >
> > On 02/18/2017 05:06 AM, Francois Saint-Jacques wrote:
> >> This socket option is undocumented. Applies on the latest version
> >> (man-pages-4.09-511).
> >>
> >> diff --git a/man7/socket.7 b/man7/socket.7
> >> index 3efd7a5d8..1a3ffa253 100644
> >> --- a/man7/socket.7
> >> +++ b/man7/socket.7
> >> @@ -490,6 +490,26 @@ flag on a socket
> >> operation.
> >> Expects an integer boolean flag.
> >> .TP
> >> +.BR SO_INCOMING_CPU " (getsockopt since Linux 3.19, setsockopt since
> >> Linux 4.4)"
> >> +.\" getsocktop 2c8c56e15df3d4c2af3d656e44feb18789f75837
> >> +.\" setsocktop 70da268b569d32a9fddeea85dc18043de9d89f89
> >> +Sets or gets the cpu affinity of a socket. Expects an integer flag.
> >> +.sp
> >> +.in +4n
> >> +.nf
> >> +int cpu = 1;
> >> +socklen_t len = sizeof(cpu);
> >> +setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
> >> +.fi
> >> +.in
> >> +.sp
> >> +The typical use case is one listener per RX queue, as the associated listener
> >> +should only accept flows handled in softirq by the same cpu. This provides
> >> +optimal NUMA behavior and keep cpu caches hot.
> >> +.TP
> >> .B SO_KEEPALIVE
> >> Enable sending of keep-alive messages on connection-oriented sockets.
> >> Expects an integer boolean flag.
> >
> > Thank you! Patch applied.
> >
> > I have tried to enhance the description somewhat. I'm not sure whether
> > what I've written is quite correct (or whether it should be further
> > extended). Eric, could you please take a look at the following, and let
> > me know if anything needs fixing:
> >
> > SO_INCOMING_CPU (gettable since Linux 3.19, settable since Linux
> > 4.4)
> > Sets or gets the CPU affinity of a socket. Expects an
> > integer flag.
> >
> > int cpu = 1;
> > socklen_t len = sizeof(cpu);
> > setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
> >
> > Because all of the packets for a single stream (i.e., all
> > packets for the same 4-tuple) arrive on the single RX queue
> > that is associated with a particular CPU, the typical use
> > case is to employ one listening process per RX queue, with
> > the incoming flow being handled by a listener on the same
> > CPU that is handling the RX queue. This provides optimal
> > NUMA behavior and keeps CPU caches hot.
> >
> > Cheers,
> >
> > Michael
> >
>
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [patch] socket.7: Document SO_INCOMING_CPU
[not found] ` <1492621535.22296.8.camel-XN9IlZ5yJG9HTL0Zs8A6p+yfmBU6pStAUsxypvmhUTTZJqsBc5GL+g@public.gmane.org>
@ 2017-04-19 18:48 ` Michael Kerrisk (man-pages)
[not found] ` <326b99c3-cc86-3abd-1069-2b3a52d9ba47-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
0 siblings, 1 reply; 8+ messages in thread
From: Michael Kerrisk (man-pages) @ 2017-04-19 18:48 UTC (permalink / raw)
To: Eric Dumazet
Cc: mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w, Francois Saint-Jacques,
linux-man-u79uwXL29TY76Z2rM5mHXA, netdev-u79uwXL29TY76Z2rM5mHXA,
Eric Dumazet
Hi Eric,
[reodering for clarity]
>> On 02/19/2017 09:55 PM, Michael Kerrisk (man-pages) wrote:
>>> [CC += Eric, so that he might review]
>>>
>>> Hello Francois,
>>>
>>> On 02/18/2017 05:06 AM, Francois Saint-Jacques wrote:
>>>> This socket option is undocumented. Applies on the latest version
>>>> (man-pages-4.09-511).
>>>>
>>>> diff --git a/man7/socket.7 b/man7/socket.7
>>>> index 3efd7a5d8..1a3ffa253 100644
>>>> --- a/man7/socket.7
>>>> +++ b/man7/socket.7
>>>> @@ -490,6 +490,26 @@ flag on a socket
>>>> operation.
>>>> Expects an integer boolean flag.
>>>> .TP
>>>> +.BR SO_INCOMING_CPU " (getsockopt since Linux 3.19, setsockopt since
>>>> Linux 4.4)"
>>>> +.\" getsocktop 2c8c56e15df3d4c2af3d656e44feb18789f75837
>>>> +.\" setsocktop 70da268b569d32a9fddeea85dc18043de9d89f89
>>>> +Sets or gets the cpu affinity of a socket. Expects an integer flag.
>>>> +.sp
>>>> +.in +4n
>>>> +.nf
>>>> +int cpu = 1;
>>>> +socklen_t len = sizeof(cpu);
>>>> +setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
>>>> +.fi
>>>> +.in
>>>> +.sp
>>>> +The typical use case is one listener per RX queue, as the associated listener
>>>> +should only accept flows handled in softirq by the same cpu. This provides
>>>> +optimal NUMA behavior and keep cpu caches hot.
>>>> +.TP
>>>> .B SO_KEEPALIVE
>>>> Enable sending of keep-alive messages on connection-oriented sockets.
>>>> Expects an integer boolean flag.
>>>
>>> Thank you! Patch applied.
>>>
>>> I have tried to enhance the description somewhat. I'm not sure whether
>>> what I've written is quite correct (or whether it should be further
>>> extended). Eric, could you please take a look at the following, and let
>>> me know if anything needs fixing:
>>>
>>> SO_INCOMING_CPU (gettable since Linux 3.19, settable since Linux
>>> 4.4)
>>> Sets or gets the CPU affinity of a socket. Expects an
>>> integer flag.
>>>
>>> int cpu = 1;
>>> socklen_t len = sizeof(cpu);
>>> setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
>>>
>>> Because all of the packets for a single stream (i.e., all
>>> packets for the same 4-tuple) arrive on the single RX queue
>>> that is associated with a particular CPU, the typical use
>>> case is to employ one listening process per RX queue, with
>>> the incoming flow being handled by a listener on the same
>>> CPU that is handling the RX queue. This provides optimal
>>> NUMA behavior and keeps CPU caches hot.
> Hi Michael
>
> Sorry for the delay.
Thanks for the reply, but I think you are assuming I know more than
I do. I'd like you to elaborate a little please. See below.
> Note that setting the option is not supported if SO_REUSEPORT is used.
Please define "not supported". Does this yield an API diagnostic?
If so, what is it?
> Socket will be selected from an array, either by a hash or BPF program
> that has no access to this information.
Sorry -- I'm lost here. How does this comment relate to the proposed
man page text above?
Thanks,
Michael
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [patch] socket.7: Document SO_INCOMING_CPU
[not found] ` <326b99c3-cc86-3abd-1069-2b3a52d9ba47-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
@ 2017-04-19 20:13 ` Eric Dumazet
2017-04-20 14:43 ` Michael Kerrisk (man-pages)
0 siblings, 1 reply; 8+ messages in thread
From: Eric Dumazet @ 2017-04-19 20:13 UTC (permalink / raw)
To: Michael Kerrisk (man-pages)
Cc: Francois Saint-Jacques, linux-man-u79uwXL29TY76Z2rM5mHXA,
netdev-u79uwXL29TY76Z2rM5mHXA, Eric Dumazet
On Wed, 2017-04-19 at 20:48 +0200, Michael Kerrisk (man-pages) wrote:
> Hi Eric,
>
> [reodering for clarity]
>
> >> On 02/19/2017 09:55 PM, Michael Kerrisk (man-pages) wrote:
> >>> [CC += Eric, so that he might review]
> >>>
> >>> Hello Francois,
> >>>
> >>> On 02/18/2017 05:06 AM, Francois Saint-Jacques wrote:
> >>>> This socket option is undocumented. Applies on the latest version
> >>>> (man-pages-4.09-511).
> >>>>
> >>>> diff --git a/man7/socket.7 b/man7/socket.7
> >>>> index 3efd7a5d8..1a3ffa253 100644
> >>>> --- a/man7/socket.7
> >>>> +++ b/man7/socket.7
> >>>> @@ -490,6 +490,26 @@ flag on a socket
> >>>> operation.
> >>>> Expects an integer boolean flag.
> >>>> .TP
> >>>> +.BR SO_INCOMING_CPU " (getsockopt since Linux 3.19, setsockopt since
> >>>> Linux 4.4)"
> >>>> +.\" getsocktop 2c8c56e15df3d4c2af3d656e44feb18789f75837
> >>>> +.\" setsocktop 70da268b569d32a9fddeea85dc18043de9d89f89
> >>>> +Sets or gets the cpu affinity of a socket. Expects an integer flag.
> >>>> +.sp
> >>>> +.in +4n
> >>>> +.nf
> >>>> +int cpu = 1;
> >>>> +socklen_t len = sizeof(cpu);
> >>>> +setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
> >>>> +.fi
> >>>> +.in
> >>>> +.sp
> >>>> +The typical use case is one listener per RX queue, as the associated listener
> >>>> +should only accept flows handled in softirq by the same cpu. This provides
> >>>> +optimal NUMA behavior and keep cpu caches hot.
> >>>> +.TP
> >>>> .B SO_KEEPALIVE
> >>>> Enable sending of keep-alive messages on connection-oriented sockets.
> >>>> Expects an integer boolean flag.
> >>>
> >>> Thank you! Patch applied.
> >>>
> >>> I have tried to enhance the description somewhat. I'm not sure whether
> >>> what I've written is quite correct (or whether it should be further
> >>> extended). Eric, could you please take a look at the following, and let
> >>> me know if anything needs fixing:
> >>>
> >>> SO_INCOMING_CPU (gettable since Linux 3.19, settable since Linux
> >>> 4.4)
> >>> Sets or gets the CPU affinity of a socket. Expects an
> >>> integer flag.
> >>>
> >>> int cpu = 1;
> >>> socklen_t len = sizeof(cpu);
> >>> setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
> >>>
> >>> Because all of the packets for a single stream (i.e., all
> >>> packets for the same 4-tuple) arrive on the single RX queue
> >>> that is associated with a particular CPU, the typical use
> >>> case is to employ one listening process per RX queue, with
> >>> the incoming flow being handled by a listener on the same
> >>> CPU that is handling the RX queue. This provides optimal
> >>> NUMA behavior and keeps CPU caches hot.
>
> > Hi Michael
> >
> > Sorry for the delay.
>
> Thanks for the reply, but I think you are assuming I know more than
> I do. I'd like you to elaborate a little please. See below.
>
> > Note that setting the option is not supported if SO_REUSEPORT is used.
>
> Please define "not supported". Does this yield an API diagnostic?
> If so, what is it?
>
> > Socket will be selected from an array, either by a hash or BPF program
> > that has no access to this information.
>
> Sorry -- I'm lost here. How does this comment relate to the proposed
> man page text above?
Simply that :
If an application uses both SO_INCOMING_CPU and SO_REUSEPORT, then
SO_REUSEPORT logic, selecting the socket to receive the packet, ignores
SO_INCOMING_CPU setting.
This does not need to be documented, because it is an implementation
detail/bug that could be changed, if someone cares enough.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [patch] socket.7: Document SO_INCOMING_CPU
2017-04-19 20:13 ` Eric Dumazet
@ 2017-04-20 14:43 ` Michael Kerrisk (man-pages)
0 siblings, 0 replies; 8+ messages in thread
From: Michael Kerrisk (man-pages) @ 2017-04-20 14:43 UTC (permalink / raw)
To: Eric Dumazet
Cc: mtk.manpages, Francois Saint-Jacques, linux-man, netdev, Eric Dumazet
On 04/19/2017 10:13 PM, Eric Dumazet wrote:
> On Wed, 2017-04-19 at 20:48 +0200, Michael Kerrisk (man-pages) wrote:
>> Hi Eric,
>>
>> [reodering for clarity]
>>
>>>> On 02/19/2017 09:55 PM, Michael Kerrisk (man-pages) wrote:
>>>>> [CC += Eric, so that he might review]
>>>>>
>>>>> Hello Francois,
>>>>>
>>>>> On 02/18/2017 05:06 AM, Francois Saint-Jacques wrote:
>>>>>> This socket option is undocumented. Applies on the latest version
>>>>>> (man-pages-4.09-511).
>>>>>>
>>>>>> diff --git a/man7/socket.7 b/man7/socket.7
>>>>>> index 3efd7a5d8..1a3ffa253 100644
>>>>>> --- a/man7/socket.7
>>>>>> +++ b/man7/socket.7
>>>>>> @@ -490,6 +490,26 @@ flag on a socket
>>>>>> operation.
>>>>>> Expects an integer boolean flag.
>>>>>> .TP
>>>>>> +.BR SO_INCOMING_CPU " (getsockopt since Linux 3.19, setsockopt since
>>>>>> Linux 4.4)"
>>>>>> +.\" getsocktop 2c8c56e15df3d4c2af3d656e44feb18789f75837
>>>>>> +.\" setsocktop 70da268b569d32a9fddeea85dc18043de9d89f89
>>>>>> +Sets or gets the cpu affinity of a socket. Expects an integer flag.
>>>>>> +.sp
>>>>>> +.in +4n
>>>>>> +.nf
>>>>>> +int cpu = 1;
>>>>>> +socklen_t len = sizeof(cpu);
>>>>>> +setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
>>>>>> +.fi
>>>>>> +.in
>>>>>> +.sp
>>>>>> +The typical use case is one listener per RX queue, as the associated listener
>>>>>> +should only accept flows handled in softirq by the same cpu. This provides
>>>>>> +optimal NUMA behavior and keep cpu caches hot.
>>>>>> +.TP
>>>>>> .B SO_KEEPALIVE
>>>>>> Enable sending of keep-alive messages on connection-oriented sockets.
>>>>>> Expects an integer boolean flag.
>>>>>
>>>>> Thank you! Patch applied.
>>>>>
>>>>> I have tried to enhance the description somewhat. I'm not sure whether
>>>>> what I've written is quite correct (or whether it should be further
>>>>> extended). Eric, could you please take a look at the following, and let
>>>>> me know if anything needs fixing:
>>>>>
>>>>> SO_INCOMING_CPU (gettable since Linux 3.19, settable since Linux
>>>>> 4.4)
>>>>> Sets or gets the CPU affinity of a socket. Expects an
>>>>> integer flag.
>>>>>
>>>>> int cpu = 1;
>>>>> socklen_t len = sizeof(cpu);
>>>>> setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu, &len);
>>>>>
>>>>> Because all of the packets for a single stream (i.e., all
>>>>> packets for the same 4-tuple) arrive on the single RX queue
>>>>> that is associated with a particular CPU, the typical use
>>>>> case is to employ one listening process per RX queue, with
>>>>> the incoming flow being handled by a listener on the same
>>>>> CPU that is handling the RX queue. This provides optimal
>>>>> NUMA behavior and keeps CPU caches hot.
>>
>>> Hi Michael
>>>
>>> Sorry for the delay.
>>
>> Thanks for the reply, but I think you are assuming I know more than
>> I do. I'd like you to elaborate a little please. See below.
>>
>>> Note that setting the option is not supported if SO_REUSEPORT is used.
>>
>> Please define "not supported". Does this yield an API diagnostic?
>> If so, what is it?
>>
>>> Socket will be selected from an array, either by a hash or BPF program
>>> that has no access to this information.
>>
>> Sorry -- I'm lost here. How does this comment relate to the proposed
>> man page text above?
>
> Simply that :
>
> If an application uses both SO_INCOMING_CPU and SO_REUSEPORT, then
> SO_REUSEPORT logic, selecting the socket to receive the packet, ignores
> SO_INCOMING_CPU setting.
>
> This does not need to be documented, because it is an implementation
> detail/bug that could be changed, if someone cares enough.
Okay, thanks, Eric. I'll just merge the page text as it currently
is then.
Cheers,
Michael
--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2017-04-20 14:43 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-02-18 4:06 [patch] socket.7: Document SO_INCOMING_CPU Francois Saint-Jacques
[not found] ` <CABNn7+rHfUTjMtm3Biqfx83G6Rr7pY-LuHXY=JqA6N3H_SstZg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2017-02-19 20:55 ` Michael Kerrisk (man-pages)
2017-02-20 15:26 ` Francois Saint-Jacques
[not found] ` <63815aac-9c8f-c599-9422-5c312cefc9e8-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2017-04-19 13:20 ` Michael Kerrisk (man-pages)
2017-04-19 17:05 ` Eric Dumazet
[not found] ` <1492621535.22296.8.camel-XN9IlZ5yJG9HTL0Zs8A6p+yfmBU6pStAUsxypvmhUTTZJqsBc5GL+g@public.gmane.org>
2017-04-19 18:48 ` Michael Kerrisk (man-pages)
[not found] ` <326b99c3-cc86-3abd-1069-2b3a52d9ba47-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2017-04-19 20:13 ` Eric Dumazet
2017-04-20 14:43 ` Michael Kerrisk (man-pages)
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.