Why not use all the syn queues? in the function "tcp_conn_request", I have some questions.

Why not use all the syn queues? in the function "tcp_conn_request", I have some questions.

[Ttttabcd]

Hello everyone,recently I am looking at the source code for handling TCP three-way handshake(Linux Kernel version 4.18.5).

I found some strange places in the source code for handling syn messages.

in the function "tcp_conn_request"

This code will be executed when we don't enable the syn cookies.

		if (!net->ipv4.sysctl_tcp_syncookies &&
		    (net->ipv4.sysctl_max_syn_backlog - inet_csk_reqsk_queue_len(sk) <
		     (net->ipv4.sysctl_max_syn_backlog >> 2)) &&
		    !tcp_peer_is_proven(req, dst)) {
			/* Without syncookies last quarter of
			 * backlog is filled with destinations,
			 * proven to be alive.
			 * It means that we continue to communicate
			 * to destinations, already remembered
			 * to the moment of synflood.
			 */
			pr_drop_req(req, ntohs(tcp_hdr(skb)->source),
				    rsk_ops->family);
			goto drop_and_release;
		}

But why don't we use all the syn queues?

Why do we need to leave the size of (net->ipv4.sysctl_max_syn_backlog >> 2) in the queue?

Even if the system is attacked by a syn flood, there is no need to leave a part. Why do we need to leave a part?

The value of sysctl_max_syn_backlog is the maximum length of the queue only if syn cookies are enabled.


This is the first strange place, here is another strange place

	__u32 isn = TCP_SKB_CB(skb)->tcp_tw_isn;

	if ((net->ipv4.sysctl_tcp_syncookies == 2 ||
	     inet_csk_reqsk_queue_is_full(sk)) && !isn) {

	if (!want_cookie && !isn) {

The value of "isn" comes from TCP_SKB_CB(skb)->tcp_tw_isn, then it is judged twice whether its value is indeed 0.

But "tcp_tw_isn" is initialized in the function "tcp_v4_fill_cb"

	TCP_SKB_CB(skb)->tcp_tw_isn = 0;

So it has always been 0, I used printk to test, and the result is always 0.

Since it is always 0, why do you need to judge twice?

This is two strange places I found.

Can anyone tell me why the code here is written like this?

Re: Why not use all the syn queues? in the function "tcp_conn_request", I have some questions.

[Eric Dumazet]

On 09/03/2018 10:31 PM, Ttttabcd wrote:
> Hello everyone,recently I am looking at the source code for handling TCP three-way handshake(Linux Kernel version 4.18.5).
> 
> I found some strange places in the source code for handling syn messages.
> 
> in the function "tcp_conn_request"
> 
> This code will be executed when we don't enable the syn cookies.
> 
> 		if (!net->ipv4.sysctl_tcp_syncookies &&
> 		    (net->ipv4.sysctl_max_syn_backlog - inet_csk_reqsk_queue_len(sk) <
> 		     (net->ipv4.sysctl_max_syn_backlog >> 2)) &&
> 		    !tcp_peer_is_proven(req, dst)) {
> 			/* Without syncookies last quarter of
> 			 * backlog is filled with destinations,
> 			 * proven to be alive.
> 			 * It means that we continue to communicate
> 			 * to destinations, already remembered
> 			 * to the moment of synflood.
> 			 */
> 			pr_drop_req(req, ntohs(tcp_hdr(skb)->source),
> 				    rsk_ops->family);
> 			goto drop_and_release;
> 		}
> 
> But why don't we use all the syn queues?


Isn't it explained in the comment ?

Anyway, I am not sure anyone disables syn cookies.

Re: Why not use all the syn queues? in the function "tcp_conn_request", I have some questions.

[Neal Cardwell]

On Tue, Sep 4, 2018 at 1:48 AM Ttttabcd <ttttabcd@protonmail.com> wrote:
>
> Hello everyone,recently I am looking at the source code for handling TCP three-way handshake(Linux Kernel version 4.18.5).
>
> I found some strange places in the source code for handling syn messages.
>
> in the function "tcp_conn_request"
>
> This code will be executed when we don't enable the syn cookies.
>
>                 if (!net->ipv4.sysctl_tcp_syncookies &&
>                     (net->ipv4.sysctl_max_syn_backlog - inet_csk_reqsk_queue_len(sk) <
>                      (net->ipv4.sysctl_max_syn_backlog >> 2)) &&
>                     !tcp_peer_is_proven(req, dst)) {
>                         /* Without syncookies last quarter of
>                          * backlog is filled with destinations,
>                          * proven to be alive.
>                          * It means that we continue to communicate
>                          * to destinations, already remembered
>                          * to the moment of synflood.
>                          */
>                         pr_drop_req(req, ntohs(tcp_hdr(skb)->source),
>                                     rsk_ops->family);
>                         goto drop_and_release;
>                 }
>
> But why don't we use all the syn queues?

If tcp_peer_is_proven() returns true then we do allow ourselves to use
the whole queue.

> Why do we need to leave the size of (net->ipv4.sysctl_max_syn_backlog >> 2) in the queue?
>
> Even if the system is attacked by a syn flood, there is no need to leave a part. Why do we need to leave a part?

The comment describes the rationale. If syncookies are disabled, then
the last quarter of the backlog is reserved for filling with
destinations that were proven to be alive, according to
tcp_peer_is_proven() (which uses RTTs measured in previous
connections). The idea is that if there is a SYN flood, we do not want
to use all of our queue budget on attack traffic but instead want to
reserve some queue space for SYNs from real remote machines that we
have actually contacted in the past.

> The value of sysctl_max_syn_backlog is the maximum length of the queue only if syn cookies are enabled.

Even if syncookies are disabled, sysctl_max_syn_backlog is the maximum
length of the queue.

> This is the first strange place, here is another strange place
>
>         __u32 isn = TCP_SKB_CB(skb)->tcp_tw_isn;
>
>         if ((net->ipv4.sysctl_tcp_syncookies == 2 ||
>              inet_csk_reqsk_queue_is_full(sk)) && !isn) {
>
>         if (!want_cookie && !isn) {
>
> The value of "isn" comes from TCP_SKB_CB(skb)->tcp_tw_isn, then it is judged twice whether its value is indeed 0.
>
> But "tcp_tw_isn" is initialized in the function "tcp_v4_fill_cb"
>
>         TCP_SKB_CB(skb)->tcp_tw_isn = 0;
>
> So it has always been 0, I used printk to test, and the result is always 0.

That field is also set in tcp_timewait_state_process():

  TCP_SKB_CB(skb)->tcp_tw_isn = isn;

So there can be cases where it is not 0.

Hope that helps,
neal

Re: Why not use all the syn queues? in the function "tcp_conn_request", I have some questions.

[Ttttabcd]

Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On 4 September 2018 9:06 PM, Neal Cardwell <ncardwell@google.com> wrote:

> On Tue, Sep 4, 2018 at 1:48 AM Ttttabcd ttttabcd@protonmail.com wrote:
>
> > Hello everyone,recently I am looking at the source code for handling TCP three-way handshake(Linux Kernel version 4.18.5).
> > I found some strange places in the source code for handling syn messages.
> > in the function "tcp_conn_request"
> > This code will be executed when we don't enable the syn cookies.
> >
> >                 if (!net->ipv4.sysctl_tcp_syncookies &&
> >                     (net->ipv4.sysctl_max_syn_backlog - inet_csk_reqsk_queue_len(sk) <
> >                      (net->ipv4.sysctl_max_syn_backlog >> 2)) &&
> >                     !tcp_peer_is_proven(req, dst)) {
> >                         /* Without syncookies last quarter of
> >                          * backlog is filled with destinations,
> >                          * proven to be alive.
> >                          * It means that we continue to communicate
> >                          * to destinations, already remembered
> >                          * to the moment of synflood.
> >                          */
> >                         pr_drop_req(req, ntohs(tcp_hdr(skb)->source),
> >                                     rsk_ops->family);
> >                         goto drop_and_release;
> >                 }
> >
> >
> > But why don't we use all the syn queues?
>
> If tcp_peer_is_proven() returns true then we do allow ourselves to use
> the whole queue.
>
> > Why do we need to leave the size of (net->ipv4.sysctl_max_syn_backlog >> 2) in the queue?
> > Even if the system is attacked by a syn flood, there is no need to leave a part. Why do we need to leave a part?
>
> The comment describes the rationale. If syncookies are disabled, then
> the last quarter of the backlog is reserved for filling with
> destinations that were proven to be alive, according to
> tcp_peer_is_proven() (which uses RTTs measured in previous
> connections). The idea is that if there is a SYN flood, we do not want
> to use all of our queue budget on attack traffic but instead want to
> reserve some queue space for SYNs from real remote machines that we
> have actually contacted in the past.
>
> > The value of sysctl_max_syn_backlog is the maximum length of the queue only if syn cookies are enabled.
>
> Even if syncookies are disabled, sysctl_max_syn_backlog is the maximum
> length of the queue.
>
> > This is the first strange place, here is another strange place
> >
> >         __u32 isn = TCP_SKB_CB(skb)->tcp_tw_isn;
> >
> >         if ((net->ipv4.sysctl_tcp_syncookies == 2 ||
> >              inet_csk_reqsk_queue_is_full(sk)) && !isn) {
> >
> >         if (!want_cookie && !isn) {
> >
> >
> > The value of "isn" comes from TCP_SKB_CB(skb)->tcp_tw_isn, then it is judged twice whether its value is indeed 0.
> > But "tcp_tw_isn" is initialized in the function "tcp_v4_fill_cb"
> >
> >         TCP_SKB_CB(skb)->tcp_tw_isn = 0;
> >
> >
> > So it has always been 0, I used printk to test, and the result is always 0.
>
> That field is also set in tcp_timewait_state_process():
>
> TCP_SKB_CB(skb)->tcp_tw_isn = isn;
>
> So there can be cases where it is not 0.
>
> Hope that helps,
> neal

Thank you very much, I understand

Re: Why not use all the syn queues? in the function "tcp_conn_request", I have some questions.

[Ttttabcd]

Thank you very much for your previous answer, sorry for the inconvenience.

But now I want to ask you one more question.

The question is why we need two variables to control the syn queue?

The first is the "backlog" parameter of the "listen" system call that controls the maximum length limit of the syn queue, it also controls the accept queue.

The second is /proc/sys/net/ipv4/tcp_max_syn_backlog, which also controls the maximum length limit of the syn queue.

So simply changing one of them and wanting to increase the syn queue is not working.

In our last discussion, I understood tcp_max_syn_backlog will retain the last quarter to the IP that has been proven to be alive

But if tcp_max_syn_backlog is very large, the syn queue will be filled as well.

So I don't understand why not just use a variable to control the syn queue.

For example, just use tcp_max_syn_backlog, which is the maximum length limit for the syn queue, and it can also be retained to prove that the IP remains the last quarter.

The backlog parameter of the listen system call only controls the accpet queue.

I feel this is more reasonable. If I don't look at the source code, I really can't guess the backlog parameter actually controls the syn queue.

I always thought that it only controlled the accept queue before I looked at the source code, because the man page is written like this.


Here is the man page's original words.

The behavior of the backlog argument on TCP sockets changed with Linux 2.2. Now it specifies the queue length for completely established sockets waiting to be accepted, instead of the number of incomplete connection requests. The maximum length of the queue for incomplete sockets can be set using /proc/sys/net/ipv4/tcp_max_syn_backlog. When syncookies are enabled there is no logical maximum length and this setting is ignored. See tcp(7) for more information.

Re: Why not use all the syn queues? in the function "tcp_conn_request", I have some questions.

[Neal Cardwell]

On Sat, Sep 8, 2018 at 11:23 AM Ttttabcd <ttttabcd@protonmail.com> wrote:
>
> Thank you very much for your previous answer, sorry for the inconvenience.
>
> But now I want to ask you one more question.
>
> The question is why we need two variables to control the syn queue?
>
> The first is the "backlog" parameter of the "listen" system call that controls the maximum length limit of the syn queue, it also controls the accept queue.

By default, and essentially always in practice (AFAIK), Linux
installations enable syncookies. With syncookies, there is essentially
no limit on the syn queue, or number of incomplete passive connections
(as the man page you quoted notes). So in practice the listen()
parameter usually controls only the accept queue.

> The second is /proc/sys/net/ipv4/tcp_max_syn_backlog, which also controls the maximum length limit of the syn queue.
>
> So simply changing one of them and wanting to increase the syn queue is not working.
>
> In our last discussion, I understood tcp_max_syn_backlog will retain the last quarter to the IP that has been proven to be alive

That discussion pertains to a code path that is relevant if syncookies
are disabled, which is very uncommon (see above).

> But if tcp_max_syn_backlog is very large, the syn queue will be filled as well.
>
> So I don't understand why not just use a variable to control the syn queue.
>
> For example, just use tcp_max_syn_backlog, which is the maximum length limit for the syn queue, and it can also be retained to prove that the IP remains the last quarter.
>
> The backlog parameter of the listen system call only controls the accpet queue.
>
> I feel this is more reasonable. If I don't look at the source code, I really can't guess the backlog parameter actually controls the syn queue.
>
> I always thought that it only controlled the accept queue before I looked at the source code, because the man page is written like this.

Keep in mind that the semantics of the listen() argument and the
/proc/sys/net/ipv4/tcp_max_syn_backlog sysctl knob, as described in
the man page, are part of the Linux kernel's user-visible API. So, in
essence, they cannot be changed. Changing the semantics of system
calls and sysctl knobs breaks applications and system configuration
scripts. :-)

neal

Re: Why not use all the syn queues? in the function "tcp_conn_request", I have some questions.

[Ttttabcd]

Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Sunday, 9 September 2018 02:24, Neal Cardwell <ncardwell@google.com> wrote:


> By default, and essentially always in practice (AFAIK), Linux
> installations enable syncookies. With syncookies, there is essentially
> no limit on the syn queue, or number of incomplete passive connections
> (as the man page you quoted notes). So in practice the listen()
> parameter usually controls only the accept queue.
>

>
> That discussion pertains to a code path that is relevant if syncookies
> are disabled, which is very uncommon (see above).
>

Yes, when I tested, I disabled syncookies. I want to know how the kernel will handle syn attacks if syncookies are disabled.


> Keep in mind that the semantics of the listen() argument and the
> /proc/sys/net/ipv4/tcp_max_syn_backlog sysctl knob, as described in
> the man page, are part of the Linux kernel's user-visible API. So, in
> essence, they cannot be changed. Changing the semantics of system
> calls and sysctl knobs breaks applications and system configuration
> scripts. :-)

So, as you said

Is there a historical issue with two variables controlling the syn queue?