From: Kuniyuki Iwashima <kuniyu@amazon.co.jp>
To: <edumazet@google.com>
Cc: <ast@kernel.org>, <benh@amazon.com>, <bpf@vger.kernel.org>,
<daniel@iogearbox.net>, <davem@davemloft.net>,
<eric.dumazet@gmail.com>, <kafai@fb.com>, <kuba@kernel.org>,
<kuni1840@gmail.com>, <kuniyu@amazon.co.jp>,
<linux-kernel@vger.kernel.org>, <netdev@vger.kernel.org>
Subject: Re: [PATCH v1 bpf-next 03/11] tcp: Migrate TCP_ESTABLISHED/TCP_SYN_RECV sockets in accept queues.
Date: Fri, 4 Dec 2020 00:41:59 +0900 [thread overview]
Message-ID: <20201203154159.76404-1-kuniyu@amazon.co.jp> (raw)
In-Reply-To: <CANn89i+MtOWryWzeDXnG-waOnqY8SxVxb_Q2Y4C=FdPGsKXivA@mail.gmail.com>
From: Eric Dumazet <edumazet@google.com>
Date: Thu, 3 Dec 2020 15:31:53 +0100
> On Thu, Dec 3, 2020 at 3:14 PM Kuniyuki Iwashima <kuniyu@amazon.co.jp> wrote:
> >
> > From: Eric Dumazet <eric.dumazet@gmail.com>
> > Date: Tue, 1 Dec 2020 16:25:51 +0100
> > > On 12/1/20 3:44 PM, Kuniyuki Iwashima wrote:
> > > > This patch lets reuseport_detach_sock() return a pointer of struct sock,
> > > > which is used only by inet_unhash(). If it is not NULL,
> > > > inet_csk_reqsk_queue_migrate() migrates TCP_ESTABLISHED/TCP_SYN_RECV
> > > > sockets from the closing listener to the selected one.
> > > >
> > > > Listening sockets hold incoming connections as a linked list of struct
> > > > request_sock in the accept queue, and each request has reference to a full
> > > > socket and its listener. In inet_csk_reqsk_queue_migrate(), we only unlink
> > > > the requests from the closing listener's queue and relink them to the head
> > > > of the new listener's queue. We do not process each request and its
> > > > reference to the listener, so the migration completes in O(1) time
> > > > complexity. However, in the case of TCP_SYN_RECV sockets, we take special
> > > > care in the next commit.
> > > >
> > > > By default, the kernel selects a new listener randomly. In order to pick
> > > > out a different socket every time, we select the last element of socks[] as
> > > > the new listener. This behaviour is based on how the kernel moves sockets
> > > > in socks[]. (See also [1])
> > > >
> > > > Basically, in order to redistribute sockets evenly, we have to use an eBPF
> > > > program called in the later commit, but as the side effect of such default
> > > > selection, the kernel can redistribute old requests evenly to new listeners
> > > > for a specific case where the application replaces listeners by
> > > > generations.
> > > >
> > > > For example, we call listen() for four sockets (A, B, C, D), and close the
> > > > first two by turns. The sockets move in socks[] like below.
> > > >
> > > > socks[0] : A <-. socks[0] : D socks[0] : D
> > > > socks[1] : B | => socks[1] : B <-. => socks[1] : C
> > > > socks[2] : C | socks[2] : C --'
> > > > socks[3] : D --'
> > > >
> > > > Then, if C and D have newer settings than A and B, and each socket has a
> > > > request (a, b, c, d) in their accept queue, we can redistribute old
> > > > requests evenly to new listeners.
> > > >
> > > > socks[0] : A (a) <-. socks[0] : D (a + d) socks[0] : D (a + d)
> > > > socks[1] : B (b) | => socks[1] : B (b) <-. => socks[1] : C (b + c)
> > > > socks[2] : C (c) | socks[2] : C (c) --'
> > > > socks[3] : D (d) --'
> > > >
> > > > Here, (A, D) or (B, C) can have different application settings, but they
> > > > MUST have the same settings at the socket API level; otherwise, unexpected
> > > > error may happen. For instance, if only the new listeners have
> > > > TCP_SAVE_SYN, old requests do not have SYN data, so the application will
> > > > face inconsistency and cause an error.
> > > >
> > > > Therefore, if there are different kinds of sockets, we must attach an eBPF
> > > > program described in later commits.
> > > >
> > > > Link: https://lore.kernel.org/netdev/CAEfhGiyG8Y_amDZ2C8dQoQqjZJMHjTY76b=KBkTKcBtA=dhdGQ@mail.gmail.com/
> > > > Reviewed-by: Benjamin Herrenschmidt <benh@amazon.com>
> > > > Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.co.jp>
> > > > ---
> > > > include/net/inet_connection_sock.h | 1 +
> > > > include/net/sock_reuseport.h | 2 +-
> > > > net/core/sock_reuseport.c | 10 +++++++++-
> > > > net/ipv4/inet_connection_sock.c | 30 ++++++++++++++++++++++++++++++
> > > > net/ipv4/inet_hashtables.c | 9 +++++++--
> > > > 5 files changed, 48 insertions(+), 4 deletions(-)
> > > >
> > > > diff --git a/include/net/inet_connection_sock.h b/include/net/inet_connection_sock.h
> > > > index 7338b3865a2a..2ea2d743f8fc 100644
> > > > --- a/include/net/inet_connection_sock.h
> > > > +++ b/include/net/inet_connection_sock.h
> > > > @@ -260,6 +260,7 @@ struct dst_entry *inet_csk_route_child_sock(const struct sock *sk,
> > > > struct sock *inet_csk_reqsk_queue_add(struct sock *sk,
> > > > struct request_sock *req,
> > > > struct sock *child);
> > > > +void inet_csk_reqsk_queue_migrate(struct sock *sk, struct sock *nsk);
> > > > void inet_csk_reqsk_queue_hash_add(struct sock *sk, struct request_sock *req,
> > > > unsigned long timeout);
> > > > struct sock *inet_csk_complete_hashdance(struct sock *sk, struct sock *child,
> > > > diff --git a/include/net/sock_reuseport.h b/include/net/sock_reuseport.h
> > > > index 0e558ca7afbf..09a1b1539d4c 100644
> > > > --- a/include/net/sock_reuseport.h
> > > > +++ b/include/net/sock_reuseport.h
> > > > @@ -31,7 +31,7 @@ struct sock_reuseport {
> > > > extern int reuseport_alloc(struct sock *sk, bool bind_inany);
> > > > extern int reuseport_add_sock(struct sock *sk, struct sock *sk2,
> > > > bool bind_inany);
> > > > -extern void reuseport_detach_sock(struct sock *sk);
> > > > +extern struct sock *reuseport_detach_sock(struct sock *sk);
> > > > extern struct sock *reuseport_select_sock(struct sock *sk,
> > > > u32 hash,
> > > > struct sk_buff *skb,
> > > > diff --git a/net/core/sock_reuseport.c b/net/core/sock_reuseport.c
> > > > index fd133516ac0e..60d7c1f28809 100644
> > > > --- a/net/core/sock_reuseport.c
> > > > +++ b/net/core/sock_reuseport.c
> > > > @@ -216,9 +216,11 @@ int reuseport_add_sock(struct sock *sk, struct sock *sk2, bool bind_inany)
> > > > }
> > > > EXPORT_SYMBOL(reuseport_add_sock);
> > > >
> > > > -void reuseport_detach_sock(struct sock *sk)
> > > > +struct sock *reuseport_detach_sock(struct sock *sk)
> > > > {
> > > > struct sock_reuseport *reuse;
> > > > + struct bpf_prog *prog;
> > > > + struct sock *nsk = NULL;
> > > > int i;
> > > >
> > > > spin_lock_bh(&reuseport_lock);
> > > > @@ -242,8 +244,12 @@ void reuseport_detach_sock(struct sock *sk)
> > > >
> > > > reuse->num_socks--;
> > > > reuse->socks[i] = reuse->socks[reuse->num_socks];
> > > > + prog = rcu_dereference(reuse->prog);
> > > >
> > > > if (sk->sk_protocol == IPPROTO_TCP) {
> > > > + if (reuse->num_socks && !prog)
> > > > + nsk = i == reuse->num_socks ? reuse->socks[i - 1] : reuse->socks[i];
> > > > +
> > > > reuse->num_closed_socks++;
> > > > reuse->socks[reuse->max_socks - reuse->num_closed_socks] = sk;
> > > > } else {
> > > > @@ -264,6 +270,8 @@ void reuseport_detach_sock(struct sock *sk)
> > > > call_rcu(&reuse->rcu, reuseport_free_rcu);
> > > > out:
> > > > spin_unlock_bh(&reuseport_lock);
> > > > +
> > > > + return nsk;
> > > > }
> > > > EXPORT_SYMBOL(reuseport_detach_sock);
> > > >
> > > > diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c
> > > > index 1451aa9712b0..b27241ea96bd 100644
> > > > --- a/net/ipv4/inet_connection_sock.c
> > > > +++ b/net/ipv4/inet_connection_sock.c
> > > > @@ -992,6 +992,36 @@ struct sock *inet_csk_reqsk_queue_add(struct sock *sk,
> > > > }
> > > > EXPORT_SYMBOL(inet_csk_reqsk_queue_add);
> > > >
> > > > +void inet_csk_reqsk_queue_migrate(struct sock *sk, struct sock *nsk)
> > > > +{
> > > > + struct request_sock_queue *old_accept_queue, *new_accept_queue;
> > > > +
> > > > + old_accept_queue = &inet_csk(sk)->icsk_accept_queue;
> > > > + new_accept_queue = &inet_csk(nsk)->icsk_accept_queue;
> > > > +
> > > > + spin_lock(&old_accept_queue->rskq_lock);
> > > > + spin_lock(&new_accept_queue->rskq_lock);
> > >
> > > Are you sure lockdep is happy with this ?
> > >
> > > I would guess it should complain, because :
> > >
> > > lock(A);
> > > lock(B);
> > > ...
> > > unlock(B);
> > > unlock(A);
> > >
> > > will fail when the opposite action happens eventually
> > >
> > > lock(B);
> > > lock(A);
> > > ...
> > > unlock(A);
> > > unlock(B);
> >
> > I enabled lockdep and did not see warnings of lockdep.
> >
> > Also, the inversion deadlock does not happen in this case.
> > In reuseport_detach_sock(), sk is moved backward in socks[] and poped out
> > from the eBPF map, so the old listener will not be selected as the new
> > listener.
>
> Until the socket is closed, reallocated and used again. LOCKDEP has no
> idea about soreuseport logic.
>
> If you run your tests long enough, lockdep should complain at some point.
>
> git grep -n double_lock
Thank you, I will change the code like double_lock().
And I will try to continue testing lockdep without this change for
curiosity!
next prev parent reply other threads:[~2020-12-03 15:47 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-01 14:44 [PATCH v1 bpf-next 00/11] Socket migration for SO_REUSEPORT Kuniyuki Iwashima
2020-12-01 14:44 ` [PATCH v1 bpf-next 01/11] tcp: Keep TCP_CLOSE sockets in the reuseport group Kuniyuki Iwashima
2020-12-05 1:31 ` Martin KaFai Lau
2020-12-06 4:38 ` Kuniyuki Iwashima
2020-12-01 14:44 ` [PATCH v1 bpf-next 02/11] bpf: Define migration types for SO_REUSEPORT Kuniyuki Iwashima
2020-12-01 14:44 ` [PATCH v1 bpf-next 03/11] tcp: Migrate TCP_ESTABLISHED/TCP_SYN_RECV sockets in accept queues Kuniyuki Iwashima
2020-12-01 15:25 ` Eric Dumazet
2020-12-03 14:14 ` Kuniyuki Iwashima
2020-12-03 14:31 ` Eric Dumazet
2020-12-03 15:41 ` Kuniyuki Iwashima [this message]
2020-12-07 20:33 ` Martin KaFai Lau
2020-12-08 6:31 ` Kuniyuki Iwashima
2020-12-08 7:34 ` Martin KaFai Lau
2020-12-08 8:17 ` Kuniyuki Iwashima
2020-12-09 3:09 ` Martin KaFai Lau
2020-12-09 8:05 ` Kuniyuki Iwashima
2020-12-09 16:57 ` Kuniyuki Iwashima
2020-12-10 1:53 ` Martin KaFai Lau
2020-12-10 5:58 ` Kuniyuki Iwashima
2020-12-10 19:33 ` Martin KaFai Lau
2020-12-14 17:16 ` Kuniyuki Iwashima
2020-12-05 1:42 ` Martin KaFai Lau
2020-12-06 4:41 ` Kuniyuki Iwashima
[not found] ` <20201205160307.91179-1-kuniyu@amazon.co.jp>
2020-12-07 20:14 ` Martin KaFai Lau
2020-12-08 6:27 ` Kuniyuki Iwashima
2020-12-08 8:13 ` Martin KaFai Lau
2020-12-08 9:02 ` Kuniyuki Iwashima
2020-12-08 6:54 ` Martin KaFai Lau
2020-12-08 7:42 ` Kuniyuki Iwashima
2020-12-01 14:44 ` [PATCH v1 bpf-next 04/11] tcp: Migrate TFO requests causing RST during TCP_SYN_RECV Kuniyuki Iwashima
2020-12-01 15:30 ` Eric Dumazet
2020-12-01 14:44 ` [PATCH v1 bpf-next 05/11] tcp: Migrate TCP_NEW_SYN_RECV requests Kuniyuki Iwashima
2020-12-01 15:13 ` Eric Dumazet
2020-12-03 14:12 ` Kuniyuki Iwashima
2020-12-10 0:07 ` Martin KaFai Lau
2020-12-10 5:15 ` Kuniyuki Iwashima
2020-12-10 18:49 ` Martin KaFai Lau
2020-12-14 17:03 ` Kuniyuki Iwashima
2020-12-15 2:58 ` Martin KaFai Lau
2020-12-16 16:41 ` Kuniyuki Iwashima
2020-12-16 22:24 ` Martin KaFai Lau
2020-12-01 14:44 ` [PATCH v1 bpf-next 06/11] bpf: Introduce two attach types for BPF_PROG_TYPE_SK_REUSEPORT Kuniyuki Iwashima
2020-12-02 2:04 ` Andrii Nakryiko
2020-12-02 19:19 ` Martin KaFai Lau
2020-12-03 4:24 ` Martin KaFai Lau
2020-12-03 14:16 ` Kuniyuki Iwashima
2020-12-04 5:56 ` Martin KaFai Lau
2020-12-06 4:32 ` Kuniyuki Iwashima
2020-12-01 14:44 ` [PATCH v1 bpf-next 07/11] libbpf: Set expected_attach_type " Kuniyuki Iwashima
2020-12-01 14:44 ` [PATCH v1 bpf-next 08/11] bpf: Add migration to sk_reuseport_(kern|md) Kuniyuki Iwashima
2020-12-01 14:44 ` [PATCH v1 bpf-next 09/11] bpf: Support bpf_get_socket_cookie_sock() for BPF_PROG_TYPE_SK_REUSEPORT Kuniyuki Iwashima
2020-12-04 19:58 ` Martin KaFai Lau
2020-12-06 4:36 ` Kuniyuki Iwashima
2020-12-01 14:44 ` [PATCH v1 bpf-next 10/11] bpf: Call bpf_run_sk_reuseport() for socket migration Kuniyuki Iwashima
2020-12-01 14:44 ` [PATCH v1 bpf-next 11/11] bpf: Test BPF_SK_REUSEPORT_SELECT_OR_MIGRATE Kuniyuki Iwashima
2020-12-05 1:50 ` Martin KaFai Lau
2020-12-06 4:43 ` Kuniyuki Iwashima
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201203154159.76404-1-kuniyu@amazon.co.jp \
--to=kuniyu@amazon.co.jp \
--cc=ast@kernel.org \
--cc=benh@amazon.com \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=eric.dumazet@gmail.com \
--cc=kafai@fb.com \
--cc=kuba@kernel.org \
--cc=kuni1840@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).