* [PATCH v2 net-next 0/2] tcp: final (?) round of mem pressure fixes
@ 2022-06-14 17:17 Eric Dumazet
2022-06-14 17:17 ` [PATCH v2 net-next 1/2] tcp: fix over estimation in sk_forced_mem_schedule() Eric Dumazet
` (2 more replies)
0 siblings, 3 replies; 15+ messages in thread
From: Eric Dumazet @ 2022-06-14 17:17 UTC (permalink / raw)
To: David S . Miller, Jakub Kicinski, Paolo Abeni
Cc: netdev, Soheil Hassas Yeganeh, Wei Wang, Shakeel Butt,
Neal Cardwell, Eric Dumazet, Eric Dumazet
From: Eric Dumazet <edumazet@google.com>
While working on prior patch series (e10b02ee5b6c "Merge branch
'net-reduce-tcp_memory_allocated-inflation'"), I found that we
could still have frozen TCP flows under memory pressure.
I thought we had solved this in 2015, but the fix was not complete.
v2: deal with zerocopy tx paths.
Eric Dumazet (2):
tcp: fix over estimation in sk_forced_mem_schedule()
tcp: fix possible freeze in tx path under memory pressure
net/ipv4/tcp.c | 33 +++++++++++++++++++++++++++++----
net/ipv4/tcp_output.c | 7 ++++---
2 files changed, 33 insertions(+), 7 deletions(-)
--
2.36.1.476.g0c4daa206d-goog
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v2 net-next 1/2] tcp: fix over estimation in sk_forced_mem_schedule()
2022-06-14 17:17 [PATCH v2 net-next 0/2] tcp: final (?) round of mem pressure fixes Eric Dumazet
@ 2022-06-14 17:17 ` Eric Dumazet
2022-06-14 17:40 ` Soheil Hassas Yeganeh
2022-06-14 17:40 ` Shakeel Butt
2022-06-14 17:17 ` [PATCH v2 net-next 2/2] tcp: fix possible freeze in tx path under memory pressure Eric Dumazet
2022-06-15 11:56 ` [PATCH v2 net-next 0/2] tcp: final (?) round of mem pressure fixes David Miller
2 siblings, 2 replies; 15+ messages in thread
From: Eric Dumazet @ 2022-06-14 17:17 UTC (permalink / raw)
To: David S . Miller, Jakub Kicinski, Paolo Abeni
Cc: netdev, Soheil Hassas Yeganeh, Wei Wang, Shakeel Butt,
Neal Cardwell, Eric Dumazet, Eric Dumazet
From: Eric Dumazet <edumazet@google.com>
sk_forced_mem_schedule() has a bug similar to ones fixed
in commit 7c80b038d23e ("net: fix sk_wmem_schedule() and
sk_rmem_schedule() errors")
While this bug has little chance to trigger in old kernels,
we need to fix it before the following patch.
Fixes: d83769a580f1 ("tcp: fix possible deadlock in tcp_send_fin()")
Signed-off-by: Eric Dumazet <edumazet@google.com>
---
net/ipv4/tcp_output.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
index 8ab98e1aca6797a51eaaf8886680d2001a616948..18c913a2347a984ae8cf2793bb8991e59e5e94ab 100644
--- a/net/ipv4/tcp_output.c
+++ b/net/ipv4/tcp_output.c
@@ -3362,11 +3362,12 @@ void tcp_xmit_retransmit_queue(struct sock *sk)
*/
void sk_forced_mem_schedule(struct sock *sk, int size)
{
- int amt;
+ int delta, amt;
- if (size <= sk->sk_forward_alloc)
+ delta = size - sk->sk_forward_alloc;
+ if (delta <= 0)
return;
- amt = sk_mem_pages(size);
+ amt = sk_mem_pages(delta);
sk->sk_forward_alloc += amt << PAGE_SHIFT;
sk_memory_allocated_add(sk, amt);
--
2.36.1.476.g0c4daa206d-goog
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH v2 net-next 2/2] tcp: fix possible freeze in tx path under memory pressure
2022-06-14 17:17 [PATCH v2 net-next 0/2] tcp: final (?) round of mem pressure fixes Eric Dumazet
2022-06-14 17:17 ` [PATCH v2 net-next 1/2] tcp: fix over estimation in sk_forced_mem_schedule() Eric Dumazet
@ 2022-06-14 17:17 ` Eric Dumazet
2022-06-14 17:42 ` Soheil Hassas Yeganeh
` (2 more replies)
2022-06-15 11:56 ` [PATCH v2 net-next 0/2] tcp: final (?) round of mem pressure fixes David Miller
2 siblings, 3 replies; 15+ messages in thread
From: Eric Dumazet @ 2022-06-14 17:17 UTC (permalink / raw)
To: David S . Miller, Jakub Kicinski, Paolo Abeni
Cc: netdev, Soheil Hassas Yeganeh, Wei Wang, Shakeel Butt,
Neal Cardwell, Eric Dumazet, Eric Dumazet
From: Eric Dumazet <edumazet@google.com>
Blamed commit only dealt with applications issuing small writes.
Issue here is that we allow to force memory schedule for the sk_buff
allocation, but we have no guarantee that sendmsg() is able to
copy some payload in it.
In this patch, I make sure the socket can use up to tcp_wmem[0] bytes.
For example, if we consider tcp_wmem[0] = 4096 (default on x86),
and initial skb->truesize being 1280, tcp_sendmsg() is able to
copy up to 2816 bytes under memory pressure.
Before this patch a sendmsg() sending more than 2816 bytes
would either block forever (if persistent memory pressure),
or return -EAGAIN.
For bigger MTU networks, it is advised to increase tcp_wmem[0]
to avoid sending too small packets.
v2: deal with zero copy paths.
Fixes: 8e4d980ac215 ("tcp: fix behavior for epoll edge trigger")
Signed-off-by: Eric Dumazet <edumazet@google.com>
---
net/ipv4/tcp.c | 33 +++++++++++++++++++++++++++++----
1 file changed, 29 insertions(+), 4 deletions(-)
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 14ebb4ec4a51f3c55501aa53423ce897599e8637..56083c2497f0b695c660256aa43f8a743d481697 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -951,6 +951,23 @@ static int tcp_downgrade_zcopy_pure(struct sock *sk, struct sk_buff *skb)
return 0;
}
+static int tcp_wmem_schedule(struct sock *sk, int copy)
+{
+ int left;
+
+ if (likely(sk_wmem_schedule(sk, copy)))
+ return copy;
+
+ /* We could be in trouble if we have nothing queued.
+ * Use whatever is left in sk->sk_forward_alloc and tcp_wmem[0]
+ * to guarantee some progress.
+ */
+ left = sock_net(sk)->ipv4.sysctl_tcp_wmem[0] - sk->sk_wmem_queued;
+ if (left > 0)
+ sk_forced_mem_schedule(sk, min(left, copy));
+ return min(copy, sk->sk_forward_alloc);
+}
+
static struct sk_buff *tcp_build_frag(struct sock *sk, int size_goal, int flags,
struct page *page, int offset, size_t *size)
{
@@ -986,7 +1003,11 @@ static struct sk_buff *tcp_build_frag(struct sock *sk, int size_goal, int flags,
tcp_mark_push(tp, skb);
goto new_segment;
}
- if (tcp_downgrade_zcopy_pure(sk, skb) || !sk_wmem_schedule(sk, copy))
+ if (tcp_downgrade_zcopy_pure(sk, skb))
+ return NULL;
+
+ copy = tcp_wmem_schedule(sk, copy);
+ if (!copy)
return NULL;
if (can_coalesce) {
@@ -1334,8 +1355,11 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size)
copy = min_t(int, copy, pfrag->size - pfrag->offset);
- if (tcp_downgrade_zcopy_pure(sk, skb) ||
- !sk_wmem_schedule(sk, copy))
+ if (tcp_downgrade_zcopy_pure(sk, skb))
+ goto wait_for_space;
+
+ copy = tcp_wmem_schedule(sk, copy);
+ if (!copy)
goto wait_for_space;
err = skb_copy_to_page_nocache(sk, &msg->msg_iter, skb,
@@ -1362,7 +1386,8 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size)
skb_shinfo(skb)->flags |= SKBFL_PURE_ZEROCOPY;
if (!skb_zcopy_pure(skb)) {
- if (!sk_wmem_schedule(sk, copy))
+ copy = tcp_wmem_schedule(sk, copy);
+ if (!copy)
goto wait_for_space;
}
--
2.36.1.476.g0c4daa206d-goog
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v2 net-next 1/2] tcp: fix over estimation in sk_forced_mem_schedule()
2022-06-14 17:17 ` [PATCH v2 net-next 1/2] tcp: fix over estimation in sk_forced_mem_schedule() Eric Dumazet
@ 2022-06-14 17:40 ` Soheil Hassas Yeganeh
2022-06-14 18:39 ` Wei Wang
2022-06-14 17:40 ` Shakeel Butt
1 sibling, 1 reply; 15+ messages in thread
From: Soheil Hassas Yeganeh @ 2022-06-14 17:40 UTC (permalink / raw)
To: Eric Dumazet
Cc: David S . Miller, Jakub Kicinski, Paolo Abeni, netdev, Wei Wang,
Shakeel Butt, Neal Cardwell, Eric Dumazet
On Tue, Jun 14, 2022 at 1:17 PM Eric Dumazet <eric.dumazet@gmail.com> wrote:
>
> From: Eric Dumazet <edumazet@google.com>
>
> sk_forced_mem_schedule() has a bug similar to ones fixed
> in commit 7c80b038d23e ("net: fix sk_wmem_schedule() and
> sk_rmem_schedule() errors")
>
> While this bug has little chance to trigger in old kernels,
> we need to fix it before the following patch.
>
> Fixes: d83769a580f1 ("tcp: fix possible deadlock in tcp_send_fin()")
> Signed-off-by: Eric Dumazet <edumazet@google.com>
Acked-by: Soheil Hassas Yeganeh <soheil@google.com>
> ---
> net/ipv4/tcp_output.c | 7 ++++---
> 1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
> index 8ab98e1aca6797a51eaaf8886680d2001a616948..18c913a2347a984ae8cf2793bb8991e59e5e94ab 100644
> --- a/net/ipv4/tcp_output.c
> +++ b/net/ipv4/tcp_output.c
> @@ -3362,11 +3362,12 @@ void tcp_xmit_retransmit_queue(struct sock *sk)
> */
> void sk_forced_mem_schedule(struct sock *sk, int size)
> {
> - int amt;
> + int delta, amt;
>
> - if (size <= sk->sk_forward_alloc)
> + delta = size - sk->sk_forward_alloc;
> + if (delta <= 0)
> return;
> - amt = sk_mem_pages(size);
> + amt = sk_mem_pages(delta);
> sk->sk_forward_alloc += amt << PAGE_SHIFT;
> sk_memory_allocated_add(sk, amt);
>
> --
> 2.36.1.476.g0c4daa206d-goog
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 net-next 1/2] tcp: fix over estimation in sk_forced_mem_schedule()
2022-06-14 17:17 ` [PATCH v2 net-next 1/2] tcp: fix over estimation in sk_forced_mem_schedule() Eric Dumazet
2022-06-14 17:40 ` Soheil Hassas Yeganeh
@ 2022-06-14 17:40 ` Shakeel Butt
1 sibling, 0 replies; 15+ messages in thread
From: Shakeel Butt @ 2022-06-14 17:40 UTC (permalink / raw)
To: Eric Dumazet
Cc: David S . Miller, Jakub Kicinski, Paolo Abeni, netdev,
Soheil Hassas Yeganeh, Wei Wang, Neal Cardwell, Eric Dumazet
On Tue, Jun 14, 2022 at 10:17:33AM -0700, Eric Dumazet wrote:
> From: Eric Dumazet <edumazet@google.com>
>
> sk_forced_mem_schedule() has a bug similar to ones fixed
> in commit 7c80b038d23e ("net: fix sk_wmem_schedule() and
> sk_rmem_schedule() errors")
>
> While this bug has little chance to trigger in old kernels,
> we need to fix it before the following patch.
>
> Fixes: d83769a580f1 ("tcp: fix possible deadlock in tcp_send_fin()")
> Signed-off-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Shakeel Butt <shakeelb@google.com>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 net-next 2/2] tcp: fix possible freeze in tx path under memory pressure
2022-06-14 17:17 ` [PATCH v2 net-next 2/2] tcp: fix possible freeze in tx path under memory pressure Eric Dumazet
@ 2022-06-14 17:42 ` Soheil Hassas Yeganeh
2022-06-14 18:41 ` Wei Wang
2022-06-14 20:31 ` Shakeel Butt
2022-06-17 10:08 ` Jason A. Donenfeld
2 siblings, 1 reply; 15+ messages in thread
From: Soheil Hassas Yeganeh @ 2022-06-14 17:42 UTC (permalink / raw)
To: Eric Dumazet
Cc: David S . Miller, Jakub Kicinski, Paolo Abeni, netdev, Wei Wang,
Shakeel Butt, Neal Cardwell, Eric Dumazet
On Tue, Jun 14, 2022 at 1:17 PM Eric Dumazet <eric.dumazet@gmail.com> wrote:
>
> From: Eric Dumazet <edumazet@google.com>
>
> Blamed commit only dealt with applications issuing small writes.
>
> Issue here is that we allow to force memory schedule for the sk_buff
> allocation, but we have no guarantee that sendmsg() is able to
> copy some payload in it.
>
> In this patch, I make sure the socket can use up to tcp_wmem[0] bytes.
>
> For example, if we consider tcp_wmem[0] = 4096 (default on x86),
> and initial skb->truesize being 1280, tcp_sendmsg() is able to
> copy up to 2816 bytes under memory pressure.
>
> Before this patch a sendmsg() sending more than 2816 bytes
> would either block forever (if persistent memory pressure),
> or return -EAGAIN.
>
> For bigger MTU networks, it is advised to increase tcp_wmem[0]
> to avoid sending too small packets.
>
> v2: deal with zero copy paths.
>
> Fixes: 8e4d980ac215 ("tcp: fix behavior for epoll edge trigger")
> Signed-off-by: Eric Dumazet <edumazet@google.com>
Acked-by: Soheil Hassas Yeganeh <soheil@google.com>
Very nice find! Thank you!
> ---
> net/ipv4/tcp.c | 33 +++++++++++++++++++++++++++++----
> 1 file changed, 29 insertions(+), 4 deletions(-)
>
> diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
> index 14ebb4ec4a51f3c55501aa53423ce897599e8637..56083c2497f0b695c660256aa43f8a743d481697 100644
> --- a/net/ipv4/tcp.c
> +++ b/net/ipv4/tcp.c
> @@ -951,6 +951,23 @@ static int tcp_downgrade_zcopy_pure(struct sock *sk, struct sk_buff *skb)
> return 0;
> }
>
> +static int tcp_wmem_schedule(struct sock *sk, int copy)
> +{
> + int left;
> +
> + if (likely(sk_wmem_schedule(sk, copy)))
> + return copy;
> +
> + /* We could be in trouble if we have nothing queued.
> + * Use whatever is left in sk->sk_forward_alloc and tcp_wmem[0]
> + * to guarantee some progress.
> + */
> + left = sock_net(sk)->ipv4.sysctl_tcp_wmem[0] - sk->sk_wmem_queued;
> + if (left > 0)
> + sk_forced_mem_schedule(sk, min(left, copy));
> + return min(copy, sk->sk_forward_alloc);
> +}
> +
> static struct sk_buff *tcp_build_frag(struct sock *sk, int size_goal, int flags,
> struct page *page, int offset, size_t *size)
> {
> @@ -986,7 +1003,11 @@ static struct sk_buff *tcp_build_frag(struct sock *sk, int size_goal, int flags,
> tcp_mark_push(tp, skb);
> goto new_segment;
> }
> - if (tcp_downgrade_zcopy_pure(sk, skb) || !sk_wmem_schedule(sk, copy))
> + if (tcp_downgrade_zcopy_pure(sk, skb))
> + return NULL;
> +
> + copy = tcp_wmem_schedule(sk, copy);
> + if (!copy)
> return NULL;
>
> if (can_coalesce) {
> @@ -1334,8 +1355,11 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size)
>
> copy = min_t(int, copy, pfrag->size - pfrag->offset);
>
> - if (tcp_downgrade_zcopy_pure(sk, skb) ||
> - !sk_wmem_schedule(sk, copy))
> + if (tcp_downgrade_zcopy_pure(sk, skb))
> + goto wait_for_space;
> +
> + copy = tcp_wmem_schedule(sk, copy);
> + if (!copy)
> goto wait_for_space;
>
> err = skb_copy_to_page_nocache(sk, &msg->msg_iter, skb,
> @@ -1362,7 +1386,8 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size)
> skb_shinfo(skb)->flags |= SKBFL_PURE_ZEROCOPY;
>
> if (!skb_zcopy_pure(skb)) {
> - if (!sk_wmem_schedule(sk, copy))
> + copy = tcp_wmem_schedule(sk, copy);
> + if (!copy)
> goto wait_for_space;
> }
>
> --
> 2.36.1.476.g0c4daa206d-goog
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 net-next 1/2] tcp: fix over estimation in sk_forced_mem_schedule()
2022-06-14 17:40 ` Soheil Hassas Yeganeh
@ 2022-06-14 18:39 ` Wei Wang
0 siblings, 0 replies; 15+ messages in thread
From: Wei Wang @ 2022-06-14 18:39 UTC (permalink / raw)
To: Soheil Hassas Yeganeh
Cc: Eric Dumazet, David S . Miller, Jakub Kicinski, Paolo Abeni,
netdev, Shakeel Butt, Neal Cardwell, Eric Dumazet
On Tue, Jun 14, 2022 at 10:41 AM Soheil Hassas Yeganeh
<soheil@google.com> wrote:
>
> On Tue, Jun 14, 2022 at 1:17 PM Eric Dumazet <eric.dumazet@gmail.com> wrote:
> >
> > From: Eric Dumazet <edumazet@google.com>
> >
> > sk_forced_mem_schedule() has a bug similar to ones fixed
> > in commit 7c80b038d23e ("net: fix sk_wmem_schedule() and
> > sk_rmem_schedule() errors")
> >
> > While this bug has little chance to trigger in old kernels,
> > we need to fix it before the following patch.
> >
> > Fixes: d83769a580f1 ("tcp: fix possible deadlock in tcp_send_fin()")
> > Signed-off-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Wei Wang <weiwan@google.com>
>
> Acked-by: Soheil Hassas Yeganeh <soheil@google.com>
>
> > ---
> > net/ipv4/tcp_output.c | 7 ++++---
> > 1 file changed, 4 insertions(+), 3 deletions(-)
> >
> > diff --git a/net/ipv4/tcp_output.c b/net/ipv4/tcp_output.c
> > index 8ab98e1aca6797a51eaaf8886680d2001a616948..18c913a2347a984ae8cf2793bb8991e59e5e94ab 100644
> > --- a/net/ipv4/tcp_output.c
> > +++ b/net/ipv4/tcp_output.c
> > @@ -3362,11 +3362,12 @@ void tcp_xmit_retransmit_queue(struct sock *sk)
> > */
> > void sk_forced_mem_schedule(struct sock *sk, int size)
> > {
> > - int amt;
> > + int delta, amt;
> >
> > - if (size <= sk->sk_forward_alloc)
> > + delta = size - sk->sk_forward_alloc;
> > + if (delta <= 0)
> > return;
> > - amt = sk_mem_pages(size);
> > + amt = sk_mem_pages(delta);
> > sk->sk_forward_alloc += amt << PAGE_SHIFT;
> > sk_memory_allocated_add(sk, amt);
> >
> > --
> > 2.36.1.476.g0c4daa206d-goog
> >
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 net-next 2/2] tcp: fix possible freeze in tx path under memory pressure
2022-06-14 17:42 ` Soheil Hassas Yeganeh
@ 2022-06-14 18:41 ` Wei Wang
[not found] ` <CANn89iJYwSoL-XvkcUK1am2wYQjToMnNRG-9pD9qY8+L=GyqaA@mail.gmail.com>
0 siblings, 1 reply; 15+ messages in thread
From: Wei Wang @ 2022-06-14 18:41 UTC (permalink / raw)
To: Soheil Hassas Yeganeh
Cc: Eric Dumazet, David S . Miller, Jakub Kicinski, Paolo Abeni,
netdev, Shakeel Butt, Neal Cardwell, Eric Dumazet
On Tue, Jun 14, 2022 at 10:42 AM Soheil Hassas Yeganeh
<soheil@google.com> wrote:
>
> On Tue, Jun 14, 2022 at 1:17 PM Eric Dumazet <eric.dumazet@gmail.com> wrote:
> >
> > From: Eric Dumazet <edumazet@google.com>
> >
> > Blamed commit only dealt with applications issuing small writes.
> >
> > Issue here is that we allow to force memory schedule for the sk_buff
> > allocation, but we have no guarantee that sendmsg() is able to
> > copy some payload in it.
> >
> > In this patch, I make sure the socket can use up to tcp_wmem[0] bytes.
> >
> > For example, if we consider tcp_wmem[0] = 4096 (default on x86),
> > and initial skb->truesize being 1280, tcp_sendmsg() is able to
> > copy up to 2816 bytes under memory pressure.
> >
> > Before this patch a sendmsg() sending more than 2816 bytes
> > would either block forever (if persistent memory pressure),
> > or return -EAGAIN.
> >
> > For bigger MTU networks, it is advised to increase tcp_wmem[0]
> > to avoid sending too small packets.
> >
> > v2: deal with zero copy paths.
> >
> > Fixes: 8e4d980ac215 ("tcp: fix behavior for epoll edge trigger")
> > Signed-off-by: Eric Dumazet <edumazet@google.com>
>
> Acked-by: Soheil Hassas Yeganeh <soheil@google.com>
>
> Very nice find! Thank you!
>
> > ---
> > net/ipv4/tcp.c | 33 +++++++++++++++++++++++++++++----
> > 1 file changed, 29 insertions(+), 4 deletions(-)
> >
> > diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
> > index 14ebb4ec4a51f3c55501aa53423ce897599e8637..56083c2497f0b695c660256aa43f8a743d481697 100644
> > --- a/net/ipv4/tcp.c
> > +++ b/net/ipv4/tcp.c
> > @@ -951,6 +951,23 @@ static int tcp_downgrade_zcopy_pure(struct sock *sk, struct sk_buff *skb)
> > return 0;
> > }
> >
> > +static int tcp_wmem_schedule(struct sock *sk, int copy)
> > +{
> > + int left;
> > +
> > + if (likely(sk_wmem_schedule(sk, copy)))
> > + return copy;
> > +
> > + /* We could be in trouble if we have nothing queued.
> > + * Use whatever is left in sk->sk_forward_alloc and tcp_wmem[0]
> > + * to guarantee some progress.
> > + */
> > + left = sock_net(sk)->ipv4.sysctl_tcp_wmem[0] - sk->sk_wmem_queued;
> > + if (left > 0)
> > + sk_forced_mem_schedule(sk, min(left, copy));
> > + return min(copy, sk->sk_forward_alloc);
> > +}
> > +
> > static struct sk_buff *tcp_build_frag(struct sock *sk, int size_goal, int flags,
> > struct page *page, int offset, size_t *size)
> > {
> > @@ -986,7 +1003,11 @@ static struct sk_buff *tcp_build_frag(struct sock *sk, int size_goal, int flags,
> > tcp_mark_push(tp, skb);
> > goto new_segment;
> > }
> > - if (tcp_downgrade_zcopy_pure(sk, skb) || !sk_wmem_schedule(sk, copy))
> > + if (tcp_downgrade_zcopy_pure(sk, skb))
> > + return NULL;
Do we need to take care of the call to sk_wmem_schedule() inside
tcp_downgrade_zcopy_pure()?
> > +
> > + copy = tcp_wmem_schedule(sk, copy);
> > + if (!copy)
> > return NULL;
> >
> > if (can_coalesce) {
> > @@ -1334,8 +1355,11 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size)
> >
> > copy = min_t(int, copy, pfrag->size - pfrag->offset);
> >
> > - if (tcp_downgrade_zcopy_pure(sk, skb) ||
> > - !sk_wmem_schedule(sk, copy))
> > + if (tcp_downgrade_zcopy_pure(sk, skb))
> > + goto wait_for_space;
> > +
> > + copy = tcp_wmem_schedule(sk, copy);
> > + if (!copy)
> > goto wait_for_space;
> >
> > err = skb_copy_to_page_nocache(sk, &msg->msg_iter, skb,
> > @@ -1362,7 +1386,8 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size)
> > skb_shinfo(skb)->flags |= SKBFL_PURE_ZEROCOPY;
> >
> > if (!skb_zcopy_pure(skb)) {
> > - if (!sk_wmem_schedule(sk, copy))
> > + copy = tcp_wmem_schedule(sk, copy);
> > + if (!copy)
> > goto wait_for_space;
> > }
> >
> > --
> > 2.36.1.476.g0c4daa206d-goog
> >
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 net-next 2/2] tcp: fix possible freeze in tx path under memory pressure
[not found] ` <CANn89iJYwSoL-XvkcUK1am2wYQjToMnNRG-9pD9qY8+L=GyqaA@mail.gmail.com>
@ 2022-06-14 20:17 ` Wei Wang
0 siblings, 0 replies; 15+ messages in thread
From: Wei Wang @ 2022-06-14 20:17 UTC (permalink / raw)
To: Eric Dumazet
Cc: Soheil Hassas Yeganeh, Eric Dumazet, David S . Miller,
Jakub Kicinski, Paolo Abeni, netdev, Shakeel Butt, Neal Cardwell
On Tue, Jun 14, 2022 at 12:10 PM Eric Dumazet <edumazet@google.com> wrote:
>
>
>
> On Tue, Jun 14, 2022, 11:41 AM Wei Wang <weiwan@google.com> wrote:
>>
>> On Tue, Jun 14, 2022 at 10:42 AM Soheil Hassas Yeganeh
>> <soheil@google.com> wrote:
>> >
>> > On Tue, Jun 14, 2022 at 1:17 PM Eric Dumazet <eric.dumazet@gmail.com> wrote:
>> > >
>> > > From: Eric Dumazet <edumazet@google.com>
>> > >
>> > > Blamed commit only dealt with applications issuing small writes.
>> > >
>> > > Issue here is that we allow to force memory schedule for the sk_buff
>> > > allocation, but we have no guarantee that sendmsg() is able to
>> > > copy some payload in it.
>> > >
>> > > In this patch, I make sure the socket can use up to tcp_wmem[0] bytes.
>> > >
>> > > For example, if we consider tcp_wmem[0] = 4096 (default on x86),
>> > > and initial skb->truesize being 1280, tcp_sendmsg() is able to
>> > > copy up to 2816 bytes under memory pressure.
>> > >
>> > > Before this patch a sendmsg() sending more than 2816 bytes
>> > > would either block forever (if persistent memory pressure),
>> > > or return -EAGAIN.
>> > >
>> > > For bigger MTU networks, it is advised to increase tcp_wmem[0]
>> > > to avoid sending too small packets.
>> > >
>> > > v2: deal with zero copy paths.
>> > >
>> > > Fixes: 8e4d980ac215 ("tcp: fix behavior for epoll edge trigger")
>> > > Signed-off-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Wei Wang <weiwan@google.com>
>> >
>> > Acked-by: Soheil Hassas Yeganeh <soheil@google.com>
>> >
>> > Very nice find! Thank you!
>> >
>> > > ---
>> > > net/ipv4/tcp.c | 33 +++++++++++++++++++++++++++++----
>> > > 1 file changed, 29 insertions(+), 4 deletions(-)
>> > >
>> > > diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
>> > > index 14ebb4ec4a51f3c55501aa53423ce897599e8637..56083c2497f0b695c660256aa43f8a743d481697 100644
>> > > --- a/net/ipv4/tcp.c
>> > > +++ b/net/ipv4/tcp.c
>> > > @@ -951,6 +951,23 @@ static int tcp_downgrade_zcopy_pure(struct sock *sk, struct sk_buff *skb)
>> > > return 0;
>> > > }
>> > >
>> > > +static int tcp_wmem_schedule(struct sock *sk, int copy)
>> > > +{
>> > > + int left;
>> > > +
>> > > + if (likely(sk_wmem_schedule(sk, copy)))
>> > > + return copy;
>> > > +
>> > > + /* We could be in trouble if we have nothing queued.
>> > > + * Use whatever is left in sk->sk_forward_alloc and tcp_wmem[0]
>> > > + * to guarantee some progress.
>> > > + */
>> > > + left = sock_net(sk)->ipv4.sysctl_tcp_wmem[0] - sk->sk_wmem_queued;
>> > > + if (left > 0)
>> > > + sk_forced_mem_schedule(sk, min(left, copy));
>> > > + return min(copy, sk->sk_forward_alloc);
>> > > +}
>> > > +
>> > > static struct sk_buff *tcp_build_frag(struct sock *sk, int size_goal, int flags,
>> > > struct page *page, int offset, size_t *size)
>> > > {
>> > > @@ -986,7 +1003,11 @@ static struct sk_buff *tcp_build_frag(struct sock *sk, int size_goal, int flags,
>> > > tcp_mark_push(tp, skb);
>> > > goto new_segment;
>> > > }
>> > > - if (tcp_downgrade_zcopy_pure(sk, skb) || !sk_wmem_schedule(sk, copy))
>> > > + if (tcp_downgrade_zcopy_pure(sk, skb))
>> > > + return NULL;
>>
>> Do we need to take care of the call to sk_wmem_schedule() inside
>> tcp_downgrade_zcopy_pure()?
>
>
> We can not. Payload has been added already, and will be sent eventually.
Ack. Thanks for the explanation.
>>
>>
>> > > +
>> > > + copy = tcp_wmem_schedule(sk, copy);
>> > > + if (!copy)
>> > > return NULL;
>> > >
>> > > if (can_coalesce) {
>> > > @@ -1334,8 +1355,11 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size)
>> > >
>> > > copy = min_t(int, copy, pfrag->size - pfrag->offset);
>> > >
>> > > - if (tcp_downgrade_zcopy_pure(sk, skb) ||
>> > > - !sk_wmem_schedule(sk, copy))
>> > > + if (tcp_downgrade_zcopy_pure(sk, skb))
>> > > + goto wait_for_space;
>> > > +
>> > > + copy = tcp_wmem_schedule(sk, copy);
>> > > + if (!copy)
>> > > goto wait_for_space;
>> > >
>> > > err = skb_copy_to_page_nocache(sk, &msg->msg_iter, skb,
>> > > @@ -1362,7 +1386,8 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size)
>> > > skb_shinfo(skb)->flags |= SKBFL_PURE_ZEROCOPY;
>> > >
>> > > if (!skb_zcopy_pure(skb)) {
>> > > - if (!sk_wmem_schedule(sk, copy))
>> > > + copy = tcp_wmem_schedule(sk, copy);
>> > > + if (!copy)
>> > > goto wait_for_space;
>> > > }
>> > >
>> > > --
>> > > 2.36.1.476.g0c4daa206d-goog
>> > >
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 net-next 2/2] tcp: fix possible freeze in tx path under memory pressure
2022-06-14 17:17 ` [PATCH v2 net-next 2/2] tcp: fix possible freeze in tx path under memory pressure Eric Dumazet
2022-06-14 17:42 ` Soheil Hassas Yeganeh
@ 2022-06-14 20:31 ` Shakeel Butt
2022-06-17 10:08 ` Jason A. Donenfeld
2 siblings, 0 replies; 15+ messages in thread
From: Shakeel Butt @ 2022-06-14 20:31 UTC (permalink / raw)
To: Eric Dumazet
Cc: David S . Miller, Jakub Kicinski, Paolo Abeni, netdev,
Soheil Hassas Yeganeh, Wei Wang, Neal Cardwell, Eric Dumazet
On Tue, Jun 14, 2022 at 10:17 AM Eric Dumazet <eric.dumazet@gmail.com> wrote:
>
> From: Eric Dumazet <edumazet@google.com>
>
> Blamed commit only dealt with applications issuing small writes.
>
> Issue here is that we allow to force memory schedule for the sk_buff
> allocation, but we have no guarantee that sendmsg() is able to
> copy some payload in it.
>
> In this patch, I make sure the socket can use up to tcp_wmem[0] bytes.
>
> For example, if we consider tcp_wmem[0] = 4096 (default on x86),
> and initial skb->truesize being 1280, tcp_sendmsg() is able to
> copy up to 2816 bytes under memory pressure.
>
> Before this patch a sendmsg() sending more than 2816 bytes
> would either block forever (if persistent memory pressure),
> or return -EAGAIN.
>
> For bigger MTU networks, it is advised to increase tcp_wmem[0]
> to avoid sending too small packets.
>
> v2: deal with zero copy paths.
>
> Fixes: 8e4d980ac215 ("tcp: fix behavior for epoll edge trigger")
> Signed-off-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Shakeel Butt <shakeelb@google.com>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 net-next 0/2] tcp: final (?) round of mem pressure fixes
2022-06-14 17:17 [PATCH v2 net-next 0/2] tcp: final (?) round of mem pressure fixes Eric Dumazet
2022-06-14 17:17 ` [PATCH v2 net-next 1/2] tcp: fix over estimation in sk_forced_mem_schedule() Eric Dumazet
2022-06-14 17:17 ` [PATCH v2 net-next 2/2] tcp: fix possible freeze in tx path under memory pressure Eric Dumazet
@ 2022-06-15 11:56 ` David Miller
2022-06-16 6:00 ` Eric Dumazet
2 siblings, 1 reply; 15+ messages in thread
From: David Miller @ 2022-06-15 11:56 UTC (permalink / raw)
To: eric.dumazet
Cc: kuba, pabeni, netdev, soheil, weiwan, shakeelb, ncardwell, edumazet
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Tue, 14 Jun 2022 10:17:32 -0700
> From: Eric Dumazet <edumazet@google.com>
>
> While working on prior patch series (e10b02ee5b6c "Merge branch
> 'net-reduce-tcp_memory_allocated-inflation'"), I found that we
> could still have frozen TCP flows under memory pressure.
>
> I thought we had solved this in 2015, but the fix was not complete.
>
> v2: deal with zerocopy tx paths.
Does not apply cleanly to net, please respin.
Thank you.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 net-next 0/2] tcp: final (?) round of mem pressure fixes
2022-06-15 11:56 ` [PATCH v2 net-next 0/2] tcp: final (?) round of mem pressure fixes David Miller
@ 2022-06-16 6:00 ` Eric Dumazet
2022-06-16 6:07 ` Eric Dumazet
0 siblings, 1 reply; 15+ messages in thread
From: Eric Dumazet @ 2022-06-16 6:00 UTC (permalink / raw)
To: David Miller
Cc: Eric Dumazet, Jakub Kicinski, Paolo Abeni, netdev,
Soheil Hassas Yeganeh, Wei Wang, Shakeel Butt, Neal Cardwell
On Wed, Jun 15, 2022 at 4:56 AM David Miller <davem@davemloft.net> wrote:
>
> From: Eric Dumazet <eric.dumazet@gmail.com>
> Date: Tue, 14 Jun 2022 10:17:32 -0700
>
> > From: Eric Dumazet <edumazet@google.com>
> >
> > While working on prior patch series (e10b02ee5b6c "Merge branch
> > 'net-reduce-tcp_memory_allocated-inflation'"), I found that we
> > could still have frozen TCP flows under memory pressure.
> >
> > I thought we had solved this in 2015, but the fix was not complete.
> >
> > v2: deal with zerocopy tx paths.
>
> Does not apply cleanly to net, please respin.
>
> Thank you.
I was targeting net-next tree for this old bug, and planning to
prepare stable backports.
Tell me if you prefer to respin to net tree, probably later today
because I am traveling.
Thanks.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 net-next 0/2] tcp: final (?) round of mem pressure fixes
2022-06-16 6:00 ` Eric Dumazet
@ 2022-06-16 6:07 ` Eric Dumazet
0 siblings, 0 replies; 15+ messages in thread
From: Eric Dumazet @ 2022-06-16 6:07 UTC (permalink / raw)
To: David Miller
Cc: Eric Dumazet, Jakub Kicinski, Paolo Abeni, netdev,
Soheil Hassas Yeganeh, Wei Wang, Shakeel Butt, Neal Cardwell
On Wed, Jun 15, 2022 at 11:00 PM Eric Dumazet <edumazet@google.com> wrote:
>
> On Wed, Jun 15, 2022 at 4:56 AM David Miller <davem@davemloft.net> wrote:
> >
> > From: Eric Dumazet <eric.dumazet@gmail.com>
> > Date: Tue, 14 Jun 2022 10:17:32 -0700
> >
> > > From: Eric Dumazet <edumazet@google.com>
> > >
> > > While working on prior patch series (e10b02ee5b6c "Merge branch
> > > 'net-reduce-tcp_memory_allocated-inflation'"), I found that we
> > > could still have frozen TCP flows under memory pressure.
> > >
> > > I thought we had solved this in 2015, but the fix was not complete.
> > >
> > > v2: deal with zerocopy tx paths.
> >
> > Does not apply cleanly to net, please respin.
> >
> > Thank you.
>
> I was targeting net-next tree for this old bug, and planning to
> prepare stable backports.
This was because in net-next we got rid of SK_MEM_QUANTUM, and to
limit merge conflicts.
>
> Tell me if you prefer to respin to net tree, probably later today
> because I am traveling.
>
> Thanks.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 net-next 2/2] tcp: fix possible freeze in tx path under memory pressure
2022-06-14 17:17 ` [PATCH v2 net-next 2/2] tcp: fix possible freeze in tx path under memory pressure Eric Dumazet
2022-06-14 17:42 ` Soheil Hassas Yeganeh
2022-06-14 20:31 ` Shakeel Butt
@ 2022-06-17 10:08 ` Jason A. Donenfeld
2022-06-17 10:13 ` Jason A. Donenfeld
2 siblings, 1 reply; 15+ messages in thread
From: Jason A. Donenfeld @ 2022-06-17 10:08 UTC (permalink / raw)
To: Eric Dumazet
Cc: David S . Miller, Jakub Kicinski, Paolo Abeni, netdev,
Soheil Hassas Yeganeh, Wei Wang, Shakeel Butt, Neal Cardwell,
Eric Dumazet
Hi,
On Tue, Jun 14, 2022 at 10:17:34AM -0700, Eric Dumazet wrote:
> From: Eric Dumazet <edumazet@google.com>
>
> Blamed commit only dealt with applications issuing small writes.
>
> Issue here is that we allow to force memory schedule for the sk_buff
> allocation, but we have no guarantee that sendmsg() is able to
> copy some payload in it.
>
> In this patch, I make sure the socket can use up to tcp_wmem[0] bytes.
>
> For example, if we consider tcp_wmem[0] = 4096 (default on x86),
> and initial skb->truesize being 1280, tcp_sendmsg() is able to
> copy up to 2816 bytes under memory pressure.
>
> Before this patch a sendmsg() sending more than 2816 bytes
> would either block forever (if persistent memory pressure),
> or return -EAGAIN.
>
> For bigger MTU networks, it is advised to increase tcp_wmem[0]
> to avoid sending too small packets.
>
> v2: deal with zero copy paths.
I think this might have gotten double applied:
https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=849b425cd091e1804af964b771761cfbefbafb43
https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=f54755f6a11accb2db5ef17f8f75aad0875aefdc
and now net-next builds are broken:
../../../../../../../../net/ipv4/tcp.c:971:12: error: redefinition of ‘tcp_wmem_schedule’
971 | static int tcp_wmem_schedule(struct sock *sk, int copy) | ^~~~~~~~~~~~~~~~~
../../../../../../../../net/ipv4/tcp.c:954:12: note: previous definition of ‘tcp_wmem_schedule’ with type ‘int(struct sock *, int)’
954 | static int tcp_wmem_schedule(struct sock *sk, int copy) | ^~~~~~~~~~~~~~~~~
../../../../../../../../net/ipv4/tcp.c:954:12: warning: ‘tcp_wmem_schedule’ defined but not used [-Wunused-function] make[5]: *** [/home/wgci/tmp/2813390.12234/tmp.0PMBO65tGf/scripts/Makefile.build:249: net
/ipv4/tcp.o] Error 1 make[4]: *** [/home/wgci/tmp/2813390.12234/tmp.0PMBO65tGf/scripts/Makefile.build:466: net/ipv4] Error 2
make[4]: *** Waiting for unfinished jobs....
Jason
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2 net-next 2/2] tcp: fix possible freeze in tx path under memory pressure
2022-06-17 10:08 ` Jason A. Donenfeld
@ 2022-06-17 10:13 ` Jason A. Donenfeld
0 siblings, 0 replies; 15+ messages in thread
From: Jason A. Donenfeld @ 2022-06-17 10:13 UTC (permalink / raw)
To: Eric Dumazet
Cc: David S . Miller, Jakub Kicinski, Paolo Abeni, netdev,
Soheil Hassas Yeganeh, Wei Wang, Shakeel Butt, Neal Cardwell,
Eric Dumazet
On Fri, Jun 17, 2022 at 12:08:14PM +0200, Jason A. Donenfeld wrote:
> Hi,
>
> On Tue, Jun 14, 2022 at 10:17:34AM -0700, Eric Dumazet wrote:
> > From: Eric Dumazet <edumazet@google.com>
> >
> > Blamed commit only dealt with applications issuing small writes.
> >
> > Issue here is that we allow to force memory schedule for the sk_buff
> > allocation, but we have no guarantee that sendmsg() is able to
> > copy some payload in it.
> >
> > In this patch, I make sure the socket can use up to tcp_wmem[0] bytes.
> >
> > For example, if we consider tcp_wmem[0] = 4096 (default on x86),
> > and initial skb->truesize being 1280, tcp_sendmsg() is able to
> > copy up to 2816 bytes under memory pressure.
> >
> > Before this patch a sendmsg() sending more than 2816 bytes
> > would either block forever (if persistent memory pressure),
> > or return -EAGAIN.
> >
> > For bigger MTU networks, it is advised to increase tcp_wmem[0]
> > to avoid sending too small packets.
> >
> > v2: deal with zero copy paths.
>
> I think this might have gotten double applied:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=849b425cd091e1804af964b771761cfbefbafb43
> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=f54755f6a11accb2db5ef17f8f75aad0875aefdc
>
> and now net-next builds are broken:
>
> ../../../../../../../../net/ipv4/tcp.c:971:12: error: redefinition of ‘tcp_wmem_schedule’
> 971 | static int tcp_wmem_schedule(struct sock *sk, int copy) | ^~~~~~~~~~~~~~~~~
> ../../../../../../../../net/ipv4/tcp.c:954:12: note: previous definition of ‘tcp_wmem_schedule’ with type ‘int(struct sock *, int)’
> 954 | static int tcp_wmem_schedule(struct sock *sk, int copy) | ^~~~~~~~~~~~~~~~~
> ../../../../../../../../net/ipv4/tcp.c:954:12: warning: ‘tcp_wmem_schedule’ defined but not used [-Wunused-function] make[5]: *** [/home/wgci/tmp/2813390.12234/tmp.0PMBO65tGf/scripts/Makefile.build:249: net
> /ipv4/tcp.o] Error 1 make[4]: *** [/home/wgci/tmp/2813390.12234/tmp.0PMBO65tGf/scripts/Makefile.build:466: net/ipv4] Error 2
> make[4]: *** Waiting for unfinished jobs....
Ah, fixed already:
https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=fd8b330ce1bb79ba00047435cc213b14f886bf1f
>
> Jason
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2022-06-17 10:13 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-14 17:17 [PATCH v2 net-next 0/2] tcp: final (?) round of mem pressure fixes Eric Dumazet
2022-06-14 17:17 ` [PATCH v2 net-next 1/2] tcp: fix over estimation in sk_forced_mem_schedule() Eric Dumazet
2022-06-14 17:40 ` Soheil Hassas Yeganeh
2022-06-14 18:39 ` Wei Wang
2022-06-14 17:40 ` Shakeel Butt
2022-06-14 17:17 ` [PATCH v2 net-next 2/2] tcp: fix possible freeze in tx path under memory pressure Eric Dumazet
2022-06-14 17:42 ` Soheil Hassas Yeganeh
2022-06-14 18:41 ` Wei Wang
[not found] ` <CANn89iJYwSoL-XvkcUK1am2wYQjToMnNRG-9pD9qY8+L=GyqaA@mail.gmail.com>
2022-06-14 20:17 ` Wei Wang
2022-06-14 20:31 ` Shakeel Butt
2022-06-17 10:08 ` Jason A. Donenfeld
2022-06-17 10:13 ` Jason A. Donenfeld
2022-06-15 11:56 ` [PATCH v2 net-next 0/2] tcp: final (?) round of mem pressure fixes David Miller
2022-06-16 6:00 ` Eric Dumazet
2022-06-16 6:07 ` Eric Dumazet
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).