* [PATCH v5 1/2] tcp: Add TCP_INFO counter for packets received out-of-order
@ 2019-09-13 23:23 Thomas Higdon
2019-09-13 23:23 ` [PATCH v5 2/2] tcp: Add snd_wnd to TCP_INFO Thomas Higdon
` (2 more replies)
0 siblings, 3 replies; 9+ messages in thread
From: Thomas Higdon @ 2019-09-13 23:23 UTC (permalink / raw)
To: netdev
Cc: Jonathan Lemon, Dave Jones, Eric Dumazet, Neal Cardwell,
Dave Taht, Yuchung Cheng, Soheil Hassas Yeganeh
For receive-heavy cases on the server-side, we want to track the
connection quality for individual client IPs. This counter, similar to
the existing system-wide TCPOFOQueue counter in /proc/net/netstat,
tracks out-of-order packet reception. By providing this counter in
TCP_INFO, it will allow understanding to what degree receive-heavy
sockets are experiencing out-of-order delivery and packet drops
indicating congestion.
Please note that this is similar to the counter in NetBSD TCP_INFO, and
has the same name.
Also note that we avoid increasing the size of the tcp_sock struct by
taking advantage of a hole.
Signed-off-by: Thomas Higdon <tph@fb.com>
---
changes since v4:
- optimize placement of rcv_ooopack to avoid increasing tcp_sock struct
size
include/linux/tcp.h | 2 ++
include/uapi/linux/tcp.h | 2 ++
net/ipv4/tcp.c | 2 ++
net/ipv4/tcp_input.c | 1 +
4 files changed, 7 insertions(+)
diff --git a/include/linux/tcp.h b/include/linux/tcp.h
index f3a85a7fb4b1..99617e528ea2 100644
--- a/include/linux/tcp.h
+++ b/include/linux/tcp.h
@@ -354,6 +354,8 @@ struct tcp_sock {
#define BPF_SOCK_OPS_TEST_FLAG(TP, ARG) 0
#endif
+ u32 rcv_ooopack; /* Received out-of-order packets, for tcpinfo */
+
/* Receiver side RTT estimation */
u32 rcv_rtt_last_tsecr;
struct {
diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h
index b3564f85a762..20237987ccc8 100644
--- a/include/uapi/linux/tcp.h
+++ b/include/uapi/linux/tcp.h
@@ -270,6 +270,8 @@ struct tcp_info {
__u64 tcpi_bytes_retrans; /* RFC4898 tcpEStatsPerfOctetsRetrans */
__u32 tcpi_dsack_dups; /* RFC4898 tcpEStatsStackDSACKDups */
__u32 tcpi_reord_seen; /* reordering events seen */
+
+ __u32 tcpi_rcv_ooopack; /* Out-of-order packets received */
};
/* netlink attributes types for SCM_TIMESTAMPING_OPT_STATS */
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 94df48bcecc2..4cf58208270e 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -2653,6 +2653,7 @@ int tcp_disconnect(struct sock *sk, int flags)
tp->rx_opt.saw_tstamp = 0;
tp->rx_opt.dsack = 0;
tp->rx_opt.num_sacks = 0;
+ tp->rcv_ooopack = 0;
/* Clean up fastopen related fields */
@@ -3295,6 +3296,7 @@ void tcp_get_info(struct sock *sk, struct tcp_info *info)
info->tcpi_bytes_retrans = tp->bytes_retrans;
info->tcpi_dsack_dups = tp->dsack_dups;
info->tcpi_reord_seen = tp->reord_seen;
+ info->tcpi_rcv_ooopack = tp->rcv_ooopack;
unlock_sock_fast(sk, slow);
}
EXPORT_SYMBOL_GPL(tcp_get_info);
diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c
index 706cbb3b2986..2ef333354026 100644
--- a/net/ipv4/tcp_input.c
+++ b/net/ipv4/tcp_input.c
@@ -4555,6 +4555,7 @@ static void tcp_data_queue_ofo(struct sock *sk, struct sk_buff *skb)
tp->pred_flags = 0;
inet_csk_schedule_ack(sk);
+ tp->rcv_ooopack += max_t(u16, 1, skb_shinfo(skb)->gso_segs);
NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPOFOQUEUE);
seq = TCP_SKB_CB(skb)->seq;
end_seq = TCP_SKB_CB(skb)->end_seq;
--
2.17.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v5 2/2] tcp: Add snd_wnd to TCP_INFO
2019-09-13 23:23 [PATCH v5 1/2] tcp: Add TCP_INFO counter for packets received out-of-order Thomas Higdon
@ 2019-09-13 23:23 ` Thomas Higdon
2019-09-13 23:36 ` Yuchung Cheng
` (2 more replies)
2019-09-14 15:43 ` [PATCH v5 1/2] tcp: Add TCP_INFO counter for packets received out-of-order Neal Cardwell
2019-09-16 14:39 ` David Miller
2 siblings, 3 replies; 9+ messages in thread
From: Thomas Higdon @ 2019-09-13 23:23 UTC (permalink / raw)
To: netdev
Cc: Jonathan Lemon, Dave Jones, Eric Dumazet, Neal Cardwell,
Dave Taht, Yuchung Cheng, Soheil Hassas Yeganeh
Neal Cardwell mentioned that snd_wnd would be useful for diagnosing TCP
performance problems --
> (1) Usually when we're diagnosing TCP performance problems, we do so
> from the sender, since the sender makes most of the
> performance-critical decisions (cwnd, pacing, TSO size, TSQ, etc).
> From the sender-side the thing that would be most useful is to see
> tp->snd_wnd, the receive window that the receiver has advertised to
> the sender.
This serves the purpose of adding an additional __u32 to avoid the
would-be hole caused by the addition of the tcpi_rcvi_ooopack field.
Signed-off-by: Thomas Higdon <tph@fb.com>
---
changes since v4:
- clarify comment
include/uapi/linux/tcp.h | 4 ++++
net/ipv4/tcp.c | 1 +
2 files changed, 5 insertions(+)
diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h
index 20237987ccc8..81e697978e8b 100644
--- a/include/uapi/linux/tcp.h
+++ b/include/uapi/linux/tcp.h
@@ -272,6 +272,10 @@ struct tcp_info {
__u32 tcpi_reord_seen; /* reordering events seen */
__u32 tcpi_rcv_ooopack; /* Out-of-order packets received */
+
+ __u32 tcpi_snd_wnd; /* peer's advertised receive window after
+ * scaling (bytes)
+ */
};
/* netlink attributes types for SCM_TIMESTAMPING_OPT_STATS */
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 4cf58208270e..79c325a07ba5 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -3297,6 +3297,7 @@ void tcp_get_info(struct sock *sk, struct tcp_info *info)
info->tcpi_dsack_dups = tp->dsack_dups;
info->tcpi_reord_seen = tp->reord_seen;
info->tcpi_rcv_ooopack = tp->rcv_ooopack;
+ info->tcpi_snd_wnd = tp->snd_wnd;
unlock_sock_fast(sk, slow);
}
EXPORT_SYMBOL_GPL(tcp_get_info);
--
2.17.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v5 2/2] tcp: Add snd_wnd to TCP_INFO
2019-09-13 23:23 ` [PATCH v5 2/2] tcp: Add snd_wnd to TCP_INFO Thomas Higdon
@ 2019-09-13 23:36 ` Yuchung Cheng
2019-09-14 15:45 ` Neal Cardwell
2019-09-16 14:39 ` David Miller
2 siblings, 0 replies; 9+ messages in thread
From: Yuchung Cheng @ 2019-09-13 23:36 UTC (permalink / raw)
To: Thomas Higdon
Cc: netdev, Jonathan Lemon, Dave Jones, Eric Dumazet, Neal Cardwell,
Dave Taht, Soheil Hassas Yeganeh
On Fri, Sep 13, 2019 at 4:23 PM Thomas Higdon <tph@fb.com> wrote:
>
> Neal Cardwell mentioned that snd_wnd would be useful for diagnosing TCP
> performance problems --
> > (1) Usually when we're diagnosing TCP performance problems, we do so
> > from the sender, since the sender makes most of the
> > performance-critical decisions (cwnd, pacing, TSO size, TSQ, etc).
> > From the sender-side the thing that would be most useful is to see
> > tp->snd_wnd, the receive window that the receiver has advertised to
> > the sender.
>
> This serves the purpose of adding an additional __u32 to avoid the
> would-be hole caused by the addition of the tcpi_rcvi_ooopack field.
>
> Signed-off-by: Thomas Higdon <tph@fb.com>
> ---
Acked-by: Yuchung Cheng <ycheng@google.com>
> changes since v4:
> - clarify comment
> include/uapi/linux/tcp.h | 4 ++++
> net/ipv4/tcp.c | 1 +
> 2 files changed, 5 insertions(+)
>
> diff --git a/include/uapi/linux/tcp.h b/include/uapi/linux/tcp.h
> index 20237987ccc8..81e697978e8b 100644
> --- a/include/uapi/linux/tcp.h
> +++ b/include/uapi/linux/tcp.h
> @@ -272,6 +272,10 @@ struct tcp_info {
> __u32 tcpi_reord_seen; /* reordering events seen */
>
> __u32 tcpi_rcv_ooopack; /* Out-of-order packets received */
> +
> + __u32 tcpi_snd_wnd; /* peer's advertised receive window after
> + * scaling (bytes)
> + */
> };
>
> /* netlink attributes types for SCM_TIMESTAMPING_OPT_STATS */
> diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
> index 4cf58208270e..79c325a07ba5 100644
> --- a/net/ipv4/tcp.c
> +++ b/net/ipv4/tcp.c
> @@ -3297,6 +3297,7 @@ void tcp_get_info(struct sock *sk, struct tcp_info *info)
> info->tcpi_dsack_dups = tp->dsack_dups;
> info->tcpi_reord_seen = tp->reord_seen;
> info->tcpi_rcv_ooopack = tp->rcv_ooopack;
> + info->tcpi_snd_wnd = tp->snd_wnd;
> unlock_sock_fast(sk, slow);
> }
> EXPORT_SYMBOL_GPL(tcp_get_info);
> --
> 2.17.1
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v5 1/2] tcp: Add TCP_INFO counter for packets received out-of-order
2019-09-13 23:23 [PATCH v5 1/2] tcp: Add TCP_INFO counter for packets received out-of-order Thomas Higdon
2019-09-13 23:23 ` [PATCH v5 2/2] tcp: Add snd_wnd to TCP_INFO Thomas Higdon
@ 2019-09-14 15:43 ` Neal Cardwell
2019-09-16 17:42 ` Thomas Higdon
2019-09-16 14:39 ` David Miller
2 siblings, 1 reply; 9+ messages in thread
From: Neal Cardwell @ 2019-09-14 15:43 UTC (permalink / raw)
To: Thomas Higdon
Cc: netdev, Jonathan Lemon, Dave Jones, Eric Dumazet, Dave Taht,
Yuchung Cheng, Soheil Hassas Yeganeh
On Fri, Sep 13, 2019 at 7:23 PM Thomas Higdon <tph@fb.com> wrote:
>
> For receive-heavy cases on the server-side, we want to track the
> connection quality for individual client IPs. This counter, similar to
> the existing system-wide TCPOFOQueue counter in /proc/net/netstat,
> tracks out-of-order packet reception. By providing this counter in
> TCP_INFO, it will allow understanding to what degree receive-heavy
> sockets are experiencing out-of-order delivery and packet drops
> indicating congestion.
>
> Please note that this is similar to the counter in NetBSD TCP_INFO, and
> has the same name.
>
> Also note that we avoid increasing the size of the tcp_sock struct by
> taking advantage of a hole.
>
> Signed-off-by: Thomas Higdon <tph@fb.com>
> ---
> changes since v4:
> - optimize placement of rcv_ooopack to avoid increasing tcp_sock struct
> size
Acked-by: Neal Cardwell <ncardwell@google.com>
Thanks, Thomas, for adding this!
After this is merged, would you mind sending a patch to add support to
the "ss" command line tool to print these 2 new fields?
My favorite recent example of such a patch to ss is Eric's change:
https://git.kernel.org/pub/scm/network/iproute2/iproute2.git/commit/misc/ss.c?id=5eead6270a19f00464052d4084f32182cfe027ff
thanks,
neal
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v5 2/2] tcp: Add snd_wnd to TCP_INFO
2019-09-13 23:23 ` [PATCH v5 2/2] tcp: Add snd_wnd to TCP_INFO Thomas Higdon
2019-09-13 23:36 ` Yuchung Cheng
@ 2019-09-14 15:45 ` Neal Cardwell
2019-09-14 17:57 ` Soheil Hassas Yeganeh
2019-09-16 14:39 ` David Miller
2 siblings, 1 reply; 9+ messages in thread
From: Neal Cardwell @ 2019-09-14 15:45 UTC (permalink / raw)
To: Thomas Higdon
Cc: netdev, Jonathan Lemon, Dave Jones, Eric Dumazet, Dave Taht,
Yuchung Cheng, Soheil Hassas Yeganeh
On Fri, Sep 13, 2019 at 7:23 PM Thomas Higdon <tph@fb.com> wrote:
>
> Neal Cardwell mentioned that snd_wnd would be useful for diagnosing TCP
> performance problems --
> > (1) Usually when we're diagnosing TCP performance problems, we do so
> > from the sender, since the sender makes most of the
> > performance-critical decisions (cwnd, pacing, TSO size, TSQ, etc).
> > From the sender-side the thing that would be most useful is to see
> > tp->snd_wnd, the receive window that the receiver has advertised to
> > the sender.
>
> This serves the purpose of adding an additional __u32 to avoid the
> would-be hole caused by the addition of the tcpi_rcvi_ooopack field.
>
> Signed-off-by: Thomas Higdon <tph@fb.com>
> ---
> changes since v4:
> - clarify comment
Acked-by: Neal Cardwell <ncardwell@google.com>
Thanks!
neal
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v5 2/2] tcp: Add snd_wnd to TCP_INFO
2019-09-14 15:45 ` Neal Cardwell
@ 2019-09-14 17:57 ` Soheil Hassas Yeganeh
0 siblings, 0 replies; 9+ messages in thread
From: Soheil Hassas Yeganeh @ 2019-09-14 17:57 UTC (permalink / raw)
To: Neal Cardwell
Cc: Thomas Higdon, netdev, Jonathan Lemon, Dave Jones, Eric Dumazet,
Dave Taht, Yuchung Cheng
On Sat, Sep 14, 2019 at 11:45 AM Neal Cardwell <ncardwell@google.com> wrote:
>
> On Fri, Sep 13, 2019 at 7:23 PM Thomas Higdon <tph@fb.com> wrote:
> >
> > Neal Cardwell mentioned that snd_wnd would be useful for diagnosing TCP
> > performance problems --
> > > (1) Usually when we're diagnosing TCP performance problems, we do so
> > > from the sender, since the sender makes most of the
> > > performance-critical decisions (cwnd, pacing, TSO size, TSQ, etc).
> > > From the sender-side the thing that would be most useful is to see
> > > tp->snd_wnd, the receive window that the receiver has advertised to
> > > the sender.
> >
> > This serves the purpose of adding an additional __u32 to avoid the
> > would-be hole caused by the addition of the tcpi_rcvi_ooopack field.
> >
> > Signed-off-by: Thomas Higdon <tph@fb.com>
> > ---
> > changes since v4:
> > - clarify comment
>
> Acked-by: Neal Cardwell <ncardwell@google.com>
Acked-by: Soheil Hassas Yeganeh <soheil@google.com>
Thank you for adding the new field!
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v5 1/2] tcp: Add TCP_INFO counter for packets received out-of-order
2019-09-13 23:23 [PATCH v5 1/2] tcp: Add TCP_INFO counter for packets received out-of-order Thomas Higdon
2019-09-13 23:23 ` [PATCH v5 2/2] tcp: Add snd_wnd to TCP_INFO Thomas Higdon
2019-09-14 15:43 ` [PATCH v5 1/2] tcp: Add TCP_INFO counter for packets received out-of-order Neal Cardwell
@ 2019-09-16 14:39 ` David Miller
2 siblings, 0 replies; 9+ messages in thread
From: David Miller @ 2019-09-16 14:39 UTC (permalink / raw)
To: tph
Cc: netdev, jonathan.lemon, dsj, edumazet, ncardwell, dave.taht,
ycheng, soheil
From: Thomas Higdon <tph@fb.com>
Date: Fri, 13 Sep 2019 23:23:34 +0000
> For receive-heavy cases on the server-side, we want to track the
> connection quality for individual client IPs. This counter, similar to
> the existing system-wide TCPOFOQueue counter in /proc/net/netstat,
> tracks out-of-order packet reception. By providing this counter in
> TCP_INFO, it will allow understanding to what degree receive-heavy
> sockets are experiencing out-of-order delivery and packet drops
> indicating congestion.
>
> Please note that this is similar to the counter in NetBSD TCP_INFO, and
> has the same name.
>
> Also note that we avoid increasing the size of the tcp_sock struct by
> taking advantage of a hole.
>
> Signed-off-by: Thomas Higdon <tph@fb.com>
Applied.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v5 2/2] tcp: Add snd_wnd to TCP_INFO
2019-09-13 23:23 ` [PATCH v5 2/2] tcp: Add snd_wnd to TCP_INFO Thomas Higdon
2019-09-13 23:36 ` Yuchung Cheng
2019-09-14 15:45 ` Neal Cardwell
@ 2019-09-16 14:39 ` David Miller
2 siblings, 0 replies; 9+ messages in thread
From: David Miller @ 2019-09-16 14:39 UTC (permalink / raw)
To: tph
Cc: netdev, jonathan.lemon, dsj, edumazet, ncardwell, dave.taht,
ycheng, soheil
From: Thomas Higdon <tph@fb.com>
Date: Fri, 13 Sep 2019 23:23:35 +0000
> Neal Cardwell mentioned that snd_wnd would be useful for diagnosing TCP
> performance problems --
>> (1) Usually when we're diagnosing TCP performance problems, we do so
>> from the sender, since the sender makes most of the
>> performance-critical decisions (cwnd, pacing, TSO size, TSQ, etc).
>> From the sender-side the thing that would be most useful is to see
>> tp->snd_wnd, the receive window that the receiver has advertised to
>> the sender.
>
> This serves the purpose of adding an additional __u32 to avoid the
> would-be hole caused by the addition of the tcpi_rcvi_ooopack field.
>
> Signed-off-by: Thomas Higdon <tph@fb.com>
Applied.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v5 1/2] tcp: Add TCP_INFO counter for packets received out-of-order
2019-09-14 15:43 ` [PATCH v5 1/2] tcp: Add TCP_INFO counter for packets received out-of-order Neal Cardwell
@ 2019-09-16 17:42 ` Thomas Higdon
0 siblings, 0 replies; 9+ messages in thread
From: Thomas Higdon @ 2019-09-16 17:42 UTC (permalink / raw)
To: Neal Cardwell
Cc: netdev, Jonathan Lemon, Dave Jones, Eric Dumazet, Dave Taht,
Yuchung Cheng, Soheil Hassas Yeganeh
On Sat, Sep 14, 2019 at 11:43:25AM -0400, Neal Cardwell wrote:
> On Fri, Sep 13, 2019 at 7:23 PM Thomas Higdon <tph@fb.com> wrote:
> >
> > For receive-heavy cases on the server-side, we want to track the
> > connection quality for individual client IPs. This counter, similar to
> > the existing system-wide TCPOFOQueue counter in /proc/net/netstat,
> > tracks out-of-order packet reception. By providing this counter in
> > TCP_INFO, it will allow understanding to what degree receive-heavy
> > sockets are experiencing out-of-order delivery and packet drops
> > indicating congestion.
> >
> > Please note that this is similar to the counter in NetBSD TCP_INFO, and
> > has the same name.
> >
> > Also note that we avoid increasing the size of the tcp_sock struct by
> > taking advantage of a hole.
> >
> > Signed-off-by: Thomas Higdon <tph@fb.com>
> > ---
> > changes since v4:
> > - optimize placement of rcv_ooopack to avoid increasing tcp_sock struct
> > size
>
>
> Acked-by: Neal Cardwell <ncardwell@google.com>
>
> Thanks, Thomas, for adding this!
>
> After this is merged, would you mind sending a patch to add support to
> the "ss" command line tool to print these 2 new fields?
>
> My favorite recent example of such a patch to ss is Eric's change:
> https://git.kernel.org/pub/scm/network/iproute2/iproute2.git/commit/misc/ss.c?id=5eead6270a19f00464052d4084f32182cfe027ff
Yes, and thank you for the help in getting this into a good state!
From looking at that "ss" patch, it seems like we would need to wait
until iproute2-next's include/uapi/linux/tcp.h has received a merge from
kernel net-next before we'd be able to apply a patch for "ss" that uses
the new fields.
In the meantime, as you've asked, I will go ahead and send a patch for
iproute2-next's "ss" with the assumption that these tcpinfo changes have
already been merged.
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2019-09-16 17:42 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-13 23:23 [PATCH v5 1/2] tcp: Add TCP_INFO counter for packets received out-of-order Thomas Higdon
2019-09-13 23:23 ` [PATCH v5 2/2] tcp: Add snd_wnd to TCP_INFO Thomas Higdon
2019-09-13 23:36 ` Yuchung Cheng
2019-09-14 15:45 ` Neal Cardwell
2019-09-14 17:57 ` Soheil Hassas Yeganeh
2019-09-16 14:39 ` David Miller
2019-09-14 15:43 ` [PATCH v5 1/2] tcp: Add TCP_INFO counter for packets received out-of-order Neal Cardwell
2019-09-16 17:42 ` Thomas Higdon
2019-09-16 14:39 ` David Miller
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).