All of
 help / color / mirror / Atom feed
From: Willem de Bruijn <>
To: Kelly Littlepage <>
Cc: David Miller <>,
	Alexey Kuznetsov <>,
	Hideaki YOSHIFUJI <>,
	Jakub Kicinski <>,
	Network Development <>,
	Iris Liu <>, Mike Maloney <>,
	Eric Dumazet <>,
	Soheil Hassas Yeganeh <>
Subject: Re: [PATCH] net: tcp: fix rx timestamp behavior for tcp_recvmsg
Date: Tue, 5 May 2020 16:23:21 -0400	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

On Mon, May 4, 2020 at 12:30 PM Kelly Littlepage <> wrote:
> Timestamping cmsgs are not returned when the user buffer supplied to
> recvmsg is too small to copy at least one skbuff in entirety.

In general a tcp reader should not make any assumptions on
packetization of the bytestream, including the number of skbs that
might have made up the bytestream.

> Support
> for TCP rx timestamps came from commit 98aaa913b4ed ("tcp: Extend
> SOF_TIMESTAMPING_RX_SOFTWARE to TCP recvmsg") which noted that the cmsg
> should "return the timestamp corresponding to the highest sequence
> number data returned." The commit further notes that when coalescing
> skbs code should "maintain the invariant of returning the timestamp of
> the last byte in the recvmsg buffer."

This states that if a byte range spans multiple timestamps, only the
last one is returned.

> This is consistent with Section 1.4 of timestamping.txt, a document that
> discusses expected behavior when timestamping streaming protocols. It's
> worth noting that Section 1.4 alludes to a "buffer" in a way that might
> have resulted in the current behavior:
> > The SO_TIMESTAMPING interface supports timestamping of bytes in a
> bytestream. Each request is interpreted as a request for when the entire
> contents of the buffer has passed a timestamping point....In practice,
> timestamps can be correlated with segments of a bytestream consistently,
> if both semantics of the timestamp and the timing of measurement are
> chosen correctly....For bytestreams, we chose that a timestamp is
> generated only when all bytes have passed a point.
> An interpretation of skbs as delineators for timestamping points makes
> sense for tx timestamps but poses implementation challenges on the rx
> side. Under the current API unless tcp_recvmsg happens to return bytes
> copied from precisely one skb there's no useful mapping from bytes to
> timestamps. Some sequences of reads will result in timestamps getting
> lost

That's a known caveat, see above. This patch does not change that.

> and others will result in the user receiving a timestamp from the
> second to last skb that tcp_recvmsg copied from instead of the last.

On Tx, the idea was to associate a timestamp with the last byte in the
send buffer, so that a timestamp for this seqno informs us of the
upper bound on latency of all bytes in the send buffer.

On Rx, we currently return the timestamp of the last skb of which the
last byte is read, which is associated with a byte in the recv buffer,
but it is not necessarily the last one. Nor the first. As such it is
not clear what it defines.

Your patch addresses this by instead always returning the timestamp
associated with the last byte in the recv buffer. The same timestamp
could then be returned again for a subsequent recv call, if the entire
recv buffer is filled from the same skb. Which is fine.

That sounds correct to me.

> The
> proposed change addresses both problems while remaining consistent with
> 1.4 and the wording of commit 98aaa913b4ed ("tcp: Extend
> Co-developed-by: Iris Liu <>
> Signed-off-by: Iris Liu <>
> Signed-off-by: Kelly Littlepage <>
> ---
>  net/ipv4/tcp.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
> index 6d87de434377..e72bd651d21a 100644
> --- a/net/ipv4/tcp.c
> +++ b/net/ipv4/tcp.c
> @@ -2154,13 +2154,15 @@ int tcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, int nonblock,
>                         tp->urg_data = 0;
>                         tcp_fast_path_check(sk);
>                 }
> -               if (used + offset < skb->len)
> -                       continue;
>                 if (TCP_SKB_CB(skb)->has_rxtstamp) {
>                         tcp_update_recv_tstamps(skb, &tss);
>                         cmsg_flags |= 2;
>                 }
> +
> +               if (used + offset < skb->len)
> +                       continue;
> +
>                 if (TCP_SKB_CB(skb)->tcp_flags & TCPHDR_FIN)
>                         goto found_fin_ok;
>                 if (!(flags & MSG_PEEK))
> --
> 2.26.2

  reply	other threads:[~2020-05-05 20:24 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-05-04 16:29 [PATCH] net: tcp: fix rx timestamp behavior for tcp_recvmsg Kelly Littlepage
2020-05-05 20:23 ` Willem de Bruijn [this message]
2020-05-07 21:40   ` Willem de Bruijn
2020-05-08  0:50     ` [PATCH v2] net: tcp: fixes commit 98aaa913b4ed ("tcp: Extend SOF_TIMESTAMPING_RX_SOFTWARE to TCP recvmsg") Kelly Littlepage
2020-05-08 13:51       ` Willem de Bruijn
2020-05-08 14:23         ` [PATCH v3] net: tcp: fix rx timestamp behavior for tcp_recvmsg Kelly Littlepage
2020-05-08 14:45           ` Eric Dumazet
2020-05-08 14:56             ` Soheil Hassas Yeganeh
2020-05-08 15:31               ` Willem de Bruijn
2020-05-08 18:29           ` Jakub Kicinski
2020-05-08 19:58             ` [PATCH v4] " Kelly Littlepage
2020-05-08 23:17               ` Jakub Kicinski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='' \ \ \ \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.