From mboxrd@z Thu Jan 1 00:00:00 1970 From: Neal Cardwell Subject: Re: Linux ECN Handling Date: Wed, 20 Dec 2017 15:17:04 -0500 Message-ID: References: <20171019124312.GE16796@breakpoint.cc> <5A006CF6.1020608@iogearbox.net> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Cc: Eric Dumazet , Yuchung Cheng , Daniel Borkmann , Netdev , Florian Westphal , Mohammad Alizadeh , Lawrence Brakmo To: Steve Ibanez Return-path: Received: from mail-wr0-f176.google.com ([209.85.128.176]:38600 "EHLO mail-wr0-f176.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932496AbdLTUR1 (ORCPT ); Wed, 20 Dec 2017 15:17:27 -0500 Received: by mail-wr0-f176.google.com with SMTP id o2so23326083wro.5 for ; Wed, 20 Dec 2017 12:17:27 -0800 (PST) In-Reply-To: Sender: netdev-owner@vger.kernel.org List-ID: On Wed, Dec 20, 2017 at 2:20 PM, Steve Ibanez wrote: > > Hi Neal, > > I added in some more printk statements and it does indeed look like > all of these calls you listed are being invoked successfully. I guess > this isn't too surprising given what the inet_csk_schedule_ack() and > inet_csk_ack_scheduled() functions are doing: > > static inline void inet_csk_schedule_ack(struct sock *sk) > { > inet_csk(sk)->icsk_ack.pending |= ICSK_ACK_SCHED; > } > > static inline int inet_csk_ack_scheduled(const struct sock *sk) > { > return inet_csk(sk)->icsk_ack.pending & ICSK_ACK_SCHED; > } > > So through the code path that you listed, the inet_csk_schedule_ack() > function sets the ICSK_ACK_SCHED bit and then the tcp_ack_snd_check() > function just checks that the ICSK_ACK_SCHED bit is indeed set. >inet_csk_schedule_ack > Do you know how I can verify that setting the ICSK_ACK_SCHED bit > actually results in an ACK being sent? Hmm. I don't think in this case we can verify that setting the ICSK_ACK_SCHED bit actually results in an ACK being sent. Because AFAICT in this case it seems like an ACK is not sent. :-) This is based on both the tcpdumps on Dec 5 and your detective work yesterday ("The tcp_rcv_established() function calls tcp_ack_snd_check() at the end of step5 and then the return statement indicated below is invoked, which prevents the __tcp_ack_snd_check() function from running.") So AFAICT the puzzle is: how is the icsk_ack.pending ICSK_ACK_SCHED bit being cleared between the inet_csk_schedule_ack() call and the tcp_ack_snd_check() call, without (apparently) an actual ACK being sent on the wire? AFAICT the ICSK_ACK_SCHED bit is not supposed to be cleared unless we get to this sequence: tcp_transmit_skb() if (likely(tcb->tcp_flags & TCPHDR_ACK)) tcp_event_ack_sent(sk, tcp_skb_pcount(skb)); -> inet_csk_clear_xmit_timer(sk, ICSK_TIME_DACK); icsk->icsk_ack.blocked = icsk->icsk_ack.pending = 0; I don't have a theory that fits all of those data points, unless this is a bi-directional transfer (is it?) and between the inet_csk_schedule_ack() call and the tcp_ack_snd_check() call the TCP connection sends a data packet (in tcp_data_snd_check()) and then it is dropped for some reason before the packet make it to the tcpdump sniffing point. Perhaps because of a qdisc limit or something? I guess a possible next step would be, while processing an incoming skb with the cwr bit set, the code could set a new debugging field in the tcp_sock (tp->processing_cwr), and then you could check this field in tcp_transmit_skb() and printk if (1) there is an attempted queue_xmit() cal and (2) if the queue_xmit() fails (the err > 0 case). That's a long shot, but the only idea I have at this point. thanks, neal