From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.2 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3BADC10F00 for ; Sat, 23 Feb 2019 01:07:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 87E2E20700 for ; Sat, 23 Feb 2019 01:07:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=fb.com header.i=@fb.com header.b="PSX07Ck0" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727659AbfBWBHa (ORCPT ); Fri, 22 Feb 2019 20:07:30 -0500 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:54468 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727648AbfBWBH3 (ORCPT ); Fri, 22 Feb 2019 20:07:29 -0500 Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x1N13Kf4006329 for ; Fri, 22 Feb 2019 17:07:29 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=QLG66rWgrxCE17qC3jT5Gkv6zmMeppEQFp4+mzjL3cU=; b=PSX07Ck02YWbj4zPbOlxHmTRcLpoNVcAeQ+mTUKpqJhYdN1ffc6Y4LFckoz0OSzs5BMq ut7KGlRgxin4cJAILOc8iaIJVA552Z2r64HTNDMYRdsnRud2pXJ5iLFhWLMdhJGHvvwz 9hk+UxTXKiDe1bUHx3yh0ViDRvYUFqnMGzM= Received: from mail.thefacebook.com ([199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2qtu86r524-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Fri, 22 Feb 2019 17:07:29 -0800 Received: from mx-out.facebook.com (2620:10d:c081:10::13) by mail.thefacebook.com (2620:10d:c081:35::130) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1531.3; Fri, 22 Feb 2019 17:07:28 -0800 Received: by devbig009.ftw2.facebook.com (Postfix, from userid 10340) id 5ECB15AE1524; Fri, 22 Feb 2019 17:07:27 -0800 (PST) Smtp-Origin-Hostprefix: devbig From: brakmo Smtp-Origin-Hostname: devbig009.ftw2.facebook.com To: netdev CC: Martin Lau , Alexei Starovoitov , Daniel Borkmann , Eric Dumazet , Kernel Team Smtp-Origin-Cluster: ftw2c04 Subject: [PATCH v2 bpf-next 5/9] bpf: Add bpf helper bpf_tcp_check_probe_timer Date: Fri, 22 Feb 2019 17:06:59 -0800 Message-ID: <20190223010703.678070-6-brakmo@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190223010703.678070-1-brakmo@fb.com> References: <20190223010703.678070-1-brakmo@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 Content-Type: text/plain X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-02-23_01:,, signatures=0 X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This patch adds a new bpf helper BPF_FUNC_tcp_check_probe_timer "int bpf_check_tcp_probe_timer(struct tcp_bpf_sock *tp, u32 when_us)". It is added to BPF_PROG_TYPE_CGROUP_SKB typed bpf_prog which currently can be attached to the ingress and egress path. To ensure it is only called from BPF_CGROUP_INET_EGRESS, the attr->expected_attach_type must be specified as BPF_CGROUP_INET_EGRESS during load time if the prog uses this new helper. The newly added prog->enforce_expected_attach_type bit will also be set if this new helper is used. This bit is for backward compatibility reason because currently prog->expected_attach_type has been ignored in BPF_PROG_TYPE_CGROUP_SKB. During attach time, prog->expected_attach_type is only enforced if the prog->enforce_expected_attach_type bit is set. i.e. prog->expected_attach_type is only enforced if this new helper is used by the prog. The function forces when_us to be at least TCP_TIMEOUT_MIN (currently 2 jiffies) and no more than TCP_RTO_MIN (currently 200ms). When using a bpf_prog to limit the egress bandwidth of a cgroup, it can happen that we drop a packet of a connection that has no packets out. In this case, the connection may not retry sending the packet until the probe timer fires. Since the default value of the probe timer is at least 200ms, this can introduce link underutiliation (i.e. the cgroup egress bandwidth being smaller than the specified rate) thus increased tail latency. This helper function allows for setting a smaller probe timer. Signed-off-by: Lawrence Brakmo --- include/uapi/linux/bpf.h | 12 +++++++++++- net/core/filter.c | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+), 1 deletion(-) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index fc646f3eaf9b..5d0bed852800 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -2372,6 +2372,15 @@ union bpf_attr { * current value is ect (ECN capable). Works with IPv6 and IPv4. * Return * 1 if set, 0 if not set. + * + * int bpf_tcp_check_probe_timer(struct bpf_tcp_sock *tp, int when_us) + * Description + * Checks that there are no packets out and there is no pending + * timer. If both of these are true, it bounds when_us by + * TCP_TIMEOUT_MIN (2 jiffies) or TCP_RTO_MIN (200ms) and + * sets the probe timer. + * Return + * 0 */ #define __BPF_FUNC_MAPPER(FN) \ FN(unspec), \ @@ -2472,7 +2481,8 @@ union bpf_attr { FN(sk_fullsock), \ FN(tcp_sock), \ FN(tcp_enter_cwr), \ - FN(skb_ecn_set_ce), + FN(skb_ecn_set_ce), \ + FN(tcp_check_probe_timer), /* integer value in 'imm' field of BPF_CALL instruction selects which helper * function eBPF program intends to call diff --git a/net/core/filter.c b/net/core/filter.c index 955369c6ed30..7d7026768840 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -5456,6 +5456,31 @@ static const struct bpf_func_proto bpf_skb_ecn_set_ce_proto = { .ret_type = RET_INTEGER, .arg1_type = ARG_PTR_TO_CTX, }; + +BPF_CALL_2(bpf_tcp_check_probe_timer, struct tcp_sock *, tp, u32, when_us) +{ + struct sock *sk = (struct sock *) tp; + unsigned long when = usecs_to_jiffies(when_us); + + if (!tp->packets_out && !inet_csk(sk)->icsk_pending) { + if (when < TCP_TIMEOUT_MIN) + when = TCP_TIMEOUT_MIN; + else if (when > TCP_RTO_MIN) + when = TCP_RTO_MIN; + + tcp_reset_xmit_timer(sk, ICSK_TIME_PROBE0, + when, TCP_RTO_MAX, NULL); + } + return 0; +} + +static const struct bpf_func_proto bpf_tcp_check_probe_timer_proto = { + .func = bpf_tcp_check_probe_timer, + .gpl_only = false, + .ret_type = RET_INTEGER, + .arg1_type = ARG_PTR_TO_TCP_SOCK, + .arg2_type = ARG_ANYTHING, +}; #endif /* CONFIG_INET */ bool bpf_helper_changes_pkt_data(void *func) @@ -5624,6 +5649,13 @@ cg_skb_func_proto(enum bpf_func_id func_id, struct bpf_prog *prog) } case BPF_FUNC_skb_ecn_set_ce: return &bpf_skb_ecn_set_ce_proto; + case BPF_FUNC_tcp_check_probe_timer: + if (prog->expected_attach_type == BPF_CGROUP_INET_EGRESS) { + prog->enforce_expected_attach_type = 1; + return &bpf_tcp_check_probe_timer_proto; + } else { + return NULL; + } #endif default: return sk_filter_func_proto(func_id, prog); -- 2.17.1