From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C10AC3276E for ; Thu, 2 Jan 2020 22:17:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 401DD21582 for ; Thu, 2 Jan 2020 22:17:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1578003424; bh=/GNAuNFE5e987/ISVIjYsKkE1UFI+Kx0wFE1vVGjSA0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=Gl/LQ9gnPdfZMnIb2jyhSuUAlqfEEqaY9g/ivlADx7WRctHtz6cPXpcmrdMkrg2Iz H5kPsUXV05QHRknYaS3sJ27JphmYkgNHinsUbHX7Do2msjftrMoNNG+7IgVpKB8kUX A6QjA9xBgyjv6gfggBlyivRBZ0eMqm9h6j+7LezI= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727392AbgABWRD (ORCPT ); Thu, 2 Jan 2020 17:17:03 -0500 Received: from mail.kernel.org ([198.145.29.99]:59192 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727905AbgABWQ6 (ORCPT ); Thu, 2 Jan 2020 17:16:58 -0500 Received: from localhost (83-86-89-107.cable.dynamic.v4.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 00069227BF; Thu, 2 Jan 2020 22:16:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1578003418; bh=/GNAuNFE5e987/ISVIjYsKkE1UFI+Kx0wFE1vVGjSA0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uay4+HsLjWkVjZcBfMFFLdZxT7OK2ioOaUo58VS92fmPfkPVe7DxuKAk3d29znTTJ 2siwinEvLnMB/Wt0jFDfB85znkxtgLGq7D8rckg+Lvuhw9yh3Fot7ykr0LHaAe1/jv UnQewgK7/0O2rMkjwvlUe5kVlLQTUbVxPyV7B/Y0= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Eric Dumazet , Soheil Hassas Yeganeh , Neal Cardwell , "David S. Miller" Subject: [PATCH 5.4 153/191] net_sched: sch_fq: properly set sk->sk_pacing_status Date: Thu, 2 Jan 2020 23:07:15 +0100 Message-Id: <20200102215845.838130304@linuxfoundation.org> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200102215829.911231638@linuxfoundation.org> References: <20200102215829.911231638@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Eric Dumazet [ Upstream commit bb3d0b8bf5be61ab1d6f472c43cbf34de17e796b ] If fq_classify() recycles a struct fq_flow because a socket structure has been reallocated, we do not set sk->sk_pacing_status immediately, but later if the flow becomes detached. This means that any flow requiring pacing (BBR, or SO_MAX_PACING_RATE) might fallback to TCP internal pacing, which requires a per-socket high resolution timer, and therefore more cpu cycles. Fixes: 218af599fa63 ("tcp: internal implementation for pacing") Signed-off-by: Eric Dumazet Cc: Soheil Hassas Yeganeh Cc: Neal Cardwell Acked-by: Soheil Hassas Yeganeh Signed-off-by: David S. Miller Signed-off-by: Greg Kroah-Hartman --- net/sched/sch_fq.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) --- a/net/sched/sch_fq.c +++ b/net/sched/sch_fq.c @@ -301,6 +301,9 @@ static struct fq_flow *fq_classify(struc f->socket_hash != sk->sk_hash)) { f->credit = q->initial_quantum; f->socket_hash = sk->sk_hash; + if (q->rate_enable) + smp_store_release(&sk->sk_pacing_status, + SK_PACING_FQ); if (fq_flow_is_throttled(f)) fq_flow_unset_throttled(q, f); f->time_next_packet = 0ULL; @@ -322,8 +325,12 @@ static struct fq_flow *fq_classify(struc fq_flow_set_detached(f); f->sk = sk; - if (skb->sk == sk) + if (skb->sk == sk) { f->socket_hash = sk->sk_hash; + if (q->rate_enable) + smp_store_release(&sk->sk_pacing_status, + SK_PACING_FQ); + } f->credit = q->initial_quantum; rb_link_node(&f->fq_node, parent, p); @@ -428,17 +435,9 @@ static int fq_enqueue(struct sk_buff *sk f->qlen++; qdisc_qstats_backlog_inc(sch, skb); if (fq_flow_is_detached(f)) { - struct sock *sk = skb->sk; - fq_flow_add_tail(&q->new_flows, f); if (time_after(jiffies, f->age + q->flow_refill_delay)) f->credit = max_t(u32, f->credit, q->quantum); - if (sk && q->rate_enable) { - if (unlikely(smp_load_acquire(&sk->sk_pacing_status) != - SK_PACING_FQ)) - smp_store_release(&sk->sk_pacing_status, - SK_PACING_FQ); - } q->inactive_flows--; }