From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <netdev-owner@kernel.org>
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on
	aws-us-west-2-korg-lkml-1.web.codeaurora.org
X-Spam-Level: 
X-Spam-Status: No, score=-6.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE,
	SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0
Received: from mail.kernel.org (mail.kernel.org [198.145.29.99])
	by smtp.lore.kernel.org (Postfix) with ESMTP id 43F81C2D0E4
	for <netdev@archiver.kernel.org>; Tue, 24 Nov 2020 19:24:34 +0000 (UTC)
Received: from vger.kernel.org (vger.kernel.org [23.128.96.18])
	by mail.kernel.org (Postfix) with ESMTP id BFDBC2087C
	for <netdev@archiver.kernel.org>; Tue, 24 Nov 2020 19:24:33 +0000 (UTC)
Authentication-Results: mail.kernel.org;
	dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="1Q71YsNA"
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S2390973AbgKXTYd (ORCPT <rfc822;netdev@archiver.kernel.org>);
        Tue, 24 Nov 2020 14:24:33 -0500
Received: from mail.kernel.org ([198.145.29.99]:52140 "EHLO mail.kernel.org"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S2390919AbgKXTYc (ORCPT <rfc822;netdev@vger.kernel.org>);
        Tue, 24 Nov 2020 14:24:32 -0500
Received: from kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com (unknown [163.114.132.1])
        (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
        (No client certificate requested)
        by mail.kernel.org (Postfix) with ESMTPSA id 83D11206B5;
        Tue, 24 Nov 2020 19:24:31 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org;
        s=default; t=1606245871;
        bh=26bRg9JP+0a2d2OvPj8wlUn5vRsch0gJj3neTfcrD0Q=;
        h=Date:From:To:Cc:Subject:In-Reply-To:References:From;
        b=1Q71YsNAx5T+3j5Air2cYa/uJZvuWZglGVBIM0cL7ZR8lIV06ITl5JBwlmCB9GOld
         8WyTkvN+7JpZr4v67NJeTuUuPlfyxnWfV5GeNPqcQQpbOtTtzsIrdCN++Nq0IIb+Yx
         /j41Iabd/GBwS8M87/IQcZgaCfXew2Te7x2it08M=
Date:   Tue, 24 Nov 2020 11:24:30 -0800
From:   Jakub Kicinski <kuba@kernel.org>
To:     wenxu@ucloud.cn
Cc:     marcelo.leitner@gmail.com, vladbu@nvidia.com, jhs@mojatatu.com,
        xiyou.wangcong@gmail.com, netdev@vger.kernel.org
Subject: Re: [PATCH v3 net-next 3/3] net/sched: sch_frag: add generic packet
 fragment support.
Message-ID: <20201124112430.64143482@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
In-Reply-To: <1605829116-10056-4-git-send-email-wenxu@ucloud.cn>
References: <1605829116-10056-1-git-send-email-wenxu@ucloud.cn>
        <1605829116-10056-4-git-send-email-wenxu@ucloud.cn>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Precedence: bulk
List-ID: <netdev.vger.kernel.org>
X-Mailing-List: netdev@vger.kernel.org

On Fri, 20 Nov 2020 07:38:36 +0800 wenxu@ucloud.cn wrote:
> +int tcf_dev_queue_xmit(struct sk_buff *skb, int (*xmit)(struct sk_buff *skb))
> +{
> +	xmit_hook_func *xmit_hook;
> +
> +	xmit_hook = rcu_dereference(tcf_xmit_hook);
> +	if (xmit_hook)
> +		return xmit_hook(skb, xmit);
> +	else
> +		return xmit(skb);
> +}
> +EXPORT_SYMBOL_GPL(tcf_dev_queue_xmit);

I'm concerned about the performance impact of these indirect calls.

Did you check what code compiler will generate? What the impact with
retpolines enabled is going to be?

Now that sch_frag is no longer a module this could be simplified.

First of all - xmit_hook can only be sch_frag_xmit_hook, so please use
that directly. 

	if (READ_ONCE(tcf_xmit_hook_count)) 
		sch_frag_xmit_hook(...
	else
		dev_queue_xmit(...

The abstraction is costly and not necessary right now IMO.

Then probably the counter should be:

	u32 __read_mostly tcf_xmit_hook_count;

To avoid byte loads and having it be places in an unlucky cache line.

You could also make the helper a static inline in a header.


Unless I'm not giving the compiler enough credit and the performance
impact of this patch with retpolines on is indiscernible, but that'd
need to be proven by testing...