From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9F8BCA9EAF for ; Mon, 21 Oct 2019 17:59:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B5AE02089C for ; Mon, 21 Oct 2019 17:59:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="VQYM3XE2" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728613AbfJUR7Z (ORCPT ); Mon, 21 Oct 2019 13:59:25 -0400 Received: from mail-qt1-f195.google.com ([209.85.160.195]:45674 "EHLO mail-qt1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727211AbfJUR7Z (ORCPT ); Mon, 21 Oct 2019 13:59:25 -0400 Received: by mail-qt1-f195.google.com with SMTP id c21so22324227qtj.12 for ; Mon, 21 Oct 2019 10:59:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=qTXoN8DCBGrFsp6FGfmWDhOfzDwC9R9U4AbtM6Ud4RI=; b=VQYM3XE2e9vlSKbQAnbJfNlkSmKh9vWxBYD6O5c6J6ZCFGLbMOpEgE9xZT+veN9KA9 lp8y+/hnRJSNGrKlhnsXoL9bcRAIXGJvq0HwBXgp4sHuD039nu8TmDfvEScsfVSo4CG2 K5v+Ax1wFzl12GV/6B99FsorGsspTTOy3stU6u5brECD/ngF4tz7hNQHl3bM1ye1EkHf 5rb2OH0pF/vvb4H1BpBKlJoo122t+fVcAkRdeXCGzTz9N9ajfnpUh3V+0S+ATSHq7Ggt FvAe5qS3/xdQHkgCTRztDlZzdEz8HsgvRLPtUtpUQr8CYzI9ivHBUGml+5LifV6YVmG0 gUew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=qTXoN8DCBGrFsp6FGfmWDhOfzDwC9R9U4AbtM6Ud4RI=; b=iK/zlCa2rbUVn49I/ySMBNSEmUuo8IXSplvaxWgUd5AUAskA2b+TEXcfIul1dFCsZ+ zSAwOgRxGDD3Hcwvs96y6vlhBUfgfyy0gls1Vk113Koll7LKEyftvZC6I5ILXlhDN51+ emdMpnY6mUeHcQaX5H231i0727qYsV3BP102QGqxMjRHmdXg7NbiK8QSaeIE3VdxKdIW NrZ52tLXrl9Bb6iFoB+8d+mqeFgW2/k9UFrsHKX6AwUTrJUrUv2kmqB5xzsLGZ1hjrDR XCq7rNQOasCl4l+45g61+4qOFPzv+ziyNhuSt6TSoQMYvEC94TmTShDozVV7w0bOxe2B iZXQ== X-Gm-Message-State: APjAAAVgeIilHHwmvW7ia2MO7rvCs1umgAxCaAG8p3V355jKrQ0cgGSd OWOXnpDxD0cUjZiLM36KDZgIaxxmAg0hjOwL97Q= X-Google-Smtp-Source: APXvYqybsIgL1jG66ZLAXek4GPYDYlb77LPF3UmlxbnygEHLjDLXWDLw024TND3ply1KFrkp6SO21nriMI4aYTFR7Xk= X-Received: by 2002:ac8:28e3:: with SMTP id j32mr20462368qtj.212.1571680764235; Mon, 21 Oct 2019 10:59:24 -0700 (PDT) MIME-Version: 1.0 References: <1571135440-24313-1-git-send-email-xiangxia.m.yue@gmail.com> <1571135440-24313-6-git-send-email-xiangxia.m.yue@gmail.com> In-Reply-To: From: William Tu Date: Mon, 21 Oct 2019 10:58:43 -0700 Message-ID: Subject: Re: [ovs-dev] [PATCH net-next v4 05/10] net: openvswitch: optimize flow-mask looking up To: Tonghao Zhang Cc: Greg Rose , pravin shelar , "" , Linux Kernel Network Developers Content-Type: text/plain; charset="UTF-8" Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org > > Hi Tonghao, > > > > Does this improve performance? After all, the original code simply > > check whether the mask is NULL, then goto next mask. > I tested the performance, but I disable the mask cache, and use the > dpdk-pktgen to generate packets: > The test ovs flow: > ovs-dpctl add-dp system@ovs-system > ovs-dpctl add-if system@ovs-system eth6 > ovs-dpctl add-if system@ovs-system eth7 > > for m in $(seq 1 100 | xargs printf '%.2x\n'); do > ovs-dpctl add-flow ovs-system > "in_port(1),eth(dst=00:$m:00:00:00:00/ff:ff:ff:ff:ff:$m),eth_type(0x0800),ipv4(frag=no)" > 2 > done > > ovs-dpctl add-flow ovs-system > "in_port(1),eth(dst=98:03:9b:6e:4a:f5/ff:ff:ff:ff:ff:ff),eth_type(0x0800),ipv4(frag=no)" > 2 > ovs-dpctl add-flow ovs-system > "in_port(2),eth(dst=98:03:9b:6e:4a:f4/ff:ff:ff:ff:ff:ff),eth_type(0x0800),ipv4(frag=no)" > 1 > > for m in $(seq 101 160 | xargs printf '%.2x\n'); do > ovs-dpctl add-flow ovs-system > "in_port(1),eth(dst=00:$m:00:00:00:00/ff:ff:ff:ff:ff:$m),eth_type(0x0800),ipv4(frag=no)" > 2 > done > > for m in $(seq 1 100 | xargs printf '%.2x\n'); do > ovs-dpctl del-flow ovs-system > "in_port(1),eth(dst=00:$m:00:00:00:00/ff:ff:ff:ff:ff:$m),eth_type(0x0800),ipv4(frag=no)" > done > > Without this patch: 982481pps (64B) > With this patch: 1112495 pps (64B), about 13% improve > Hi Tonghao, Thanks for doing the measurement. Based on the result (skipping 100 NULL mask lookup with 13% improvement), and with additional overhead of mask cache being invalidate while refilling these 100 gap, I'd argue that this patch is not necessary. But let's wait for others comments. Regards, William > > However, with your patch, isn't this invalidated the mask cache entry which > > point to the "M" you swap to the front? See my commands inline. > > > > > > > > Signed-off-by: Tonghao Zhang > > > Tested-by: Greg Rose > > > --- > > > static struct table_instance *table_instance_expand(struct table_instance *ti, > > > @@ -704,21 +704,33 @@ static struct table_instance *table_instance_expand(struct table_instance *ti, > > > return table_instance_rehash(ti, ti->n_buckets * 2, ufid); > > > } > > > > > > -static void tbl_mask_array_delete_mask(struct mask_array *ma, > > > - struct sw_flow_mask *mask) > > > +static void tbl_mask_array_del_mask(struct flow_table *tbl, > > > + struct sw_flow_mask *mask) > > > { > > > - int i; > > > + struct mask_array *ma = ovsl_dereference(tbl->mask_array); > > > + int i, ma_count = READ_ONCE(ma->count); > > > > > > /* Remove the deleted mask pointers from the array */ > > > - for (i = 0; i < ma->max; i++) { > > > - if (mask == ovsl_dereference(ma->masks[i])) { > > > - RCU_INIT_POINTER(ma->masks[i], NULL); > > > - ma->count--; > > > - kfree_rcu(mask, rcu); > > > - return; > > > - } > > > + for (i = 0; i < ma_count; i++) { > > > + if (mask == ovsl_dereference(ma->masks[i])) > > > + goto found; > > > } > > > + > > > BUG(); > > > + return; > > > + > > > +found: > > > + WRITE_ONCE(ma->count, ma_count -1); > > > + > > > + rcu_assign_pointer(ma->masks[i], ma->masks[ma_count -1]); > > > + RCU_INIT_POINTER(ma->masks[ma_count -1], NULL); > > > > So when you swap the ma->masks[ma_count -1], the mask cache entry > > who's 'mask_index == ma_count' become all invalid? > Yes, a little tricky. > > Regards, > > William > >