From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E130C38A24 for ; Thu, 7 May 2020 16:12:43 +0000 (UTC) Received: from dpdk.org (dpdk.org [92.243.14.124]) by mail.kernel.org (Postfix) with ESMTP id B20C62083B for ; Thu, 7 May 2020 16:12:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=intel.onmicrosoft.com header.i=@intel.onmicrosoft.com header.b="Z5klEkEj" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B20C62083B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0CC4C1DC7B; Thu, 7 May 2020 18:12:41 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id 0FD251DC2A for ; Thu, 7 May 2020 18:12:38 +0200 (CEST) IronPort-SDR: fLH2zGl9rdZSoIdW6axlYweZcEn0djGnKO4WVBRZXiG3nyhWJ5lS3MScEGhozK/3gp/5GU/JK4 tn/iY7C/30Zg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 May 2020 09:12:18 -0700 IronPort-SDR: K5tyIiuw3pLc8kmJKCBAQT2iQg4+BeUM9rfjzYIlBW5MKXTb2f0c6QJu1bz6ndsDDghMmrMbP0 s824s3MIk60Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,364,1583222400"; d="scan'208";a="305160293" Received: from fmsmsx105.amr.corp.intel.com ([10.18.124.203]) by FMSMGA003.fm.intel.com with ESMTP; 07 May 2020 09:12:18 -0700 Received: from fmsmsx120.amr.corp.intel.com (10.18.124.208) by FMSMSX105.amr.corp.intel.com (10.18.124.203) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 7 May 2020 09:12:18 -0700 Received: from FMSEDG002.ED.cps.intel.com (10.1.192.134) by fmsmsx120.amr.corp.intel.com (10.18.124.208) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 7 May 2020 09:12:17 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.104) by edgegateway.intel.com (192.55.55.69) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 7 May 2020 09:12:17 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=igt5CBmKCBIKCeG7fwVV5qOCzgheZ7kyllhNqDPfhlmC3BvLufU8NWFrfizG2vMzZZsGq9OgBgk1KYo0cJEzlwdtbSSiiIdfL4Dxqrx81OEL5j+7Akk0G3TkMUKczq/nG3R96TF0IHfYsVBoN+TUBo0XigWwU8U6GwNaV+SbW+jOzeVr4UDDhPsoWVozYryFgm8jX3jVThPcNSGXGUii+aAUZaLcwFCqBeuw/C+3jjkLFGIjwnJ2zsZfLnGqqTf5QRfK9NiuZ6yeIWKbmky7kLmsRBSNFuMLvdaAPKKH7FzRbS9a4jFVcxm2g/Eoz8E+NyPST8xcwnszLMW3GCdyuw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4cn7+dJYYQsTzwZg+mGkqUFmhbClgI/Bh9w8n/63b1g=; b=AB+Fk63WvXxcUY0caAMMvu0T/5kgNYYXE86xyGwc8FMlvYnKqCkY7SD+xk5elgID6c7NUwzknEIHNh+vfB5f72vO4eUUOv+Fh44rsJ5v2RXwRW6wJ3O8zZg3ohzGeSuCpZxx/yZHrge7D+MoonlJPwJYofI5gvJ3jpNgymBoHUVlur1OfYzszx51qo3bsMFF//4cIzMw8bl7fl6ijwI5SsDpCeUSJJj65gGxVgETdKk7A9RwmrU3VWSwc1xKQcM2soKmD+RjC25DrprlV3/Seo/QGRiAM/Bzs8y1cPKJM9w3uSdTNDEI84hrJa8Du5BHlI1MoJ2eJABLXK+jOb06KQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4cn7+dJYYQsTzwZg+mGkqUFmhbClgI/Bh9w8n/63b1g=; b=Z5klEkEjhjo78h558ZI+CeR7bKu8GnPL5ZAYrb/OwXUh4j719Zz1CkqkUXYriuDTp6kur2Bd4Pyj60sjX6hNinUwhD6eifBBZV14ewCHCuVCdNzOxcN6xFjPXi6Cpkm3so0/lWGcTQUheF5a6hgwMnEmL6Xw5XAfln9cmJxB69M= Received: from BYAPR11MB3301.namprd11.prod.outlook.com (2603:10b6:a03:7f::26) by BYAPR11MB2920.namprd11.prod.outlook.com (2603:10b6:a03:82::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2958.21; Thu, 7 May 2020 16:12:16 +0000 Received: from BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f8cb:58cd:e958:fff4]) by BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f8cb:58cd:e958:fff4%6]) with mapi id 15.20.2958.030; Thu, 7 May 2020 16:12:15 +0000 From: "Ananyev, Konstantin" To: Anoob Joseph , Akhil Goyal , "Nicolau, Radu" CC: Narayana Prasad , "dev@dpdk.org" Thread-Topic: [PATCH v3] examples/ipsec-secgw: add per core packet stats Thread-Index: AQHWI6SkbuuME/sCGUWlN/0x2cLQmKicvv3g Date: Thu, 7 May 2020 16:12:15 +0000 Message-ID: References: <1587647245-10524-1-git-send-email-anoobj@marvell.com> <1588769253-10405-1-git-send-email-anoobj@marvell.com> In-Reply-To: <1588769253-10405-1-git-send-email-anoobj@marvell.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 authentication-results: marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.151.188] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 1962d88d-f0dd-4ca6-0bb5-08d7f2a168df x-ms-traffictypediagnostic: BYAPR11MB2920: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-forefront-prvs: 03965EFC76 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: vAJaj/no6XCiS55N9eGzO1sMTthq45mep3qS4Svttm3FKE82GvQCD8wH55c0uxVLb13vM1ic202eB/eDUoFXkBBXeEYAzIvxQSNSSlHxvGLs+MnUBCr9FEUsAUCQHWVoY7Bk+Xb1y9gUSd6k21hYJUk0Zt4TFRjvZYy7s9j/IeuRyXb7I18QORndXm/W+zDwaepOAq3cGuxo1rcOCKtzIy+YJAA54rg/qieBD0HLFj0AuBhkgowkGVBd6+JxHKDF6NkIPC2PDzRVQMaV0gi6ivDIbSA71ZnxzXMjGMA5XGyToRJASt0jJGd6++CcPvNwKakQEqpyHdZZvk2W9YBtgZYUn+7rWD36WuEhHdz3Zk20zrEikeGw1cylAL//fhFBinO6824M/bidH/7yT3rZZ75GRI7niZxLWs+ttzJqkQwMHa69PestdqQ6JoUXiocu7Di9szdYJV80pJsAqvGyd+gJq5R+wOLj83AAEDKs4RVonzle55YnKWeKw8zvbbP1F3scFEbT++Yu+MPSkc35pA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3301.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(136003)(396003)(346002)(376002)(366004)(39860400002)(33430700001)(86362001)(52536014)(4326008)(54906003)(478600001)(110136005)(33440700001)(9686003)(6636002)(316002)(55016002)(5660300002)(33656002)(76116006)(71200400001)(2906002)(66556008)(7696005)(66476007)(66946007)(83280400001)(83310400001)(83290400001)(8936002)(64756008)(66446008)(26005)(83320400001)(186003)(8676002)(6506007)(83300400001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: jrMuPQHoTURPEjgKwTx8dhFQScOPooOGlY7I4r7BODQULvRvRLFnMLvoCSqF6IUbn7Qxh2FJV//yGl09RcTkuVjd2Mo00J8bwLjR5H2BuniOS4Z4m1dWwVk+5ZKLYjz+O1MFxPn69GfF7eAayamNlfGmcUnDKUmmmuTneIpPGBZ7Hjnr3YP25MUUOABOAZSZSL8ALgeKfZ1uUx066qvjMEvHdIiYhkpsRJjvKjXHy191Gm+fs6jleDnD5iIwD3JXyPMpGGOY/L5Fi6hrszp3JuRIJE1RiVG+Tdy6rYk+SdLQ/uI6o317gVO+WRQcRamv1rkfKvS5GMBy8/xpqiOQ2z4N7lPS7WaZk05hebpfbFOqZPF7U8mNk/Otjuot6R0VpZbDFx7rPH469herFWJag1XLQwxFH616V+o4K6JlYKg2kghjoc4W1QaIIWZjZZXhDhfk+zsS//ovk/xN67u9DDn8BBQdVtxheHukE+BQ146uGER3fNqh638sqvAwfc6RTTQrMjChEEzyTmLob5CPLxx8OT0EHn76o8gOgEpEoVMkcpqVKZeLrWudZNtfAVf+AAQ3CxFC8kcVll5A7Asfb5DKb/mh0ZhLLNwz2DUrXhapDvBrK+sa6Zt4CimjEoIN5l9Ye3/fvNIw2kT0V0NccnKrL/Pw3H22p/ecvgA4Q/qWAAJBl7fxEgvJ1F8EaYFcvhbGnEtypq+RjIf6wTKwKoYDz97fvMDPd14/HfyVl65gxmxJwWPZOxJ5h8lNmprhW7utsRnLZh32xkMgnSgVQ8mksLGkah4/t/WlnDA0stYy5qEI7+/3cRZwbYw2YcpB Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 1962d88d-f0dd-4ca6-0bb5-08d7f2a168df X-MS-Exchange-CrossTenant-originalarrivaltime: 07 May 2020 16:12:15.7889 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: VEYH3QmAghKnt3Xhq98uQqzbiPBXPXg9j62YBjxvDo+jwrWi2N5DjZtLYFxqLjyZvClTpvZSzmQbv7hJsMW1X+de467Pru1A9xq7VGCyYGE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB2920 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v3] examples/ipsec-secgw: add per core packet stats X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > @@ -1099,6 +1151,10 @@ ipsec_poll_mode_worker(void) > const uint64_t drain_tsc =3D (rte_get_tsc_hz() + US_PER_S - 1) > / US_PER_S * BURST_TX_DRAIN_US; > struct lcore_rx_queue *rxql; > +#if (STATS_INTERVAL > 0) > + const uint64_t timer_period =3D STATS_INTERVAL * rte_get_timer_hz(); > + uint64_t timer_tsc =3D 0; > +#endif /* STATS_INTERVAL */ >=20 > prev_tsc =3D 0; > lcore_id =3D rte_lcore_id(); > @@ -1159,6 +1215,19 @@ ipsec_poll_mode_worker(void) > drain_tx_buffers(qconf); > drain_crypto_buffers(qconf); > prev_tsc =3D cur_tsc; > +#if (STATS_INTERVAL > 0) > + if (lcore_id =3D=3D rte_get_master_lcore()) { > + /* advance the timer */ > + timer_tsc +=3D diff_tsc; > + > + /* if timer has reached its timeout */ > + if (unlikely(timer_tsc >=3D timer_period)) { > + print_stats(); > + /* reset the timer */ > + timer_tsc =3D 0; > + } > + } > +#endif /* STATS_INTERVAL */ I still don't understand why to do it in data-path thread. As I said in previous comments, in DPDK there is a control thread that can be used for such house-keeping tasks. Why not to use it (via rte_alarm or so) and keep data-path threads less affected. > } >=20 > for (i =3D 0; i < qconf->nb_rx_queue; ++i) { > @@ -1169,8 +1238,10 @@ ipsec_poll_mode_worker(void) > nb_rx =3D rte_eth_rx_burst(portid, queueid, > pkts, MAX_PKT_BURST); >=20 > - if (nb_rx > 0) > + if (nb_rx > 0) { > + core_stats_update_rx(nb_rx); > process_pkts(qconf, pkts, nb_rx, portid); > + } >=20 > /* dequeue and process completed crypto-ops */ > if (is_unprotected_port(portid)) > diff --git a/examples/ipsec-secgw/ipsec-secgw.h b/examples/ipsec-secgw/ip= sec-secgw.h > index 4b53cb5..5b3561f 100644 > --- a/examples/ipsec-secgw/ipsec-secgw.h > +++ b/examples/ipsec-secgw/ipsec-secgw.h > @@ -6,6 +6,8 @@ >=20 > #include >=20 > +#define STATS_INTERVAL 0 Shouldn't it be: #ifndef STATS_INTERVAL #define STATS_INTERVAL 0 #endif ? To allow user specify statis interval via EXTRA_CFLAGS=3D'-DSTATS_INTERVAL= =3D10' or so. > + > #define NB_SOCKETS 4 >=20 > #define MAX_PKT_BURST 32 > @@ -69,6 +71,17 @@ struct ethaddr_info { > uint64_t src, dst; > }; >=20 > +#if (STATS_INTERVAL > 0) > +struct ipsec_core_statistics { > + uint64_t tx; > + uint64_t rx; > + uint64_t dropped; > + uint64_t burst_rx; > +} __rte_cache_aligned; > + > +struct ipsec_core_statistics core_statistics[RTE_MAX_LCORE]; > +#endif /* STATS_INTERVAL */ > + > extern struct ethaddr_info ethaddr_tbl[RTE_MAX_ETHPORTS]; >=20 > /* Port mask to identify the unprotected ports */ > @@ -85,4 +98,59 @@ is_unprotected_port(uint16_t port_id) > return unprotected_port_mask & (1 << port_id); > } >=20 > +static inline void > +core_stats_update_rx(int n) > +{ > +#if (STATS_INTERVAL > 0) > + int lcore_id =3D rte_lcore_id(); > + core_statistics[lcore_id].rx +=3D n; > + if (n =3D=3D MAX_PKT_BURST) > + core_statistics[lcore_id].burst_rx +=3D n; > +#else > + RTE_SET_USED(n); > +#endif /* STATS_INTERVAL */ > +} > + > +static inline void > +core_stats_update_tx(int n) > +{ > +#if (STATS_INTERVAL > 0) > + int lcore_id =3D rte_lcore_id(); > + core_statistics[lcore_id].tx +=3D n; > +#else > + RTE_SET_USED(n); > +#endif /* STATS_INTERVAL */ > +} > + > +static inline void > +core_stats_update_drop(int n) > +{ > +#if (STATS_INTERVAL > 0) > + int lcore_id =3D rte_lcore_id(); > + core_statistics[lcore_id].dropped +=3D n; > +#else > + RTE_SET_USED(n); > +#endif /* STATS_INTERVAL */ > +} > + > +/* helper routine to free bulk of packets */ > +static inline void > +free_pkts(struct rte_mbuf *mb[], uint32_t n) > +{ > + uint32_t i; > + > + for (i =3D 0; i !=3D n; i++) > + rte_pktmbuf_free(mb[i]); > + > + core_stats_update_drop(n); > +} > + > +/* helper routine to free single packet */ > +static inline void > +free_pkt(struct rte_mbuf *mb) > +{ > + rte_pktmbuf_free(mb); > + core_stats_update_drop(1); Probably just: free_pkts(&mb, 1); ? > +} > + > #endif /* _IPSEC_SECGW_H_ */