From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44483C433DF for ; Mon, 6 Jul 2020 23:23:04 +0000 (UTC) Received: from dpdk.org (dpdk.org [92.243.14.124]) by mail.kernel.org (Postfix) with ESMTP id C04F220720 for ; Mon, 6 Jul 2020 23:23:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=intel.onmicrosoft.com header.i=@intel.onmicrosoft.com header.b="rJrPiCE5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C04F220720 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 9A82F1DC0A; Tue, 7 Jul 2020 01:23:02 +0200 (CEST) Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id 3F17C1DB9D for ; Tue, 7 Jul 2020 01:23:00 +0200 (CEST) IronPort-SDR: js60hKC9f97QRY9D/xb9ZWigzCl/UioQ9+dcafLWlX6lExs5e2r55vj46kHSlGJSb6HMTajKdZ vFV8hYzGYKCg== X-IronPort-AV: E=McAfee;i="6000,8403,9674"; a="145016098" X-IronPort-AV: E=Sophos;i="5.75,321,1589266800"; d="scan'208";a="145016098" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Jul 2020 16:22:59 -0700 IronPort-SDR: 6qFy6pxlkJ6rbNLmuJToG7kpwEoYLPjg3VmdSg88SsY9gEEUa8VVccJ9zwiIG2F1BUi1RTBuaQ xhHKe23UH80Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,321,1589266800"; d="scan'208";a="482857993" Received: from orsmsx105.amr.corp.intel.com ([10.22.225.132]) by fmsmga006.fm.intel.com with ESMTP; 06 Jul 2020 16:22:58 -0700 Received: from orsmsx163.amr.corp.intel.com (10.22.240.88) by ORSMSX105.amr.corp.intel.com (10.22.225.132) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 6 Jul 2020 16:22:58 -0700 Received: from ORSEDG002.ED.cps.intel.com (10.7.248.5) by ORSMSX163.amr.corp.intel.com (10.22.240.88) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 6 Jul 2020 16:22:58 -0700 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.176) by edgegateway.intel.com (134.134.137.101) with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 6 Jul 2020 16:22:39 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CM1uaGKZVixTuKEmTwOXjGDpVgiaDdfQAUrjhSIHLonIGJYiPKkBhU2HxCwzr8koUKuoN7O7Ygt62Kf/2j9yz11zfzV1ChQPBDYJUKQ/WQJYUAjKPeVxNrMiWC+XzrUx/KVFxWHD6smUlDidKp75WaMXbJyWjNNoT0UAplAnkLmIOBrzOIJZic1Jjk+Zppr8uX3PTaQITGU10QP2ZI873STUnijY17rRXbtW5YXtstBEHKANNkcYpDvWG9i/faHFky8zRkmpbRVcvQpcGBJrlS2L6q9voOol5lKoy+/2rosKO04YtswyzR7dYSn7UxSB6V3Cs6CccAW5t8f7Hi6ICQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RHfCBnpkSKOFkiW7lbhqDC3FT6nnkHgvLh6jZoq2v6U=; b=c0OI92TdmaVJHa+lpOomA2DDjRvUPjaxxNwbWzHqvgBA3/6w3flkpI7GJ81Kf0TeIdb7RqUQYBeM7kF1ShTKqo/HC8MgadA+93rbvUPs1h1Q8XbbWzcl6GNgs5eufcrmPYHG1ZlZqjnE9A79riWComt5HVNsgtXBPW/5D8ZYItF7CfNb0cx/gDYoIUCVACvq6hp61zDH1ubhghtVhBedKoOp6eHfQuydCGvwniOnR6Qd05PfEkVY+jLgSnHKlDd7+ED+oWHi/JMU7r1hXnb7QO+EYlVhrtrbbT0FO0+TlQJQWCUhzKMpEIab4kGjekTz/6yJJI+V9at8H8/py8w0IA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RHfCBnpkSKOFkiW7lbhqDC3FT6nnkHgvLh6jZoq2v6U=; b=rJrPiCE5EMK80LhljuQyuJzj9OnKpnxPTB4fA9iMQTnYN1rL7p9mNMEETH5AWNGw7DAqqOnyyp0V6EjT4qniWbu/hojKtC4EQEPu3N6sW87BqpEsGQpdLFnJDTZKIWbZ/HyLeXBs8IB4b+CAvzMhSkXtfNPOwDHbU6RltIzgfCU= Received: from BYAPR11MB3301.namprd11.prod.outlook.com (2603:10b6:a03:7f::26) by BY5PR11MB3896.namprd11.prod.outlook.com (2603:10b6:a03:187::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3153.27; Mon, 6 Jul 2020 23:22:28 +0000 Received: from BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f160:29ab:b8f9:4189]) by BYAPR11MB3301.namprd11.prod.outlook.com ([fe80::f160:29ab:b8f9:4189%6]) with mapi id 15.20.3153.029; Mon, 6 Jul 2020 23:22:28 +0000 From: "Ananyev, Konstantin" To: David Marchand , "dev@dpdk.org" CC: "jerinjacobk@gmail.com" , "Richardson, Bruce" , "mdr@ashroe.eu" , "thomas@monjalon.net" , "arybchenko@solarflare.com" , "ktraynor@redhat.com" , "Stokes, Ian" , "i.maximets@ovn.org" , "olivier.matz@6wind.com" Thread-Topic: [PATCH v6 00/10] Register non-EAL threads as lcore Thread-Index: AQHWU9dyliTRCOKW406YMG0EeP5T86j7LOwg Date: Mon, 6 Jul 2020 23:22:28 +0000 Message-ID: References: <20200610144506.30505-1-david.marchand@redhat.com> <20200706205234.8040-1-david.marchand@redhat.com> In-Reply-To: <20200706205234.8040-1-david.marchand@redhat.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 authentication-results: redhat.com; dkim=none (message not signed) header.d=none;redhat.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.151.179] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 3b6b633a-85b4-42a5-ccb3-08d82203735b x-ms-traffictypediagnostic: BY5PR11MB3896: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:6790; x-forefront-prvs: 04569283F9 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: M5GUsUNk8bv+w0PxManPicNrIotHHnflUMC3pCIoAOJdBLHRuDFs/wsQuF+dCxEYOgtBuxIv0LthADyJ/1Yvlulu0WUc2zIxRLlxf64Y6NlUl+Ox19B0jaAiXJ3weeAuE28wr9ZIZ9oUVh/Phqyh9IVzueTKgP8Lap5Fhg58j1QgEAnV4fdi71+8pvXgYnVyL9TvMeODDGm8EUt+7nm39CugDj/O1BQYkpjKVBUUspMd5A5MKFbsAUA+Am2dC1BY8PDKrXN1wa1gVVBS1IIWavhpGkv1rlgTpAVQ211gwiYwNljlSVNXd5T4I5mRDXzlj56+sdd0zvLID4c7Lp3sln18h/0nyVzpG0ncnsYy9Mx0SX3mSHasepMS3Pk4dT9jJxoP9QTT95S+keDx3YVU4g== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3301.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(396003)(346002)(376002)(366004)(39860400002)(136003)(5660300002)(54906003)(33656002)(52536014)(76116006)(478600001)(83380400001)(110136005)(86362001)(966005)(4326008)(316002)(186003)(7696005)(8936002)(71200400001)(8676002)(66476007)(66946007)(9686003)(66556008)(64756008)(66446008)(26005)(2906002)(6506007)(55016002); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: bDKMCXRsvEuxXu7MvV9vkK7RKI+KLINkKxcuRyZCd+b27FRhVxOfac4pslrJZkI3qj2/MxRFuAfRgQJI1RaDBOzj5gv/7kegDb4c93z6KbJK0sfst+zBEHiWmkO82LRCprjqytN1b6/Vbmfg5nARkeFt8nRtGrzIio9Obd1vyVa9vVKXyll8hh1dXJ5UdhnfkfsiRYLUSd2qxmryRPJelSbqHzSethejmbkuUN+KWafqAX1lDhH/penp7W+rI0jlVnvFuMTwtxaMZukHLSOptJE1p8fvjLErN16hcIm3wEUxuLzJASf5trHS8zTEO9nXIUbeTUpxONXyRmdscPE3ailKVlbRH0pY83OBgeTz8wMiR6WJ3Wo0zQ3DnfRAZsflFgp2P5RbQia68O6WvkXmNjwoBVEIc1ffunXcaWplPLEWHsweS/8cE+hNbWWgRFtpiQW1C6ktV0gLVd8DwMmFKWHtZH64IKpitDsmBbFmG3s9H65hCQI9pdg/ynL/e4KI Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3301.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3b6b633a-85b4-42a5-ccb3-08d82203735b X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Jul 2020 23:22:28.6168 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: gsEEIwyNWBX9pUUCRoXoGHj20e2h71E+AL/Drktd+va9VHezqDuOWIBMTTOhL9nKO6EPEyCeDRyzO48Ff32tuNgZMo1yq0xPjB+vIya2yJY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR11MB3896 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v6 00/10] Register non-EAL threads as lcore X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi David, =20 > OVS and some other applications have been hacking into DPDK internals to > fake EAL threads and avoid performance penalty of only having non-EAL > threads. >=20 > This series proposes to add a new type of lcores and maps those threads > to such lcores. > non-EAL threads won't run the DPDK eal mainloop. > As a consequence, part of the EAL threads API cannot work. >=20 > Having new lcores appearing during the process lifetime is not expected > by some DPDK components. This is addressed by introducing init/uninit > callacks invoked when hotplugging of such lcore. >=20 > There is still some work/discussion: > - refuse new lcore role in incompatible EAL threads API (or document it > only as those API were already incompatible?), > - think about deprecation notices for existing RTE_FOREACH_LCORE macros > and consorts, it is probably worth discussing on how to iterate over > lcores, >=20 > For the interested parties, I have a patch [1] against dpdk-latest OVS > branch that makes use of this series (this patch probably won't work with > v5, it will be rebased once dpdk side is ready). >=20 > 1: https://patchwork.ozlabs.org/project/openvswitch/patch/20200626123017.= 28555-1-david.marchand@redhat.com/ >=20 > Changes since v5: > - fixed windows build, >=20 > Changes since v4: > - added separate API to control mp feature activation, > - addressed Konstantin and Olivier comments, >=20 > Changes since v3: > - added init failure when trying to use in conjunction with multiprocess, > - addressed Andrew comments, >=20 > Changes since v2: > - fixed windows build error due to missing trace stub, > - fixed bug when rolling back on lcore register, >=20 > Changes since v1: > - rebased on master (conflicts on merged Windows series), > - separated lcore role code cleanup in a patch, > - tried to use a single naming, so kept non-EAL threads as the main > notion. non-EAL threads are then distinguished between registered and > unregistered non-EAL threads, > - added unit tests (still missing some coverage, marked with a FIXME), > - reworked callbacks call under a common rwlock lock which protects > lcores allocations and callbacks registration, > - introduced lcore iterators and converted the bucket mempool driver, >=20 LGTM, just 2 nits see below. Apart from that: Series Acked-by: Konstantin Ananyev 1. +void +rte_lcore_callback_unregister(void *handle) +{ + struct rte_config *cfg =3D rte_eal_get_configuration(); + struct lcore_callback *callback =3D handle; + unsigned int lcore_id; Seems like forgot to add formal parameter check here: if (callback =3D=3D NULL) ...=20 + + rte_rwlock_write_lock(&lcore_lock); + if (callback->uninit =3D=3D NULL) 2. +bool +rte_mp_disable(void) +{ + return set_mp_status(MP_STATUS_DISABLED); +} Probably name it rte_eal_multiprocess_enable (or so) to make it clear from naming and follow more closely our own name convention. + +bool +eal_enable_multiprocess(void) +{ + return set_mp_status(MP_STATUS_ENABLED); +} Might be worth to make that function public too. Then user will have a proper pair to use: rte_eal_multiprocess_(enable|disable).