From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F90BC433E0 for ; Wed, 17 Feb 2021 12:09:28 +0000 (UTC) Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by mail.kernel.org (Postfix) with ESMTP id 902D661481 for ; Wed, 17 Feb 2021 12:09:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 902D661481 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=dev-bounces@dpdk.org Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 56E5340690; Wed, 17 Feb 2021 13:09:26 +0100 (CET) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id BA6D04067A for ; Wed, 17 Feb 2021 13:09:24 +0100 (CET) IronPort-SDR: IarsZ41x4Fz1+N5heSYQ5K5TR4rQe6o9+GO3xeiDLG5jpWyVBUvLRi8zLij6bftKQ92vtPuBms Myhf0objzATg== X-IronPort-AV: E=McAfee;i="6000,8403,9897"; a="247245603" X-IronPort-AV: E=Sophos;i="5.81,184,1610438400"; d="scan'208";a="247245603" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Feb 2021 04:09:23 -0800 IronPort-SDR: O16NdPYuUOUkroFy+S9DDjVT6dLkhsOGuGHmc3Ebg9/fMfZxav/ihlcCKOEyeahE1h2yd8CG+1 5/PEOVdIk3QA== X-IronPort-AV: E=Sophos;i="5.81,184,1610438400"; d="scan'208";a="399930123" Received: from aburakov-mobl.ger.corp.intel.com (HELO [10.213.233.84]) ([10.213.233.84]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Feb 2021 04:09:21 -0800 To: "Van Haaren, Harry" , "Richardson, Bruce" Cc: "dev@dpdk.org" References: <20210216094300.27889-1-bruce.richardson@intel.com> <313c223f-bf1c-9307-75f8-0a0c1da7fd21@intel.com> <20210216104652.GB136@bricha3-MOBL.ger.corp.intel.com> <42706d4c-f8de-55c5-1161-b1e54c77599e@intel.com> <20210216173057.GE136@bricha3-MOBL.ger.corp.intel.com> From: "Burakov, Anatoly" Message-ID: Date: Wed, 17 Feb 2021 12:09:18 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH] eal: support using 0 as coremask for no-affinitization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 16-Feb-21 5:44 PM, Van Haaren, Harry wrote: >> -----Original Message----- >> From: Bruce Richardson >> Sent: Tuesday, February 16, 2021 5:31 PM >> To: Van Haaren, Harry >> Cc: Burakov, Anatoly ; dev@dpdk.org >> Subject: Re: [dpdk-dev] [PATCH] eal: support using 0 as coremask for no- >> affinitization >> >> On Tue, Feb 16, 2021 at 05:22:25PM +0000, Van Haaren, Harry wrote: >>>> -----Original Message----- >>>> From: Burakov, Anatoly >>>> Sent: Tuesday, February 16, 2021 10:53 AM >>>> To: Richardson, Bruce ; Van Haaren, Harry >>>> >>>> Cc: dev@dpdk.org >>>> Subject: Re: [dpdk-dev] [PATCH] eal: support using 0 as coremask for no- >>>> affinitization >>>> >>>> On 16-Feb-21 10:46 AM, Bruce Richardson wrote: >>>>> On Tue, Feb 16, 2021 at 10:36:13AM +0000, Burakov, Anatoly wrote: >>>>>> On 16-Feb-21 9:43 AM, Bruce Richardson wrote: >>>>>>> Allow the user to specify that they don't want any core pinning from >> DPDK >>>>>>> by passing in the coremask of 0. >>>>>>> --- >>>>>> >>>>>> I haven't checked what happens yet, but down the line we also set affinity >>>>>> for service cores as well as interrupt thread. what would be the semantics >>>>>> of those in this particular case? do we want the same ability for service >>>>>> cores (i.e. pick a non-affinitized core)? And where does interrupt thread >>>>>> affinitize in this case (presumably, nowhere too)? >>>>>> >>>>> I have not checked the service core setup, because a) I forgot about them >>>>> and b) I'm not sure how their affinity rules work with respect to the main >>>>> lcore mask. On the other hand I did check out that the lcore mask for all >>>>> non-pinned threads, or control threads, is the full set of bits as >>>>> expected. >>>>> >>>>> /Bruce >>>>> >>>> >>>> +Harry, >>>> >>>> I believe service core mask must not overlap with lcore masks, so >>>> presumably using 0 as lcore mask would make it so that any service core >>>> mask will be valid (which is presumably what we want?). >>> >>> Services cores -S list or -s *must* overlap with the RTE lcores, EAL >>> then"steals" the service cores from the application lcores, code that >> implements here: >>> http://git.dpdk.org/dpdk- >> stable/tree/lib/librte_eal/common/eal_common_options.c?h=20.11#n657 >>> >>>> Should service cores also have a "just pick a core" parameter? >>> >>> I'm not sure, depends on what the bigger goal is here. >>> Assuming we're enabling this for ROLE_RTE threads, then >>> it would seem to me that ROLE_SERVICE and control threads >>> would require similar treatment? >>> >> Control threads are affinitised to all cores not in the coremask, which >> means in this case that they can run anywhere on the system the OS chooses. > > Ah ok, fair enough yes. > >> In case of service cores, it would seem that using service cores with an >> empty coremask is just not compatible. I would assume that this >> incompatibility already exists when one has a coremask with only one core >> already in it. > > Yes, correct, it would leave zero lcores for ROLE_RTE, meaning no lcores for the application. > A possible solution would be to special case a zero service core mask and apply the same > treatment as ROLE_RTE coremask? > > Others likely have better ideas - I don't have time to follow DPDK threading/pinning topic > closely at the moment. > I don't think it's a good idea to disallow service cores functionality in this case, but i don't have a way to solve this, other than implementing similar 0x0 coremask for service cores and assume it always means "one core affinitized to wherever the OS feels like it". After all, with lcore mask 0x0 we assume user wants one single core only, so following that, one single service core is a valid extrapolation IMO. Perhaps specifying the number of l/s cores when using 0x0 would be interesting, but IMO unless there's ask for it, i'd rather not overcomplicate things and go with similar semantics for service cores, and just allow a 0x0 coremask that means only one unaffinitized service core will be created. Thoughts? -- Thanks, Anatoly