From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71C80C433DB for ; Wed, 17 Feb 2021 13:37:35 +0000 (UTC) Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by mail.kernel.org (Postfix) with ESMTP id CCF4C64E28 for ; Wed, 17 Feb 2021 13:37:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CCF4C64E28 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=dev-bounces@dpdk.org Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A957840690; Wed, 17 Feb 2021 14:37:33 +0100 (CET) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id 704E24067A for ; Wed, 17 Feb 2021 14:37:31 +0100 (CET) IronPort-SDR: OilRpkIaxJykQ4M2xF9BwXO6IE/EOUJbE9xE8AvHAV5SZL9oV88ugvuz3ew2oknVGunXd6yVXN yIZ3mSO8mzaA== X-IronPort-AV: E=McAfee;i="6000,8403,9897"; a="247261146" X-IronPort-AV: E=Sophos;i="5.81,184,1610438400"; d="scan'208";a="247261146" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Feb 2021 05:37:28 -0800 IronPort-SDR: ghEJQr6vzVb5BZ0dL1Ti0PilqZ5uaPscUMne52pTSRagNUqlINdkH8jHQ/Dwg+b2R4BUAH3tdN JEvO1ZjcpOKw== X-IronPort-AV: E=Sophos;i="5.81,184,1610438400"; d="scan'208";a="399954417" Received: from aburakov-mobl.ger.corp.intel.com (HELO [10.213.233.84]) ([10.213.233.84]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Feb 2021 05:37:27 -0800 To: Bruce Richardson , "Van Haaren, Harry" Cc: "dev@dpdk.org" References: <20210216094300.27889-1-bruce.richardson@intel.com> <313c223f-bf1c-9307-75f8-0a0c1da7fd21@intel.com> <20210216104652.GB136@bricha3-MOBL.ger.corp.intel.com> <42706d4c-f8de-55c5-1161-b1e54c77599e@intel.com> <20210216173057.GE136@bricha3-MOBL.ger.corp.intel.com> <20210217132649.GA1171@bricha3-MOBL.ger.corp.intel.com> From: "Burakov, Anatoly" Message-ID: Date: Wed, 17 Feb 2021 13:37:25 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <20210217132649.GA1171@bricha3-MOBL.ger.corp.intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH] eal: support using 0 as coremask for no-affinitization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 17-Feb-21 1:26 PM, Bruce Richardson wrote: > On Wed, Feb 17, 2021 at 12:14:36PM +0000, Van Haaren, Harry wrote: >>> -----Original Message----- >>> From: Burakov, Anatoly >>> Sent: Wednesday, February 17, 2021 12:09 PM >>> To: Van Haaren, Harry ; Richardson, Bruce >>> >>> Cc: dev@dpdk.org >>> Subject: Re: [dpdk-dev] [PATCH] eal: support using 0 as coremask for no- >>> affinitization >>> >>> On 16-Feb-21 5:44 PM, Van Haaren, Harry wrote: >>>>> -----Original Message----- >>>>> From: Bruce Richardson >>>>> Sent: Tuesday, February 16, 2021 5:31 PM >>>>> To: Van Haaren, Harry >>>>> Cc: Burakov, Anatoly ; dev@dpdk.org >>>>> Subject: Re: [dpdk-dev] [PATCH] eal: support using 0 as coremask for no- >>>>> affinitization >>>>> >>>>> On Tue, Feb 16, 2021 at 05:22:25PM +0000, Van Haaren, Harry wrote: >>>>>>> -----Original Message----- >>>>>>> From: Burakov, Anatoly >>>>>>> Sent: Tuesday, February 16, 2021 10:53 AM >>>>>>> To: Richardson, Bruce ; Van Haaren, Harry >>>>>>> >>>>>>> Cc: dev@dpdk.org >>>>>>> Subject: Re: [dpdk-dev] [PATCH] eal: support using 0 as coremask for no- >>>>>>> affinitization >>>>>>> >>>>>>> On 16-Feb-21 10:46 AM, Bruce Richardson wrote: >>>>>>>> On Tue, Feb 16, 2021 at 10:36:13AM +0000, Burakov, Anatoly wrote: >>>>>>>>> On 16-Feb-21 9:43 AM, Bruce Richardson wrote: >>>>>>>>>> Allow the user to specify that they don't want any core pinning from >>>>> DPDK >>>>>>>>>> by passing in the coremask of 0. >>>>>>>>>> --- >>>>>>>>> >>>>>>>>> I haven't checked what happens yet, but down the line we also set >>> affinity >>>>>>>>> for service cores as well as interrupt thread. what would be the >>> semantics >>>>>>>>> of those in this particular case? do we want the same ability for service >>>>>>>>> cores (i.e. pick a non-affinitized core)? And where does interrupt thread >>>>>>>>> affinitize in this case (presumably, nowhere too)? >>>>>>>>> >>>>>>>> I have not checked the service core setup, because a) I forgot about them >>>>>>>> and b) I'm not sure how their affinity rules work with respect to the main >>>>>>>> lcore mask. On the other hand I did check out that the lcore mask for all >>>>>>>> non-pinned threads, or control threads, is the full set of bits as >>>>>>>> expected. >>>>>>>> >>>>>>>> /Bruce >>>>>>>> >>>>>>> >>>>>>> +Harry, >>>>>>> >>>>>>> I believe service core mask must not overlap with lcore masks, so >>>>>>> presumably using 0 as lcore mask would make it so that any service core >>>>>>> mask will be valid (which is presumably what we want?). >>>>>> >>>>>> Services cores -S list or -s *must* overlap with the RTE lcores, EAL >>>>>> then"steals" the service cores from the application lcores, code that >>>>> implements here: >>>>>> http://git.dpdk.org/dpdk- >>>>> stable/tree/lib/librte_eal/common/eal_common_options.c?h=20.11#n657 >>>>>> >>>>>>> Should service cores also have a "just pick a core" parameter? >>>>>> >>>>>> I'm not sure, depends on what the bigger goal is here. >>>>>> Assuming we're enabling this for ROLE_RTE threads, then >>>>>> it would seem to me that ROLE_SERVICE and control threads >>>>>> would require similar treatment? >>>>>> >>>>> Control threads are affinitised to all cores not in the coremask, which >>>>> means in this case that they can run anywhere on the system the OS chooses. >>>> >>>> Ah ok, fair enough yes. >>>> >>>>> In case of service cores, it would seem that using service cores with an >>>>> empty coremask is just not compatible. I would assume that this >>>>> incompatibility already exists when one has a coremask with only one core >>>>> already in it. >>>> >>>> Yes, correct, it would leave zero lcores for ROLE_RTE, meaning no lcores for the >>> application. >>>> A possible solution would be to special case a zero service core mask and apply >>> the same >>>> treatment as ROLE_RTE coremask? >>>> >>>> Others likely have better ideas - I don't have time to follow DPDK >>> threading/pinning topic >>>> closely at the moment. >>>> >>> >>> I don't think it's a good idea to disallow service cores functionality >>> in this case, but i don't have a way to solve this, other than >>> implementing similar 0x0 coremask for service cores and assume it always >>> means "one core affinitized to wherever the OS feels like it". After >>> all, with lcore mask 0x0 we assume user wants one single core only, so >>> following that, one single service core is a valid extrapolation IMO. >> >> OK with me - seems reasonable. >> >>> Perhaps specifying the number of l/s cores when using 0x0 would be >>> interesting, but IMO unless there's ask for it, i'd rather not >>> overcomplicate things and go with similar semantics for service cores, >>> and just allow a 0x0 coremask that means only one unaffinitized service >>> core will be created. >>> >>> Thoughts? >> >> Agree with keeping-it-simple if possible, and agree that unaffinitized with >> a single service-core with a 0x0 mask makes sense. >> >> Most important to me is to maintain backward compatibility with existing >> usage of -S and -s, but this shouldn't break anything? (Famous last words..) >> > > Not sure I entirely follow all of this. Is the suggestion just to extend -s > processing to allow "0" as coremask too? That would be independent then of > any core masks passed in for -c/-l options, right? As well as working well > with this patch, it would also solve the issue of using single core with a > coremask of e.g. 0x1 too, I think. > > Is my understanding correct? > > /Bruce > Yes, that's exactly what i meant :) Sorry for being long-winded and unclear. -- Thanks, Anatoly