From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA0C5C47DD9 for ; Sun, 25 Feb 2024 16:28:32 +0000 (UTC) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6275B402C8; Sun, 25 Feb 2024 17:28:31 +0100 (CET) Received: from mail.lysator.liu.se (mail.lysator.liu.se [130.236.254.3]) by mails.dpdk.org (Postfix) with ESMTP id 12693402BA for ; Sun, 25 Feb 2024 17:28:30 +0100 (CET) Received: from mail.lysator.liu.se (localhost [127.0.0.1]) by mail.lysator.liu.se (Postfix) with ESMTP id 96C146DF7 for ; Sun, 25 Feb 2024 17:28:29 +0100 (CET) Received: by mail.lysator.liu.se (Postfix, from userid 1004) id 8B5BB6E4D; Sun, 25 Feb 2024 17:28:29 +0100 (CET) Received: from [192.168.1.59] (h-62-63-215-114.A163.priv.bahnhof.se [62.63.215.114]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mail.lysator.liu.se (Postfix) with ESMTPSA id D5CBD6F10; Sun, 25 Feb 2024 17:28:27 +0100 (CET) Message-ID: <67cf1df8-b4b4-4d7c-a462-e84e3ee3e16e@lysator.liu.se> Date: Sun, 25 Feb 2024 17:28:27 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC v4 5/6] service: keep per-lcore state in lcore variable To: =?UTF-8?Q?Mattias_R=C3=B6nnblom?= , dev@dpdk.org Cc: =?UTF-8?Q?Morten_Br=C3=B8rup?= , Stephen Hemminger References: <20240220084908.488252-2-mattias.ronnblom@ericsson.com> <20240225150330.517225-1-mattias.ronnblom@ericsson.com> <20240225150330.517225-6-mattias.ronnblom@ericsson.com> Content-Language: en-US From: =?UTF-8?Q?Mattias_R=C3=B6nnblom?= In-Reply-To: <20240225150330.517225-6-mattias.ronnblom@ericsson.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Scanned: ClamAV using ClamSMTP X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On 2024-02-25 16:03, Mattias Rönnblom wrote: > Replace static array of cache-aligned structs with an lcore variable, > to slightly benefit code simplicity and performance. > > RFC v4: > * Remove strange-looking lcore value lookup potentially containing > invalid lcore id. (Morten Brørup) > * Replace misplaced tab with space. (Morten Brørup) > > Signed-off-by: Mattias Rönnblom > --- > lib/eal/common/rte_service.c | 120 ++++++++++++++++++++--------------- > 1 file changed, 69 insertions(+), 51 deletions(-) > > diff --git a/lib/eal/common/rte_service.c b/lib/eal/common/rte_service.c > index d959c91459..7fbae704ed 100644 > --- a/lib/eal/common/rte_service.c > +++ b/lib/eal/common/rte_service.c > @@ -11,6 +11,7 @@ > > #include > #include > +#include > #include > #include > #include > @@ -75,7 +76,7 @@ struct core_state { > > static uint32_t rte_service_count; > static struct rte_service_spec_impl *rte_services; > -static struct core_state *lcore_states; > +static RTE_LCORE_VAR_HANDLE(struct core_state, lcore_states); > static uint32_t rte_service_library_initialized; > > int32_t > @@ -101,11 +102,12 @@ rte_service_init(void) > goto fail_mem; > } > > - lcore_states = rte_calloc("rte_service_core_states", RTE_MAX_LCORE, > - sizeof(struct core_state), RTE_CACHE_LINE_SIZE); > - if (!lcore_states) { > - EAL_LOG(ERR, "error allocating core states array"); > - goto fail_mem; > + if (lcore_states == NULL) > + RTE_LCORE_VAR_ALLOC(lcore_states); > + else { > + struct core_state *cs; > + RTE_LCORE_VAR_FOREACH_VALUE(cs, lcore_states) > + memset(cs, 0, sizeof(struct core_state)); > } > > int i; > @@ -122,7 +124,6 @@ rte_service_init(void) > return 0; > fail_mem: > rte_free(rte_services); > - rte_free(lcore_states); > return -ENOMEM; > } > > @@ -136,7 +137,6 @@ rte_service_finalize(void) > rte_eal_mp_wait_lcore(); > > rte_free(rte_services); > - rte_free(lcore_states); > > rte_service_library_initialized = 0; > } > @@ -286,7 +286,6 @@ rte_service_component_register(const struct rte_service_spec *spec, > int32_t > rte_service_component_unregister(uint32_t id) > { > - uint32_t i; > struct rte_service_spec_impl *s; > SERVICE_VALID_GET_OR_ERR_RET(id, s, -EINVAL); > > @@ -294,9 +293,10 @@ rte_service_component_unregister(uint32_t id) > > s->internal_flags &= ~(SERVICE_F_REGISTERED); > > + struct core_state *cs; > /* clear the run-bit in all cores */ > - for (i = 0; i < RTE_MAX_LCORE; i++) > - lcore_states[i].service_mask &= ~(UINT64_C(1) << id); > + RTE_LCORE_VAR_FOREACH_VALUE(cs, lcore_states) > + cs->service_mask &= ~(UINT64_C(1) << id); > > memset(&rte_services[id], 0, sizeof(struct rte_service_spec_impl)); > > @@ -454,7 +454,10 @@ rte_service_may_be_active(uint32_t id) > return -EINVAL; > > for (i = 0; i < lcore_count; i++) { > - if (lcore_states[ids[i]].service_active_on_lcore[id]) > + struct core_state *cs = > + RTE_LCORE_VAR_LCORE_PTR(ids[i], lcore_states); > + > + if (cs->service_active_on_lcore[id]) > return 1; > } > > @@ -464,7 +467,7 @@ rte_service_may_be_active(uint32_t id) > int32_t > rte_service_run_iter_on_app_lcore(uint32_t id, uint32_t serialize_mt_unsafe) > { > - struct core_state *cs = &lcore_states[rte_lcore_id()]; > + struct core_state *cs = RTE_LCORE_VAR_PTR(lcore_states); > struct rte_service_spec_impl *s; > > SERVICE_VALID_GET_OR_ERR_RET(id, s, -EINVAL); > @@ -486,8 +489,7 @@ service_runner_func(void *arg) > { > RTE_SET_USED(arg); > uint8_t i; > - const int lcore = rte_lcore_id(); > - struct core_state *cs = &lcore_states[lcore]; > + struct core_state *cs = RTE_LCORE_VAR_PTR(lcore_states); > > rte_atomic_store_explicit(&cs->thread_active, 1, rte_memory_order_seq_cst); > > @@ -533,13 +535,17 @@ service_runner_func(void *arg) > int32_t > rte_service_lcore_may_be_active(uint32_t lcore) > { > - if (lcore >= RTE_MAX_LCORE || !lcore_states[lcore].is_service_core) > + struct core_state *cs; > + > + if (lcore >= RTE_MAX_LCORE || !cs->is_service_core) This doesn't work, since 'cs' is not yet initialized. I'll fix it v5.