From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.5 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68E4DC433E0 for ; Fri, 19 Jun 2020 16:23:08 +0000 (UTC) Received: from dpdk.org (dpdk.org [92.243.14.124]) by mail.kernel.org (Postfix) with ESMTP id CD9772168B for ; Fri, 19 Jun 2020 16:23:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="YGMZQiqt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CD9772168B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7C73D1BF44; Fri, 19 Jun 2020 18:23:06 +0200 (CEST) Received: from us-smtp-1.mimecast.com (us-smtp-2.mimecast.com [205.139.110.61]) by dpdk.org (Postfix) with ESMTP id 62BDD1BEE0 for ; Fri, 19 Jun 2020 18:23:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592583784; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mvkNRJkJE0yBlLJ4s8E7oGPdnU+lxWoZtCMSjF/UVR0=; b=YGMZQiqtZhdbczGA1eLarmWZ4XRwrqdvYJ0odSDsmoloBEvOrPDSQQyrliopZ07gKYHICV wIEtsmTVBTaZE0a3g6vpCR6iVZIEnxIsN2bv0Hl+bJ/8ncguR4pgt1g+/sXAhNABbuphYI 2uRFZYQz842FyiHNKrIt3aF02Mx7cKM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-6-Rpw3v5NmNN-B3458aK43Pg-1; Fri, 19 Jun 2020 12:22:59 -0400 X-MC-Unique: Rpw3v5NmNN-B3458aK43Pg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7724F1800D42; Fri, 19 Jun 2020 16:22:57 +0000 (UTC) Received: from dmarchan.remote.csb (unknown [10.40.193.186]) by smtp.corp.redhat.com (Postfix) with ESMTP id CB03360BF4; Fri, 19 Jun 2020 16:22:54 +0000 (UTC) From: David Marchand To: dev@dpdk.org Cc: jerinjacobk@gmail.com, bruce.richardson@intel.com, mdr@ashroe.eu, ktraynor@redhat.com, ian.stokes@intel.com, i.maximets@ovn.org Date: Fri, 19 Jun 2020 18:22:35 +0200 Message-Id: <20200619162244.8239-1-david.marchand@redhat.com> In-Reply-To: <20200610144506.30505-1-david.marchand@redhat.com> References: <20200610144506.30505-1-david.marchand@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david.marchand@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v2 0/9] Register non-EAL threads as lcore X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" OVS and some other applications have been hacking into DPDK internals to fake EAL threads and avoid performance penalty of only having non-EAL threads. This series proposes to add a new type of lcores and maps those threads to such lcores. non-EAL threads won't run the DPDK eal mainloop. As a consequence, part of the EAL threads API cannot work. Having new lcores appearing during the process lifetime is not expected by some DPDK components. This is addressed by introducing init/uninit callacks invoked when hotplugging of such lcore. There is still some work: - refuse new lcore role in incompatible EAL threads API (or document it only as those API were already incompatible?), - finish unit tests in patch 7, - think about deprecation notices for existing RTE_FOREACH_LCORE macros and consorts, it is probably worth discussing on how to iterate over lcores, For the interested parties, I have a patch [1] against dpdk-latest OVS branch that makes use of this series. 1: https://github.com/david-marchand/ovs/commit/dpdk-latest-external-lcores Changes since v1: - rebased on master (conflicts on merged Windows series), - separated lcore role code cleanup in a patch, - tried to use a single naming, so kept non-EAL threads as the main notion. non-EAL threads are then distinguished between registered and unregistered non-EAL threads, - added unit tests (still missing some coverage, marked with a FIXME), - reworked callbacks call under a common rwlock lock which protects lcores allocations and callbacks registration, - introduced lcore iterators and converted the bucket mempool driver, -- David Marchand David Marchand (9): eal: relocate per thread symbols to common eal: fix multiple definition of per lcore thread id eal: introduce thread init helper eal: introduce thread uninit helper eal: move lcore role code eal: register non-EAL threads as lcores eal: add lcore init callbacks eal: add lcore iterators mempool/bucket: handle non-EAL lcores MAINTAINERS | 1 + app/test/Makefile | 1 + app/test/autotest_data.py | 6 + app/test/meson.build | 2 + app/test/test_lcores.c | 234 +++++++++++++++++ doc/guides/howto/debug_troubleshoot.rst | 5 +- .../prog_guide/env_abstraction_layer.rst | 22 +- doc/guides/prog_guide/mempool_lib.rst | 2 +- drivers/mempool/bucket/rte_mempool_bucket.c | 131 ++++++---- lib/librte_eal/common/eal_common_lcore.c | 241 +++++++++++++++++- lib/librte_eal/common/eal_common_thread.c | 126 ++++++--- lib/librte_eal/common/eal_common_trace.c | 49 +++- lib/librte_eal/common/eal_private.h | 19 ++ lib/librte_eal/common/eal_thread.h | 26 +- lib/librte_eal/common/eal_trace.h | 1 + lib/librte_eal/freebsd/eal.c | 16 +- lib/librte_eal/freebsd/eal_thread.c | 38 +-- lib/librte_eal/include/rte_eal.h | 12 +- lib/librte_eal/include/rte_lcore.h | 167 ++++++++++-- lib/librte_eal/linux/eal.c | 17 +- lib/librte_eal/linux/eal_thread.c | 38 +-- lib/librte_eal/rte_eal_version.map | 15 ++ lib/librte_eal/windows/eal.c | 3 +- lib/librte_eal/windows/eal_thread.c | 14 +- lib/librte_mempool/rte_mempool.h | 11 +- 25 files changed, 960 insertions(+), 237 deletions(-) create mode 100644 app/test/test_lcores.c -- 2.23.0