All of lore.kernel.org
 help / color / mirror / Atom feed
From: Dmitry Kozlyuk <dkozlyuk@oss.nvidia.com>
To: <dev@dpdk.org>
Subject: [dpdk-dev] [PATCH v6 0/3] eal: add memory pre-allocation from existing files
Date: Mon, 11 Oct 2021 11:56:41 +0300	[thread overview]
Message-ID: <20211011085644.2716490-1-dkozlyuk@nvidia.com> (raw)
In-Reply-To: <20210921081632.858873-1-dkozlyuk@nvidia.com>

Hugepage allocation from the system takes time, resulting in slow
startup or sporadic delays later. Most of the time spent in kernel
is zero-filling memory for security reasons, which may be irrelevant
in a controlled environment. The bottleneck is memory access speed,
so for speeduup the amount of memory cleared must be reduced.
We propose a new EAL option --mem-file FILE1,FILE2,... to quickly
allocate dirty pages from existing files and clean it as necessary.
A new malloc_perf_autotest is provided to estimate the impact.
More details are explained in relevant patches.

v6: fix copyright line (Thomas), add SPDX header for the new test file
    (BTW, why didn't the CI complain in previous versions?)
v5: rebase
v4: getmntent() -> getmntent_r(), better error detection (John Levon)
v3: fix hugepage mount point detection
v2: fix CI failures

Dmitry Kozlyuk (2):
  eal/linux: make hugetlbfs analysis reusable
  app/test: add allocator performance autotest

Viacheslav Ovsiienko (1):
  eal: add memory pre-allocation from existing files

 app/test/meson.build                          |   2 +
 app/test/test_malloc_perf.c                   | 161 +++++++++
 doc/guides/linux_gsg/linux_eal_parameters.rst |  17 +
 lib/eal/common/eal_common_dynmem.c            |   6 +
 lib/eal/common/eal_common_options.c           |  23 ++
 lib/eal/common/eal_internal_cfg.h             |   4 +
 lib/eal/common/eal_memalloc.h                 |   8 +-
 lib/eal/common/eal_options.h                  |   2 +
 lib/eal/common/malloc_elem.c                  |   5 +
 lib/eal/common/malloc_heap.h                  |   8 +
 lib/eal/common/rte_malloc.c                   |  16 +-
 lib/eal/include/rte_memory.h                  |   4 +-
 lib/eal/linux/eal.c                           |  28 ++
 lib/eal/linux/eal_hugepage_info.c             | 158 ++++++---
 lib/eal/linux/eal_hugepage_info.h             |  39 +++
 lib/eal/linux/eal_memalloc.c                  | 328 +++++++++++++++++-
 16 files changed, 739 insertions(+), 70 deletions(-)
 create mode 100644 app/test/test_malloc_perf.c
 create mode 100644 lib/eal/linux/eal_hugepage_info.h

-- 
2.25.1


  parent reply	other threads:[~2021-10-11  8:57 UTC|newest]

Thread overview: 48+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-07-05 12:49 [dpdk-dev] [PATCH 21.11 0/3] eal: add memory pre-allocation from existing files Dmitry Kozlyuk
2021-07-05 12:49 ` [dpdk-dev] [PATCH 21.11 1/3] eal/linux: make hugetlbfs analysis reusable Dmitry Kozlyuk
2021-07-05 12:49 ` [dpdk-dev] [PATCH 21.11 2/3] eal: add memory pre-allocation from existing files Dmitry Kozlyuk
2021-07-05 12:49 ` [dpdk-dev] [PATCH 21.11 3/3] app/test: add allocator performance autotest Dmitry Kozlyuk
2021-07-16 11:08 ` [dpdk-dev] [PATCH 21.11 v2 0/3] eal: add memory pre-allocation from existing files Dmitry Kozlyuk
2021-07-16 11:08   ` [dpdk-dev] [PATCH 21.11 v2 1/3] eal/linux: make hugetlbfs analysis reusable Dmitry Kozlyuk
2021-07-16 11:08   ` [dpdk-dev] [PATCH 21.11 v2 2/3] eal: add memory pre-allocation from existing files Dmitry Kozlyuk
2021-07-16 11:08   ` [dpdk-dev] [PATCH 21.11 v2 3/3] app/test: add allocator performance autotest Dmitry Kozlyuk
2021-08-09  9:45   ` [dpdk-dev] [PATCH 21.11 v2 0/3] eal: add memory pre-allocation from existing files Dmitry Kozlyuk
2021-08-30  8:21     ` Dmitry Kozlyuk
2021-09-14 10:34   ` [dpdk-dev] [PATCH v3 " Dmitry Kozlyuk
2021-09-14 10:34     ` [dpdk-dev] [PATCH v3 1/3] eal/linux: make hugetlbfs analysis reusable Dmitry Kozlyuk
2021-09-14 12:48       ` John Levon
2021-09-14 12:57         ` Dmitry Kozlyuk
2021-09-16 12:08       ` John Levon
2021-09-14 10:34     ` [dpdk-dev] [PATCH v3 2/3] eal: add memory pre-allocation from existing files Dmitry Kozlyuk
2021-09-14 10:34     ` [dpdk-dev] [PATCH v3 3/3] app/test: add allocator performance autotest Dmitry Kozlyuk
2021-09-20 12:52     ` [dpdk-dev] [PATCH v4 0/3] eal: add memory pre-allocation from existing files dkozlyuk
2021-09-20 12:53       ` [dpdk-dev] [PATCH v4 1/3] eal/linux: make hugetlbfs analysis reusable dkozlyuk
2021-09-20 12:53       ` [dpdk-dev] [PATCH v4 2/3] eal: add memory pre-allocation from existing files dkozlyuk
2021-09-20 12:53       ` [dpdk-dev] [PATCH v4 3/3] app/test: add allocator performance autotest dkozlyuk
2021-09-21  8:16       ` [dpdk-dev] [PATCH v5 0/3] eal: add memory pre-allocation from existing files dkozlyuk
2021-09-21  8:16         ` [dpdk-dev] [PATCH v5 1/3] eal/linux: make hugetlbfs analysis reusable dkozlyuk
2021-09-22 13:52           ` John Levon
2021-10-05 17:36           ` Thomas Monjalon
2021-10-08 15:33             ` John Levon
2021-10-08 15:50               ` Dmitry Kozlyuk
2021-09-21  8:16         ` [dpdk-dev] [PATCH v5 2/3] eal: add memory pre-allocation from existing files dkozlyuk
2021-09-21  8:16         ` [dpdk-dev] [PATCH v5 3/3] app/test: add allocator performance autotest dkozlyuk
2021-10-11  8:56         ` Dmitry Kozlyuk [this message]
2021-10-11  8:56           ` [dpdk-dev] [PATCH v6 1/3] eal/linux: make hugetlbfs analysis reusable Dmitry Kozlyuk
2021-10-13  8:16             ` David Marchand
2021-10-13  9:21               ` Dmitry Kozlyuk
2021-10-11  8:56           ` [dpdk-dev] [PATCH v6 2/3] eal: add memory pre-allocation from existing files Dmitry Kozlyuk
2021-10-12 15:37             ` David Marchand
2021-10-12 15:55               ` Dmitry Kozlyuk
2021-10-12 17:32                 ` David Marchand
2021-10-12 21:09                   ` Dmitry Kozlyuk
2021-10-13 10:18                     ` David Marchand
2021-11-08 14:27                       ` Dmitry Kozlyuk
2021-11-08 17:45                         ` David Marchand
2021-10-11  8:56           ` [dpdk-dev] [PATCH v6 3/3] app/test: add allocator performance autotest Dmitry Kozlyuk
2021-10-12 13:53             ` Aaron Conole
2021-10-12 14:48               ` Dmitry Kozlyuk
2021-10-15 13:47                 ` Aaron Conole
2021-10-11 18:52           ` [dpdk-dev] [PATCH v6 0/3] eal: add memory pre-allocation from existing files Thomas Monjalon
2021-10-11 21:12             ` [dpdk-dev] [dpdk-ci] " Lincoln Lavoie
2021-10-12  6:54               ` Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211011085644.2716490-1-dkozlyuk@nvidia.com \
    --to=dkozlyuk@oss.nvidia.com \
    --cc=dev@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.