From mboxrd@z Thu Jan  1 00:00:00 1970
From: Thomas Monjalon <thomas.monjalon@6wind.com>
Subject: Re: [PATCH v2] mem: balanced allocation of hugepages
Date: Mon, 10 Apr 2017 12:03:50 +0200
Message-ID: <2538172.OtvTCRC0DK@xps13>
References: <1487250070-13973-1-git-send-email-i.maximets@samsung.com>
 <CGME20170410080425eucas1p27fd424ae58151f13b1a7a3723aa4ad1e@eucas1p2.samsung.com>
 <1491811459-1647-1-git-send-email-i.maximets@samsung.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7Bit
Cc: Ilya Maximets <i.maximets@samsung.com>, dev@dpdk.org,
 David Marchand <david.marchand@6wind.com>,
 Heetae Ahn <heetae82.ahn@samsung.com>,
 Yuanhan Liu <yuanhan.liu@linux.intel.com>,
 Jianfeng Tan <jianfeng.tan@intel.com>, Neil Horman <nhorman@tuxdriver.com>,
 Yulong Pei <yulong.pei@intel.com>
To: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
Return-path: <dev-bounces@dpdk.org>
Received: from mail-wm0-f54.google.com (mail-wm0-f54.google.com [74.125.82.54])
 by dpdk.org (Postfix) with ESMTP id A35DF2BB0
 for <dev@dpdk.org>; Mon, 10 Apr 2017 12:03:52 +0200 (CEST)
Received: by mail-wm0-f54.google.com with SMTP id t189so33219166wmt.1
 for <dev@dpdk.org>; Mon, 10 Apr 2017 03:03:52 -0700 (PDT)
In-Reply-To: <1491811459-1647-1-git-send-email-i.maximets@samsung.com>
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

2017-04-10 11:04, Ilya Maximets:
> Currently EAL allocates hugepages one by one not paying
> attention from which NUMA node allocation was done.
> 
> Such behaviour leads to allocation failure if number of
> available hugepages for application limited by cgroups
> or hugetlbfs and memory requested not only from the first
> socket.
> 
> Example:
> 	# 90 x 1GB hugepages availavle in a system
> 
> 	cgcreate -g hugetlb:/test
> 	# Limit to 32GB of hugepages
> 	cgset -r hugetlb.1GB.limit_in_bytes=34359738368 test
> 	# Request 4GB from each of 2 sockets
> 	cgexec -g hugetlb:test testpmd --socket-mem=4096,4096 ...
> 
> 	EAL: SIGBUS: Cannot mmap more hugepages of size 1024 MB
> 	EAL: 32 not 90 hugepages of size 1024 MB allocated
> 	EAL: Not enough memory available on socket 1!
> 	     Requested: 4096MB, available: 0MB
> 	PANIC in rte_eal_init():
> 	Cannot init memory
> 
> 	This happens beacause all allocated pages are
> 	on socket 0.
> 
> Fix this issue by setting mempolicy MPOL_PREFERRED for each
> hugepage to one of requested nodes in a round-robin fashion.
> In this case all allocated pages will be fairly distributed
> between all requested nodes.
> 
> New config option RTE_LIBRTE_EAL_NUMA_AWARE_HUGEPAGES
> introduced and disabled by default because of external
> dependency from libnuma.
> 
> Fixes: 77988fc08dc5 ("mem: fix allocating all free hugepages")
> 
> Signed-off-by: Ilya Maximets <i.maximets@samsung.com>

Status: Changes Requested
per Sergio advice: "I would be inclined towards v3 targeting v17.08."