From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alejandro Lucero Subject: Re: [PATCH v3 2/6] mem: use address hint for mapping hugepages Date: Mon, 29 Oct 2018 16:40:54 +0000 Message-ID: References: <1538743527-8285-1-git-send-email-alejandro.lucero@netronome.com> <1538743527-8285-3-git-send-email-alejandro.lucero@netronome.com> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Cc: Thomas Monjalon , "Burakov, Anatoly" , dev To: darek.stojaczyk@gmail.com Return-path: Received: from mail-ed1-f65.google.com (mail-ed1-f65.google.com [209.85.208.65]) by dpdk.org (Postfix) with ESMTP id 9A72C1C01 for ; Mon, 29 Oct 2018 17:41:05 +0100 (CET) Received: by mail-ed1-f65.google.com with SMTP id x2-v6so7903786eds.3 for ; Mon, 29 Oct 2018 09:41:05 -0700 (PDT) In-Reply-To: List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Dariousz, On Mon, Oct 29, 2018 at 4:08 PM Dariusz Stojaczyk wrote: > On Fri, Oct 5, 2018 at 2:47 PM Alejandro Lucero > wrote: > > > > Linux kernel uses a really high address as starting address for > > serving mmaps calls. If there exist addressing limitations and > > IOVA mode is VA, this starting address is likely too high for > > those devices. However, it is possible to use a lower address in > > the process virtual address space as with 64 bits there is a lot > > of available space. > > > > This patch adds an address hint as starting address for 64 bits > > systems and increments the hint for next invocations. If the mmap > > call does not use the hint address, repeat the mmap call using > > the hint address incremented by page size. > > > > Signed-off-by: Alejandro Lucero > > Reviewed-by: Anatoly Burakov > > --- > > lib/librte_eal/common/eal_common_memory.c | 34 > ++++++++++++++++++++++++++++++- > > 1 file changed, 33 insertions(+), 1 deletion(-) > > > > diff --git a/lib/librte_eal/common/eal_common_memory.c > b/lib/librte_eal/common/eal_common_memory.c > > index c482f0d..853c44c 100644 > > --- a/lib/librte_eal/common/eal_common_memory.c > > +++ b/lib/librte_eal/common/eal_common_memory.c > > @@ -37,6 +37,23 @@ > > static void *next_baseaddr; > > static uint64_t system_page_sz; > > > > +#ifdef RTE_ARCH_64 > > +/* > > + * Linux kernel uses a really high address as starting address for > serving > > + * mmaps calls. If there exists addressing limitations and IOVA mode is > VA, > > + * this starting address is likely too high for those devices. However, > it > > + * is possible to use a lower address in the process virtual address > space > > + * as with 64 bits there is a lot of available space. > > + * > > + * Current known limitations are 39 or 40 bits. Setting the starting > address > > + * at 4GB implies there are 508GB or 1020GB for mapping the available > > + * hugepages. This is likely enough for most systems, although a device > with > > + * addressing limitations should call rte_eal_check_dma_mask for > ensuring all > > + * memory is within supported range. > > + */ > > +static uint64_t baseaddr = 0x100000000; > > +#endif > > This breaks running with ASAN unless a custom --base-virtaddr option > is specified. The default base-virtaddr introduced by this patch falls > into an area that's already reserved by ASAN. > > See here: > https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm > The only available address space starts at 0x10007fff8000, which > unfortunately doesn't fit in 39 bits. > > Right now the very first eal_get_virtual_area() in EAL initialization > is used with 4KB pagesize, meaning that DPDK will try to mmap at each > 4KB-aligned offset all the way from 0x100000000 to 0x10007fff8000, > which takes quite a long, long time. > > I'm not sure about the solution to this problem, but I verify that > starting DPDK 18.11-rc1 with `--base-virtaddr 0x200000000000` works > just fine under ASAN. > > Do we have documentation about using Address Sanitizer? I understand the goal but, which is the cost? Do you have numbers about the impact on performance? Solving this is not trivial. I would say someone interested in this but using a hardware with addressing limitations needs to choose. Could it be possible to modify the virtual addresses used by default? I guess the shadow regions can be higher that the default ones. > D. > > > > > >