From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09A59C43331 for ; Thu, 2 Apr 2020 22:04:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AEBB62072E for ; Thu, 2 Apr 2020 22:04:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="DBCwI+Ak" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AEBB62072E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3AE8E8E0008; Thu, 2 Apr 2020 18:04:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 35F0E8E0007; Thu, 2 Apr 2020 18:04:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24CDE8E0008; Thu, 2 Apr 2020 18:04:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0150.hostedemail.com [216.40.44.150]) by kanga.kvack.org (Postfix) with ESMTP id 0A0558E0007 for ; Thu, 2 Apr 2020 18:04:05 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id B7744180AD80F for ; Thu, 2 Apr 2020 22:04:04 +0000 (UTC) X-FDA: 76664293608.20.glove67_6752dc72e337 X-HE-Tag: glove67_6752dc72e337 X-Filterd-Recvd-Size: 10462 Received: from mail-qk1-f195.google.com (mail-qk1-f195.google.com [209.85.222.195]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Thu, 2 Apr 2020 22:04:04 +0000 (UTC) Received: by mail-qk1-f195.google.com with SMTP id y3so1153286qky.8 for ; Thu, 02 Apr 2020 15:04:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=utBibNePo92+gYMMwoMvWIa8z78xVhyHF++ockV8SaM=; b=DBCwI+AkUnzvhIiSa2YXAXjf/NjJjiCz+mlvQJ1Kn1vmlEpC/dyMAiAGwZQ3BBrfGC pTqKpWEQs9lLpUVNIQ50DY1jI/+zO8tt1bOZfcknZ679/H/7VT2OXoM7tFjGpiVYfoet FflBpV+QRSjQsXd5L2Dxubls/Mkl/vX5HeogVOR+60J/gOMgzEzStZ43v1F7xo6VjLUC LKMs1CdJNWCUDi92dC4gn+mtWKyII4VHLu1BiCESTL5SLPEJew/mSWt4KqIfR1ZjHc+f nLoARjQTyqiUANxIIW7flGC9IUPmSKec1uYSQvTuz/n6dKB7DOgOTkmNN5aGzKZorYPn d3aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=utBibNePo92+gYMMwoMvWIa8z78xVhyHF++ockV8SaM=; b=tJOfeK8YMGt5SdGyZ5zTvekgX0ttxi2sqir/4exf/rbSjTho9XMZdNvawq1nb40cij k/KZ6IBxtALx4qW4z4I3ajaSrQnIAwrgYz4z+19F7OgsIOp1CyEhYdh7jsrhWtLcUUFW 41UV8ZVZ3s6pNCImfcDXMgzO81b+GxXJ2Q11l2+tbUVjNh9feD2MjrO4RrExc+N+W8og E7fsz/wXZy1LlW2Td2/X081zfoNdfpTc0k2vMxNnSMu0/gKv+ZskBiX+bssBuDCXrnTG dTeHo5SBU91V4C/HauddaY+4xbaGaqAG2/EU6cANkDQAEB/Y1o23SEkNNzC8wC3aqIta dGig== X-Gm-Message-State: AGi0PuY4kLPcsY4HOrkaEyPLOtt74s/2fmDMOdqqGixodYLAMh3CDqck hkK1yPK++sEvbYp7MEsNqtExZImxlvPXisojwYU= X-Google-Smtp-Source: APiQypIi2zfAZ5cBAiAI/mKFN/149Bb7guyTMDvqOGAmmLX127Ww2DV7ZGMq6yuKRnCDJZx/CuFputdqdBmkrCdm768= X-Received: by 2002:ae9:e711:: with SMTP id m17mr5924637qka.129.1585865043642; Thu, 02 Apr 2020 15:04:03 -0700 (PDT) MIME-Version: 1.0 References: <20200326212718.3798742-1-aslan@fb.com> <65164fdb-38b0-e239-84cc-ab22b92214c5@suse.cz> In-Reply-To: <65164fdb-38b0-e239-84cc-ab22b92214c5@suse.cz> From: Aslan Bakirov Date: Thu, 2 Apr 2020 23:03:52 +0100 Message-ID: Subject: Re: [PATCH 1/2] mm: cma: NUMA node interface To: Vlastimil Babka Cc: Aslan Bakirov , akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel-team@fb.com, riel@surriel.com, Roman Gushchin , mhocko@kernel.org, hannes@cmpxchg.org Content-Type: multipart/alternative; boundary="000000000000968f9905a255fa61" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --000000000000968f9905a255fa61 Content-Type: text/plain; charset="UTF-8" On Thu, Apr 2, 2020 at 4:48 PM Vlastimil Babka wrote: > On 3/26/20 10:27 PM, Aslan Bakirov wrote: > > I've noticed that there is no interfaces exposed by CMA which would let > me > > to declare contigous memory on particular NUMA node. > > > > This patchset adds the ability to try to allocate contiguous memory on > > specific node. > > I would say more explicitly that 'try' here means it will fallback to other > nodes if the specific one doesn't work. At least AFAICS that's what it > does by > calling memblock_alloc_range_nid() with exact_nid=false. > Thanks for the comments, Vlastimil. I created the next version with mentioning fallback case in the changelog. > > > Implement a new method for declaring contigous memory on particular node > > and keep cma_declare_contiguous() as a wrapper. > > Should there be also support for using this node spcification in the cma=X > boot > param? > > For this, I would wait for a real usecase for now, because adding it is easy but removing is hard. > > Signed-off-by: Aslan Bakirov > > ... > > > --- a/mm/cma.c > > +++ b/mm/cma.c > > @@ -220,7 +220,7 @@ int __init cma_init_reserved_mem(phys_addr_t base, > phys_addr_t size, > > } > > > > /** > > - * cma_declare_contiguous() - reserve custom contiguous area > > + * cma_declare_contiguous_nid() - reserve custom contiguous area > > * @base: Base address of the reserved area optional, use 0 for any > > * @size: Size of the reserved area (in bytes), > > * @limit: End address of the reserved memory (optional, 0 for any). > > @@ -229,6 +229,7 @@ int __init cma_init_reserved_mem(phys_addr_t base, > phys_addr_t size, > > * @fixed: hint about where to place the reserved area > > * @name: The name of the area. See function cma_init_reserved_mem() > > * @res_cma: Pointer to store the created cma region. > > + * @nid: nid of the free area to find, %NUMA_NO_NODE for any node > > The bit about fallback should be also specified here. > > > * > > * This function reserves memory from early allocator. It should be > > * called by arch specific code once the early allocator (memblock or > bootmem) > > @@ -238,10 +239,10 @@ int __init cma_init_reserved_mem(phys_addr_t base, > phys_addr_t size, > > * If @fixed is true, reserve contiguous area at exactly @base. If > false, > > * reserve in range from @base to @limit. > > */ > > -int __init cma_declare_contiguous(phys_addr_t base, > > +int __init cma_declare_contiguous_nid(phys_addr_t base, > > phys_addr_t size, phys_addr_t limit, > > phys_addr_t alignment, unsigned int order_per_bit, > > - bool fixed, const char *name, struct cma **res_cma) > > + bool fixed, const char *name, struct cma > **res_cma, int nid) > > { > > phys_addr_t memblock_end = memblock_end_of_DRAM(); > > phys_addr_t highmem_start; > > --000000000000968f9905a255fa61 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Apr 2, 2020 at 4:48 PM Vlasti= mil Babka <vbabka@suse.cz> wrot= e:
On 3/26/20 10= :27 PM, Aslan Bakirov wrote:
> I've noticed that there is no interfaces exposed by CMA which woul= d let me
> to declare contigous memory on particular NUMA node.
>
> This patchset adds the ability to try to allocate contiguous memory on=
> specific node.

I would say more explicitly that 'try' here means it will fallback = to other
nodes if the specific one doesn't work. At least AFAICS that's what= it does by
calling memblock_alloc_range_nid() with exact_nid=3Dfalse.
=

Thanks for the comments, Vlastimil. I created the next = version with mentioning fallback case in the changelog.=C2=A0

> Implement a new method for declaring contigous memory on particular no= de
> and keep cma_declare_contiguous() as a wrapper.

Should there be also support for using this node spcification in the cma=3D= X boot
param?

For this, I would wait for a real usecase=C2=A0for no= w, because adding it is easy but removing is=C2=A0hard.
=C2=A0
> Signed-off-by: Aslan Bakirov <aslan@fb.com>

...

> --- a/mm/cma.c
> +++ b/mm/cma.c
> @@ -220,7 +220,7 @@ int __init cma_init_reserved_mem(phys_addr_t base,= phys_addr_t size,
>=C2=A0 }
>=C2=A0
>=C2=A0 /**
> - * cma_declare_contiguous() - reserve custom contiguous area
> + * cma_declare_contiguous_nid() - reserve custom contiguous area
>=C2=A0 =C2=A0* @base: Base address of the reserved area optional, use 0= for any
>=C2=A0 =C2=A0* @size: Size of the reserved area (in bytes),
>=C2=A0 =C2=A0* @limit: End address of the reserved memory (optional, 0 = for any).
> @@ -229,6 +229,7 @@ int __init cma_init_reserved_mem(phys_addr_t base,= phys_addr_t size,
>=C2=A0 =C2=A0* @fixed: hint about where to place the reserved area
>=C2=A0 =C2=A0* @name: The name of the area. See function cma_init_reser= ved_mem()
>=C2=A0 =C2=A0* @res_cma: Pointer to store the created cma region.
> + * @nid: nid of the free area to find, %NUMA_NO_NODE for any node

The bit about fallback should be also specified here.

>=C2=A0 =C2=A0*
>=C2=A0 =C2=A0* This function reserves memory from early allocator. It s= hould be
>=C2=A0 =C2=A0* called by arch specific code once the early allocator (m= emblock or bootmem)
> @@ -238,10 +239,10 @@ int __init cma_init_reserved_mem(phys_addr_t bas= e, phys_addr_t size,
>=C2=A0 =C2=A0* If @fixed is true, reserve contiguous area at exactly @b= ase.=C2=A0 If false,
>=C2=A0 =C2=A0* reserve in range from @base to @limit.
>=C2=A0 =C2=A0*/
> -int __init cma_declare_contiguous(phys_addr_t base,
> +int __init cma_declare_contiguous_nid(phys_addr_t base,
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0phys_addr_t size, phys_addr_t limit,
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0phys_addr_t alignment, unsigned int order_per_bit,
> -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0bool fixed, const char *name, struct cma **res_cma)
> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0bool fixed, const char *name, struct cma **res_cma, int nid)
>=C2=A0 {
>=C2=A0 =C2=A0 =C2=A0 =C2=A0phys_addr_t memblock_end =3D memblock_end_of= _DRAM();
>=C2=A0 =C2=A0 =C2=A0 =C2=A0phys_addr_t highmem_start;

--000000000000968f9905a255fa61--