From: Meneghini, John <John.Meneghini at netapp.com>
To: spdk@lists.01.org
Subject: Re: [SPDK] spdk_malloc vs. malloc
Date: Thu, 03 May 2018 20:14:36 +0000 [thread overview]
Message-ID: <7B51D052-0906-4E4C-BE69-2EB587C95E27@netapp.com> (raw)
In-Reply-To: 1525375288.22849.48.camel@intel.com
[-- Attachment #1: Type: text/plain, Size: 3295 bytes --]
Yes, this is the same discussion we had last time about POSIX.
Yes, we are using include/stdinc.h to abstract malloc() and calloc(). These are not big patches. This isn't the issue.
The issue is: spdk_malloc and spdk_calloc are logical APIs to abstract the POSIX malloc and calloc calls. We chose to NOT use those API names originally and went with the spdk_dma_{} api names. My concern is that taking those API names now will eliminate the possibility of any other use in the future.
I am not asking to "fully abstract POSIX". I am simply saying that the malloc(), calloc() and free() POSIX APIs suffer from some problems. This is why DPDK implemented rte_malloc() and rte_calloc(). As we move forward to build production quality products out of SPDK we will want to abstract malloc, calloc and free; especially in the libraries. I don't care about what the applications do. I want a better memory management APIs in the libraries.
I'd be happy to simply map spdk_malloc/calloc() to rte_malloc/calloc() and I don't see the point introducing an spdk_malloc() that only extends spdk_dma_malloc(). I'd rather absorb an API change to spdk_dma_malloc(). We change many APIs in SPDK from release to release. I don't see why this API can't be changed - with agreement from the SPDK community.
/*
* Allocate memory on default heap.
*/
void *
rte_malloc(const char *type, size_t size, unsigned align)
{
return rte_malloc_socket(type, size, align, SOCKET_ID_ANY);
}
/*
* Allocate zero'd memory on default heap.
*/
void *
rte_calloc(const char *type, size_t num, size_t size, unsigned align)
{
return rte_zmalloc(type, num * size, align);
}
/John
On 5/3/18, 3:21 PM, "Walker, Benjamin" <benjamin.walker(a)intel.com> wrote:
On Thu, 2018-05-03 at 18:23 +0000, Meneghini, John wrote:
> > If the user passes flags == 0 to the new spdk_malloc() call, this could be
> implemented by malloc() or equivalent behind the scenes,
>
> So, does this mean you’re willing to change all calls to malloc(size) with
> spdk_malloc(size, 0, NULL, SPDK_ENV_SOCKET_ID_ANY, SPDK_MALLOC_SHARE)?
>
> Is that the plan?
>
Hi John,
SPDK explicitly depends on POSIX throughout the code base. We do have an
environment abstraction layer designed to make porting SPDK to various
environments (i.e. away from DPDK) easier, but this only abstracts operations
that are not available through standard POSIX calls. We're concerned that fully
abstracting POSIX would introduce a significant barrier to entry for new
contributors to the project, while only enabling one additional use case that
we're aware of. I understand that this one use case happens to be yours, and so
this is a challenge for you.
In your code today, I assume you carry patches to replace the POSIX calls with
your available alternatives. We've attempted to make carrying these patches
reasonably easy by moving all of the POSIX includes into a single header
(include/stdinc.h). Since you're already carrying those patches, can you simply
choose a different name for your replacement for malloc/calloc?
Thanks,
Ben
next reply other threads:[~2018-05-03 20:14 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-05-03 20:14 Meneghini, John [this message]
-- strict thread matches above, loose matches on Subject: below --
2018-05-08 19:01 [SPDK] spdk_malloc vs. malloc Rodriguez, Edwin
2018-05-08 18:10 Walker, Benjamin
2018-05-08 17:57 Rodriguez, Edwin
2018-05-08 17:53 Walker, Benjamin
2018-05-08 17:35 Rodriguez, Edwin
2018-05-07 17:38 Rodriguez, Edwin
2018-05-07 17:18 Walker, Benjamin
2018-05-05 0:17 Harris, James R
2018-05-04 22:37 Rodriguez, Edwin
2018-05-04 22:29 Meneghini, John
2018-05-03 23:24 Walker, Benjamin
2018-05-03 19:21 Walker, Benjamin
2018-05-03 18:23 Meneghini, John
2018-05-03 18:09 Harris, James R
2018-05-03 18:05 Verkamp, Daniel
2018-05-03 17:53 Meneghini, John
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=7B51D052-0906-4E4C-BE69-2EB587C95E27@netapp.com \
--to=spdk@lists.01.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.