All of lore.kernel.org
 help / color / mirror / Atom feed
From: Arnd Bergmann <arnd@arndb.de>
To: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Michal Nazarewicz <mina86@mina86.com>,
	Ankita Garg <ankita@in.ibm.com>,
	Daniel Walker <dwalker@codeaurora.org>,
	Jesse Barker <jesse.barker@linaro.org>,
	Mel Gorman <mel@csn.ul.ie>,
	linux-kernel@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
	linux-mm@kvack.org, Kyungmin Park <kyungmin.park@samsung.com>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	linux-arm-kernel@lists.infradead.org,
	linux-media@vger.kernel.org
Subject: Re: [Linaro-mm-sig] [PATCH 08/10] mm: cma: Contiguous Memory Allocator added
Date: Fri, 17 Jun 2011 18:08:44 +0200	[thread overview]
Message-ID: <201106171808.44178.arnd@arndb.de> (raw)
In-Reply-To: <BANLkTi=XTJuF4np7+rYHzJqWK20OxMrBsw@mail.gmail.com>

On Wednesday 15 June 2011, Daniel Vetter wrote:
> On Tue, Jun 14, 2011 at 20:30, Arnd Bergmann <arnd@arndb.de> wrote:
> > On Tuesday 14 June 2011 18:58:35 Michal Nazarewicz wrote:
> >> Ah yes, I forgot that separate regions for different purposes could
> >> decrease fragmentation.
> >
> > That is indeed a good point, but having a good allocator algorithm
> > could also solve this. I don't know too much about these allocation
> > algorithms, but there are probably multiple working approaches to this.
> 
> imo no allocator algorithm is gonna help if you have comparably large,
> variable-sized contiguous allocations out of a restricted address range.
> It might work well enough if there are only a few sizes and/or there's
> decent headroom. But for really generic workloads this would require
> sync objects and eviction callbacks (i.e. what Thomas Hellstrom pushed
> with ttm).

The requirements are quite different depending on what system you
look at. In a lot of cases, the constraints are not that tight at all,
and CMA will easily help to turn "works sometimes" into "works almost
always". Let's get there first and then look into the harder problems.

Unfortunately, memory allocation gets nondeterministic in the corner
cases, you can simply get the system into a state where you don't
have enough memory when you try to do too many things at once. This
may sound like a platitude but it's really what is behind all this:

If we had unlimited amounts of RAM, we would never need CMA, we could
simply set aside a lot of memory at boot time. Having one CMA area
with movable page eviction lets you build systems capable of doing
the same thing with less RAM than without CMA. Adding more complexity
lets you reduce that amount further.

The other aspects that have been mentioned about bank affinity and
SRAM are pretty orthogonal to the allocation, so we should also
treat them separately.

> So if this is only a requirement on very few platforms and can be
> cheaply fixed with multiple cma allocation areas (heck, we have
> slabs for the same reasons in the kernel), it might be a sensible
> compromise.

Yes, we can probably add it later when we find out what the limits
of the generic approach are. I don't really mind having the per-device
pointers to CMA areas, we just need to come up with a good way to
initialize them.

	Arnd

WARNING: multiple messages have this Message-ID (diff)
From: Arnd Bergmann <arnd@arndb.de>
To: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Michal Nazarewicz <mina86@mina86.com>,
	Ankita Garg <ankita@in.ibm.com>,
	Daniel Walker <dwalker@codeaurora.org>,
	Jesse Barker <jesse.barker@linaro.org>,
	Mel Gorman <mel@csn.ul.ie>,
	linux-kernel@vger.kernel.org, linaro-mm-sig@lists.linaro.org,
	linux-mm@kvack.org, Kyungmin Park <kyungmin.park@samsung.com>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	linux-arm-kernel@lists.infradead.org,
	linux-media@vger.kernel.org
Subject: Re: [Linaro-mm-sig] [PATCH 08/10] mm: cma: Contiguous Memory Allocator added
Date: Fri, 17 Jun 2011 18:08:44 +0200	[thread overview]
Message-ID: <201106171808.44178.arnd@arndb.de> (raw)
In-Reply-To: <BANLkTi=XTJuF4np7+rYHzJqWK20OxMrBsw@mail.gmail.com>

On Wednesday 15 June 2011, Daniel Vetter wrote:
> On Tue, Jun 14, 2011 at 20:30, Arnd Bergmann <arnd@arndb.de> wrote:
> > On Tuesday 14 June 2011 18:58:35 Michal Nazarewicz wrote:
> >> Ah yes, I forgot that separate regions for different purposes could
> >> decrease fragmentation.
> >
> > That is indeed a good point, but having a good allocator algorithm
> > could also solve this. I don't know too much about these allocation
> > algorithms, but there are probably multiple working approaches to this.
> 
> imo no allocator algorithm is gonna help if you have comparably large,
> variable-sized contiguous allocations out of a restricted address range.
> It might work well enough if there are only a few sizes and/or there's
> decent headroom. But for really generic workloads this would require
> sync objects and eviction callbacks (i.e. what Thomas Hellstrom pushed
> with ttm).

The requirements are quite different depending on what system you
look at. In a lot of cases, the constraints are not that tight at all,
and CMA will easily help to turn "works sometimes" into "works almost
always". Let's get there first and then look into the harder problems.

Unfortunately, memory allocation gets nondeterministic in the corner
cases, you can simply get the system into a state where you don't
have enough memory when you try to do too many things at once. This
may sound like a platitude but it's really what is behind all this:

If we had unlimited amounts of RAM, we would never need CMA, we could
simply set aside a lot of memory at boot time. Having one CMA area
with movable page eviction lets you build systems capable of doing
the same thing with less RAM than without CMA. Adding more complexity
lets you reduce that amount further.

The other aspects that have been mentioned about bank affinity and
SRAM are pretty orthogonal to the allocation, so we should also
treat them separately.

> So if this is only a requirement on very few platforms and can be
> cheaply fixed with multiple cma allocation areas (heck, we have
> slabs for the same reasons in the kernel), it might be a sensible
> compromise.

Yes, we can probably add it later when we find out what the limits
of the generic approach are. I don't really mind having the per-device
pointers to CMA areas, we just need to come up with a good way to
initialize them.

	Arnd

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: arnd@arndb.de (Arnd Bergmann)
To: linux-arm-kernel@lists.infradead.org
Subject: [Linaro-mm-sig] [PATCH 08/10] mm: cma: Contiguous Memory Allocator added
Date: Fri, 17 Jun 2011 18:08:44 +0200	[thread overview]
Message-ID: <201106171808.44178.arnd@arndb.de> (raw)
In-Reply-To: <BANLkTi=XTJuF4np7+rYHzJqWK20OxMrBsw@mail.gmail.com>

On Wednesday 15 June 2011, Daniel Vetter wrote:
> On Tue, Jun 14, 2011 at 20:30, Arnd Bergmann <arnd@arndb.de> wrote:
> > On Tuesday 14 June 2011 18:58:35 Michal Nazarewicz wrote:
> >> Ah yes, I forgot that separate regions for different purposes could
> >> decrease fragmentation.
> >
> > That is indeed a good point, but having a good allocator algorithm
> > could also solve this. I don't know too much about these allocation
> > algorithms, but there are probably multiple working approaches to this.
> 
> imo no allocator algorithm is gonna help if you have comparably large,
> variable-sized contiguous allocations out of a restricted address range.
> It might work well enough if there are only a few sizes and/or there's
> decent headroom. But for really generic workloads this would require
> sync objects and eviction callbacks (i.e. what Thomas Hellstrom pushed
> with ttm).

The requirements are quite different depending on what system you
look at. In a lot of cases, the constraints are not that tight at all,
and CMA will easily help to turn "works sometimes" into "works almost
always". Let's get there first and then look into the harder problems.

Unfortunately, memory allocation gets nondeterministic in the corner
cases, you can simply get the system into a state where you don't
have enough memory when you try to do too many things at once. This
may sound like a platitude but it's really what is behind all this:

If we had unlimited amounts of RAM, we would never need CMA, we could
simply set aside a lot of memory at boot time. Having one CMA area
with movable page eviction lets you build systems capable of doing
the same thing with less RAM than without CMA. Adding more complexity
lets you reduce that amount further.

The other aspects that have been mentioned about bank affinity and
SRAM are pretty orthogonal to the allocation, so we should also
treat them separately.

> So if this is only a requirement on very few platforms and can be
> cheaply fixed with multiple cma allocation areas (heck, we have
> slabs for the same reasons in the kernel), it might be a sensible
> compromise.

Yes, we can probably add it later when we find out what the limits
of the generic approach are. I don't really mind having the per-device
pointers to CMA areas, we just need to come up with a good way to
initialize them.

	Arnd

  parent reply	other threads:[~2011-06-17 16:09 UTC|newest]

Thread overview: 178+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-06-10  9:54 [PATCHv10 0/10] Contiguous Memory Allocator Marek Szyprowski
2011-06-10  9:54 ` Marek Szyprowski
2011-06-10  9:54 ` Marek Szyprowski
2011-06-10  9:54 ` [PATCH 01/10] lib: bitmap: Added alignment offset for bitmap_find_next_zero_area() Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54 ` [PATCH 02/10] lib: genalloc: Generic allocator improvements Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10 11:24   ` Alan Cox
2011-06-10 11:24     ` Alan Cox
2011-06-10 11:24     ` Alan Cox
2011-06-10 12:22     ` Marek Szyprowski
2011-06-10 12:22       ` Marek Szyprowski
2011-06-10 12:22       ` Marek Szyprowski
2011-06-10 12:52       ` Alan Cox
2011-06-10 12:52         ` Alan Cox
2011-06-10 12:52         ` Alan Cox
2011-06-10 17:16         ` Michal Nazarewicz
2011-06-10 17:16           ` Michal Nazarewicz
2011-06-10 17:16           ` Michal Nazarewicz
2011-06-14 15:49         ` [Linaro-mm-sig] " Jordan Crouse
2011-06-14 15:49           ` Jordan Crouse
2011-06-14 15:49           ` Jordan Crouse
2011-06-10  9:54 ` [PATCH 03/10] mm: move some functions from memory_hotplug.c to page_isolation.c Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54 ` [PATCH 04/10] mm: alloc_contig_freed_pages() added Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54 ` [PATCH 05/10] mm: alloc_contig_range() added Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54 ` [PATCH 06/10] mm: MIGRATE_CMA migration type added Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54 ` [PATCH 07/10] mm: MIGRATE_CMA isolation functions added Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54 ` [PATCH 08/10] mm: cma: Contiguous Memory Allocator added Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10 16:21   ` Arnd Bergmann
2011-06-10 16:21     ` Arnd Bergmann
2011-06-10 16:21     ` Arnd Bergmann
2011-06-13  9:05     ` Marek Szyprowski
2011-06-13  9:05       ` Marek Szyprowski
2011-06-13  9:05       ` Marek Szyprowski
2011-06-14 13:49       ` Arnd Bergmann
2011-06-14 13:49         ` Arnd Bergmann
2011-06-14 13:49         ` Arnd Bergmann
2011-06-14 13:55         ` Michal Nazarewicz
2011-06-14 13:55           ` Michal Nazarewicz
2011-06-14 13:55           ` Michal Nazarewicz
2011-06-14 16:03           ` Arnd Bergmann
2011-06-14 16:03             ` Arnd Bergmann
2011-06-14 16:03             ` Arnd Bergmann
2011-06-14 16:58             ` Michal Nazarewicz
2011-06-14 16:58               ` Michal Nazarewicz
2011-06-14 16:58               ` Michal Nazarewicz
2011-06-14 18:30               ` Arnd Bergmann
2011-06-14 18:30                 ` Arnd Bergmann
2011-06-14 18:30                 ` Arnd Bergmann
2011-06-14 18:40                 ` Michal Nazarewicz
2011-06-14 18:40                   ` Michal Nazarewicz
2011-06-14 18:40                   ` Michal Nazarewicz
2011-06-15  7:11                 ` Marek Szyprowski
2011-06-15  7:11                   ` Marek Szyprowski
2011-06-15  7:11                   ` Marek Szyprowski
2011-06-15  7:37                   ` Arnd Bergmann
2011-06-15  7:37                     ` Arnd Bergmann
2011-06-15  7:37                     ` Arnd Bergmann
2011-06-15  8:14                     ` Marek Szyprowski
2011-06-15  8:14                       ` Marek Szyprowski
2011-06-15  8:14                       ` Marek Szyprowski
2011-06-16  0:48                     ` [Linaro-mm-sig] " Philip Balister
2011-06-16  0:48                       ` Philip Balister
2011-06-16  0:48                       ` Philip Balister
2011-06-16  7:03                       ` Arnd Bergmann
2011-06-16  7:03                         ` Arnd Bergmann
2011-06-16  7:03                         ` Arnd Bergmann
2011-06-16  7:03                         ` Arnd Bergmann
2011-06-22  7:03                     ` Hans Verkuil
2011-06-22  7:03                       ` Hans Verkuil
2011-06-22  7:03                       ` Hans Verkuil
2011-06-22  7:32                       ` Michal Nazarewicz
2011-06-22  7:32                         ` Michal Nazarewicz
2011-06-22  7:32                         ` Michal Nazarewicz
2011-06-22 12:42                       ` Arnd Bergmann
2011-06-22 12:42                         ` Arnd Bergmann
2011-06-22 12:42                         ` Arnd Bergmann
2011-06-22 13:15                         ` Marek Szyprowski
2011-06-22 13:15                           ` Marek Szyprowski
2011-06-22 13:15                           ` Marek Szyprowski
2011-06-22 13:39                           ` Arnd Bergmann
2011-06-22 13:39                             ` Arnd Bergmann
2011-06-22 13:39                             ` Arnd Bergmann
2011-06-22 13:39                             ` Arnd Bergmann
2011-06-22 16:04                             ` Michal Nazarewicz
2011-06-22 16:04                               ` Michal Nazarewicz
2011-06-22 16:04                               ` Michal Nazarewicz
2011-06-22 15:54                           ` Michal Nazarewicz
2011-06-22 15:54                             ` Michal Nazarewicz
2011-06-22 15:54                             ` Michal Nazarewicz
2011-06-15 11:53                 ` Daniel Vetter
2011-06-15 11:53                   ` Daniel Vetter
2011-06-15 11:53                   ` Daniel Vetter
2011-06-15 13:12                   ` Thomas Hellstrom
2011-06-15 13:12                     ` Thomas Hellstrom
2011-06-15 13:12                     ` Thomas Hellstrom
2011-06-17 16:08                   ` Arnd Bergmann [this message]
2011-06-17 16:08                     ` Arnd Bergmann
2011-06-17 16:08                     ` Arnd Bergmann
2011-06-14 17:01             ` Daniel Stone
2011-06-14 17:01               ` Daniel Stone
2011-06-14 17:01               ` Daniel Stone
2011-06-14 18:58               ` Zach Pfeffer
2011-06-14 18:58                 ` Zach Pfeffer
2011-06-14 18:58                 ` Zach Pfeffer
2011-06-14 20:42                 ` Arnd Bergmann
2011-06-14 20:42                   ` Arnd Bergmann
2011-06-14 20:42                   ` Arnd Bergmann
2011-06-14 21:01                   ` Jordan Crouse
2011-06-14 21:01                     ` Jordan Crouse
2011-06-14 21:01                     ` Jordan Crouse
2011-06-15 11:27                     ` Arnd Bergmann
2011-06-15 11:27                       ` Arnd Bergmann
2011-06-15 11:27                       ` Arnd Bergmann
2011-06-15 11:27                       ` Arnd Bergmann
2011-06-15  6:29                   ` Subash Patel
2011-06-15  8:36                   ` Marek Szyprowski
2011-06-15  8:36                     ` Marek Szyprowski
2011-06-15  8:36                     ` Marek Szyprowski
2011-06-15 21:39                     ` Larry Bassel
2011-06-15 21:39                       ` Larry Bassel
2011-06-15 21:39                       ` Larry Bassel
2011-06-15 22:06                       ` Arnd Bergmann
2011-06-15 22:06                         ` Arnd Bergmann
2011-06-15 22:06                         ` Arnd Bergmann
2011-06-16 17:01                         ` Larry Bassel
2011-06-16 17:01                           ` Larry Bassel
2011-06-16 17:01                           ` Larry Bassel
2011-06-17 12:45                           ` Arnd Bergmann
2011-06-17 12:45                             ` Arnd Bergmann
2011-06-17 12:45                             ` Arnd Bergmann
2011-07-04  5:25                         ` Ankita Garg
2011-07-04  5:25                           ` Ankita Garg
2011-07-04  5:25                           ` Ankita Garg
2011-07-04 14:45                           ` Arnd Bergmann
2011-07-04 14:45                             ` Arnd Bergmann
2011-07-04 14:45                             ` Arnd Bergmann
2011-06-16  3:20                       ` Zach Pfeffer
2011-06-16  3:20                         ` Zach Pfeffer
2011-06-16  3:20                         ` Zach Pfeffer
2011-06-15  9:26                   ` Michal Nazarewicz
2011-06-15  9:26                     ` Michal Nazarewicz
2011-06-15  9:26                     ` Michal Nazarewicz
2011-06-15 11:20                     ` Arnd Bergmann
2011-06-15 11:20                       ` Arnd Bergmann
2011-06-15 11:20                       ` Arnd Bergmann
2011-06-15 11:30                       ` Michal Nazarewicz
2011-06-15 11:30                         ` Michal Nazarewicz
2011-06-15 11:30                         ` Michal Nazarewicz
2011-06-15  6:01             ` Subash Patel
2011-06-15  6:01               ` Subash Patel
2011-06-15  6:01               ` Subash Patel
2011-06-15  8:02         ` Marek Szyprowski
2011-06-15  8:02           ` Marek Szyprowski
2011-06-15  8:02           ` Marek Szyprowski
2011-06-15 11:14           ` Arnd Bergmann
2011-06-15 11:14             ` Arnd Bergmann
2011-06-15 11:14             ` Arnd Bergmann
2011-06-10  9:54 ` [PATCH 09/10] ARM: integrate CMA with dma-mapping subsystem Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54 ` [PATCH 10/10] ARM: S5PV210: add CMA support for FIMC devices on Aquila board Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski
2011-06-10  9:54   ` Marek Szyprowski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=201106171808.44178.arnd@arndb.de \
    --to=arnd@arndb.de \
    --cc=akpm@linux-foundation.org \
    --cc=ankita@in.ibm.com \
    --cc=daniel.vetter@ffwll.ch \
    --cc=dwalker@codeaurora.org \
    --cc=jesse.barker@linaro.org \
    --cc=kamezawa.hiroyu@jp.fujitsu.com \
    --cc=kyungmin.park@samsung.com \
    --cc=linaro-mm-sig@lists.linaro.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-media@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mel@csn.ul.ie \
    --cc=mina86@mina86.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.