Re: [ATTEND] many topics

[Vlastimil Babka <vbabka@suse.cz>]

On 01/23/2017 08:34 PM, NeilBrown wrote:
> On Tue, Jan 24 2017, Theodore Ts'o wrote:
>
>> On Sun, Jan 22, 2017 at 10:05:44PM -0800, Matthew Wilcox wrote:
>>>
>>> I don't have a clear picture in my mind of when Java promotes objects
>>> from nursery to tenure
>>
>> It's typically on the order of minutes.   :-)
>>
>>> ... which is not too different from my lack of
>>> understanding of what the MM layer considers "temporary" :-)  Is it
>>> acceptable usage to allocate a SCSI command (guaranteed to be freed
>>> within 30 seconds) from the temporary area?  Or should it only be used
>>> for allocations where the thread of control is not going to sleep between
>>> allocation and freeing?
>>
>> What the mm folks have said is that it's to prevent fragmentation.  If
>> that's the optimization, whether or not you the process is allocating
>> the memory sleeps for a few hundred milliseconds, or even seconds, is
>> really in the noise compared with the average lifetime of an inode in
>> the inode cache, or a page in the page cache....
>>
>> Why do you think it matters whether or not we sleep?  I've not heard
>> any explanation for the assumption for why this might be important.
>
> Because "TEMPORARY" implies a limit to the amount of time, and sleeping
> is the thing that causes a process to take a large amount of time.  It
> seems like an obvious connection to me.

There's no simple connection to time, it depends on the larger picture - what's 
the state of the allocator and what other allocations/free's are happening 
around this one. Perhaps let me try to explain what the flag does and what 
benefits are expected.

GFP_TEMPORARY, compared to GFP_KERNEL, adds __GFP_RECLAIMABLE, which tries to 
place the allocation within MIGRATE_RECLAIMABLE pageblocks - GFP_KERNEL implies 
MIGRATE_UNMOVABLE pageblocks, and userspace allocations are typically 
MIGRATE_MOVABLE. The main goal of this "mobility grouping" is to prevent the 
unmovable pages spreading all over the memory, making it impossible to get 
larger blocks by defragmentation (compaction). Ideally we would have all these 
problematic pages fit neatly into the smallest possible number of pageblocks 
that can accomodate them. But we can't know in advance how many, and we don't 
know their lifetimes, so there are various heuristics for relabeling pageblocks 
between the 3 types as we exceed the existing ones.

Now GFP_TEMPORARY means we tell the allocator about the relatively shorter 
lifetime, so it places the allocation within the RECLAIMABLE pageblocks, which 
are also used for slab caches that have shrinkers. The expected benefit of this 
is that we potentially prevent growing the number of UNMOVABLE pageblocks 
(either directly by this allocation, or a subsequent GFP_KERNEL one, that would 
otherwise fit within the existing pageblocks). While the RECLAIMABLE pages also 
cannot be defragmented (at least currently, there are some proposals for the 
slab caches...), we can at least shrink them, so the negative impact on 
compaction is considered less severe in the longer term.

> Imagine I want to allocate a large contiguous region in the
> ZONE_MOVEABLE region.  I find a mostly free region, so I just need to
> move those last few pages.  If there is a limit on how long a process
> can sleep while holding an allocation from ZONE_MOVEABLE, then I know
> how long, at most, I need to wait before those pages become either free
> or movable.  If those processes can wait indefinitely, then I might have
> to wait indefinitely to get this large region.

Yeah so this is not relevant, because GFP_TEMPORARY does not make the allocation 
__GFP_MOVABLE, so it still is not allowed to end up within a ZONE_MOVABLE zone. 
Unfortunately the issue similar to that you mention does still exist due to 
uncontrolled pinning of the movable pages, which affects both ZONE_MOVABLE and 
CMA, but that's another story...

> "temporary" doesn't mean anything without a well defined time limit.
>
> But maybe I completely misunderstand.

HTH,
Vlastimil

> NeilBrown
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>