Re: [PATCH 5/9] mm: Let sparse_{add,remove}_one_section receive a node_id

From: Boaz Harrosh <boaz@plexistor.com>
To: Dave Hansen <dave.hansen@intel.com>,
	Ross Zwisler <ross.zwisler@linux.intel.com>,
	Jens Axboe <axboe@fb.com>,
	Matthew Wilcox <matthew.r.wilcox@intel.com>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	linux-nvdimm@ml01.01.org, Toshi Kani <toshi.kani@hp.com>,
	linux-mm@kvack.org
Cc: Andrew Morton <akpm@linux-foundation.org>,
	linux-kernel <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 5/9] mm: Let sparse_{add,remove}_one_section receive a node_id
Date: Wed, 10 Sep 2014 13:07:23 +0300	[thread overview]
Message-ID: <541022DB.9090000@plexistor.com> (raw)
In-Reply-To: <540F48BA.2090304@intel.com>

On 09/09/2014 09:36 PM, Dave Hansen wrote:
> On 09/09/2014 08:45 AM, Boaz Harrosh wrote:
>> This is for add_persistent_memory that will want a section of pages
>> allocated but without any zone associated. This is because belonging
>> to a zone will give the memory to the page allocators, but
>> persistent_memory belongs to a block device, and is not available for
>> regular volatile usage.
> 
> I don't think we should be taking patches like this in to the kernel
> until we've seen the other side of it.  Where is the page allocator code
> which will see a page belonging to no zone?  Am I missing it in this set?
> 

It is not missing. It will never be.

These pages do not belong to any allocator. They are not allocate-able
pages. In fact they are not "memory" they are "storage"

These pages belong wholesomely to a block-device. In turn the block
device grants ownership of a partition of this pages to an FS.
The FS loaded has its own block allocation schema. Which internally
circulate each pages usage around. But the page never goes beyond its
FS.

> I see about 80 or so calls to page_zone() in the kernel.  How will a
> zone-less page look to all of these sites?
> 

None of these 80 call site will be reached! the pages are always used
below the FS, like send them on the network, or send them to a slower
block device via a BIO. I have a full fledge FS on top of this code
and it all works very smoothly, and stable. (And fast ;))

It is up to the pMem-based FS to manage its pages's ref count so they are
never released outside of its own block allocator.

at the end of the day, struct pages has nothing to do with zones
and allocators and "memory", as it says in Documentation struct
page is a facility to track the state of a physical page in the
system. All the other structures are higher in the stack above
the physical layer, struct-pages for me are the upper API of the
memory physical layer. Which are in common with pmem, higher
on the stack where with memory we have a zone, pmem has a block-device.
Higher where we have page allocators, pmem has an FS block allocator,
higher where we have a slab, pmem has files for user consumption.

pmem is storage, which shares the physical layer with memory, and
this is what this patch describes. There will be no more mm interaction
at all for pmem. The rest of the picture is all there in plain site as
part of this patchset, the pmem.c driver then an FS on top of that. What
else do you need to see?

Thanks
Boaz