Re: another pmem variant V2

From: Boaz Harrosh <boaz@plexistor.com>
To: Christoph Hellwig <hch@lst.de>
Cc: linux-nvdimm@ml01.01.org, linux-fsdevel@vger.kernel.org,
	linux-kernel@vger.kernel.org, x86@kernel.org,
	ross.zwisler@linux.intel.com, axboe@kernel.dk
Subject: Re: another pmem variant V2
Date: Thu, 26 Mar 2015 19:31:03 +0200	[thread overview]
Message-ID: <55144257.8050004@plexistor.com> (raw)
In-Reply-To: <20150326171858.GA25575@lst.de>

On 03/26/2015 07:18 PM, Christoph Hellwig wrote:
> On Thu, Mar 26, 2015 at 06:57:47PM +0200, Boaz Harrosh wrote:
>> For one this auto discovery of yours is very (very) nice but is a bit
>> inconvenience. Before I would reserve a big chuck on each NUMA range
>> on Kernel's memmap=  And then at pmem map= would slice and dice it
>> as I want hot style on modprobe with no need for reboot. Now I need
>> to do it on reboot theoretically. (You know xfstest needs lots of devices
>> some big some small ;-))
> 
> Slicing up a block device based on kernel options is not exactly a smart
> idea.  We have partitions that are perfectly fine for that.  If you
> really don't are about persistance of your partitioning you can just
> set up a device mapper table.  No need to reinvent the wheel.
> 

I know! fdisk is my friend, I know.

But I hope you are not ignoring my real problem. any two memmap= ranges
will halt the boot. Specially if they are dis-contiguous.

Also I need the contiguous variant split into two devices because
they might belong to two NUMA nodes. It is very hard to manage
if a NUMA crossing is in a middle of a single pmemX device
The way we like to configure it is that each /dev/pmem belongs to
a single NUMA node. And in a multy device setup each CPU node
allocates from "his" pmem device If there is space.
(And it lets me set application affinity if need to)

BTW: Will device mapper let me call ->direct_access()

Thanks
Boaz