Re: [PATCH RFC 0/7] block: Introduce CBD (CXL Block Device)

From: Dongsheng Yang <dongsheng.yang@easystack.cn>
To: Gregory Price <gregory.price@memverge.com>
Cc: Dan Williams <dan.j.williams@intel.com>,
	axboe@kernel.dk, John Groves <John@groves.net>,
	linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-cxl@vger.kernel.org,
	Dongsheng Yang <dongsheng.yang.linux@gmail.com>
Subject: Re: [PATCH RFC 0/7] block: Introduce CBD (CXL Block Device)
Date: Fri, 26 Apr 2024 09:25:53 +0800	[thread overview]
Message-ID: <bbf692ec-2109-baf2-aaae-7859a8315025@easystack.cn> (raw)
In-Reply-To: <ZikhwAAIGFG0UU23@memverge.com>

在 2024/4/24 星期三 下午 11:14, Gregory Price 写道:
> On Wed, Apr 24, 2024 at 02:33:28PM +0800, Dongsheng Yang wrote:
>>
>>
>> 在 2024/4/24 星期三 下午 12:29, Dan Williams 写道:
>>> Dongsheng Yang wrote:
>>>> From: Dongsheng Yang <dongsheng.yang.linux@gmail.com>
>>>>
>>>> Hi all,
>>>> 	This patchset introduce cbd (CXL block device). It's based on linux 6.8, and available at:
>>>> 	https://github.com/DataTravelGuide/linux
>>>>
>>> [..]
>>>> (4) dax is not supported yet:
>>>> 	same with famfs, dax device is not supported here, because dax device does not support
>>>> dev_dax_iomap so far. Once dev_dax_iomap is supported, CBD can easily support DAX mode.
>>>
>>> I am glad that famfs is mentioned here, it demonstrates you know about
>>> it. However, unfortunately this cover letter does not offer any analysis
>>> of *why* the Linux project should consider this additional approach to
>>> the inter-host shared-memory enabling problem.
>>>
>>> To be clear I am neutral at best on some of the initiatives around CXL
>>> memory sharing vs pooling, but famfs at least jettisons block-devices
>>> and gets closer to a purpose-built memory semantic.
>>>
>>> So my primary question is why would Linux need both famfs and cbd? I am
>>> sure famfs would love feedback and help vs developing competing efforts.
>>
>> Hi,
>> 	Thanks for your reply, IIUC about FAMfs, the data in famfs is stored in
>> shared memory, and related nodes can share the data inside this file system;
>> whereas cbd does not store data in shared memory, it uses shared memory as a
>> channel for data transmission, and the actual data is stored in the backend
>> block device of remote nodes. In cbd, shared memory works more like network
>> to connect different hosts.
>>
> 
> Couldn't you basically just allocate a file for use as a uni-directional
> buffer on top of FAMFS and achieve the same thing without the need for
> additional kernel support? Similar in a sense to allocating a file on
> network storage and pinging the remote host when it's ready (except now
> it's fast!)

I'm not entirely sure I follow your suggestion. I guess it means that 
cbd would no longer directly manage the pmem device, but allocate files 
on famfs to transfer data. I didn't do it this way because I considered 
at least a few points: one of them is, cbd_transport actually requires a 
DAX device to access shared memory, and cbd has very simple requirements 
for space management, so there's no need to rely on a file system layer, 
which would increase architectural complexity.

However, we still need cbd_blkdev to provide a block device, so it 
doesn't achieve "achieve the same without the need for additional kernel 
support".

Could you please provide more specific details about your suggestion?
> 
> (The point here is not "FAMFS is better" or "CBD is better", simply
> trying to identify the function that will ultimately dictate the form).

Thank you for your clarification. totally aggree with it, discussions 
always make the issues clearer.

Thanx
> 
> ~Gregory
> 