Re: [PATCH RFC] fs: New zonefs file system

[Damien Le Moal <Damien.LeMoal@wdc.com>]

Jeff,

On 2019/07/19 23:25, Jeff Moyer wrote:
> OK, I can see how a file system eases adoption across multiple
> languages, and may, in some cases, be easier to adopt by applications.
> However, I'm not a fan of the file system interface for this usage.
> Once you present a file system, there are certain expectations from
> users, and this fs breaks most of them.

Your comments got me thinking more about zonefs specifications/features and I am
now wondering if I am not pushing this too far in terms of simplicity. So here
is a new RFC/Question to chew on... While keeping as a target the concept of
"file == zone" or as close to it as possible, what do you think zonefs minimal
feature set should be ?

One idea I have since a while back now is this:
1) If a zone is unused, do not show a file for it. This means adding a dynamic
"zone allocation" code and supporting O_CREAT on open, unlink, etc. So have more
normal file system calls behave as with a normal FS.
2) Allow file names to be decided by the user instead of using a fixed names.
Again, have O_CREAT behave as expected
3) Potentially allow files to grow beyond a single zone, while keeping the space
allocation unit as a zone.

Thinking of our current LevelDB/RocksDB use cases, (1) and (2) would allow even
further simplifying the support code since with these, the SSTable file
management can essentially stay completely untouched.

(3) is not necessary for LSM-Tree type use cases since typically zones are large
and so aligning SSTables to zones the most efficient approach. However, I can
see other use cases that would benefit from (3). One example would be
Surveillance system video recording or any system dealing with high bitrate
Video. E.g. A 256 MB zone size is only 100s of high definition broadcasting (20
Mbps or so). SO managing storage space in such big chunks is OK with such use cases.

These 3 additional features would make zonefs much closer to a regular FS
behavior while keeping its IO path simple enough to be in par with fast raw
block device accesses. Additional metadata management, completely absent for
now, would be needed though. But by not allowing directories (flat namespace),
this metadata management would be reduced to an inode table and a bitmap for
zone use management. Anything beyond these features and I think we would be
better off with a regular file system.

Thoughts ?


-- 
Damien Le Moal
Western Digital Research