[LSF/MM/BPF TOPIC] File system synthesis inside of computational storage

[LSF/MM/BPF TOPIC] File system synthesis inside of computational storage

[Viacheslav A.Dubeyko]

Hello,

Computational storage creates the way to offload as user data as metadata processing into storage space. So, I would like to discuss the feasibility of building a storage device that is capable of synthesizing a file system by itself.

First of all, it is well known that designing and implementing a file system architecture is a very time-consuming and difficult process. Also this process includes a significant amount of debugging and bug fix efforts. But real life and evolving technologies require much faster data processing. So, file system technologies cannot satisfy the challenges of real life very frequently. Generally speaking, if computational storage is capable of designing and evolving file system architecture under I/O load then it could be a very interesting solution.

Any machine learning approach is based on training by input data. So, I/O requests can be such training data. It is possible to imagine a ML engine inside of computational storage that can analyze the requests to store and to retrieve the data and to build the file system’s metadata structures on the fly. Generally speaking, computational storage can implement a file system architecture inside of a storage device without any preliminary development of a file system driver.

Initially, the ML engine inside of computational storage could be equipped with the initial set of metadata structure primitives (array, queue, bitmap, tree). By receiving I/O requests, ML engine can use at first the simplest metadata structures and to gather the modification/access statistics. For example, initially, it is possible to use a simple array of extents to account file’s content location. Then, with a growing amount of statistics, ML engine can synthesize more sophisticated and efficient metadata structures and start to use, for example, a tree that consumes existing arrays as tree’s nodes. As a result, a file system can be constructed by an ML engine taking into account the peculiarities of files’ lifetime and workload.

Any opinions related to this crazy idea?

Thanks,
Slava.