From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753665Ab2HSLej (ORCPT ); Sun, 19 Aug 2012 07:34:39 -0400 Received: from mail.nonattached.net ([193.160.39.60]:55054 "EHLO mail.nonattached.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751410Ab2HSLef (ORCPT ); Sun, 19 Aug 2012 07:34:35 -0400 Subject: Re: [PATCH] fs: Introducing Lanyard Filesystem From: Dan Luedtke To: Marco Stornelli Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, lanyfs@librelist.com In-Reply-To: <5030BC7E.3030000@gmail.com> References: <1345333117-2826-1-git-send-email-mail@danrl.de> <1345371135.4441.2.camel@tunafish> <5030BC7E.3030000@gmail.com> Content-Type: text/plain; charset="UTF-8" Date: Sun, 19 Aug 2012 15:34:24 +0200 Message-ID: <1345383264.4441.56.camel@tunafish> Mime-Version: 1.0 X-Mailer: Evolution 2.32.3 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, 2012-08-19 at 12:14 +0200, Marco Stornelli wrote: > You say that you wrote a new fs because of some lacks in the other fs > ("...I kind of invented a very simple filesystem that solves the issues > I had with other filesystems...."). I read the website (very quickly > actually) but I didn't find the point: I had to be a little bit stingy with that, I haven't submitted my thesis document yet, and I don't want to publish it before that. I may not be allowed to do so before it is officially submission, IANAL. At the moment, the website lacks of the information you were looking for, sorry for that. The thesis contains also a lot of statistical data and fancy diagrams and stuff like that. I analyzed about 600k file stored on various removable storage devices. 80 volunteers sent in data about their devices, generated by a program (windows) and scripts (linux, bsd, osx) I wrote for that purpose. The data shows that people use more complex filesystems as soon as they are confronted with problems (mostly the 4GB limit). After that they have problems getting their data accessed by other systems. I derived from that, that we need a filesystem that is so simple that even unpopular operating systems can implement it without having their business plan explode. The data will be made public (like open data) after submitting the document. Others are then free to derive statements from it, may then verify or contradict my conclusions. > what are pros and cons of this fs > compared with existing fs? Pros: - Simplicity. LanyFS avoids any unnecessary complexity. Example: I had a lot of problems reading and writing data on the Arduino platform. Most of it was, because the platform wasn't capable of dealing with FAT32 when files grew. I don't know if LanyFS performs better on Arduino, I haven't written a driver for Arduino yet, I am glad I managed to get it working on Linux for the moment. - Interoperability. LanyFS was designed to unite those features which are common on most operating systems. [...] All information, including file and directory metadata, is stored in the most purposive format without honoring the habits of any particular operating system. Example: I have a nice FullHD movie file which is about 8GB in size. I wanted to play that video file on a RaspberryPI connected to a TV. FAT32 wasn't able to store the file because it was larger than 4GB. I then tried using ext3, but the ownership information from my workstation (were the file was copied from) did not match the ones the RaspberryPI had, since I usually do not synchronize user profiles between workstations and embedded devices. There are workarounds, one might say, but shouldn't it be much more easier to transfer files from one device to another? There are use cases were modes/permissions and ownership of files does not matter, where quota and accounting isn't necessary, where access times don't need to be stored since they would never get updated anyway. Flexibility. LanyFS adopts to the underlying storage device by adjusting its parameters accordingly. It can address block devices starting from 4 KiB up to 64 ZiB with minimal overhead by parameterizing the filesystem at formatting time. Example: At the university we sometimes work with smart cards and their sometimes "strange" filesystems. I am not an expert on smart cards and stuff like that, but I was being told that a more simple filesystem would be welcome. AFAIK we are talking about "disks" with a size of less than 1MB. I don't think this is a big feature, most other fs can be adjusted to perfectly fit a particular purpose, too. Yet there still is the demand for something "much more simpler". Cons: - LanyFS is featureless. It lacks of features. This is considered a feature by some people, and a big con by others :) - Use of recursion. It may not scale in some cases (e.g. very large files on embedded platforms). - Current restrictions due to early stage. Max blocksize is 4k, but 4MB might be better. The LWN article pointed to by Alan talks about this. - No use of MTD/UBI features. LanyFS expects a block layer abstraction beneath it, this is because it targets highly mobile devices, and those devices also happen to be rotating disk (external hard drives) sometimes. It is not optimized for flash-only devices, although most of the devices it may run on one day will probably be flash-based. > What problems does it solve? - I can now watch >4GB movie files from USB thumb drive :) - Providing a storage device interface will be easier for vendors. TVs, car entertainment systems, digital cameras, sensor devices, navigation systems and all kind of other devices often use FAT32 since it just works on everyones computer, but it is limit in size. Other fs are there to hop in, but vendors often refuse to implement them since they have to be compatible with a lot of features. They won't face such problems with LanyFS, it just has no features. Of course, I am just a student, I have no money and no market power, so LanyFS might never make it into other major operating systems or all the devices around us. - Look around, there are so many devices that do not provide a storage interface at the moment, but they easily could. I'd rather see my electricity meter log my power consumption to a thumb drive than sending it into the cloud every 5 minutes. This whole thing isn't about the patch anymore, is it? Should we go off list to not make so much noise disturbing other developers? Thank you very much for your interest in LanyFS and please don't hesitate to point out errors in the code, too. Thanks Dan -- Dan Luedtke http://www.danrl.de