All of lore.kernel.org
 help / color / mirror / Atom feed
From: Richard Weinberger <richard.weinberger@gmail.com>
To: Amir Goldstein <amir73il@gmail.com>
Cc: ezemtsov@google.com,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	Theodore Tso <tytso@mit.edu>, Miklos Szeredi <miklos@szeredi.hu>
Subject: Re: Initial patches for Incremental FS
Date: Thu, 2 May 2019 20:16:48 +0200	[thread overview]
Message-ID: <CAFLxGvyfeuwhX=9Fu2XAoT7mxgKmr7T=238y8Mf8yAZnNXnOhg@mail.gmail.com> (raw)
In-Reply-To: <CAOQ4uxhmDjYY5_UVWYAWXPtD1jFh3H5Bqn1qn6Fam0KZZjyprw@mail.gmail.com>

On Thu, May 2, 2019 at 1:21 PM Amir Goldstein <amir73il@gmail.com> wrote:
>
> On Thu, May 2, 2019 at 12:04 AM <ezemtsov@google.com> wrote:
> >
> > Hi All,
> >
> > Please take a look at Incremental FS.
> >
> > Incremental FS is special-purpose Linux virtual file system that allows
> > execution of a program while its binary and resource files are still being
> > lazily downloaded over the network, USB etc. It is focused on incremental
> > delivery for a small number (under 100) of big files (more than 10 megabytes each).
> > Incremental FS doesn’t allow direct writes into files and, once loaded, file
> > content never changes. Incremental FS doesn’t use a block device, instead it
> > saves data into a backing file located on a regular file-system.
> >
> > What’s it for?
> >
> > It allows running big Android apps before their binaries and resources are
> > fully loaded to an Android device. If an app reads something not loaded yet,
> > it needs to wait for the data block to be fetched, but in most cases hot blocks
> > can be loaded in advance and apps can run smoothly and almost instantly.
>
> This sounds very useful.
>
> Why does it have to be a new special-purpose Linux virtual file?
> Why not FUSE, which is meant for this purpose?
> Those are things that you should explain when you are proposing a new
> filesystem,
> but I will answer for you - because FUSE page fault will incur high
> latency also after
> blocks are locally available in your backend store. Right?
>
> How about fscache support for FUSE then?
> You can even write your own fscache backend if the existing ones don't
> fit your needs for some reason.
>
> Do you know of the project https://vfsforgit.org/?
> Not exactly the same use case but very similar.
> There is ongoing work on a Linux port developed by GitHub.com:
> https://github.com/github/libprojfs
>
> Piling logic into the kernel is not the answer.
> Adding the missing interfaces to the kernel is the answer.

I wonder whether userfaultfd can but used for that use-case too?

-- 
Thanks,
//richard

  parent reply	other threads:[~2019-05-02 18:17 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-05-02  4:03 Initial patches for Incremental FS ezemtsov
2019-05-02  4:03 ` [PATCH 1/6] incfs: Add first files of incrementalfs ezemtsov
2019-05-02 19:06   ` Miklos Szeredi
2019-05-02 20:41   ` Randy Dunlap
2019-05-07 15:57   ` Jann Horn
2019-05-07 17:13   ` Greg KH
2019-05-07 17:18   ` Greg KH
2019-05-02  4:03 ` [PATCH 2/6] incfs: Backing file format ezemtsov
2019-05-02  4:03 ` [PATCH 3/6] incfs: Management of in-memory FS data structures ezemtsov
2019-05-02  4:03 ` [PATCH 4/6] incfs: Integration with VFS layer ezemtsov
2019-05-02  4:03 ` [PATCH 6/6] incfs: Integration tests for incremental-fs ezemtsov
2019-05-02 11:19 ` Initial patches for Incremental FS Amir Goldstein
2019-05-02 13:10   ` Theodore Ts'o
2019-05-02 13:26     ` Al Viro
2019-05-03  4:23       ` Eugene Zemtsov
2019-05-03  5:19         ` Amir Goldstein
2019-05-08 20:09           ` Eugene Zemtsov
2019-05-09  8:15             ` Amir Goldstein
     [not found]               ` <CAK8JDrEQnXTcCtAPkb+S4r4hORiKh_yX=0A0A=LYSVKUo_n4OA@mail.gmail.com>
2019-05-21  1:32                 ` Yurii Zubrytskyi
2019-05-22  8:32                   ` Miklos Szeredi
2019-05-22 17:25                     ` Yurii Zubrytskyi
2019-05-23  4:25                       ` Miklos Szeredi
2019-05-29 21:06                         ` Yurii Zubrytskyi
2019-05-30  9:22                           ` Miklos Szeredi
2019-05-30 22:45                             ` Yurii Zubrytskyi
2019-05-31  9:02                               ` Miklos Szeredi
2019-05-22 10:54                   ` Amir Goldstein
2019-05-03  7:23         ` Richard Weinberger
2019-05-03 10:22         ` Miklos Szeredi
2019-05-02 13:46     ` Amir Goldstein
2019-05-02 18:16   ` Richard Weinberger [this message]
2019-05-02 18:33     ` Richard Weinberger
2019-05-02 13:47 ` J. R. Okajima

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAFLxGvyfeuwhX=9Fu2XAoT7mxgKmr7T=238y8Mf8yAZnNXnOhg@mail.gmail.com' \
    --to=richard.weinberger@gmail.com \
    --cc=amir73il@gmail.com \
    --cc=ezemtsov@google.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=miklos@szeredi.hu \
    --cc=tytso@mit.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.