Re: robinhood, fanotify name info events and lustre changelog

[Dominique Martinet <asmadeus@codewreck.org>]

Answering what I can until Quentin chips back in.

Amir Goldstein wrote on Sat, May 30, 2020:
> Nice. thanks for explaining that.
> I suppose you need to store the calculated path attribute for things like
> index queries on the database?

Either querying for a subtree or simply printing the path (rbh-find
would print path by default, like find does)

> > So, to be fair, full paths _are_ computed solely from information in the
> > changelog records, even though it requires a bit of processing on the side.
> > No additional query to the filesystem for that.
> 
> As I wrote, that fact that robinhood trusts the information in changelog
> records doesn't mean that information needs to arrive from the kernel.
> The adapter code should use information provided by fanotify events
> then use open_by_handle_at(2) for directory fid to finds its current
> path in the filesystem then feed that information to a robinhood change
> record.

I can agree with that - it's not because for lustre we made the decision
to be able to run without querying the filesystem at all that it has to
hold true for all type of inputs.

> I would be happy to work with you on a POC for adapting fanotify
> test code with robinhood v4, but before I invest time on that, I would
> need to know there is a good chance that people are going to test and
> use robinhood with Linux vfs.
>
> Do you have actual users requesting to use robinhood with non-Lustre
> fs?

I would run it at home, but that isn't much :D
As I wrote previously we have users for large nfs shares out of lustre,
but I honestly don't think there will be much use for local filesystems
at least in the short term.

Filesystem indexers like tracker[1] or similar would definitely get much
more use for that; from an objective point of view I wouldn't suggest
you spend time on robinhood for this: local filesytems are rarely large
enough to warrant using something like robinhood, and without something
like fanotify we wouldn't be efficient for a local disk with hundreds of
millions of files anyway because of the prohibitive rescan cost - so
it's a bit like chicken and egg maybe, I don't know, but if you want
many users to test different configurations I wouldn't recommend
robinhood (OTOH, we run CI tests so would be happy to add that to the
tests once it's available on vanilla kernels; but that's still not real
users)

[1] https://wiki.gnome.org/Projects/Tracker


> May I ask, what is the reason for embarking on the project to decouple
> robinhood v4 API from Lustre changelog API?
> Is it because you had other fsevent producers in mind?

I've been planning to at least add some recursive-inotifywatch a
subfolder at least (like watchman does) before these new fanotify events
came up, so I might be partly to blame for that.

There also are advantages for lustre though; the point is to be able to
ingest changelogs directly with some daemon (it's only at proof of
concept level for v4 at this point), but also to split the load by
involving multiple lustre clients.
So you would get a pool of lustre clients to read changelogs, a pool of
lustre clients to stat files as required to enrich the fsevents (file
size etc), and a pool of servers to read fsevents and commit changes to
the database (this part is still at the design level afaik)


Hope this all makes sense,
-- 
Dominique