From: Amir Goldstein <amir73il@gmail.com>
To: Jan Kara <jack@suse.cz>
Cc: linux-fsdevel <linux-fsdevel@vger.kernel.org>
Subject: thoughts about fanotify and HSM
Date: Sun, 11 Sep 2022 21:12:06 +0300 [thread overview]
Message-ID: <CAOQ4uxhrQ7hySTyHM0Atq=uzbNdHyGV5wfadJarhAu1jDFOUTg@mail.gmail.com> (raw)
Hi Jan,
I wanted to consult with you about preliminary design thoughts
for implementing a hierarchical storage manager (HSM)
with fanotify.
I have been in contact with some developers in the past
who were interested in using fanotify to implement HSM
(to replace old DMAPI implementation).
Basically, FAN_OPEN_PERM + FAN_MARK_FILESYSTEM
should be enough to implement a basic HSM, but it is not
sufficient for implementing more advanced HSM features.
Some of the HSM feature that I would like are:
- blocking hook before access to file range and fill that range
- blocking hook before lookup of child and optionally create child
My thoughts on the UAPI were:
- Allow new combination of FAN_CLASS_PRE_CONTENT
and FAN_REPORT_FID/DFID_NAME
- This combination does not allow any of the existing events
in mask
- It Allows only new events such as FAN_PRE_ACCESS
FAN_PRE_MODIFY and FAN_PRE_LOOKUP
- FAN_PRE_ACCESS and FAN_PRE_MODIFY can have
optional file range info
- All the FAN_PRE_ events are called outside vfs locks and
specifically before sb_writers lock as in my fsnotify_pre_modify [1]
POC
That last part is important because the HSM daemon will
need to make modifications to the accessed file/directory
before allowing the operation to proceed.
Naturally that opens the possibility for new userspace
deadlocks. Nothing that is not already possible with permission
event, but maybe deadlocks that are more inviting to trip over.
I am not sure if we need to do anything about this, but we
could make it easier to ignore events from the HSM daemon
itself if we want to, to make the userspace implementation easier.
Another thing that might be good to do is provide an administrative
interface to iterate and abort pending fanotify permission/pre-content
events.
You must have noticed the overlap between my old persistent
change tracking journal and this design. The referenced branch
is from that old POC.
I do believe that the use cases somewhat overlap and that the
same building blocks could be used to implement a persistent
change journal in userspace as you suggested back then.
Thoughts?
Amir.
[1] https://github.com/amir73il/linux/commits/fsnotify_pre_modify
next reply other threads:[~2022-09-11 18:12 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-11 18:12 Amir Goldstein [this message]
2022-09-12 12:57 ` thoughts about fanotify and HSM Jan Kara
2022-09-12 16:38 ` Amir Goldstein
[not found] ` <BY5PR07MB652953061D3A2243F66F0798A3449@BY5PR07MB6529.namprd07.prod.outlook.com>
2022-09-13 2:41 ` Amir Goldstein
2022-09-14 7:27 ` Amir Goldstein
2022-09-14 10:30 ` Jan Kara
2022-09-14 11:52 ` Amir Goldstein
2022-09-20 18:19 ` Amir Goldstein
2022-09-22 10:48 ` Jan Kara
2022-09-22 13:03 ` Amir Goldstein
2022-09-26 15:27 ` Jan Kara
2022-09-28 12:29 ` Amir Goldstein
2022-09-29 10:01 ` Jan Kara
2022-10-07 13:58 ` Amir Goldstein
2022-10-12 15:44 ` Jan Kara
2022-10-12 16:28 ` Amir Goldstein
2022-10-13 12:16 ` Amir Goldstein
2022-11-03 12:57 ` Jan Kara
2022-11-03 13:38 ` Amir Goldstein
2022-10-28 12:50 ` Amir Goldstein
2022-11-03 16:30 ` Jan Kara
2022-11-04 8:17 ` Amir Goldstein
2022-11-07 11:10 ` Jan Kara
2022-11-07 14:13 ` Amir Goldstein
2022-11-14 19:17 ` Jan Kara
2022-11-14 20:08 ` Amir Goldstein
2022-11-15 10:16 ` Jan Kara
2022-11-15 13:08 ` Amir Goldstein
2022-11-16 10:56 ` Jan Kara
2022-11-16 16:24 ` Amir Goldstein
2022-11-17 12:38 ` Amir Goldstein
2022-11-23 10:49 ` Jan Kara
2022-11-23 13:07 ` Amir Goldstein
2022-11-21 16:40 ` Amir Goldstein
2022-11-23 12:11 ` Jan Kara
2022-11-23 13:30 ` Amir Goldstein
2022-11-23 10:10 ` Jan Kara
2022-11-23 15:16 ` Amir Goldstein
[not found] ` <BY5PR07MB6529795F49FB4E923AFCB062A3449@BY5PR07MB6529.namprd07.prod.outlook.com>
2022-09-14 9:29 ` Jan Kara
2022-09-21 23:27 ` Dave Chinner
2022-09-22 4:35 ` Amir Goldstein
2022-09-23 7:57 ` Dave Chinner
2022-09-23 11:22 ` Amir Goldstein
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAOQ4uxhrQ7hySTyHM0Atq=uzbNdHyGV5wfadJarhAu1jDFOUTg@mail.gmail.com' \
--to=amir73il@gmail.com \
--cc=jack@suse.cz \
--cc=linux-fsdevel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).