From: Satya Tangirala <satyat@google.com> To: "Theodore Y. Ts'o" <tytso@mit.edu> Cc: Jaegeuk Kim <jaegeuk@kernel.org>, Eric Biggers <ebiggers@kernel.org>, Chao Yu <chao@kernel.org>, Jens Axboe <axboe@kernel.dk>, "Darrick J . Wong" <darrick.wong@oracle.com>, linux-kernel@vger.kernel.org, linux-fscrypt@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net, linux-xfs@vger.kernel.org, linux-block@vger.kernel.org, linux-ext4@vger.kernel.org Subject: Re: [PATCH v7 0/8] add support for direct I/O with fscrypt using blk-crypto Date: Thu, 3 Dec 2020 23:57:18 +0000 [thread overview] Message-ID: <X8l7XgWNz5sO/LQ6@google.com> (raw) In-Reply-To: <20201117171526.GD445084@mit.edu> On Tue, Nov 17, 2020 at 12:15:26PM -0500, Theodore Y. Ts'o wrote: > What is the expected use case for Direct I/O using fscrypt? This > isn't a problem which is unique to fscrypt, but one of the really > unfortunate aspects of the DIO interface is the silent fallback to > buffered I/O. We've lived with this because DIO goes back decades, > and the original use case was to keep enterprise databases happy, and > the rules around what is necessary for DIO to work was relatively well > understood. > > But with fscrypt, there's going to be some additional requirements > (e.g., using inline crypto) required or else DIO silently fall back to > buffered I/O for encrypted files. Depending on the intended use case > of DIO with fscrypt, this caveat might or might not be unfortunately > surprising for applications. > > I wonder if we should have some kind of interface so we can more > explicitly allow applications to query exactly what the requirements > might be for a particular file vis-a-vis Direct I/O. What are the > memory alignment requirements, what are the file offset alignment > requirements, what are the write size requirements, for a particular > file. > (Credit to Eric for the description of use cases that I'm copying/summarizing here). The primary motivation for this patch series is Android - some devices use zram with cold page writeback enabled to an encrypted swap file, so direct I/O is needed to avoid double-caching the data in the swap file. In general, this patch is useful for avoiding double caching any time a loopback device is created in an encrypted directory. We also expect this to be useful for databases that want to use direct I/O but also want to encrypt data at the FS level. I do think having a good way to tell userspace about the DIO requirements would be great to have. Userspace does have ways to access to most, but not all, of the information it needs to figure out the DIO requirements (I don't think userspace has any way of figuring out if inline encryption hardware is available right now), so it would be nice if there was a good/unified API for getting those requirements. Do you think we'll need that before these patches can go in though? I do think the patches as is are useful for their primary use case even without the ability to explicitly query for the DIO requirements, because Android devices are predictable w.r.t inline encryption support (devices ship with either blk-crypto-fallback or have inline encryption hardware, and the patchset's requirements are met in either case). And even when used on machines without such predictability, this patch is at worst the same as the current situation, and at best an improvement. > - Ted
WARNING: multiple messages have this Message-ID (diff)
From: Satya Tangirala via Linux-f2fs-devel <linux-f2fs-devel@lists.sourceforge.net> To: "Theodore Y. Ts'o" <tytso@mit.edu> Cc: Jens Axboe <axboe@kernel.dk>, linux-xfs@vger.kernel.org, "Darrick J . Wong" <darrick.wong@oracle.com>, linux-kernel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net, Eric Biggers <ebiggers@kernel.org>, linux-fscrypt@vger.kernel.org, linux-block@vger.kernel.org, Jaegeuk Kim <jaegeuk@kernel.org>, linux-ext4@vger.kernel.org Subject: Re: [f2fs-dev] [PATCH v7 0/8] add support for direct I/O with fscrypt using blk-crypto Date: Thu, 3 Dec 2020 23:57:18 +0000 [thread overview] Message-ID: <X8l7XgWNz5sO/LQ6@google.com> (raw) In-Reply-To: <20201117171526.GD445084@mit.edu> On Tue, Nov 17, 2020 at 12:15:26PM -0500, Theodore Y. Ts'o wrote: > What is the expected use case for Direct I/O using fscrypt? This > isn't a problem which is unique to fscrypt, but one of the really > unfortunate aspects of the DIO interface is the silent fallback to > buffered I/O. We've lived with this because DIO goes back decades, > and the original use case was to keep enterprise databases happy, and > the rules around what is necessary for DIO to work was relatively well > understood. > > But with fscrypt, there's going to be some additional requirements > (e.g., using inline crypto) required or else DIO silently fall back to > buffered I/O for encrypted files. Depending on the intended use case > of DIO with fscrypt, this caveat might or might not be unfortunately > surprising for applications. > > I wonder if we should have some kind of interface so we can more > explicitly allow applications to query exactly what the requirements > might be for a particular file vis-a-vis Direct I/O. What are the > memory alignment requirements, what are the file offset alignment > requirements, what are the write size requirements, for a particular > file. > (Credit to Eric for the description of use cases that I'm copying/summarizing here). The primary motivation for this patch series is Android - some devices use zram with cold page writeback enabled to an encrypted swap file, so direct I/O is needed to avoid double-caching the data in the swap file. In general, this patch is useful for avoiding double caching any time a loopback device is created in an encrypted directory. We also expect this to be useful for databases that want to use direct I/O but also want to encrypt data at the FS level. I do think having a good way to tell userspace about the DIO requirements would be great to have. Userspace does have ways to access to most, but not all, of the information it needs to figure out the DIO requirements (I don't think userspace has any way of figuring out if inline encryption hardware is available right now), so it would be nice if there was a good/unified API for getting those requirements. Do you think we'll need that before these patches can go in though? I do think the patches as is are useful for their primary use case even without the ability to explicitly query for the DIO requirements, because Android devices are predictable w.r.t inline encryption support (devices ship with either blk-crypto-fallback or have inline encryption hardware, and the patchset's requirements are met in either case). And even when used on machines without such predictability, this patch is at worst the same as the current situation, and at best an improvement. > - Ted _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
next prev parent reply other threads:[~2020-12-03 23:58 UTC|newest] Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-11-17 14:07 [PATCH v7 0/8] add support for direct I/O with fscrypt using blk-crypto Satya Tangirala 2020-11-17 14:07 ` [f2fs-dev] " Satya Tangirala via Linux-f2fs-devel 2020-11-17 14:07 ` [PATCH v7 1/8] block: ensure bios are not split in middle of crypto data unit Satya Tangirala 2020-11-17 14:07 ` [f2fs-dev] " Satya Tangirala via Linux-f2fs-devel 2020-11-17 23:31 ` Eric Biggers 2020-11-17 23:31 ` [f2fs-dev] " Eric Biggers 2020-11-18 0:38 ` Satya Tangirala 2020-11-18 0:38 ` [f2fs-dev] " Satya Tangirala via Linux-f2fs-devel 2020-11-25 22:12 ` Eric Biggers 2020-11-25 22:12 ` [f2fs-dev] " Eric Biggers 2020-12-02 22:52 ` Satya Tangirala 2020-12-02 22:52 ` [f2fs-dev] " Satya Tangirala via Linux-f2fs-devel 2020-11-25 22:15 ` Eric Biggers 2020-11-25 22:15 ` [f2fs-dev] " Eric Biggers 2020-11-17 14:07 ` [PATCH v7 2/8] blk-crypto: don't require user buffer alignment Satya Tangirala 2020-11-17 14:07 ` [f2fs-dev] " Satya Tangirala via Linux-f2fs-devel 2020-11-17 23:51 ` Eric Biggers 2020-11-17 23:51 ` [f2fs-dev] " Eric Biggers 2020-11-17 14:07 ` [PATCH v7 3/8] fscrypt: add functions for direct I/O support Satya Tangirala 2020-11-17 14:07 ` [f2fs-dev] " Satya Tangirala via Linux-f2fs-devel 2020-11-17 14:07 ` [PATCH v7 4/8] direct-io: add support for fscrypt using blk-crypto Satya Tangirala 2020-11-17 14:07 ` [f2fs-dev] " Satya Tangirala via Linux-f2fs-devel 2020-11-17 14:07 ` [PATCH v7 5/8] iomap: support direct I/O with " Satya Tangirala 2020-11-17 14:07 ` [f2fs-dev] " Satya Tangirala via Linux-f2fs-devel 2020-11-17 14:07 ` [PATCH v7 6/8] ext4: " Satya Tangirala 2020-11-17 14:07 ` [f2fs-dev] " Satya Tangirala via Linux-f2fs-devel 2020-12-03 13:45 ` Theodore Y. Ts'o 2020-12-03 13:45 ` [f2fs-dev] " Theodore Y. Ts'o 2020-11-17 14:07 ` [PATCH v7 7/8] f2fs: " Satya Tangirala 2020-11-17 14:07 ` [f2fs-dev] " Satya Tangirala via Linux-f2fs-devel 2020-11-17 14:07 ` [PATCH v7 8/8] fscrypt: update documentation for direct I/O support Satya Tangirala 2020-11-17 14:07 ` [f2fs-dev] " Satya Tangirala via Linux-f2fs-devel 2020-11-18 2:38 ` Eric Biggers 2020-11-18 2:38 ` [f2fs-dev] " Eric Biggers 2020-11-17 17:15 ` [PATCH v7 0/8] add support for direct I/O with fscrypt using blk-crypto Theodore Y. Ts'o 2020-11-17 17:15 ` [f2fs-dev] " Theodore Y. Ts'o 2020-11-17 17:20 ` Darrick J. Wong 2020-11-17 17:20 ` [f2fs-dev] " Darrick J. Wong 2020-12-03 23:57 ` Satya Tangirala [this message] 2020-12-03 23:57 ` Satya Tangirala via Linux-f2fs-devel 2020-11-18 2:51 ` Eric Biggers 2020-11-18 2:51 ` [f2fs-dev] " Eric Biggers
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=X8l7XgWNz5sO/LQ6@google.com \ --to=satyat@google.com \ --cc=axboe@kernel.dk \ --cc=chao@kernel.org \ --cc=darrick.wong@oracle.com \ --cc=ebiggers@kernel.org \ --cc=jaegeuk@kernel.org \ --cc=linux-block@vger.kernel.org \ --cc=linux-ext4@vger.kernel.org \ --cc=linux-f2fs-devel@lists.sourceforge.net \ --cc=linux-fscrypt@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-xfs@vger.kernel.org \ --cc=tytso@mit.edu \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.