* [PATCH] fallocate.2: add FALLOC_FL_PUNCH_HOLE flag definition @ 2010-11-15 17:07 Josef Bacik 2011-09-19 5:08 ` Michael Kerrisk 0 siblings, 1 reply; 6+ messages in thread From: Josef Bacik @ 2010-11-15 17:07 UTC (permalink / raw) To: mtk.manpages, linux-man, linux-fsdevel, linux-kernel This patch adds the relevant documenation for using fallocate with the FALLOC_FL_PUNCH_HOLE mode. This information comes from the implementation of FALLOC_FL_PUNCH_HOLE that I've posted along with this man page update to outline the definition for hole punching. Obviously this patch is dependant upon the current implementation being accepted. Thanks, Signed-off-by: Josef Bacik <josef@redhat.com> --- man2/fallocate.2 | 15 ++++++++++++--- 1 files changed, 12 insertions(+), 3 deletions(-) diff --git a/man2/fallocate.2 b/man2/fallocate.2 index 42eefbd..1b28916 100644 --- a/man2/fallocate.2 +++ b/man2/fallocate.2 @@ -31,9 +31,8 @@ bytes. The .I mode -argument determines the operation to be performed on the given range. -Currently only one flag is supported for -.IR mode : +argument determines the operation to be performed on the given range, the +value can be one of the following: .TP .B FALLOC_FL_KEEP_SIZE This flag allocates and initializes to zero the disk space @@ -50,6 +49,16 @@ the file size (as reported by .BR stat (2)) even if it is less than .IR offset + len . +.TP +.B FALLOC_FL_PUNCH_HOLE +This flag de-allocates the disk space within the range specified by +.I offset +and +.IR len . +This call preserves the file size (as reported by +.BR stat (2)), +the only change will be to the amount of space used by the file. After a +successful call, subsequent reads from this range will return 0's. .\" .\" Note from Amit Arora: .\" There were few more flags which were discussed, but none of -- 1.6.6.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] fallocate.2: add FALLOC_FL_PUNCH_HOLE flag definition 2010-11-15 17:07 [PATCH] fallocate.2: add FALLOC_FL_PUNCH_HOLE flag definition Josef Bacik @ 2011-09-19 5:08 ` Michael Kerrisk 2011-09-19 6:32 ` Dave Chinner 2011-09-21 4:11 ` Michael Kerrisk 0 siblings, 2 replies; 6+ messages in thread From: Michael Kerrisk @ 2011-09-19 5:08 UTC (permalink / raw) To: Josef Bacik Cc: linux-man, linux-fsdevel, linux-kernel, Lucian Adrian Grijincu, Al Viro, Dave Chinner Hi Josef (Dave C, please also note comments below), On Mon, Nov 15, 2010 at 6:07 PM, Josef Bacik <josef@redhat.com> wrote: > This patch adds the relevant documenation for using fallocate with the > FALLOC_FL_PUNCH_HOLE mode. This information comes from the implementation of > FALLOC_FL_PUNCH_HOLE that I've posted along with this man page update to outline > the definition for hole punching. Obviously this patch is dependant upon the > current implementation being accepted. Thanks, > > Signed-off-by: Josef Bacik <josef@redhat.com> > --- > man2/fallocate.2 | 15 ++++++++++++--- > 1 files changed, 12 insertions(+), 3 deletions(-) > > diff --git a/man2/fallocate.2 b/man2/fallocate.2 > index 42eefbd..1b28916 100644 > --- a/man2/fallocate.2 > +++ b/man2/fallocate.2 > @@ -31,9 +31,8 @@ bytes. > > The > .I mode > -argument determines the operation to be performed on the given range. > -Currently only one flag is supported for > -.IR mode : > +argument determines the operation to be performed on the given range, the > +value can be one of the following: > .TP > .B FALLOC_FL_KEEP_SIZE > This flag allocates and initializes to zero the disk space > @@ -50,6 +49,16 @@ the file size (as reported by > .BR stat (2)) > even if it is less than > .IR offset + len . > +.TP > +.B FALLOC_FL_PUNCH_HOLE > +This flag de-allocates the disk space within the range specified by > +.I offset > +and > +.IR len . > +This call preserves the file size (as reported by > +.BR stat (2)), > +the only change will be to the amount of space used by the file. After a > +successful call, subsequent reads from this range will return 0's. > .\" > .\" Note from Amit Arora: > .\" There were few more flags which were discussed, but none of > -- > 1.6.6.1 Thanks for this patch. As noted in another mail, Lucian also sent a patch for FALLOC_FL_PUNCH_HOLE, and I applied his patch first, and then added some pieces from yours, as well as some of my own edits. However, the addition of a second class of operation to the man page made it clear that some significant restructuring of the page is required. So I substantially reworked the page, including the preexisting material on the default "file allocation" operation (Dave C please note). Josef, Lucian, could I ask you to check ("man -l <file>") the final text (below) for FALLOC_FL_PUNCH_HOLE (including the EPERM error under ERRORS)? Dave, could I ask you to review my reworked text for the "Allocating disk space" subsection? Thanks, Michael .\" Copyright (c) 2007 Silicon Graphics, Inc. All Rights Reserved .\" Written by Dave Chinner <dgc@sgi.com> .\" May be distributed as per GNU General Public License version 2. .\" .\" 2011-09-19: Added FALLOC_FL_PUNCH_HOLE .\" 2011-09-19: Substantial restructuring of the page .\" .TH FALLOCATE 2 2011-09-19 "Linux" "Linux Programmer's Manual" .SH NAME fallocate \- manipulate file space .SH SYNOPSIS .nf .BR "#define _GNU_SOURCE" " /* See feature_test_macros(7) */" .B #include <fcntl.h> .BI "int fallocate(int " fd ", int " mode ", off_t " offset \ ", off_t " len "); .fi .SH DESCRIPTION This is a nonportable, Linux-specific system call. For the portable, POSIX.1-specified method of ensuring that space is allocated for a file, see .BR posix_fallocate (3). .BR fallocate () allows the caller to directly manipulate the allocated disk space for the file referred to by .I fd for the byte range starting at .I offset and continuing for .I len bytes. The .I mode argument determines the operation to be performed on the given range. Details of the supported operations are given in the subsections below. .SS Allocating disk space The default operation (i.e., .I mode is zero) of .BR fallocate () allocates and initializes to zero the disk space within the range specified by .I offset and .IR len . The file size (as reported by .BR stat (2)) will be changed if .I "offset + len" is greater than the file size. This default behavior closely resembles the behavior of the .BR posix_fallocate (3) library function, and is intended as a method of optimally implementing that function. After a successful call, subsequent writes into the range specified by .IR offset and .IR len are guaranteed not to fail because of lack of disk space. If the .B FALLOC_FL_KEEP_SIZE flag is specified in .IR mode , the behavior of the call is similar, but the file size will not be changed even if .I "offset + len" is greater than the file size. Preallocating zeroed blocks beyond the end of the file in this manner is useful for optimizing append workloads. .PP Because allocation is done in block size chunks, .BR fallocate () may allocate a larger range of disk space than was specified. .SS Deallocating file space Specifying the .BR FALLOC_FL_PUNCH_HOLE flag (available since Linux 2.6.38) in .I mode deallocates space (i.e., creates a hole) in the byte range starting at .I offset and continuing for .I len bytes. Within the specified range, partial file system blocks are zeroed, and whole file system blocks are removed from the file. After a successful call, subsequent reads from this range will return zeroes. The .BR FALLOC_FL_PUNCH_HOLE flag must be ORed with .BR FALLOC_FL_KEEP_SIZE in .IR mode ; in other words, even when punching off the end of the file, the file size (as reported by .BR stat (2)) does not change. Not all file systems support .BR FALLOC_FL_PUNCH_HOLE ; if a file system doesn't support the operation, an error is returned. .SH RETURN VALUE .BR fallocate () returns zero on success, and \-1 on failure. .SH ERRORS .TP .B EBADF .I fd is not a valid file descriptor, or is not opened for writing. .TP .B EFBIG .IR offset + len exceeds the maximum file size. .TP .B EINTR A signal was caught during execution. .TP .B EINVAL .I offset was less than 0, or .I len was less than or equal to 0. .TP .B EIO An I/O error occurred while reading from or writing to a file system. .TP .B ENODEV .I fd does not refer to a regular file or a directory. (If .I fd is a pipe or FIFO, a different error results.) .TP .B ENOSPC There is not enough space left on the device containing the file referred to by .IR fd . .TP .B ENOSYS The file system containing the file referred to by .I fd does not support this operation. .TP .B EOPNOTSUPP The .I mode is not supported by the file system containing the file referred to by .IR fd . .TP .B EPERM The file referred to by .I fd is marked immutable (see .BR chattr (1)). Or: .I mode specifies .BR FALLOC_FL_PUNCH_HOLE and the file referred to by .I fd is marked append-only (see .BR chattr (1)). .TP .B ESPIPE .I fd refers to a pipe of FIFO. .SH VERSIONS .BR fallocate () is available on Linux since kernel 2.6.23. Support is provided by glibc since version 2.10. .SH CONFORMING TO .BR fallocate () is Linux-specific. .SH SEE ALSO .BR ftruncate (2), .BR posix_fadvise (3), .BR posix_fallocate (3) -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Author of "The Linux Programming Interface"; http://man7.org/tlpi/ ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] fallocate.2: add FALLOC_FL_PUNCH_HOLE flag definition 2011-09-19 5:08 ` Michael Kerrisk @ 2011-09-19 6:32 ` Dave Chinner 2011-09-21 4:11 ` Michael Kerrisk 1 sibling, 0 replies; 6+ messages in thread From: Dave Chinner @ 2011-09-19 6:32 UTC (permalink / raw) To: Michael Kerrisk Cc: Josef Bacik, linux-man, linux-fsdevel, linux-kernel, Lucian Adrian Grijincu, Al Viro, Dave Chinner On Mon, Sep 19, 2011 at 07:08:17AM +0200, Michael Kerrisk wrote: > Hi Josef (Dave C, please also note comments below), > > On Mon, Nov 15, 2010 at 6:07 PM, Josef Bacik <josef@redhat.com> wrote: > > This patch adds the relevant documenation for using fallocate with the > > FALLOC_FL_PUNCH_HOLE mode. This information comes from the implementation of > > FALLOC_FL_PUNCH_HOLE that I've posted along with this man page update to outline > > the definition for hole punching. Obviously this patch is dependant upon the > > current implementation being accepted. Thanks, > > > > Signed-off-by: Josef Bacik <josef@redhat.com> > > --- > > man2/fallocate.2 | 15 ++++++++++++--- > > 1 files changed, 12 insertions(+), 3 deletions(-) > > > > diff --git a/man2/fallocate.2 b/man2/fallocate.2 > > index 42eefbd..1b28916 100644 > > --- a/man2/fallocate.2 > > +++ b/man2/fallocate.2 > > @@ -31,9 +31,8 @@ bytes. > > > > The > > .I mode > > -argument determines the operation to be performed on the given range. > > -Currently only one flag is supported for > > -.IR mode : > > +argument determines the operation to be performed on the given range, the > > +value can be one of the following: > > .TP > > .B FALLOC_FL_KEEP_SIZE > > This flag allocates and initializes to zero the disk space > > @@ -50,6 +49,16 @@ the file size (as reported by > > .BR stat (2)) > > even if it is less than > > .IR offset + len . > > +.TP > > +.B FALLOC_FL_PUNCH_HOLE > > +This flag de-allocates the disk space within the range specified by > > +.I offset > > +and > > +.IR len . > > +This call preserves the file size (as reported by > > +.BR stat (2)), > > +the only change will be to the amount of space used by the file. After a > > +successful call, subsequent reads from this range will return 0's. > > .\" > > .\" Note from Amit Arora: > > .\" There were few more flags which were discussed, but none of > > -- > > 1.6.6.1 > > Thanks for this patch. As noted in another mail, Lucian also sent a > patch for FALLOC_FL_PUNCH_HOLE, and I applied his patch first, and > then added some pieces from yours, as well as some of my own edits. > > However, the addition of a second class of operation to the man page > made it clear that some significant restructuring of the page is > required. So I substantially reworked the page, including the > preexisting material on the default "file allocation" operation (Dave > C please note). > > Josef, Lucian, could I ask you to check ("man -l <file>") the final > text (below) for FALLOC_FL_PUNCH_HOLE (including the EPERM error under > ERRORS)? > > Dave, could I ask you to review my reworked text for the "Allocating > disk space" subsection? Looks OK to me. Cheers, Dave. -- Dave Chinner david@fromorbit.com ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] fallocate.2: add FALLOC_FL_PUNCH_HOLE flag definition 2011-09-19 5:08 ` Michael Kerrisk 2011-09-19 6:32 ` Dave Chinner @ 2011-09-21 4:11 ` Michael Kerrisk 2011-09-21 5:25 ` Andreas Dilger 1 sibling, 1 reply; 6+ messages in thread From: Michael Kerrisk @ 2011-09-21 4:11 UTC (permalink / raw) To: Josef Bacik Cc: Eric Gouriou, inux-man, linux-fsdevel, linux-kernel, Lucian Adrian Grijincu, Al Viro, Dave Chinner [Resending with Eric Gouriou on CC] Hi Josef (Dave C, please also note comments below), On Mon, Nov 15, 2010 at 6:07 PM, Josef Bacik <josef@redhat.com> wrote: > This patch adds the relevant documenation for using fallocate with the > FALLOC_FL_PUNCH_HOLE mode. This information comes from the implementation of > FALLOC_FL_PUNCH_HOLE that I've posted along with this man page update to outline > the definition for hole punching. Obviously this patch is dependant upon the > current implementation being accepted. Thanks, > > Signed-off-by: Josef Bacik <josef@redhat.com> > --- > man2/fallocate.2 | 15 ++++++++++++--- > 1 files changed, 12 insertions(+), 3 deletions(-) > > diff --git a/man2/fallocate.2 b/man2/fallocate.2 > index 42eefbd..1b28916 100644 > --- a/man2/fallocate.2 > +++ b/man2/fallocate.2 > @@ -31,9 +31,8 @@ bytes. > > The > .I mode > -argument determines the operation to be performed on the given range. > -Currently only one flag is supported for > -.IR mode : > +argument determines the operation to be performed on the given range, the > +value can be one of the following: > .TP > .B FALLOC_FL_KEEP_SIZE > This flag allocates and initializes to zero the disk space > @@ -50,6 +49,16 @@ the file size (as reported by > .BR stat (2)) > even if it is less than > .IR offset + len . > +.TP > +.B FALLOC_FL_PUNCH_HOLE > +This flag de-allocates the disk space within the range specified by > +.I offset > +and > +.IR len . > +This call preserves the file size (as reported by > +.BR stat (2)), > +the only change will be to the amount of space used by the file. After a > +successful call, subsequent reads from this range will return 0's. > .\" > .\" Note from Amit Arora: > .\" There were few more flags which were discussed, but none of > -- > 1.6.6.1 Thanks for this patch. As noted in another mail, Lucian also sent a patch for FALLOC_FL_PUNCH_HOLE, and I applied his patch first, and then added some pieces from yours, as well as some of my own edits. However, the addition of a second class of operation to the man page made it clear that some significant restructuring of the page is required. So I substantially reworked the page, including the preexisting material on the default "file allocation" operation (Dave C please note). Josef, Lucian, could I ask you to check ("man -l <file>") the final text (below) for FALLOC_FL_PUNCH_HOLE (including the EPERM error under ERRORS)? Dave, could I ask you to review my reworked text for the "Allocating disk space" subsection? Thanks, Michael .\" Copyright (c) 2007 Silicon Graphics, Inc. All Rights Reserved .\" Written by Dave Chinner <dgc@sgi.com> .\" May be distributed as per GNU General Public License version 2. .\" .\" 2011-09-19: Added FALLOC_FL_PUNCH_HOLE .\" 2011-09-19: Substantial restructuring of the page .\" .TH FALLOCATE 2 2011-09-19 "Linux" "Linux Programmer's Manual" .SH NAME fallocate \- manipulate file space .SH SYNOPSIS .nf .BR "#define _GNU_SOURCE" " /* See feature_test_macros(7) */" .B #include <fcntl.h> .BI "int fallocate(int " fd ", int " mode ", off_t " offset \ ", off_t " len "); .fi .SH DESCRIPTION This is a nonportable, Linux-specific system call. For the portable, POSIX.1-specified method of ensuring that space is allocated for a file, see .BR posix_fallocate (3). .BR fallocate () allows the caller to directly manipulate the allocated disk space for the file referred to by .I fd for the byte range starting at .I offset and continuing for .I len bytes. The .I mode argument determines the operation to be performed on the given range. Details of the supported operations are given in the subsections below. .SS Allocating disk space The default operation (i.e., .I mode is zero) of .BR fallocate () allocates and initializes to zero the disk space within the range specified by .I offset and .IR len . The file size (as reported by .BR stat (2)) will be changed if .I "offset + len" is greater than the file size. This default behavior closely resembles the behavior of the .BR posix_fallocate (3) library function, and is intended as a method of optimally implementing that function. After a successful call, subsequent writes into the range specified by .IR offset and .IR len are guaranteed not to fail because of lack of disk space. If the .B FALLOC_FL_KEEP_SIZE flag is specified in .IR mode , the behavior of the call is similar, but the file size will not be changed even if .I "offset + len" is greater than the file size. Preallocating zeroed blocks beyond the end of the file in this manner is useful for optimizing append workloads. .PP Because allocation is done in block size chunks, .BR fallocate () may allocate a larger range of disk space than was specified. .SS Deallocating file space Specifying the .BR FALLOC_FL_PUNCH_HOLE flag (available since Linux 2.6.38) in .I mode deallocates space (i.e., creates a hole) in the byte range starting at .I offset and continuing for .I len bytes. Within the specified range, partial file system blocks are zeroed, and whole file system blocks are removed from the file. After a successful call, subsequent reads from this range will return zeroes. The .BR FALLOC_FL_PUNCH_HOLE flag must be ORed with .BR FALLOC_FL_KEEP_SIZE in .IR mode ; in other words, even when punching off the end of the file, the file size (as reported by .BR stat (2)) does not change. Not all file systems support .BR FALLOC_FL_PUNCH_HOLE ; if a file system doesn't support the operation, an error is returned. .SH RETURN VALUE .BR fallocate () returns zero on success, and \-1 on failure. .SH ERRORS .TP .B EBADF .I fd is not a valid file descriptor, or is not opened for writing. .TP .B EFBIG .IR offset + len exceeds the maximum file size. .TP .B EINTR A signal was caught during execution. .TP .B EINVAL .I offset was less than 0, or .I len was less than or equal to 0. .TP .B EIO An I/O error occurred while reading from or writing to a file system. .TP .B ENODEV .I fd does not refer to a regular file or a directory. (If .I fd is a pipe or FIFO, a different error results.) .TP .B ENOSPC There is not enough space left on the device containing the file referred to by .IR fd . .TP .B ENOSYS The file system containing the file referred to by .I fd does not support this operation. .TP .B EOPNOTSUPP The .I mode is not supported by the file system containing the file referred to by .IR fd . .TP .B EPERM The file referred to by .I fd is marked immutable (see .BR chattr (1)). Or: .I mode specifies .BR FALLOC_FL_PUNCH_HOLE and the file referred to by .I fd is marked append-only (see .BR chattr (1)). .TP .B ESPIPE .I fd refers to a pipe of FIFO. .SH VERSIONS .BR fallocate () is available on Linux since kernel 2.6.23. Support is provided by glibc since version 2.10. .SH CONFORMING TO .BR fallocate () is Linux-specific. .SH SEE ALSO .BR ftruncate (2), .BR posix_fadvise (3), .BR posix_fallocate (3) -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Author of "The Linux Programming Interface"; http://man7.org/tlpi/ -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Author of "The Linux Programming Interface"; http://man7.org/tlpi/ ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] fallocate.2: add FALLOC_FL_PUNCH_HOLE flag definition 2011-09-21 4:11 ` Michael Kerrisk @ 2011-09-21 5:25 ` Andreas Dilger 2011-09-21 5:49 ` Michael Kerrisk 0 siblings, 1 reply; 6+ messages in thread From: Andreas Dilger @ 2011-09-21 5:25 UTC (permalink / raw) To: mtk.manpages Cc: Josef Bacik, Eric Gouriou, inux-man, linux-fsdevel, linux-kernel, Lucian Adrian Grijincu, Al Viro, Dave Chinner On 2011-09-20, at 10:11 PM, Michael Kerrisk wrote: > Thanks for this patch. As noted in another mail, Lucian also sent a > patch for FALLOC_FL_PUNCH_HOLE, and I applied his patch first, and > then added some pieces from yours, as well as some of my own edits. > > However, the addition of a second class of operation to the man page > made it clear that some significant restructuring of the page is > required. So I substantially reworked the page, including the > preexisting material on the default "file allocation" operation (Dave > C please note). > > \b.TP > .B EINVAL > .I offset > was less than 0, or > .I len > was less than or equal to 0. I wasn't sure if this is a bug in the manpage or actually how it is done in the kernel, but it seems this is a kernel implementation issue... Does it make sense to return an error if len == 0? That just adds extra complexity on the part of the application, and doesn't reduce complexity in the kernel (whether the kernel checks for len == 0 and returns 0 or -EINVAL is not any different). read() and write() and malloc() all allow a length of zero to succeed, since applications may compute this length and sometimes it is zero. Cheers, Andreas ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] fallocate.2: add FALLOC_FL_PUNCH_HOLE flag definition 2011-09-21 5:25 ` Andreas Dilger @ 2011-09-21 5:49 ` Michael Kerrisk 0 siblings, 0 replies; 6+ messages in thread From: Michael Kerrisk @ 2011-09-21 5:49 UTC (permalink / raw) To: Andreas Dilger Cc: Josef Bacik, Eric Gouriou, inux-man, linux-fsdevel, linux-kernel, Lucian Adrian Grijincu, Al Viro, Dave Chinner Hello Andreas, On Wed, Sep 21, 2011 at 7:25 AM, Andreas Dilger <adilger@dilger.ca> wrote: > On 2011-09-20, at 10:11 PM, Michael Kerrisk wrote: >> Thanks for this patch. As noted in another mail, Lucian also sent a >> patch for FALLOC_FL_PUNCH_HOLE, and I applied his patch first, and >> then added some pieces from yours, as well as some of my own edits. >> >> However, the addition of a second class of operation to the man page >> made it clear that some significant restructuring of the page is >> required. So I substantially reworked the page, including the >> preexisting material on the default "file allocation" operation (Dave >> C please note). >> >> .TP >> .B EINVAL >> .I offset >> was less than 0, or >> .I len >> was less than or equal to 0. > > I wasn't sure if this is a bug in the manpage or actually how it is done > in the kernel, but it seems this is a kernel implementation issue... > > Does it make sense to return an error if len == 0? That just adds extra > complexity on the part of the application, and doesn't reduce complexity > in the kernel (whether the kernel checks for len == 0 and returns 0 or > -EINVAL is not any different). read() and write() and malloc() all allow > a length of zero to succeed, since applications may compute this length > and sometimes it is zero. Good point. I agree: for comfort of userspace application writers and for consistency with related interfaces, len == 0 shouldn't be an error. Cheers, Michael -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Author of "The Linux Programming Interface"; http://man7.org/tlpi/ ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2011-09-21 5:49 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2010-11-15 17:07 [PATCH] fallocate.2: add FALLOC_FL_PUNCH_HOLE flag definition Josef Bacik 2011-09-19 5:08 ` Michael Kerrisk 2011-09-19 6:32 ` Dave Chinner 2011-09-21 4:11 ` Michael Kerrisk 2011-09-21 5:25 ` Andreas Dilger 2011-09-21 5:49 ` Michael Kerrisk
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).