Re: man page update (fcntl(2) new set/get write hints)

Re: man page update (fcntl(2) new set/get write hints)

[Michael Kerrisk (man-pages)]

Hello Jens,

On 07/31/2017 05:24 PM, Jens Axboe wrote:
> Hi Michael,
> 
> Not sure what the best way to submit this is, but I wrote an
> update for the fcntl man page to cover the new set/get write
> hints commands that are added in 4.13. Attaching the patch, but
> you can also find it below, in a fork of your github repo:
> 
> https://github.com/axboe/man-pages/commits/fcntl-write-hints

Thanks for the patch. Some questions below.

> From 3eee36f14a650e50f14b42ae1d4fec33e78e6254 Mon Sep 17 00:00:00 2001
> From: Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
> Date: Mon, 26 Jun 2017 16:58:29 -0600
> Subject: [PATCH] man2/fcntl.2: add description of new read/write hint commands
> 
> This adds a description of the new read/write hint commands:
> 
> F_GET_RW_HINT		Get inode read/write hint
> F_SET_RW_HINT		Set inode read/write hint
> F_GET_FILE_RW_HINT	Get file read/write hint
> F_SET_FILE_RW_HINT	Set file read/write hint
> 
> and the currently available values, as of Linux 4.13.
> 
> Signed-off-by: Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
> ---
>  man2/fcntl.2 | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 58 insertions(+)
> 
> diff --git a/man2/fcntl.2 b/man2/fcntl.2
> index 70758d2fcd99..07eacc9564fa 100644
> --- a/man2/fcntl.2
> +++ b/man2/fcntl.2
> @@ -5,6 +5,7 @@
>  .\" and Copyright (C) 2002-2010, 2014 Michael Kerrisk;
>  .\" and Copyright (C) 2014 Jeff Layton
>  .\" and Copyright (C) 2014 David Herrmann
> +.\" and Copyright (C) 2017 Jens Axboe
>  .\"
>  .\" %%%LICENSE_START(VERBATIM)
>  .\" Permission is granted to make and distribute verbatim copies of this
> @@ -61,6 +62,8 @@
>  .\"	Document F_SETPIPE_SZ and F_GETPIPE_SZ.
>  .\" 2014-07-08, David Herrmann <dh.herrmann-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>  .\"     Document F_ADD_SEALS and F_GET_SEALS
> +.\" 2017-06-26, Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
> +.\"     Document F_{GET,SET}_RW_HINT and F_{GET,SET}_FILE_RW_HINT
>  .\"
>  .TH FCNTL 2 2017-05-03 "Linux" "Linux Programmer's Manual"
>  .SH NAME
> @@ -1490,6 +1493,61 @@ Furthermore, if there are any asynchronous I/O operations
>  .RB ( io_submit (2))
>  pending on the file,
>  all outstanding writes will be discarded.
> +.SS File read/write hints
> +Write life time hints can be used to inform the kernel about the relative
> +expected life time of writes on a given inode or file. An application may
> +use this interface for separating writes into different write classes,
> +so that multiple users or applications running on a single storage back-end
> +can aggregate their IO patterns in a consistent manner. However, there are
> +no functional semantics implied by these flags, and different IO classes
> +can use the write life time hints in arbitrary ways so long as they are used
> +consistently.

I read the above, and it is still not clear to me what a "write lifetime" 
is. Can you add some text to explain? (I'm guessing that it means the 
amount of time before the data is overwritten by another write, but I'm
not sure of that, and it needs to be more explicit in the text.)

> +.TP
> +.BR F_GET_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
> +Returns the value of the read/write hint associated with the underlying inode
> +for the file descriptor.
> +.TP
> +.BR F_SET_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
> +Sets the read/write hint value associated with the underlying inode for the
> +file descriptor.
> +.TP
> +.BR F_GET_FILE_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
> +Returns the value of the read/write hint associated with the file descriptor.

Do you mean here "file descriptor" or "file description (i.e., the
open file handle)? Maybe you mean the former, but I want to confirm.

> +.TP
> +.BR F_SET_FILE_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
> +Sets the read/write hint value associated with the file descriptor.

Same as above comment.

> +.P
> +If a given file descriptor has not been assigned a read/write hint, then it
> +shall use the value assigned to the inode, if any. The following read/write
> +hints are valid since kernel 4.13:
> +.TP
> +.BR RWH_WRITE_LIFE_NOT_SET
> +No specific hint has been set. This is the default value.
> +.TP
> +.BR RWH_WRITE_LIFE_NONE
> +No specific write life time is associated with this file or inode.
> +.TP
> +.BR RWH_WRITE_LIFE_SHORT
> +Data written to this file or inode is expected to have a short life time.
> +.TP
> +.BR RWH_WRITE_LIFE_MEDIUM
> +Data written to this file or inode is expected to have a life time longer than
> +data written with
> +.BR RWH_WRITE_LIFE_SHORT .
> +.TP
> +.BR RWH_WRITE_LIFE_LONG
> +Data written to this file or inode is expected to have a life time longer than
> +data written with
> +.BR RWH_WRITE_LIFE_MEDIUM .
> +.TP
> +.BR RWH_WRITE_LIFE_EXTREME
> +Data written to this file or inode is expected to have a life time longer than
> +data written with
> +.BR RWH_WRITE_LIFE_LONG .
> +.P
> +All the write specific hints are relative to each other, and no individual
> +absolute meaning should be attributed to them.
> +.P
>  .SH RETURN VALUE
>  For a successful call, the return value depends on the operation:
>  .TP 0.9i

Thanks,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: man page update (fcntl(2) new set/get write hints)

[Jens Axboe]

On 08/23/2017 04:41 PM, Michael Kerrisk (man-pages) wrote:
> Hello Jens,
> 
> On 07/31/2017 05:24 PM, Jens Axboe wrote:
>> Hi Michael,
>>
>> Not sure what the best way to submit this is, but I wrote an
>> update for the fcntl man page to cover the new set/get write
>> hints commands that are added in 4.13. Attaching the patch, but
>> you can also find it below, in a fork of your github repo:
>>
>> https://github.com/axboe/man-pages/commits/fcntl-write-hints
> 
> Thanks for the patch. Some questions below.
> 
>> From 3eee36f14a650e50f14b42ae1d4fec33e78e6254 Mon Sep 17 00:00:00 2001
>> From: Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
>> Date: Mon, 26 Jun 2017 16:58:29 -0600
>> Subject: [PATCH] man2/fcntl.2: add description of new read/write hint commands
>>
>> This adds a description of the new read/write hint commands:
>>
>> F_GET_RW_HINT		Get inode read/write hint
>> F_SET_RW_HINT		Set inode read/write hint
>> F_GET_FILE_RW_HINT	Get file read/write hint
>> F_SET_FILE_RW_HINT	Set file read/write hint
>>
>> and the currently available values, as of Linux 4.13.
>>
>> Signed-off-by: Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
>> ---
>>  man2/fcntl.2 | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 58 insertions(+)
>>
>> diff --git a/man2/fcntl.2 b/man2/fcntl.2
>> index 70758d2fcd99..07eacc9564fa 100644
>> --- a/man2/fcntl.2
>> +++ b/man2/fcntl.2
>> @@ -5,6 +5,7 @@
>>  .\" and Copyright (C) 2002-2010, 2014 Michael Kerrisk;
>>  .\" and Copyright (C) 2014 Jeff Layton
>>  .\" and Copyright (C) 2014 David Herrmann
>> +.\" and Copyright (C) 2017 Jens Axboe
>>  .\"
>>  .\" %%%LICENSE_START(VERBATIM)
>>  .\" Permission is granted to make and distribute verbatim copies of this
>> @@ -61,6 +62,8 @@
>>  .\"	Document F_SETPIPE_SZ and F_GETPIPE_SZ.
>>  .\" 2014-07-08, David Herrmann <dh.herrmann-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>>  .\"     Document F_ADD_SEALS and F_GET_SEALS
>> +.\" 2017-06-26, Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
>> +.\"     Document F_{GET,SET}_RW_HINT and F_{GET,SET}_FILE_RW_HINT
>>  .\"
>>  .TH FCNTL 2 2017-05-03 "Linux" "Linux Programmer's Manual"
>>  .SH NAME
>> @@ -1490,6 +1493,61 @@ Furthermore, if there are any asynchronous I/O operations
>>  .RB ( io_submit (2))
>>  pending on the file,
>>  all outstanding writes will be discarded.
>> +.SS File read/write hints
>> +Write life time hints can be used to inform the kernel about the relative
>> +expected life time of writes on a given inode or file. An application may
>> +use this interface for separating writes into different write classes,
>> +so that multiple users or applications running on a single storage back-end
>> +can aggregate their IO patterns in a consistent manner. However, there are
>> +no functional semantics implied by these flags, and different IO classes
>> +can use the write life time hints in arbitrary ways so long as they are used
>> +consistently.
> 
> I read the above, and it is still not clear to me what a "write lifetime" 
> is. Can you add some text to explain? (I'm guessing that it means the 
> amount of time before the data is overwritten by another write, but I'm
> not sure of that, and it needs to be more explicit in the text.)

You are right, it's exactly what it refers to. It's essentially data
lifetime hints for a write command. I'd be happy to take suggestions to
improve the description of it.

>> +.TP
>> +.BR F_GET_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>> +Returns the value of the read/write hint associated with the underlying inode
>> +for the file descriptor.
>> +.TP
>> +.BR F_SET_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>> +Sets the read/write hint value associated with the underlying inode for the
>> +file descriptor.
>> +.TP
>> +.BR F_GET_FILE_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>> +Returns the value of the read/write hint associated with the file descriptor.
> 
> Do you mean here "file descriptor" or "file description (i.e., the
> open file handle)? Maybe you mean the former, but I want to confirm.

I do mean file descriptor.

>> +.TP
>> +.BR F_SET_FILE_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>> +Sets the read/write hint value associated with the file descriptor.
> 
> Same as above comment.

Ditto.

-- 
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: man page update (fcntl(2) new set/get write hints)

[Michael Kerrisk (man-pages)]

Hello Jens,

On 24 August 2017 at 16:39, Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org> wrote:
> On 08/23/2017 04:41 PM, Michael Kerrisk (man-pages) wrote:
>> Hello Jens,
>>
>> On 07/31/2017 05:24 PM, Jens Axboe wrote:
>>> Hi Michael,
>>>
>>> Not sure what the best way to submit this is, but I wrote an
>>> update for the fcntl man page to cover the new set/get write
>>> hints commands that are added in 4.13. Attaching the patch, but
>>> you can also find it below, in a fork of your github repo:
>>>
>>> https://github.com/axboe/man-pages/commits/fcntl-write-hints
>>
>> Thanks for the patch. Some questions below.
>>
>>> From 3eee36f14a650e50f14b42ae1d4fec33e78e6254 Mon Sep 17 00:00:00 2001
>>> From: Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
>>> Date: Mon, 26 Jun 2017 16:58:29 -0600
>>> Subject: [PATCH] man2/fcntl.2: add description of new read/write hint commands
>>>
>>> This adds a description of the new read/write hint commands:
>>>
>>> F_GET_RW_HINT                Get inode read/write hint
>>> F_SET_RW_HINT                Set inode read/write hint
>>> F_GET_FILE_RW_HINT   Get file read/write hint
>>> F_SET_FILE_RW_HINT   Set file read/write hint
>>>
>>> and the currently available values, as of Linux 4.13.
>>>
>>> Signed-off-by: Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
>>> ---
>>>  man2/fcntl.2 | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>  1 file changed, 58 insertions(+)
>>>
>>> diff --git a/man2/fcntl.2 b/man2/fcntl.2
>>> index 70758d2fcd99..07eacc9564fa 100644
>>> --- a/man2/fcntl.2
>>> +++ b/man2/fcntl.2
>>> @@ -5,6 +5,7 @@
>>>  .\" and Copyright (C) 2002-2010, 2014 Michael Kerrisk;
>>>  .\" and Copyright (C) 2014 Jeff Layton
>>>  .\" and Copyright (C) 2014 David Herrmann
>>> +.\" and Copyright (C) 2017 Jens Axboe
>>>  .\"
>>>  .\" %%%LICENSE_START(VERBATIM)
>>>  .\" Permission is granted to make and distribute verbatim copies of this
>>> @@ -61,6 +62,8 @@
>>>  .\" Document F_SETPIPE_SZ and F_GETPIPE_SZ.
>>>  .\" 2014-07-08, David Herrmann <dh.herrmann-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>>>  .\"     Document F_ADD_SEALS and F_GET_SEALS
>>> +.\" 2017-06-26, Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
>>> +.\"     Document F_{GET,SET}_RW_HINT and F_{GET,SET}_FILE_RW_HINT
>>>  .\"
>>>  .TH FCNTL 2 2017-05-03 "Linux" "Linux Programmer's Manual"
>>>  .SH NAME
>>> @@ -1490,6 +1493,61 @@ Furthermore, if there are any asynchronous I/O operations
>>>  .RB ( io_submit (2))
>>>  pending on the file,
>>>  all outstanding writes will be discarded.
>>> +.SS File read/write hints
>>> +Write life time hints can be used to inform the kernel about the relative
>>> +expected life time of writes on a given inode or file. An application may
>>> +use this interface for separating writes into different write classes,
>>> +so that multiple users or applications running on a single storage back-end
>>> +can aggregate their IO patterns in a consistent manner. However, there are
>>> +no functional semantics implied by these flags, and different IO classes
>>> +can use the write life time hints in arbitrary ways so long as they are used
>>> +consistently.
>>
>> I read the above, and it is still not clear to me what a "write lifetime"
>> is. Can you add some text to explain? (I'm guessing that it means the
>> amount of time before the data is overwritten by another write, but I'm
>> not sure of that, and it needs to be more explicit in the text.)
>
> You are right, it's exactly what it refers to. It's essentially data
> lifetime hints for a write command. I'd be happy to take suggestions to
> improve the description of it.

Well, a sentence something like my above is probably a start, but I'd
prefer you worded something, since it might contain more insight than
anything that I propose.

>>> +.TP
>>> +.BR F_GET_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>>> +Returns the value of the read/write hint associated with the underlying inode
>>> +for the file descriptor.
>>> +.TP
>>> +.BR F_SET_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>>> +Sets the read/write hint value associated with the underlying inode for the
>>> +file descriptor.
>>> +.TP
>>> +.BR F_GET_FILE_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>>> +Returns the value of the read/write hint associated with the file descriptor.
>>
>> Do you mean here "file descriptor" or "file description (i.e., the
>> open file handle)? Maybe you mean the former, but I want to confirm.
>
> I do mean file descriptor.

So, what are the semantics if a file descriptor is duplicated using
dup(2) or similar? If I understand correctly, then the write lifetime
hint has no effect for the new file descriptor, right?

Cheers,

Michael

>>> +.TP
>>> +.BR F_SET_FILE_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>>> +Sets the read/write hint value associated with the file descriptor.
>>
>> Same as above comment.
>
> Ditto.
>
> --
> Jens Axboe
>



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: man page update (fcntl(2) new set/get write hints)

[Jens Axboe]

[-- Attachment #1: Type: text/plain, Size: 5188 bytes --]

On 08/24/2017 04:10 PM, Michael Kerrisk (man-pages) wrote:
> Hello Jens,
> 
> On 24 August 2017 at 16:39, Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org> wrote:
>> On 08/23/2017 04:41 PM, Michael Kerrisk (man-pages) wrote:
>>> Hello Jens,
>>>
>>> On 07/31/2017 05:24 PM, Jens Axboe wrote:
>>>> Hi Michael,
>>>>
>>>> Not sure what the best way to submit this is, but I wrote an
>>>> update for the fcntl man page to cover the new set/get write
>>>> hints commands that are added in 4.13. Attaching the patch, but
>>>> you can also find it below, in a fork of your github repo:
>>>>
>>>> https://github.com/axboe/man-pages/commits/fcntl-write-hints
>>>
>>> Thanks for the patch. Some questions below.
>>>
>>>> From 3eee36f14a650e50f14b42ae1d4fec33e78e6254 Mon Sep 17 00:00:00 2001
>>>> From: Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
>>>> Date: Mon, 26 Jun 2017 16:58:29 -0600
>>>> Subject: [PATCH] man2/fcntl.2: add description of new read/write hint commands
>>>>
>>>> This adds a description of the new read/write hint commands:
>>>>
>>>> F_GET_RW_HINT                Get inode read/write hint
>>>> F_SET_RW_HINT                Set inode read/write hint
>>>> F_GET_FILE_RW_HINT   Get file read/write hint
>>>> F_SET_FILE_RW_HINT   Set file read/write hint
>>>>
>>>> and the currently available values, as of Linux 4.13.
>>>>
>>>> Signed-off-by: Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
>>>> ---
>>>>  man2/fcntl.2 | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>  1 file changed, 58 insertions(+)
>>>>
>>>> diff --git a/man2/fcntl.2 b/man2/fcntl.2
>>>> index 70758d2fcd99..07eacc9564fa 100644
>>>> --- a/man2/fcntl.2
>>>> +++ b/man2/fcntl.2
>>>> @@ -5,6 +5,7 @@
>>>>  .\" and Copyright (C) 2002-2010, 2014 Michael Kerrisk;
>>>>  .\" and Copyright (C) 2014 Jeff Layton
>>>>  .\" and Copyright (C) 2014 David Herrmann
>>>> +.\" and Copyright (C) 2017 Jens Axboe
>>>>  .\"
>>>>  .\" %%%LICENSE_START(VERBATIM)
>>>>  .\" Permission is granted to make and distribute verbatim copies of this
>>>> @@ -61,6 +62,8 @@
>>>>  .\" Document F_SETPIPE_SZ and F_GETPIPE_SZ.
>>>>  .\" 2014-07-08, David Herrmann <dh.herrmann-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>>>>  .\"     Document F_ADD_SEALS and F_GET_SEALS
>>>> +.\" 2017-06-26, Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
>>>> +.\"     Document F_{GET,SET}_RW_HINT and F_{GET,SET}_FILE_RW_HINT
>>>>  .\"
>>>>  .TH FCNTL 2 2017-05-03 "Linux" "Linux Programmer's Manual"
>>>>  .SH NAME
>>>> @@ -1490,6 +1493,61 @@ Furthermore, if there are any asynchronous I/O operations
>>>>  .RB ( io_submit (2))
>>>>  pending on the file,
>>>>  all outstanding writes will be discarded.
>>>> +.SS File read/write hints
>>>> +Write life time hints can be used to inform the kernel about the relative
>>>> +expected life time of writes on a given inode or file. An application may
>>>> +use this interface for separating writes into different write classes,
>>>> +so that multiple users or applications running on a single storage back-end
>>>> +can aggregate their IO patterns in a consistent manner. However, there are
>>>> +no functional semantics implied by these flags, and different IO classes
>>>> +can use the write life time hints in arbitrary ways so long as they are used
>>>> +consistently.
>>>
>>> I read the above, and it is still not clear to me what a "write lifetime"
>>> is. Can you add some text to explain? (I'm guessing that it means the
>>> amount of time before the data is overwritten by another write, but I'm
>>> not sure of that, and it needs to be more explicit in the text.)
>>
>> You are right, it's exactly what it refers to. It's essentially data
>> lifetime hints for a write command. I'd be happy to take suggestions to
>> improve the description of it.
> 
> Well, a sentence something like my above is probably a start, but I'd
> prefer you worded something, since it might contain more insight than
> anything that I propose.

Could perhaps insert a sentence (after the first one) like the following:

"Life time refers to the expected time the data will live on media, before
being overwritten or erased."

>>>> +.BR F_GET_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>>>> +Returns the value of the read/write hint associated with the underlying inode
>>>> +for the file descriptor.
>>>> +.TP
>>>> +.BR F_SET_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>>>> +Sets the read/write hint value associated with the underlying inode for the
>>>> +file descriptor.
>>>> +.TP
>>>> +.BR F_GET_FILE_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>>>> +Returns the value of the read/write hint associated with the file descriptor.
>>>
>>> Do you mean here "file descriptor" or "file description (i.e., the
>>> open file handle)? Maybe you mean the former, but I want to confirm.
>>
>> I do mean file descriptor.
> 
> So, what are the semantics if a file descriptor is duplicated using
> dup(2) or similar? If I understand correctly, then the write lifetime
> hint has no effect for the new file descriptor, right?

If it's dup(2)'ed, then the new file descriptor will refer to the same
hints as the previous. See attached test file.

-- 
Jens Axboe


[-- Attachment #2: test-writehints.c --]
[-- Type: text/x-csrc, Size: 3194 bytes --]

/*
 * test-writehints.c: test file/inode write hint setting/getting
 */
#include <stdio.h>
#include <fcntl.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdbool.h>
#include <inttypes.h>
#include <assert.h>

#ifndef F_GET_RW_HINT
#define F_LINUX_SPECIFIC_BASE	1024
#define F_GET_RW_HINT		(F_LINUX_SPECIFIC_BASE + 11)
#define F_SET_RW_HINT		(F_LINUX_SPECIFIC_BASE + 12)
#define F_GET_FILE_RW_HINT	(F_LINUX_SPECIFIC_BASE + 13)
#define F_SET_FILE_RW_HINT	(F_LINUX_SPECIFIC_BASE + 14)

#define RWH_WRITE_LIFE_NOT_SET	0
#define RWH_WRITE_LIFE_NONE	1
#define RWH_WRITE_LIFE_SHORT	2
#define RWH_WRITE_LIFE_MEDIUM	3
#define RWH_WRITE_LIFE_LONG	4
#define RWH_WRITE_LIFE_EXTREME	5

#endif

static int __get_write_hint(int fd, int cmd)
{
	uint64_t hint;
	int ret;

	ret = fcntl(fd, cmd, &hint);
	if (ret < 0) {
		perror("fcntl: F_GET_RW_FILE_HINT");
		return -1;
	}

	return hint;
}

static int get_file_write_hint(int fd)
{
	return __get_write_hint(fd, F_GET_FILE_RW_HINT);
}

static int get_inode_write_hint(int fd)
{
	return __get_write_hint(fd, F_GET_RW_HINT);
}

static void set_file_write_hint(int fd, uint64_t hint)
{
	uint64_t set_hint = hint;
	int ret;

	ret = fcntl(fd, F_SET_FILE_RW_HINT, &set_hint);
	if (ret < 0) {
		perror("fcntl: F_RW_SET_HINT");
		return;
	}
}

static void set_inode_write_hint(int fd, uint64_t hint)
{
	uint64_t set_hint = hint;
	int ret;

	ret = fcntl(fd, F_SET_RW_HINT, &set_hint);
	if (ret < 0) {
		perror("fcntl: F_RW_SET_HINT");
		return;
	}
}

int main(int argc, char *argv[])
{
	char filename[] = "/tmp/writehintsXXXXXX";
	int ihint, fhint, fd, dupfd;

	fd = mkstemp(filename);
	if (fd < 0) {
		perror("open");
		return 2;
	}

	/*
	 * Default hints for both file and inode should be NOT_SET
	 */
	fhint = get_file_write_hint(fd);
	if (fhint < 0)
		return 0;
	ihint = get_inode_write_hint(fd);
	assert(fhint == ihint);
	assert(fhint == RWH_WRITE_LIFE_NOT_SET);

	/*
	 * Set inode hint, check file hint returns the right hint
	 */
	set_inode_write_hint(fd, RWH_WRITE_LIFE_SHORT);
	fhint = get_file_write_hint(fd);
	ihint = get_inode_write_hint(fd);
	assert(fhint == ihint);
	assert(fhint == RWH_WRITE_LIFE_SHORT);

	/*
	 * Check that a duped descriptor returns the right hint
	 */
	dupfd = dup(fd);
	fhint = get_file_write_hint(dupfd);
	ihint = get_inode_write_hint(dupfd);
	assert(fhint == RWH_WRITE_LIFE_SHORT);
	assert(ihint == RWH_WRITE_LIFE_SHORT);

	/*
	 * Now set file hint, ensure that this is now the hint we get
	 */
	set_file_write_hint(fd, RWH_WRITE_LIFE_LONG);
	fhint = get_file_write_hint(fd);
	ihint = get_inode_write_hint(fd);
	assert(fhint == RWH_WRITE_LIFE_LONG);
	assert(ihint == RWH_WRITE_LIFE_SHORT);

	/*
	 * Clear inode write hint, ensure that file still returns the set hint
	 */
	set_inode_write_hint(fd, RWH_WRITE_LIFE_NOT_SET);
	fhint = get_file_write_hint(fd);
	ihint = get_inode_write_hint(fd);
	assert(fhint == RWH_WRITE_LIFE_LONG);
	assert(ihint == RWH_WRITE_LIFE_NOT_SET);

	/*
	 * Clear file write hint, ensure that now returns cleared
	 */
	set_file_write_hint(fd, RWH_WRITE_LIFE_NOT_SET);
	fhint = get_file_write_hint(fd);
	assert(fhint == RWH_WRITE_LIFE_NOT_SET);

	close(fd);
	close(dupfd);
	unlink(filename);
	return 0;
}

Re: man page update (fcntl(2) new set/get write hints)

[Michael Kerrisk (man-pages)]

Hi Jens,

On 25 August 2017 at 22:43, Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org> wrote:
> On 08/24/2017 04:10 PM, Michael Kerrisk (man-pages) wrote:
>> Hello Jens,
>>
>> On 24 August 2017 at 16:39, Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org> wrote:
>>> On 08/23/2017 04:41 PM, Michael Kerrisk (man-pages) wrote:
>>>> Hello Jens,
>>>>
>>>> On 07/31/2017 05:24 PM, Jens Axboe wrote:
>>>>> Hi Michael,
>>>>>
>>>>> Not sure what the best way to submit this is, but I wrote an
>>>>> update for the fcntl man page to cover the new set/get write
>>>>> hints commands that are added in 4.13. Attaching the patch, but
>>>>> you can also find it below, in a fork of your github repo:
>>>>>
>>>>> https://github.com/axboe/man-pages/commits/fcntl-write-hints
>>>>
>>>> Thanks for the patch. Some questions below.
>>>>
>>>>> From 3eee36f14a650e50f14b42ae1d4fec33e78e6254 Mon Sep 17 00:00:00 2001
>>>>> From: Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
>>>>> Date: Mon, 26 Jun 2017 16:58:29 -0600
>>>>> Subject: [PATCH] man2/fcntl.2: add description of new read/write hint commands
>>>>>
>>>>> This adds a description of the new read/write hint commands:
>>>>>
>>>>> F_GET_RW_HINT                Get inode read/write hint
>>>>> F_SET_RW_HINT                Set inode read/write hint
>>>>> F_GET_FILE_RW_HINT   Get file read/write hint
>>>>> F_SET_FILE_RW_HINT   Set file read/write hint
>>>>>
>>>>> and the currently available values, as of Linux 4.13.
>>>>>
>>>>> Signed-off-by: Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
>>>>> ---
>>>>>  man2/fcntl.2 | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>  1 file changed, 58 insertions(+)
>>>>>
>>>>> diff --git a/man2/fcntl.2 b/man2/fcntl.2
>>>>> index 70758d2fcd99..07eacc9564fa 100644
>>>>> --- a/man2/fcntl.2
>>>>> +++ b/man2/fcntl.2
>>>>> @@ -5,6 +5,7 @@
>>>>>  .\" and Copyright (C) 2002-2010, 2014 Michael Kerrisk;
>>>>>  .\" and Copyright (C) 2014 Jeff Layton
>>>>>  .\" and Copyright (C) 2014 David Herrmann
>>>>> +.\" and Copyright (C) 2017 Jens Axboe
>>>>>  .\"
>>>>>  .\" %%%LICENSE_START(VERBATIM)
>>>>>  .\" Permission is granted to make and distribute verbatim copies of this
>>>>> @@ -61,6 +62,8 @@
>>>>>  .\" Document F_SETPIPE_SZ and F_GETPIPE_SZ.
>>>>>  .\" 2014-07-08, David Herrmann <dh.herrmann-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>>>>>  .\"     Document F_ADD_SEALS and F_GET_SEALS
>>>>> +.\" 2017-06-26, Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
>>>>> +.\"     Document F_{GET,SET}_RW_HINT and F_{GET,SET}_FILE_RW_HINT
>>>>>  .\"
>>>>>  .TH FCNTL 2 2017-05-03 "Linux" "Linux Programmer's Manual"
>>>>>  .SH NAME
>>>>> @@ -1490,6 +1493,61 @@ Furthermore, if there are any asynchronous I/O operations
>>>>>  .RB ( io_submit (2))
>>>>>  pending on the file,
>>>>>  all outstanding writes will be discarded.
>>>>> +.SS File read/write hints
>>>>> +Write life time hints can be used to inform the kernel about the relative
>>>>> +expected life time of writes on a given inode or file. An application may
>>>>> +use this interface for separating writes into different write classes,
>>>>> +so that multiple users or applications running on a single storage back-end
>>>>> +can aggregate their IO patterns in a consistent manner. However, there are
>>>>> +no functional semantics implied by these flags, and different IO classes
>>>>> +can use the write life time hints in arbitrary ways so long as they are used
>>>>> +consistently.
>>>>
>>>> I read the above, and it is still not clear to me what a "write lifetime"
>>>> is. Can you add some text to explain? (I'm guessing that it means the
>>>> amount of time before the data is overwritten by another write, but I'm
>>>> not sure of that, and it needs to be more explicit in the text.)
>>>
>>> You are right, it's exactly what it refers to. It's essentially data
>>> lifetime hints for a write command. I'd be happy to take suggestions to
>>> improve the description of it.
>>
>> Well, a sentence something like my above is probably a start, but I'd
>> prefer you worded something, since it might contain more insight than
>> anything that I propose.
>
> Could perhaps insert a sentence (after the first one) like the following:
>
> "Life time refers to the expected time the data will live on media, before
> being overwritten or erased."

Sounds good.

>>>>> +.BR F_GET_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>>>>> +Returns the value of the read/write hint associated with the underlying inode
>>>>> +for the file descriptor.
>>>>> +.TP
>>>>> +.BR F_SET_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>>>>> +Sets the read/write hint value associated with the underlying inode for the
>>>>> +file descriptor.
>>>>> +.TP
>>>>> +.BR F_GET_FILE_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>>>>> +Returns the value of the read/write hint associated with the file descriptor.
>>>>
>>>> Do you mean here "file descriptor" or "file description (i.e., the
>>>> open file handle)? Maybe you mean the former, but I want to confirm.
>>>
>>> I do mean file descriptor.
>>
>> So, what are the semantics if a file descriptor is duplicated using
>> dup(2) or similar? If I understand correctly, then the write lifetime
>> hint has no effect for the new file descriptor, right?
>
> If it's dup(2)'ed, then the new file descriptor will refer to the same
> hints as the previous. See attached test file.

But then isn't this exactly the point I asked about: are the hints
private to a file descriptor or are they associated with the open file
description (open file table entry, "struct file")? You said "I do
mean file descriptor", but actually I understand what you just said
now as "hints are associated with the open file description, which may
be referred to by multiple duplicated file descriptors". Can you
clarify?

Thanks,

Michael



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: man page update (fcntl(2) new set/get write hints)

[Jens Axboe]

On 08/25/2017 02:51 PM, Michael Kerrisk (man-pages) wrote:
>>>>> Do you mean here "file descriptor" or "file description (i.e., the
>>>>> open file handle)? Maybe you mean the former, but I want to confirm.
>>>>
>>>> I do mean file descriptor.
>>>
>>> So, what are the semantics if a file descriptor is duplicated using
>>> dup(2) or similar? If I understand correctly, then the write lifetime
>>> hint has no effect for the new file descriptor, right?
>>
>> If it's dup(2)'ed, then the new file descriptor will refer to the same
>> hints as the previous. See attached test file.
> 
> But then isn't this exactly the point I asked about: are the hints
> private to a file descriptor or are they associated with the open file
> description (open file table entry, "struct file")? You said "I do
> mean file descriptor", but actually I understand what you just said
> now as "hints are associated with the open file description, which may
> be referred to by multiple duplicated file descriptors". Can you
> clarify?

You are right, I misunderstood your original question. They do follow
the file description. So the dup'ed one will return the same as the
original, even if the hints on the original fd get modified. That is the
expected behavior.

-- 
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: man page update (fcntl(2) new set/get write hints)

[Michael Kerrisk (man-pages)]

Hi Jens,

On 08/25/2017 10:55 PM, Jens Axboe wrote:
> On 08/25/2017 02:51 PM, Michael Kerrisk (man-pages) wrote:
>>>>>> Do you mean here "file descriptor" or "file description (i.e., the
>>>>>> open file handle)? Maybe you mean the former, but I want to confirm.
>>>>>
>>>>> I do mean file descriptor.
>>>>
>>>> So, what are the semantics if a file descriptor is duplicated using
>>>> dup(2) or similar? If I understand correctly, then the write lifetime
>>>> hint has no effect for the new file descriptor, right?
>>>
>>> If it's dup(2)'ed, then the new file descriptor will refer to the same
>>> hints as the previous. See attached test file.
>>
>> But then isn't this exactly the point I asked about: are the hints
>> private to a file descriptor or are they associated with the open file
>> description (open file table entry, "struct file")? You said "I do
>> mean file descriptor", but actually I understand what you just said
>> now as "hints are associated with the open file description, which may
>> be referred to by multiple duplicated file descriptors". Can you
>> clarify?
> 
> You are right, I misunderstood your original question. They do follow
> the file description. So the dup'ed one will return the same as the
> original, even if the hints on the original fd get modified. That is the
> expected behavior.

So, I am still confused. I was wondering whether the hints are 
associated with the open file description (OFD), rather than the 
file descriptor. You said yes, then say that the dup'ed file 
descriptor will have the same hints even if the hints on the
original file descriptor are modified. To me that sounds like:
the hints are associated with the file descriptor, and not the
OFD, and during dup(2) the hints are *copied* to the the new
file descriptor, with the result that after the dup(2) the hints
can be modified independently for the two file descriptors.

Can you clarify please?

Thanks,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: man page update (fcntl(2) new set/get write hints)

[Jens Axboe]

On 08/28/2017 01:19 PM, Michael Kerrisk (man-pages) wrote:
> Hi Jens,
> 
> On 08/25/2017 10:55 PM, Jens Axboe wrote:
>> On 08/25/2017 02:51 PM, Michael Kerrisk (man-pages) wrote:
>>>>>>> Do you mean here "file descriptor" or "file description (i.e., the
>>>>>>> open file handle)? Maybe you mean the former, but I want to confirm.
>>>>>>
>>>>>> I do mean file descriptor.
>>>>>
>>>>> So, what are the semantics if a file descriptor is duplicated using
>>>>> dup(2) or similar? If I understand correctly, then the write lifetime
>>>>> hint has no effect for the new file descriptor, right?
>>>>
>>>> If it's dup(2)'ed, then the new file descriptor will refer to the same
>>>> hints as the previous. See attached test file.
>>>
>>> But then isn't this exactly the point I asked about: are the hints
>>> private to a file descriptor or are they associated with the open file
>>> description (open file table entry, "struct file")? You said "I do
>>> mean file descriptor", but actually I understand what you just said
>>> now as "hints are associated with the open file description, which may
>>> be referred to by multiple duplicated file descriptors". Can you
>>> clarify?
>>
>> You are right, I misunderstood your original question. They do follow
>> the file description. So the dup'ed one will return the same as the
>> original, even if the hints on the original fd get modified. That is the
>> expected behavior.
> 
> So, I am still confused. I was wondering whether the hints are 
> associated with the open file description (OFD), rather than the 
> file descriptor. You said yes, then say that the dup'ed file 
> descriptor will have the same hints even if the hints on the
> original file descriptor are modified. To me that sounds like:
> the hints are associated with the file descriptor, and not the
> OFD, and during dup(2) the hints are *copied* to the the new
> file descriptor, with the result that after the dup(2) the hints
> can be modified independently for the two file descriptors.
> 
> Can you clarify please?

No, that's not how it behaves. If you dup(2) the file descriptor, then
the dup'ed descriptor will return the same hint as was set on the
original.  If you change/clear the hint on the original, the dup'ed
descriptor will now return the new hint.


-- 
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: man page update (fcntl(2) new set/get write hints)

[Michael Kerrisk (man-pages)]

On 08/28/2017 10:15 PM, Jens Axboe wrote:
> On 08/28/2017 01:19 PM, Michael Kerrisk (man-pages) wrote:
>> Hi Jens,
>>
>> On 08/25/2017 10:55 PM, Jens Axboe wrote:
>>> On 08/25/2017 02:51 PM, Michael Kerrisk (man-pages) wrote:
>>>>>>>> Do you mean here "file descriptor" or "file description (i.e., the
>>>>>>>> open file handle)? Maybe you mean the former, but I want to confirm.
>>>>>>>
>>>>>>> I do mean file descriptor.
>>>>>>
>>>>>> So, what are the semantics if a file descriptor is duplicated using
>>>>>> dup(2) or similar? If I understand correctly, then the write lifetime
>>>>>> hint has no effect for the new file descriptor, right?
>>>>>
>>>>> If it's dup(2)'ed, then the new file descriptor will refer to the same
>>>>> hints as the previous. See attached test file.
>>>>
>>>> But then isn't this exactly the point I asked about: are the hints
>>>> private to a file descriptor or are they associated with the open file
>>>> description (open file table entry, "struct file")? You said "I do
>>>> mean file descriptor", but actually I understand what you just said
>>>> now as "hints are associated with the open file description, which may
>>>> be referred to by multiple duplicated file descriptors". Can you
>>>> clarify?
>>>
>>> You are right, I misunderstood your original question. They do follow
>>> the file description. So the dup'ed one will return the same as the
>>> original, even if the hints on the original fd get modified. That is the
>>> expected behavior.
>>
>> So, I am still confused. I was wondering whether the hints are 
>> associated with the open file description (OFD), rather than the 
>> file descriptor. You said yes, then say that the dup'ed file 
>> descriptor will have the same hints even if the hints on the
>> original file descriptor are modified. To me that sounds like:
>> the hints are associated with the file descriptor, and not the
>> OFD, and during dup(2) the hints are *copied* to the the new
>> file descriptor, with the result that after the dup(2) the hints
>> can be modified independently for the two file descriptors.
>>
>> Can you clarify please?
> 
> No, that's not how it behaves. If you dup(2) the file descriptor, then
> the dup'ed descriptor will return the same hint as was set on the
> original.  If you change/clear the hint on the original, the dup'ed
> descriptor will now return the new hint.

Okay -- thanks. I'd misunderstood your earlier words. Okay, I've
hacked this text to arrive at new text below. Could you please check
it? Also, there are some details that are still missing. Could you take 
a look at the questions below please.

[[[
    File read/write hints
       Write lifetime hints can be used to inform the  kernel  about  the
       relative  expected  lifetime  of  writes on a given inode or via a
       particular open file description.  (See open(2) for an explanation
       of open file desriptions.)  In this context, the term "write life‐
       time" means the expected time the data will live on media,  before
       being overwritten or erased.

       An  application  may use the different hint values specified below
       to separate writes into different write classes, so that  multiple
       users  or  applications  running  on a single storage back-end can
       aggregate their I/O patterns in  a  consistent  manner.   However,
       there are no functional semantics implied by these flags, and dif‐
       ferent I/O classes can use the write lifetime hints  in  arbitrary
       ways, so long as the hints are used consistently.

QUESTIONS:
* What are write classes?
* What are I/O classes?
* What is the purpose of using read/write hints? I assume it's a
  performance point, but the text is not explicit about that.
* You variously wrote "read/write hints" and "write hints". Let's make 
  it consistent. Which is the preferred term?

       The  following  operations  can be applied to the file descriptor,
       fd:

       F_GET_RW_HINT (uint64_t *; since Linux 4.13)
              Returns the value of the read/write  hint  associated  with
              the underlying inode referred to by fd.

       F_SET_RW_HINT (uint64_t *; since Linux 4.13)
              Sets the read/write hint value associated with the underly‐
              ing inode referred to by fd.

       F_GET_FILE_RW_HINT (uint64_t *; since Linux 4.13)
              Returns the value of the read/write  hint  associated  with
              the open file description referred to by fd.

       F_SET_FILE_RW_HINT (uint64_t *; since Linux 4.13)
              Sets  the  read/write  hint  value associated with the open
              file description referred to by fd.

       If an open file description has not  been  assigned  a  read/write
       hint, then it shall use the value assigned to the inode, if any.

       The following read/write hints are valid since Linux 4.13:

       RWH_WRITE_LIFE_NOT_SET
              No specific hint has been set.  This is the default value.

       RWH_WRITE_LIFE_NONE
              No  specific write lifetime is associated with this file or
              inode.

       RWH_WRITE_LIFE_SHORT
              Data written to this inode or via this open  file  descrip‐
              tion is expected to have a short lifetime.

       RWH_WRITE_LIFE_MEDIUM
              Data  written  to this inode or via this open file descrip‐
              tion is expected to have a lifetime longer than data  writ‐
              ten with RWH_WRITE_LIFE_SHORT.

       RWH_WRITE_LIFE_LONG
              Data  written  to this inode or via this open file descrip‐
              tion is expected to have a lifetime longer than data  writ‐
              ten with RWH_WRITE_LIFE_MEDIUM.

       RWH_WRITE_LIFE_EXTREME
              Data  written  to this inode or via this open file descrip‐
              tion is expected to have a lifetime longer than data  writ‐
              ten with RWH_WRITE_LIFE_LONG.

       All  the  write-specific  hints are relative to each other, and no
       individual absolute meaning should be attributed to them.
]]]

Cheers,

Michael



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: man page update (fcntl(2) new set/get write hints)

[Michael Kerrisk (man-pages)]

Hi Jens,

Ping!

Cheers,

Michael


On 08/28/2017 11:16 PM, Michael Kerrisk (man-pages) wrote:
> On 08/28/2017 10:15 PM, Jens Axboe wrote:
>> On 08/28/2017 01:19 PM, Michael Kerrisk (man-pages) wrote:
>>> Hi Jens,
>>>
>>> On 08/25/2017 10:55 PM, Jens Axboe wrote:
>>>> On 08/25/2017 02:51 PM, Michael Kerrisk (man-pages) wrote:
>>>>>>>>> Do you mean here "file descriptor" or "file description (i.e., the
>>>>>>>>> open file handle)? Maybe you mean the former, but I want to confirm.
>>>>>>>>
>>>>>>>> I do mean file descriptor.
>>>>>>>
>>>>>>> So, what are the semantics if a file descriptor is duplicated using
>>>>>>> dup(2) or similar? If I understand correctly, then the write lifetime
>>>>>>> hint has no effect for the new file descriptor, right?
>>>>>>
>>>>>> If it's dup(2)'ed, then the new file descriptor will refer to the same
>>>>>> hints as the previous. See attached test file.
>>>>>
>>>>> But then isn't this exactly the point I asked about: are the hints
>>>>> private to a file descriptor or are they associated with the open file
>>>>> description (open file table entry, "struct file")? You said "I do
>>>>> mean file descriptor", but actually I understand what you just said
>>>>> now as "hints are associated with the open file description, which may
>>>>> be referred to by multiple duplicated file descriptors". Can you
>>>>> clarify?
>>>>
>>>> You are right, I misunderstood your original question. They do follow
>>>> the file description. So the dup'ed one will return the same as the
>>>> original, even if the hints on the original fd get modified. That is the
>>>> expected behavior.
>>>
>>> So, I am still confused. I was wondering whether the hints are 
>>> associated with the open file description (OFD), rather than the 
>>> file descriptor. You said yes, then say that the dup'ed file 
>>> descriptor will have the same hints even if the hints on the
>>> original file descriptor are modified. To me that sounds like:
>>> the hints are associated with the file descriptor, and not the
>>> OFD, and during dup(2) the hints are *copied* to the the new
>>> file descriptor, with the result that after the dup(2) the hints
>>> can be modified independently for the two file descriptors.
>>>
>>> Can you clarify please?
>>
>> No, that's not how it behaves. If you dup(2) the file descriptor, then
>> the dup'ed descriptor will return the same hint as was set on the
>> original.  If you change/clear the hint on the original, the dup'ed
>> descriptor will now return the new hint.
> 
> Okay -- thanks. I'd misunderstood your earlier words. Okay, I've
> hacked this text to arrive at new text below. Could you please check
> it? Also, there are some details that are still missing. Could you take 
> a look at the questions below please.
> 
> [[[
>     File read/write hints
>        Write lifetime hints can be used to inform the  kernel  about  the
>        relative  expected  lifetime  of  writes on a given inode or via a
>        particular open file description.  (See open(2) for an explanation
>        of open file desriptions.)  In this context, the term "write life‐
>        time" means the expected time the data will live on media,  before
>        being overwritten or erased.
> 
>        An  application  may use the different hint values specified below
>        to separate writes into different write classes, so that  multiple
>        users  or  applications  running  on a single storage back-end can
>        aggregate their I/O patterns in  a  consistent  manner.   However,
>        there are no functional semantics implied by these flags, and dif‐
>        ferent I/O classes can use the write lifetime hints  in  arbitrary
>        ways, so long as the hints are used consistently.
> 
> QUESTIONS:
> * What are write classes?
> * What are I/O classes?
> * What is the purpose of using read/write hints? I assume it's a
>   performance point, but the text is not explicit about that.
> * You variously wrote "read/write hints" and "write hints". Let's make 
>   it consistent. Which is the preferred term?
> 
>        The  following  operations  can be applied to the file descriptor,
>        fd:
> 
>        F_GET_RW_HINT (uint64_t *; since Linux 4.13)
>               Returns the value of the read/write  hint  associated  with
>               the underlying inode referred to by fd.
> 
>        F_SET_RW_HINT (uint64_t *; since Linux 4.13)
>               Sets the read/write hint value associated with the underly‐
>               ing inode referred to by fd.
> 
>        F_GET_FILE_RW_HINT (uint64_t *; since Linux 4.13)
>               Returns the value of the read/write  hint  associated  with
>               the open file description referred to by fd.
> 
>        F_SET_FILE_RW_HINT (uint64_t *; since Linux 4.13)
>               Sets  the  read/write  hint  value associated with the open
>               file description referred to by fd.
> 
>        If an open file description has not  been  assigned  a  read/write
>        hint, then it shall use the value assigned to the inode, if any.
> 
>        The following read/write hints are valid since Linux 4.13:
> 
>        RWH_WRITE_LIFE_NOT_SET
>               No specific hint has been set.  This is the default value.
> 
>        RWH_WRITE_LIFE_NONE
>               No  specific write lifetime is associated with this file or
>               inode.
> 
>        RWH_WRITE_LIFE_SHORT
>               Data written to this inode or via this open  file  descrip‐
>               tion is expected to have a short lifetime.
> 
>        RWH_WRITE_LIFE_MEDIUM
>               Data  written  to this inode or via this open file descrip‐
>               tion is expected to have a lifetime longer than data  writ‐
>               ten with RWH_WRITE_LIFE_SHORT.
> 
>        RWH_WRITE_LIFE_LONG
>               Data  written  to this inode or via this open file descrip‐
>               tion is expected to have a lifetime longer than data  writ‐
>               ten with RWH_WRITE_LIFE_MEDIUM.
> 
>        RWH_WRITE_LIFE_EXTREME
>               Data  written  to this inode or via this open file descrip‐
>               tion is expected to have a lifetime longer than data  writ‐
>               ten with RWH_WRITE_LIFE_LONG.
> 
>        All  the  write-specific  hints are relative to each other, and no
>        individual absolute meaning should be attributed to them.
> ]]]
> 
> Cheers,
> 
> Michael
> 
> 
> 


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: man page update (fcntl(2) new set/get write hints)

[Michael Kerrisk (man-pages)]

Hi Jens,

Ping^2!

Cheers,

Michael


On 3 September 2017 at 02:45, Michael Kerrisk (man-pages)
<mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Hi Jens,
>
> Ping!
>
> Cheers,
>
> Michael
>
>
> On 08/28/2017 11:16 PM, Michael Kerrisk (man-pages) wrote:
>> On 08/28/2017 10:15 PM, Jens Axboe wrote:
>>> On 08/28/2017 01:19 PM, Michael Kerrisk (man-pages) wrote:
>>>> Hi Jens,
>>>>
>>>> On 08/25/2017 10:55 PM, Jens Axboe wrote:
>>>>> On 08/25/2017 02:51 PM, Michael Kerrisk (man-pages) wrote:
>>>>>>>>>> Do you mean here "file descriptor" or "file description (i.e., the
>>>>>>>>>> open file handle)? Maybe you mean the former, but I want to confirm.
>>>>>>>>>
>>>>>>>>> I do mean file descriptor.
>>>>>>>>
>>>>>>>> So, what are the semantics if a file descriptor is duplicated using
>>>>>>>> dup(2) or similar? If I understand correctly, then the write lifetime
>>>>>>>> hint has no effect for the new file descriptor, right?
>>>>>>>
>>>>>>> If it's dup(2)'ed, then the new file descriptor will refer to the same
>>>>>>> hints as the previous. See attached test file.
>>>>>>
>>>>>> But then isn't this exactly the point I asked about: are the hints
>>>>>> private to a file descriptor or are they associated with the open file
>>>>>> description (open file table entry, "struct file")? You said "I do
>>>>>> mean file descriptor", but actually I understand what you just said
>>>>>> now as "hints are associated with the open file description, which may
>>>>>> be referred to by multiple duplicated file descriptors". Can you
>>>>>> clarify?
>>>>>
>>>>> You are right, I misunderstood your original question. They do follow
>>>>> the file description. So the dup'ed one will return the same as the
>>>>> original, even if the hints on the original fd get modified. That is the
>>>>> expected behavior.
>>>>
>>>> So, I am still confused. I was wondering whether the hints are
>>>> associated with the open file description (OFD), rather than the
>>>> file descriptor. You said yes, then say that the dup'ed file
>>>> descriptor will have the same hints even if the hints on the
>>>> original file descriptor are modified. To me that sounds like:
>>>> the hints are associated with the file descriptor, and not the
>>>> OFD, and during dup(2) the hints are *copied* to the the new
>>>> file descriptor, with the result that after the dup(2) the hints
>>>> can be modified independently for the two file descriptors.
>>>>
>>>> Can you clarify please?
>>>
>>> No, that's not how it behaves. If you dup(2) the file descriptor, then
>>> the dup'ed descriptor will return the same hint as was set on the
>>> original.  If you change/clear the hint on the original, the dup'ed
>>> descriptor will now return the new hint.
>>
>> Okay -- thanks. I'd misunderstood your earlier words. Okay, I've
>> hacked this text to arrive at new text below. Could you please check
>> it? Also, there are some details that are still missing. Could you take
>> a look at the questions below please.
>>
>> [[[
>>     File read/write hints
>>        Write lifetime hints can be used to inform the  kernel  about  the
>>        relative  expected  lifetime  of  writes on a given inode or via a
>>        particular open file description.  (See open(2) for an explanation
>>        of open file desriptions.)  In this context, the term "write life‐
>>        time" means the expected time the data will live on media,  before
>>        being overwritten or erased.
>>
>>        An  application  may use the different hint values specified below
>>        to separate writes into different write classes, so that  multiple
>>        users  or  applications  running  on a single storage back-end can
>>        aggregate their I/O patterns in  a  consistent  manner.   However,
>>        there are no functional semantics implied by these flags, and dif‐
>>        ferent I/O classes can use the write lifetime hints  in  arbitrary
>>        ways, so long as the hints are used consistently.
>>
>> QUESTIONS:
>> * What are write classes?
>> * What are I/O classes?
>> * What is the purpose of using read/write hints? I assume it's a
>>   performance point, but the text is not explicit about that.
>> * You variously wrote "read/write hints" and "write hints". Let's make
>>   it consistent. Which is the preferred term?
>>
>>        The  following  operations  can be applied to the file descriptor,
>>        fd:
>>
>>        F_GET_RW_HINT (uint64_t *; since Linux 4.13)
>>               Returns the value of the read/write  hint  associated  with
>>               the underlying inode referred to by fd.
>>
>>        F_SET_RW_HINT (uint64_t *; since Linux 4.13)
>>               Sets the read/write hint value associated with the underly‐
>>               ing inode referred to by fd.
>>
>>        F_GET_FILE_RW_HINT (uint64_t *; since Linux 4.13)
>>               Returns the value of the read/write  hint  associated  with
>>               the open file description referred to by fd.
>>
>>        F_SET_FILE_RW_HINT (uint64_t *; since Linux 4.13)
>>               Sets  the  read/write  hint  value associated with the open
>>               file description referred to by fd.
>>
>>        If an open file description has not  been  assigned  a  read/write
>>        hint, then it shall use the value assigned to the inode, if any.
>>
>>        The following read/write hints are valid since Linux 4.13:
>>
>>        RWH_WRITE_LIFE_NOT_SET
>>               No specific hint has been set.  This is the default value.
>>
>>        RWH_WRITE_LIFE_NONE
>>               No  specific write lifetime is associated with this file or
>>               inode.
>>
>>        RWH_WRITE_LIFE_SHORT
>>               Data written to this inode or via this open  file  descrip‐
>>               tion is expected to have a short lifetime.
>>
>>        RWH_WRITE_LIFE_MEDIUM
>>               Data  written  to this inode or via this open file descrip‐
>>               tion is expected to have a lifetime longer than data  writ‐
>>               ten with RWH_WRITE_LIFE_SHORT.
>>
>>        RWH_WRITE_LIFE_LONG
>>               Data  written  to this inode or via this open file descrip‐
>>               tion is expected to have a lifetime longer than data  writ‐
>>               ten with RWH_WRITE_LIFE_MEDIUM.
>>
>>        RWH_WRITE_LIFE_EXTREME
>>               Data  written  to this inode or via this open file descrip‐
>>               tion is expected to have a lifetime longer than data  writ‐
>>               ten with RWH_WRITE_LIFE_LONG.
>>
>>        All  the  write-specific  hints are relative to each other, and no
>>        individual absolute meaning should be attributed to them.
>> ]]]
>>
>> Cheers,
>>
>> Michael
>>
>>
>>
>
>
> --
> Michael Kerrisk
> Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
> Linux/UNIX System Programming Training: http://man7.org/training/



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: man page update (fcntl(2) new set/get write hints)

[Florian Weimer]

On 08/24/2017 12:41 AM, Michael Kerrisk (man-pages) wrote:
> +.TP
> +.BR F_GET_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
> +Returns the value of the read/write hint associated with the underlying inode
> +for the file descriptor.
> +.TP
> +.BR F_SET_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
> +Sets the read/write hint value associated with the underlying inode for the
> +file descriptor.

Surely the arguments are of pointer type for reads *and* writes?

There is no convenient way 32-bit userspace can pass a 64-bit integer
through are variadic dispatcher interface such as ioctl/fcntl/futex.

Thanks,
Florian
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: man page update (fcntl(2) new set/get write hints)

[Jens Axboe]

On 08/28/2017 03:07 AM, Florian Weimer wrote:
> On 08/24/2017 12:41 AM, Michael Kerrisk (man-pages) wrote:
>> +.TP
>> +.BR F_GET_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>> +Returns the value of the read/write hint associated with the underlying inode
>> +for the file descriptor.
>> +.TP
>> +.BR F_SET_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>> +Sets the read/write hint value associated with the underlying inode for the
>> +file descriptor.
> 
> Surely the arguments are of pointer type for reads *and* writes?
> 
> There is no convenient way 32-bit userspace can pass a 64-bit integer
> through are variadic dispatcher interface such as ioctl/fcntl/futex.

Yes, they are pointers to a u64 for both the get/set side. The return type
is 0/Exxxx.

-- 
Jens Axboe

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: man page update (fcntl(2) new set/get write hints)

[Michael Kerrisk (man-pages)]

On 08/28/2017 11:07 AM, Florian Weimer wrote:
> On 08/24/2017 12:41 AM, Michael Kerrisk (man-pages) wrote:
>> +.TP
>> +.BR F_GET_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>> +Returns the value of the read/write hint associated with the underlying inode
>> +for the file descriptor.
>> +.TP
>> +.BR F_SET_RW_HINT " (\fIuint64_t\fP; since Linux 4.13)"
>> +Sets the read/write hint value associated with the underlying inode for the
>> +file descriptor.
> 
> Surely the arguments are of pointer type for reads *and* writes?
> 
> There is no convenient way 32-bit userspace can pass a 64-bit integer
> through are variadic dispatcher interface such as ioctl/fcntl/futex.

Thanks, Florian. I corrected this.

Cheers,

Michael



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html