Linux-man Archive on lore.kernel.org
 help / color / Atom feed
* [v2 PATCH] move_pages.2: Returning positive value is a new error case
@ 2020-01-29 21:48 Yang Shi
  2020-01-30  9:06 ` Vlastimil Babka
  0 siblings, 1 reply; 6+ messages in thread
From: Yang Shi @ 2020-01-29 21:48 UTC (permalink / raw)
  To: mhocko, mtk.manpages, akpm
  Cc: yang.shi, linux-man, linux-api, linux-mm, linux-kernel

Since commit a49bd4d71637 ("mm, numa: rework do_pages_move"),
the semantic of move_pages() has changed to return the number of
non-migrated pages if they were result of a non-fatal reasons (usually a
busy page).  This was an unintentional change that hasn't been noticed
except for LTP tests which checked for the documented behavior.

There are two ways to go around this change.  We can even get back to the
original behavior and return -EAGAIN whenever migrate_pages is not able
to migrate pages due to non-fatal reasons.  Another option would be to
simply continue with the changed semantic and extend move_pages
documentation to clarify that -errno is returned on an invalid input or
when migration simply cannot succeed (e.g. -ENOMEM, -EBUSY) or the
number of pages that couldn't have been migrated due to ephemeral
reasons (e.g. page is pinned or locked for other reasons).

We decided to keep the second option in kernel because this behavior is in
place for some time without anybody complaining and possibly new users
depending on it.  Also it allows to have a slightly easier error handling
as the caller knows that it is worth to retry when err > 0.

Update man pages to reflect the new semantic.

Acked-by: Michal Hocko <mhocko@suse.com>
Cc: Michael Kerrisk <mtk.manpages@gmail.com>
Signed-off-by: Yang Shi <yang.shi@linux.alibaba.com>
---
v2: * Added notes about status array per Michal.
    * Added Michal's Acked-by.

 man2/move_pages.2 | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/man2/move_pages.2 b/man2/move_pages.2
index 1bf1053..83d5c81 100644
--- a/man2/move_pages.2
+++ b/man2/move_pages.2
@@ -104,7 +104,9 @@ pages that need to be moved.
 is an array of integers that return the status of each page.
 The array contains valid values only if
 .BR move_pages ()
-did not return an error.
+did not return an error.  Pre-initialization of the array to -1 or
+similar value which cannot represent a real numa node could help to
+identify pages that have been migrated
 .PP
 .I flags
 specify what types of pages to move.
@@ -164,9 +166,13 @@ returns zero.
 .\" do the right thing?
 On error, it returns \-1, and sets
 .I errno
-to indicate the error.
+to indicate the error. Or positive value to report the number of
+non-migrated pages.
 .SH ERRORS
 .TP
+.B Positive value
+The number of non-migrated pages if they were result of a non-fatal
+reasons since version 4.17.
 .B E2BIG
 Too many pages to move.
 Since Linux 2.6.29,
-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [v2 PATCH] move_pages.2: Returning positive value is a new error case
  2020-01-29 21:48 [v2 PATCH] move_pages.2: Returning positive value is a new error case Yang Shi
@ 2020-01-30  9:06 ` Vlastimil Babka
  2020-01-30 12:02   ` Michal Hocko
  0 siblings, 1 reply; 6+ messages in thread
From: Vlastimil Babka @ 2020-01-30  9:06 UTC (permalink / raw)
  To: Yang Shi, mhocko, mtk.manpages, akpm
  Cc: linux-man, linux-api, linux-mm, linux-kernel

On 1/29/20 10:48 PM, Yang Shi wrote:
> Since commit a49bd4d71637 ("mm, numa: rework do_pages_move"),
> the semantic of move_pages() has changed to return the number of
> non-migrated pages if they were result of a non-fatal reasons (usually a
> busy page).  This was an unintentional change that hasn't been noticed
> except for LTP tests which checked for the documented behavior.
> 
> There are two ways to go around this change.  We can even get back to the
> original behavior and return -EAGAIN whenever migrate_pages is not able

The manpage says EBUSY, not EAGAIN? And should its description be
updated too? I.e. that it's no longer returned since 4.17?

> to migrate pages due to non-fatal reasons.  Another option would be to
> simply continue with the changed semantic and extend move_pages
> documentation to clarify that -errno is returned on an invalid input or
> when migration simply cannot succeed (e.g. -ENOMEM, -EBUSY) or the
> number of pages that couldn't have been migrated due to ephemeral
> reasons (e.g. page is pinned or locked for other reasons).
> 
> We decided to keep the second option in kernel because this behavior is in
> place for some time without anybody complaining and possibly new users
> depending on it.  Also it allows to have a slightly easier error handling
> as the caller knows that it is worth to retry when err > 0.
> 
> Update man pages to reflect the new semantic.
> 
> Acked-by: Michal Hocko <mhocko@suse.com>
> Cc: Michael Kerrisk <mtk.manpages@gmail.com>
> Signed-off-by: Yang Shi <yang.shi@linux.alibaba.com>
> ---
> v2: * Added notes about status array per Michal.
>     * Added Michal's Acked-by.
> 
>  man2/move_pages.2 | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/man2/move_pages.2 b/man2/move_pages.2
> index 1bf1053..83d5c81 100644
> --- a/man2/move_pages.2
> +++ b/man2/move_pages.2
> @@ -104,7 +104,9 @@ pages that need to be moved.
>  is an array of integers that return the status of each page.
>  The array contains valid values only if
>  .BR move_pages ()
> -did not return an error.
> +did not return an error.  Pre-initialization of the array to -1 or
> +similar value which cannot represent a real numa node could help to
> +identify pages that have been migrated
>  .PP
>  .I flags
>  specify what types of pages to move.
> @@ -164,9 +166,13 @@ returns zero.
>  .\" do the right thing?
>  On error, it returns \-1, and sets
>  .I errno
> -to indicate the error.
> +to indicate the error. Or positive value to report the number of
> +non-migrated pages.
>  .SH ERRORS
>  .TP
> +.B Positive value
> +The number of non-migrated pages if they were result of a non-fatal
> +reasons since version 4.17.
>  .B E2BIG
>  Too many pages to move.
>  Since Linux 2.6.29,
> 


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [v2 PATCH] move_pages.2: Returning positive value is a new error case
  2020-01-30  9:06 ` Vlastimil Babka
@ 2020-01-30 12:02   ` Michal Hocko
  2020-01-30 12:56     ` Vlastimil Babka
  0 siblings, 1 reply; 6+ messages in thread
From: Michal Hocko @ 2020-01-30 12:02 UTC (permalink / raw)
  To: Vlastimil Babka
  Cc: Yang Shi, mtk.manpages, akpm, linux-man, linux-api, linux-mm,
	linux-kernel

On Thu 30-01-20 10:06:28, Vlastimil Babka wrote:
> On 1/29/20 10:48 PM, Yang Shi wrote:
> > Since commit a49bd4d71637 ("mm, numa: rework do_pages_move"),
> > the semantic of move_pages() has changed to return the number of
> > non-migrated pages if they were result of a non-fatal reasons (usually a
> > busy page).  This was an unintentional change that hasn't been noticed
> > except for LTP tests which checked for the documented behavior.
> > 
> > There are two ways to go around this change.  We can even get back to the
> > original behavior and return -EAGAIN whenever migrate_pages is not able
> 
> The manpage says EBUSY, not EAGAIN? And should its description be
> updated too?

The idea was that we _could_ return EAGAIN from the syscall if
migrate_pages > 0.

> I.e. that it's no longer returned since 4.17?

I am pretty sure this will require a deeper consideration. Do we return
EIO/EINVAL?
-- 
Michal Hocko
SUSE Labs

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [v2 PATCH] move_pages.2: Returning positive value is a new error case
  2020-01-30 12:02   ` Michal Hocko
@ 2020-01-30 12:56     ` Vlastimil Babka
  2020-01-30 13:48       ` Michal Hocko
  0 siblings, 1 reply; 6+ messages in thread
From: Vlastimil Babka @ 2020-01-30 12:56 UTC (permalink / raw)
  To: Michal Hocko
  Cc: Yang Shi, mtk.manpages, akpm, linux-man, linux-api, linux-mm,
	linux-kernel

On 1/30/20 1:02 PM, Michal Hocko wrote:
> On Thu 30-01-20 10:06:28, Vlastimil Babka wrote:
>> On 1/29/20 10:48 PM, Yang Shi wrote:
>>> Since commit a49bd4d71637 ("mm, numa: rework do_pages_move"),
>>> the semantic of move_pages() has changed to return the number of
>>> non-migrated pages if they were result of a non-fatal reasons (usually a
>>> busy page).  This was an unintentional change that hasn't been noticed
>>> except for LTP tests which checked for the documented behavior.
>>>
>>> There are two ways to go around this change.  We can even get back to the
>>> original behavior and return -EAGAIN whenever migrate_pages is not able
>>
>> The manpage says EBUSY, not EAGAIN? And should its description be
>> updated too?
> 
> The idea was that we _could_ return EAGAIN from the syscall if
> migrate_pages > 0.
> 
>> I.e. that it's no longer returned since 4.17?
> 
> I am pretty sure this will require a deeper consideration. Do we return
> EIO/EINVAL?

I thought the manpage says we return -EBUSY, but I misread it, this part
was not about errno, but the status array. So there's nothing to update
there, sorry about the noise.

BTW, the suggestion to "Pre-initialization of the array to -1" means
effectively it's pre-initialized to -EPERM. That's fine now as -EPERM is
not one of the codes listed as possible to be returned via the array,
but perhaps it's not entirely future-proof?


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [v2 PATCH] move_pages.2: Returning positive value is a new error case
  2020-01-30 12:56     ` Vlastimil Babka
@ 2020-01-30 13:48       ` Michal Hocko
  2020-01-30 17:27         ` Yang Shi
  0 siblings, 1 reply; 6+ messages in thread
From: Michal Hocko @ 2020-01-30 13:48 UTC (permalink / raw)
  To: Vlastimil Babka
  Cc: Yang Shi, mtk.manpages, akpm, linux-man, linux-api, linux-mm,
	linux-kernel

On Thu 30-01-20 13:56:20, Vlastimil Babka wrote:
> On 1/30/20 1:02 PM, Michal Hocko wrote:
> > On Thu 30-01-20 10:06:28, Vlastimil Babka wrote:
> >> On 1/29/20 10:48 PM, Yang Shi wrote:
> >>> Since commit a49bd4d71637 ("mm, numa: rework do_pages_move"),
> >>> the semantic of move_pages() has changed to return the number of
> >>> non-migrated pages if they were result of a non-fatal reasons (usually a
> >>> busy page).  This was an unintentional change that hasn't been noticed
> >>> except for LTP tests which checked for the documented behavior.
> >>>
> >>> There are two ways to go around this change.  We can even get back to the
> >>> original behavior and return -EAGAIN whenever migrate_pages is not able
> >>
> >> The manpage says EBUSY, not EAGAIN? And should its description be
> >> updated too?
> > 
> > The idea was that we _could_ return EAGAIN from the syscall if
> > migrate_pages > 0.
> > 
> >> I.e. that it's no longer returned since 4.17?
> > 
> > I am pretty sure this will require a deeper consideration. Do we return
> > EIO/EINVAL?
> 
> I thought the manpage says we return -EBUSY, but I misread it, this part
> was not about errno, but the status array. So there's nothing to update
> there, sorry about the noise.
> 
> BTW, the suggestion to "Pre-initialization of the array to -1" means
> effectively it's pre-initialized to -EPERM. That's fine now as -EPERM is
> not one of the codes listed as possible to be returned via the array,
> but perhaps it's not entirely future-proof?

Hmm, I didn't realize EPERM is refering to 1. The wording however
suggests also any other value that cannot represent a valid NUMA node.
So maybe we should just drop the node about -1.
-- 
Michal Hocko
SUSE Labs

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [v2 PATCH] move_pages.2: Returning positive value is a new error case
  2020-01-30 13:48       ` Michal Hocko
@ 2020-01-30 17:27         ` Yang Shi
  0 siblings, 0 replies; 6+ messages in thread
From: Yang Shi @ 2020-01-30 17:27 UTC (permalink / raw)
  To: Michal Hocko, Vlastimil Babka
  Cc: mtk.manpages, akpm, linux-man, linux-api, linux-mm, linux-kernel



On 1/30/20 5:48 AM, Michal Hocko wrote:
> On Thu 30-01-20 13:56:20, Vlastimil Babka wrote:
>> On 1/30/20 1:02 PM, Michal Hocko wrote:
>>> On Thu 30-01-20 10:06:28, Vlastimil Babka wrote:
>>>> On 1/29/20 10:48 PM, Yang Shi wrote:
>>>>> Since commit a49bd4d71637 ("mm, numa: rework do_pages_move"),
>>>>> the semantic of move_pages() has changed to return the number of
>>>>> non-migrated pages if they were result of a non-fatal reasons (usually a
>>>>> busy page).  This was an unintentional change that hasn't been noticed
>>>>> except for LTP tests which checked for the documented behavior.
>>>>>
>>>>> There are two ways to go around this change.  We can even get back to the
>>>>> original behavior and return -EAGAIN whenever migrate_pages is not able
>>>> The manpage says EBUSY, not EAGAIN? And should its description be
>>>> updated too?
>>> The idea was that we _could_ return EAGAIN from the syscall if
>>> migrate_pages > 0.
>>>
>>>> I.e. that it's no longer returned since 4.17?
>>> I am pretty sure this will require a deeper consideration. Do we return
>>> EIO/EINVAL?
>> I thought the manpage says we return -EBUSY, but I misread it, this part
>> was not about errno, but the status array. So there's nothing to update
>> there, sorry about the noise.
>>
>> BTW, the suggestion to "Pre-initialization of the array to -1" means
>> effectively it's pre-initialized to -EPERM. That's fine now as -EPERM is
>> not one of the codes listed as possible to be returned via the array,
>> but perhaps it's not entirely future-proof?
> Hmm, I didn't realize EPERM is refering to 1. The wording however
> suggests also any other value that cannot represent a valid NUMA node.
> So maybe we should just drop the node about -1.

Or maybe we just say "any value which doesn't represent a valid NUMA 
node or valid error of status array"?



^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, back to index

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-29 21:48 [v2 PATCH] move_pages.2: Returning positive value is a new error case Yang Shi
2020-01-30  9:06 ` Vlastimil Babka
2020-01-30 12:02   ` Michal Hocko
2020-01-30 12:56     ` Vlastimil Babka
2020-01-30 13:48       ` Michal Hocko
2020-01-30 17:27         ` Yang Shi

Linux-man Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-man/0 linux-man/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-man linux-man/ https://lore.kernel.org/linux-man \
		linux-man@vger.kernel.org
	public-inbox-index linux-man

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-man


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git