[PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Peng Haitao]

The function iswblank() is thread safe with exceptions.

Signed-off-by: Peng Haitao <penght-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
---
 man3/iswblank.3 | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/man3/iswblank.3 b/man3/iswblank.3
index 34126ae..2d29484 100644
--- a/man3/iswblank.3
+++ b/man3/iswblank.3
@@ -13,7 +13,7 @@
 .\"   OpenGroup's Single UNIX specification http://www.UNIX-systems.org/online.html
 .\"   ISO/IEC 9899:1999
 .\"
-.TH ISWBLANK 3  2010-09-20 "GNU" "Linux Programmer's Manual"
+.TH ISWBLANK 3  2014-02-10 "GNU" "Linux Programmer's Manual"
 .SH NAME
 iswblank \- test for whitespace wide character
 .SH SYNOPSIS
@@ -66,6 +66,14 @@ function returns nonzero
 if \fIwc\fP is a wide character
 belonging to the wide-character class "blank".
 Otherwise it returns zero.
+.SH ATTRIBUTES
+.SS Multithreading (see pthreads(7))
+The
+.BR iswblank ()
+function is thread-safe with exceptions.
+It can be safely used in multithreaded applications, as long as
+.BR setlocale (3)
+is not called to change the locale during its execution.
 .SH CONFORMING TO
 POSIX.1-2001.
 .SH NOTES
-- 
1.8.4.2

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Michael Kerrisk (man-pages)]

Thank you, Haitao.

Applied.

Cheers,

Michael


On 02/10/2014 07:20 AM, Peng Haitao wrote:
> The function iswblank() is thread safe with exceptions.
> 
> Signed-off-by: Peng Haitao <penght-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org>
> ---
>  man3/iswblank.3 | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/man3/iswblank.3 b/man3/iswblank.3
> index 34126ae..2d29484 100644
> --- a/man3/iswblank.3
> +++ b/man3/iswblank.3
> @@ -13,7 +13,7 @@
>  .\"   OpenGroup's Single UNIX specification http://www.UNIX-systems.org/online.html
>  .\"   ISO/IEC 9899:1999
>  .\"
> -.TH ISWBLANK 3  2010-09-20 "GNU" "Linux Programmer's Manual"
> +.TH ISWBLANK 3  2014-02-10 "GNU" "Linux Programmer's Manual"
>  .SH NAME
>  iswblank \- test for whitespace wide character
>  .SH SYNOPSIS
> @@ -66,6 +66,14 @@ function returns nonzero
>  if \fIwc\fP is a wide character
>  belonging to the wide-character class "blank".
>  Otherwise it returns zero.
> +.SH ATTRIBUTES
> +.SS Multithreading (see pthreads(7))
> +The
> +.BR iswblank ()
> +function is thread-safe with exceptions.
> +It can be safely used in multithreaded applications, as long as
> +.BR setlocale (3)
> +is not called to change the locale during its execution.
>  .SH CONFORMING TO
>  POSIX.1-2001.
>  .SH NOTES
> 


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Carlos O'Donell]

On Mon, Feb 10, 2014 at 1:20 AM, Peng Haitao <penght-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org> wrote:
> The function iswblank() is thread safe with exceptions.
> +.SH ATTRIBUTES
> +.SS Multithreading (see pthreads(7))
> +The
> +.BR iswblank ()
> +function is thread-safe with exceptions.
> +It can be safely used in multithreaded applications, as long as
> +.BR setlocale (3)
> +is not called to change the locale during its execution.

Now that glibc 2.19 is out the door and using a standard set of
"exception" markers should we try to standardize the same markers?

For example in glibc this exception is called "locale" and iswblank is
marked "MT-Safe locale." With "locale" being described in detail in
the safety conceptions section.

For example the description you use is not quite technically correct.
It is possible to call setlocale, but you must guard the call to
setlocale and all other functions marked with the "locale" safety note
using a mutex to serialize access. That may be prohibitively
expensive, but it's an option.

In summary:
* Suggest you look at glibc 2.19 release and the safety notes used
there to simplify the linux kernel manual page notes.
* Add a manual page that specifically talked about "Safety concepts"
including thread, signal, and cancellation. Talk about things in more
detail there.

If need be Alex can repost the text for the linux kenrel man pages
project to reuse it under an alternate license.

Comments?

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Peng Haitao]

On 02/11/2014 12:03 AM, Carlos O'Donell wrote:
> On Mon, Feb 10, 2014 at 1:20 AM, Peng Haitao <penght-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org> wrote:
>> The function iswblank() is thread safe with exceptions.
>> +.SH ATTRIBUTES
>> +.SS Multithreading (see pthreads(7))
>> +The
>> +.BR iswblank ()
>> +function is thread-safe with exceptions.
>> +It can be safely used in multithreaded applications, as long as
>> +.BR setlocale (3)
>> +is not called to change the locale during its execution.
> 
> Now that glibc 2.19 is out the door and using a standard set of
> "exception" markers should we try to standardize the same markers?
> 

Yes, maybe it is better.
But in Oracle Solaris's manpages, iswblank is marked "MT-Safe with exceptions".
I also think "MT-Safe with exceptions" is OK.

> For example in glibc this exception is called "locale" and iswblank is
> marked "MT-Safe locale." With "locale" being described in detail in
> the safety conceptions section.
> 
> For example the description you use is not quite technically correct.
> It is possible to call setlocale, but you must guard the call to
> setlocale and all other functions marked with the "locale" safety note
> using a mutex to serialize access. That may be prohibitively
> expensive, but it's an option.
> 

This is a method to avoid exceptions, but I want to tell the users
iswblank() is "MT-Safe with exceptions".


The description of "locale" in glibc 2.19 is as follows:
Functions annotated with locale called concurrently with locale changes
may behave in ways that do not correspond to any of the locales active
during their execution, but an unpredictable mix thereof.

I think the description is similar to ours.

> In summary:
> * Suggest you look at glibc 2.19 release and the safety notes used
> there to simplify the linux kernel manual page notes.
> * Add a manual page that specifically talked about "Safety concepts"
> including thread, signal, and cancellation. Talk about things in more
> detail there.
> 

There is a manual page "man-pages(7)" which talks about "Safety concepts".


-- 
Best Regards,
Peng

> If need be Alex can repost the text for the linux kenrel man pages
> project to reuse it under an alternate license.
> 
> Comments?
> 
> Cheers,
> Carlos.
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Carlos O'Donell]

On Tue, Feb 11, 2014 at 8:34 PM, Peng Haitao <penght-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org> wrote:
>
> On 02/11/2014 12:03 AM, Carlos O'Donell wrote:
>> On Mon, Feb 10, 2014 at 1:20 AM, Peng Haitao <penght-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org> wrote:
>>> The function iswblank() is thread safe with exceptions.
>>> +.SH ATTRIBUTES
>>> +.SS Multithreading (see pthreads(7))
>>> +The
>>> +.BR iswblank ()
>>> +function is thread-safe with exceptions.
>>> +It can be safely used in multithreaded applications, as long as
>>> +.BR setlocale (3)
>>> +is not called to change the locale during its execution.
>>
>> Now that glibc 2.19 is out the door and using a standard set of
>> "exception" markers should we try to standardize the same markers?
>>
>
> Yes, maybe it is better.
> But in Oracle Solaris's manpages, iswblank is marked "MT-Safe with exceptions".
> I also think "MT-Safe with exceptions" is OK.

My suggestion is:

"The function iswblank() is thread safe with exceptions (locale). See
"Safety Concepts" for more information."

Then describe the locale issue once in the "Safety Concepts" section.

This reduces duplicated text and simplifies each man page.

Users that know what "locale" means can scroll quickly, users that
don't can look it up and learn.

Does that make sense?

My further suggestion is to use the same ~20 exception notes as glibc e.g.
race, const, locale, env, hostid, sigintr, init, lock, corrupt, heap,
dlopen, plugin, i18ln, fd, mem, sig, term, and cwd (we also have
!posix to indicate it is known not to conform with posix).

The glibc manual is marked with them now, you can see iswblank here:
http://www.gnu.org/software/libc/manual/html_mono/libc.html#index-iswblank-486

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Peng Haitao]

On 02/12/2014 11:31 PM, Carlos O'Donell wrote:
>>>> +The
>>>> +.BR iswblank ()
>>>> +function is thread-safe with exceptions.
>>>> +It can be safely used in multithreaded applications, as long as
>>>> +.BR setlocale (3)
>>>> +is not called to change the locale during its execution.
>>>
>>> Now that glibc 2.19 is out the door and using a standard set of
>>> "exception" markers should we try to standardize the same markers?
>>>
>>
>> Yes, maybe it is better.
>> But in Oracle Solaris's manpages, iswblank is marked "MT-Safe with exceptions".
>> I also think "MT-Safe with exceptions" is OK.
> 
> My suggestion is:
> 
> "The function iswblank() is thread safe with exceptions (locale). See
> "Safety Concepts" for more information."
> 
> Then describe the locale issue once in the "Safety Concepts" section.
> 
> This reduces duplicated text and simplifies each man page.
> 
> Users that know what "locale" means can scroll quickly, users that
> don't can look it up and learn.
> 
> Does that make sense?
> 

Your suggestion can simplify each man page, but our description make each man page clearer.

In Oracle Solaris's manpages and POSIX, when the function has exceptions,
exceptions are described in this function.

In Oracle Solaris's manpages, exceptions are described as follows:
MT-Safe with Exceptions
    See the NOTES or USAGE sections of these pages for a description of the exceptions.

Safe with Exceptions
    See the NOTES or USAGE sections of these pages for a description of the exceptions.

In POSIX, when the function has exceptions, exceptions are described in this function as follows:
The wcrtomb() function need not be thread-safe if called with a NULL ps argument.

The tmpnam() function need not be thread-safe if called with a NULL parameter. 


I don't have strong preference. If Michael and other members prefer
your suggestion, I can modify my patches.


-- 
Best Regards,
Peng

> My further suggestion is to use the same ~20 exception notes as glibc e.g.
> race, const, locale, env, hostid, sigintr, init, lock, corrupt, heap,
> dlopen, plugin, i18ln, fd, mem, sig, term, and cwd (we also have
> !posix to indicate it is known not to conform with posix).
> 
> The glibc manual is marked with them now, you can see iswblank here:
> http://www.gnu.org/software/libc/manual/html_mono/libc.html#index-iswblank-486
> 
> Cheers,
> Carlos.
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Carlos O'Donell]

On Wed, Feb 12, 2014 at 11:07 PM, Peng Haitao <penght-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org> wrote:
>> My suggestion is:
>>
>> "The function iswblank() is thread safe with exceptions (locale). See
>> "Safety Concepts" for more information."
>>
>> Then describe the locale issue once in the "Safety Concepts" section.
>>
>> This reduces duplicated text and simplifies each man page.
>>
>> Users that know what "locale" means can scroll quickly, users that
>> don't can look it up and learn.
>>
>> Does that make sense?
>>
>
> Your suggestion can simplify each man page, but our description make each man page clearer.

There is a natural tension here between being descriptive and being succinct.

> In Oracle Solaris's manpages and POSIX, when the function has exceptions,
> exceptions are described in this function.

We should strive to be better than that.

> In POSIX, when the function has exceptions, exceptions are described in this function as follows:
> The wcrtomb() function need not be thread-safe if called with a NULL ps argument.
>
> The tmpnam() function need not be thread-safe if called with a NULL parameter.
>
> I don't have strong preference. If Michael and other members prefer
> your suggestion, I can modify my patches.

I have a strong preference for harmonizing this information across the
man pages and the glibc manual.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Michael Kerrisk (man-pages)]

On 02/13/2014 05:48 AM, Carlos O'Donell wrote:
> On Wed, Feb 12, 2014 at 11:07 PM, Peng Haitao <penght-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org> wrote:
>>> My suggestion is:
>>>
>>> "The function iswblank() is thread safe with exceptions (locale). See
>>> "Safety Concepts" for more information."
>>>
>>> Then describe the locale issue once in the "Safety Concepts" section.
>>>
>>> This reduces duplicated text and simplifies each man page.
>>>
>>> Users that know what "locale" means can scroll quickly, users that
>>> don't can look it up and learn.
>>>
>>> Does that make sense?
>>>
>>
>> Your suggestion can simplify each man page, but our description make each man page clearer.
> 
> There is a natural tension here between being descriptive and being succinct.

(Yup.)

>> In Oracle Solaris's manpages and POSIX, when the function has exceptions,
>> exceptions are described in this function.
> 
> We should strive to be better than that.

(Yup ;-).)

>> In POSIX, when the function has exceptions, exceptions are described in this function as follows:
>> The wcrtomb() function need not be thread-safe if called with a NULL ps argument.
>>
>> The tmpnam() function need not be thread-safe if called with a NULL parameter.
>>
>> I don't have strong preference. If Michael and other members prefer
>> your suggestion, I can modify my patches.
> 
> I have a strong preference for harmonizing this information across the
> man pages and the glibc manual.

Sorry for being late to this thread. Carlos, I agree with you. I will take a bit 
of time now to look more closely at the glibc markings, and follow up. Thanks for
raising this.

Cheers,

Michael



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Michael Kerrisk (man-pages)]

Alexandre, Carlos,

On Wed, Feb 12, 2014 at 4:31 PM, Carlos O'Donell
<carlos-v2tUB8YBRSi3e3T8WW9gsA@public.gmane.org> wrote:
> On Tue, Feb 11, 2014 at 8:34 PM, Peng Haitao <penght-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org> wrote:
>>
>> On 02/11/2014 12:03 AM, Carlos O'Donell wrote:
>>> On Mon, Feb 10, 2014 at 1:20 AM, Peng Haitao <penght-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org> wrote:
>>>> The function iswblank() is thread safe with exceptions.
>>>> +.SH ATTRIBUTES
>>>> +.SS Multithreading (see pthreads(7))
>>>> +The
>>>> +.BR iswblank ()
>>>> +function is thread-safe with exceptions.
>>>> +It can be safely used in multithreaded applications, as long as
>>>> +.BR setlocale (3)
>>>> +is not called to change the locale during its execution.
>>>
>>> Now that glibc 2.19 is out the door and using a standard set of
>>> "exception" markers should we try to standardize the same markers?
>>>
>>
>> Yes, maybe it is better.
>> But in Oracle Solaris's manpages, iswblank is marked "MT-Safe with exceptions".
>> I also think "MT-Safe with exceptions" is OK.
>
> My suggestion is:
>
> "The function iswblank() is thread safe with exceptions (locale). See
> "Safety Concepts" for more information."
>
> Then describe the locale issue once in the "Safety Concepts" section.
>
> This reduces duplicated text and simplifies each man page.
>
> Users that know what "locale" means can scroll quickly, users that
> don't can look it up and learn.
>
> Does that make sense?
>
> My further suggestion is to use the same ~20 exception notes as glibc e.g.
> race, const, locale, env, hostid, sigintr, init, lock, corrupt, heap,
> dlopen, plugin, i18ln, fd, mem, sig, term, and cwd (we also have
> !posix to indicate it is known not to conform with posix).
>
> The glibc manual is marked with them now, you can see iswblank here:
> http://www.gnu.org/software/libc/manual/html_mono/libc.html#index-iswblank-486

I realize I could did through a lot of archived mail and answer these
questions, but I hope you can save me some time.

1. How was the information about MT-Safety etc derived. I'm assuming a
lot of arduous manual inspection, right? (Or was there some automation
involved?)

2. Does the glibc manual document the actual or the desired state of
the functions wrt to MT-Safety? Where these differ, does the manual
document that?

3. Were the glibc manual changes generated manually, or was there some
degree of automation involved? I.e., has some markup been added to the
source that allows one to generate a summary of information about
MT-safety? As far as I can tell, that is not the case, but I wanted to
check.

The motivation of the above questions is of course that I wonder what
degree of automation might be achievable in making changes to the
documentation in man-pages.

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Carlos O'Donell]

On Sun, Feb 23, 2014 at 5:16 AM, Michael Kerrisk (man-pages)
<mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> The glibc manual is marked with them now, you can see iswblank here:
>> http://www.gnu.org/software/libc/manual/html_mono/libc.html#index-iswblank-486
>
> I realize I could did through a lot of archived mail and answer these
> questions, but I hope you can save me some time.

My pleasure.

> 1. How was the information about MT-Safety etc derived. I'm assuming a
> lot of arduous manual inspection, right? (Or was there some automation
> involved?)

No automation. Manual inspection. It took Alexandre roughly a year to
get through ~1000+ functions. After a while though the previous safety
notes help you out immensely as you don't need to recurse deeply in a
function when you know the properties of the functions you are
calling.

> 2. Does the glibc manual document the actual or the desired state of
> the functions wrt to MT-Safety? Where these differ, does the manual
> document that?

The noes are preliminary so we make no promises.

We document the actual state (not perfectly true, but it's the
statement we want to make).

We do not presently indicate what is the desired state.

Some interfaces deviate from the standards, and in some cases we have
started marking that e.g. !posix.

However, we need to do a second pass to ensure that:

(a) We transition from preliminary to full guarantees about the API.

-- When we do this transition we will look at what the desired
guarantee is for each function.

For example we may *never* guarantee setlocale is MT-safe, simply
because the consequence of making it safe slows down every other
function that uses locales.

(b) We document the standard requirement for reference.

(c) We add texinfo comments when we have notes to describe why we
deviate from the standard.

> 3. Were the glibc manual changes generated manually, or was there some
> degree of automation involved? I.e., has some markup been added to the
> source that allows one to generate a summary of information about
> MT-safety? As far as I can tell, that is not the case, but I wanted to
> check.

The glibc manual changes were generated manually.

However, the notes per function are not text, but instead texinfo macros.

The macros are then turned into the text you see.

Therefore we have a meta-language in texinfo macros to describe the
safety notes for each function.

We have a regression test to ensure these notes follow the rules we
set out for them e.g. if you mark something unsafe you note why using
a valid marker for that safety note.

Markup has not been added to the source to generate a summary of
information about MT-safety.

This is actually a GSoC project for glibc:
https://sourceware.org/glibc/wiki/GSoC

See "Dynamic Documentation"

Fundamentally we can't move source code documentation into the manual
because of the licensing barrier between GFDL and LGPLv2, *however*
since gcc has the same problem with C++ we do have a known process for
doing this easily (FSF maintainer given the right to relicense the
generated docs and check them in).

I also question the validity of copyrighting some of this information.
For example I consider that the safety note information isn't
copyrightable because it's a property of the function. Descriptive
text about why we don't follow the standard would be copyrightable
though.

> The motivation of the above questions is of course that I wonder what
> degree of automation might be achievable in making changes to the
> documentation in man-pages.

We considered this initially and we want to go in that direction, but
for this initial pass we determined it would complicate the project
and decided to do a manual pass.

If you can suggest any technology we might wish to use, we could
refine our GSoC project?

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Peng Haitao]

Hi Michael,

What is the status of this thread now?

> If need be Alex can repost the text for the linux kenrel man pages
> project to reuse it under an alternate license.

Maybe we need Alex can repost "Safety Concepts" section for the man-pages?

Thanks.


-- 
Best Regards,
Peng

On 02/26/2014 01:28 AM, Carlos O'Donell wrote:
> On Sun, Feb 23, 2014 at 5:16 AM, Michael Kerrisk (man-pages)
> <mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>> The glibc manual is marked with them now, you can see iswblank here:
>>> http://www.gnu.org/software/libc/manual/html_mono/libc.html#index-iswblank-486
>>
>> I realize I could did through a lot of archived mail and answer these
>> questions, but I hope you can save me some time.
> 
> My pleasure.
> 
>> 1. How was the information about MT-Safety etc derived. I'm assuming a
>> lot of arduous manual inspection, right? (Or was there some automation
>> involved?)
> 
> No automation. Manual inspection. It took Alexandre roughly a year to
> get through ~1000+ functions. After a while though the previous safety
> notes help you out immensely as you don't need to recurse deeply in a
> function when you know the properties of the functions you are
> calling.
> 
>> 2. Does the glibc manual document the actual or the desired state of
>> the functions wrt to MT-Safety? Where these differ, does the manual
>> document that?
> 
> The noes are preliminary so we make no promises.
> 
> We document the actual state (not perfectly true, but it's the
> statement we want to make).
> 
> We do not presently indicate what is the desired state.
> 
> Some interfaces deviate from the standards, and in some cases we have
> started marking that e.g. !posix.
> 
> However, we need to do a second pass to ensure that:
> 
> (a) We transition from preliminary to full guarantees about the API.
> 
> -- When we do this transition we will look at what the desired
> guarantee is for each function.
> 
> For example we may *never* guarantee setlocale is MT-safe, simply
> because the consequence of making it safe slows down every other
> function that uses locales.
> 
> (b) We document the standard requirement for reference.
> 
> (c) We add texinfo comments when we have notes to describe why we
> deviate from the standard.
> 
>> 3. Were the glibc manual changes generated manually, or was there some
>> degree of automation involved? I.e., has some markup been added to the
>> source that allows one to generate a summary of information about
>> MT-safety? As far as I can tell, that is not the case, but I wanted to
>> check.
> 
> The glibc manual changes were generated manually.
> 
> However, the notes per function are not text, but instead texinfo macros.
> 
> The macros are then turned into the text you see.
> 
> Therefore we have a meta-language in texinfo macros to describe the
> safety notes for each function.
> 
> We have a regression test to ensure these notes follow the rules we
> set out for them e.g. if you mark something unsafe you note why using
> a valid marker for that safety note.
> 
> Markup has not been added to the source to generate a summary of
> information about MT-safety.
> 
> This is actually a GSoC project for glibc:
> https://sourceware.org/glibc/wiki/GSoC
> 
> See "Dynamic Documentation"
> 
> Fundamentally we can't move source code documentation into the manual
> because of the licensing barrier between GFDL and LGPLv2, *however*
> since gcc has the same problem with C++ we do have a known process for
> doing this easily (FSF maintainer given the right to relicense the
> generated docs and check them in).
> 
> I also question the validity of copyrighting some of this information.
> For example I consider that the safety note information isn't
> copyrightable because it's a property of the function. Descriptive
> text about why we don't follow the standard would be copyrightable
> though.
> 
>> The motivation of the above questions is of course that I wonder what
>> degree of automation might be achievable in making changes to the
>> documentation in man-pages.
> 
> We considered this initially and we want to go in that direction, but
> for this initial pass we determined it would complicate the project
> and decided to do a manual pass.
> 
> If you can suggest any technology we might wish to use, we could
> refine our GSoC project?
> 
> Cheers,
> Carlos.
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Michael Kerrisk (man-pages)]

Hi Carlos, Haitao, Qian Lei, Alex,

Following up on this at last. Sorry for the delay.

On Tue, Feb 25, 2014 at 6:28 PM, Carlos O'Donell
<carlos-v2tUB8YBRSi3e3T8WW9gsA@public.gmane.org> wrote:
> On Sun, Feb 23, 2014 at 5:16 AM, Michael Kerrisk (man-pages)
> <mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>> The glibc manual is marked with them now, you can see iswblank here:
>>> http://www.gnu.org/software/libc/manual/html_mono/libc.html#index-iswblank-486
>>
>> I realize I could did through a lot of archived mail and answer these
>> questions, but I hope you can save me some time.
>
> My pleasure.
>
>> 1. How was the information about MT-Safety etc derived. I'm assuming a
>> lot of arduous manual inspection, right? (Or was there some automation
>> involved?)
>
> No automation. Manual inspection. It took Alexandre roughly a year to
> get through ~1000+ functions. After a while though the previous safety
> notes help you out immensely as you don't need to recurse deeply in a
> function when you know the properties of the functions you are
> calling.
>
>> 2. Does the glibc manual document the actual or the desired state of
>> the functions wrt to MT-Safety? Where these differ, does the manual
>> document that?
>
> The noes are preliminary so we make no promises.
>
> We document the actual state (not perfectly true, but it's the
> statement we want to make).
>
> We do not presently indicate what is the desired state.

Okay.

> Some interfaces deviate from the standards, and in some cases we have
> started marking that e.g. !posix.
>
> However, we need to do a second pass to ensure that:
>
> (a) We transition from preliminary to full guarantees about the API.
>
> -- When we do this transition we will look at what the desired
> guarantee is for each function.
>
> For example we may *never* guarantee setlocale is MT-safe, simply
> because the consequence of making it safe slows down every other
> function that uses locales.
>
> (b) We document the standard requirement for reference.
>
> (c) We add texinfo comments when we have notes to describe why we
> deviate from the standard.

Okay.

Is there a schedule/plan to bring these notes to something more definitive?

>> 3. Were the glibc manual changes generated manually, or was there some
>> degree of automation involved? I.e., has some markup been added to the
>> source that allows one to generate a summary of information about
>> MT-safety? As far as I can tell, that is not the case, but I wanted to
>> check.
>
> The glibc manual changes were generated manually.
>
> However, the notes per function are not text, but instead texinfo macros.
>
> The macros are then turned into the text you see.
>
> Therefore we have a meta-language in texinfo macros to describe the
> safety notes for each function.
>
> We have a regression test to ensure these notes follow the rules we
> set out for them e.g. if you mark something unsafe you note why using
> a valid marker for that safety note.
>
> Markup has not been added to the source to generate a summary of
> information about MT-safety.
>
> This is actually a GSoC project for glibc:
> https://sourceware.org/glibc/wiki/GSoC
>
> See "Dynamic Documentation"
>
> Fundamentally we can't move source code documentation into the manual
> because of the licensing barrier between GFDL and LGPLv2, *however*
> since gcc has the same problem with C++ we do have a known process for
> doing this easily (FSF maintainer given the right to relicense the
> generated docs and check them in).
>
> I also question the validity of copyrighting some of this information.
> For example I consider that the safety note information isn't
> copyrightable because it's a property of the function. Descriptive
> text about why we don't follow the standard would be copyrightable
> though.
>
>> The motivation of the above questions is of course that I wonder what
>> degree of automation might be achievable in making changes to the
>> documentation in man-pages.
>
> We considered this initially and we want to go in that direction, but
> for this initial pass we determined it would complicate the project
> and decided to do a manual pass.
>
> If you can suggest any technology we might wish to use, we could
> refine our GSoC project?

Okay. I've looked in more detail at the explanations on the glibc
manual nodes linked off
http://www.gnu.org/software/libc/manual/html_node/POSIX.html#POSIX
The approach looks good to me, and I think there is value in in the
man-pages and and the glibc manual using consistent annotations.

Practically speaking, I think this means:

1. We'd create an attributes(7) man page that has much of the same
information as in the glibc manual nodes listed above. If needed we
can rewrite to avoid any licensing issues. On the other hand, if
Alex(?) wants to make that text available under a suitable license
(e.g., one of those at
https://www.kernel.org/doc/man-pages/licenses.html). (Note, GFDL
doesn't work for man-pages, as explained on that web page.)

2. We redesign the ATTRIBUTES section that we use in man-pages. I
think it would make sense to use something like the Solaris
attributes(5) approach, where there is a table of safety information.
That table would be supplemented by notes, as needed. Thus, we might
have something like:

    ATTRIBUTES

        See attributes(7).

        +-----------+---------------+-----------+
        | Interface | Attribute     | Value     |
        +-----------+---------------+-----------+
        | l64a()    | Thread-safety | MT-Unsafe |
        +-----------+---------------+-----------+

        +-------------------+---------------+---------+
       | Interface         | Attribute     | Value   |
       +-------------------+---------------+---------+
        | bindresvport()    | Thread-safety | MT-Safe |
        +-------------------+---------------+---------+

        Before glibc 2.17, the bindresvport() function uses a static variable
        that is not protected, so it is not thread-safe.
        Since glibc 2.17, it protects the static variable with a lock,
        and is thus MT-Safe.

        +----------------------------+---------------+----------------+
        | Interface                  | Attribute     | Value          |
        +----------------------------+---------------+----------------+
        | atoi(), atol(), atoll()    | Thread-safety | MT-Safe locale |
        +----------------------------+---------------+----------------+

        +--------------------------+---------------------+---------+
        | Interface                | Attribute           | Value   |
        +--------------------------+---------------------+---------+
        | pthread_setcancelstate() | Async-cancel-safety | AC-Safe |
        +--------------------------+---------------------+---------+

An alternative approach would be to have subsections under ATTRIBUES,
with separate tables for Thread-safety, Async-signal-safety, and
Async-cancel-safety. Something like this:

    ATTRIBUTES

            See attributes(7).

        Thread-safety
            +-------------------+---------+
            | Interface         | Value   |
            +-------------------+---------+
            | bindresvport()    | MT-Safe |
            +-------------------+---------+

            Before glibc 2.17, the bindresvport() function uses a static
            variable that is not protected, so it is not thread-safe.
            Since glibc 2.17, it protects the static variable with a lock,
            and is thus MT-Safe.

        Async-cancel-safety

            +--------------------------+---------+
            | Interface                | Value   |
            +--------------------------+---------+
            | pthread_setcancelstate() | AC-Safe |
            +--------------------------+---------+

Does one of those approaches eem okay? Haitao, Qian Lei, what do you
think? (I appreciate that there's a lot of pages to be fixed to follow
this new approach: I'd do that, though you could help if you want.)

Cheers,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Peng Haitao]

On 06/11/2014 04:30 PM, Michael Kerrisk (man-pages) wrote:
...
> Practically speaking, I think this means:
> 
> 1. We'd create an attributes(7) man page that has much of the same
> information as in the glibc manual nodes listed above. If needed we
> can rewrite to avoid any licensing issues. On the other hand, if
> Alex(?) wants to make that text available under a suitable license
> (e.g., one of those at
> https://www.kernel.org/doc/man-pages/licenses.html). (Note, GFDL
> doesn't work for man-pages, as explained on that web page.)
> 
> 2. We redesign the ATTRIBUTES section that we use in man-pages. I
> think it would make sense to use something like the Solaris
> attributes(5) approach, where there is a table of safety information.
> That table would be supplemented by notes, as needed. Thus, we might
> have something like:
> 
>     ATTRIBUTES
> 
>         See attributes(7).
> 
>         +-----------+---------------+-----------+
>         | Interface | Attribute     | Value     |
>         +-----------+---------------+-----------+
>         | l64a()    | Thread-safety | MT-Unsafe |
>         +-----------+---------------+-----------+
> 
>         +-------------------+---------------+---------+
>        | Interface         | Attribute     | Value   |
>        +-------------------+---------------+---------+
>         | bindresvport()    | Thread-safety | MT-Safe |
>         +-------------------+---------------+---------+
> 
>         Before glibc 2.17, the bindresvport() function uses a static variable
>         that is not protected, so it is not thread-safe.
>         Since glibc 2.17, it protects the static variable with a lock,
>         and is thus MT-Safe.
> 
>         +----------------------------+---------------+----------------+
>         | Interface                  | Attribute     | Value          |
>         +----------------------------+---------------+----------------+
>         | atoi(), atol(), atoll()    | Thread-safety | MT-Safe locale |
>         +----------------------------+---------------+----------------+
> 
>         +--------------------------+---------------------+---------+
>         | Interface                | Attribute           | Value   |
>         +--------------------------+---------------------+---------+
>         | pthread_setcancelstate() | Async-cancel-safety | AC-Safe |
>         +--------------------------+---------------------+---------+
> 
> An alternative approach would be to have subsections under ATTRIBUES,
> with separate tables for Thread-safety, Async-signal-safety, and
> Async-cancel-safety. Something like this:
> 
>     ATTRIBUTES
> 
>             See attributes(7).
> 
>         Thread-safety
>             +-------------------+---------+
>             | Interface         | Value   |
>             +-------------------+---------+
>             | bindresvport()    | MT-Safe |
>             +-------------------+---------+
> 
>             Before glibc 2.17, the bindresvport() function uses a static
>             variable that is not protected, so it is not thread-safe.
>             Since glibc 2.17, it protects the static variable with a lock,
>             and is thus MT-Safe.
> 
>         Async-cancel-safety
> 
>             +--------------------------+---------+
>             | Interface                | Value   |
>             +--------------------------+---------+
>             | pthread_setcancelstate() | AC-Safe |
>             +--------------------------+---------+
> 
> Does one of those approaches eem okay? Haitao, Qian Lei, what do you
> think? (I appreciate that there's a lot of pages to be fixed to follow
> this new approach: I'd do that, though you could help if you want.)
> 

I select the first approach.
If the approach is decided，Qian and I will help to fix to follow the new approach.
Thanks.


-- 
Best Regards,
Peng

> Cheers,
> 
> Michael
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Carlos O'Donell]

On Wed, Jun 11, 2014 at 4:30 AM, Michael Kerrisk (man-pages)
<mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Hi Carlos, Haitao, Qian Lei, Alex,
>
> Following up on this at last. Sorry for the delay.

I also meant to respond, thanks for taking the initiative.

> On Tue, Feb 25, 2014 at 6:28 PM, Carlos O'Donell
> <carlos-v2tUB8YBRSi3e3T8WW9gsA@public.gmane.org> wrote:
>> On Sun, Feb 23, 2014 at 5:16 AM, Michael Kerrisk (man-pages)
>> <mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>>> The glibc manual is marked with them now, you can see iswblank here:
>>>> http://www.gnu.org/software/libc/manual/html_mono/libc.html#index-iswblank-486
>>>
>>> I realize I could did through a lot of archived mail and answer these
>>> questions, but I hope you can save me some time.
>>
>> My pleasure.
>>
>>> 1. How was the information about MT-Safety etc derived. I'm assuming a
>>> lot of arduous manual inspection, right? (Or was there some automation
>>> involved?)
>>
>> No automation. Manual inspection. It took Alexandre roughly a year to
>> get through ~1000+ functions. After a while though the previous safety
>> notes help you out immensely as you don't need to recurse deeply in a
>> function when you know the properties of the functions you are
>> calling.
>>
>>> 2. Does the glibc manual document the actual or the desired state of
>>> the functions wrt to MT-Safety? Where these differ, does the manual
>>> document that?
>>
>> The noes are preliminary so we make no promises.
>>
>> We document the actual state (not perfectly true, but it's the
>> statement we want to make).
>>
>> We do not presently indicate what is the desired state.
>
> Okay.
>
>> Some interfaces deviate from the standards, and in some cases we have
>> started marking that e.g. !posix.
>>
>> However, we need to do a second pass to ensure that:
>>
>> (a) We transition from preliminary to full guarantees about the API.
>>
>> -- When we do this transition we will look at what the desired
>> guarantee is for each function.
>>
>> For example we may *never* guarantee setlocale is MT-safe, simply
>> because the consequence of making it safe slows down every other
>> function that uses locales.
>>
>> (b) We document the standard requirement for reference.
>>
>> (c) We add texinfo comments when we have notes to describe why we
>> deviate from the standard.
>
> Okay.
>
> Is there a schedule/plan to bring these notes to something more definitive?

At present there is no schedule to make things more definitive.

My rough timeline is to block out time for this work in 2016 (as 2015
is planned out).

I know it sounds like a long time, but it's serious work to review
these functions and even more serious work to get upstream to commit
to them in their present form. Obviously some functions will be easy
to mark, while others harder.

>>> 3. Were the glibc manual changes generated manually, or was there some
>>> degree of automation involved? I.e., has some markup been added to the
>>> source that allows one to generate a summary of information about
>>> MT-safety? As far as I can tell, that is not the case, but I wanted to
>>> check.
>>
>> The glibc manual changes were generated manually.
>>
>> However, the notes per function are not text, but instead texinfo macros.
>>
>> The macros are then turned into the text you see.
>>
>> Therefore we have a meta-language in texinfo macros to describe the
>> safety notes for each function.
>>
>> We have a regression test to ensure these notes follow the rules we
>> set out for them e.g. if you mark something unsafe you note why using
>> a valid marker for that safety note.
>>
>> Markup has not been added to the source to generate a summary of
>> information about MT-safety.
>>
>> This is actually a GSoC project for glibc:
>> https://sourceware.org/glibc/wiki/GSoC
>>
>> See "Dynamic Documentation"
>>
>> Fundamentally we can't move source code documentation into the manual
>> because of the licensing barrier between GFDL and LGPLv2, *however*
>> since gcc has the same problem with C++ we do have a known process for
>> doing this easily (FSF maintainer given the right to relicense the
>> generated docs and check them in).
>>
>> I also question the validity of copyrighting some of this information.
>> For example I consider that the safety note information isn't
>> copyrightable because it's a property of the function. Descriptive
>> text about why we don't follow the standard would be copyrightable
>> though.
>>
>>> The motivation of the above questions is of course that I wonder what
>>> degree of automation might be achievable in making changes to the
>>> documentation in man-pages.
>>
>> We considered this initially and we want to go in that direction, but
>> for this initial pass we determined it would complicate the project
>> and decided to do a manual pass.
>>
>> If you can suggest any technology we might wish to use, we could
>> refine our GSoC project?
>
> Okay. I've looked in more detail at the explanations on the glibc
> manual nodes linked off
> http://www.gnu.org/software/libc/manual/html_node/POSIX.html#POSIX
> The approach looks good to me, and I think there is value in in the
> man-pages and and the glibc manual using consistent annotations.

Thank you for looking over what we've written in glibc.

I agree, we really want to use consistent terminology across all of
the documentation for the core runtimes.

> Practically speaking, I think this means:
>
> 1. We'd create an attributes(7) man page that has much of the same
> information as in the glibc manual nodes listed above. If needed we
> can rewrite to avoid any licensing issues. On the other hand, if
> Alex(?) wants to make that text available under a suitable license
> (e.g., one of those at
> https://www.kernel.org/doc/man-pages/licenses.html). (Note, GFDL
> doesn't work for man-pages, as explained on that web page.)

Would it be OK if Alex just posted a dump of his text (he's and I are
the authors of all the text) with an alternate license such that you
can just copy-edit from that?

> 2. We redesign the ATTRIBUTES section that we use in man-pages. I
> think it would make sense to use something like the Solaris
> attributes(5) approach, where there is a table of safety information.
> That table would be supplemented by notes, as needed. Thus, we might
> have something like:
>
>     ATTRIBUTES
>
>         See attributes(7).
>
>         +-----------+---------------+-----------+
>         | Interface | Attribute     | Value     |
>         +-----------+---------------+-----------+
>         | l64a()    | Thread-safety | MT-Unsafe |
>         +-----------+---------------+-----------+
>
>         +-------------------+---------------+---------+
>        | Interface         | Attribute     | Value   |
>        +-------------------+---------------+---------+
>         | bindresvport()    | Thread-safety | MT-Safe |
>         +-------------------+---------------+---------+

Looks awesome. I agree Solaris had a good layout.

>         Before glibc 2.17, the bindresvport() function uses a static variable
>         that is not protected, so it is not thread-safe.
>         Since glibc 2.17, it protects the static variable with a lock,
>         and is thus MT-Safe.
>
>         +----------------------------+---------------+----------------+
>         | Interface                  | Attribute     | Value          |
>         +----------------------------+---------------+----------------+
>         | atoi(), atol(), atoll()    | Thread-safety | MT-Safe locale |
>         +----------------------------+---------------+----------------+

Looks good. Where `locale' means the agreed upon definition in the
attributes(7) section.

>         +--------------------------+---------------------+---------+
>         | Interface                | Attribute           | Value   |
>         +--------------------------+---------------------+---------+
>         | pthread_setcancelstate() | Async-cancel-safety | AC-Safe |
>         +--------------------------+---------------------+---------+

Looks good to me.

> An alternative approach would be to have subsections under ATTRIBUES,
> with separate tables for Thread-safety, Async-signal-safety, and
> Async-cancel-safety. Something like this:
>
>     ATTRIBUTES
>
>             See attributes(7).
>
>         Thread-safety
>             +-------------------+---------+
>             | Interface         | Value   |
>             +-------------------+---------+
>             | bindresvport()    | MT-Safe |
>             +-------------------+---------+
>
>             Before glibc 2.17, the bindresvport() function uses a static
>             variable that is not protected, so it is not thread-safe.
>             Since glibc 2.17, it protects the static variable with a lock,
>             and is thus MT-Safe.
>
>         Async-cancel-safety
>
>             +--------------------------+---------+
>             | Interface                | Value   |
>             +--------------------------+---------+
>             | pthread_setcancelstate() | AC-Safe |
>             +--------------------------+---------+
>
> Does one of those approaches eem okay? Haitao, Qian Lei, what do you
> think? (I appreciate that there's a lot of pages to be fixed to follow
> this new approach: I'd do that, though you could help if you want.)

I agree with Peng, I like the first approach of a single table.

We should also evaluate the approach of adding the glibc version to
the table if and when the version change would change the result e.g.
the table lists the historical values.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Michael Kerrisk (man-pages)]

Hi Carlos,

On Wed, Jun 11, 2014 at 4:59 PM, Carlos O'Donell
<carlos-v2tUB8YBRSi3e3T8WW9gsA@public.gmane.org> wrote:
> On Wed, Jun 11, 2014 at 4:30 AM, Michael Kerrisk (man-pages)
> <mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> Hi Carlos, Haitao, Qian Lei, Alex,
>>
>> Following up on this at last. Sorry for the delay.
>
> I also meant to respond, thanks for taking the initiative.
>
>> On Tue, Feb 25, 2014 at 6:28 PM, Carlos O'Donell
>> <carlos-v2tUB8YBRSi3e3T8WW9gsA@public.gmane.org> wrote:
>>> On Sun, Feb 23, 2014 at 5:16 AM, Michael Kerrisk (man-pages)
>>> <mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>>>> The glibc manual is marked with them now, you can see iswblank here:
>>>>> http://www.gnu.org/software/libc/manual/html_mono/libc.html#index-iswblank-486
>>>>
>>>> I realize I could did through a lot of archived mail and answer these
>>>> questions, but I hope you can save me some time.
>>>
>>> My pleasure.
>>>
>>>> 1. How was the information about MT-Safety etc derived. I'm assuming a
>>>> lot of arduous manual inspection, right? (Or was there some automation
>>>> involved?)
>>>
>>> No automation. Manual inspection. It took Alexandre roughly a year to
>>> get through ~1000+ functions. After a while though the previous safety
>>> notes help you out immensely as you don't need to recurse deeply in a
>>> function when you know the properties of the functions you are
>>> calling.
>>>
>>>> 2. Does the glibc manual document the actual or the desired state of
>>>> the functions wrt to MT-Safety? Where these differ, does the manual
>>>> document that?
>>>
>>> The noes are preliminary so we make no promises.
>>>
>>> We document the actual state (not perfectly true, but it's the
>>> statement we want to make).
>>>
>>> We do not presently indicate what is the desired state.
>>
>> Okay.
>>
>>> Some interfaces deviate from the standards, and in some cases we have
>>> started marking that e.g. !posix.
>>>
>>> However, we need to do a second pass to ensure that:
>>>
>>> (a) We transition from preliminary to full guarantees about the API.
>>>
>>> -- When we do this transition we will look at what the desired
>>> guarantee is for each function.
>>>
>>> For example we may *never* guarantee setlocale is MT-safe, simply
>>> because the consequence of making it safe slows down every other
>>> function that uses locales.
>>>
>>> (b) We document the standard requirement for reference.
>>>
>>> (c) We add texinfo comments when we have notes to describe why we
>>> deviate from the standard.
>>
>> Okay.
>>
>> Is there a schedule/plan to bring these notes to something more definitive?
>
> At present there is no schedule to make things more definitive.
>
> My rough timeline is to block out time for this work in 2016 (as 2015
> is planned out).
>
> I know it sounds like a long time, but it's serious work to review
> these functions and even more serious work to get upstream to commit
> to them in their present form. Obviously some functions will be easy
> to mark, while others harder.

The timeframe is easily understandable to me, actually. It's a big job.

[...]

>> Practically speaking, I think this means:
>>
>> 1. We'd create an attributes(7) man page that has much of the same
>> information as in the glibc manual nodes listed above. If needed we
>> can rewrite to avoid any licensing issues. On the other hand, if
>> Alex(?) wants to make that text available under a suitable license
>> (e.g., one of those at
>> https://www.kernel.org/doc/man-pages/licenses.html). (Note, GFDL
>> doesn't work for man-pages, as explained on that web page.)
>
> Would it be OK if Alex just posted a dump of his text (he's and I are
> the authors of all the text) with an alternate license such that you
> can just copy-edit from that?

That'd be perfect. My preference is the verbatim license, but any of
the other licenses at
https://www.kernel.org/doc/man-pages/licenses.html would also work
fine.

>> 2. We redesign the ATTRIBUTES section that we use in man-pages. I
>> think it would make sense to use something like the Solaris
>> attributes(5) approach, where there is a table of safety information.
>> That table would be supplemented by notes, as needed. Thus, we might
>> have something like:
>>
>>     ATTRIBUTES
>>
>>         See attributes(7).
>>
>>         +-----------+---------------+-----------+
>>         | Interface | Attribute     | Value     |
>>         +-----------+---------------+-----------+
>>         | l64a()    | Thread-safety | MT-Unsafe |
>>         +-----------+---------------+-----------+
>>
>>         +-------------------+---------------+---------+
>>        | Interface         | Attribute     | Value   |
>>        +-------------------+---------------+---------+
>>         | bindresvport()    | Thread-safety | MT-Safe |
>>         +-------------------+---------------+---------+
>
> Looks awesome. I agree Solaris had a good layout.

Okay.

>>         Before glibc 2.17, the bindresvport() function uses a static variable
>>         that is not protected, so it is not thread-safe.
>>         Since glibc 2.17, it protects the static variable with a lock,
>>         and is thus MT-Safe.
>>
>>         +----------------------------+---------------+----------------+
>>         | Interface                  | Attribute     | Value          |
>>         +----------------------------+---------------+----------------+
>>         | atoi(), atol(), atoll()    | Thread-safety | MT-Safe locale |
>>         +----------------------------+---------------+----------------+
>
> Looks good. Where `locale' means the agreed upon definition in the
> attributes(7) section.
>
>>         +--------------------------+---------------------+---------+
>>         | Interface                | Attribute           | Value   |
>>         +--------------------------+---------------------+---------+
>>         | pthread_setcancelstate() | Async-cancel-safety | AC-Safe |
>>         +--------------------------+---------------------+---------+
>
> Looks good to me.
>
>> An alternative approach would be to have subsections under ATTRIBUES,
>> with separate tables for Thread-safety, Async-signal-safety, and
>> Async-cancel-safety. Something like this:
>>
>>     ATTRIBUTES
>>
>>             See attributes(7).
>>
>>         Thread-safety
>>             +-------------------+---------+
>>             | Interface         | Value   |
>>             +-------------------+---------+
>>             | bindresvport()    | MT-Safe |
>>             +-------------------+---------+
>>
>>             Before glibc 2.17, the bindresvport() function uses a static
>>             variable that is not protected, so it is not thread-safe.
>>             Since glibc 2.17, it protects the static variable with a lock,
>>             and is thus MT-Safe.
>>
>>         Async-cancel-safety
>>
>>             +--------------------------+---------+
>>             | Interface                | Value   |
>>             +--------------------------+---------+
>>             | pthread_setcancelstate() | AC-Safe |
>>             +--------------------------+---------+
>>
>> Does one of those approaches eem okay? Haitao, Qian Lei, what do you
>> think? (I appreciate that there's a lot of pages to be fixed to follow
>> this new approach: I'd do that, though you could help if you want.)
>
> I agree with Peng, I like the first approach of a single table.

Alrighty, I need to hack a few of the pages to see that this approach
works, and then we can make the change wholesale.

> We should also evaluate the approach of adding the glibc version to
> the table if and when the version change would change the result e.g.
> the table lists the historical values.

Good point. I'll give that a little thought.

Thanks,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Michael Kerrisk (man-pages)]

On Wed, Jun 11, 2014 at 10:31 PM, Michael Kerrisk (man-pages)
<mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> Hi Carlos,
>
> On Wed, Jun 11, 2014 at 4:59 PM, Carlos O'Donell
> <carlos-v2tUB8YBRSi3e3T8WW9gsA@public.gmane.org> wrote:
>> On Wed, Jun 11, 2014 at 4:30 AM, Michael Kerrisk (man-pages)
>> <mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>> Hi Carlos, Haitao, Qian Lei, Alex,
[...]
>>> Does one of those approaches eem okay? Haitao, Qian Lei, what do you
>>> think? (I appreciate that there's a lot of pages to be fixed to follow
>>> this new approach: I'd do that, though you could help if you want.)
>>
>> I agree with Peng, I like the first approach of a single table.
>
> Alrighty, I need to hack a few of the pages to see that this approach
> works, and then we can make the change wholesale.

Okay -- take a look[1] at the change in the 'attributes_reformat'
branch that I've just pushed. That tries to cover a representative
sampling of the changes that I think we need to make. In particular,
the changed pages are:

    a64l.3
    abort.3
    abs.3
    atof.3
    bindresvport.3
    drand48.3
    fenv.3

Does this seem okay?

>> We should also evaluate the approach of adding the glibc version to
>> the table if and when the version change would change the result e.g.
>> the table lists the historical values.
>
> Good point. I'll give that a little thought.

Take a look at bindresvport.3 for an example of how we might deal with
these cases.

Cheers,

Michael

[1]   $ git pull
      $ git checkout remotes/origin/attributes_reformat

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Michael Kerrisk (man-pages)]

On Thu, Jun 12, 2014 at 1:27 PM, Michael Kerrisk (man-pages)
<mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> On Wed, Jun 11, 2014 at 10:31 PM, Michael Kerrisk (man-pages)
> <mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> Hi Carlos,
>>
>> On Wed, Jun 11, 2014 at 4:59 PM, Carlos O'Donell
>> <carlos-v2tUB8YBRSi3e3T8WW9gsA@public.gmane.org> wrote:
>>> On Wed, Jun 11, 2014 at 4:30 AM, Michael Kerrisk (man-pages)
>>> <mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>>> Hi Carlos, Haitao, Qian Lei, Alex,
> [...]
>>>> Does one of those approaches eem okay? Haitao, Qian Lei, what do you
>>>> think? (I appreciate that there's a lot of pages to be fixed to follow
>>>> this new approach: I'd do that, though you could help if you want.)
>>>
>>> I agree with Peng, I like the first approach of a single table.
>>
>> Alrighty, I need to hack a few of the pages to see that this approach
>> works, and then we can make the change wholesale.
>
> Okay -- take a look[1] at the change in the 'attributes_reformat'
> branch that I've just pushed. That tries to cover a representative
> sampling of the changes that I think we need to make. In particular,
> the changed pages are:
>
>     a64l.3
>     abort.3
>     abs.3
>     atof.3
>     bindresvport.3
>     drand48.3
>     fenv.3
>
> Does this seem okay?
>
>>> We should also evaluate the approach of adding the glibc version to
>>> the table if and when the version change would change the result e.g.
>>> the table lists the historical values.
>>
>> Good point. I'll give that a little thought.
>
> Take a look at bindresvport.3 for an example of how we might deal with
> these cases.
>
> Cheers,
>
> Michael
>
> [1]   $ git pull
>       $ git checkout remotes/origin/attributes_reformat

For good measure, I also added

    pthread_setcancelstate.3

to show two different attribute types in a table.

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Carlos O'Donell]

On Thu, Jun 12, 2014 at 8:16 AM, Michael Kerrisk (man-pages)
<mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> [1]   $ git pull
>>       $ git checkout remotes/origin/attributes_reformat
>
> For good measure, I also added
>
>     pthread_setcancelstate.3
>
> to show two different attribute types in a table.

Looks amazing. Exactly the right way forward. Wicked job.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Michael Kerrisk (man-pages)]

Hello Haitao and Qian Lei,

Any thoughts on the changes proposed in that Git branch (see below).

Cheers,

Michael


On 06/12/2014 02:16 PM, Michael Kerrisk (man-pages) wrote:
> On Thu, Jun 12, 2014 at 1:27 PM, Michael Kerrisk (man-pages)
> <mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>> On Wed, Jun 11, 2014 at 10:31 PM, Michael Kerrisk (man-pages)
>> <mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>> Hi Carlos,
>>>
>>> On Wed, Jun 11, 2014 at 4:59 PM, Carlos O'Donell
>>> <carlos-v2tUB8YBRSi3e3T8WW9gsA@public.gmane.org> wrote:
>>>> On Wed, Jun 11, 2014 at 4:30 AM, Michael Kerrisk (man-pages)
>>>> <mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>>>>> Hi Carlos, Haitao, Qian Lei, Alex,
>> [...]
>>>>> Does one of those approaches eem okay? Haitao, Qian Lei, what do you
>>>>> think? (I appreciate that there's a lot of pages to be fixed to follow
>>>>> this new approach: I'd do that, though you could help if you want.)
>>>>
>>>> I agree with Peng, I like the first approach of a single table.
>>>
>>> Alrighty, I need to hack a few of the pages to see that this approach
>>> works, and then we can make the change wholesale.
>>
>> Okay -- take a look[1] at the change in the 'attributes_reformat'
>> branch that I've just pushed. That tries to cover a representative
>> sampling of the changes that I think we need to make. In particular,
>> the changed pages are:
>>
>>     a64l.3
>>     abort.3
>>     abs.3
>>     atof.3
>>     bindresvport.3
>>     drand48.3
>>     fenv.3
>>
>> Does this seem okay?
>>
>>>> We should also evaluate the approach of adding the glibc version to
>>>> the table if and when the version change would change the result e.g.
>>>> the table lists the historical values.
>>>
>>> Good point. I'll give that a little thought.
>>
>> Take a look at bindresvport.3 for an example of how we might deal with
>> these cases.
>>
>> Cheers,
>>
>> Michael
>>
>> [1]   $ git pull
>>       $ git checkout remotes/origin/attributes_reformat
> 
> For good measure, I also added
> 
>     pthread_setcancelstate.3
> 
> to show two different attribute types in a table.
> 
> Cheers,
> 
> Michael
> 
> 


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Peng Haitao]

On 06/16/2014 11:53 AM, Michael Kerrisk (man-pages) wrote:
> Hello Haitao and Qian Lei,
> 
> Any thoughts on the changes proposed in that Git branch (see below).
> 

Sorry for the late reply. 

>>> Okay -- take a look[1] at the change in the 'attributes_reformat'
>>> branch that I've just pushed. That tries to cover a representative
>>> sampling of the changes that I think we need to make. In particular,
>>> the changed pages are:
>>>
>>>     a64l.3
>>>     abort.3
>>>     abs.3
>>>     atof.3
>>>     bindresvport.3
>>>     drand48.3
>>>     fenv.3
>>>
>>> Does this seem okay?
>>>

Well done, thanks.

atof.3 has an error: 
".SS Multithreading (see pthreads(7))" should be deleted.


-- 
Best Regards,
Peng

>>>>> We should also evaluate the approach of adding the glibc version to
>>>>> the table if and when the version change would change the result e.g.
>>>>> the table lists the historical values.
>>>>
>>>> Good point. I'll give that a little thought.
>>>
>>> Take a look at bindresvport.3 for an example of how we might deal with
>>> these cases.
>>>
>>> Cheers,
>>>
>>> Michael
>>>
>>> [1]   $ git pull
>>>       $ git checkout remotes/origin/attributes_reformat
>>
>> For good measure, I also added
>>
>>     pthread_setcancelstate.3
>>
>> to show two different attribute types in a table.
>>
>> Cheers,
>>
>> Michael
>>
>>
> 
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Michael Kerrisk (man-pages)]

On 06/16/2014 09:49 AM, Peng Haitao wrote:
> 
> On 06/16/2014 11:53 AM, Michael Kerrisk (man-pages) wrote:
>> Hello Haitao and Qian Lei,
>>
>> Any thoughts on the changes proposed in that Git branch (see below).
>>
> 
> Sorry for the late reply. 
> 
>>>> Okay -- take a look[1] at the change in the 'attributes_reformat'
>>>> branch that I've just pushed. That tries to cover a representative
>>>> sampling of the changes that I think we need to make. In particular,
>>>> the changed pages are:
>>>>
>>>>     a64l.3
>>>>     abort.3
>>>>     abs.3
>>>>     atof.3
>>>>     bindresvport.3
>>>>     drand48.3
>>>>     fenv.3
>>>>
>>>> Does this seem okay?
>>>>
> 
> Well done, thanks.

Good. Could I ask you and Qian Lei to submit future patches in that format.

Carlos, I await some text from you and/or Alex under a license as discussed.

> atof.3 has an error: 
> ".SS Multithreading (see pthreads(7))" should be deleted.

Fixed. Thanks for spotting that.

Cheers,

Michael



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Michael Kerrisk (man-pages)]

On 06/17/2014 07:40 AM, Michael Kerrisk (man-pages) wrote:
> On 06/16/2014 09:49 AM, Peng Haitao wrote:
>>
>> On 06/16/2014 11:53 AM, Michael Kerrisk (man-pages) wrote:
>>> Hello Haitao and Qian Lei,
>>>
>>> Any thoughts on the changes proposed in that Git branch (see below).
>>>
>>
>> Sorry for the late reply. 
>>
>>>>> Okay -- take a look[1] at the change in the 'attributes_reformat'
>>>>> branch that I've just pushed. That tries to cover a representative
>>>>> sampling of the changes that I think we need to make. In particular,
>>>>> the changed pages are:
>>>>>
>>>>>     a64l.3
>>>>>     abort.3
>>>>>     abs.3
>>>>>     atof.3
>>>>>     bindresvport.3
>>>>>     drand48.3
>>>>>     fenv.3
>>>>>
>>>>> Does this seem okay?
>>>>>
>>
>> Well done, thanks.
> 
> Good. Could I ask you and Qian Lei to submit future patches in that format.

Hello Haitao, Qian Lei,

I meant also to add that I'll think about a plan of how we could update the 
existing pages. Would either of you have any time to help with this task?

Thanks,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Peng Haitao]

On 06/17/2014 01:47 PM, Michael Kerrisk (man-pages) wrote:
>>>
>>> Well done, thanks.
>>
>> Good. Could I ask you and Qian Lei to submit future patches in that format.
> 

Yes, no problem.

> Hello Haitao, Qian Lei,
> 
> I meant also to add that I'll think about a plan of how we could update the 
> existing pages. Would either of you have any time to help with this task?
> 

I am pleased to update the existing pages and should have time to help with this task.
Thanks.


-- 
Best Regards,
Peng

> Thanks,
> 
> Michael
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[Peng Haitao]

On 02/12/2014 11:31 PM, Carlos O'Donell wrote:
>> Yes, maybe it is better.
>> But in Oracle Solaris's manpages, iswblank is marked "MT-Safe with exceptions".
>> I also think "MT-Safe with exceptions" is OK.
> 
> My suggestion is:
> 
> "The function iswblank() is thread safe with exceptions (locale). See
> "Safety Concepts" for more information."
> 

I think we should tell users where can look up "Safety Concepts".
The description may be:

"The function iswblank() is thread safe with exceptions (locale). See
"Safety Concepts" for more information in pthreads(7)."

Maybe pthreads(7) is man-pages(7) or others.

Before the decision is made, I will only send the patch without exceptions.
Thanks.


-- 
Best Regards,
Peng

> Then describe the locale issue once in the "Safety Concepts" section.
> 
> This reduces duplicated text and simplifies each man page.
> 
> Users that know what "locale" means can scroll quickly, users that
> don't can look it up and learn.
> 
> Does that make sense?
> 
> My further suggestion is to use the same ~20 exception notes as glibc e.g.
> race, const, locale, env, hostid, sigintr, init, lock, corrupt, heap,
> dlopen, plugin, i18ln, fd, mem, sig, term, and cwd (we also have
> !posix to indicate it is known not to conform with posix).
> 
> The glibc manual is marked with them now, you can see iswblank here:
> http://www.gnu.org/software/libc/manual/html_mono/libc.html#index-iswblank-486
> 
> Cheers,
> Carlos.
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] iswblank.3: ATTRIBUTES: Note function that is thread safe with exceptions

[walter harms]

Am 25.02.2014 06:35, schrieb Peng Haitao:
> 
> On 02/12/2014 11:31 PM, Carlos O'Donell wrote:
>>> Yes, maybe it is better.
>>> But in Oracle Solaris's manpages, iswblank is marked "MT-Safe with exceptions".
>>> I also think "MT-Safe with exceptions" is OK.
>>
>> My suggestion is:
>>
>> "The function iswblank() is thread safe with exceptions (locale). See
>> "Safety Concepts" for more information."
>>
> 
> I think we should tell users where can look up "Safety Concepts".
> The description may be:
> 
> "The function iswblank() is thread safe with exceptions (locale). See
> "Safety Concepts" for more information in pthreads(7)."
> 

Good idea, maybe also (locale(7)) ?

re,
 wh

> Maybe pthreads(7) is man-pages(7) or others.
> 
> Before the decision is made, I will only send the patch without exceptions.
> Thanks.
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html