Re: [PATCH] man7/mdoc_samples.7: srcfix: Avoid a warning about a wrong section

Re: [PATCH] man7/mdoc_samples.7: srcfix: Avoid a warning about a wrong section

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 2968 bytes --]

At 2019-02-27T10:09:44+0100, Michael Kerrisk (man-pages) wrote:
> Bjarni,
> 
> On 2/16/19 7:03 PM, Bjarni Ingi Gislason wrote:
> > Usage: .Rv -std in sections 2 and 3 only (#1672)
> > 
> >   The output from "nroff" and "groff" is unchanged
> 
> Can you please elaborate on what the problem problem is (.,
> how do you see the warning)?

I can't answer the parenthetical definitively--I think Bjarni has
customized local versions of man-db and groff that enable or supplement
warnings, making a kind of lint tool for man pages.

But I think I can speak to the underlying change.  The names of some
symbols in the mdoc macro package got their "doc-" prefixes restored to
them in groff 1.22.4.  They had been getting rewritten by "the stripper"
in the groff source tree, a sed script which applies a bunch of
transforms to a few of the macro packages to make them run more
efficiently.  Groff's parser does not tokenize its input, let alone
compile it down to an intermediate representation, so for instance if
you have a 72-character comment line inside a macro definition or loop,
thus:

.\" ********************************************************************

...groff will re-parse the 73 characters (counting the newline) every
time the macro is called or the loop iterates.

Unfortunately it is only the stripped version of the macro packages that
get installed, which makes them pretty hostile to user comprehension,
like JavaScript minification.

Opinions on the utility of the stripper script among the groff
development team are mixed.  One thing no one cared to defend was using
the stripper to change the names of string or number registers, as those
are part of the interface of a macro package, not cosmetic or stylistic
stuff.  The whole issue arose because the stripper script inadvertently
renamed a symbol in "mom", an entirely different macro package, contrary
to the knowledge and intentions of its developer.

Here is the commit in question.

https://lists.gnu.org/archive/html/groff-commit/2017-11/msg00098.html

Come to think of it, because this _was_ an interface change for mdoc,
this change should have been documented in the NEWS file for groff
1.22.4.  That it was not was my oversight, and I apologize.

> > Signed-off-by: Bjarni Ingi Gislason <bjarniig@rhi.hi.is>
> > ---
> >  man7/mdoc.samples.7 | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/man7/mdoc.samples.7 b/man7/mdoc.samples.7
> > index 6d1887481..7109afba4 100644
> > --- a/man7/mdoc.samples.7
> > +++ b/man7/mdoc.samples.7
> > @@ -1668,11 +1668,11 @@ will generate the following text:
> >  .Pp
> >  .\" fake section 3 to avoid error message from Rv
> >  .\".ds cH 3
> > -.ds section 3
> > +.ds doc-section 3
> >  .Rv -std atexit
> >  .\" and back to 7 again
> >  .\".ds cH 7
> > -.ds section 7
> > +.ds doc-section 7
> >  .Pp
> >  The
> >  .Fl std

Regards,
Branden

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] man7/mdoc_samples.7: srcfix: Avoid a warning about a wrong section

[Ingo Schwarze]

Hello Micheal, dear colleagues,

G. Branden Robinson wrote on Wed, Feb 27, 2019 at 08:48:50PM +1100:
> At 2019-02-27T10:09:44+0100, Michael Kerrisk (man-pages) wrote:
>> On 2/16/19 7:03 PM, Bjarni Ingi Gislason wrote:

>>> Usage: .Rv -std in sections 2 and 3 only (#1672)
>>>   The output from "nroff" and "groff" is unchanged

>> Can you please elaborate on what the problem problem is (.,
>> how do you see the warning)?

> I can't answer the parenthetical definitively--I think Bjarni has
> customized local versions of man-db and groff that enable or supplement
> warnings, making a kind of lint tool for man pages.

And some of those emit warnings that i would definitely call bogus,
plus Bjarni is unusually bad at explaining what he is doing, and
also unusually bad at distinguishing real problems from linter noise.
So in general, don't waste too much time on whatever Bjarni may be
saying.

In this case, there happens to be an actual problem - even though the
patch Bjarni sent is wrong and misleading, as it is usual with him.

> But I think I can speak to the underlying change.  The names of some
> symbols in the mdoc macro package got their "doc-" prefixes restored to
> them in groff 1.22.4.  They had been getting rewritten by "the stripper"
> in the groff source tree, a sed script which applies a bunch of
> transforms to a few of the macro packages to make them run more
> efficiently.  Groff's parser does not tokenize its input, let alone
> compile it down to an intermediate representation, so for instance if
> you have a 72-character comment line inside a macro definition or loop,
> thus:
> 
> .\" ********************************************************************
> 
> ...groff will re-parse the 73 characters (counting the newline) every
> time the macro is called or the loop iterates.
> 
> Unfortunately it is only the stripped version of the macro packages that
> get installed, which makes them pretty hostile to user comprehension,
> like JavaScript minification.
> 
> Opinions on the utility of the stripper script among the groff
> development team are mixed.  One thing no one cared to defend was using
> the stripper to change the names of string or number registers,

So far, i consider Brandon's description accurate and to the point.

> as those are part of the interface of a macro package,

I think this phrase invites a misunderstanding.  With a library or
macro package, the term "interface" is usually understood as "user
interface" or "API", i.e. what application programs (or end-user
documents like manual pages in the case of a macro package) are
supposed to use.

But the .doc-* macros are specifically *NOT* intended for use by
manual pages.  They are supposed to be *INTERNAL* implementation
details of the mdoc(7) macro package, and as such, they can change
at any time without any notice whatsoever.  The .doc-* prefix is
specifically intended to stress the internal character of these
macros and avoid clashes with user-defined macros - even though
defining macros inside manual pages certainly wouldn't be good
style, it still makes sense that a macro set keeps the global
namespace as clean as possible.  Just like a library might start
the names of private, internal symbols that application programs
are not supposed to access with, e.g., an "_" prefix.

> not cosmetic or stylistic
> stuff.  The whole issue arose because the stripper script inadvertently
> renamed a symbol in "mom", an entirely different macro package, contrary
> to the knowledge and intentions of its developer.

Exactly, *that* was the reason for the change: automatically changing
symbol names in libraries by simplistic scripts is a bad idea because
it is likely to sooner or later break in unexpected ways and cause bugs.

> Here is the commit in question.
> 
> https://lists.gnu.org/archive/html/groff-commit/2017-11/msg00098.html
> 
> Come to think of it, because this _was_ an interface change for mdoc,
> this change should have been documented in the NEWS file for groff
> 1.22.4.  That it was not was my oversight, and I apologize.

Branden, i respectfully disagree, and quite strongly.  You did nothing
wrong.  This was not an interface change, certainly not in the sense
that needs to be announced to users.  It was a purely internal
implementation change, entirely irrelevant for users.

The two actual problems are both within the Linux man-pages project,
not within groff:

 1. While back in the early 1990ies, Cynthia Livingston's
    mdoc.samples(7) manual page was an important document and the
    de-facto language definition of the mdoc(7) language, it has
    been outdated for a long time now.  The current groff_mdoc(7)
    manual page is based on it but contains large numbers of important
    improvements by Werner Lemberg and others.  As an alternative
    language definition that is slightly more concise without being
    less precise and complete, the mdoc(7) manual page is available
    from the mandoc(1) distribution (mandoc.bsd.lv).  If there are
    any contradictions between groff_mdoc(7) and mdoc(7), those are
    unintended and i ought to fix them.

    So i really believe that the Linux man-pages project ought to
    stop distributing the woefully outdated mdoc.samples(7) manual
    page.  If you want to include documentation for the mdoc language,
    i suggest that you either include a copy of the current version
    of the groff_mdoc(7) manual from the groff(1) distribution or
    of the mdoc(7) manual from the mandoc(1) distribution, whichever
    you think harmonizes better with the Linux man-pages project.
    Both are BSD-style licensed, so there should be no licensing
    issues.

    I'm not sure whether it is better for you to include or not
    include it.  There is probably value in having mdoc(7) documentation
    out of the box with the Linux man-pages project.  Then again,
    having groff_mdoc(7) in both the Linux man-pages package and
    in the groff package - or having mdoc(7) in both the Linux
    man-pages project and the mandoc(1) package - might cause
    packaging conflicts for some distributions.  I don't rightly
    know how such conflicts are typically handled by Linux
    distributions.  Not being able to install the Linux man-pages
    pages project, groff(1) and mandoc(1) all together on the same
    Linux machine would certainly be a bad situation...

    By the way, the mdoc(7) manual page distributed by the Linux
    man-pages project also makes very little sense.  It is a partial
    repetition of information from groff_mdoc(7)/[mandoc-]mdoc(7),
    but so compressed that it is mostly unintelligible.  Besides,
    it is incomplete: e.g. .Lk, .Mt, .Dx, .Ox, .Nx, .Ta, .%U, .Bk,
    .Ek, .Lb, .In, .Ft, .Ms, .Brq, .Bro, .Brc, .Ex are missing -
    it seems outdated by at lest 25 years.  Also, some claims are
    outright wrong - for example, you *cannot* use .UR/.UE in an
    mdoc(7) document, and i cannot remember ever having seen an
    implementation of a .UN macro anywhere.  Some macros descriptions
    are also wrong, e.g. .Fd is *not* intended for "function
    declarations", and .Vt is *not* "Fortran only".  And so on.

 2. I don't recommend keeping the old mdoc.samples(7) and mdoc(7)
    manual pages, but if you think you must do that for some reason,
    then you must at least revert this bogus commit:

    commit 15a12343170f78bed753787e7886dc30b244aced
    Author: Bjarni Ingi Gislason <bjarniig@rhi.hi.is>
    Date:   Fri Jun 8 02:44:11 2012 +1200
    Signed-off-by: Michael Kerrisk <mtk.manpages@gmail.com>

    A manual page *must not* access internal implementation details
    of the manual page macros like the doc-section string variable.

    It is just like with compiler warnings in general: you cannot
    get code 100% free of compiler warnings, small numbers of false
    positives will always remain, and if you try too hard, then you
    will eventually introduce bugs that are worse than living with
    a small number of compiler warning false positives.

    (Aside: as you will see when running a real linter like "mandoc
    -T lint" on your ancient version of mdoc.samples(7), there *are*
    many dozens of syntactic and logical aspects that could indeed
    be improved and that Bjarni consistently missed when sending
    his patches.  But given the outdated content, the page is hardly
    worth such polishing.)

Yours,
  Ingo

Re: [PATCH] man7/mdoc_samples.7: srcfix: Avoid a warning about a wrong section

[Michael Kerrisk (man-pages)]

Hello Branden,
On Wed, 27 Feb 2019 at 10:48, G. Branden Robinson
<g.branden.robinson@gmail.com> wrote:
>
> At 2019-02-27T10:09:44+0100, Michael Kerrisk (man-pages) wrote:
> > Bjarni,
> >
> > On 2/16/19 7:03 PM, Bjarni Ingi Gislason wrote:
> > > Usage: .Rv -std in sections 2 and 3 only (#1672)
> > >
> > >   The output from "nroff" and "groff" is unchanged
> >
> > Can you please elaborate on what the problem problem is (.,
> > how do you see the warning)?
>
> I can't answer the parenthetical definitively--I think Bjarni has
> customized local versions of man-db and groff that enable or supplement
> warnings, making a kind of lint tool for man pages.
>
> But I think I can speak to the underlying change.  The names of some
> symbols in the mdoc macro package got their "doc-" prefixes restored to
> them in groff 1.22.4.  They had been getting rewritten by "the stripper"
> in the groff source tree, a sed script which applies a bunch of
> transforms to a few of the macro packages to make them run more
> efficiently.  Groff's parser does not tokenize its input, let alone
> compile it down to an intermediate representation, so for instance if
> you have a 72-character comment line inside a macro definition or loop,
> thus:
>
> .\" ********************************************************************
>
> ...groff will re-parse the 73 characters (counting the newline) every
> time the macro is called or the loop iterates.
>
> Unfortunately it is only the stripped version of the macro packages that
> get installed, which makes them pretty hostile to user comprehension,
> like JavaScript minification.
>
> Opinions on the utility of the stripper script among the groff
> development team are mixed.  One thing no one cared to defend was using
> the stripper to change the names of string or number registers, as those
> are part of the interface of a macro package, not cosmetic or stylistic
> stuff.  The whole issue arose because the stripper script inadvertently
> renamed a symbol in "mom", an entirely different macro package, contrary
> to the knowledge and intentions of its developer.
>
> Here is the commit in question.
>
> https://lists.gnu.org/archive/html/groff-commit/2017-11/msg00098.html
>
> Come to think of it, because this _was_ an interface change for mdoc,
> this change should have been documented in the NEWS file for groff
> 1.22.4.  That it was not was my oversight, and I apologize.

So, summary: should I apply Bjarni's patch? Or does this lead to
back-compatibility problems?

Cheers,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: [PATCH] man7/mdoc_samples.7: srcfix: Avoid a warning about a wrong section

[G. Branden Robinson]

[-- Attachment #1.1: Type: text/plain, Size: 2897 bytes --]

At 2019-02-27T13:37:46+0100, Michael Kerrisk (man-pages) wrote:
> So, summary: should I apply Bjarni's patch? Or does this lead to
> back-compatibility problems?

For maximum back-compatibility, you'd leave all of the string defines
(.ds requests) active and uncommented.

Zooming out a little bit, it appears that the proposed patch may be
getting layered on top of an _earlier_ patch to shut up warnings from
the macro package while rendering mdoc.samples(7).

Looking at the patch once again, and noting the context, I see three
workarounds in a row, first pushed and then popped.

--- a/man7/mdoc.samples.7
+++ b/man7/mdoc.samples.7
@@ -1668,11 +1668,11 @@ will generate the following text:
 .Pp
 .\" fake section 3 to avoid error message from Rv
 .\".ds cH 3
-.ds section 3
+.ds doc-section 3
 .Rv -std atexit
 .\" and back to 7 again
 .\".ds cH 7
-.ds section 7
+.ds doc-section 7
 .Pp
 The
 .Fl std

The warning in question is this:

Usage: .Rv -std in sections 2 and 3 only (#1672)

I'm starting to think the correct thing to do here is to see how we can
get the page to render correctly without having to define strings at
all.  Or, as Ingo suggested, drop the page altogether.

The purpose of the mdoc macro ".Rv" is to document a function's return
value.  I've attached groff's implementation of this macro for the
curious.

The macro complains if it's called from a page that isn't in the two
sections set aside for documenting function interfaces (2 and 3).  Being
a gallery of examples, the page is trying to trick .Rv into not spewing
a diagnostic message by using insider knowledge of the test .Rv
implements to determine whether it should emit one.  This is going to be
a fragile thing, hence the 3 different strings that have been used for
the purpose: "doc-section" (unstripped groff_mdoc, groff 1.22.4),
"section" (stripped groff_mdoc, groff 1.22.3 and earlier), and "cH" (no
idea).

While I consider myself competent in man(7), I defer to Ingo's expertise
on mdoc(7)[1].  But I don't think any of the above--the commented out
.ds, the old (-) one, or the new (+)--is a model solution for resolving
what .Rv is complaining about.

Therefore my suggestion is to either live with the warning, chopping out
_all_ these .ds kludges from the page--leaving one comment noting the
warning we expect--or to drop the page altogether.

Regards,
Branden

[1] As I have said elsewhere, I admire mdoc's goals and design--it is
semantically-oriented, which man(7) very badly is not.  Unfortunately I
find mdoc unergonomic due in part to its self-restriction to the
two-character macro namespace of legacy *roff implementations, which in
my opinion makes the language too hard to acquire, especially for the
occasional user.  And most man page writers are only occasional users.
(I have other complaints about mdoc's documentation and pedagogy, but I
will leave them for another discussion.)

[-- Attachment #1.2: mdoc_rv_impl.tmac --]
[-- Type: text/plain, Size: 2251 bytes --]

.\" NS Rv user macro
.\" NS   return values
.\" NS
.\" NS width register 'Rv' set in doc-common
.\" NS
.\" NS local variables:
.\" NS   doc-str-Rv-std-prefix
.\" NS   doc-str-Rv-std-suffix
.\" NS   doc-str-Rv-stds-prefix
.\" NS   doc-str-Rv-stds-and
.\" NS   doc-str-Rv-stds-suffix
.\" NS   doc-str-Rv-std0
.
.ds doc-str-Rv-std-prefix "The
.ds doc-str-Rv-std-suffix "function returns the value\~0 if successful;
.as doc-str-Rv-std-suffix " otherwise the value\~\-1 is returned and
.as doc-str-Rv-std-suffix " the global variable \*[doc-Va-font]errno\f[]
.as doc-str-Rv-std-suffix " is set to indicate the error.
.
.ds doc-str-Rv-stds-prefix "The
.ds doc-str-Rv-stds-and    "and
.ds doc-str-Rv-stds-suffix "functions return the value\~0 if successful;
.as doc-str-Rv-stds-suffix " otherwise the value\~\-1 is returned and
.as doc-str-Rv-stds-suffix " the global variable \*[doc-Va-font]errno\f[]
.as doc-str-Rv-stds-suffix " is set to indicate the error.
.
.ds doc-str-Rv-std0 "Upon successful completion, the value\~0 is returned;
.as doc-str-Rv-std0 " otherwise the value\~\-1 is returned and
.as doc-str-Rv-std0 " the global variable \*[doc-Va-font]errno\f[]
.as doc-str-Rv-std0 " is set to indicate the error.
.
.de Rv
.
.\" XXX: what does this function without '-std'?
.
.  if \n[doc-arg-limit] \{\
.    tm Usage: .Rv not callable by other macros (#\n[.c])
.    doc-reset-args
.    return
.  \}
.
.  if !\n[.$] \{\
.    tm Usage: .Rv [-std] [<function> ...] (#\n[.c])
.    return
.  \}
.
.  if "\$1"-std" \{\
.    nr doc-reg-Rv \*[doc-section]
.    if ((\n[doc-reg-Rv] < 2) : (\n[doc-reg-Rv] > 3)) \
.      tm Usage: .Rv -std in sections 2 and 3 only (#\n[.c])
.    br
.    shift
.    ie (\n[.$] > 1) \{\
.      nop \)\*[doc-str-Rv-stds-prefix]
.      nr doc-reg-Rv 1
.      while (\n[doc-reg-Rv] < \n[.$]) \{\
.        ie (\n[.$] > 2) \
.          Fn \$\n[doc-reg-Rv] ,
.        el \
.          Fn \$\n[doc-reg-Rv]
.        nr doc-reg-Rv +1
.      \}
.      nop \)\*[doc-str-Rv-stds-and]
.      Fn \$\n[.$]
.      nop \)\*[doc-str-Rv-stds-suffix]
.    \}
.    el \{ .ie (\n[.$] == 1) \{\
.      nop \)\*[doc-str-Rv-std-prefix]
.      Fn \$1
.      nop \)\*[doc-str-Rv-std-suffix]
.    \}
.    el \{\
.      nop \)\*[doc-str-Rv-std0]
.  \}\}\}
..

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] man7/mdoc_samples.7: srcfix: Avoid a warning about a wrong section

[G. Branden Robinson]

[-- Attachment #1: Type: text/plain, Size: 3156 bytes --]

At 2019-02-27T13:34:29+0100, Ingo Schwarze wrote:
> > as those are part of the interface of a macro package,
> 
> I think this phrase invites a misunderstanding.  With a library or
> macro package, the term "interface" is usually understood as "user
> interface" or "API", i.e. what application programs (or end-user
> documents like manual pages in the case of a macro package) are
> supposed to use.
> 
> But the .doc-* macros are specifically *NOT* intended for use by
> manual pages.  They are supposed to be *INTERNAL* implementation
> details of the mdoc(7) macro package, and as such, they can change
> at any time without any notice whatsoever.  The .doc-* prefix is
> specifically intended to stress the internal character of these
> macros and avoid clashes with user-defined macros - even though
> defining macros inside manual pages certainly wouldn't be good
> style, it still makes sense that a macro set keeps the global
> namespace as clean as possible.  Just like a library might start
> the names of private, internal symbols that application programs
> are not supposed to access with, e.g., an "_" prefix.

Ah, okay, I stand corrected.  groff doesn't have namespaces or true
lexical scopes--though I would not dare assert that they could not be
hacked into existence--and I am too unfamiliar with mdoc to know which
symbols are intended for use by man page writers and which aren't.

> > Come to think of it, because this _was_ an interface change for mdoc,
> > this change should have been documented in the NEWS file for groff
> > 1.22.4.  That it was not was my oversight, and I apologize.
> 
> Branden, i respectfully disagree, and quite strongly.  You did nothing
> wrong.  This was not an interface change, certainly not in the sense
> that needs to be announced to users.  It was a purely internal
> implementation change, entirely irrelevant for users.

Ah, okay.  Well, if I _didn't_ screw up, I'll accept that.  :)

>     I'm not sure whether it is better for you to include or not
>     include it.  There is probably value in having mdoc(7) documentation
>     out of the box with the Linux man-pages project.  Then again,
>     having groff_mdoc(7) in both the Linux man-pages package and
>     in the groff package - or having mdoc(7) in both the Linux
>     man-pages project and the mandoc(1) package - might cause
>     packaging conflicts for some distributions.

This should not be a problem for any self-respecting distribution.
Package maintainers frequently choose not to "ship" certain files in
certain packages due to conflicts/overlap, and sophisticated package
management systems (like dpkg) allow for migration of files between
packages across upgrades, with rollback in case of problems.

>     I don't rightly know how such conflicts are typically handled by
>     Linux distributions.  Not being able to install the Linux
>     man-pages pages project, groff(1) and mandoc(1) all together on
>     the same Linux machine would certainly be a bad situation...

I don't think has been a technical challenge for at least 25 years.

Regards,
Branden

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] man7/mdoc_samples.7: srcfix: Avoid a warning about a wrong section

[Michael Kerrisk (man-pages)]

> Or, as Ingo suggested, drop the page altogether.

Exactly.

Thanks,

Michael


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: [PATCH] man7/mdoc_samples.7: srcfix: Avoid a warning about a wrong section

[Michael Kerrisk (man-pages)]

Hello Ingo,

Thank you for your very detailed mail. I'll leave aside
the history of people's interactions, and cut to the chase
at the end of this mail...

[...]

> The two actual problems are both within the Linux man-pages project,
> not within groff:
> 
>  1. While back in the early 1990ies, Cynthia Livingston's
>     mdoc.samples(7) manual page was an important document and the
>     de-facto language definition of the mdoc(7) language, it has
>     been outdated for a long time now.  The current groff_mdoc(7)
>     manual page is based on it but contains large numbers of important
>     improvements by Werner Lemberg and others.  As an alternative
>     language definition that is slightly more concise without being
>     less precise and complete, the mdoc(7) manual page is available
>     from the mandoc(1) distribution (mandoc.bsd.lv).  If there are
>     any contradictions between groff_mdoc(7) and mdoc(7), those are
>     unintended and i ought to fix them.
> 
>     So i really believe that the Linux man-pages project ought to
>     stop distributing the woefully outdated mdoc.samples(7) manual
>     page.  If you want to include documentation for the mdoc language,
>     i suggest that you either include a copy of the current version
>     of the groff_mdoc(7) manual from the groff(1) distribution or
>     of the mdoc(7) manual from the mandoc(1) distribution, whichever
>     you think harmonizes better with the Linux man-pages project.
>     Both are BSD-style licensed, so there should be no licensing
>     issues.
> 
>     I'm not sure whether it is better for you to include or not
>     include it.  There is probably value in having mdoc(7) documentation
>     out of the box with the Linux man-pages project.  Then again,
>     having groff_mdoc(7) in both the Linux man-pages package and
>     in the groff package - or having mdoc(7) in both the Linux
>     man-pages project and the mandoc(1) package - might cause
>     packaging conflicts for some distributions.  I don't rightly
>     know how such conflicts are typically handled by Linux
>     distributions.  Not being able to install the Linux man-pages
>     pages project, groff(1) and mandoc(1) all together on the same
>     Linux machine would certainly be a bad situation...
> 
>     By the way, the mdoc(7) manual page distributed by the Linux
>     man-pages project also makes very little sense.  It is a partial
>     repetition of information from groff_mdoc(7)/[mandoc-]mdoc(7),
>     but so compressed that it is mostly unintelligible.  Besides,
>     it is incomplete: e.g. .Lk, .Mt, .Dx, .Ox, .Nx, .Ta, .%U, .Bk,
>     .Ek, .Lb, .In, .Ft, .Ms, .Brq, .Bro, .Brc, .Ex are missing -
>     it seems outdated by at lest 25 years.  Also, some claims are
>     outright wrong - for example, you *cannot* use .UR/.UE in an
>     mdoc(7) document, and i cannot remember ever having seen an
>     implementation of a .UN macro anywhere.  Some macros descriptions
>     are also wrong, e.g. .Fd is *not* intended for "function
>     declarations", and .Vt is *not* "Fortran only".  And so on.
> 
>  2. I don't recommend keeping the old mdoc.samples(7) and mdoc(7)
>     manual pages, but if you think you must do that for some reason,
>     then you must at least revert this bogus commit:

I am *not at all* attached to keeping to these pages. Their 
presence in the project has always felt a bit anomalous to me.

Back when I took over maintainership in 2004, there were a small
number of pages that used mdoc markup, and so it seemed wise
to keep these pages. Over time, most of those few pages were
converted to 'man' markup, and today the only other page in the
project that still uses mdoc markup is in queue(3). So, there is
just about zero value in having 'mdoc' documentation come with
the "Linux man-pages" box.

Since I seldom use mdoc markup myself, I've had no reason to 
monitor pages such as groff_mdoc(7) or the mdoc(7) page
provided my ther 'mandoc' project and compare them with
the pages provided by "Linux man-pages". Now I've had a
closer look. It's sad.

I've removed mdoc(7) and mdoc.samples(7) from "Linux -man-pages".

That felt good.

Thank you for your input, Ingo!

Cheers,

Michael



-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: [PATCH] man7/mdoc_samples.7: srcfix: Avoid a warning about a wrong section

[Ingo Schwarze]

Hi Michael,

Michael Kerrisk (man-pages) wrote on Wed, Feb 27, 2019 at 03:28:19PM +0100:

> I am *not at all* attached to keeping to these pages. Their 
> presence in the project has always felt a bit anomalous to me.
> 
> Back when I took over maintainership in 2004, there were a small
> number of pages that used mdoc markup, and so it seemed wise
> to keep these pages. Over time, most of those few pages were
> converted to 'man' markup, and today the only other page in the
> project that still uses mdoc markup is in queue(3). So, there is
> just about zero value in having 'mdoc' documentation come with
> the "Linux man-pages" box.
> 
> Since I seldom use mdoc markup myself, I've had no reason to 
> monitor pages such as groff_mdoc(7) or the mdoc(7) page
> provided my ther 'mandoc' project and compare them with
> the pages provided by "Linux man-pages". Now I've had a
> closer look. It's sad.
> 
> I've removed mdoc(7) and mdoc.samples(7) from "Linux -man-pages".
> 
> That felt good.

Yes, that sounds entirely reasonable.

Even though the Linux man-pages project is very portable and can
be installed and used on almost any operating system, i think most
users probably use it on various distributions of Linux-based
operating systems.  And Linux-based operating systems almost
invariably provide a GNU troff (groff) package installed by default,
so i expect the vast majority of your users will still have mdoc
documentation on their system, in the form of groff_mdoc(7).

The mandoc package is also slowly becoming more popular on Linux.
For example, optional packages exist for Debian, Ubuntu, Gentoo,
Arch, Alpine, Void, Slackware, Crux, ...  So quite a few Linux users
may already have Kristaps' modern mdoc(7) manual page in addition
to the classical groff_mdoc(7).

Free non-Linux systems that offer access to the Linux man-pages project
anyway - see https://man.openbsd.org/Linux-4.16/ for an example -
typically include mandoc(1) by default (e.g. FreeBSD, OpenBSD,
NetBSD, DragonFly, illumos, Minix, ...) - so those already have
the mdoc(7) manual.

I would no doubt like to see mdoc(7) used more in Linux contexts,
too, because Cynthia managed to reach an excellent balance between
simplicity, conciseness, mnemonic value, and semantic markup power -
the semantic search facilities mdoc(7) supports nowadays are very
valuable in practice, and i think the difficulty of learning it is
often vastly exaggerated even for casual users.  Just look at the
MACRO OVERVIEW in mdoc(7), there aren't really that many macros:
  https://man.openbsd.org/mdoc.7#MACRO_OVERVIEW
If you want to see something that is *really* excessively complicated,
try DocBook for a change...  :-/

Anyway, that doesn't mean the Linux man-pages project has to document
mdoc(7) given that it is not currently using it in any significant way.

Yours,
  Ingo

Re: [PATCH] man7/mdoc_samples.7: srcfix: Avoid a warning about a wrong section

[Michael Kerrisk (man-pages)]

> The mandoc package is also slowly becoming more popular on Linux.
> For example, optional packages exist for Debian, Ubuntu, Gentoo,
> Arch, Alpine, Void, Slackware, Crux, ...  So quite a few Linux users
> may already have Kristaps' modern mdoc(7) manual page in addition
> to the classical groff_mdoc(7).

[...]

> I would no doubt like to see mdoc(7) used more in Linux contexts,
> too, because Cynthia managed to reach an excellent balance between
> simplicity, conciseness, mnemonic value, and semantic markup power -
> the semantic search facilities mdoc(7) supports nowadays are very
> valuable in practice, and i think the difficulty of learning it is
> often vastly exaggerated even for casual users.  Just look at the
> MACRO OVERVIEW in mdoc(7), there aren't really that many macros:
>   https://man.openbsd.org/mdoc.7#MACRO_OVERVIEW
> If you want to see something that is *really* excessively complicated,
> try DocBook for a change...  :-/
>
> Anyway, that doesn't mean the Linux man-pages project has to document
> mdoc(7) given that it is not currently using it in any significant way.

Just to be clear, I'm not against mdoc in any way. (I consider it
unfortunate that man markup isn't at all semantic.)

It's just that the project that I inherited had > 95% pages in man
format, and the existence of a handful of pages in mdoc format was
painful when (in the early days) I was trying to do various automated
global automated edits to get greater consistency of formatting across
pages. I wouldn't have been unhappy if all of the pages had instead
been in mdoc format, but that was not the history.

Cheers,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: [PATCH] man7/mdoc_samples.7: srcfix: Avoid a warning about a wrong section

[Ingo Schwarze]

Hi Michael,

Michael Kerrisk (man-pages) wrote on Fri, Mar 01, 2019 at 10:49:41AM +0100:

> Just to be clear, I'm not against mdoc in any way. (I consider it
> unfortunate that man markup isn't at all semantic.)
> 
> It's just that the project that I inherited had > 95% pages in man
> format, and the existence of a handful of pages in mdoc format was
> painful when (in the early days) I was trying to do various automated
> global automated edits to get greater consistency of formatting across
> pages.

Sure.  Consistent style eases maintenance.

> I wouldn't have been unhappy if all of the pages had instead
> been in mdoc format, but that was not the history.

The reason probably is that nothing (except bitrot) happens without
somebody doing the work.

Both the original UC California CSRG BSD project and the modern
*BSD projects deriving from it have always been adamant about
following the original UNIX tradition that manual pages are crucial,
are the format of choice for documentation.  Cynthia Livingston
spent substantial work in 1990/1991 on converting the whole corpus
to mdoc(7), and Jason McIntyre did additional conversions in OpenBSD
starting in 2003.  Illumos declared mdoc(7) the new default language
for new documentation around 2014, iirc.

It might have contributed to nobody porting up that work to GNU
land and to Linux land because in spite of the existence of the
Linux man pages project, the situation with respect to documentation
wasn't (and isn't) quite as clear in those lands as it is in BSD
land.

Given that the problem has been solved almost exhaustively for the
BSDs nearly 30 years ago, my focus hasn't been on writing tools to
support man(7) to mdoc(7) conversions either - so that is still
manual work even today; chaining doclifter with docbook2mdoc, as a
potential idea, has been around for many years, but has never been
developed to maturity for practical use at any scale.  Instead, when
working on converters, i worked on converters that are more relevant
in BSD contexts: mdoc to man (relevant for portable software having
mdoc pages that also wants to be installable on mdoc-lacking OSes),
pod2mdoc (relevant for converting OpenSSL to LibreSSL docs),
mdoc to markdown (relevant for those poor souls who are forced by
project policies to provide markdown versions of their documentation),
and so on...

But developing the doclifter|docbook2mdoc toolchain to practical
usability would likely be possible if there were a substantial need,
even though some manual postprocession would almost certainly remain
required.

Yours,
  Ingo