Documentation Breakage at 2.5.6

Documentation Breakage at 2.5.6

[Randall S. Becker]

Hi All,

I'm trying to upgrade the NonStop port from 2.3.7 upward eventually to
2.15.1 and hit a snag on documentation. The xmlto component is a bit new to
me and I hit the following error:

    XMLTO git-remote-testgit.1
xmlto: /home/git/git/Documentation/git-remote-testgit.xml does not validate
(status 3)
xmlto: Fix document syntax or use --skip-validation option
I/O error : Attempt to load network entity
http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd
/home/git/git/Documentation/git-remote-testgit.xml:2: warning: failed to
load external entity
"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"
D DocBook XML V4.5//EN"
"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"
 
^
I/O error : Attempt to load network entity
http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd
warning: failed to load external entity
"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"
validity error : Could not load the external subset
http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd

The -skip-validation option just takes me to a different problem validating
via sourceforge URL that appears not to exist anymore, although I had to
modify ./git/Documention/Makefile, which is vexing.

    XMLTO git-remote-testgit.1
I/O error : Attempt to load network entity
http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl
warning: failed to load external entity
"http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl"
compilation error: file /tmp/xmlto-xsl.ie6J8p line 4 element import
xsl:import : unable to load
http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl
Makefile:328: recipe for target 'git-remote-testgit.1' failed

Any advice on getting past this? It would be nice to get git help to working
again. An answer of "you need to get past 2.5.6" is ok too as long as I know
where I'm going.

Thanks,
Randall

-- Brief whoami: NonStop&UNIX developer since approximately
UNIX(421664400)/NonStop(211288444200000000) 
-- In my real life, I talk too much.

Re: Documentation Breakage at 2.5.6

[Ævar Arnfjörð Bjarmason]

On Wed, Dec 06 2017, Randall S. Becker jotted:

> Hi All,
>
> I'm trying to upgrade the NonStop port from 2.3.7 upward eventually to
> 2.15.1 and hit a snag on documentation. The xmlto component is a bit new to
> me and I hit the following error:
>
>     XMLTO git-remote-testgit.1
> xmlto: /home/git/git/Documentation/git-remote-testgit.xml does not validate
> (status 3)
> xmlto: Fix document syntax or use --skip-validation option
> I/O error : Attempt to load network entity
> http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd
> /home/git/git/Documentation/git-remote-testgit.xml:2: warning: failed to
> load external entity
> "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"
> D DocBook XML V4.5//EN"
> "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"
>
> ^
> I/O error : Attempt to load network entity
> http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd
> warning: failed to load external entity
> "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"
> validity error : Could not load the external subset
> http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd
>
> The -skip-validation option just takes me to a different problem validating
> via sourceforge URL that appears not to exist anymore, although I had to
> modify ./git/Documention/Makefile, which is vexing.
>
>     XMLTO git-remote-testgit.1
> I/O error : Attempt to load network entity
> http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl
> warning: failed to load external entity
> "http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl"
> compilation error: file /tmp/xmlto-xsl.ie6J8p line 4 element import
> xsl:import : unable to load
> http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl
> Makefile:328: recipe for target 'git-remote-testgit.1' failed
>
> Any advice on getting past this? It would be nice to get git help to working
> again. An answer of "you need to get past 2.5.6" is ok too as long as I know
> where I'm going.

I don't know if this helps, but here with xmlto 0.0.28 on Debian if I
apply this the docs still build:

    diff --git a/Documentation/texi.xsl b/Documentation/texi.xsl
    index 0f8ff07eca..332a65558d 100644
    --- a/Documentation/texi.xsl
    +++ b/Documentation/texi.xsl
    @@ -7,7 +7,7 @@
     <xsl:output method="xml"
                encoding="UTF-8"
                doctype-public="-//OASIS//DTD DocBook XML V4.5//EN"
    -           doctype-system="http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" />
    +           doctype-system="http://example.org/docbook/xml/4.5/docbookx.dtd" />

So whatever's needing to remote fetch those resources doesn't seem to
cause the same error for me.


> -- Brief whoami: NonStop&UNIX developer since approximately
> UNIX(421664400)/NonStop(211288444200000000)
> -- In my real life, I talk too much.

Re: Documentation Breakage at 2.5.6

[Jeff King]

On Wed, Dec 06, 2017 at 09:14:57AM +0100, Ævar Arnfjörð Bjarmason wrote:

> > I'm trying to upgrade the NonStop port from 2.3.7 upward eventually to
> > 2.15.1 and hit a snag on documentation. The xmlto component is a bit new to
> > me and I hit the following error:

Did it work before in v2.3.7? If so, can you bisect to the breakage?

> >     XMLTO git-remote-testgit.1
> > xmlto: /home/git/git/Documentation/git-remote-testgit.xml does not validate
> > (status 3)
> > xmlto: Fix document syntax or use --skip-validation option
> > I/O error : Attempt to load network entity
> > http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd
> > /home/git/git/Documentation/git-remote-testgit.xml:2: warning: failed to
> > load external entity
> > "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"
> > D DocBook XML V4.5//EN"
> > "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"
> >
> > ^
> > I/O error : Attempt to load network entity
> > http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd
> > warning: failed to load external entity
> > "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd"
> > validity error : Could not load the external subset
> > http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd

Those URLs are the "official" names of the docbook DTDs. But normally
you'd have a local copy, along with a mapping from the official name to
your local copy. The XML term for that mapping is a "catalog", and it
looks something like this:

  $ grep oasis-open /etc/xml/catalog
  <delegateSystem systemIdStartString="http://www.oasis-open.org/docbook/xml/" catalog="file:///etc/xml/docbook-xml.xml"/>

That just points to another local catalog, which has:

  $ grep 4.5/docbookx.dtd /etc/xml/docbook-xml.xml
  <delegateSystem systemIdStartString="http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" catalog="file:///usr/share/xml/docbook/schema/dtd/4.5/catalog.xml"/>
  <delegateSystem systemIdStartString="http://docbook.org/xml/4.5/docbookx.dtd" catalog="file:///usr/share/xml/docbook/schema/dtd/4.5/catalog.xml"/>

So my guess is that your problem is one of:

  1. You don't have docbook 4.5 installed on your system.

or

  2. You don't have a correctly built catalog file, or xmlto isn't
     pointing to it for some reason (on Debian, this is normally built
     by the post-install script of packages that contain xml).

And xmlto (actually, probably xsltproc that it's calling) is unwilling
or unable to hit the network to pull down those entities.

Those are all somewhat vague guesses based on past troubles I've had
with broken xml setups. I'm far from an expert on xml processing (and
I'd just as soon keep it that way).

> I don't know if this helps, but here with xmlto 0.0.28 on Debian if I
> apply this the docs still build:
> 
>     diff --git a/Documentation/texi.xsl b/Documentation/texi.xsl
>     index 0f8ff07eca..332a65558d 100644
>     --- a/Documentation/texi.xsl
>     +++ b/Documentation/texi.xsl
>     @@ -7,7 +7,7 @@
>      <xsl:output method="xml"
>                 encoding="UTF-8"
>                 doctype-public="-//OASIS//DTD DocBook XML V4.5//EN"
>     -           doctype-system="http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" />
>     +           doctype-system="http://example.org/docbook/xml/4.5/docbookx.dtd" />
> 
> So whatever's needing to remote fetch those resources doesn't seem to
> cause the same error for me.

I think that would come into play only if you try to build
"gitman.info", which isn't one of the default targets.

The string that Randall is seeing is in git-remote-testgit.xml, so it's
probably be generated by the "docbook" backend of asciidoc.

One alternative is to try to avoid docbook entirely. The only way to get
manpages with asciidoc is to generate docbook and then process it, but:

 - you can generate HTML directly (and "make -C Documentation html" does
   this). Perhaps not as nice, but you still at least have some
   documentation.

 - asciidoctor can generate manpages directly. I don't think our
   Makefile supports that now, but it might not be too hard to hack in
   (we already have some basic asciidoctor support). I'm not sure how
   hard it would be to get Ruby running on NonStop

And of course one final option is to generate the manpages elsewhere and
copy them in, since they're platform-independent. In fact, that's what
quick-install-man should do (you just have to clone Junio's git-manpages
repository -- see the INSTALL file).

-Peff

RE: Documentation Breakage at 2.5.6

[Randall S. Becker]

-----Original Message-----
On December 6, 2017 3:49 AM, Jeff King wrote:
>On Wed, Dec 06, 2017 at 09:14:57AM +0100, Ævar Arnfjörð Bjarmason wrote:
>> > I'm trying to upgrade the NonStop port from 2.3.7 upward eventually 
>> > to
>> > 2.15.1 and hit a snag on documentation. The xmlto component is a bit 
>> > new to me and I hit the following error:
>Did it work before in v2.3.7? If so, can you bisect to the breakage?
It worked fine at 2.3.7. No seeming dependency on docbook at that point - it was never on my system.

>One alternative is to try to avoid docbook entirely. The only way to get manpages with asciidoc is to generate docbook and then process it, but:
I have asciidoc installed, but using it via Make?

> - you can generate HTML directly (and "make -C Documentation html" does
>  this). Perhaps not as nice, but you still at least have some
>   documentation.
Not an option. I need git help to work.

> - asciidoctor can generate manpages directly. I don't think our
>   Makefile supports that now, but it might not be too hard to hack in
>  (we already have some basic asciidoctor support). I'm not sure how
 > hard it would be to get Ruby running on NonStop
Ruby runs fine. I'm a bit out of my configuration depth here.

>And of course one final option is to generate the manpages elsewhere and copy them in, since they're platform-independent.
>In fact, that's what quick-install-man should do (you just have to clone Junio's >git-manpages repository -- see the INSTALL file).

I've gone down this path and it works. Much cleaner in fact. Dependencies of docbook (jade) are too reliant on GCC C++ forms to port to the platform - not to mention being SVN, which is culturally uncomfortable 😉

One request to Junio: Would it be possible to tag the commits to align with the tags in the main repo? That way, I can build a nice little Jenkins job to automatically fetch the correct commit for man pages when packaging up a release.

-Peff

Re: Documentation Breakage at 2.5.6

[Junio C Hamano]

"Randall S. Becker" <rsbecker@nexbridge.com> writes:

> One request to Junio: Would it be possible to tag the commits to
> align with the tags in the main repo? That way, I can build a nice
> little Jenkins job to automatically fetch the correct commit for
> man pages when packaging up a release.

Sorry, I missed this due to an overlong line in the message.

I am not interested in doing anything more than absolute minimum in
the history that records generated cruft.  We already describe the
mainline commit object names in the messages; perhaps that is
sufficient?

        commit daa88a54a985ed1ef258800c742223c2a8f0caaa
        Author: Junio C Hamano <gitster@pobox.com>
        Date:   Wed Dec 6 10:04:03 2017 -0800

            Autogenerated manpages for v2.15.1-354-g95ec6

        commit 466a3070abecf4081a12d8e07c770689506440b9
        Author: Junio C Hamano <gitster@pobox.com>
        Date:   Wed Nov 29 10:12:49 2017 +0900

            Autogenerated manpages for v2.15.1-271-g1a4e4

        commit be681f4100647ab93fd19cb5066fcaa2cb79204b
        Author: Junio C Hamano <gitster@pobox.com>
        Date:   Mon Nov 27 12:33:30 2017 +0900

            Autogenerated manpages for v2.15.0-374-g5f995

The primary reason why I do not want to tag them is because the tree
the documentation sources were taken from is *not* the only thing
that affects these autogenerated cruft.  The AsciiDoc toolchain that
happen to be installed on the box the day I ran the documentation
tools is an obvious difference, and I do not want to make them
appear any more definitive and official.  "This is *the* manpage for
release v2.15.1" is the message I do not want to give.

RE: Documentation Breakage at 2.5.6

[Randall S. Becker]

-----Original Message-----
On December 8, 2017 5:29 PM Junio C Hamano wrote:
>"Randall S. Becker" <rsbecker@nexbridge.com> writes:
>> One request to Junio: Would it be possible to tag the commits to align 
>> with the tags in the main repo? That way, I can build a nice little 
>> Jenkins job to automatically fetch the correct commit for man pages 
>> when packaging up a release.
>I am not interested in doing anything more than absolute minimum in the history that records generated cruft.  We already describe the mainline commit object names in the messages; perhaps that is >sufficient?
>        commit daa88a54a985ed1ef258800c742223c2a8f0caaa
>       Author: Junio C Hamano <gitster@pobox.com>
>       Date:   Wed Dec 6 10:04:03 2017 -0800
>
>           Autogenerated manpages for v2.15.1-354-g95ec6
>The primary reason why I do not want to tag them is because the tree the documentation sources were taken from is *not* the only thing that affects these autogenerated cruft.  The AsciiDoc toolchain that >happen to be installed on the box the day I ran the documentation tools is an obvious difference, and I do not want to make them appear any more definitive and official.  "This is *the* manpage for release >v2.15.1" is the message I do not want to give.

No worries. I will push on with trying to get asciidoc to build so that I can generate the man pages. That probably won't happen soon, so I'll keep MacGyvering. I do get generating is the better solution, but I would rather focus my own efforts on keeping up with git (that ports fairly easily and is essential to work life) than burning off $DAYJOB hours on asciidoc and/or xmlto.

Cheers,
Randall