Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab <mchehab@s-opensource.com>]

Em Thu, 2 Mar 2017 19:16:47 +0100
Markus Heiser <markus.heiser@darmarit.de> escreveu:

> > Am 02.03.2017 um 17:29 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> > 
> > Em Thu, 2 Mar 2017 17:13:25 +0100
> > Markus Heiser <markus.heiser@darmarit.de> escreveu:
> >   
> >>> Am 02.03.2017 um 16:49 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> >>>   
> >>>>> You can test it with virtualenv  https://virtualenv.pypa.io/en/stable/userguide/
> >>>>> 
> >>>>> In short:
> >>>>> 
> >>>>> $ cd kernel-src
> >>>>> $ virtualenv myenv
> >>>>> $ source myenv/bin/activate
> >>>>> $ pip install 'Sphinx==1.3.1'
> >>>>> $ make ....      
> >>>> 
> >>>> Hmm... at least here, building docs-next with Sphinx 1.3.1 inside a
> >>>> virtualenv is broken:
> >>>> 
> >>>> writing output... [ 16%] media/intro                                            
> >>>> Exception occurred:
> >>>> File "/devel/v4l/patchwork/myenv-1.3/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
> >>>>   assert not self.context, 'len(context) = %s' % len(self.context)
> >>>> AssertionError: len(context) = 1
> >>>> The full traceback has been saved in /tmp/sphinx-err-W7CPtT.log, if you want to report the issue to the developers.
> >>>> Please also report this if it was a user error, so that a better error message can be provided next time.
> >>>> A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> >>>> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> >>>> make[1]: *** [htmldocs] Error 1
> >>>> Makefile:1450: recipe for target 'htmldocs' failed
> >>>> make: *** [htmldocs] Error 2
> >>>> 
> >>>> Perhaps it is time for us to move minimal requirements to 1.4?    
> >>> 
> >>> Hmm... the same happened with 1.4.9 inside virtualenv. It built fine
> >>> with 1.5.2 (for htmldocs).
> >>> 
> >>> Thanks,
> >>> Mauro
> >>> 
> >>> -
> >>> 
> >>> This is the error log with 1.4:
> >>> 
> >>> # Sphinx version: 1.4.9    
> >> ....  
> >>> File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py", line 187, in walkabout
> >>>   visitor.dispatch_departure(self)
> >>> File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py", line 1895, in dispatch_departure
> >>>   return method(node)
> >>> File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
> >>>   assert not self.context, 'len(context) = %s' % len(self.context)
> >>> AssertionError: len(context) = 1    
> >> 
> >> I guess this is a error from newer docutils, so we have to fix docutils version also.
> >> 
> >> Sphinx itself specifies "docutils>=0.11"
> >> 
> >>  https://github.com/sphinx-doc/sphinx/blob/1.3.1/setup.py#L52
> >> 
> >> So I guess it uses a up to date docutils or the ducutils which are already installed system wide.  
> > 
> > The system-wide docutils is this one:
> > 
> > 	python2-docutils-0.13.1-3.fc25.noarch
> > 	python3-docutils-0.13.1-3.fc25.noarch  
> 
> Sorry my mistake, in the traceback we see
> 
> File "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/writers/_html_base.py", line 671, in depart_document
>   assert not self.context, 'len(context) = %s' % len(self.context)
> 
> ... which means: system wide installation does not matter.
> 
> > Btw, I tested also with virtualenv-3/pip3 and the same issue happens there.
> > 
> > Manually installing version 0.11 makes it to work again.  
> 
> Yes, its only about "docutils>=0.11" in Sphinx 1.3 dependencies. 
> In Sphinx 1.5 the error:
> 
>   https://github.com/sphinx-doc/sphinx/issues/3212
> 
> is fixed:
> 
>   https://github.com/tk0miya/sphinx/commit/73663f63672f22304810ce6bb9787490ad250127
> 
> But this will never be fixed downwards.

Crap. This kind of patch should be backported to Sphinx 1.3/1.4,
or Python's PIP repository should be fixed to require docutils
version to be either 0.11 or 0.12, if one installs version 1.3
or 1.4 of Sphinx.

The way it is, PIP repository is broken!

> All this is about semantic versioning. If you want to promise your
> builds, you have to name which versions of dependencies you support.
> I guess this is nothing new for kernel developers ;)

No. At the Kernel, we do everything possible to not break APIs.

If this were not the case, you wouldn't be able to run Sphinx 1.5 
with legacy Kernel versions ;-)

> The problem is, that PEP440 defines not only ONE version scheme
> 
>  """Some hard to read version identifiers are permitted by
>     this scheme in order to better accommodate the wide range
>     of versioning practices across existing public and private
>     Python projects."""
> 
> In practice, the python projects use slightly different schemes
> which not follow one rule like <main>.<compatible feature>.<patch> 
> From history packaging in Python is the hell, it becomes better, but
> the problem with slightly different version schemes still exist.

IMHO, the way Python and python libraries break compatibility is crazy.

Good packaging sense says that, if APIs can break on every single new
release (with seems to be the case of docutils), then a package
depending on such bad-developed library should require the exact
version(s) it is known to work.

When we're discussing about the docs toolchain, I mentioned that I
was afraid that the Python development model would cause this sort
of issues. Unfortunately, it seems that my concerns were pertinent :-(

> How can this info help us? Now we know, that we have to stick Sphinx
> and docutils by patch-levels we are willing to support. Or with
> your words ....
> 
> > Considering that Sphinx require a specific docutils package for it to
> > work, perhaps it is time for us to consider to use the virtenv
> > enchantments at make docs targets :-p  
> 
> 
> This is very easy, if we use a requiremts.txt file where we 
> stick the versions and run the sphinx in this build in a
> virtualenv which is build up by this requirements.txt.
> 
>   https://pip.pypa.io/en/stable/user_guide/#requirements-files
> 
> To summarize, I recommend a Makefile.sphinx cmd which does
> something like:
> 
>   virtualenv output/myenv
>   source output/myenv/bin/activate
>   pip install -r requirements.txt
>   sphinx-build ...
>   
> I guess this is something we should discuss with Jon, he
> is also familiar with it virtualenv. 

Yeah, making kernel build to depend on network can be a problem.

Maybe one way would be to have a sort of "prepare" script that
would be network-dependent, and will install whatever needed to
build the docs. If called, make *docs will use the virtualenv.
Otherwise, it would print a warning message saying that the doc
build is not reliable, but would try to use whatever installed
on the machine.

Thanks,
Mauro