Documentation: build failure with sphinx >= 3.0.0: exception: cannot import name 'c_funcptr_sig_re' from 'sphinx.domains.c' (was: Re: Kernel build failed ...SPHINX extension error)

Documentation: build failure with sphinx >= 3.0.0: exception: cannot import name 'c_funcptr_sig_re' from 'sphinx.domains.c' (was: Re: Kernel build failed ...SPHINX extension error)

[Salvatore Bonaccorso]

Hi 
On Thu, Apr 09, 2020 at 05:04:54AM +0530, Bhaskar Chowdhury wrote:
> On 13:25 Wed 08 Apr 2020, Jonathan Corbet wrote:
> > On Wed, 8 Apr 2020 17:07:05 +0530
> > Bhaskar Chowdhury <unixbhaskar@gmail.com> wrote:
> > 
> > > Extension error:
> > > Could not import extension cdomain (exception: cannot import name
> > > 'c_funcptr_sig_re' from 'sphinx.domains.c'
> > > (/usr/lib/python3.8/site-packages/sphinx/domains/c.py))
> > > Apr 08 16:48:46 enabling CJK for LaTeX builder
> > > Apr 08 16:48:46   CC      kernel/power/poweroff.o
> > > make[1]: *** [Documentation/Makefile:81: htmldocs] Error 2
> > > make: *** [Makefile:1549: htmldocs] Error 2
> > > make: *** Waiting for unfinished jobs....
> > 
> > This is weird, to say the least.  But I think the "python3.8" in the
> > message above says everything you need to know.  If you're running with
> > an unreleased version of Python, it's not entirely surprising that you
> > might run into trouble with a complex package.
> > 
> > jon
> 
> Thank you Jon...will investigate more and keep your suggestion in mind.

The problem is actually related to changes happening in Sphinx 3.0.0.
There is the followign issue filled upstream:

https://github.com/sphinx-doc/sphinx/issues/7421

'c_funcptr_sig_re' was removed upstream in sphinx v3.0.0b1 and so the
kernel documentation build fails. This is reproducible with a recent
sphinx version (in attached case it is 3.2.0):

$ make PYTHON=python3 xmldocs
  SPHINX  xmldocs --> file:///home/build/linux/Documentation/output/xml
make[2]: Nothing to be done for 'xml'.
Running Sphinx v3.2.0

Extension error:
Could not import extension cdomain (exception: cannot import name 'c_funcptr_sig_re' from 'sphinx.domains.c' (/usr/lib/python3/dist-packages/sphinx/domains/c.py))
make[1]: *** [Documentation/Makefile:115: xmldocs] Error 2
make: *** [Makefile:1655: xmldocs] Error 2

Distribution reports related to this issue:
https://bugs.debian.org/963636
https://bugs.archlinux.org/task/66178
https://bugs.archlinux.org/task/66156

Regards,
Salvatore

Re: Documentation: build failure with sphinx >= 3.0.0: exception: cannot import name 'c_funcptr_sig_re' from 'sphinx.domains.c'

[Salvatore Bonaccorso]

Hi,

On Sun, Aug 09, 2020 at 03:23:27PM +0200, Salvatore Bonaccorso wrote:
> Hi 
> On Thu, Apr 09, 2020 at 05:04:54AM +0530, Bhaskar Chowdhury wrote:
> > On 13:25 Wed 08 Apr 2020, Jonathan Corbet wrote:
> > > On Wed, 8 Apr 2020 17:07:05 +0530
> > > Bhaskar Chowdhury <unixbhaskar@gmail.com> wrote:
> > > 
> > > > Extension error:
> > > > Could not import extension cdomain (exception: cannot import name
> > > > 'c_funcptr_sig_re' from 'sphinx.domains.c'
> > > > (/usr/lib/python3.8/site-packages/sphinx/domains/c.py))
> > > > Apr 08 16:48:46 enabling CJK for LaTeX builder
> > > > Apr 08 16:48:46   CC      kernel/power/poweroff.o
> > > > make[1]: *** [Documentation/Makefile:81: htmldocs] Error 2
> > > > make: *** [Makefile:1549: htmldocs] Error 2
> > > > make: *** Waiting for unfinished jobs....
> > > 
> > > This is weird, to say the least.  But I think the "python3.8" in the
> > > message above says everything you need to know.  If you're running with
> > > an unreleased version of Python, it's not entirely surprising that you
> > > might run into trouble with a complex package.
> > > 
> > > jon
> > 
> > Thank you Jon...will investigate more and keep your suggestion in mind.
> 
> The problem is actually related to changes happening in Sphinx 3.0.0.
> There is the followign issue filled upstream:
> 
> https://github.com/sphinx-doc/sphinx/issues/7421
> 
> 'c_funcptr_sig_re' was removed upstream in sphinx v3.0.0b1 and so the
> kernel documentation build fails. This is reproducible with a recent
> sphinx version (in attached case it is 3.2.0):
> 
> $ make PYTHON=python3 xmldocs
>   SPHINX  xmldocs --> file:///home/build/linux/Documentation/output/xml
> make[2]: Nothing to be done for 'xml'.
> Running Sphinx v3.2.0
> 
> Extension error:
> Could not import extension cdomain (exception: cannot import name 'c_funcptr_sig_re' from 'sphinx.domains.c' (/usr/lib/python3/dist-packages/sphinx/domains/c.py))
> make[1]: *** [Documentation/Makefile:115: xmldocs] Error 2
> make: *** [Makefile:1655: xmldocs] Error 2
> 
> Distribution reports related to this issue:
> https://bugs.debian.org/963636
> https://bugs.archlinux.org/task/66178
> https://bugs.archlinux.org/task/66156

As a workaround to make the documentation build again (but known that
parts of the documentation will be broken), we could drop the cdomain
extension.

Regards,
Salvatore

From 9e81028c7e5f8f33f67a768b9c306536fd8e688f Mon Sep 17 00:00:00 2001
From: Salvatore Bonaccorso <carnil@debian.org>
Date: Wed, 12 Aug 2020 09:11:51 +0200
Subject: [PATCH] Documentation: Drop use of 'cdomain' sphinx extension module

'c_funcptr_sig_re' was removed upstream in sphinx v3.0.0b1 and so the
kernel documentation build fails sphinx 3.x.

As a (temporary) workaround to make the documentation build, the
'cdomain' sphinx extension can be removed from the Documentation build
configuration file, although some parts using the feature will be
broken.

https://github.com/sphinx-doc/sphinx/issues/7421 describes the
workaround.

Cc: Markus Heiser <markus.heiser@darmarit.de>
Cc: Justin M. Forbes <jforbes@fedoraproject.org>
Link: https://lore.kernel.org/linux-doc/20200809132327.GA145573@eldamar.local/
Link: https://bugs.archlinux.org/task/66178
Link: https://bugs.archlinux.org/task/66156
Link: https://bugs.debian.org/963636
Signed-off-by: Salvatore Bonaccorso <carnil@debian.org>
---
 Documentation/conf.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/conf.py b/Documentation/conf.py
index c503188880d9..b5b2be8eec22 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -36,7 +36,7 @@ needs_sphinx = '1.3'
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain',
+extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include',
               'kfigure', 'sphinx.ext.ifconfig', 'automarkup',
               'maintainers_include', 'sphinx.ext.autosectionlabel' ]
 
-- 
2.28.0

Re: Documentation: build failure with sphinx >= 3.0.0: exception: cannot import name 'c_funcptr_sig_re' from 'sphinx.domains.c'

[Markus Heiser]

Am 12.08.20 um 09:30 schrieb Salvatore Bonaccorso:
[..]
>> The problem is actually related to changes happening in Sphinx 3.0.0.
>> There is the followign issue filled upstream:
>>
>> https://github.com/sphinx-doc/sphinx/issues/7421
>>
>> 'c_funcptr_sig_re' was removed upstream in sphinx v3.0.0b1 and so the
>> kernel documentation build fails. This is reproducible with a recent
>> sphinx version (in attached case it is 3.2.0):
>>
>> $ make PYTHON=python3 xmldocs
>>    SPHINX  xmldocs --> file:///home/build/linux/Documentation/output/xml
>> make[2]: Nothing to be done for 'xml'.
>> Running Sphinx v3.2.0
>>
>> Extension error:
>> Could not import extension cdomain (exception: cannot import name 'c_funcptr_sig_re' from 'sphinx.domains.c' (/usr/lib/python3/dist-packages/sphinx/domains/c.py))
>> make[1]: *** [Documentation/Makefile:115: xmldocs] Error 2
>> make: *** [Makefile:1655: xmldocs] Error 2
>>
>> Distribution reports related to this issue:
>> https://bugs.debian.org/963636
>> https://bugs.archlinux.org/task/66178
>> https://bugs.archlinux.org/task/66156
> 
> As a workaround to make the documentation build again (but known that
> parts of the documentation will be broken), we could drop the cdomain
> extension.
> 
> Regards,
> Salvatore

@jon, do you have time to implement a patch?
.. sorry, I'am in a hurry :o

In the linked github issue you find also a patch that fixes a
cdomain.py.  I tested the patch (shee below) in the linux kernel.

   For me it works.

BTW: The patch of Documentation/sphinx/requirements.txt
was only nedded for a test.

   -- Markus --


---


diff --git a/Documentation/sphinx/cdomain.py b/Documentation/sphinx/cdomain.py
index cbac8e608dc4..65e15d48891e 100644
--- a/Documentation/sphinx/cdomain.py
+++ b/Documentation/sphinx/cdomain.py
@@ -31,16 +31,35 @@ u"""
         arguments types of function-like macros.

  """
+import re

  from docutils import nodes
  from docutils.parsers.rst import directives

  import sphinx
  from sphinx import addnodes
-from sphinx.domains.c import c_funcptr_sig_re, c_sig_re
  from sphinx.domains.c import CObject as Base_CObject
  from sphinx.domains.c import CDomain as Base_CDomain

+# C&P from commit https://github.com/return42/linuxdoc/commit/48f09de2
+# fixes 
https://github.com/sphinx-doc/sphinx/commit/0f49e30c51b5cc5055cda5b4b294c2dd9d1df573#r38750737
+
+# pylint: disable=invalid-name
+c_sig_re = re.compile(
+    r'''^([^(]*?)          # return type
+        ([\w:.]+)  \s*     # thing name (colon allowed for C++)
+        (?: \((.*)\) )?    # optionally arguments
+        (\s+const)? $      # const specifier
+    ''', re.VERBOSE)
+
+c_funcptr_sig_re = re.compile(
+    r'''^([^(]+?)          # return type
+        (\( [^()]+ \)) \s* # name in parentheses
+        \( (.*) \)         # arguments
+        (\s+const)? $      # const specifier
+    ''', re.VERBOSE)
+# pylint: enable=invalid-name
+
  __version__  = '1.0'

  # Get Sphinx version
diff --git a/Documentation/sphinx/requirements.txt 
b/Documentation/sphinx/requirements.txt
index 489f6626de67..f7486fd7ec85 100644
--- a/Documentation/sphinx/requirements.txt
+++ b/Documentation/sphinx/requirements.txt
@@ -1,3 +1,3 @@
  docutils
-Sphinx==2.4.4
+Sphinx==3.2.0
  sphinx_rtd_theme

Re: Documentation: build failure with sphinx >= 3.0.0: exception: cannot import name 'c_funcptr_sig_re' from 'sphinx.domains.c'

[Markus Heiser]

Am 12.08.20 um 10:21 schrieb Markus Heiser:
> 
> Am 12.08.20 um 09:30 schrieb Salvatore Bonaccorso:
> [..]
>>> The problem is actually related to changes happening in Sphinx 3.0.0.
>>> There is the followign issue filled upstream:
>>>
>>> https://github.com/sphinx-doc/sphinx/issues/7421
>>>
>>> 'c_funcptr_sig_re' was removed upstream in sphinx v3.0.0b1 and so the
>>> kernel documentation build fails. This is reproducible with a recent
>>> sphinx version (in attached case it is 3.2.0):
>>>
>>> $ make PYTHON=python3 xmldocs
>>>    SPHINX  xmldocs --> file:///home/build/linux/Documentation/output/xml
>>> make[2]: Nothing to be done for 'xml'.
>>> Running Sphinx v3.2.0
>>>
>>> Extension error:
>>> Could not import extension cdomain (exception: cannot import name 
>>> 'c_funcptr_sig_re' from 'sphinx.domains.c' 
>>> (/usr/lib/python3/dist-packages/sphinx/domains/c.py))
>>> make[1]: *** [Documentation/Makefile:115: xmldocs] Error 2
>>> make: *** [Makefile:1655: xmldocs] Error 2
>>>
>>> Distribution reports related to this issue:
>>> https://bugs.debian.org/963636
>>> https://bugs.archlinux.org/task/66178
>>> https://bugs.archlinux.org/task/66156
>>
>> As a workaround to make the documentation build again (but known that
>> parts of the documentation will be broken), we could drop the cdomain
>> extension.
>>
>> Regards,
>> Salvatore
> 
> @jon, do you have time to implement a patch?
> .. sorry, I'am in a hurry :o
> 
> In the linked github issue you find also a patch that fixes a
> cdomain.py.  I tested the patch (shee below) in the linux kernel.
> 
>    For me it works.

Sorry, I have to correct:

   a Patch needs more work

sphinx-doc has changed the way how domains are implemented [1].

-- Markus --

[1] https://github.com/sphinx-doc/sphinx/commit/0f49e30c

Re: Documentation: build failure with sphinx >= 3.0.0: exception: cannot import name 'c_funcptr_sig_re' from 'sphinx.domains.c'

[Jonathan Corbet]

On Wed, 12 Aug 2020 10:21:40 +0200
Markus Heiser <markus.heiser@darmarit.de> wrote:

> @jon, do you have time to implement a patch?
> .. sorry, I'am in a hurry :o

This is on my list ... but life is busy at the moment.  I would not be
distressed if somebody beat me to it.

jon