From: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
To: Linux Doc Mailing List <linux-doc@vger.kernel.org>
Cc: "Nícolas F. R. A. Prado" <nfraprado@protonmail.com>,
"Jonathan Corbet" <corbet@lwn.net>,
"Mauro Carvalho Chehab" <mchehab+huawei@kernel.org>,
linux-kernel@vger.kernel.org
Subject: [PATCH v6 13/80] docs: automarkup.py: make it ready for Sphinx 3.1+
Date: Tue, 13 Oct 2020 13:53:28 +0200 [thread overview]
Message-ID: <57010ba2cb9e313270c6ff56702803c017cb168d.1602589096.git.mchehab+huawei@kernel.org> (raw)
In-Reply-To: <cover.1602589096.git.mchehab+huawei@kernel.org>
From: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
While Sphinx 2 used a single c:type role for struct, union, enum and
typedef, Sphinx 3 uses a specific role for each one.
To keep backward compatibility, detect the Sphinx version and use the
correct roles for that version.
Also, Sphinx 3 is more strict with its C domain and generated warnings,
exposing issues in the parsing.
To fix the warnings, make the C regexes use ASCII, ensure the
expressions only match the beginning of words and skip trying to
cross-reference C reserved words.
Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
---
Documentation/sphinx/automarkup.py | 69 ++++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 9 deletions(-)
diff --git a/Documentation/sphinx/automarkup.py b/Documentation/sphinx/automarkup.py
index a1b0f554cd82..fd1e927408ad 100644
--- a/Documentation/sphinx/automarkup.py
+++ b/Documentation/sphinx/automarkup.py
@@ -22,13 +22,34 @@ from itertools import chain
# :c:func: block (i.e. ":c:func:`mmap()`s" flakes out), so the last
# bit tries to restrict matches to things that won't create trouble.
#
-RE_function = re.compile(r'(([\w_][\w\d_]+)\(\))')
-RE_type = re.compile(r'(struct|union|enum|typedef)\s+([\w_][\w\d_]+)')
+RE_function = re.compile(r'\b(([a-zA-Z_]\w+)\(\))', flags=re.ASCII)
+
+#
+# Sphinx 2 uses the same :c:type role for struct, union, enum and typedef
+#
+RE_generic_type = re.compile(r'\b(struct|union|enum|typedef)\s+([a-zA-Z_]\w+)',
+ flags=re.ASCII)
+
+#
+# Sphinx 3 uses a different C role for each one of struct, union, enum and
+# typedef
+#
+RE_struct = re.compile(r'\b(struct)\s+([a-zA-Z_]\w+)', flags=re.ASCII)
+RE_union = re.compile(r'\b(union)\s+([a-zA-Z_]\w+)', flags=re.ASCII)
+RE_enum = re.compile(r'\b(enum)\s+([a-zA-Z_]\w+)', flags=re.ASCII)
+RE_typedef = re.compile(r'\b(typedef)\s+([a-zA-Z_]\w+)', flags=re.ASCII)
+
#
# Detects a reference to a documentation page of the form Documentation/... with
# an optional extension
#
-RE_doc = re.compile(r'Documentation(/[\w\-_/]+)(\.\w+)*')
+RE_doc = re.compile(r'\bDocumentation(/[\w\-_/]+)(\.\w+)*')
+
+#
+# Reserved C words that we should skip when cross-referencing
+#
+Skipnames = [ 'for', 'if', 'register', 'sizeof', 'struct', 'unsigned' ]
+
#
# Many places in the docs refer to common system calls. It is
@@ -48,9 +69,22 @@ def markup_refs(docname, app, node):
#
# Associate each regex with the function that will markup its matches
#
- markup_func = {RE_type: markup_c_ref,
- RE_function: markup_c_ref,
- RE_doc: markup_doc_ref}
+ markup_func_sphinx2 = {RE_doc: markup_doc_ref,
+ RE_function: markup_c_ref,
+ RE_generic_type: markup_c_ref}
+
+ markup_func_sphinx3 = {RE_doc: markup_doc_ref,
+ RE_function: markup_c_ref,
+ RE_struct: markup_c_ref,
+ RE_union: markup_c_ref,
+ RE_enum: markup_c_ref,
+ RE_typedef: markup_c_ref}
+
+ if sphinx.__version__[0] == '3':
+ markup_func = markup_func_sphinx3
+ else:
+ markup_func = markup_func_sphinx2
+
match_iterators = [regex.finditer(t) for regex in markup_func]
#
# Sort all references by the starting position in text
@@ -79,8 +113,24 @@ def markup_refs(docname, app, node):
# type_name) with an appropriate cross reference.
#
def markup_c_ref(docname, app, match):
- class_str = {RE_function: 'c-func', RE_type: 'c-type'}
- reftype_str = {RE_function: 'function', RE_type: 'type'}
+ class_str = {RE_function: 'c-func',
+ # Sphinx 2 only
+ RE_generic_type: 'c-type',
+ # Sphinx 3+ only
+ RE_struct: 'c-struct',
+ RE_union: 'c-union',
+ RE_enum: 'c-enum',
+ RE_typedef: 'c-type',
+ }
+ reftype_str = {RE_function: 'function',
+ # Sphinx 2 only
+ RE_generic_type: 'type',
+ # Sphinx 3+ only
+ RE_struct: 'struct',
+ RE_union: 'union',
+ RE_enum: 'enum',
+ RE_typedef: 'type',
+ }
cdom = app.env.domains['c']
#
@@ -89,7 +139,8 @@ def markup_c_ref(docname, app, match):
target = match.group(2)
target_text = nodes.Text(match.group(0))
xref = None
- if not (match.re == RE_function and target in Skipfuncs):
+ if not ((match.re == RE_function and target in Skipfuncs)
+ or (target in Skipnames)):
lit_text = nodes.literal(classes=['xref', 'c', class_str[match.re]])
lit_text += target_text
pxref = addnodes.pending_xref('', refdomain = 'c',
--
2.26.2
next prev parent reply other threads:[~2020-10-13 12:02 UTC|newest]
Thread overview: 108+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-13 11:53 [PATCH v6 00/80] htmldoc build fixes with Sphinx 2.x and 3.x Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 01/80] scripts: kernel-doc: add support for typedef enum Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 02/80] scripts: kernel-doc: make it more compatible with Sphinx 3.x Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 03/80] scripts: kernel-doc: use a less pedantic markup for funcs on " Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 04/80] scripts: kernel-doc: fix troubles with line counts Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 05/80] scripts: kernel-doc: reimplement -nofunction argument Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 06/80] scripts: kernel-doc: fix typedef identification Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 07/80] scripts: kernel-doc: don't mangle with parameter list Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 08/80] scripts: kernel-doc: allow passing desired Sphinx C domain dialect Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 09/80] scripts: kernel-doc: fix line number handling Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 10/80] scripts: kernel-doc: try to use c:function if possible Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 11/80] docs: cdomain.py: add support for a new Sphinx 3.1+ tag Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 12/80] docs: cdomain.py: extend it to handle new Sphinx 3.x tags Mauro Carvalho Chehab
2020-10-13 11:53 ` Mauro Carvalho Chehab [this message]
2020-10-13 11:53 ` [PATCH v6 14/80] docs: kerneldoc.py: append the name of the parsed doc file Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 15/80] docs: kerneldoc.py: add support for kerneldoc -nosymbol Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 16/80] media: docs: make CEC documents compatible with Sphinx 3.1+ Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 17/80] media: docs: make V4L documents more " Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 18/80] media: docs: make DVB " Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 19/80] media: docs: make MC " Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 20/80] media: docs: make RC " Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 21/80] media: cec-core.rst: don't use c:type for structs Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 22/80] docs: remove some replace macros like |struct foo| Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 23/80] docs: get rid of :c:type explicit declarations for structs Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 24/80] docs: trace-uses.rst: remove bogus c-domain tags Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 25/80] docs: it_IT: fix namespace collisions at locking.rst Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 26/80] docs: net: ieee802154.rst: fix C expressions Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 27/80] docs: genericirq.rst: don't document chip.c functions twice Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 28/80] docs: kernel-api.rst: drop kernel/irq/manage.c kernel-doc tag Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 29/80] docs: remove sound API duplication Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 30/80] docs: basics.rst: move kernel-doc workqueue markups to workqueue.rst Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 31/80] docs: scsi: target.rst: remove iSCSI transport class kernel-doc markup Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 32/80] docs: device_link.rst: remove duplicated kernel-doc include Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 33/80] docs: basics.rst: get rid of rcu kernel-doc macros Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 34/80] docs: pstore-blk.rst: fix kernel-doc tags Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 35/80] docs: fs: fscrypt.rst: get rid of :c:type: tags Mauro Carvalho Chehab
2020-10-13 17:25 ` Eric Biggers
2020-10-14 6:59 ` Mauro Carvalho Chehab
2020-10-14 21:59 ` Eric Biggers
2020-10-15 5:32 ` Mauro Carvalho Chehab
2020-10-15 16:36 ` Eric Biggers
2020-10-15 16:56 ` Mauro Carvalho Chehab
2020-10-15 21:26 ` Jonathan Corbet
2020-10-13 11:53 ` [PATCH v6 36/80] docs: devices.rst: get rid of :c:type macros Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 37/80] docs: sound: writing-an-alsa-driver.rst: get rid of :c:type Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 38/80] docs: block: blk-mq.rst: " Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 39/80] docs: writing-an-alsa-driver.rst: fix some bad c:func: markups Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 40/80] docs: fpga: replace :c:member: macros Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 41/80] docs: kgdb.rst: fix :c:type: usages Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 42/80] docs: libata.rst: fix a wrong usage of :c:type: tag Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 43/80] docs: infrastructure.rst: don't include firmware kernel-doc Mauro Carvalho Chehab
2020-10-13 11:53 ` [PATCH v6 44/80] docs: gpu: i915.rst: Fix several C duplication warnings Mauro Carvalho Chehab
2020-10-16 11:01 ` Joonas Lahtinen
2020-10-16 11:37 ` Mauro Carvalho Chehab
2020-10-16 11:39 ` Lionel Landwerlin
2020-10-16 11:50 ` Jani Nikula
2020-10-16 12:02 ` Lionel Landwerlin
2020-10-16 12:03 ` Lionel Landwerlin
2020-10-13 11:54 ` [PATCH v6 45/80] docs: devices.rst: fix a C reference markup Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 46/80] docs: it_IT: hacking.rst: fix a typo on a markup Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 47/80] docs: mei.rst: fix a C expression markup Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 48/80] docs: basics.rst: avoid duplicated C function declaration Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 49/80] docs: conf.py: fix c:function support with Sphinx 3.x Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 50/80] docs: conf.py: change the Sphinx 3.x+ text Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 51/80] docs: infrastructure.rst: exclude device_link_state from device.h Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 52/80] docs: zh_CN: amu.rst: fix document title markup Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 53/80] media: uAPI: buffer.rst: remove a left-over documentation Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 54/80] math64.h: kernel-docs: Convert some markups into normal comments Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 55/80] memblock: get rid of a :c:type leftover Mauro Carvalho Chehab
2020-10-13 12:02 ` Mike Rapoport
2020-10-13 11:54 ` [PATCH v6 56/80] dt-bindings: fix references to files converted to yaml Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 57/80] net: appletalk: Kconfig: Fix docs location Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 58/80] drivers: net: hamradio: fix document location Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 59/80] docs: powerpc: syscall64-abi.rst: fix a malformed table Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 60/80] block: bio: fix a warning at the kernel-doc markups Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 61/80] kunit: test.h: solve kernel-doc warnings Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 62/80] docs: bio: fix a kerneldoc markup Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 63/80] drivers: core: fix kernel-doc markup for dev_err_probe() Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 64/80] kunit: test.h: fix a bad kernel-doc markup Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 65/80] docs: amdgpu: fix a warning when building the documentation Mauro Carvalho Chehab
2020-10-13 13:19 ` Alex Deucher
2020-10-13 11:54 ` [PATCH v6 66/80] locking/refcount: document the new "oldp" pointer value Mauro Carvalho Chehab
2020-10-13 12:47 ` Peter Zijlstra
2020-10-13 11:54 ` [PATCH v6 67/80] usb: docs: document altmode register/unregister functions Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 68/80] nl80211: docs: add a description for s1g_cap parameter Mauro Carvalho Chehab
2020-10-13 16:45 ` Thomas Pedersen
2020-10-13 18:47 ` Johannes Berg
2020-10-13 20:41 ` Mauro Carvalho Chehab
2020-10-13 20:43 ` Johannes Berg
2020-10-13 11:54 ` [PATCH v6 69/80] IB/srpt: docs: add a description for cq_size member Mauro Carvalho Chehab
2020-10-15 4:32 ` Bart Van Assche
2020-10-13 11:54 ` [PATCH v6 70/80] rcu/tree: docs: document bkvcache new members at struct kfree_rcu_cpu Mauro Carvalho Chehab
2020-10-13 16:34 ` Paul E. McKenney
2020-10-13 20:46 ` Mauro Carvalho Chehab
2020-10-14 1:55 ` Paul E. McKenney
2020-10-13 11:54 ` [PATCH v6 71/80] Input: sparse-keymap: add a description for @sw Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 72/80] drm/amd/display: kernel-doc: document force_timing_sync Mauro Carvalho Chehab
2020-10-13 13:16 ` Alex Deucher
2020-10-13 11:54 ` [PATCH v6 73/80] drm: kernel-doc: drm_dp_helper.h: fix a typo Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 74/80] gpu: docs: amdgpu.rst: get rid of wrong kernel-doc markups Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 75/80] drm: kernel-doc: add description for a new function parameter Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 76/80] docs: virt: user_mode_linux_howto_v2.rst: fix a literal block markup Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 77/80] workqueue: fix a kernel-doc warning Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 78/80] mm/doc: fix a literal block markup Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 79/80] drm: drm_edid: remove a duplicated kernel-doc declaration Mauro Carvalho Chehab
2020-10-13 11:54 ` [PATCH v6 80/80] PM / devfreq: remove a duplicated kernel-doc markup Mauro Carvalho Chehab
2020-10-15 15:49 ` [PATCH v6 00/80] htmldoc build fixes with Sphinx 2.x and 3.x Daniel Vetter
2020-10-15 16:30 ` Mauro Carvalho Chehab
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=57010ba2cb9e313270c6ff56702803c017cb168d.1602589096.git.mchehab+huawei@kernel.org \
--to=mchehab+huawei@kernel.org \
--cc=corbet@lwn.net \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=nfraprado@protonmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).