* [PATCH 0/3] Add a way to build only media docs
@ 2016-08-06 12:00 Mauro Carvalho Chehab
2016-08-06 12:00 ` [PATCH 1/3] doc-rst: support additional Sphinx build config override Mauro Carvalho Chehab
` (4 more replies)
0 siblings, 5 replies; 8+ messages in thread
From: Mauro Carvalho Chehab @ 2016-08-06 12:00 UTC (permalink / raw)
To: Linux Media Mailing List, linux-doc, Jonathan Corbet
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab
Being able to build just the media docs is important for us due to several
reasons:
1) Media developers community hosts a copy of the media documentation at linuxtv.org
with the very latest under development documents;
2) Nitpicking to identify broken references is important to identify documentation gaps
that need to be addressed on future releases;
3) As media maintainers check patch per patch if a documentation gap is introduced, building
media documentation should be as fast as possible.
This patchset adds a media file adding nitpick support and an extra build target that will
compile only the media documentation. It also groups all media documentation into one
section on the main Kernel document, with is, IMHO, a good thing as we start adding more
stuff there.
Jon,
I'd love to see this patch merged early at the -rc cycle, in order to avoid merge
conflicts when people start converting other docbooks to Sphinx, as it touches
at the main Makefile and at the Sphinx common stuff. Also, as I'll need to patch my
build scripts to check for documentation issues with Sphinx, I need them on my
master branch, as otherwise my workflow will be broken until the next Kernel release.
So, If you're ok with this patch series, can you submit to Linus on early -rc? Or
if you prefer, I can do it myself, with your ack.
Thanks!
Mauro
PS.: I would prefer to have a more generic way to add support to build documentation
for only one subsystem, but, as we also need to load an extra python module to be
able to enable nitpick mode, I opted, for now, on not doing it too generic. We can rework
on it later, as other subsystems would need a similar feature.
Markus Heiser (1):
doc-rst: support additional Sphinx build config override
Mauro Carvalho Chehab (2):
doc-rst: add an option to build media documentation in nitpick mode
doc-rst: remove a bogus comment from Documentation/index.rst
Documentation/Makefile.sphinx | 10 ++++-
Documentation/conf.py | 9 ++++
Documentation/index.rst | 7 +--
Documentation/media/conf_nitpick.py | 85 +++++++++++++++++++++++++++++++++++++
Documentation/media/index.rst | 12 ++++++
Documentation/sphinx/load_config.py | 25 +++++++++++
Makefile | 6 +++
7 files changed, 146 insertions(+), 8 deletions(-)
create mode 100644 Documentation/media/conf_nitpick.py
create mode 100644 Documentation/media/index.rst
create mode 100644 Documentation/sphinx/load_config.py
--
2.7.4
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 1/3] doc-rst: support additional Sphinx build config override
2016-08-06 12:00 [PATCH 0/3] Add a way to build only media docs Mauro Carvalho Chehab
@ 2016-08-06 12:00 ` Mauro Carvalho Chehab
2016-08-06 12:00 ` [PATCH 2/3] doc-rst: add an option to build media documentation in nitpick mode Mauro Carvalho Chehab
` (3 subsequent siblings)
4 siblings, 0 replies; 8+ messages in thread
From: Mauro Carvalho Chehab @ 2016-08-06 12:00 UTC (permalink / raw)
To: Linux Media Mailing List, linux-doc, Jonathan Corbet
Cc: Markus Heiser, Mauro Carvalho Chehab, Mauro Carvalho Chehab
From: Markus Heiser <markus.heiser@darmarIT.de>
Load an additional configuration file into conf.py namespace.
The name of the configuration file is taken from the environment
SPHINX_CONF. The external configuration file extends (or overwrites) the
configuration values from the origin conf.py. With this you are
able to maintain *build themes*.
E.g. to create your own nit-picking *build theme*, create a file
Documentation/conf_nitpick.py::
nitpicky=True
nitpick_ignore = [
("c:func", "clock_gettime"),
...
]
and run make with SPHINX_CONF environment::
make SPHINX_CONF=conf_nitpick.py htmldocs
Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
Documentation/conf.py | 9 +++++++++
Documentation/sphinx/load_config.py | 25 +++++++++++++++++++++++++
2 files changed, 34 insertions(+)
create mode 100644 Documentation/sphinx/load_config.py
diff --git a/Documentation/conf.py b/Documentation/conf.py
index 96b7aa66c89c..d5027750503a 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -20,6 +20,8 @@ import os
# documentation root, use os.path.abspath to make it absolute, like shown here.
sys.path.insert(0, os.path.abspath('sphinx'))
+from load_config import loadConfig
+
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
@@ -419,3 +421,10 @@ pdf_documents = [
# line arguments.
kerneldoc_bin = '../scripts/kernel-doc'
kerneldoc_srctree = '..'
+
+
+# ------------------------------------------------------------------------------
+# Since loadConfig overwrites settings from the global namespace, it has to be
+# the last statement in the conf.py file
+# ------------------------------------------------------------------------------
+loadConfig(globals())
diff --git a/Documentation/sphinx/load_config.py b/Documentation/sphinx/load_config.py
new file mode 100644
index 000000000000..44bdd2271d13
--- /dev/null
+++ b/Documentation/sphinx/load_config.py
@@ -0,0 +1,25 @@
+# -*- coding: utf-8; mode: python -*-
+# pylint: disable=R0903, C0330, R0914, R0912, E0401
+
+import os
+from sphinx.util.pycompat import execfile_
+
+# ------------------------------------------------------------------------------
+def loadConfig(namespace):
+# ------------------------------------------------------------------------------
+
+ u"""Load an additional configuration file into *namespace*.
+
+ The name of the configuration file is taken from the environment
+ ``SPHINX_CONF``. The external configuration file extends (or overwrites) the
+ configuration values from the origin ``conf.py``. With this you are able to
+ maintain *build themes*. """
+
+ config_file = os.environ.get("SPHINX_CONF", None)
+ if config_file is not None and os.path.exists(config_file):
+ config_file = os.path.abspath(config_file)
+ config = namespace.copy()
+ config['__file__'] = config_file
+ execfile_(config_file, config)
+ del config['__file__']
+ namespace.update(config)
--
2.7.4
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 2/3] doc-rst: add an option to build media documentation in nitpick mode
2016-08-06 12:00 [PATCH 0/3] Add a way to build only media docs Mauro Carvalho Chehab
2016-08-06 12:00 ` [PATCH 1/3] doc-rst: support additional Sphinx build config override Mauro Carvalho Chehab
@ 2016-08-06 12:00 ` Mauro Carvalho Chehab
2016-08-06 12:00 ` [PATCH 3/3] doc-rst: remove a bogus comment from Documentation/index.rst Mauro Carvalho Chehab
` (2 subsequent siblings)
4 siblings, 0 replies; 8+ messages in thread
From: Mauro Carvalho Chehab @ 2016-08-06 12:00 UTC (permalink / raw)
To: Linux Media Mailing List, linux-doc, Jonathan Corbet
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, Michal Marek, linux-kbuild
While writing the media documentation, it is important to be able
to check if all symbols that are internal to the documentation were
cross-referenced, as this ensures that newer patches won't be
introducing documentation gaps.
So, add a way to build only the media documentation in nitpick
mode.
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
Documentation/Makefile.sphinx | 10 ++++-
Documentation/index.rst | 5 +--
Documentation/media/conf_nitpick.py | 85 +++++++++++++++++++++++++++++++++++++
Documentation/media/index.rst | 12 ++++++
Makefile | 6 +++
5 files changed, 112 insertions(+), 6 deletions(-)
create mode 100644 Documentation/media/conf_nitpick.py
create mode 100644 Documentation/media/index.rst
diff --git a/Documentation/Makefile.sphinx b/Documentation/Makefile.sphinx
index b10b6c598ae2..bbd7cd46f4a9 100644
--- a/Documentation/Makefile.sphinx
+++ b/Documentation/Makefile.sphinx
@@ -33,12 +33,17 @@ PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
KERNELDOC = $(srctree)/scripts/kernel-doc
KERNELDOC_CONF = -D kerneldoc_srctree=$(srctree) -D kerneldoc_bin=$(KERNELDOC)
-ALLSPHINXOPTS = -D version=$(KERNELVERSION) -D release=$(KERNELRELEASE) -d $(BUILDDIR)/.doctrees $(KERNELDOC_CONF) $(PAPEROPT_$(PAPER)) -c $(srctree)/$(src) $(SPHINXOPTS) $(srctree)/$(src)
+ALLSPHINXOPTS = -D version=$(KERNELVERSION) -D release=$(KERNELRELEASE) -d $(BUILDDIR)/.doctrees $(KERNELDOC_CONF) $(PAPEROPT_$(PAPER)) -c $(srctree)/$(src) $(SPHINXOPTS)
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
quiet_cmd_sphinx = SPHINX $@
- cmd_sphinx = BUILDDIR=$(BUILDDIR) $(SPHINXBUILD) -b $2 $(ALLSPHINXOPTS) $(BUILDDIR)/$2
+ cmd_sphinx = BUILDDIR=$(BUILDDIR) $(SPHINXBUILD) -b $2 $(ALLSPHINXOPTS) $(srctree)/$(src)$3 $(BUILDDIR)/$2
+
+# Build only the media docs, in nitpick mode
+mediadocs:
+ $(MAKE) BUILDDIR=$(BUILDDIR) SPHINX_CONF=media/conf_nitpick.py -f $(srctree)/Documentation/media/Makefile htmldocs
+ $(call cmd,sphinx,html,/media)
htmldocs:
$(MAKE) BUILDDIR=$(BUILDDIR) -f $(srctree)/Documentation/media/Makefile $@
@@ -70,6 +75,7 @@ cleandocs:
dochelp:
@echo ' Linux kernel internal documentation in different formats (Sphinx):'
@echo ' htmldocs - HTML'
+ @echo ' mediadocs - built only media books in HTML on nitpick mode'
@echo ' pdfdocs - PDF'
@echo ' epubdocs - EPUB'
@echo ' xmldocs - XML'
diff --git a/Documentation/index.rst b/Documentation/index.rst
index e0fc72963e87..02255c1806f6 100644
--- a/Documentation/index.rst
+++ b/Documentation/index.rst
@@ -14,10 +14,7 @@ Contents:
:maxdepth: 2
kernel-documentation
- media/media_uapi
- media/media_kapi
- media/dvb-drivers/index
- media/v4l-drivers/index
+ media/index
gpu/index
Indices and tables
diff --git a/Documentation/media/conf_nitpick.py b/Documentation/media/conf_nitpick.py
new file mode 100644
index 000000000000..9034d7f19753
--- /dev/null
+++ b/Documentation/media/conf_nitpick.py
@@ -0,0 +1,85 @@
+nitpicky=True
+
+# It is possible to run Sphinx in nickpick mode with:
+# make SPHINXOPTS=-n htmldocs
+# In such case, it will complain about lots of missing references that
+# 1) are just typedefs like: bool, __u32, etc;
+# 2) It will complain for things like: enum, NULL;
+# 3) It will complain for symbols that should be on different
+# books (but currently aren't ported to ReST)
+# The list below has a list of such symbols to be ignored in nitpick mode
+#
+nitpick_ignore = [
+ ("c:func", "clock_gettime"),
+ ("c:func", "close"),
+ ("c:func", "container_of"),
+ ("c:func", "determine_valid_ioctls"),
+ ("c:func", "ERR_PTR"),
+ ("c:func", "ioctl"),
+ ("c:func", "IS_ERR"),
+ ("c:func", "mmap"),
+ ("c:func", "open"),
+ ("c:func", "pci_name"),
+ ("c:func", "poll"),
+ ("c:func", "PTR_ERR"),
+ ("c:func", "read"),
+ ("c:func", "release"),
+ ("c:func", "set"),
+ ("c:func", "struct fd_set"),
+ ("c:func", "struct pollfd"),
+ ("c:func", "usb_make_path"),
+ ("c:func", "write"),
+ ("c:type", "atomic_t"),
+ ("c:type", "bool"),
+ ("c:type", "buf_queue"),
+ ("c:type", "device"),
+ ("c:type", "device_driver"),
+ ("c:type", "device_node"),
+ ("c:type", "enum"),
+ ("c:type", "file"),
+ ("c:type", "i2c_adapter"),
+ ("c:type", "i2c_board_info"),
+ ("c:type", "i2c_client"),
+ ("c:type", "ktime_t"),
+ ("c:type", "led_classdev_flash"),
+ ("c:type", "list_head"),
+ ("c:type", "lock_class_key"),
+ ("c:type", "module"),
+ ("c:type", "mutex"),
+ ("c:type", "pci_dev"),
+ ("c:type", "pdvbdev"),
+ ("c:type", "poll_table_struct"),
+ ("c:type", "s32"),
+ ("c:type", "s64"),
+ ("c:type", "sd"),
+ ("c:type", "spi_board_info"),
+ ("c:type", "spi_device"),
+ ("c:type", "spi_master"),
+ ("c:type", "struct fb_fix_screeninfo"),
+ ("c:type", "struct pollfd"),
+ ("c:type", "struct timeval"),
+ ("c:type", "struct video_capability"),
+ ("c:type", "u16"),
+ ("c:type", "u32"),
+ ("c:type", "u64"),
+ ("c:type", "u8"),
+ ("c:type", "union"),
+ ("c:type", "usb_device"),
+
+ ("cpp:type", "boolean"),
+ ("cpp:type", "fd"),
+ ("cpp:type", "fd_set"),
+ ("cpp:type", "int16_t"),
+ ("cpp:type", "NULL"),
+ ("cpp:type", "off_t"),
+ ("cpp:type", "pollfd"),
+ ("cpp:type", "size_t"),
+ ("cpp:type", "ssize_t"),
+ ("cpp:type", "timeval"),
+ ("cpp:type", "__u16"),
+ ("cpp:type", "__u32"),
+ ("cpp:type", "__u64"),
+ ("cpp:type", "uint16_t"),
+ ("cpp:type", "uint32_t"),
+ ("cpp:type", "video_system_t"),
+]
diff --git a/Documentation/media/index.rst b/Documentation/media/index.rst
new file mode 100644
index 000000000000..e85c557eeea3
--- /dev/null
+++ b/Documentation/media/index.rst
@@ -0,0 +1,12 @@
+Linux Media Subsystem Documentation
+===================================
+
+Contents:
+
+.. toctree::
+ :maxdepth: 2
+
+ media_uapi
+ media_kapi
+ dvb-drivers/index
+ v4l-drivers/index
diff --git a/Makefile b/Makefile
index 35603556023e..08ef6c1a807b 100644
--- a/Makefile
+++ b/Makefile
@@ -1439,6 +1439,12 @@ $(DOC_TARGETS): scripts_basic FORCE
$(Q)$(MAKE) $(build)=Documentation -f $(srctree)/Documentation/Makefile.sphinx $@
$(Q)$(MAKE) $(build)=Documentation/DocBook $@
+DOC_NITPIC_TARGETS := mediadocs
+PHONY += $(DOC_NITPIC_TARGETS)
+$(DOC_NITPIC_TARGETS): scripts_basic FORCE
+ $(Q)$(MAKE) $(build)=scripts build_docproc build_check-lc_ctype
+ $(Q)$(MAKE) $(build)=Documentation -f $(srctree)/Documentation/Makefile.sphinx $@
+
else # KBUILD_EXTMOD
###
--
2.7.4
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 3/3] doc-rst: remove a bogus comment from Documentation/index.rst
2016-08-06 12:00 [PATCH 0/3] Add a way to build only media docs Mauro Carvalho Chehab
2016-08-06 12:00 ` [PATCH 1/3] doc-rst: support additional Sphinx build config override Mauro Carvalho Chehab
2016-08-06 12:00 ` [PATCH 2/3] doc-rst: add an option to build media documentation in nitpick mode Mauro Carvalho Chehab
@ 2016-08-06 12:00 ` Mauro Carvalho Chehab
2016-08-06 12:06 ` [PATCH 0/3] Add a way to build only media docs Mauro Carvalho Chehab
2016-08-07 9:55 ` Markus Heiser
4 siblings, 0 replies; 8+ messages in thread
From: Mauro Carvalho Chehab @ 2016-08-06 12:00 UTC (permalink / raw)
To: Linux Media Mailing List, linux-doc, Jonathan Corbet
Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab
There's a bogus document at the documentation index saying
that there was nothing there, likely a left-over from the
initial Sphinx patches. Get rid of it!
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
Documentation/index.rst | 2 --
1 file changed, 2 deletions(-)
diff --git a/Documentation/index.rst b/Documentation/index.rst
index 02255c1806f6..bdd9525e05aa 100644
--- a/Documentation/index.rst
+++ b/Documentation/index.rst
@@ -6,8 +6,6 @@
Welcome to The Linux Kernel's documentation!
============================================
-Nothing for you to see here *yet*. Please move along.
-
Contents:
.. toctree::
--
2.7.4
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 0/3] Add a way to build only media docs
2016-08-06 12:00 [PATCH 0/3] Add a way to build only media docs Mauro Carvalho Chehab
` (2 preceding siblings ...)
2016-08-06 12:00 ` [PATCH 3/3] doc-rst: remove a bogus comment from Documentation/index.rst Mauro Carvalho Chehab
@ 2016-08-06 12:06 ` Mauro Carvalho Chehab
2016-08-07 9:55 ` Markus Heiser
4 siblings, 0 replies; 8+ messages in thread
From: Mauro Carvalho Chehab @ 2016-08-06 12:06 UTC (permalink / raw)
To: Linux Media Mailing List, linux-doc, Jonathan Corbet
Cc: Mauro Carvalho Chehab
Em Sat, 6 Aug 2016 09:00:31 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:
> Being able to build just the media docs is important for us due to several
> reasons:
>
> 1) Media developers community hosts a copy of the media documentation at linuxtv.org
> with the very latest under development documents;
>
> 2) Nitpicking to identify broken references is important to identify documentation gaps
> that need to be addressed on future releases;
>
> 3) As media maintainers check patch per patch if a documentation gap is introduced, building
> media documentation should be as fast as possible.
>
> This patchset adds a media file adding nitpick support and an extra build target that will
> compile only the media documentation. It also groups all media documentation into one
> section on the main Kernel document, with is, IMHO, a good thing as we start adding more
> stuff there.
>
> Jon,
>
> I'd love to see this patch merged early at the -rc cycle, in order to avoid merge
> conflicts when people start converting other docbooks to Sphinx, as it touches
> at the main Makefile and at the Sphinx common stuff. Also, as I'll need to patch my
> build scripts to check for documentation issues with Sphinx, I need them on my
> master branch, as otherwise my workflow will be broken until the next Kernel release.
>
> So, If you're ok with this patch series, can you submit to Linus on early -rc? Or
> if you prefer, I can do it myself, with your ack.
Forgot to comment, but those patches are applied against ustream master
branch. I applied them against my development tree at:
https://git.linuxtv.org//mchehab/experimental.git/log/?h=docs-next
Regards,
Mauro
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 0/3] Add a way to build only media docs
2016-08-06 12:00 [PATCH 0/3] Add a way to build only media docs Mauro Carvalho Chehab
` (3 preceding siblings ...)
2016-08-06 12:06 ` [PATCH 0/3] Add a way to build only media docs Mauro Carvalho Chehab
@ 2016-08-07 9:55 ` Markus Heiser
2016-08-07 12:38 ` Mauro Carvalho Chehab
4 siblings, 1 reply; 8+ messages in thread
From: Markus Heiser @ 2016-08-07 9:55 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Media Mailing List, linux-doc, Jonathan Corbet,
Mauro Carvalho Chehab
Am 06.08.2016 um 14:00 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> Being able to build just the media docs is important for us due to several
> reasons:
>
> 1) Media developers community hosts a copy of the media documentation at linuxtv.org
> with the very latest under development documents;
>
> 2) Nitpicking to identify broken references is important to identify documentation gaps
> that need to be addressed on future releases;
>
> 3) As media maintainers check patch per patch if a documentation gap is introduced,
Take into account: the gap is detected by "open references" (from the reST-header
files), but Sphinx tries to bind a ":ref:" to any anchor which fits (no matter where
it comes from). I mean, if we add more and more content or e.g. interpshinx objects,
we did not know which targets are available and the risk to get a *false bind* grows
with the content.
I don't know, may be in some future you will come into trouble, when you want
to refer targets outside of the ./media folder and *detect gaps* in the docs.
This need not be a problem, but you should be aware of, that your "test" is
only a test on open/closed links.
> building
> media documentation should be as fast as possible.
>
> This patchset adds a media file adding nitpick support and an extra build target that will
> compile only the media documentation. It also groups all media documentation into one
> section on the main Kernel document, with is, IMHO, a good thing as we start adding more
> stuff there.
>
> Jon,
>
> I'd love to see this patch merged early at the -rc cycle, in order to avoid merge
> conflicts when people start converting other docbooks to Sphinx, as it touches
> at the main Makefile and at the Sphinx common stuff. Also, as I'll need to patch my
> build scripts to check for documentation issues with Sphinx, I need them on my
> master branch, as otherwise my workflow will be broken until the next Kernel release.
>
> So, If you're ok with this patch series, can you submit to Linus on early -rc? Or
> if you prefer, I can do it myself, with your ack.
>
> Thanks!
> Mauro
>
> PS.: I would prefer to have a more generic way to add support to build documentation
> for only one subsystem, but, as we also need to load an extra python module to be
> able to enable nitpick mode, I opted, for now, on not doing it too generic. We can rework
> on it later, as other subsystems would need a similar feature.
Within my POC I called it "books", may it is better to call them "sphinx-projects",
but for the first and the concept it should good enough to stay with "books" / see
Makefile.reST [1] ::
# Sphinx projects, we call them *books* which is more common to authors.
BOOKS_FOLDER = $(obj)/Documentation
BOOKS=$(filter-out books/intro, \
$(patsubst $(BOOKS_FOLDER)/%/conf.py,books/%,$(wildcard $(BOOKS_FOLDER)/*/conf.py)) \
)
# fine grained targets
BOOKS_HTML = $(patsubst %,%.html, $(BOOKS))
BOOKS_CLEAN = $(patsubst %,%.clean, $(BOOKS))
BOOKS_MAN = $(patsubst %,%.man, $(BOOKS))
BOOKS_PDF = $(patsubst %,%.pdf, $(BOOKS))
[1] https://github.com/return42/sphkerneldoc/blob/master/doc/Makefile#L40
In words: A "book" (or sphinx-project) is a folder with a conf.py in
and the "folder-name" is the name of the related target ::
Documentation/*/conf.py
And the selective targets are
make books/[media|...].[html|man|pdf|clean|...]
By example, build only the html of media::
make books/media.html
The integration into the main Makefile is generic by adding the
wildcard target "books%"
modified Makefile
@@ -478,7 +478,7 @@ version_h := include/generated/uapi/linux/version.h
old_version_h := include/linux/version.h
no-dot-config-targets := clean mrproper distclean \
- cscope gtags TAGS tags help% %docs check% coccicheck \
+ cscope gtags TAGS tags help% %docs books% check% coccicheck \
$(version_h) headers_% archheaders archscripts \
kernelversion %src-pkg
+# more fine grained documentation targets
+books/%:
+ $(Q)$(MAKE) $(build)=Documentation -f Documentation/Makefile.reST $@
+
If you are interested in, I prepare a patch series for the Makefiles and the
conf.py. Any suggestions about the prefix "books"? ... I preferred "books" over
"sphinx-project" or similar, because it is pregnant and short (less to type).
-- Markus --
>
>
> Markus Heiser (1):
> doc-rst: support additional Sphinx build config override
>
> Mauro Carvalho Chehab (2):
> doc-rst: add an option to build media documentation in nitpick mode
> doc-rst: remove a bogus comment from Documentation/index.rst
>
> Documentation/Makefile.sphinx | 10 ++++-
> Documentation/conf.py | 9 ++++
> Documentation/index.rst | 7 +--
> Documentation/media/conf_nitpick.py | 85 +++++++++++++++++++++++++++++++++++++
> Documentation/media/index.rst | 12 ++++++
> Documentation/sphinx/load_config.py | 25 +++++++++++
> Makefile | 6 +++
> 7 files changed, 146 insertions(+), 8 deletions(-)
> create mode 100644 Documentation/media/conf_nitpick.py
> create mode 100644 Documentation/media/index.rst
> create mode 100644 Documentation/sphinx/load_config.py
>
> --
> 2.7.4
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 0/3] Add a way to build only media docs
2016-08-07 9:55 ` Markus Heiser
@ 2016-08-07 12:38 ` Mauro Carvalho Chehab
2016-08-08 9:20 ` Markus Heiser
0 siblings, 1 reply; 8+ messages in thread
From: Mauro Carvalho Chehab @ 2016-08-07 12:38 UTC (permalink / raw)
To: Markus Heiser
Cc: Linux Media Mailing List, linux-doc, Jonathan Corbet,
Mauro Carvalho Chehab
Em Sun, 7 Aug 2016 11:55:27 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:
>
> Am 06.08.2016 um 14:00 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
>
> > Being able to build just the media docs is important for us due to several
> > reasons:
> >
> > 1) Media developers community hosts a copy of the media documentation at linuxtv.org
> > with the very latest under development documents;
> >
> > 2) Nitpicking to identify broken references is important to identify documentation gaps
> > that need to be addressed on future releases;
> >
> > 3) As media maintainers check patch per patch if a documentation gap is introduced,
>
>
> Take into account: the gap is detected by "open references" (from the reST-header
> files), but Sphinx tries to bind a ":ref:" to any anchor which fits (no matter where
> it comes from). I mean, if we add more and more content or e.g. interpshinx objects,
> we did not know which targets are available and the risk to get a *false bind* grows
> with the content.
>
> I don't know, may be in some future you will come into trouble, when you want
> to refer targets outside of the ./media folder and *detect gaps* in the docs.
>
> This need not be a problem, but you should be aware of, that your "test" is
> only a test on open/closed links.
Yeah, we'll still have some references outside the media system. There
are already a few of them at the kABI books, and to get mistakes there,
we'll need to use nitpick mode for the entire stuff, with can be a
nightmare, specially since right now we have 300+ missing references just
inside the media books. We need to address those first, before being able
to address the external ones.
> > building
> > media documentation should be as fast as possible.
> >
> > This patchset adds a media file adding nitpick support and an extra build target that will
> > compile only the media documentation. It also groups all media documentation into one
> > section on the main Kernel document, with is, IMHO, a good thing as we start adding more
> > stuff there.
> >
> > Jon,
> >
> > I'd love to see this patch merged early at the -rc cycle, in order to avoid merge
> > conflicts when people start converting other docbooks to Sphinx, as it touches
> > at the main Makefile and at the Sphinx common stuff. Also, as I'll need to patch my
> > build scripts to check for documentation issues with Sphinx, I need them on my
> > master branch, as otherwise my workflow will be broken until the next Kernel release.
> >
> > So, If you're ok with this patch series, can you submit to Linus on early -rc? Or
> > if you prefer, I can do it myself, with your ack.
> >
> > Thanks!
> > Mauro
> >
> > PS.: I would prefer to have a more generic way to add support to build documentation
> > for only one subsystem, but, as we also need to load an extra python module to be
> > able to enable nitpick mode, I opted, for now, on not doing it too generic. We can rework
> > on it later, as other subsystems would need a similar feature.
>
> Within my POC I called it "books", may it is better to call them "sphinx-projects",
> but for the first and the concept it should good enough to stay with "books" / see
> Makefile.reST [1] ::
>
> # Sphinx projects, we call them *books* which is more common to authors.
The nomenclature can be very confusing, specially when we're grouping
different documents altogether.
The way I see, at the media subsystem, there are 4 books:
- uAPI - with has internally 5 parts;
- kAPI;
- V4L drivers;
- DVB drivers.
Yet, IMHO, the best is to put one master index.rst pointing to all of
them, so we actually have a media "book set", and just one "override"
conf.py for all of them. So, for now, I guess it is OK to group them
altogether when checking for references, although in reality, at least
in the first moment, I'm only interested on fixing bad references at
the uAPI and kAPI books. The other two books have lots of legacy stuff
that needs to be updated before even looking into any bad references there.
Yet, at the makefile, I don't mind calling the book set as "BOOK".
> BOOKS_FOLDER = $(obj)/Documentation
> BOOKS=$(filter-out books/intro, \
> $(patsubst $(BOOKS_FOLDER)/%/conf.py,books/%,$(wildcard $(BOOKS_FOLDER)/*/conf.py)) \
> )
>
> # fine grained targets
> BOOKS_HTML = $(patsubst %,%.html, $(BOOKS))
> BOOKS_CLEAN = $(patsubst %,%.clean, $(BOOKS))
> BOOKS_MAN = $(patsubst %,%.man, $(BOOKS))
> BOOKS_PDF = $(patsubst %,%.pdf, $(BOOKS))
>
> [1] https://github.com/return42/sphkerneldoc/blob/master/doc/Makefile#L40
>
> In words: A "book" (or sphinx-project) is a folder with a conf.py in
> and the "folder-name" is the name of the related target ::
>
> Documentation/*/conf.py
>
> And the selective targets are
>
> make books/[media|...].[html|man|pdf|clean|...]
>
> By example, build only the html of media::
>
> make books/media.html
>
> The integration into the main Makefile is generic by adding the
> wildcard target "books%"
>
> modified Makefile
> @@ -478,7 +478,7 @@ version_h := include/generated/uapi/linux/version.h
> old_version_h := include/linux/version.h
>
> no-dot-config-targets := clean mrproper distclean \
> - cscope gtags TAGS tags help% %docs check% coccicheck \
> + cscope gtags TAGS tags help% %docs books% check% coccicheck \
> $(version_h) headers_% archheaders archscripts \
> kernelversion %src-pkg
>
> +# more fine grained documentation targets
> +books/%:
> + $(Q)$(MAKE) $(build)=Documentation -f Documentation/Makefile.reST $@
> +
>
>
> If you are interested in, I prepare a patch series for the Makefiles and the
> conf.py.
Interesting approach. That would work for me. Not sure what the others
think about that.
Yet, my understanding is that we should have just one conf.py
for the "all-in-one" book (e. g. Documentation/index.rst), plus
additional ones to be used where we want to build standalone books
with nitpick mode enabled.
So, we should keep having the
make htmldocs
target that would build the all-in-one book, plus some other way to
build those books that would have a conf.py that would be adding the
extra nitpick options.
I really don't mind if we'll be compiling those with:
make books/media.html
or with something like:
make SPHINXDIRS="./media" htmldocs
Actually, the last syntax is better, IMHO, because people are already
familiar with something similar to that.
Provided that it would work and the patch would be simple enough to
be added on an early -rc Kernel (or alternatively, if it can be
sent via my master devel branch with Jon's ack).
> Any suggestions about the prefix "books"? ... I preferred "books" over
> "sphinx-project" or similar, because it is pregnant and short (less to type).
>
> -- Markus --
>
> >
> >
> > Markus Heiser (1):
> > doc-rst: support additional Sphinx build config override
> >
> > Mauro Carvalho Chehab (2):
> > doc-rst: add an option to build media documentation in nitpick mode
> > doc-rst: remove a bogus comment from Documentation/index.rst
> >
> > Documentation/Makefile.sphinx | 10 ++++-
> > Documentation/conf.py | 9 ++++
> > Documentation/index.rst | 7 +--
> > Documentation/media/conf_nitpick.py | 85 +++++++++++++++++++++++++++++++++++++
> > Documentation/media/index.rst | 12 ++++++
> > Documentation/sphinx/load_config.py | 25 +++++++++++
> > Makefile | 6 +++
> > 7 files changed, 146 insertions(+), 8 deletions(-)
> > create mode 100644 Documentation/media/conf_nitpick.py
> > create mode 100644 Documentation/media/index.rst
> > create mode 100644 Documentation/sphinx/load_config.py
> >
> > --
> > 2.7.4
> >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-media" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>
Thanks,
Mauro
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 0/3] Add a way to build only media docs
2016-08-07 12:38 ` Mauro Carvalho Chehab
@ 2016-08-08 9:20 ` Markus Heiser
0 siblings, 0 replies; 8+ messages in thread
From: Markus Heiser @ 2016-08-08 9:20 UTC (permalink / raw)
To: Mauro Carvalho Chehab
Cc: Linux Media Mailing List, linux-doc, Jonathan Corbet,
Mauro Carvalho Chehab
Am 07.08.2016 um 14:38 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> Em Sun, 7 Aug 2016 11:55:27 +0200
> Markus Heiser <markus.heiser@darmarit.de> escreveu:
>
>>
>> Am 06.08.2016 um 14:00 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
>>
>>> Being able to build just the media docs is important for us due to several
>>> reasons:
>>>
>>> 1) Media developers community hosts a copy of the media documentation at linuxtv.org
>>> with the very latest under development documents;
>>>
>>> 2) Nitpicking to identify broken references is important to identify documentation gaps
>>> that need to be addressed on future releases;
>>>
>>> 3) As media maintainers check patch per patch if a documentation gap is introduced,
>>
>>
>> Take into account: the gap is detected by "open references" (from the reST-header
>> files), but Sphinx tries to bind a ":ref:" to any anchor which fits (no matter where
>> it comes from). I mean, if we add more and more content or e.g. interpshinx objects,
>> we did not know which targets are available and the risk to get a *false bind* grows
>> with the content.
>>
>> I don't know, may be in some future you will come into trouble, when you want
>> to refer targets outside of the ./media folder and *detect gaps* in the docs.
>>
>> This need not be a problem, but you should be aware of, that your "test" is
>> only a test on open/closed links.
>
> Yeah, we'll still have some references outside the media system. There
> are already a few of them at the kABI books, and to get mistakes there,
> we'll need to use nitpick mode for the entire stuff, with can be a
> nightmare, specially since right now we have 300+ missing references just
> inside the media books. We need to address those first, before being able
> to address the external ones.
>
>>> building
>>> media documentation should be as fast as possible.
>>>
>>> This patchset adds a media file adding nitpick support and an extra build target that will
>>> compile only the media documentation. It also groups all media documentation into one
>>> section on the main Kernel document, with is, IMHO, a good thing as we start adding more
>>> stuff there.
>>>
>>> Jon,
>>>
>>> I'd love to see this patch merged early at the -rc cycle, in order to avoid merge
>>> conflicts when people start converting other docbooks to Sphinx, as it touches
>>> at the main Makefile and at the Sphinx common stuff. Also, as I'll need to patch my
>>> build scripts to check for documentation issues with Sphinx, I need them on my
>>> master branch, as otherwise my workflow will be broken until the next Kernel release.
>>>
>>> So, If you're ok with this patch series, can you submit to Linus on early -rc? Or
>>> if you prefer, I can do it myself, with your ack.
>>>
>>> Thanks!
>>> Mauro
>>>
>>> PS.: I would prefer to have a more generic way to add support to build documentation
>>> for only one subsystem, but, as we also need to load an extra python module to be
>>> able to enable nitpick mode, I opted, for now, on not doing it too generic. We can rework
>>> on it later, as other subsystems would need a similar feature.
>>
>> Within my POC I called it "books", may it is better to call them "sphinx-projects",
>> but for the first and the concept it should good enough to stay with "books" / see
>> Makefile.reST [1] ::
>>
>> # Sphinx projects, we call them *books* which is more common to authors.
>
> The nomenclature can be very confusing, specially when we're grouping
> different documents altogether.
>
> The way I see, at the media subsystem, there are 4 books:
> - uAPI - with has internally 5 parts;
> - kAPI;
> - V4L drivers;
> - DVB drivers.
>
> Yet, IMHO, the best is to put one master index.rst pointing to all of
> them, so we actually have a media "book set", and just one "override"
> conf.py for all of them. So, for now, I guess it is OK to group them
> altogether when checking for references, although in reality, at least
> in the first moment, I'm only interested on fixing bad references at
> the uAPI and kAPI books. The other two books have lots of legacy stuff
> that needs to be updated before even looking into any bad references there.
>
> Yet, at the makefile, I don't mind calling the book set as "BOOK".
>
>
>> BOOKS_FOLDER = $(obj)/Documentation
>> BOOKS=$(filter-out books/intro, \
>> $(patsubst $(BOOKS_FOLDER)/%/conf.py,books/%,$(wildcard $(BOOKS_FOLDER)/*/conf.py)) \
>> )
>>
>> # fine grained targets
>> BOOKS_HTML = $(patsubst %,%.html, $(BOOKS))
>> BOOKS_CLEAN = $(patsubst %,%.clean, $(BOOKS))
>> BOOKS_MAN = $(patsubst %,%.man, $(BOOKS))
>> BOOKS_PDF = $(patsubst %,%.pdf, $(BOOKS))
>>
>> [1] https://github.com/return42/sphkerneldoc/blob/master/doc/Makefile#L40
>>
>> In words: A "book" (or sphinx-project) is a folder with a conf.py in
>> and the "folder-name" is the name of the related target ::
>>
>> Documentation/*/conf.py
>>
>> And the selective targets are
>>
>> make books/[media|...].[html|man|pdf|clean|...]
>>
>> By example, build only the html of media::
>>
>> make books/media.html
>>
>> The integration into the main Makefile is generic by adding the
>> wildcard target "books%"
>>
>> modified Makefile
>> @@ -478,7 +478,7 @@ version_h := include/generated/uapi/linux/version.h
>> old_version_h := include/linux/version.h
>>
>> no-dot-config-targets := clean mrproper distclean \
>> - cscope gtags TAGS tags help% %docs check% coccicheck \
>> + cscope gtags TAGS tags help% %docs books% check% coccicheck \
>> $(version_h) headers_% archheaders archscripts \
>> kernelversion %src-pkg
>>
>> +# more fine grained documentation targets
>> +books/%:
>> + $(Q)$(MAKE) $(build)=Documentation -f Documentation/Makefile.reST $@
>> +
>>
>>
>> If you are interested in, I prepare a patch series for the Makefiles and the
>> conf.py.
>
> Interesting approach. That would work for me. Not sure what the others
> think about that.
>
> Yet, my understanding is that we should have just one conf.py
> for the "all-in-one" book (e. g. Documentation/index.rst), plus
> additional ones to be used where we want to build standalone books
> with nitpick mode enabled.
>
> So, we should keep having the
> make htmldocs
>
> target that would build the all-in-one book, plus some other way to
> build those books that would have a conf.py that would be adding the
> extra nitpick options.
>
> I really don't mind if we'll be compiling those with:
> make books/media.html
>
> or with something like:
> make SPHINXDIRS="./media" htmldocs
>
> Actually, the last syntax is better, IMHO, because people are already
> familiar with something similar to that.
>
> Provided that it would work and the patch would be simple enough to
> be added on an early -rc Kernel (or alternatively, if it can be
> sent via my master devel branch with Jon's ack).
Hi Mauro,
I agree with all, nevertheless I think, building a partial product should
be a make target. Anyway, I prepare a patch (with SPHINXDIRS="./media")
on top of your patch series.
-- Markus --
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2016-08-08 9:20 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-08-06 12:00 [PATCH 0/3] Add a way to build only media docs Mauro Carvalho Chehab
2016-08-06 12:00 ` [PATCH 1/3] doc-rst: support additional Sphinx build config override Mauro Carvalho Chehab
2016-08-06 12:00 ` [PATCH 2/3] doc-rst: add an option to build media documentation in nitpick mode Mauro Carvalho Chehab
2016-08-06 12:00 ` [PATCH 3/3] doc-rst: remove a bogus comment from Documentation/index.rst Mauro Carvalho Chehab
2016-08-06 12:06 ` [PATCH 0/3] Add a way to build only media docs Mauro Carvalho Chehab
2016-08-07 9:55 ` Markus Heiser
2016-08-07 12:38 ` Mauro Carvalho Chehab
2016-08-08 9:20 ` Markus Heiser
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.