* [PATCH 0/7] add reST/sphinx-doc to linux documentation
@ 2016-06-06 16:32 Markus Heiser
2016-06-06 16:32 ` [PATCH 1/7] python: add scripts/site-packages Markus Heiser
` (8 more replies)
0 siblings, 9 replies; 23+ messages in thread
From: Markus Heiser @ 2016-06-06 16:32 UTC (permalink / raw)
To: corbet
Cc: jani.nikula, daniel.vetter, grant.likely, mchehab, keithp,
linux-kernel, linux-doc, hverkuil, Heiser, Markus
From: "Heiser, Markus" <markus.heiser@darmarIT.de>
Hi Jonathan, Jani, all -
I merged the work from sphkerneldoc POC [1], into branch (on v4.7-rc2)
git://github.com/return42/linux.git linux-doc-reST
The specification of the implementation is the (new) kernel-doc-HOWTO book which
is a part of the patch series (also online avaliable at [2]).
The kernel-doc-HOWTO book is a rewrite of the kernel-doc-nano-HOWTO.txt,
taking reST extension into account. It serves several purposes:
* user guide for kernel developers
* specification of the kernel-doc parser
* test case
The kernel_doc.py parser is a python implementation of the kernel-doc-HOWTO, the
kernel-doc perl script is not needed for reST builds. The python variant of the
kernel-doc parser supports the markup modes:
* kernel-doc (vintage)
* reST
This series also includes the reST directives describe in the kernel-doc-HOWTO:
* kernel-doc (uses/imports the parser from kernel_doc.py)
* flat-table (table notation with meaningfull diffs)
The Python/Sphinx extensions placed in the scripts/site-packages folder. With
this folder a minimal python infrastructure is provided.
The basic sphinx-build infrastructure consists of:
* Documentation/Makefile.reST & Documentation/conf.py
Makefile and basic 'sphinx config' file to build the various reST
documents and output formats. Provides the basic sphinx-doc build
infrastructure including the *sphinx-subprojects* feature. With this
feature each book can be build and distributed stand-alone.
* Documentation/books
In this folder, the books with reST markup are placed. To
provide *sphinx-subprojects*, each book has its one folder and
a (optional) Documentation/books/{book-name}.conf file
which *overwrites* the basic configuration from Documentation/conf.py
Any comments are welcome, but please read [2] first.
With regards
-- Markus --
[1] https://return42.github.io/sphkerneldoc
[2] https://return42.github.io/sphkerneldoc/books/kernel-doc-HOWTO
Heiser, Markus (7):
python: add scripts/site-packages
sphinx-doc: add basic sphinx-build infrastructure
kernel-doc-HOWTO: add kernel-doc specification
linuxdoc: add python package linuxdoc
kernel-doc parser: inital python implementation
kernel-doc directive: initial implementation
flat-table directive: initial implementation
Documentation/.gitignore | 2 +
Documentation/Makefile.reST | 111 +
Documentation/books/.gitignore | 0
Documentation/books/kernel-doc-HOWTO.conf | 200 ++
.../books/kernel-doc-HOWTO/all-in-a-tumble-src.rst | 12 +
.../books/kernel-doc-HOWTO/all-in-a-tumble.h | 271 ++
.../books/kernel-doc-HOWTO/all-in-a-tumble.rst | 11 +
Documentation/books/kernel-doc-HOWTO/csv_table.txt | 6 +
Documentation/books/kernel-doc-HOWTO/index.rst | 46 +
.../kernel-doc-HOWTO/kernel-doc-components.rst | 35 +
.../kernel-doc-HOWTO/kernel-doc-directive.rst | 199 ++
.../books/kernel-doc-HOWTO/kernel-doc-examples.rst | 16 +
.../books/kernel-doc-HOWTO/kernel-doc-intro.rst | 85 +
.../books/kernel-doc-HOWTO/kernel-doc-syntax.rst | 171 ++
.../kernel-doc-HOWTO/reST-kernel-doc-mode.rst | 120 +
Documentation/books/kernel-doc-HOWTO/refs.txt | 15 +
.../books/kernel-doc-HOWTO/table-markup.rst | 499 ++++
.../books/kernel-doc-HOWTO/test/parser_test.h | 332 +++
.../kernel-doc-HOWTO/vintage-kernel-doc-mode.rst | 100 +
Documentation/conf.py | 357 +++
Documentation/sphinx-static/theme_overrides.css | 45 +
Documentation/sphinx-tex/Makefile | 88 +
scripts/site-python/.gitignore | 1 +
scripts/site-python/README | 14 +
scripts/site-python/linuxdoc/__init__.py | 8 +
scripts/site-python/linuxdoc/kernel_doc.py | 2764 ++++++++++++++++++++
scripts/site-python/linuxdoc/rstFlatTable.py | 356 +++
scripts/site-python/linuxdoc/rstKernelDoc.py | 369 +++
28 files changed, 6233 insertions(+)
create mode 100644 Documentation/.gitignore
create mode 100644 Documentation/Makefile.reST
create mode 100644 Documentation/books/.gitignore
create mode 100644 Documentation/books/kernel-doc-HOWTO.conf
create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble-src.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.h
create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/csv_table.txt
create mode 100644 Documentation/books/kernel-doc-HOWTO/index.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-components.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-directive.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-examples.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-intro.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-syntax.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/reST-kernel-doc-mode.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/refs.txt
create mode 100644 Documentation/books/kernel-doc-HOWTO/table-markup.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/test/parser_test.h
create mode 100644 Documentation/books/kernel-doc-HOWTO/vintage-kernel-doc-mode.rst
create mode 100644 Documentation/conf.py
create mode 100644 Documentation/sphinx-static/theme_overrides.css
create mode 100644 Documentation/sphinx-tex/Makefile
create mode 100644 scripts/site-python/.gitignore
create mode 100644 scripts/site-python/README
create mode 100644 scripts/site-python/linuxdoc/__init__.py
create mode 100755 scripts/site-python/linuxdoc/kernel_doc.py
create mode 100755 scripts/site-python/linuxdoc/rstFlatTable.py
create mode 100755 scripts/site-python/linuxdoc/rstKernelDoc.py
--
v4.7-rc-2
^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 1/7] python: add scripts/site-packages
2016-06-06 16:32 [PATCH 0/7] add reST/sphinx-doc to linux documentation Markus Heiser
@ 2016-06-06 16:32 ` Markus Heiser
2016-06-06 16:32 ` [PATCH 2/7] sphinx-doc: add basic sphinx-build infrastructure Markus Heiser
` (7 subsequent siblings)
8 siblings, 0 replies; 23+ messages in thread
From: Markus Heiser @ 2016-06-06 16:32 UTC (permalink / raw)
To: corbet
Cc: jani.nikula, daniel.vetter, grant.likely, mchehab, keithp,
linux-kernel, linux-doc, hverkuil, Heiser, Markus
From: "Heiser, Markus" <markus.heiser@darmarIT.de>
With folder scripts/site-packages a minimal python infrastructure is
provided.
The ``site-python`` folder gather python extensions (packages and
modules). With a PYTHONPATH [1] or a sys.path [2] entry pointing to
this folder, these extensions can be imported from any python script. If
you are unfamiliar with python packages consult [3] at least.
[1] https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPATH
[2] https://docs.python.org/3/install/#inst-search-path
[3] https://docs.python.org/3/tutorial/modules.html#packages
Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
---
scripts/site-python/.gitignore | 1 +
scripts/site-python/README | 14 ++++++++++++++
2 files changed, 15 insertions(+)
create mode 100644 scripts/site-python/.gitignore
create mode 100644 scripts/site-python/README
diff --git a/scripts/site-python/.gitignore b/scripts/site-python/.gitignore
new file mode 100644
index 0000000..0d20b64
--- /dev/null
+++ b/scripts/site-python/.gitignore
@@ -0,0 +1 @@
+*.pyc
diff --git a/scripts/site-python/README b/scripts/site-python/README
new file mode 100644
index 0000000..04ebf7b
--- /dev/null
+++ b/scripts/site-python/README
@@ -0,0 +1,14 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+===========
+site-python
+===========
+
+The ``site-python`` folder gather python extensions (packages and modules).
+With a PYTHONPATH [1] or a sys.path [2] entry pointing to this folder, these
+extensions can be imported from any python script. If you are unfamiliar with
+python packages consult [3] at least.
+
+[1] https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPATH
+[2] https://docs.python.org/3/install/#inst-search-path
+[3] https://docs.python.org/3/tutorial/modules.html#packages
--
v4.7-rc-2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 2/7] sphinx-doc: add basic sphinx-build infrastructure
2016-06-06 16:32 [PATCH 0/7] add reST/sphinx-doc to linux documentation Markus Heiser
2016-06-06 16:32 ` [PATCH 1/7] python: add scripts/site-packages Markus Heiser
@ 2016-06-06 16:32 ` Markus Heiser
2016-06-06 16:32 ` [PATCH 3/7] kernel-doc-HOWTO: add kernel-doc specification Markus Heiser
` (6 subsequent siblings)
8 siblings, 0 replies; 23+ messages in thread
From: Markus Heiser @ 2016-06-06 16:32 UTC (permalink / raw)
To: corbet
Cc: jani.nikula, daniel.vetter, grant.likely, mchehab, keithp,
linux-kernel, linux-doc, hverkuil, Heiser, Markus
From: "Heiser, Markus" <markus.heiser@darmarIT.de>
Documentation/Makefile.reST and Documentation/conf.py
Makefile and basic 'sphinx config' file to build the various reST
documents and output formats. Provides the basic sphinx-doc build
infrastructure including the *sphinx-subprojects* feature. With this
feature each book can be build and distributed stand-alone.
Documentation/sphinx-static and Documentation/sphinx-tex
Paths that contain sphinx-doc custom static files (such as style
sheets).
Documentation/books
In this folder, the books with reST markup are placed. To
provide *sphinx-subprojects*, each book has its one folder and
a (optional) Documentation/books/{book-name}.conf file
which *overwrites* the basic configuration from Documentation/conf.py
The concept of the infrastructure was taken from the sphkerneldoc
project [1]
[1] http://return42.github.io/sphkerneldoc
Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
---
Documentation/.gitignore | 2 +
Documentation/Makefile.reST | 111 ++++++++
Documentation/books/.gitignore | 0
Documentation/conf.py | 357 ++++++++++++++++++++++++
Documentation/sphinx-static/theme_overrides.css | 45 +++
Documentation/sphinx-tex/Makefile | 88 ++++++
6 files changed, 603 insertions(+)
create mode 100644 Documentation/.gitignore
create mode 100644 Documentation/Makefile.reST
create mode 100644 Documentation/books/.gitignore
create mode 100644 Documentation/conf.py
create mode 100644 Documentation/sphinx-static/theme_overrides.css
create mode 100644 Documentation/sphinx-tex/Makefile
diff --git a/Documentation/.gitignore b/Documentation/.gitignore
new file mode 100644
index 0000000..1886a596
--- /dev/null
+++ b/Documentation/.gitignore
@@ -0,0 +1,2 @@
+cache/
+dist/
\ No newline at end of file
diff --git a/Documentation/Makefile.reST b/Documentation/Makefile.reST
new file mode 100644
index 0000000..67bdcd0
--- /dev/null
+++ b/Documentation/Makefile.reST
@@ -0,0 +1,111 @@
+# -*- coding: utf-8; mode: make -*-
+# Makefile for restructuredText (reST) content
+
+export PYTHONPATH := $(abspath ../scripts/site-python/):$(PYTHONPATH)
+
+ifndef obj
+ obj := .
+endif
+
+CACHE := $(obj)/cache
+BOOKS_FOLDER := books
+DIST := $(obj)/dist
+
+# External programs used
+
+SPHINXBUILD := sphinx-build
+PDFLATEX := pdflatex
+
+BOOKS := $(filter %/, $(wildcard $(BOOKS_FOLDER)/*/))
+BOOKS := $(BOOKS:%/=%)
+BOOKS_HTML := $(patsubst %, %.html, $(BOOKS))
+BOOKS_HTML_CLEAN := $(patsubst %, %.clean, $(BOOKS))
+
+# sphinx-doc setup
+SPHINXOPTS :=
+PAPER :=
+
+PAPEROPT_a4 = -D latex_paper_size=a4
+PAPEROPT_letter = -D latex_paper_size=letter
+ALLSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
+# the i18n builder cannot share the environment and doctrees with the others
+I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
+
+FMT = cat
+ifeq ($(shell which fmt >/dev/null 2>&1; echo $$?), 0)
+FMT = fmt
+endif
+
+# ------------------------------------------------------------------------------
+# requirements
+# ------------------------------------------------------------------------------
+
+.PHONY: sphinx-doc
+
+ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
+sphinx-doc: msg-SphinxDoc
+ $(error The '$(SPHINXBUILD)' command was not found)
+else
+sphinx-doc:
+ @:
+endif
+
+msg-SphinxDoc:
+ @echo "\nsphinx-doc:\n\n\
+ Make sure you have an updated Sphinx installed, grab it from\
+ http://sphinx-doc.org or install it from the python package\
+ manager (pip). \n\n\
+ On debian based OS these requirements are installed by::\n\n\
+ sudo apt-get install pip \n\n\
+ pip install Sphinx\n" | $(FMT)
+
+# ------------------------------------------------------------------------------
+# usage
+# ------------------------------------------------------------------------------
+
+.PHONY: help help-requirements
+help:
+ @echo "Please use \`make <target>' where <target> is one of ..."
+ @echo
+ @echo "all-HTML : builds all HTML outputs/targets"
+ @echo
+ @echo "make {book/*} : builds only the HTML of {book}, valid"
+ @echo " values for {book/*} are: \n\n $(BOOKS_HTML)" | $(FMT)
+ @echo
+ @echo "The output formats are placed into folder:"
+ @echo
+ @echo " $(DIST)"
+ @echo
+
+help-requirements: msg-SphinxDoc
+
+
+# ------------------------------------------------------------------------------
+# main targets
+# ------------------------------------------------------------------------------
+
+.PHONY: all-HTML clean-HTML $(BOOKS_HTML)
+
+all-HTML: sphinx-doc $(BOOKS_HTML)
+clean-HTML: clean-books.html
+
+$(DIST):
+ mkdir -p $(DIST)
+
+# ------------------------------------------------------------------------------
+# BOOKs
+# ------------------------------------------------------------------------------
+
+# reST-book --> HTML (sphinx-doc projects)
+# ----------------------------------------
+
+$(BOOKS_HTML): sphinx-doc | $(DIST)
+ @echo "building $@ ..."
+ SPHPROJ_CONF=$(patsubst %.html,%.conf,$@) $(SPHINXBUILD) -c . -b html -d $(CACHE)/doctrees/$(patsubst %.html,%,$@) $(patsubst %.html,%,$@) $(DIST)/$(patsubst %.html,%,$@)
+
+$(BOOKS_HTML_CLEAN):
+ rm -rf $(DIST)/$(patsubst %.clean,%,$@) $(CACHE)/doctrees/$(patsubst %.clean,%,$@)
+
+.PHONY: clean-books.html
+clean-books.html: $(BOOKS_HTML_CLEAN)
+
diff --git a/Documentation/books/.gitignore b/Documentation/books/.gitignore
new file mode 100644
index 0000000..e69de29
diff --git a/Documentation/conf.py b/Documentation/conf.py
new file mode 100644
index 0000000..50a7a67
--- /dev/null
+++ b/Documentation/conf.py
@@ -0,0 +1,357 @@
+# -*- coding: utf-8 -*-
+#
+# This is the common sphinx-build configuration used by the kernel documentation
+# book's build process. About config values consult:
+#
+# * http://www.sphinx-doc.org/en/stable/config.html
+#
+# Project (book) specific configuration is read from a file given by the
+# SPHPROJ_CONF environment (see function loadPrjConfig).
+
+import sys
+import os
+from os.path import join as pathjoin
+from os.path import abspath, dirname, splitext, basename, exists
+
+BASE_FOLDER = abspath(pathjoin(dirname(__file__)))
+
+# ------------------------------------------------------------------------------
+def loadPrjConfig():
+# ------------------------------------------------------------------------------
+
+ from sphinx.util.pycompat import execfile_
+ from sphinx.util.osutil import cd
+
+ config_file = os.environ.get("SPHPROJ_CONF", None)
+ if config_file is not None and exists(config_file):
+ config_file = abspath(config_file)
+ main_name = splitext(basename(config_file))[0]
+ config = globals().copy()
+ config.update({
+ "main_name" : main_name
+ })
+ config['__file__'] = config_file
+ execfile_(config_file, config)
+ globals().update(config)
+
+# ------------------------------------------------------------------------------
+# extensions
+# ------------------------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here or add it to the PYTHONPATH
+
+# ------------------------------------------------------------------------------
+# General configuration
+# ------------------------------------------------------------------------------
+
+# The default language to highlight source code in.
+highlight_language = "none"
+
+# If your documentation needs a minimal Sphinx version, state it here.
+#needs_sphinx = '1.0'
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = None
+
+# external links
+# --------------
+#
+# usage: lorem :man:`dvbv5-scan` ipsum
+
+extlinks = {
+ 'man' : ('http://manpages.ubuntu.com/cgi-bin/search.py?q=%s', ' ')
+ , 'deb' : ('http://packages.ubuntu.com/xenial/%s', ' ')
+ }
+
+# Intersphinx
+# -----------
+#
+# usage: lorem :ref:`dtv_get_frontend <linux:dtv_get_frontend>` ipsum
+
+intersphinx_mapping = {}
+intersphinx_mapping['linux'] = ('https://return42.github.io/sphkerneldoc/linux_src_doc/', None)
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+ "linuxdoc.rstFlatTable"
+ , "linuxdoc.rstKernelDoc"
+ , 'sphinx.ext.autodoc'
+ , 'sphinx.ext.extlinks'
+ #, 'sphinx.ext.autosummary'
+ #, 'sphinx.ext.doctest'
+ , 'sphinx.ext.todo'
+ , 'sphinx.ext.coverage'
+ #, 'sphinx.ext.pngmath'
+ #, 'sphinx.ext.mathjax'
+ , 'sphinx.ext.viewcode'
+ , 'sphinx.ext.intersphinx'
+ , 'sphinx.ext.ifconfig'
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['sphinx-templates']
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+# source_suffix = ['.rst', '.md']
+source_suffix = '.rst'
+
+# The encoding of source files.
+source_encoding = 'utf-8-sig'
+
+# The master toctree document.
+master_doc = 'index'
+
+# There are two options for replacing |today|: either, you set today to some
+# non-false value, then it is used:
+#today = ''
+# Else, today_fmt is used as the format for a strftime call.
+#today_fmt = '%B %d, %Y'
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+exclude_patterns = ['sphinx-tex', 'sphinx-static', 'sphinx-templates']
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = 'sphinx'
+
+# A list of ignored prefixes for module index sorting.
+#modindex_common_prefix = []
+
+# If true, keep warnings as "system message" paragraphs in the built documents.
+keep_warnings = False
+
+# If true, `todo` and `todoList` produce output, else they produce nothing.
+todo_include_todos = True
+
+# If true, fatal errors (like missing function descripions) raise an
+# error. Default: True
+kernel_doc_raise_error = False
+
+# ------------------------------------------------------------------------------
+# Options for HTML output
+# ------------------------------------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages. See the documentation for
+# a list of builtin themes.
+html_theme = 'sphinx_rtd_theme'
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further. For a list of options available for each theme, see the
+# documentation.
+#html_theme_options = {}
+
+# Add any paths that contain custom themes here, relative to this directory.
+#html_theme_path = []
+
+# The name of an image file (relative to this directory) to place at the top
+# of the sidebar.
+#html_logo = pathjoin(BASE_FOLDER, "logo.gif")
+
+# The name of an image file (within the static path) to use as favicon of the
+# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
+# pixels large.
+#html_favicon = None
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['sphinx-static']
+
+html_context = {
+ 'css_files': [
+ '_static/theme_overrides.css',
+ ],
+}
+
+# Add any extra paths that contain custom files (such as robots.txt or
+# .htaccess) here, relative to this directory. These files are copied
+# directly to the root of the documentation.
+#html_extra_path = []
+
+# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
+# using the given strftime format.
+#html_last_updated_fmt = '%b %d, %Y'
+
+# If true, SmartyPants will be used to convert quotes and dashes to
+# typographically correct entities.
+#html_use_smartypants = True
+
+# Custom sidebar templates, maps document names to template names.
+#html_sidebars = {}
+
+# Additional templates that should be rendered to pages, maps page names to
+# template names.
+#html_additional_pages = {}
+
+# If false, no module index is generated.
+#html_domain_indices = True
+
+# If false, no index is generated.
+#html_use_index = True
+
+# If true, the index is split into individual pages for each letter.
+#html_split_index = False
+
+# If true, links to the reST sources are added to the pages.
+html_show_sourcelink = True
+
+# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
+html_show_sphinx = False
+
+# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
+html_show_copyright = True
+
+# If true, an OpenSearch description file will be output, and all pages will
+# contain a <link> tag referring to it. The value of this option must be the
+# base URL from which the finished HTML is served.
+#html_use_opensearch = ''
+
+# This is the file name suffix for HTML files (e.g. ".xhtml").
+#html_file_suffix = None
+
+# Language to be used for generating the HTML full-text search index.
+# Sphinx supports the following languages:
+# 'da', 'de', 'en', 'es', 'fi', 'fr', 'hu', 'it', 'ja'
+# 'nl', 'no', 'pt', 'ro', 'ru', 'sv', 'tr'
+html_search_language = 'en'
+
+# A dictionary with options for the search language support, empty by default.
+# Now only 'ja' uses this config value
+#html_search_options = {'type': 'default'}
+
+# The name of a javascript file (relative to the configuration directory) that
+# implements a search results scorer. If empty, the default will be used.
+#html_search_scorer = 'scorer.js'
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'Kernel-Doc'
+
+# ------------------------------------------------------------------------------
+# Options for LaTeX output
+# ------------------------------------------------------------------------------
+
+# The name of an image file (relative to this directory) to place at the top of
+# the title page.
+#latex_logo = pathjoin(BASE_FOLDER, "sphinx-tex", "logo.png")
+
+#latex_additional_files = [
+# pathjoin(BASE_FOLDER, "sphinx-tex", "Makefile")
+#]
+
+# If true, show page references after internal links.
+# latex_show_pagerefs = False
+
+# If true, show URL addresses after external links.
+#latex_show_urls = False
+
+# Documents to append as an appendix to all manuals.
+#latex_appendices = []
+
+# If false, no module index is generated.
+#latex_domain_indices = True
+
+# ------------------------------------------------------------------------------
+# Options for manual page output
+# ------------------------------------------------------------------------------
+
+# If true, show URL addresses after external links.
+#man_show_urls = False
+
+# ------------------------------------------------------------------------------
+# Options for Texinfo output
+# ------------------------------------------------------------------------------
+
+# Documents to append as an appendix to all manuals.
+#texinfo_appendices = []
+
+# If false, no module index is generated.
+#texinfo_domain_indices = True
+
+# How to display URL addresses: 'footnote', 'no', or 'inline'.
+#texinfo_show_urls = 'footnote'
+
+# If true, do not generate a @detailmenu in the "Top" node's menu.
+#texinfo_no_detailmenu = False
+
+# ------------------------------------------------------------------------------
+# Options for Epub output
+# ------------------------------------------------------------------------------
+
+# Bibliographic Dublin Core info.
+# epub_title = project
+# epub_author = author
+# epub_publisher = author
+# epub_copyright = copyright
+
+# The basename for the epub file. It defaults to the project name.
+#epub_basename = project
+
+# The HTML theme for the epub output. Since the default themes are not
+# optimized for small screen space, using the same theme for HTML and epub
+# output is usually not wise. This defaults to 'epub', a theme designed to save
+# visual space.
+#epub_theme = 'epub'
+
+# The language of the text. It defaults to the language option
+# or 'en' if the language is not set.
+#epub_language = ''
+
+# The scheme of the identifier. Typical schemes are ISBN or URL.
+#epub_scheme = ''
+
+# The unique identifier of the text. This can be a ISBN number
+# or the project homepage.
+#epub_identifier = ''
+
+# A unique identification for the text.
+#epub_uid = ''
+
+# A tuple containing the cover image and cover page html template filenames.
+#epub_cover = ()
+
+# A sequence of (type, uri, title) tuples for the guide element of content.opf.
+#epub_guide = ()
+
+# HTML files that should be inserted before the pages created by sphinx.
+# The format is a list of tuples containing the path and title.
+#epub_pre_files = []
+
+# HTML files that should be inserted after the pages created by sphinx.
+# The format is a list of tuples containing the path and title.
+#epub_post_files = []
+
+# A list of files that should not be packed into the epub file.
+epub_exclude_files = ['search.html']
+
+# The depth of the table of contents in toc.ncx.
+#epub_tocdepth = 3
+
+# Allow duplicate toc entries.
+#epub_tocdup = True
+
+# Choose between 'default' and 'includehidden'.
+#epub_tocscope = 'default'
+
+# Fix unsupported image types using the Pillow.
+#epub_fix_images = False
+
+# Scale large images.
+#epub_max_image_width = 0
+
+# How to display URL addresses: 'footnote', 'no', or 'inline'.
+#epub_show_urls = 'inline'
+
+# If false, no index is generated.
+#epub_use_index = True
+
+# ------------------------------------------------------------------------------
+loadPrjConfig()
+# ------------------------------------------------------------------------------
diff --git a/Documentation/sphinx-static/theme_overrides.css b/Documentation/sphinx-static/theme_overrides.css
new file mode 100644
index 0000000..cd1277c
--- /dev/null
+++ b/Documentation/sphinx-static/theme_overrides.css
@@ -0,0 +1,45 @@
+/* -*- coding: utf-8; mode: css -*-
+ *
+ * This is a customization of the "Read The Doc" (RTD) theme.
+ *
+ */
+
+@media screen {
+
+ /* inline literal: drop the borderbox and red color */
+
+ code,.rst-content tt,.rst-content code {
+ color: inherit;
+ border: none;
+ background: inherit;
+ font-size: 85%;
+ }
+
+ .rst-content tt.literal,.rst-content tt.literal,.rst-content code.literal {
+ color: inherit;
+ }
+
+ /* captions:
+ *
+ * - captions should have 100% (not 85%) font size
+ * - hide the permalink symbol as long as link is not hovered
+ */
+
+ caption, .wy-table caption, .rst-content table.field-list caption {
+ font-size: 100%;
+ }
+
+ caption a.headerlink { opacity: 0; }
+ caption a.headerlink:hover { opacity: 1; }
+
+ /* table:
+ *
+ * - Sequences of whitespace should collapse into a single whitespace.
+ * - make the overflow auto (scrollbar if needed)
+ * - align caption "left" ("center" is unsuitable on vast tables)
+ */
+
+ .wy-table-responsive table td { white-space: normal; }
+ .wy-table-responsive { overflow: auto; }
+ .rst-content table.docutils caption { text-align: left; font-size: 100%; }
+}
diff --git a/Documentation/sphinx-tex/Makefile b/Documentation/sphinx-tex/Makefile
new file mode 100644
index 0000000..d2f7621
--- /dev/null
+++ b/Documentation/sphinx-tex/Makefile
@@ -0,0 +1,88 @@
+# Makefile for Sphinx LaTeX output
+
+ALLDOCS = $(basename $(wildcard *.tex))
+ALLPDF = $(addsuffix .pdf,$(ALLDOCS))
+ALLDVI = $(addsuffix .dvi,$(ALLDOCS))
+
+# Prefix for archive names
+ARCHIVEPRREFIX =
+
+# Additional LaTeX options
+LATEXOPTS = # -interaction=batchmode
+LATEX_ENV = max_print_line=120
+
+# format: pdf or dvi
+FMT = pdf
+
+LATEX = $(LATEX_ENV) latex
+PDFLATEX = $(LATEX_ENV) FIXME_xelatex
+MAKEINDEX = $(LATEX_ENV) makeindex
+
+all: $(ALLPDF)
+
+all-pdf: $(ALLPDF)
+
+dist-pdf:
+ mkdir -p $(DIST_BOOKS)
+ cp $(ALLPDF) $(DIST_BOOKS)
+
+all-dvi: $(ALLDVI)
+
+all-ps: all-dvi
+ for f in *.dvi; do dvips $$f; done
+
+all-pdf-ja:
+ for f in *.pdf *.png *.gif *.jpg *.jpeg; do extractbb $$f; done
+ for f in *.tex; do platex -kanji=utf8 $(LATEXOPTS) $$f; done
+ for f in *.tex; do platex -kanji=utf8 $(LATEXOPTS) $$f; done
+ for f in *.tex; do platex -kanji=utf8 $(LATEXOPTS) $$f; done
+ -for f in *.idx; do mendex -U -f -d "`basename $$f .idx`.dic" -s python.ist $$f; done
+ for f in *.tex; do platex -kanji=utf8 $(LATEXOPTS) $$f; done
+ for f in *.tex; do platex -kanji=utf8 $(LATEXOPTS) $$f; done
+ for f in *.dvi; do dvipdfmx $$f; done
+
+zip: all-$(FMT)
+ mkdir $(ARCHIVEPREFIX)docs-$(FMT)
+ cp $(ALLPDF) $(ARCHIVEPREFIX)docs-$(FMT)
+ zip -q -r -9 $(ARCHIVEPREFIX)docs-$(FMT).zip $(ARCHIVEPREFIX)docs-$(FMT)
+ rm -r $(ARCHIVEPREFIX)docs-$(FMT)
+
+tar: all-$(FMT)
+ mkdir $(ARCHIVEPREFIX)docs-$(FMT)
+ cp $(ALLPDF) $(ARCHIVEPREFIX)docs-$(FMT)
+ tar cf $(ARCHIVEPREFIX)docs-$(FMT).tar $(ARCHIVEPREFIX)docs-$(FMT)
+ rm -r $(ARCHIVEPREFIX)docs-$(FMT)
+
+gz: tar
+ gzip -9 < $(ARCHIVEPREFIX)docs-$(FMT).tar > $(ARCHIVEPREFIX)docs-$(FMT).tar.gz
+
+bz2: tar
+ bzip2 -9 -k $(ARCHIVEPREFIX)docs-$(FMT).tar
+
+xz: tar
+ xz -9 -k $(ARCHIVEPREFIX)docs-$(FMT).tar
+
+# The number of LaTeX runs is quite conservative, but I don't expect it
+# to get run often, so the little extra time won't hurt.
+%.dvi: %.tex
+ $(LATEX) $(LATEXOPTS) '$<'
+ $(LATEX) $(LATEXOPTS) '$<'
+ $(LATEX) $(LATEXOPTS) '$<'
+ -$(MAKEINDEX) -s python.ist '$(basename $<).idx'
+ $(LATEX) $(LATEXOPTS) '$<'
+ $(LATEX) $(LATEXOPTS) '$<'
+
+%.pdf: %.tex
+ $(PDFLATEX) $(LATEXOPTS) '$<'
+ $(PDFLATEX) $(LATEXOPTS) '$<'
+ $(PDFLATEX) $(LATEXOPTS) '$<'
+ -$(MAKEINDEX) -s python.ist '$(basename $<).idx'
+ $(PDFLATEX) $(LATEXOPTS) '$<'
+ $(PDFLATEX) $(LATEXOPTS) '$<'
+
+clean:
+ rm -f *.log *.ind *.aux *.toc *.syn *.idx *.out *.ilg *.pla *.ps *.tar *.tar.gz *.tar.bz2 *.tar.xz $(ALLPDF) $(ALLDVI)
+
+.PHONY: all all-pdf all-dvi all-ps clean zip tar gz bz2 xz
+.PHONY: all-pdf-ja
+
--
v4.7-rc-2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 3/7] kernel-doc-HOWTO: add kernel-doc specification
2016-06-06 16:32 [PATCH 0/7] add reST/sphinx-doc to linux documentation Markus Heiser
2016-06-06 16:32 ` [PATCH 1/7] python: add scripts/site-packages Markus Heiser
2016-06-06 16:32 ` [PATCH 2/7] sphinx-doc: add basic sphinx-build infrastructure Markus Heiser
@ 2016-06-06 16:32 ` Markus Heiser
2016-06-09 18:05 ` Randy Dunlap
2016-06-06 16:32 ` [PATCH 4/7] linuxdoc: add python package linuxdoc Markus Heiser
` (5 subsequent siblings)
8 siblings, 1 reply; 23+ messages in thread
From: Markus Heiser @ 2016-06-06 16:32 UTC (permalink / raw)
To: corbet
Cc: jani.nikula, daniel.vetter, grant.likely, mchehab, keithp,
linux-kernel, linux-doc, hverkuil, Heiser, Markus
From: "Heiser, Markus" <markus.heiser@darmarIT.de>
The kernel-doc-HOWTO book is a rewrite of the kernel-doc-nano-HOWTO.txt,
taking reST extension into account.
The kernel-doc-HOWTO is a user guide for kernel developers that also
serves as a specification.
The format for this documentation is called the *kernel-doc* format. With kernel
version 4.x the restructuredText (reST_) markup was added to the *kernel-doc*
format. The kernel-doc parser supports the markup modes:
* kernel-doc (vintage)
* reST
This document gives hints on how to use these markups and how to refer
and extract documentation from source files.
A online kernel-doc-HOWTO is available at [1]
[1] http://return42.github.io/sphkerneldoc/books/kernel-doc-HOWTO
Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
---
Documentation/books/kernel-doc-HOWTO.conf | 200 +++++++++
.../books/kernel-doc-HOWTO/all-in-a-tumble-src.rst | 12 +
.../books/kernel-doc-HOWTO/all-in-a-tumble.h | 271 +++++++++++
.../books/kernel-doc-HOWTO/all-in-a-tumble.rst | 11 +
Documentation/books/kernel-doc-HOWTO/csv_table.txt | 6 +
Documentation/books/kernel-doc-HOWTO/index.rst | 46 ++
.../kernel-doc-HOWTO/kernel-doc-components.rst | 35 ++
.../kernel-doc-HOWTO/kernel-doc-directive.rst | 199 ++++++++
.../books/kernel-doc-HOWTO/kernel-doc-examples.rst | 16 +
.../books/kernel-doc-HOWTO/kernel-doc-intro.rst | 85 ++++
.../books/kernel-doc-HOWTO/kernel-doc-syntax.rst | 171 +++++++
.../kernel-doc-HOWTO/reST-kernel-doc-mode.rst | 120 +++++
Documentation/books/kernel-doc-HOWTO/refs.txt | 15 +
.../books/kernel-doc-HOWTO/table-markup.rst | 499 +++++++++++++++++++++
.../books/kernel-doc-HOWTO/test/parser_test.h | 332 ++++++++++++++
.../kernel-doc-HOWTO/vintage-kernel-doc-mode.rst | 100 +++++
16 files changed, 2118 insertions(+)
create mode 100644 Documentation/books/kernel-doc-HOWTO.conf
create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble-src.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.h
create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/csv_table.txt
create mode 100644 Documentation/books/kernel-doc-HOWTO/index.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-components.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-directive.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-examples.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-intro.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-syntax.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/reST-kernel-doc-mode.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/refs.txt
create mode 100644 Documentation/books/kernel-doc-HOWTO/table-markup.rst
create mode 100644 Documentation/books/kernel-doc-HOWTO/test/parser_test.h
create mode 100644 Documentation/books/kernel-doc-HOWTO/vintage-kernel-doc-mode.rst
diff --git a/Documentation/books/kernel-doc-HOWTO.conf b/Documentation/books/kernel-doc-HOWTO.conf
new file mode 100644
index 0000000..4e8c32a
--- /dev/null
+++ b/Documentation/books/kernel-doc-HOWTO.conf
@@ -0,0 +1,200 @@
+# -*- coding: utf-8; mode: python -*-
+#
+# This is the project specific sphinx-build configuration, which is loaded from
+# the base configuration file (``../conf.py``). About config values consult:
+#
+# * http://www.sphinx-doc.org/en/stable/config.html
+#
+# While setting values here, please take care to not overwrite common needed
+# configurations. This means, do not *overwrite* composite values (e.g. the
+# list- or dictionary-value of "latex_elements" resp. "extensions") by
+# thoughtless assignments. Manipulate composite values always by *update*
+# (dict-values) or extend (list-values). Nevertheless, if you know what you are
+# doing, you are free to *overwrite* values to your needs.
+#
+# useful preseted names:
+#
+# * BASE_FOLDER: the folder where the top conf.py is located
+# * main_name: the basename of this project-folder
+
+# ------------------------------------------------------------------------------
+# General configuration
+# ------------------------------------------------------------------------------
+
+project = u'kernel-doc HOWTO'
+copyright = u'2016, Linux documentation authors'
+author = u'Linux contributors'
+
+extlinks.update({ })
+
+intersphinx_mapping.update({ })
+
+extensions.extend([
+ # 'sphinx.ext.pngmath'
+ #, 'sphinx.ext.mathjax'
+])
+
+# The version info for the project you're documenting, acts as replacement for
+# |version| and |release|, also used in various other places throughout the
+# built documents.
+#
+# The short X.Y version.
+version = u'1.0'
+# The full version, including alpha/beta/rc tags.
+# release = u'0'
+
+# ------------------------------------------------------------------------------
+# Options for HTML output
+# ------------------------------------------------------------------------------
+
+# The name for this set of Sphinx documents. If None, it defaults to
+# "<project> v<release> documentation".
+#html_title = None
+
+# A shorter title for the navigation bar. Default is the same as html_title.
+#html_short_title = None
+
+# The name of an image file (relative to this directory) to place at the top
+# of the sidebar.
+#html_logo = pathjoin(BASE_FOLDER, "_tex", "logo.png")
+
+# The name of an image file (within the static path) to use as favicon of the
+# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
+# pixels large.
+#html_favicon = None
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path.extend([])
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = main_name
+
+# ------------------------------------------------------------------------------
+# Options for XeLaTeX output
+# ------------------------------------------------------------------------------
+
+xelatex_documents = [
+ # A book with default DIN A4 and 12pt
+ dict(docname = master_doc
+ , targetname = "%s.tex" % main_name
+ , documentclass = "darmarITArticle")]
+
+# This value determines the topmost sectioning unit. It should be chosen from
+# ``part``, ``chapter`` or ``section``. The default is ``None``; the topmost
+# sectioning unit is switched by documentclass. ``section`` is used if
+# documentclass will be ``howto``, otherwise ``chapter`` will be used.
+
+latex_toplevel_sectioning = 'chapter'
+
+# ------------------------------------------------------------------------------
+# Options for manual page output
+# ------------------------------------------------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+# man_pages = [
+# (master_doc, 'kernel-doc', u'Kernel-Doc',
+# [author], 1)
+# ]
+
+# If true, show URL addresses after external links.
+#man_show_urls = False
+
+# ------------------------------------------------------------------------------
+# Options for Texinfo output
+# ------------------------------------------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+# dir menu entry, description, category)
+# texinfo_documents = [
+# (master_doc, 'Kernel-Doc', u'Kernel-Doc Documentation',
+# author, 'Kernel-Doc', 'One line description of project.',
+# 'Miscellaneous'),
+# ]
+
+# Documents to append as an appendix to all manuals.
+#texinfo_appendices = []
+
+# If false, no module index is generated.
+#texinfo_domain_indices = True
+
+# How to display URL addresses: 'footnote', 'no', or 'inline'.
+#texinfo_show_urls = 'footnote'
+
+# If true, do not generate a @detailmenu in the "Top" node's menu.
+#texinfo_no_detailmenu = False
+
+# ------------------------------------------------------------------------------
+# Options for Epub output
+# ------------------------------------------------------------------------------
+
+# Bibliographic Dublin Core info.
+# epub_title = project
+# epub_author = author
+# epub_publisher = author
+# epub_copyright = copyright
+
+# The basename for the epub file. It defaults to the project name.
+#epub_basename = project
+
+# The HTML theme for the epub output. Since the default themes are not
+# optimized for small screen space, using the same theme for HTML and epub
+# output is usually not wise. This defaults to 'epub', a theme designed to save
+# visual space.
+#epub_theme = 'epub'
+
+# The language of the text. It defaults to the language option
+# or 'en' if the language is not set.
+#epub_language = ''
+
+# The scheme of the identifier. Typical schemes are ISBN or URL.
+#epub_scheme = ''
+
+# The unique identifier of the text. This can be a ISBN number
+# or the project homepage.
+#epub_identifier = ''
+
+# A unique identification for the text.
+#epub_uid = ''
+
+# A tuple containing the cover image and cover page html template filenames.
+#epub_cover = ()
+
+# A sequence of (type, uri, title) tuples for the guide element of content.opf.
+#epub_guide = ()
+
+# HTML files that should be inserted before the pages created by sphinx.
+# The format is a list of tuples containing the path and title.
+#epub_pre_files.extend([])
+
+# HTML files that should be inserted after the pages created by sphinx.
+# The format is a list of tuples containing the path and title.
+#epub_post_files.extend([])
+
+# A list of files that should not be packed into the epub file.
+epub_exclude_files.extend([])
+
+# The depth of the table of contents in toc.ncx.
+#epub_tocdepth = 3
+
+# Allow duplicate toc entries.
+#epub_tocdup = True
+
+# Choose between 'default' and 'includehidden'.
+#epub_tocscope = 'default'
+
+# Fix unsupported image types using the Pillow.
+#epub_fix_images = False
+
+# Scale large images.
+#epub_max_image_width = 0
+
+# How to display URL addresses: 'footnote', 'no', or 'inline'.
+#epub_show_urls = 'inline'
+
+# If false, no index is generated.
+#epub_use_index = True
+
diff --git a/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble-src.rst b/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble-src.rst
new file mode 100644
index 0000000..9d360ae
--- /dev/null
+++ b/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble-src.rst
@@ -0,0 +1,12 @@
+.. -*- coding: utf-8; mode: rst -*-
+.. include:: refs.txt
+
+.. _all-in-a-tumble-src:
+
+================
+Examples Sources
+================
+
+.. literalinclude:: ./all-in-a-tumble.h
+ :linenos:
+ :language: c
diff --git a/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.h b/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.h
new file mode 100644
index 0000000..d72c8bd
--- /dev/null
+++ b/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.h
@@ -0,0 +1,271 @@
+/* parse-markup: reST */
+
+/* parse-SNIP: intro */
+/**
+ * DOC: kernel-doc intro
+ *
+ * This file / chapter includes all the refered examples of the kernel-doc-HOWTO
+ * and additional tests.
+ *
+ * The content itself is nonsens / don’t look to close ;-)
+ */
+/* parse-SNAP: */
+
+
+/* parse-SNIP: hello-world */
+#include<stdio.h>
+int main() {
+ printf("Hello World\n");
+ return 0;
+}
+/* parse-SNAP: */
+
+
+/* parse-SNIP: my_struct */
+/**
+ * struct my_struct - short description
+ * @a: first member
+ * @b: second member
+ *
+ * Longer description
+ */
+struct my_struct {
+ int a;
+ int b;
+/* private: */
+ int c;
+};
+/* parse-SNAP: */
+
+
+/* parse-SNIP: my_long_struct */
+/**
+ * struct my_struct - short description
+ * @a: first member
+ * @b: second member
+ *
+ * Longer description
+ */
+struct my_long_struct {
+ int a;
+ int b;
+ /**
+ * @c: This is longer description of C.
+ *
+ * You can use paragraphs to describe arguments
+ * using this method.
+ */
+ int c;
+};
+/* parse-SNAP: */
+
+
+/* parse-SNIP: theory-of-operation */
+/**
+ * DOC: Theory of Operation
+ *
+ * The whizbang foobar is a dilly of a gizmo. It can do whatever you
+ * want it to do, at any time. It reads your mind. Here's how it works.
+ *
+ * foo bar splat
+ *
+ * The only drawback to this gizmo is that is can sometimes damage
+ * hardware, software, or its subject(s).
+ */
+/* parse-SNAP: */
+
+
+
+/* parse-SNIP: user_function */
+/**
+ * user_function - function that can only be called in user context
+ * @a: some argument
+ * Context: !in_interrupt()
+ *
+ * This function makes no sense, it is only kernel-doc demonstration.
+ *
+ * ::
+ *
+ * Example:
+ * x = user_function(22);
+ *
+ * Return:
+ * Returns first argument
+ */
+int user_function(int a)
+{
+ return a;
+}
+/* parse-SNAP: */
+
+
+
+
+/* parse-markup: kernel-doc */
+
+/**
+ * vintage - short description of this function
+ * @parameter_a: first argument
+ * @parameter_b: second argument
+ * Context: in_gizmo_mode().
+ *
+ * Long description. This function has two integer arguments. The first is
+ * @parameter_a and the second is @parameter_b.
+ *
+ * Example:
+ * user_function(22);
+ *
+ * Return:
+ * Sum of @parameter_a and @parameter_b.
+ *
+ * highlighting:
+ *
+ * - vintage() : function
+ * - @parameter_a : name of a parameter
+ * - $ENVVAR : environmental variable
+ * - &my_struct : name of a structure (up to two words including ``struct``)
+ * - %CONST : name of a constant.
+ *
+ * Parser Mode: *vintage* kernel-doc mode
+ *
+ * Within the *vintage kernel-doc mode* dogged ignores any whitespace or inline
+ * markup.
+ *
+ * - Inline markup like *emphasis* or **emphasis strong**
+ * - Literals and/or block indent:
+ *
+ * a + b
+ *
+ * In kernel-doc *vintage* mode, there are no special block or inline markups
+ * available. Markups like the one above result in ambiguous reST markup which
+ * could produce error messages in the subsequently sphinx-build
+ * process. Unexpected outputs are mostly the result.
+ *
+ * This is a link https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/
+ * to the Linux kernel source tree
+ *
+ * colon markup: sectioning by colon markup in vintage mode is partial ugly. ;-)
+ *
+ */
+int vintage(int parameter_a, char parameter_b)
+{
+ return a + b;
+}
+
+
+/* parse-markup: reST */
+
+/**
+ * rst_mode - short description of this function
+ * @a: first argument
+ * @b: second argument
+ * Context: :c:func:`in_gizmo_mode`.
+ *
+ * Long description. This function has two integer arguments. The first is
+ * ``parameter_a`` and the second is ``parameter_b``.
+ *
+ * As long as the reST / sphinx-doc toolchain uses `intersphinx
+ * <http://www.sphinx-doc.org/en/stable/ext/intersphinx.html>`__ you can refer
+ * definitions *outside* like :c:type:`struct media_device <media_device>`. If
+ * the description of ``media_device`` struct is found in any of the intersphinx
+ * locations, a hyperref to this target is genarated a build time.
+ *
+ * Example:
+ *
+ * .. code-block:: c
+ *
+ * user_function(22);
+ *
+ * Return:
+ * Sum of ``parameter_a`` and the second is ``parameter_b``.
+ *
+ * highlighting:
+ *
+ * Because reST markup syntax conflicts with the highlighting markup from the
+ * *vintage* mode, these *vintage* highlighting markup is not available in
+ * reST-mode. reST brings it's own markup to refer and highlight function,
+ * structs or whatever definition :
+ *
+ * - :c:func:`rst_mode` : function
+ * - ``$ENVVAR``: environmental variable
+ * - :c:type:`my_struct` : name of a structure
+ * - ``parameter_a`` : name of a parameter
+ * - ``CONST`` : name of a constant.
+ *
+ * Parser Mode:
+ *
+ * This is an example with activated reST additions, in this section you will
+ * find some common inline markups.
+ *
+ * Within the *reST mode* the kernel-doc parser pass through all markups to the
+ * reST toolchain, except the *vintage highlighting* but including any
+ * whitespace. With this, the full reST markup is available in the comments.
+ *
+ * This is a link to the `Linux kernel source tree
+ * <https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/>`_.
+ *
+ * This description is only to show some reST inline markups like *emphasise*
+ * and **emphasis strong**. The follwing is a demo of a reST list markup:
+ *
+ * Definition list:
+ * :def1: lorem
+ * :def2: ipsum
+ *
+ * Ordered List:
+ * - item one
+ * - item two
+ * - item three with
+ * a linebreak
+ *
+ * Literal blocks:
+ * The next example shows a literal block::
+ *
+ * +------+ +------+
+ * |\ |\ /| /|
+ * | +----+-+ +-+----+ |
+ * | | | | | | | |
+ * +-+----+ | | +----+-+
+ * \| \| |/ |/
+ * +------+ +------+
+ *
+ * Highlighted code blocks:
+ * The next example shows a code example, with highlighting C syntax in the
+ * output.
+ *
+ * .. code-block:: c
+ *
+ * // Hello World program
+ *
+ * #include<stdio.h>
+ *
+ * int main()
+ * {
+ * printf("Hello World");
+ * }
+ *
+ *
+ * reST sectioning:
+ *
+ * colon markup: sectioning by colon markup in reST mode is less ugly. ;-)
+ *
+ * A kernel-doc section like *this* section is translated into a reST
+ * *subsection*. This means, you can only use the follwing *sub-levels* within a
+ * kernel-doc section.
+ *
+ * a subsubsection
+ * ^^^^^^^^^^^^^^^
+ *
+ * lorem ipsum
+ *
+ * a paragraph
+ * """""""""""
+ *
+ * lorem ipsum
+ *
+ */
+
+int rst_mode(int a, char b)
+{
+ return a + b;
+}
+
diff --git a/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.rst b/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.rst
new file mode 100644
index 0000000..17b127b
--- /dev/null
+++ b/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.rst
@@ -0,0 +1,11 @@
+.. -*- coding: utf-8; mode: rst -*-
+.. include:: refs.txt
+
+.. _all-in-a-tumble:
+
+=================
+Rendered Examples
+=================
+
+.. kernel-doc:: ./all-in-a-tumble.h
+ :module: example
diff --git a/Documentation/books/kernel-doc-HOWTO/csv_table.txt b/Documentation/books/kernel-doc-HOWTO/csv_table.txt
new file mode 100644
index 0000000..8a14541
--- /dev/null
+++ b/Documentation/books/kernel-doc-HOWTO/csv_table.txt
@@ -0,0 +1,6 @@
+stub col row 1, column, "loremLorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy
+eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam
+voluptua."
+stub col row 1, "At vero eos et accusam et justo duo dolores et ea rebum. Stet clita
+kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.", column
+stub col row 1, column, column
diff --git a/Documentation/books/kernel-doc-HOWTO/index.rst b/Documentation/books/kernel-doc-HOWTO/index.rst
new file mode 100644
index 0000000..1893303
--- /dev/null
+++ b/Documentation/books/kernel-doc-HOWTO/index.rst
@@ -0,0 +1,46 @@
+.. -*- coding: utf-8; mode: rst -*-
+.. include:: refs.txt
+
+.. _kernel-doc-howto:
+
+================
+kernel-doc HOWTO
+================
+
+In order to provide embedded, 'C' friendly, easy to maintain, but consistent and
+extractable documentation of the functions and data structures in the Linux
+kernel, the Linux kernel has adopted a consistent style for documenting
+functions and their parameters, and structures and their members.
+
+The format for this documentation is called the *kernel-doc* format. With kernel
+version 4.x the restructuredText (reST_) markup was added to the *kernel-doc*
+format. The kernel-doc parser supports the markup modes:
+:ref:`vintage-kernel-doc-mode` / :ref:`reST-kernel-doc-mode`. This document
+gives hints on how to use these markups and how to refer and extract
+documentation from source files.
+
+The kernel-doc parser extracts the kernel-doc descriptions from the source files
+and produced reST markup as base format. With reST as base format the
+documentation building process changed also. The building process is now based
+on sphinx-doc_ and the DocBook documents will be migrated to reST gradually.
+
+.. toctree::
+ :maxdepth: 1
+
+ kernel-doc-intro
+ kernel-doc-syntax
+ vintage-kernel-doc-mode
+ reST-kernel-doc-mode
+ kernel-doc-directive
+ table-markup
+ kernel-doc-components
+ kernel-doc-examples
+
+The examples in this HOWTO using the kernel-doc comments from the example file
+:ref:`all-in-a-tumble-src`. They are rendered in the
+chapter :ref:`all-in-a-tumble`.
+
+.. only:: html
+
+ * :ref:`genindex`
+
diff --git a/Documentation/books/kernel-doc-HOWTO/kernel-doc-components.rst b/Documentation/books/kernel-doc-HOWTO/kernel-doc-components.rst
new file mode 100644
index 0000000..f515f04
--- /dev/null
+++ b/Documentation/books/kernel-doc-HOWTO/kernel-doc-components.rst
@@ -0,0 +1,35 @@
+.. -*- coding: utf-8; mode: rst -*-
+.. include:: refs.txt
+
+.. _kernel-doc-components:
+
+===================================
+Components of the kernel-doc system
+===================================
+
+Many places in the source tree have extractable kernel-doc documentation. The
+components of this system are:
+
+Documentation/Makefile.reST and Documentation/conf.py
+ Makefile and basic `sphinx config`_ file to build the various reST documents
+ and output formats. Provides the basic sphinx-doc_ build infrastructure
+ including the *sphinx-subprojects* feature. With this feature each book can be
+ build and distributed stand-alone. Cross reference between *subprojects* will
+ be ensured by `intersphinx`_.
+
+Documentation/sphinx-static and Documentation/sphinx-tex
+ Paths that contain sphinx-doc_ custom static files (such as style sheets).
+
+Documentation/books
+ In this folder, the books with reST markup are placed. To provide
+ *sphinx-subprojects*, each book has its one folder and a (optional)
+ ``Documentation/books/{book-name}.conf`` file which *overwrites* the basic
+ configuration from ``Documentation/conf.py`` (settings see `sphinx config`_)
+
+scripts/site-python/linuxdoc
+ This folder includes python extensions related to the linux documentation
+ processes.
+
+
+
+
diff --git a/Documentation/books/kernel-doc-HOWTO/kernel-doc-directive.rst b/Documentation/books/kernel-doc-HOWTO/kernel-doc-directive.rst
new file mode 100644
index 0000000..5ae5c76
--- /dev/null
+++ b/Documentation/books/kernel-doc-HOWTO/kernel-doc-directive.rst
@@ -0,0 +1,199 @@
+.. -*- coding: utf-8; mode: rst -*-
+.. include:: refs.txt
+
+.. _kernel-doc-directive:
+
+================================
+Use kernel-doc in reST documents
+================================
+
+There exists a `reST-directive
+<http://www.sphinx-doc.org/en/stable/rest.html#directives>`_ named
+``kernel-doc`` to integrate kernel-doc comments into a reST document (e.g. in a
+*book*). The directive comes with options to fine grain control which parts
+should be placed into the reST document. With no options given, the complete
+kernel-doc comments from a source file will be inserted. So, the first and very
+simple example is:
+
+.. code-block:: rst
+
+ My Media Book
+ =============
+
+ .. kernel-doc:: include/media/media-device.h
+
+With this small example the kernel-doc comments from the media-device.h will be
+inserted direct under the chapter "My Media Book". The "DOC:" sections, the function
+and the type descriptions will be inserted in the order they appear in the source file.
+Mostly you want to select more fine grained, read on to see how.
+
+kernel-doc config
+=================
+
+Within the sphinx-doc config file (``config.py``) you can set the following
+option.
+
+kernel_doc_raise_error: ``True``
+ If true, fatal errors (like missing function descriptions) raise an error. The
+ default is ``True``. Because this might break your build process, you can
+ change the value to ``False``.
+
+ In this example, the documentation of definition ``no_longer_exists`` is
+ required.
+
+ .. code-block:: rst
+
+ .. kernel-doc:: ./all-in-a-tumble.h
+ :functions: no_longer_exist
+
+ Since this definition not exists (anymore), the following TODO entry is
+ inserted, when ``kernel_doc_raise_error`` is ``False``.
+
+ .. kernel-doc:: ./all-in-a-tumble.h
+ :functions: no_longer_exist
+
+
+kernel-doc options
+==================
+
+Here is a short overview of the options:
+
+.. code-block:: rst
+
+ .. kernel-doc:: <filename>
+ :doc: <section title>
+ :export:
+ :internal:
+ :functions: <function [, functions [, ...]]>
+ :module: <prefix-id>
+ :snippets: <snippet [, snippets [, ...]]>
+ :language: <snippet-lang>
+ :linenos:
+ :debug:
+
+The argument ``<filename>`` is required, it points to a source file in the
+kernel source tree. The pathname is relativ to kernel's root folder. The
+options have the following meaning, but be aware that not all combinations of
+these options make sense:
+
+``doc <section title>``
+ Inserts the contents of the ``DOC:`` section titled ``<section title>``.
+ Spaces are allowed in ``<section title>``; do not quote the ``<section
+ title>``.
+
+``export``
+ Inserts the documentation of function, struct or whatever definition that is
+ exported using EXPORT_SYMBOL (``EXPORT_SYMBOL()``, ``EXPORT_SYMBOL_GPL()`` &
+ ``EXPORT_SYMBOL_GPL_FUTURE()``). Assume that a the all exported symbols are
+ documented.
+
+``internal``
+ Inserts the documentation of function, struct or whatever definition that
+ is documented, but not **not** exported using EXPORT_SYMBOL.
+
+``functions <name [, names [, ...]]>``
+ Inserts the documentation of function(s), struct(s) or whatever
+ definition(s) named ``name``.
+
+``module <prefix-id>``
+ The option ``:module: <id-prefix>`` sets a module-name. The module-name is
+ used as a prefix for automatic generated IDs (reference anchors).
+
+``snippets <name [, names [, ...]]>``
+ Inserts the source-code passage(s) marked with the snippet ``name``. The
+ snippet is inserted with a `code-block:: <http://www.sphinx-doc.org/en/stable/markup/code.html>`_
+ directive.
+
+ The next options make only sense in conjunction with option ``snippets``:
+
+ ``:language: <highlighter>``
+ Set highlighting language of the snippet code-block.
+
+ ``:linenos:``
+ Set line numbers in the snippet code-block.
+
+``debug``
+ Inserts a code-block with the generated reST source. This might somtimes
+ helpful to see how the kernel-doc parser transforms the kernel-doc markup to
+ reST markup.
+
+ducumentation blocks
+====================
+
+The following example inserts the documentation block with the title "Theory of
+Operation".
+
+.. code-block:: rst
+
+ .. kernel-doc:: ./all-in-a-tumble.h
+ :doc: Theory of Operation
+ :module: example
+
+With the module name "example" the title refers by:
+
+.. code-block:: rst
+
+ Rendered example: :ref:`example.theory-of-operation`
+
+Rendered example: :ref:`example.theory-of-operation`
+
+functions
+=========
+
+The following example inserts the documentation of struct 'user_function'.
+
+.. code-block:: rst
+
+ .. kernel-doc:: ./all-in-a-tumble.h
+ :functions: user_function
+ :module: example
+
+.. code-block:: rst
+
+ * Rendered example by ID with module prefix: :ref:`example.user_function`
+ * Function reference: :c:func:`user_function`
+
+* Rendered example by ID with module prefix: :ref:`example.user_function`
+* Function reference: :c:func:`user_function`
+
+
+structs, unions, enums and typedefs
+===================================
+
+The following example inserts the documentation of struct 'my_long_struct'.
+
+.. code-block:: rst
+
+ .. kernel-doc:: ./all-in-a-tumble.h
+ :functions: my_long_struct
+ :module: example
+
+.. code-block:: rst
+
+ * Rendered example by ID with module prefix: :ref:`example.my_long_struct`
+ * Type reference: :c:type:`my_long_struct` or the alternativ notation
+ with title :c:type:`struct my_long_struct <my_long_struct>`
+
+* Rendered example by ID with module prefix: :ref:`example.my_long_struct`
+* Type reference: :c:type:`my_long_struct` or the alternativ notation
+ with title :c:type:`struct my_long_struct <my_long_struct>`
+
+Snippets
+========
+
+The kernel-doc Parser supports a comment-markup for
+:ref:`kernel-doc-syntax-snippets`. By example; The directive of the shown
+code-snippet below is:
+
+.. code-block:: rst
+
+ .. kernel-doc:: ./all-in-a-tumble.h
+ :snippets: hello-world
+ :language: c
+ :linenos:
+
+.. kernel-doc:: ./all-in-a-tumble.h
+ :snippets: hello-world
+ :language: c
+ :linenos:
+
diff --git a/Documentation/books/kernel-doc-HOWTO/kernel-doc-examples.rst b/Documentation/books/kernel-doc-HOWTO/kernel-doc-examples.rst
new file mode 100644
index 0000000..4c3ba3b
--- /dev/null
+++ b/Documentation/books/kernel-doc-HOWTO/kernel-doc-examples.rst
@@ -0,0 +1,16 @@
+.. -*- coding: utf-8; mode: rst -*-
+.. include:: refs.txt
+
+.. _kernel-doc-examples:
+
+=====================
+Examples & test cases
+=====================
+
+.. toctree::
+ :maxdepth: 2
+
+ all-in-a-tumble-src
+ all-in-a-tumble
+
+
diff --git a/Documentation/books/kernel-doc-HOWTO/kernel-doc-intro.rst b/Documentation/books/kernel-doc-HOWTO/kernel-doc-intro.rst
new file mode 100644
index 0000000..8cad730
--- /dev/null
+++ b/Documentation/books/kernel-doc-HOWTO/kernel-doc-intro.rst
@@ -0,0 +1,85 @@
+.. -*- coding: utf-8; mode: rst -*-
+.. include:: refs.txt
+
+.. _kernel-doc-intro:
+
+================
+kernel-doc intro
+================
+
+In order to provide good documentation of kernel functions and data structures,
+please use the following conventions to format your kernel-doc comments in Linux
+kernel source.
+
+We also look to provide kernel-doc formatted documentation for functions
+externally visible to other kernel files (not marked "static"). We also
+recommend providing kernel-doc formatted documentation for private (file
+"static") routines, for consistency of kernel source code layout. But this is
+lower priority and at the discretion of the MAINTAINER of that kernel source
+file. Data structures visible in kernel include files should also be documented
+using kernel-doc formatted comments.
+
+The opening comment mark ``/**`` is reserved for kernel-doc comments. Only
+comments so marked will be considered by the kernel-doc tools, and any comment
+so marked must be in kernel-doc format. The closing comment marker for
+kernel-doc comments can be either ``*/`` or ``**/``, but ``*/`` is preferred in
+the Linux kernel tree.
+
+.. hint::
+
+ 1. Do not use ``/**`` to be begin a comment block unless the comment block
+ contains kernel-doc formatted comments.
+
+ 2. We definitely need kernel-doc formatted documentation for functions that
+ are exported to loadable modules using EXPORT_SYMBOL.
+
+ 3. Kernel-doc comments should be placed just before the function or data
+ structure being described.
+
+
+Example kernel-doc function comment:
+
+.. code-block:: c
+
+ /**
+ * foobar() - short function description of foobar
+ * @arg1: Describe the first argument to foobar.
+ * @arg2: Describe the second argument to foobar.
+ * One can provide multiple line descriptions
+ * for arguments.
+ *
+ * A longer description, with more discussion of the function foobar()
+ * that might be useful to those using or modifying it. Begins with
+ * empty comment line, and may include additional embedded empty
+ * comment lines.
+ *
+ * The longer description can have multiple paragraphs.
+ *
+ * Return: Describe the return value of foobar.
+ */
+
+The short description following the subject can span multiple lines and ends
+with an ``@name`` description, an empty line or the end of the comment block.
+The kernel-doc function comments describe each parameter to the function, in
+order, with the ``@name`` lines. The ``@name`` descriptions must begin on the
+very next line following this opening short function description line, with no
+intervening empty comment lines. If a function parameter is ``...`` (varargs),
+it should be listed in kernel-doc notation as::
+
+ * @...: description
+
+The return value, if any, should be described in a dedicated section named
+``Return``. Beside functions you can also write documentation for structs,
+unions, enums and typedefs. Example kernel-doc data structure comment.::
+
+ /**
+ * struct blah - the basic blah structure
+ * @mem1: describe the first member of struct blah
+ * @mem2: describe the second member of struct blah,
+ * perhaps with more lines and words.
+ *
+ * Longer description of this structure.
+ */
+
+The kernel-doc data structure comments describe each structure member in the
+data structure, with the ``@name`` lines.
diff --git a/Documentation/books/kernel-doc-HOWTO/kernel-doc-syntax.rst b/Documentation/books/kernel-doc-HOWTO/kernel-doc-syntax.rst
new file mode 100644
index 0000000..3037941
--- /dev/null
+++ b/Documentation/books/kernel-doc-HOWTO/kernel-doc-syntax.rst
@@ -0,0 +1,171 @@
+.. -*- coding: utf-8; mode: rst -*-
+.. include:: refs.txt
+
+.. _kernel-doc-syntax:
+
+==================
+kernel-doc syntax
+==================
+
+In the following examples,
+
+* ``(...)?`` signifies optional structure and
+* ``(...)*`` signifies 0 or more structure elements
+
+The definition (or DOC) name is the section header. The section header names
+must be unique per source file.
+
+.. _kernel-doc-syntax-functions:
+
+functions
+=========
+
+The format of the block comment is like this::
+
+ /**
+ * function_name(:)? (- short description)?
+ (* @parameterx: (description of parameter x)?)*
+ (* a blank line)?
+ * (Description:)? (Description of function)?
+ * (section header: (section description)? )*
+ (*)?*/
+
+All *description* text can span multiple lines, although the
+``function_name`` & its short description are traditionally on a single line.
+Description text may also contain blank lines (i.e., lines that contain only a
+"*").
+
+.. ????
+.. Avoid putting a spurious blank line after the function name, or else the
+.. description will be repeated!
+.. ????
+
+So, the trivial example would be:
+
+.. code-block:: c
+
+ /**
+ * my_function
+ */
+
+If the Description: header tag is omitted, then there must be a blank line
+after the last parameter specification.:
+
+.. code-block:: c
+
+ /**
+ * my_function - does my stuff
+ * @my_arg: its mine damnit
+ *
+ * Does my stuff explained.
+ */
+
+or, could also use:
+
+.. code-block:: c
+
+ /**
+ * my_function - does my stuff
+ * @my_arg: its mine damnit
+ * Description: Does my stuff explained.
+ */
+
+You can also add additional sections. When documenting kernel functions you
+should document the ``Context:`` of the function, e.g. whether the functions can
+be called form interrupts. Unlike other sections you can end it with an empty
+line.
+
+A non-void function should have a ``Return:`` section describing the return
+value(s). Example-sections should contain the string ``EXAMPLE`` so that
+they are marked appropriately in the output format.
+
+.. kernel-doc:: ./all-in-a-tumble.h
+ :snippets: user_function
+ :language: c
+ :linenos:
+
+Rendered example: :ref:`example.user_function`
+
+.. _kernel-doc-syntax-misc-types:
+
+structs, unions, enums and typedefs
+===================================
+
+Beside functions you can also write documentation for structs, unions, enums and
+typedefs. Instead of the function name you must write the name of the
+declaration; the ``struct``, ``union``, ``enum`` or ``typedef`` must always
+precede the name. Nesting of declarations is not supported. Use the
+``@argument`` mechanism to document members or constants.
+
+Inside a struct description, you can use the 'private:' and 'public:' comment
+tags. Structure fields that are inside a 'private:' area are not listed in the
+generated output documentation. The 'private:' and 'public:' tags must begin
+immediately following a ``/*`` comment marker. They may optionally include
+comments between the ``:`` and the ending ``*/`` marker.
+
+.. kernel-doc:: ./all-in-a-tumble.h
+ :snippets: my_struct
+ :language: c
+ :linenos:
+
+Rendered example: :ref:`example.my_struct`
+
+All descriptions can be multiline, except the short function description.
+For really longs structs, you can also describe arguments inside the body of
+the struct.
+
+.. kernel-doc:: ./all-in-a-tumble.h
+ :snippets: my_long_struct
+ :language: c
+ :linenos:
+
+Rendered example: :ref:`example.my_long_struct`
+
+This should be used only for struct and enum members.
+
+.. _kernel-doc-syntax-doc:
+
+ducumentation blocks
+====================
+
+To facilitate having source code and comments close together, you can include
+kernel-doc documentation blocks that are *free-form* comments instead of being
+kernel-doc for functions, structures, unions, enums, or typedefs. This could be
+used for something like a theory of operation for a driver or library code, for
+example.
+
+This is done by using a ``DOC:`` section keyword with a section title. A small
+example:
+
+.. kernel-doc:: ./all-in-a-tumble.h
+ :snippets: theory-of-operation
+ :language: c
+ :linenos:
+
+Rendered example: :ref:`example.theory-of-operation`
+
+.. _kernel-doc-syntax-snippets:
+
+Snippets
+========
+
+The kernel-doc Parser supports a comment-markup for snippets out of the source
+code. To start a region to snip insert::
+
+ /* parse-SNIP: <snippet-name> */
+
+The snippet region stops with a new snippet region or at the next::
+
+ /* parse-SNAP: */
+
+A small example:
+
+.. code-block:: c
+
+ /* parse-SNIP: hello-world */
+ #include<stdio.h>
+ int main() {
+ printf("Hello World\n");
+ return 0;
+ }
+ /* parse-SNAP: */
diff --git a/Documentation/books/kernel-doc-HOWTO/reST-kernel-doc-mode.rst b/Documentation/books/kernel-doc-HOWTO/reST-kernel-doc-mode.rst
new file mode 100644
index 0000000..5cfe59f
--- /dev/null
+++ b/Documentation/books/kernel-doc-HOWTO/reST-kernel-doc-mode.rst
@@ -0,0 +1,120 @@
+.. -*- coding: utf-8; mode: rst -*-
+.. include:: refs.txt
+
+.. _reST-kernel-doc-mode:
+
+====================
+reST kernel-doc mode
+====================
+
+To get in use of the fully reST_ add the following comment (e.g.) at the top of
+your source code file (or at any line reST content starts).::
+
+ /* parse-markup: reST */
+
+In reST mode the kernel-doc parser pass through all text / markups unchanged to
+the reST toolchain including any whitespace. To toggle back to
+:ref:`vintage-kernel-doc-mode` type the following line::
+
+ /* parse-markup: kernel-doc */
+
+Within reST mode, most of the *vintage* kernel-doc markup -- as described in
+:ref:`kernel-doc-syntax` -- stays unchanged, except the vintage-highlighting
+markup and the treatment of whitespaces in description blocks. The *vintage
+highlighting* markup is not supported in reST mode, because it conflicts with
+the reST markup syntax.
+
+The reST syntax brings it's own markup to refer and highlight function,
+structs or whatever definition e.g.:
+
+* functions ...
+
+ .. code-block:: rst
+
+ :c:func:`foo_func`
+
+* structs ...
+
+ .. code-block:: rst
+
+ :c:type:`stuct foo_struct`
+
+If you are familiar with the vintage style, first this might be a cons-change
+for you, but take in account, that you get a expressive ASCII markup on the
+pro-side.
+
+
+reST section structure
+======================
+
+Since a section title in reST mode needs a line break after the colon, the colon
+handling is less ugly (:ref:`vintage-mode-quirks`). E.g.::
+
+ prints out: hello world
+
+is rendered as expected in one line. If non text follows the colon, a section
+is inserted. To avoid sectioning in any case, place a space in front of the column.::
+
+ lorem list :
+
+ * lorem
+ * ipsum
+
+On the opposite, super-short sections like::
+
+ Return: sum of a and b
+
+are no longer supported, you have to enter at least one line break::
+
+ Return:
+ sum of a and b
+
+Beside these *sectioning* of the kernel-doc syntax, reST has it's own chapter,
+section etc. markup (e.g. see `Sections
+<http://www.sphinx-doc.org/en/stable/rest.html#sections>`_). Normally, there are
+no heading levels assigned to certain characters as the structure is determined
+from the succession of headings. However, there is a common convention, which is
+used by the kernel-doc parser also:
+
+* ``#`` with overline, for parts
+* ``*`` with overline, for chapters
+* ``=`` for sections
+* ``-`` for subsections
+* ``^`` for subsubsections
+* ``"`` for paragraphs
+
+Within kernel-doc comments you should use this sectioning with care. A
+kernel-doc section like the "Return" section above is translated into a reST
+section with the following markup.
+
+.. code-block:: rst
+
+ Return
+ ------
+
+ sum of a and b
+
+As you see, a kernel-doc section is at reST *subsection* level. This means, you
+can only use the following *sub-levels* within a kernel-doc section.
+
+* ``^`` for subsubsections
+* ``"`` for paragraphs
+
+
+further references
+==================
+
+Here are some handy links about reST_ and the `Sphinx markup constructs`_:
+
+* reST_ primer, `reST (quickref)`_, `reST (spec)`_
+* `Sphinx markup constructs`_
+* `sphinx domains`_
+* `sphinx cross refences`_
+* `intersphinx`_, `sphinx.ext.intersphinx`_
+* `sphinx-doc`_, `sphinx-doc FAQ`_
+* `docutils`_, `docutils FAQ`_
+
+In absence of a more detailed C style guide for documentation, the `Python's
+Style Guide for documentating
+<https://docs.python.org/devguide/documenting.html#style-guide>`_ provides a
+good orientation.
diff --git a/Documentation/books/kernel-doc-HOWTO/refs.txt b/Documentation/books/kernel-doc-HOWTO/refs.txt
new file mode 100644
index 0000000..6d96765
--- /dev/null
+++ b/Documentation/books/kernel-doc-HOWTO/refs.txt
@@ -0,0 +1,15 @@
+.. _`reST (spec)`: http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html
+.. _`reST (quickref)`: http://docutils.sourceforge.net/docs/user/rst/quickref.html
+
+.. _`reST`: http://www.sphinx-doc.org/en/stable/rest.html
+.. _`Sphinx markup constructs`: http://www.sphinx-doc.org/en/stable/markup/index.html
+.. _`sphinx-doc`: http://www.sphinx-doc.org/
+.. _`sphinx-doc FAQ`: http://www.sphinx-doc.org/en/stable/faq.html
+.. _`sphinx domains`: http://www.sphinx-doc.org/en/stable/domains.html
+.. _`sphinx cross refences`: http://www.sphinx-doc.org/en/stable/markup/inline.html#cross-referencing-arbitrary-locations
+.. _`sphinx.ext.intersphinx`: http://www.sphinx-doc.org/en/stable/ext/intersphinx.html#module-sphinx.ext.intersphinx
+.. _`intersphinx`: http://www.sphinx-doc.org/en/stable/ext/intersphinx.html
+.. _`sphinx config`: http://www.sphinx-doc.org/en/stable/config.html
+
+.. _`docutils`: http://docutils.sourceforge.net/docs/index.html
+.. _`docutils FAQ`: http://docutils.sourceforge.net/FAQ.html
diff --git a/Documentation/books/kernel-doc-HOWTO/table-markup.rst b/Documentation/books/kernel-doc-HOWTO/table-markup.rst
new file mode 100644
index 0000000..38a6de0
--- /dev/null
+++ b/Documentation/books/kernel-doc-HOWTO/table-markup.rst
@@ -0,0 +1,499 @@
+.. -*- coding: utf-8; mode: rst -*-
+
+.. _`Emacs Table Mode`: https://www.emacswiki.org/emacs/TableMode
+.. _`Online Tables Generator`: http://www.tablesgenerator.com/text_tables
+.. _`OASIS XML Exchange Table Model`: https://www.oasis-open.org/specs/tm9901.html
+
+
+.. _xref_table_concerns:
+
+============
+About tables
+============
+
+First: the rest-flat-table_ directive is preferred in the Linux kernel tree.
+
+Internaly, the docutils uses a representation according to the `OASIS XML
+Exchange Table Model`_ (same as DocBook). The *OASIS Table Model* gives a huge
+bandwith of possibilities to form tables. This often seduce authors to force a
+specific layout. Misuse of tables in this manner is not recommended, because it
+breaks the separation of *presentation from content* which most often ends in
+problems in range of output formats. Tables (and preformated text like source
+code listings) should be used advisedly. In a HTML output, the horizontal and
+vertical expansion is handled by a scrollbar. On print medias (paper / pdf)
+there is no scrollbar, automaticaly (page-) breaking a table (or line-breaking a
+preformated text) in the layout process ends mostly in a unwanted results.
+
+.. hint::
+
+ Tables and preformated text in itself violate the separation of *presentation
+ from content*, but we will never be able to entirely renounce them. Use them
+ with care, if your content should be rendered well, in wide variation of
+ output formats.
+
+
+ASCII-art tables
+================
+
+ASCII-art tables might be comfortable for readers of the text-files, but they
+have huge disadvantages in the creation and modifying. First, they are hard to
+edit. Think about adding a row or a column to a ASCII-art table or adding a
+paraggraph in a cell, it is a nightmare on big tables. Second the diff of
+modifing ASCII-art tables is not meaningfull, e.g. widening a cell generates a
+diff in which also changes are included, which are only ascribable to the
+ASCII-art (see also :ref:`list-table-directives`).
+
+* `Emacs Table Mode`_
+* `Online Tables Generator`_
+
+
+Simple tables
+-------------
+
+simple tables allow *colspan* but not *rowspan*:
+
+.. code-block:: none
+
+ ====== ====== ======
+ Inputs Output
+ ------------- ------
+ A B A or B
+ ====== ====== ======
+ False
+ --------------------
+ True
+ --------------------
+ True False True
+ ------ ------ ------
+ False True
+ ====== =============
+
+Rendered as:
+
+====== ====== ======
+ Inputs Output
+------------- ------
+A B A or B
+====== ====== ======
+False
+--------------------
+True
+--------------------
+True False True
+------ ------ ------
+False True
+====== =============
+
+
+Grid tables
+-----------
+
+grid tables allow colspan *colspan* and *rowspan*:
+
+.. code-block:: rst
+
+ +------------+------------+-----------+
+ | Header 1 | Header 2 | Header 3 |
+ +============+============+===========+
+ | body row 1 | column 2 | column 3 |
+ +------------+------------+-----------+
+ | body row 2 | Cells may span columns.|
+ +------------+------------+-----------+
+ | body row 3 | Cells may | - Cells |
+ +------------+ span rows. | - contain |
+ | body row 4 | | - blocks. |
+ +------------+------------+-----------+
+
+Rendered as:
+
++------------+------------+-----------+
+| Header 1 | Header 2 | Header 3 |
++============+============+===========+
+| body row 1 | column 2 | column 3 |
++------------+------------+-----------+
+| body row 2 | Cells may span columns.|
++------------+------------+-----------+
+| body row 3 | Cells may | - Cells |
++------------+ span rows. | - contain |
+| body row 4 | | - blocks. |
++------------+------------+-----------+
+
+.. _list-table-directives:
+
+List table directives
+=====================
+
+The *list table* formats are double stage list, compared to the ASCII-art they
+migth not be as comfortable for readers of the text-files. Their advantage is,
+that they are easy to create/modify and that the diff of a modification is much
+more meaningfull, because it is limited to the modified content.
+
+.. _rest-list-table:
+
+list-table
+----------
+
+The ``list-tables`` has no ability to *colspan* nor *rowspan*:
+
+.. code-block:: rst
+
+ .. list-table:: table title
+ :header-rows: 1
+ :stub-columns: 1
+
+ * - ..
+ - head col 1
+ - head col 2
+
+ * - stub col row 1
+ - column
+ - column
+
+ * - stub col row 2
+ - column
+ - column
+
+ * - stub col row 3
+ - column
+ - column
+
+
+Rendered as:
+
+.. list-table:: table title
+ :header-rows: 1
+ :stub-columns: 1
+
+ * - ..
+ - head col 1
+ - head col 2
+
+ * - stub col row 1
+ - column
+ - column
+
+ * - stub col row 2
+ - column
+ - column
+
+ * - stub col row 3
+ - column
+ - column
+
+.. _rest-flat-table:
+
+flat-table
+----------
+
+The ``flat-table`` (:py:class:`FlatTable`) is a double-stage list similar
+to the ``list-table`` with some additional features:
+
+* *column-span*: with the role ``cspan`` a cell can be extended through
+ additional columns
+
+* *row-span*: with the role ``rspan`` a cell can be extended through
+ additional rows
+
+* *auto span* rightmost cell of a table row over the missing cells on the right
+ side of that table-row. With Option ``:fill-cells:`` this behavior can
+ changed from *auto span* to *auto fill*, which automaticly inserts (empty)
+ cells instead of spanning the last cell.
+
+options:
+
+:header-rows: [int] count of header rows
+:stub-columns: [int] count of stub columns
+:widths: [[int] [int] ... ] widths of columns
+:fill-cells: instead of autospann missing cells, insert missing cells
+
+roles:
+
+:cspan: [int] additionale columns (*morecols*)
+:rspan: [int] additionale rows (*morerows*)
+
+The example below shows how to use this markup. The first level of the staged
+list is the *table-row*. In the *table-row* there is only one markup allowed,
+the list of the cells in this *table-row*. Exception are *comments* ( ``..`` )
+and *targets* (e.g. a ref to :ref:`row 2 of table's body <row body 2>`).
+
+.. code-block:: rst
+
+ .. flat-table:: table title
+ :header-rows: 2
+ :stub-columns: 1
+ :widths: 1 1 1 1 2
+
+ * - :rspan:`1` head / stub
+ - :cspan:`3` head 1.1-4
+
+ * - head 2.1
+ - head 2.2
+ - head 2.3
+ - head 2.4
+
+ * .. row body 1 / this is a comment
+
+ - row 1
+ - :rspan:`2` cell 1-3.1
+ - cell 1.2
+ - cell 1.3
+ - cell 1.4
+
+ * .. Comments and targets are allowed on *table-row* stage.
+ .. _`row body 2`:
+
+ - row 2
+ - cell 2.2
+ - :rspan:`1` :cspan:`1`
+ cell 2.3 with a span over
+
+ * col 3-4 &
+ * row 2-3
+
+ * - row 3
+ - cell 3.2
+
+ * - row 4
+ - cell 4.1
+ - cell 4.2
+ - cell 4.3
+ - cell 4.4
+
+ * - row 5
+ - cell 5.1 with automatic span to rigth end
+
+ * - row 6
+ - cell 6.1
+ - ..
+
+
+Rendered as:
+
+ .. flat-table:: table title
+ :header-rows: 2
+ :stub-columns: 1
+ :widths: 1 1 1 1 2
+
+ * - :rspan:`1` head / stub
+ - :cspan:`3` head 1.1-4
+
+ * - head 2.1
+ - head 2.2
+ - head 2.3
+ - head 2.4
+
+ * .. row body 1 / this is a comment
+
+ - row 1
+ - :rspan:`2` cell 1-3.1
+ - cell 1.2
+ - cell 1.3
+ - cell 1.4
+
+ * .. Comments and targets are allowed on *table-row* stage.
+ .. _`row body 2`:
+
+ - row 2
+ - cell 2.2
+ - :rspan:`1` :cspan:`1`
+ cell 2.3 with a span over
+
+ * col 3-4 &
+ * row 2-3
+
+ * - row 3
+ - cell 3.2
+
+ * - row 4
+ - cell 4.1
+ - cell 4.2
+ - cell 4.3
+ - cell 4.4
+
+ * - row 5
+ - cell 5.1 with automatic span to rigth end
+
+ * - row 6
+ - cell 6.1
+ - ..
+
+
+CSV table
+=========
+
+CSV table might be the choice if you want to include CSV-data from a outstanding
+(build) process into your documentation.
+
+.. code-block:: rst
+
+ .. csv-table:: table title
+ :header: , Header1, Header2
+ :widths: 15, 10, 30
+ :stub-columns: 1
+ :file: csv_table.txt
+
+Content of file ``csv_table.txt``:
+
+.. literalinclude:: csv_table.txt
+
+Rendered as:
+
+.. csv-table:: table title
+ :header: , Header1, Header2
+ :widths: 15, 10, 30
+ :stub-columns: 1
+ :file: csv_table.txt
+
+
+Nested Tables
+=============
+
+Nested tables are ugly, don't use them! This part here is only to show what you
+should never do. They are ugly because they cause huge problems in many output
+formats and there is always no need for nested tables.
+
+.. code-block:: rst
+
+ +-----------+----------------------------------------------------+
+ | W/NW cell | N/NE cell |
+ | +-------------+--------------------------+-----------+
+ | | W/NW center | N/NE center | E/SE cell |
+ | | +------------+-------------+ |
+ | | | +--------+ | E/SE center | |
+ | | | | nested | | | |
+ | | | +--------+ | | |
+ | | | | table | | | |
+ | | | +--------+ | | |
+ | +-------------+------------+ | |
+ | | S/SE center | | |
+ +-----------+--------------------------+-------------+ |
+ | S/SW cell | |
+ +----------------------------------------------------+-----------+
+
+Rendered as: Not supported by all sphinx-builders, don't use nested tables!!!
+
+
+raw HTML tables
+===============
+
+If HTML is the only format you want to render, you could use a raw-import of a
+HTML table markup. But be aware, this breaks the separation of *presentation from
+content*. HTML-Tables are only rendered within a HTML output.
+
+.. code-block:: html
+
+ <div class="wy-table-responsive">
+ <table class="docutils">
+ <thead>
+ <tr style="font-weight: bold;">
+ <td>Owner Module/Drivers</td>
+ <td>Group</td>
+ <td>Property Name</td>
+ <td>Type</td>
+ <td>Property Values</td>
+ <td>Object attached</td>
+ <td>Description/Restrictions</td>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td rowspan="4">DRM</td>
+ <td>Generic</td>
+ <td>"rotation"</td>
+ <td>BITMASK</td>
+ <td>{ 0, "rotate-0" }, { 1, "rotate-90" }, { 2, "rotate-180" }, { 3,
+ "rotate-270" }, { 4, "reflect-x" }, { 5, "reflect-y" }</td>
+ <td>CRTC, Plane</td>
+ <td>rotate-(degrees) rotates the image by the specified amount in
+ degrees in counter clockwise direction. reflect-x and reflect-y
+ reflects the image along the specified axis prior to rotation</td>
+ </tr>
+
+ <tr>
+ <td rowspan="3">Connector</td>
+ <td>"EDID"</td>
+ <td>BLOB | IMMUTABLE</td>
+ <td>0</td>
+ <td>Connector</td>
+ <td>Contains id of edid blob ptr object.</td>
+ </tr>
+
+ <tr>
+ <td>"DPMS"</td>
+ <td>ENUM</td>
+ <td>{ "On", "Standby", "Suspend", "Off" }</td>
+ <td>Connector</td>
+ <td>Contains DPMS operation mode value.</td>
+ </tr>
+
+ <tr>
+ <td>"PATH"</td>
+ <td>BLOB | IMMUTABLE</td>
+ <td>0</td>
+ <td>Connector</td>
+ <td>Contains topology path to a connector.</td>
+ </tr>
+ </tbody>
+ </table>
+ </div>
+
+
+
+.. raw:: html
+
+ <div class="wy-table-responsive">
+ <table class="docutils">
+ <thead>
+ <tr style="font-weight: bold;">
+ <td>Owner Module/Drivers</td>
+ <td>Group</td>
+ <td>Property Name</td>
+ <td>Type</td>
+ <td>Property Values</td>
+ <td>Object attached</td>
+ <td>Description/Restrictions</td>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td rowspan="4">DRM</td>
+ <td>Generic</td>
+ <td>"rotation"</td>
+ <td>BITMASK</td>
+ <td>{ 0, "rotate-0" }, { 1, "rotate-90" }, { 2, "rotate-180" }, { 3,
+ "rotate-270" }, { 4, "reflect-x" }, { 5, "reflect-y" }</td>
+ <td>CRTC, Plane</td>
+ <td>rotate-(degrees) rotates the image by the specified amount in
+ degrees in counter clockwise direction. reflect-x and reflect-y
+ reflects the image along the specified axis prior to rotation</td>
+ </tr>
+
+ <tr>
+ <td rowspan="3">Connector</td>
+ <td>"EDID"</td>
+ <td>BLOB | IMMUTABLE</td>
+ <td>0</td>
+ <td>Connector</td>
+ <td>Contains id of edid blob ptr object.</td>
+ </tr>
+
+ <tr>
+ <td>"DPMS"</td>
+ <td>ENUM</td>
+ <td>{ "On", "Standby", "Suspend", "Off" }</td>
+ <td>Connector</td>
+ <td>Contains DPMS operation mode value.</td>
+ </tr>
+
+ <tr>
+ <td>"PATH"</td>
+ <td>BLOB | IMMUTABLE</td>
+ <td>0</td>
+ <td>Connector</td>
+ <td>Contains topology path to a connector.</td>
+ </tr>
+ </tbody>
+ </table>
+ </div>
+
+.. include:: refs.txt
diff --git a/Documentation/books/kernel-doc-HOWTO/test/parser_test.h b/Documentation/books/kernel-doc-HOWTO/test/parser_test.h
new file mode 100644
index 0000000..455b74c
--- /dev/null
+++ b/Documentation/books/kernel-doc-HOWTO/test/parser_test.h
@@ -0,0 +1,332 @@
+/* parse-markup: kernel-doc */
+
+/**
+ * struct v4l2_subdev_core_ops - Define core ops callbacks for subdevs
+ *
+ * @log_status: callback for VIDIOC_LOG_STATUS ioctl handler code.
+ *
+ * @s_io_pin_config: configure one or more chip I/O pins for chips that
+ * multiplex different internal signal pads out to IO pins. This function
+ * takes a pointer to an array of 'n' pin configuration entries, one for
+ * each pin being configured. This function could be called at times
+ * other than just subdevice initialization.
+ *
+ * @init: initialize the sensor registers to some sort of reasonable default
+ * values. Do not use for new drivers and should be removed in existing
+ * drivers.
+ *
+ * @load_fw: load firmware.
+ *
+ * @reset: generic reset command. The argument selects which subsystems to
+ * reset. Passing 0 will always reset the whole chip. Do not use for new
+ * drivers without discussing this first on the linux-media mailinglist.
+ * There should be no reason normally to reset a device.
+ *
+ * @s_gpio: set GPIO pins. Very simple right now, might need to be extended with
+ * a direction argument if needed.
+ *
+ * @queryctrl: callback for VIDIOC_QUERYCTL ioctl handler code.
+ *
+ * @g_ctrl: callback for VIDIOC_G_CTRL ioctl handler code.
+ *
+ * @s_ctrl: callback for VIDIOC_S_CTRL ioctl handler code.
+ *
+ * @g_ext_ctrls: callback for VIDIOC_G_EXT_CTRLS ioctl handler code.
+ *
+ * @s_ext_ctrls: callback for VIDIOC_S_EXT_CTRLS ioctl handler code.
+ *
+ * @try_ext_ctrls: callback for VIDIOC_TRY_EXT_CTRLS ioctl handler code.
+ *
+ * @querymenu: callback for VIDIOC_QUERYMENU ioctl handler code.
+ *
+ * @ioctl: called at the end of ioctl() syscall handler at the V4L2 core.
+ * used to provide support for private ioctls used on the driver.
+ *
+ * @compat_ioctl32: called when a 32 bits application uses a 64 bits Kernel,
+ * in order to fix data passed from/to userspace.
+ *
+ * @g_register: callback for VIDIOC_G_REGISTER ioctl handler code.
+ *
+ * @s_register: callback for VIDIOC_G_REGISTER ioctl handler code.
+ *
+ * @s_power: puts subdevice in power saving mode (on == 0) or normal operation
+ * mode (on == 1).
+ *
+ * @interrupt_service_routine: Called by the bridge chip's interrupt service
+ * handler, when an interrupt status has be raised due to this subdev,
+ * so that this subdev can handle the details. It may schedule work to be
+ * performed later. It must not sleep. *Called from an IRQ context*.
+ *
+ * @subscribe_event: used by the drivers to request the control framework that
+ * for it to be warned when the value of a control changes.
+ *
+ * @unsubscribe_event: remove event subscription from the control framework.
+ *
+ * @registered_async: the subdevice has been registered async.
+ *
+ * This is a copy&paste of a more complexe struct, just for testing.
+ */
+struct v4l2_subdev_core_ops {
+ int (*log_status)(struct v4l2_subdev *sd);
+ int (*s_io_pin_config)(struct v4l2_subdev *sd, size_t n,
+ struct v4l2_subdev_io_pin_config *pincfg);
+ int (*init)(struct v4l2_subdev *sd, u32 val);
+ int (*load_fw)(struct v4l2_subdev *sd);
+ int (*reset)(struct v4l2_subdev *sd, u32 val);
+ int (*s_gpio)(struct v4l2_subdev *sd, u32 val);
+ int (*queryctrl)(struct v4l2_subdev *sd, struct v4l2_queryctrl *qc);
+ int (*g_ctrl)(struct v4l2_subdev *sd, struct v4l2_control *ctrl);
+ int (*s_ctrl)(struct v4l2_subdev *sd, struct v4l2_control *ctrl);
+ int (*g_ext_ctrls)(struct v4l2_subdev *sd, struct v4l2_ext_controls *ctrls);
+ int (*s_ext_ctrls)(struct v4l2_subdev *sd, struct v4l2_ext_controls *ctrls);
+ int (*try_ext_ctrls)(struct v4l2_subdev *sd, struct v4l2_ext_controls *ctrls);
+ int (*querymenu)(struct v4l2_subdev *sd, struct v4l2_querymenu *qm);
+ long (*ioctl)(struct v4l2_subdev *sd, unsigned int cmd, void *arg);
+#ifdef CONFIG_COMPAT
+ long (*compat_ioctl32)(struct v4l2_subdev *sd, unsigned int cmd,
+ unsigned long arg);
+#endif
+#ifdef CONFIG_VIDEO_ADV_DEBUG
+ int (*g_register)(struct v4l2_subdev *sd, struct v4l2_dbg_register *reg);
+ int (*s_register)(struct v4l2_subdev *sd, const struct v4l2_dbg_register *reg);
+#endif
+ int (*s_power)(struct v4l2_subdev *sd, int on);
+ int (*interrupt_service_routine)(struct v4l2_subdev *sd,
+ u32 status, bool *handled);
+ int (*subscribe_event)(struct v4l2_subdev *sd, struct v4l2_fh *fh,
+ struct v4l2_event_subscription *sub);
+ int (*unsubscribe_event)(struct v4l2_subdev *sd, struct v4l2_fh *fh,
+ struct v4l2_event_subscription *sub);
+ int (*registered_async)(struct v4l2_subdev *sd);
+};
+
+/**
+ * DOC: Media Controller
+ *
+ * The media controller userspace API is documented in DocBook format in
+ * Documentation/DocBook/media/v4l/media-controller.xml. This document focus
+ * on the kernel-side implementation of the media framework.
+ *
+ * * Abstract media device model:
+ *
+ * Discovering a device internal topology, and configuring it at runtime, is one
+ * of the goals of the media framework. To achieve this, hardware devices are
+ * modelled as an oriented graph of building blocks called entities connected
+ * through pads.
+ *
+ * An entity is a basic media hardware building block. It can correspond to
+ * a large variety of logical blocks such as physical hardware devices
+ * (CMOS sensor for instance), logical hardware devices (a building block
+ * in a System-on-Chip image processing pipeline), DMA channels or physical
+ * connectors.
+ *
+ * A pad is a connection endpoint through which an entity can interact with
+ * other entities. Data (not restricted to video) produced by an entity
+ * flows from the entity's output to one or more entity inputs. Pads should
+ * not be confused with physical pins at chip boundaries.
+ *
+ * A link is a point-to-point oriented connection between two pads, either
+ * on the same entity or on different entities. Data flows from a source
+ * pad to a sink pad.
+ *
+ *
+ * * Media device:
+ *
+ * A media device is represented by a struct &media_device instance, defined in
+ * include/media/media-device.h. Allocation of the structure is handled by the
+ * media device driver, usually by embedding the &media_device instance in a
+ * larger driver-specific structure.
+ *
+ * Drivers register media device instances by calling
+ * __media_device_register() via the macro media_device_register()
+ * and unregistered by calling
+ * media_device_unregister().
+ *
+ * * Entities, pads and links:
+ *
+ * - Entities
+ *
+ * Entities are represented by a struct &media_entity instance, defined in
+ * include/media/media-entity.h. The structure is usually embedded into a
+ * higher-level structure, such as a v4l2_subdev or video_device instance,
+ * although drivers can allocate entities directly.
+ *
+ * Drivers initialize entity pads by calling
+ * media_entity_pads_init().
+ *
+ * Drivers register entities with a media device by calling
+ * media_device_register_entity()
+ * and unregistred by calling
+ * media_device_unregister_entity().
+ *
+ * - Interfaces
+ *
+ * Interfaces are represented by a struct &media_interface instance, defined in
+ * include/media/media-entity.h. Currently, only one type of interface is
+ * defined: a device node. Such interfaces are represented by a struct
+ * &media_intf_devnode.
+ *
+ * Drivers initialize and create device node interfaces by calling
+ * media_devnode_create()
+ * and remove them by calling:
+ * media_devnode_remove().
+ *
+ * - Pads
+ *
+ * Pads are represented by a struct &media_pad instance, defined in
+ * include/media/media-entity.h. Each entity stores its pads in a pads array
+ * managed by the entity driver. Drivers usually embed the array in a
+ * driver-specific structure.
+ *
+ * Pads are identified by their entity and their 0-based index in the pads
+ * array.
+ * Both information are stored in the &media_pad structure, making the
+ * &media_pad pointer the canonical way to store and pass link references.
+ *
+ * Pads have flags that describe the pad capabilities and state.
+ *
+ * %MEDIA_PAD_FL_SINK indicates that the pad supports sinking data.
+ * %MEDIA_PAD_FL_SOURCE indicates that the pad supports sourcing data.
+ *
+ * NOTE: One and only one of %MEDIA_PAD_FL_SINK and %MEDIA_PAD_FL_SOURCE must
+ * be set for each pad.
+ *
+ * - Links
+ *
+ * Links are represented by a struct &media_link instance, defined in
+ * include/media/media-entity.h. There are two types of links:
+ *
+ * 1. pad to pad links:
+ *
+ * Associate two entities via their PADs. Each entity has a list that points
+ * to all links originating at or targeting any of its pads.
+ * A given link is thus stored twice, once in the source entity and once in
+ * the target entity.
+ *
+ * Drivers create pad to pad links by calling:
+ * media_create_pad_link() and remove with media_entity_remove_links().
+ *
+ * 2. interface to entity links:
+ *
+ * Associate one interface to a Link.
+ *
+ * Drivers create interface to entity links by calling:
+ * media_create_intf_link() and remove with media_remove_intf_links().
+ *
+ * NOTE:
+ *
+ * Links can only be created after having both ends already created.
+ *
+ * Links have flags that describe the link capabilities and state. The
+ * valid values are described at media_create_pad_link() and
+ * media_create_intf_link().
+ *
+ * Graph traversal:
+ *
+ * The media framework provides APIs to iterate over entities in a graph.
+ *
+ * To iterate over all entities belonging to a media device, drivers can use
+ * the media_device_for_each_entity macro, defined in
+ * include/media/media-device.h.
+ *
+ * struct media_entity *entity;
+ *
+ * media_device_for_each_entity(entity, mdev) {
+ * // entity will point to each entity in turn
+ * ...
+ * }
+ *
+ * Drivers might also need to iterate over all entities in a graph that can be
+ * reached only through enabled links starting at a given entity. The media
+ * framework provides a depth-first graph traversal API for that purpose.
+ *
+ * Note that graphs with cycles (whether directed or undirected) are *NOT*
+ * supported by the graph traversal API. To prevent infinite loops, the graph
+ * traversal code limits the maximum depth to MEDIA_ENTITY_ENUM_MAX_DEPTH,
+ * currently defined as 16.
+ *
+ * Drivers initiate a graph traversal by calling
+ * media_entity_graph_walk_start()
+ *
+ * The graph structure, provided by the caller, is initialized to start graph
+ * traversal at the given entity.
+ *
+ * Drivers can then retrieve the next entity by calling
+ * media_entity_graph_walk_next()
+ *
+ * When the graph traversal is complete the function will return NULL.
+ *
+ * Graph traversal can be interrupted at any moment. No cleanup function call
+ * is required and the graph structure can be freed normally.
+ *
+ * Helper functions can be used to find a link between two given pads, or a pad
+ * connected to another pad through an enabled link
+ * media_entity_find_link() and media_entity_remote_pad()
+ *
+ * Use count and power handling:
+ *
+ * Due to the wide differences between drivers regarding power management
+ * needs, the media controller does not implement power management. However,
+ * the &media_entity structure includes a use_count field that media drivers
+ * can use to track the number of users of every entity for power management
+ * needs.
+ *
+ * The &media_entity.@use_count field is owned by media drivers and must not be
+ * touched by entity drivers. Access to the field must be protected by the
+ * &media_device.@graph_mutex lock.
+ *
+ * Links setup:
+ *
+ * Link properties can be modified at runtime by calling
+ * media_entity_setup_link()
+ *
+ * Pipelines and media streams:
+ *
+ * When starting streaming, drivers must notify all entities in the pipeline to
+ * prevent link states from being modified during streaming by calling
+ * media_entity_pipeline_start().
+ *
+ * The function will mark all entities connected to the given entity through
+ * enabled links, either directly or indirectly, as streaming.
+ *
+ * The &media_pipeline instance pointed to by the pipe argument will be stored
+ * in every entity in the pipeline. Drivers should embed the &media_pipeline
+ * structure in higher-level pipeline structures and can then access the
+ * pipeline through the &media_entity pipe field.
+ *
+ * Calls to media_entity_pipeline_start() can be nested. The pipeline pointer
+ * must be identical for all nested calls to the function.
+ *
+ * media_entity_pipeline_start() may return an error. In that case, it will
+ * clean up any of the changes it did by itself.
+ *
+ * When stopping the stream, drivers must notify the entities with
+ * media_entity_pipeline_stop().
+ *
+ * If multiple calls to media_entity_pipeline_start() have been made the same
+ * number of media_entity_pipeline_stop() calls are required to stop streaming.
+ * The &media_entity pipe field is reset to NULL on the last nested stop call.
+ *
+ * Link configuration will fail with -%EBUSY by default if either end of the
+ * link is a streaming entity. Links that can be modified while streaming must
+ * be marked with the %MEDIA_LNK_FL_DYNAMIC flag.
+ *
+ * If other operations need to be disallowed on streaming entities (such as
+ * changing entities configuration parameters) drivers can explicitly check the
+ * media_entity stream_count field to find out if an entity is streaming. This
+ * operation must be done with the media_device graph_mutex held.
+ *
+ * Link validation:
+ *
+ * Link validation is performed by media_entity_pipeline_start() for any
+ * entity which has sink pads in the pipeline. The
+ * &media_entity.@link_validate() callback is used for that purpose. In
+ * @link_validate() callback, entity driver should check that the properties of
+ * the source pad of the connected entity and its own sink pad match. It is up
+ * to the type of the entity (and in the end, the properties of the hardware)
+ * what matching actually means.
+ *
+ * Subsystems should facilitate link validation by providing subsystem specific
+ * helper functions to provide easy access for commonly needed information, and
+ * in the end provide a way to use driver-specific callbacks.
+ */
diff --git a/Documentation/books/kernel-doc-HOWTO/vintage-kernel-doc-mode.rst b/Documentation/books/kernel-doc-HOWTO/vintage-kernel-doc-mode.rst
new file mode 100644
index 0000000..a49bc25
--- /dev/null
+++ b/Documentation/books/kernel-doc-HOWTO/vintage-kernel-doc-mode.rst
@@ -0,0 +1,100 @@
+.. -*- coding: utf-8; mode: rst -*-
+.. include:: refs.txt
+
+.. _vintage-kernel-doc-mode:
+
+=======================
+Vintage kernel-doc mode
+=======================
+
+All kernel-doc markup is processed as described in :ref:`kernel-doc-syntax`, all
+descriptive text is further processed, scanning for the following special
+patterns, which are highlighted appropriately.
+
+* ``funcname()`` - function
+* ``$ENVVAR`` - environmental variable
+* ``&struct name`` - name of a structure (up to two words including ``struct``)
+* ``@parameter`` - name of a parameter
+* ``%CONST`` - name of a constant.
+
+These highlighted patterns are not used when you are using the reST addition
+(:ref:`reST-kernel-doc-mode`). This is, because reST brings it's own markup to
+refer and highlight function, structs or whatever definition.
+
+Within the *vintage* kernel-doc mode the kernel-doc parser highlights the pattern
+above, but he also dogged ignores any whitespace formatting/markup.
+
+.. hint::
+
+ Formatting with whitespaces is substantial for ASCII markups. By this, it's
+ recommended to use the :ref:`reST-kernel-doc-mode` on any new or changed
+ comment.
+
+
+.. _vintage-mode-quirks:
+
+vintage mode quirks
+===================
+
+In the following, you will find some quirks of the *vintage* kernel-doc mode.
+
+* Since a colon introduce a new section, you can't use colons. E.g. a comment
+ line like::
+
+ prints out: hello world
+
+ will result in a section with the title "prints out" and a paragraph with only
+ "hello world" in, this is mostly not what you expect. To avoid sectioning,
+ place a space in front of the column::
+
+ prints out : hello world
+
+* The multi-line descriptive text you provide does *not* recognize
+ line breaks, so if you try to format some text nicely, as in::
+
+ Return:
+ 0 - cool
+ 1 - invalid arg
+ 2 - out of memory
+
+ this will all run together and produce::
+
+ Return: 0 - cool 1 - invalid arg 2 - out of memory
+
+* If the descriptive text you provide has lines that begin with some phrase
+ followed by a colon, each of those phrases will be taken as a new section
+ heading, which means you should similarly try to avoid text like::
+
+ Return:
+ 0: cool
+ 1: invalid arg
+ 2: out of memory
+
+ every line of which would start a new section. Again, probably not what you
+ were after.
+
+Determined by the historical development of the kernel-doc comments, the
+*vintage* kernel-doc comments contain characters like "*" or strings with
+e.g. leading/trailing underscore ("_"), which are inline markups in reST. Here a
+short example from a *vintage* comment::
+
+ <SNIP> -----
+ * In contrast to the other drm_get_*_name functions this one here returns a
+ * const pointer and hence is threadsafe.
+ <SNAP> -----
+
+Within reST markup (the new bas format), the wildcard in the string
+``drm_get_*_name`` has to be masked: ``drm_get_\\*_name``. Some more examples
+from reST markup:
+
+* Emphasis "*": like ``*emphasis*`` or ``**emphasis strong**``
+* Leading "_" : is a *anchor* in reST markup (``_foo``).
+* Trailing "_: is a reference in reST markup (``foo_``).
+* interpreted text: "`"
+* inline literals: "``"
+* substitution references: "|"
+
+As long as you in the *vintage* kernel-doc mode, these special strings will be
+masked in the reST output and can't be used as *plain-text markup*.
+
+
--
v4.7-rc-2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 4/7] linuxdoc: add python package linuxdoc
2016-06-06 16:32 [PATCH 0/7] add reST/sphinx-doc to linux documentation Markus Heiser
` (2 preceding siblings ...)
2016-06-06 16:32 ` [PATCH 3/7] kernel-doc-HOWTO: add kernel-doc specification Markus Heiser
@ 2016-06-06 16:32 ` Markus Heiser
2016-06-06 16:32 ` [PATCH 5/7] kernel-doc parser: inital python implementation Markus Heiser
` (4 subsequent siblings)
8 siblings, 0 replies; 23+ messages in thread
From: Markus Heiser @ 2016-06-06 16:32 UTC (permalink / raw)
To: corbet
Cc: jani.nikula, daniel.vetter, grant.likely, mchehab, keithp,
linux-kernel, linux-doc, hverkuil, Heiser, Markus
From: "Heiser, Markus" <markus.heiser@darmarIT.de>
The linuxdoc package includes python extensions related to the linux
documentation processes.
Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
---
scripts/site-python/linuxdoc/__init__.py | 8 ++++++++
1 file changed, 8 insertions(+)
create mode 100644 scripts/site-python/linuxdoc/__init__.py
diff --git a/scripts/site-python/linuxdoc/__init__.py b/scripts/site-python/linuxdoc/__init__.py
new file mode 100644
index 0000000..6edc554
--- /dev/null
+++ b/scripts/site-python/linuxdoc/__init__.py
@@ -0,0 +1,8 @@
+# -*- coding: utf-8; mode: python -*-
+u"""
+ linuxdoc
+ ~~~~~~~~
+
+ The linuxdoc package includes python extensions related to the linux
+ documentation processes.
+"""
--
v4.7-rc-2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 5/7] kernel-doc parser: inital python implementation
2016-06-06 16:32 [PATCH 0/7] add reST/sphinx-doc to linux documentation Markus Heiser
` (3 preceding siblings ...)
2016-06-06 16:32 ` [PATCH 4/7] linuxdoc: add python package linuxdoc Markus Heiser
@ 2016-06-06 16:32 ` Markus Heiser
2016-06-06 16:32 ` [PATCH 6/7] kernel-doc directive: initial implementation Markus Heiser
` (3 subsequent siblings)
8 siblings, 0 replies; 23+ messages in thread
From: Markus Heiser @ 2016-06-06 16:32 UTC (permalink / raw)
To: corbet
Cc: jani.nikula, daniel.vetter, grant.likely, mchehab, keithp,
linux-kernel, linux-doc, hverkuil, Heiser, Markus
From: "Heiser, Markus" <markus.heiser@darmarIT.de>
The kernel-doc parser extracts documention from linux kernel's source
code comments. This kernel-doc parser implements the specification given
in the Documention/books/kernel-doc-HOWTO [1] and replaces the
kernel-doc perl script.
This module provides an API -- which is used by a sphinx-doc generator
extension -- and a commandline interface, see ``--help``::
$ ./kernel_doc.py --help
About a kernel-doc parser rewrite:
The rewrite is written in python and has additional features for a
smooth integration into the sphinx-doc buildchain. In addition, this
rewrite brings the functionalities, which has been spread
in *docproc* and make files (e.g. working with *EXPORTED_SYMBOLS*)
back to the kernel-doc parse process. In combination with
a (separate) *kernel-doc* reST directive which uses this module, the
documentation generation becomes much more clear and flexible.
[1] http://return42.github.io/sphkerneldoc/books/kernel-doc-HOWTO
Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
---
scripts/site-python/linuxdoc/kernel_doc.py | 2764 ++++++++++++++++++++++++++++
1 file changed, 2764 insertions(+)
create mode 100755 scripts/site-python/linuxdoc/kernel_doc.py
diff --git a/scripts/site-python/linuxdoc/kernel_doc.py b/scripts/site-python/linuxdoc/kernel_doc.py
new file mode 100755
index 0000000..ae84b2b
--- /dev/null
+++ b/scripts/site-python/linuxdoc/kernel_doc.py
@@ -0,0 +1,2764 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8; mode: python -*-
+# pylint: disable= C0330, C0103, C0410, R0903, R0913, R0912, R0914, R0915, C0302
+
+u"""
+ kernel_doc
+ ~~~~~~~~~~
+
+ Implementation of the ``kernel-doc`` parser.
+
+ :copyright: Copyright (C) 2016 Markus Heiser
+ :license: GPL V3.0, see LICENSE for details.
+
+ The kernel-doc parser extracts documention from linux kernel's source code
+ comments. This is a rewrite of the kernel-doc perl script.
+
+ This module provides an API -- which could used by a sphinx-doc generator
+ extension -- and a commandline interface, see ``--help``::
+
+ $ ./kernel_doc.py --help
+
+ About the rewrite:
+
+ The rewrite has additional features like *parse options* (see below) for a
+ smooth integration of restructuredText (reST) markup in the kernel's
+ source code comments. In addition, this rewrite brings the
+ functionalities, which has been spread in *docproc* and make files
+ (e.g. working with *EXPORTED_SYMBOLS*) back to the kernel-doc parse
+ process. In combination with a (separate) *kernel-doc* reST directive
+ which uses this module, the documentation generation becomes much more
+ clear and flexible.
+
+ About reST integration into the kerne-doc markup:
+
+ In general, the markup of the linux kernel's source code comments remains
+ unchanged and the reST markup within the comments is passed through the
+ output. A closer lookup to the *kernel-doc* and *reST* markup revals, that
+ there are some conflicts between reST (inline) markup and kernel-doc
+ markup. This conflicts could be handled fine grained with the *parse
+ options*. Parse options are comments like: ``/* parse-{option-name}:
+ {value} */``, examples about conflicts and *parse options* see below.
+
+ As long as kernel's source code comments remain unchanged, the description
+ *here* is mainly a copy of the kernel-doc origin, except for the additions
+ which are decribed at the end.
+
+ In the following examples,
+
+ * ``(...)?`` signifies optional structure and
+ * ``(...)*`` signifies 0 or more structure elements
+
+ Format of comments.::
+
+ /**
+ * function_name(:)? (- short description)?
+ (* @parameterx: (description of parameter x)?)*
+ (* a blank line)?
+ * (Description:)? (Description of function)?
+ * (section header: (section description)? )*
+ (*)?*/
+
+ So, the trivial example would be::
+
+ /**
+ * my_function
+ */
+
+ If the Description: header tag is omitted, then there must be a blank line
+ after the last parameter specification.::
+
+ /**
+ * my_function - does my stuff
+ * @my_arg: its mine damnit
+ *
+ * Does my stuff explained.
+ */
+
+ or, could also use::
+
+ /**
+ * my_function - does my stuff
+ * @my_arg: its mine damnit
+ * Description: Does my stuff explained.
+ */
+
+ Besides functions you can also write documentation for ``structs``,
+ ``unions``, ``enums`` and ``typedefs``. Instead of the function name you
+ must write the name of the declaration; the struct / union / enum / typedef
+ must always precede the name. Nesting of declarations is not supported. Use
+ the argument mechanism to document members or constants.::
+
+ /**
+ * struct my_struct - short description
+ * @a: first member
+ * @b: second member
+ *
+ * Longer description
+ */
+ struct my_struct {
+ int a;
+ int b;
+ /* private: */
+ int c;
+ };
+
+ All descriptions can be multiline, except the short function description.
+ For really longs structs, you can also describe arguments inside the body of
+ the struct.::
+
+ /**
+ * struct my_struct - short description
+ * @a: first member
+ * @b: second member
+ *
+ * Longer description
+ */
+ struct my_struct {
+ int a;
+ int b;
+ /**
+ * @c: This is longer description of C
+ *
+ * You can use paragraphs to describe arguments
+ * using this method.
+ */
+ int c;
+ };
+
+ This should be used only for struct/enum members.
+
+ You can also add additional sections. When documenting kernel functions you
+ should document the ``Context:`` of the function, e.g. whether the functions
+ can be called form interrupts. Unlike other sections you can end it with an
+ empty line.
+
+ A non-void function should have a ``Return:`` section describing the return
+ value(s). Example-sections should contain the string ``EXAMPLE`` so that
+ they are marked appropriately in the output format.::
+
+ /**
+ * user_function - function that can only be called in user context
+ * @a: some argument
+ * Context: !in_interrupt()
+ *
+ * Some description
+ *
+ * Example:
+ * user_function(22);
+ */
+
+ All descriptive text is further processed, scanning for the following special
+ patterns, which are highlighted appropriately.
+
+ * ``funcname()`` - function
+ * ``$ENVVAR`` - environmental variable
+ * ``&struct_name`` - name of a structure (up to two words including ``struct``)
+ * ``@parameter`` - name of a parameter
+ * ``%CONST`` - name of a constant.
+
+ With the following comment lines, highlighting could be switched on or off::
+
+ /* parse-highlight: on */
+ /* parse-highlight: off */
+
+ In addition you could use the *plain-text markup* restructuredText (reST /
+ http://www.sphinx-doc.org/rest.html) in the descriptive text, which will
+ passed through.
+
+ Determined by the historical development of the kernel-doc comments, the
+ *classic* kernel-doc comments contain characters like ``*`` or strings with
+ e.g. leading/trailing underscore (``_``), which are inline markups in
+ reST. Here a schort example from a *classic* comment::
+
+ <SNIP> -----
+ * In contrast to the other drm_get_*_name functions this one here returns a
+ * const pointer and hence is threadsafe.
+ <SNAP> -----
+
+ In reST markup, the wildcard in the string ``drm_get_*_name`` has to be
+ masked: ``drm_get_\\*_name``. Some more examples from reST markup:
+
+ * Emphasis "*": like ``*emphasis*`` or ``**emphasis strong**``
+
+ * Leading "_" : is a *anchor* in reST markup (``_foo``).
+
+ * Trailing "_: is a reference in reST markup (``foo_``).
+
+ * interpreted text: "`"
+
+ * inline literals: "``"
+
+ * substitution references: "|"
+
+ These special strings will be masked in the output and can't be used as
+ *plain-text markup*. To get in use of the fully reST markup (stop masking
+ special characters) add the following comment (e.g.) at the top of your
+ source code file.::
+
+ /* parse-markup: reST */
+
+ To toggle back to *classic* kernel-doc behavior (masking special characters
+ in the output) type the following line::
+
+ /* parse-markup: kernel-doc */
+
+ Additional hints for ``kernel-doc.py`` developers:
+
+ If you like to inspect a selective part of the kernel's source comments, you
+ could toggle the global ``INSPECT`` variable with::
+
+ /* parse-INSPECT: on */
+ /* parse-INSPECT: off */
+
+ And with an additional *breakpoint* in the ``kernel_doc.py`` script::
+
+ INSPECT and CONSOLE()
+
+ a console for introspection pops up (see examples in kernel_doc.py sources).
+
+"""
+
+# ==============================================================================
+# imports
+# ==============================================================================
+
+import sys, os, argparse, codecs, collections, textwrap
+import re
+
+# ==============================================================================
+# common globals
+# ==============================================================================
+
+# The version numbering follows numbering of the specification
+# (Documentation/books/kernel-doc-HOWTO).
+__version__ = '1.0'
+
+PY3 = sys.version_info[0] == 3
+PY2 = sys.version_info[0] == 2
+
+if PY3:
+ # pylint: disable=C0103, W0622
+ unicode = str
+ basestring = str
+
+# ==============================================================================
+# regular expresssions and helper used by the parser and the translator
+# ==============================================================================
+
+class RE(object):
+ u"""regexp that stores last match (like perl's ``=~`` operator)"""
+
+ def __init__(self, *args, **kwargs):
+ self.re = re.compile(*args, **kwargs)
+ self.last_match = None
+
+ def match(self, *args, **kwargs):
+ self.last_match = self.re.match(*args, **kwargs)
+ return self.last_match
+
+ def search(self, *args, **kwargs):
+ self.last_match = self.re.search(*args, **kwargs)
+ return self.last_match
+
+ def __getattr__(self, attr):
+ return getattr(self.re, attr)
+
+ def __getitem__(self, group):
+ if group < 0 or group > self.groups - 1:
+ raise IndexError("group index out of range (max %s groups)" % self.groups )
+ if self.last_match is None:
+ raise IndexError("nothing hase matched / no groups")
+ return self.last_match.group(group + 1)
+
+# these regular expresions has been *stolen* from the kernel-doc perl script.
+
+doc_special = RE(r"\@\%\$\&") # special: [@%$&]
+doc_start = RE(r"^/\*\*\s*$") # Allow whitespace at end of comment start.
+doc_end = RE(r"\s*\*+/")
+doc_com = RE(r"\s*\*\s*")
+doc_com_body = RE(r"\s*\* ?")
+doc_decl = RE(doc_com.pattern + r"(\w+)")
+doc_decl_ident = RE(r"\s*([\w\s]+?)\s*-")
+doc_decl_purpose = RE(r"-(.*)$")
+
+# except pattern like "http://", a whitespace is required after the colon
+doc_sect_except = RE(doc_com.pattern + r"(.*?):[^\s]")
+
+#doc_sect = RE(doc_com.pattern + r"([" + doc_special.pattern + r"]?[\w\s]+):(.*)")
+# "section header:" names must be unique per function (or struct,union, typedef,
+# enum). Additional condition: the header name should have 3 characters at least!
+doc_sect = RE(doc_com.pattern
+ + r"("
+ + r"[" + doc_special.pattern + r"]\w[\w\s]*" # "@foo: lorem" or
+ + r"|" + r"\@\.\.\." # ellipsis "@...: lorem" or
+ + r"|" + r"\w[\w\s]+\w" # e.g. "Return: lorem"
+ + r")"
+ + r":(.*?)\s*$") # this matches also strings like "http://..." (doc_sect_except)
+
+doc_param = RE(doc_com.pattern
+ + r"("
+ + r"[" + doc_special.pattern + r"]\w[\w\s]*" # "@foo: lorem" or
+ + r"|" + r"\@\.\.\." # ellipsis "@...: lorem" or
+ + r")"
+ + r":(.*?)\s*$") # this matches also strings like "http://..." (doc_sect_except)
+
+reST_sect = RE(doc_com.pattern
+ + r"("
+ r"\w[\w\s]+\w"
+ + r")"
+ + r":\s*$")
+
+doc_content = RE(doc_com_body.pattern + r"(.*)")
+doc_block = RE(doc_com.pattern + r"DOC:\s*(.*)?")
+doc_split_start = RE(r"^\s*/\*\*\s*$")
+doc_split_sect = RE(r"\s*\*\s*(@[\w\s]+):(.*)")
+doc_split_end = RE(r"^\s*\*/\s*$")
+
+# match expressions used to find embedded type information
+type_constant = RE(r"\%([-_\w]+)")
+type_func = RE(r"(\w+)\(\)")
+type_param = RE(r"\@(\w+)")
+type_struct = RE(r"\&((struct\s*)*[_\w]+)")
+type_env = RE(r"(\$\w+)")
+type_enum_full = RE(r"\&(enum)\s*([_\w]+)")
+type_struct_full = RE(r"\&(struct)\s*([_\w]+)")
+
+CR_NL = RE(r"[\r\n]")
+C99_comments = RE(r"//.*$")
+C89_comments = RE(r"/\*.*?\*/")
+
+C_STRUCT = RE(r"struct\s+(\w+)\s*{(.*)}")
+C_UNION = RE(r"union\s+(\w+)\s*{(.*)}")
+C_STRUCT_UNION = RE(r"(struct|union)\s+(\w+)\s*{(.*)}")
+C_ENUM = RE(r"enum\s+(\w+)\s*{(.*)}")
+C_TYPEDEF = RE(r"typedef.*\s+(\w+)\s*;")
+
+# typedef of a function pointer
+C_FUNC_TYPEDEF = RE(r"typedef\s+(\w+)\s*\(\*\s*(\w\S+)\s*\)\s*\((.*)\);")
+
+MACRO = RE(r"^#")
+MACRO_define = RE(r"^#\s*define\s+")
+
+SYSCALL_DEFINE = RE(r"^\s*SYSCALL_DEFINE.*\(")
+SYSCALL_DEFINE0 = RE(r"^\s*SYSCALL_DEFINE0")
+
+TP_PROTO = RE(r"TP_PROTO\((.*?)\)")
+TRACE_EVENT = RE(r"TRACE_EVENT")
+TRACE_EVENT_name = RE(r"TRACE_EVENT\((.*?),")
+DEFINE_EVENT = RE(r"DEFINE_EVENT")
+DEFINE_EVENT_name = RE(r"DEFINE_EVENT\((.*?),(.*?),")
+DEFINE_SINGLE_EVENT = RE(r"DEFINE_SINGLE_EVENT")
+DEFINE_SINGLE_EVENT_name = RE(r"DEFINE_SINGLE_EVENT\((.*?),")
+
+FUNC_PROTOTYPES = [
+ # RE(r"^(\w+)\s+\(\*([a-zA-Z0-9_]+)\)\s*\(([^\(]*)\)") # match: void (*foo) (int bar);
+ RE(r"^()([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)")
+ , RE(r"^(\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)")
+ , RE(r"^(\w+\s*\*)\s*([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)")
+ , RE(r"^(\w+\s+\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)")
+ , RE(r"^(\w+\s+\w+\s*\*+)\s*([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)")
+ , RE(r"^(\w+\s+\w+\s+\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)")
+ , RE(r"^(\w+\s+\w+\s+\w+\s*\*)\s*([a-zA-Z0-9_~:]+)\s*\(([^\(]*)\)")
+ , RE(r"^()([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
+ , RE(r"^(\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
+ , RE(r"^(\w+\s*\*)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
+ , RE(r"^(\w+\s+\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
+ , RE(r"^(\w+\s+\w+\s*\*)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
+ , RE(r"^(\w+\s+\w+\s+\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
+ , RE(r"^(\w+\s+\w+\s+\w+\s*\*)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
+ , RE(r"^(\w+\s+\w+\s+\w+\s+\w+)\s+([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
+ , RE(r"^(\w+\s+\w+\s+\w+\s+\w+\s*\*)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)")
+ , RE(r"^(\w+\s+\w+\s*\*\s*\w+\s*\*\s*)\s*([a-zA-Z0-9_~:]+)\s*\(([^\{]*)\)/")
+]
+
+EXPORTED_SYMBOLS = RE(
+ r"^\s*(EXPORT_SYMBOL)(_GPL)?(_FUTURE)?\s*\(\s*(\w*)\s*\)\s*", flags=re.M)
+
+# MODULE_AUTHOR("..."); / MODULE_DESCRIPTION("..."); / MODULE_LICENSE("...");
+#
+MODULE_INFO = RE(r'^\s*(MODULE_)(AUTHOR|DESCRIPTION|LICENSE)\s*\(\s*"([^"]+)"', flags=re.M)
+
+WHITESPACE = RE(r"\s+", flags=re.UNICODE)
+
+def normalize_ws(string):
+ u"""strip needles whitespaces.
+
+ Substitute consecutive whitespaces with one single space and strip
+ trailing/leading whitespaces"""
+
+ string = WHITESPACE.sub(" ", string)
+ return string.strip()
+
+ID_CHARS = RE(r"[^A-Za-z0-9\._]")
+
+def normalize_id(ID):
+ u"""substitude invalid chars of the ID with ``-`` and mak it lowercase"""
+ return ID_CHARS.sub("-", ID).lower()
+
+def map_text(text, map_table):
+ for regexpr, substitute in map_table:
+ if substitute is not None:
+ text = regexpr.sub(substitute, text)
+ return text
+
+# ==============================================================================
+# helper
+# ==============================================================================
+
+def openTextFile(fname, mode="r", encoding="utf-8", errors="strict", buffering=1):
+ return codecs.open(
+ fname, mode=mode, encoding=encoding
+ , errors=errors, buffering=buffering)
+
+def readFile(fname, encoding="utf-8", errors="strict"):
+ with openTextFile(fname, encoding=encoding, errors=errors) as f:
+ return f.read()
+
+class Container(dict):
+ # pylint: disable=C0321
+ @property
+ def __dict__(self): return self
+ def __getattr__(self, attr): return self[attr]
+ def __setattr__(self, attr, val): self[attr] = val
+
+class OS_ENV(dict):
+ # pylint: disable=C0321
+ @property
+ def __dict__(self): return os.environ
+ def __getattr__(self, attr): return os.environ[attr]
+ def __setattr__(self, attr, val): os.environ[attr] = val
+ def get(self, attr, default=None): return os.environ.get(attr, default)
+
+OS_ENV = OS_ENV()
+
+KBUILD_VERBOSE = int(OS_ENV.get("KBUILD_VERBOSE", "0"))
+KERNELVERSION = OS_ENV.get("KERNELVERSION", "unknown kernel version")
+SRCTREE = OS_ENV.get("srctree", "")
+GIT_REF = "Linux kernel source tree: `%(rel_fname)s <https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/%(rel_fname)s>`_"
+
+# ==============================================================================
+# Logging stuff
+# ==============================================================================
+
+STREAM = Container(
+ # pipes used by the application & logger
+ appl_out = sys.__stdout__
+ , log_out = sys.__stderr__
+ , )
+
+VERBOSE = bool(KBUILD_VERBOSE)
+DEBUG = False
+INSPECT = False
+QUIET = False
+
+class SimpleLog(object):
+
+ LOG_FORMAT = "%(logclass)s: %(message)s\n"
+
+ def error(self, message, **replace):
+ message = message % replace
+ replace.update(dict(message = message, logclass = "ERROR"))
+ STREAM.log_out.write(self.LOG_FORMAT % replace)
+
+ def warn(self, message, **replace):
+ if QUIET:
+ return
+ message = message % replace
+ replace.update(dict(message = message, logclass = "WARN"))
+ STREAM.log_out.write(self.LOG_FORMAT % replace)
+
+ def info(self, message, **replace):
+ if not VERBOSE:
+ return
+ message = message % replace
+ replace.update(dict(message = message, logclass = "INFO"))
+ STREAM.log_out.write(self.LOG_FORMAT % replace)
+
+ def debug(self, message, **replace):
+ if not DEBUG:
+ return
+ message = message % replace
+ replace.update(dict(message = message, logclass = "DEBUG"))
+ STREAM.log_out.write(self.LOG_FORMAT % replace)
+
+LOG = SimpleLog()
+
+# ==============================================================================
+def main():
+# ==============================================================================
+
+ global VERBOSE, DEBUG # pylint: disable=W0603
+
+ CLI = argparse.ArgumentParser(
+ description = (
+ "Parse *kernel-doc* comments from source code"
+ " and print them (with reST markup) to stdout." )
+ , formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+
+ CLI.add_argument(
+ "files"
+ , nargs = "+"
+ , help = "source file(s) to parse.")
+
+ CLI.add_argument(
+ "--id-prefix"
+ , default = ""
+ , help = (
+ "A prefix for generated IDs. IDs are used as anchors by cross"
+ " references. They are automaticly gernerated based on the declaration"
+ " and / or section names. Declartions like 'open' or section names"
+ " like 'Intro' are very common, to make them unique, a prefix is needed."
+ " Mostly you will choose the *module* or *include filename* as prefix." ))
+
+ CLI.add_argument(
+ "--verbose", "-v"
+ , action = "store_true"
+ , help = "verbose output with log messages to stderr" )
+
+ CLI.add_argument(
+ "--debug"
+ , action = "store_true"
+ , help = "debug messages to stderr" )
+
+ CLI.add_argument(
+ "--skip-preamble"
+ , action = "store_true"
+ , help = "skip preamble in the output" )
+
+ CLI.add_argument(
+ "--skip-epilog"
+ , action = "store_true"
+ , help = "skip epilog in the output" )
+
+ CLI.add_argument(
+ "--list-internals"
+ , choices = Parser.DOC_TYPES + ["all"]
+ , nargs = "+"
+ , help = "list symbols, titles or whatever is documented, but *not* exported" )
+
+ CLI.add_argument(
+ "--list-exports"
+ , action = "store_true"
+ , help = "list all symbols exported using EXPORT_SYMBOL" )
+
+ CLI.add_argument(
+ "--use-names"
+ , nargs = "+"
+ , help = "print documentation of functions, structs or whatever title/object")
+
+ CLI.add_argument(
+ "--exported"
+ , action = "store_true"
+ , help = ("print documentation of all symbols exported"
+ " using EXPORT_SYMBOL macros" ))
+
+ CLI.add_argument(
+ "--internal"
+ , action = "store_true"
+ , help = ("print documentation of all symbols that are documented,"
+ " but not exported" ))
+
+
+ CMD = CLI.parse_args()
+ VERBOSE = CMD.verbose
+ DEBUG = CMD.debug
+
+ LOG.debug(u"CMD: %(CMD)s", CMD=CMD)
+
+ retVal = 0
+
+ for fname in CMD.files:
+ opts = ParseOptions(
+ fname = fname
+ , id_prefix = CMD.id_prefix
+ , skip_preamble = CMD.skip_preamble
+ , skip_epilog = CMD.skip_epilog
+ , out = STREAM.appl_out
+ , translator = ReSTTranslator()
+ ,)
+ opts.set_defaults()
+
+ if CMD.list_exports or CMD.list_internals:
+ opts.gather_context = True
+ opts.translator = ListTranslator(CMD.list_exports, CMD.list_internals)
+
+ elif CMD.use_names:
+ opts.use_names = CMD.use_names
+
+ elif CMD.exported or CMD.internal:
+ # gather exported symbols ...
+ src = readFile(opts.fname)
+ ctx = ParserContext()
+ Parser.gather_context(src, ctx)
+
+ opts.translator = ReSTTranslator()
+ opts.error_missing = False
+ opts.use_names = ctx.exported_symbols
+ opts.skip_names = []
+
+ if CMD.internal:
+ opts.use_names = []
+ opts.skip_names = ctx.exported_symbols
+
+ parser = Parser(opts)
+ parser.parse()
+ if parser.errors:
+ retVal = 1
+
+ return retVal
+
+# ==============================================================================
+# API
+# ==============================================================================
+
+# ------------------------------------------------------------------------------
+class TranslatorAPI(object):
+# ------------------------------------------------------------------------------
+ u"""
+ Abstract kernel-doc translator.
+
+ :cvar list HIGHLIGHT_MAP: highlight mapping
+ :cvar tuple LINE_COMMENT: tuple with start-/end- comment tags
+ """
+
+ HIGHLIGHT_MAP = [
+ ( type_constant , None )
+ , ( type_func , None )
+ , ( type_param , None )
+ , ( type_struct , None )
+ , ( type_env , None )
+ , ( type_enum_full , None )
+ , ( type_struct_full , None ) ]
+
+ LINE_COMMENT = ("# ", "")
+
+ def __init__(self):
+ self.options = None
+ self.parser = None
+
+ def setParser(self, parser):
+ self.options = parser.options
+ self.parser = parser
+
+ def highlight(self, cont):
+ u"""returns *highlighted* text"""
+ if self.options.highlight:
+ return map_text(cont, self.HIGHLIGHT_MAP)
+ return cont
+
+ @classmethod
+ def comment(cls, cont):
+ u"""returns *commented* text"""
+
+ start, end = cls.LINE_COMMENT
+ if not start and not end:
+ return cont
+
+ retVal = []
+ for line in cont.split("\n"):
+ if line.strip():
+ retVal.append(start + line + end)
+ else:
+ retVal.append("")
+ return "\n".join(retVal)
+
+ def write(self, *objects):
+ u"""Write *objects* to stream.
+
+ Write unicode-values of the *objects* to :py:attr:``self.options.out``.
+
+ :param *objects: The positional arguments are the objects with the
+ content to write.
+ """
+ for obj in objects:
+ cont = unicode(obj)
+ self.options.out.write(cont)
+
+ def write_comment(self, *objects):
+ u"""Write *objects* as comments to stream."""
+ for obj in objects:
+ cont = unicode(obj)
+ self.write(self.comment(cont))
+
+ def eof(self):
+ if self.options.eof_newline:
+ self.write("\n")
+
+ # API
+ # ---
+
+ def output_preamble(self):
+ raise NotImplementedError
+
+ def output_epilog(self):
+ raise NotImplementedError
+
+ def output_doc_section(
+ self
+ , sections = None # ctx.sections
+ , ):
+ raise NotImplementedError
+
+ def output_function_decl(
+ self
+ , function = None # ctx.decl_name
+ , return_type = None # ctx.return_type
+ , parameterlist = None # ctx.parameterlist
+ , parameterdescs = None # ctx.parameterdescs
+ , parametertypes = None # ctx.parametertypes
+ , sections = None # ctx.sections
+ , purpose = None # ctx.decl_purpose
+ , ):
+ raise NotImplementedError
+
+ def output_struct_decl(
+ self
+ , decl_name = None # ctx.decl_name
+ , decl_type = None # ctx.decl_type
+ , parameterlist = None # ctx.parameterlist
+ , parameterdescs = None # ctx.parameterdescs
+ , parametertypes = None # ctx.parametertypes
+ , sections = None # ctx.sections
+ , purpose = None # ctx.decl_purpose
+ , ):
+ raise NotImplementedError
+
+ def output_union_decl(self, *args, **kwargs):
+ self.output_struct_decl(*args, **kwargs)
+
+ def output_enum_decl(
+ self
+ , enum = None # ctx.decl_name
+ , parameterlist = None # ctx.parameterlist
+ , parameterdescs = None # ctx.parameterdescs
+ , sections = None # ctx.sections
+ , purpose = None # ctx.decl_purpose
+ , ):
+ raise NotImplementedError
+
+ def output_typedef_decl(
+ self
+ , typedef = None # ctx.decl_name
+ , sections = None # ctx.sections
+ , purpose = None # ctx.decl_purpose
+ , ):
+ raise NotImplementedError
+
+# ------------------------------------------------------------------------------
+class NullTranslator(object):
+# ------------------------------------------------------------------------------
+ u"""
+ Null translator, translates nothing, just parse.
+ """
+ HIGHLIGHT_MAP = []
+ LINE_COMMENT = ("", "")
+ # pylint: disable=C0321
+ def output_preamble(self, *args, **kwargs): pass
+ def output_epilog(self, *args, **kwargs): pass
+ def output_doc_section(self, *args, **kwargs): pass
+ def output_function_decl(self, *args, **kwargs): pass
+ def output_struct_decl(self, *args, **kwargs): pass
+ def output_union_decl(self, *args, **kwargs): pass
+ def output_enum_decl(self, *args, **kwargs): pass
+ def output_typedef_decl(self, *args, **kwargs): pass
+
+# ------------------------------------------------------------------------------
+class ListTranslator(TranslatorAPI):
+# ------------------------------------------------------------------------------
+
+ u"""
+ Generates a list of kernel-doc symbols.
+ """
+
+ def __init__(self, list_exported, list_internal_types
+ , *args, **kwargs):
+ super(ListTranslator, self).__init__(*args, **kwargs)
+
+ self.list_exported = list_exported
+ self.list_internal_types = list_internal_types
+
+ self.names = dict()
+ for t in Parser.DOC_TYPES:
+ self.names[t] = []
+
+ def get_type(self, name):
+ for t, l in self.names.items():
+ if name in l:
+ return t
+ return None
+
+ def output_preamble(self):
+ pass
+
+ def output_epilog(self):
+ pass
+
+ def output_doc_section(self, sections = None):
+ for header in sections.keys():
+ self.names["DOC"].append(header)
+
+ def output_function_decl(self, **kwargs):
+ self.names["function"].append(kwargs["function"])
+
+ def output_struct_decl(self, **kwargs):
+ self.names["struct"].append(kwargs["decl_name"])
+
+ def output_union_decl(self, **kwargs):
+ self.names["union"].append(kwargs["decl_name"])
+
+ def output_enum_decl(self, **kwargs):
+ self.names["enum"].append(kwargs["enum"])
+
+ def output_typedef_decl(self, **kwargs):
+ self.names["typedef"].append(kwargs["typedef"])
+
+ def eof(self):
+
+ if self.list_exported:
+ self.parser.info("list exported symbols")
+ for name in self.parser.ctx.exported_symbols:
+ t = self.get_type(name)
+ if t is None:
+ self.parser.warn("exported symbol '%(name)s' is undocumented"
+ , name = name)
+ t = "undocumented"
+ self.write("[exported %-10s] %s \n" % (t, name))
+
+ if self.list_internal_types:
+ self.parser.info("list internal names")
+ for t, l in self.names.items():
+ if not ("all" in self.list_internal_types
+ or t in self.list_internal_types):
+ continue
+ for name in l:
+ if name not in self.parser.ctx.exported_symbols:
+ self.write("[internal %-10s] %s \n" % (t, name))
+
+# ------------------------------------------------------------------------------
+class ReSTTranslator(TranslatorAPI):
+# ------------------------------------------------------------------------------
+
+ u"""
+ Translate kernel-doc to reST markup.
+
+ :cvar list HIGHLIGHT_map: Escape common reST (inline) markups. Classic
+ kernel-doc comments contain characters and strings like ``*`` or
+ trailing ``_``, which are inline markups in reST. These special strings
+ has to be masked in reST.
+
+ """
+ INDENT = " "
+ LINE_COMMENT = (".. ", "")
+
+ HIGHLIGHT_MAP = [
+ ( type_constant , r"``\1``" )
+ , ( type_func , r":c:func:`\1`")
+ , ( type_param , r"``\1``" )
+ , ( type_struct , r":c:type:`struct \1 <\1>`")
+ , ( type_env , r"``\1``" )
+ , ( type_enum_full , r":c:type:`\1 \2 <\2>`")
+ , ( type_struct_full , r":c:type:`\1 \2 <\2>`" )
+ , ]
+
+ MASK_REST_INLINES = [
+ (RE(r"(\*)") , r"\\\1") # emphasis
+ , (RE(r"(`)") , r"\\\1") # interpreted text & inline literals
+ , (RE(r"(_\s)") , r"\\\1") # trailing underline
+ , (RE(r"(\s)_(\w)") , r"\1_\2") # leading underline
+ , (RE(r"(\|)") , r"\\\1") # substitution references
+ , ]
+
+ FUNC_PTR = RE(r"([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)")
+ BITFIELD = RE(r"^(.*?)\s*(:.*)")
+
+ def highlight(self, text):
+ if self.options.markup == "kernel-doc":
+ text = map_text(text, self.MASK_REST_INLINES + self.HIGHLIGHT_MAP )
+ return text
+
+ def format_block(self, content):
+ u"""format the content (string)"""
+ lines = []
+ if self.options.markup == "kernel-doc":
+ lines = [ l.strip() for l in content.split("\n")]
+ elif self.options.markup == "reST":
+ lines = [ l.rstrip() for l in content.split("\n")]
+ return "\n".join(lines)
+
+ def write_anchor(self, refname):
+ ID = refname
+ if self.options.id_prefix:
+ ID = self.options.id_prefix + "." + ID
+ ID = normalize_id(ID)
+ self.write("\n.. _`%s`:\n" % ID)
+
+ HEADER_TAGS = (
+ "#" # level 0 / part with overline
+ "=" # level 1 / chapter with overline
+ "=" # level 2 / sec
+ "-" # level 3 / subsec
+ "-" # level 4 / subsubsec
+ '"' ) # level 5 / para
+
+ def write_header(self, header, sec_level=2):
+ header = self.highlight(header)
+ sectag = self.HEADER_TAGS[sec_level]
+ if sec_level < 2:
+ self.write("\n", (sectag * len(header)))
+ self.write("\n%s" % header)
+ self.write("\n", (sectag * len(header)), "\n")
+
+ def write_section(self, header, content, sec_level=2, ID=None):
+ if ID:
+ self.write_anchor(ID)
+ self.write_header(header, sec_level=sec_level)
+ if (header.lower() == "example"
+ and self.options.markup == "kernel-doc"):
+ self.write("\n.. code-block:: c\n\n")
+ for l in textwrap.dedent(content).split("\n"):
+ if not l.strip():
+ self.write("\n")
+ else:
+ self.write(self.INDENT, l, "\n")
+ else:
+ content = self.format_block(content)
+ content = self.highlight(content)
+ self.write("\n" + content)
+
+ self.write("\n")
+
+ def write_definition(self, term, definition, prefix=""):
+ term = normalize_ws(term) # term has to be a "one-liner"
+ term = self.highlight(term)
+ if definition != Parser.undescribed:
+ definition = self.format_block(definition)
+ definition = self.highlight(definition)
+ self.write("\n", prefix, term)
+ for l in textwrap.dedent(definition).split("\n"):
+ self.write("\n", prefix)
+ if l.strip():
+ self.write(self.INDENT, l)
+ self.write("\n")
+
+ def write_func_param(self, param, descr):
+ param = self.highlight(param)
+ self.write("\n", self.INDENT, param)
+
+ if descr != Parser.undescribed:
+ descr = self.format_block(descr)
+ descr = self.highlight(descr)
+ for l in textwrap.dedent(descr).split("\n"):
+ self.write("\n")
+ if l.strip():
+ self.write(self.INDENT * 2, l)
+ self.write("\n")
+
+ def output_preamble(self):
+ if self.options.mode_line:
+ self.write_comment(
+ "-*- coding: %s; mode: rst -*-\n"
+ % (getattr(self.options.out, "encoding", "utf-8") or "utf-8").lower())
+
+ if self.options.preamble:
+ self.write(self.options.preamble, "\n")
+
+ if self.options.top_title:
+ self.write_anchor(self.options.top_title)
+ self.write_header(self.options.top_title, 0)
+ if self.options.top_link:
+ self.write("\n", self.options.top_link % self.options, "\n")
+
+ def output_epilog(self):
+ if self.options.epilog is not None:
+ self.write(self.options.epilog)
+
+ def output_doc_section(self, sections = None):
+ for header, content in sections.items():
+ self.write_section(header, content, sec_level=2, ID=header)
+
+ def output_function_decl(
+ self
+ , function = None # ctx.decl_name
+ , return_type = None # ctx.return_type
+ , parameterlist = None # ctx.parameterlist
+ , parameterdescs = None # ctx.parameterdescs
+ , parametertypes = None # ctx.parametertypes
+ , sections = None # ctx.sections
+ , purpose = None # ctx.decl_purpose
+ , ):
+ self.write_anchor(function)
+ self.write_header(function, sec_level=2)
+
+ # write function definition
+
+ self.write("\n.. c:function:: ")
+ if return_type and return_type.endswith("*"):
+ self.write(return_type, function, "(")
+ else:
+ self.write(return_type, " ", function, "(")
+
+ p_list = []
+
+ for p_name in parameterlist:
+ p_type = parametertypes[p_name]
+
+ if self.FUNC_PTR.search(p_type):
+ # pointer to function
+ p_list.append("%s) %s (%s"
+ % (self.FUNC_PTR[0], p_name, self.FUNC_PTR[1]))
+ elif p_type.endswith("*"):
+ # pointer
+ p_list.append("%s%s" % (p_type, p_name))
+ else:
+ p_list.append("%s %s" % (p_type, p_name))
+
+ self.write(", ".join(p_list), ")\n")
+
+ # purpose
+
+ if purpose:
+ self.write("\n", self.INDENT, self.highlight(purpose), "\n")
+
+ # parameter descriptions
+
+ for p_name in parameterlist:
+
+ p_type = parametertypes[p_name]
+ p_name = re.sub(r"\[.*", "", p_name)
+ p_desc = parameterdescs[p_name]
+
+ param = ""
+ if self.FUNC_PTR.search(p_type):
+ # pointer to function
+ param = ":param (%s) %s (%s):" % (self.FUNC_PTR[0], p_name, self.FUNC_PTR[1])
+ elif p_type.endswith("*"):
+ # pointer & pointer to pointer
+ param = ":param %s%s:" % (p_type, p_name)
+ elif p_name == "...":
+ param = ":param %s :" % (p_name)
+ else:
+ param = ":param %s %s:" % (p_type, p_name)
+
+ self.write_func_param(param, p_desc)
+
+ # sections
+
+ for header, content in sections.items():
+ self.write_section(
+ header
+ , content
+ , sec_level = 3
+ , ID = function + "." + header)
+
+ # INSPECT and CONSOLE() # pylint: disable=W0106
+
+ def output_struct_decl(
+ self
+ , decl_name = None # ctx.decl_name
+ , decl_type = None # ctx.decl_type
+ , parameterlist = None # ctx.parameterlist
+ , parameterdescs = None # ctx.parameterdescs
+ , parametertypes = None # ctx.parametertypes
+ , sections = None # ctx.sections
+ , purpose = None # ctx.decl_purpose
+ , ):
+ self.write_anchor(decl_name)
+ self.write_header("%s %s" % (decl_type, decl_name), sec_level=2)
+
+ # write struct definition
+
+ self.write("\n.. c:type:: %s\n\n" % decl_name)
+
+ # purpose
+
+ if purpose:
+ self.write(self.INDENT, self.highlight(purpose), "\n")
+
+ # definition
+
+ self.write_anchor(decl_name + "." + Parser.section_def)
+ self.write_header(Parser.section_def, sec_level=3)
+ self.write("\n.. code-block:: c\n\n")
+ self.write(self.INDENT, decl_type, " ", decl_name, " {\n")
+
+ for p_name in parameterlist:
+ p_type = parametertypes[p_name]
+
+ if MACRO.match(p_name):
+ self.write(self.INDENT * 2, "%s\n" % p_name)
+
+ elif self.FUNC_PTR.search(p_type):
+ # pointer to function
+ self.write(
+ self.INDENT * 2
+ , "%s %s) (%s);\n" % (self.FUNC_PTR[0], p_name, self.FUNC_PTR[1]))
+
+ elif self.BITFIELD.match(p_type):
+ self.write(
+ self.INDENT * 2
+ , "%s %s%s;\n" % (self.BITFIELD[0], p_name, self.BITFIELD[1]))
+ elif p_type.endswith("*"):
+ # pointer
+ self.write(
+ self.INDENT * 2
+ , "%s%s;\n" % (p_type, p_name))
+
+ else:
+ self.write(
+ self.INDENT * 2
+ , "%s %s;\n" % (p_type, p_name))
+
+ self.write(self.INDENT, "}\n")
+
+ # member description
+
+ self.write_anchor(decl_name + "." + Parser.section_members)
+ self.write_header(Parser.section_members, sec_level=3)
+
+ for p_name in parameterlist:
+ if MACRO.match(p_name):
+ continue
+ p_name = re.sub(r"\[.*", "", p_name)
+ p_desc = parameterdescs.get(p_name, None)
+ if p_desc is None:
+ continue
+ self.write_definition(p_name, p_desc)
+
+ # sections
+
+ for header, content in sections.items():
+ self.write_section(
+ header
+ , content
+ , sec_level = 3
+ , ID = decl_name + "." + header)
+
+ # INSPECT and CONSOLE() # pylint: disable=W0106
+
+ def output_enum_decl(
+ self
+ , enum = None # ctx.decl_name
+ , parameterlist = None # ctx.parameterlist
+ , parameterdescs = None # ctx.parameterdescs
+ , sections = None # ctx.sections
+ , purpose = None # ctx.decl_purpose
+ , ):
+ self.write_anchor(enum)
+ self.write_header("enum %s" % enum, sec_level=2)
+
+ # write union definition
+
+ self.write("\n.. c:type:: %s\n\n" % enum)
+
+ # purpose
+
+ if purpose:
+ self.write(self.INDENT, self.highlight(purpose), "\n")
+
+ # definition
+
+ self.write_anchor(enum + "." + Parser.section_def)
+ self.write_header(Parser.section_def, sec_level=3)
+ self.write("\n.. code-block:: c\n\n")
+ self.write(self.INDENT, "enum ", enum, " {")
+
+ e_list = parameterlist[:]
+ while e_list:
+ e = e_list.pop(0)
+ self.write("\n", self.INDENT *2, e)
+ if e_list:
+ self.write(",")
+ self.write("\n", self.INDENT, "};\n")
+
+ # constants description
+
+ self.write_anchor(enum + "." + Parser.section_constants)
+ self.write_header(Parser.section_constants, sec_level=3)
+
+ for p_name, p_desc in parameterdescs.items():
+ self.write_definition(p_name, p_desc)
+
+ # sections
+
+ for header, content in sections.items():
+ self.write_section(
+ header
+ , content or "???"
+ , sec_level = 3
+ , ID = enum + "." + header)
+
+ # INSPECT and CONSOLE() # pylint: disable=W0106
+
+ def output_typedef_decl(
+ self
+ , typedef = None # ctx.decl_name
+ , sections = None # ctx.sections
+ , purpose = None # ctx.decl_purpose
+ , ):
+ self.write_anchor(typedef)
+ self.write_header("typedef %s" % typedef, sec_level=2)
+
+ # write typdef definition
+
+ self.write("\n.. c:type:: %s\n\n" % typedef)
+ if purpose:
+ self.write(self.INDENT, self.highlight(purpose), "\n")
+
+ for header, content in sections.items():
+ self.write_section(
+ header
+ , content or "???"
+ , sec_level = 3
+ , ID = typedef + "." + header)
+
+ # INSPECT and CONSOLE() # pylint: disable=W0106
+
+
+# ------------------------------------------------------------------------------
+class ParseOptions(Container):
+# ------------------------------------------------------------------------------
+
+ PARSE_OPTION_RE = r"^/\*+\s*parse-%s:\s*([a-zA-Z0-9_-]*?)\s*\*/+\s*$"
+ PARSE_OPTIONS = [
+ ("highlight", ["on","off"], "setOnOff")
+ , ("INSPECT", ["on","off"], "setINSPECT")
+ , ("markup", ["reST", "kernel-doc"], "setVal")
+ , ("SNIP", [], "setVal")
+ , ("SNAP", [], "snap")
+ , ]
+
+ def __init__(self, *args, **kwargs):
+
+ self.id_prefix = None # A prefix for generated IDs.
+ self.out = None # File descriptor for output.
+ self.eof_newline = True # write newline on end of file
+ self.translator = None # Translator object.
+
+ self.src_tree = SRCTREE # root of the kernel sources
+ self.rel_fname = "" # pathname relative to src_tree
+ self.fname = "" # absolute pathname
+
+ # self.encoding: the input encoding (encoding of the parsed source
+ # file), the output encoding could be seeked from the file-descriptor at
+ # self.out.
+
+ self.encoding = "utf-8"
+
+ # control which content to print
+
+ self.use_names = [] # positiv list of names to print / empty list means "print all"
+ self.skip_names = [] # negativ list of names (not to print)
+ self.error_missing = True # report missing names as errors / else warning
+
+ # self.gather_context: [True/False] Scan additional context from the
+ # parsed source. E.g.: The list of exported symbols is a part of the
+ # parser's context. If the context of exported symbols is needed, we
+ # have to parse twice. First to find exported symbols, store them in the
+ # context and a second once for *normal* parsing within this modified
+ # *context*.
+
+ self.gather_context = False
+
+ # epilog / preamble
+
+ self.skip_preamble = False
+ self.skip_epilog = False
+ self.mode_line = True # write mode-line in the very first line
+ self.top_title = "" # write a title on top of the preamble
+ self.top_link = "" # if top_title, add link to the *top* of the preamble
+ self.preamble = "" # additional text placed into the preamble
+ self.epilog = "" # text placed into the epilog
+
+ # default's of filtered PARSE_OPTIONS
+
+ self.opt_filters = dict()
+ self.markup = "kernel-doc"
+ self.highlight = True # switch highlighting on/off
+ self.add_filters(self.PARSE_OPTIONS)
+
+ # SNIP / SNAP
+ self.SNIP = None
+
+ super(ParseOptions, self).__init__(self, *args, **kwargs)
+
+ # absolute and relativ filename
+
+ if self.src_tree and self.fname and os.path.isabs(self.fname):
+ # if SCRTREE and abspath fname is given, determine relativ pathname
+ self.rel_fname = os.path.relpath(self.fname, self.src_tree)
+
+ if self.src_tree and self.fname and not os.path.isabs(self.fname):
+ # if SCRTREE and relative fname is given, drop fname and set rel_fname
+ self.rel_fname = self.fname
+ self.fname = ""
+
+ if self.src_tree and self.rel_fname:
+ self.fname = os.path.join(self.src_tree, self.rel_fname)
+ else:
+ LOG.warn("no relative pathname given / no SRCTREE: "
+ " features based on these settings might not work"
+ " as expected!")
+ if not self.fname:
+ LOG.error("no source file given!")
+
+ def set_defaults(self):
+
+ # default top title and top link
+
+ if self.fname and self.top_title == "":
+ self.top_title = os.path.basename(self.fname)
+ if self.top_title:
+ self.top_title = self.top_title % self
+
+ if self.top_link == "":
+ if self.rel_fname:
+ self.top_link = GIT_REF % self
+ else:
+ LOG.warn("missing SRCTREE, can't set *top_link* option")
+ if self.top_link:
+ self.top_link = self.top_link % self
+
+ # default preamble
+
+ if self.preamble == "":
+ self.preamble = self.translator.comment("src-file: %s" % (self.rel_fname or self.fname))
+ if self.preamble:
+ self.preamble = self.preamble % self
+
+ # default epilog
+
+ if self.epilog == "":
+ self.epilog = self.translator.comment(
+ "\nThis file was automatic generated / don't edit.")
+ if self.epilog:
+ self.epilog = self.epilog % self
+
+ def add_filters(self, parse_options):
+
+ # pylint: disable=W0613, W0603
+
+ def setINSPECT(name, val):
+ global INSPECT
+ INSPECT = bool(val == "on")
+
+ _actions = dict(
+ setOnOff = lambda name, val: ( name, bool(val == "on") )
+ , setVal = lambda name, val: ( name, val )
+ , snap = lambda name, val: ( "SNIP", "" )
+ , setINSPECT = setINSPECT
+ , )
+
+ for option, val_list, action in parse_options:
+ self.opt_filters[option] = (
+ RE(self.PARSE_OPTION_RE % option), val_list, _actions[action])
+
+ def filter_opt(self, line, parser):
+
+ for name, (regexpr, val_list, action) in self.opt_filters.items():
+ if regexpr.match(line):
+ line = None
+ value = regexpr[0]
+ if val_list and value not in val_list:
+ parser.error("unknown parse-%(name)s value: '%(value)s'"
+ , name=name, value=value)
+ else:
+ opt_val = action(name, value)
+ if opt_val is not None:
+ name, value = opt_val
+ self[name] = value
+ parser.info(
+ "set parse-option: %(name)s = '%(value)s'"
+ , name=name, value=value)
+ break
+ return line
+
+# ------------------------------------------------------------------------------
+class ParserContext(Container):
+# ------------------------------------------------------------------------------
+
+ def __init__(self, *args, **kwargs):
+ self.line_no = 0
+
+ self.contents = ""
+ self.section = Parser.section_default
+
+ # self.sections: ordered dictionary (list) of sections as they appear in
+ # the source. The sections are set by Parser.dump_section
+ self.sections = collections.OrderedDict()
+ self.sectcheck = []
+
+ self.prototype = ""
+
+ # self.parameterlist: ordered list of the parameters as they appear in
+ # the source. The parameterlist is set by Parser.push_parameter and
+ # Parser.dump_enum
+ self.parameterlist = []
+
+ # self.parametertypes: dictionary of <parameter-name>:<type>
+ # key/values of the parameters. Set by Parser.push_parameter
+ self.parametertypes = dict()
+
+ # self.parameterdescs: dictionary of <'@parameter'>:<description>
+ # key/values of the parameters. Set by Parser.dump_section
+ self.parameterdescs = dict()
+
+ # self.constants: dictionary of <'%CONST'>:<description>
+ # key/values. Set by Parser.dump_section
+ self.constants = dict()
+
+ self.decl_name = ""
+ self.decl_type = "" # [struct|union|enum|typedef|function]
+ self.decl_purpose = ""
+ self.return_type = ""
+
+ # record ranslated names
+ self.translated_names = []
+
+ #self.struct_actual = ""
+
+ # Additional context from the parsed source
+
+ # self.exported: list of exported symbols
+ self.exported_symbols = []
+
+ # self.mod_xxx: Module informations
+ self.mod_authors = []
+ self.mod_descr = ""
+ self.mod_license = ""
+
+ # SNIP / SNAP
+ self.snippets = collections.OrderedDict()
+
+ super(ParserContext, self).__init__(self, *args, **kwargs)
+
+ def new(self):
+ return self.__class__(
+ line_no = self.line_no
+ , exported_symbols = self.exported_symbols
+ , translated_names = self.translated_names
+ , snippets = self.snippets )
+
+
+class ParserBuggy(RuntimeError):
+ u"""Exception raised when the parser implementation seems buggy.
+
+ The parser implementation perform some integrity tests at runtime. This
+ exception type mainly exitst to improve the regualar expressions which are
+ used to parse and analyze the kernels source code.
+
+ In the exception message the last position the parser parsed is stored, this
+ positon may, but does not need to be related with the exception (it is only
+ an additional information which might help).
+
+ Under normal circumstances, exceptions of this type should never arise,
+ unless the implementation of the parser is buggy."""
+
+ def __init__(self, parserObj, message):
+
+ message = ("last parse position %s:%s\n"
+ % (parserObj.ctx.line_no, parserObj.self.options.fname)
+ + message)
+ super(ParserBuggy, self).__init__(message)
+ self.parserObj = parserObj
+
+# ------------------------------------------------------------------------------
+class Parser(SimpleLog):
+# ------------------------------------------------------------------------------
+
+ u"""
+ kernel-doc comments parser
+
+ States:
+
+ * 0 - normal code
+ * 1 - looking for function name
+ * 2 - scanning field start.
+ * 3 - scanning prototype.
+ * 4 - documentation block
+ * 5 - gathering documentation outside main block (see Split Doc State)
+
+ Split Doc States:
+
+ * 0 - Invalid (Before start or after finish)
+ * 1 - Is started (the /** was found inside a struct)
+ * 2 - The @parameter header was found, start accepting multi paragraph text.
+ * 3 - Finished (the */ was found)
+ * 4 - Error: Comment without header was found. Spit a error as it's not
+ proper kernel-doc and ignore the rest.
+ """
+
+ LOG_FORMAT = "%(fname)s:%(line_no)s :%(logclass)s: %(message)s\n"
+
+ # DOC_TYPES: types of documentation gathered by the parser
+ DOC_TYPES = ["DOC", "function", "struct", "union", "enum", "typedef"]
+
+ undescribed = "*undescribed*"
+
+ section_descr = "Description"
+ section_def = "Definition"
+ section_members = "Members"
+ section_constants = "Constants"
+ section_intro = "Introduction"
+ section_context = "Context"
+ section_return = "Return"
+ section_default = section_descr
+
+ def __init__(self, parse_options):
+ super(Parser, self).__init__()
+ self.options = parse_options
+ self.ctx = ParserContext()
+
+ self.translator = self.options.translator
+ self.translator.setParser(self)
+
+ # raw data akku
+ self.rawdata = ""
+
+ # flags:
+ self.state = 0
+ self.split_doc_state = 0
+ self.in_intro = True
+ self.in_doc_sect = False
+ self.in_purpose = False
+ self.brcount = 0
+ self.warnings = 0
+ self.errors = 0
+ self.anon_struct_union = False
+
+ def reset_state(self):
+ self.ctx = self.ctx.new()
+ self.state = 0
+ self.split_doc_state = 0
+ self.in_doc_sect = False
+ self.in_purpose = False
+ self.brcount = 0
+ self.anon_struct_union = False
+
+ # ------------------------------------------------------------
+ # Log
+ # ------------------------------------------------------------
+
+ def error(self, message, **replace):
+ replace.update(dict(fname=self.options.fname, line_no=self.ctx.line_no))
+ self.errors += 1
+ super(Parser, self).error(message, **replace)
+
+ def warn(self, message, **replace):
+ replace.update(dict(fname=self.options.fname, line_no=self.ctx.line_no))
+ self.warnings += 1
+ super(Parser, self).warn(message, **replace)
+
+ def info(self, message, **replace):
+ replace.update(dict(fname=self.options.fname, line_no=self.ctx.line_no))
+ super(Parser, self).info(message, **replace)
+
+ def debug(self, message, **replace):
+ replace.update(dict(fname=self.options.fname, line_no=self.ctx.line_no))
+ super(Parser, self).debug(message, **replace)
+
+ # ------------------------------------------------------------
+ # state parser
+ # ------------------------------------------------------------
+
+ def parse(self, src=None):
+ if src is not None:
+ for line in src:
+ self.feed(line)
+ else:
+ with openTextFile(self.options.fname, encoding=self.options.encoding) as src:
+ for line in src:
+ self.feed(line)
+ self.close()
+
+ @classmethod
+ def gather_context(cls, src, ctx):
+ u"""Scan source about context informations.
+
+ Scans *whole* source (e.g. :py:attr:`Parser.rawdata`) about data relevant
+ for the context (e.g. exported symbols).
+
+ Names of exported symbols gathered in :py:attr:`ParserContext.exported`.
+ The list contains names (symbols) which are exported using the
+ EXPORT_SYMBOL macro.
+
+ * ``EXPORT_SYMBOL(<name>)``
+ * ``EXPORT_SYMBOL_GPL(<name>)``
+ * ``EXPORT_SYMBOL_GPL_FUTURE(<name>)``)
+
+ .. hint::
+
+ A exported symbol does not necessarily have a corresponding source code
+ comment with a documentation.
+
+ Module information comes from the ``MODULE_xxx`` macros. Module
+ informations are gathered in ``ParserContext.module_xxx``:
+
+ * ``MODULE_AUTHOR("...")``: Author entries are collected in a list in
+ :py:attr:`ParserContext.mod_authors`
+
+ * ``MODULE_DESCRIPTION("...")``: A concatenated string in
+ :py:attr:`ParserContext.mod_descr`
+
+ * ``MODULE_LICENSE``("..."): String with comma seprated licences in
+ :py:attr:`ParserContext.mod_license`.
+
+ .. hint::
+
+ While parsing header files, about kernel-doc, you will not find the
+ ``MODULE_xxx`` macros, because they are commonly used in the ".c"
+ files.
+ """
+
+ LOG.debug("gather_context() regExp: %(pattern)s", pattern=EXPORTED_SYMBOLS.pattern)
+ for match in EXPORTED_SYMBOLS.findall(src):
+ name = match[3]
+ LOG.info("exported symbol: %(name)s", name = name)
+ ctx.exported_symbols.append(name)
+
+ LOG.debug("gather_context() regExp: %(pattern)s", pattern=MODULE_INFO.pattern)
+
+ for match in MODULE_INFO.findall(src):
+ info_type = match[1]
+ content = match[2]
+ if info_type == "AUTHOR":
+ ctx.mod_authors.append(content)
+ elif info_type == "DESCRIPTION":
+ ctx.mod_descr += content + " "
+ elif info_type == "LICENSE":
+ ctx.mod_license += content + ", "
+
+ LOG.info("mod_authors: %(x)s", x = ctx.mod_authors)
+ LOG.info("mod_descr: %(x)s", x = ctx.mod_descr)
+ LOG.info("mod_license : %(x)s", x = ctx.mod_license)
+
+ def close(self):
+ self._parse(eof=True)
+
+ def feed(self, data):
+ self.rawdata = self.rawdata + data
+ self._parse()
+
+ def _parse(self, eof=False):
+
+ if self.options.gather_context:
+ # Scan additional context from the parsed source. For this, collect
+ # all lines in self.rawdata until EOF. On EOF, scan rawdata about
+ # (e.g.) exported symbols and after this, continue with the *normal*
+ # parsing.
+ if not eof:
+ return
+ else:
+ self.gather_context(self.rawdata, self.ctx)
+
+ lines = self.rawdata.split("\n")
+
+ if not eof:
+ # keep last line, until EOF
+ self.rawdata = lines[-1]
+ lines = lines[:-1]
+
+ if self.in_intro:
+ self.dump_preamble()
+ self.in_intro = False
+
+ for l in lines:
+ self.ctx.line_no += 1
+ l = self.options.filter_opt(l, self)
+ if l is None:
+ continue
+
+ if self.options.SNIP:
+ # record snippet
+ val = self.ctx.snippets.get(self.options.SNIP, "")
+ if val or l:
+ self.ctx.snippets[self.options.SNIP] = val + l + "\n"
+
+ state = getattr(self, "state_%s" % self.state)
+ try:
+ state(l)
+ except Exception:
+ self.warn("total errors: %(errors)s / warnings: %(warnings)s"
+ , errors=self.errors, warnings=self.warnings)
+ self.warnings -= 1
+ self.error("unhandled exception in line: %(l)s", l=l)
+ raise
+
+ if eof:
+ self.dump_epilog()
+ self.translator.eof()
+
+ # log requested but missed documentation
+ log_missed = self.error
+ if not self.options.error_missing:
+ log_missed = self.warn
+
+ for name in self.options.use_names:
+ if name not in self.ctx.translated_names:
+ log_missed("no documentation for '%(name)s' found", name=name)
+
+ if self.errors or self.warnings:
+ self.warn("total errors: %(errors)s / total warnings: %(warnings)s"
+ , errors=self.errors, warnings=self.warnings)
+ self.warnings -= 1
+ global INSPECT
+ INSPECT = False
+
+ def state_0(self, line):
+ u"""state: 0 - normal code"""
+
+ if doc_start.match(line):
+ self.debug("START: kernel-doc comment / switch state 0 --> 1")
+ self.state = 1
+ self.in_doc_sect = False
+
+ def state_1(self, line):
+ u"""state: 1 - looking for function name"""
+
+ if doc_block.match(line):
+ self.debug("START: DOC block / switch state 1 --> 4")
+ self.state = 4
+ self.ctx.contents = ""
+ self.ctx.section = self.section_intro
+ if doc_block[0].strip():
+ self.ctx.section = self.sect_title(doc_block[0])
+ self.info("DOC: %(sect)s", sect=self.ctx.section)
+
+ elif doc_decl.match(line):
+ self.debug("START: declaration / switch state 1 --> 2")
+ self.state = 2
+
+ identifier = doc_decl[0].strip()
+ if doc_decl_ident.search(line):
+ identifier = doc_decl_ident[0]
+
+ self.ctx.decl_type = "function"
+ for t in ["struct", "union", "enum", "typedef"]:
+ if identifier.startswith(t):
+ self.ctx.decl_type = t
+ break
+
+ self.debug("FLAG: in_purpose=True")
+ self.in_purpose = True
+
+ self.info("scanning doc for: %(t)s '%(i)s'", t=self.ctx.decl_type, i = identifier)
+
+ self.ctx.decl_purpose = ""
+ if doc_decl_purpose.search(line):
+ self.ctx.decl_purpose = doc_decl_purpose[0].strip()
+
+ if not self.ctx.decl_purpose:
+ self.warn("missing initial short description of '%(i)s' on line -->|%(line)s|<--"
+ , i=identifier, line=line)
+
+
+ else:
+ self.warn("can't understand: -->|%(line)s|<--"
+ " - I thought it was a doc line" , line=line)
+ self.state = 0
+
+ def sect_title(self, title):
+ u"""Normalize common section titles"""
+ # fix varius notations for the "Return:" section
+
+ retVal = title
+
+ if title.lower() in ["description", ]:
+ retVal = self.section_descr
+
+ elif title.lower() in ["introduction", "intro"]:
+ retVal = self.section_intro
+
+ elif title.lower() in ["context", ]:
+ retVal = self.section_context
+
+ elif title.lower() in ["return", "returns"]:
+ retVal = self.section_return
+
+ return retVal
+
+ def state_2(self, line):
+ u"""state: 2 - scanning field start. """
+
+ new_sect = ""
+ new_cont = ""
+
+ if not doc_sect_except.match(line):
+
+ # probe different sect start pattern ...
+
+ if self.options.markup == "reST":
+ if doc_param.match(line):
+ # this is a line with a parameter definition
+ new_sect = self.sect_title(doc_param[0].strip())
+ new_cont = doc_param[1].strip()
+ elif reST_sect.match(line):
+ # this is a line with a section definition
+ new_sect = self.sect_title(reST_sect[0].strip())
+ new_cont = ""
+ else: # kernel-doc vintage mode
+ if doc_sect.match(line):
+ # this is a line with a parameter or section definition
+ new_sect = self.sect_title(doc_sect[0].strip())
+ new_cont = doc_sect[1].strip()
+
+ if new_sect:
+
+ # a new section starts *here*
+
+ self.debug("found new section --> %(sect)s", sect=new_sect)
+
+ if self.ctx.contents.strip():
+ if not self.in_doc_sect:
+ self.warn("contents before sections '%(c)s'" , c=self.ctx.contents.strip())
+ self.dump_section(self.ctx.section, self.ctx.contents)
+ self.ctx.section = self.section_default
+ self.ctx.contents = ""
+
+ self.debug("new_sect: '%(sec)s' / desc: '%(desc)s'", sec = new_sect, desc = new_cont)
+
+ self.in_doc_sect = True
+ self.in_purpose = False
+ self.debug("FLAGs: in_doc_sect=%(s)s / in_purpose=%(p)s", s=self.in_doc_sect, p=self.in_purpose)
+
+ self.ctx.section = new_sect
+ if new_cont:
+ self.ctx.contents = new_cont + "\n"
+ self.info("section: %(sec)s" , sec=self.ctx.section)
+
+ elif doc_end.match(line):
+
+ # end of the comment-block
+
+ if self.ctx.contents:
+ self.dump_section(self.ctx.section, self.ctx.contents)
+ self.ctx.section = self.section_default
+ self.ctx.contents = ""
+
+ # look for doc_com + <text> + doc_end:
+ if RE(doc_com.pattern + r"[a-zA-Z_0-9:\.]+" + doc_end.pattern).match(line):
+ self.warn("suspicious ending line")
+
+ self.ctx.prototype = ""
+ self.debug("END doc block / switch state 2 --> 3")
+ self.debug("end of doc comment, looking for prototype")
+ self.state = 3
+ self.brcount = 0
+
+ elif doc_content.match(line):
+
+ # a comment line with *content* of a section or a *purpose*
+
+ cont_line = doc_content[0]
+
+ if not cont_line.strip():
+
+ if self.in_purpose:
+
+ # empty line after short description (*purpose*) introduce the
+ # "Description" section
+
+ self.debug("found empty line in *purpose* --> start 'Description' section")
+ if self.ctx.contents.strip():
+ if not self.in_doc_sect:
+ self.warn("contents before sections '%(c)s'" , c=self.ctx.contents.strip())
+ self.dump_section(self.ctx.section, self.ctx.contents)
+ self.ctx.section = self.section_descr
+ self.ctx.contents = ""
+ self.in_doc_sect = True
+ self.in_purpose = False
+ self.debug("FLAGs: in_doc_sect=%(s)s / in_purpose=%(p)s", s=self.in_doc_sect, p=self.in_purpose)
+
+ elif (self.ctx.section.startswith("@")
+ or self.ctx.section == self.section_context):
+
+ # miguel-style comment kludge, look for blank lines after @parameter
+ # line to signify start of description
+
+ self.debug("blank lines after @parameter --> start 'Description' section")
+ self.dump_section(self.ctx.section, self.ctx.contents)
+ self.ctx.section = self.section_descr
+ self.ctx.contents = ""
+ self.in_doc_sect = True
+ self.debug("FLAGs: in_doc_sect=%(s)s / in_purpose=%(p)s", s=self.in_doc_sect, p=self.in_purpose)
+
+ else:
+ self.ctx.contents += "\n"
+
+ elif self.in_purpose:
+ # Continued declaration purpose, dismiss leading whitespace
+ if self.ctx.decl_purpose:
+ self.ctx.decl_purpose += " " + cont_line.strip()
+ else:
+ self.ctx.decl_purpose = cont_line.strip()
+ else:
+ self.ctx.contents += cont_line + "\n"
+
+ else:
+ # i dont know - bad line? ignore.
+ self.warn("bad line: '%(line)s'", line = line.strip())
+
+ def state_3(self, line):
+ u"""state: 3 - scanning prototype."""
+
+ if doc_split_start.match(line):
+ self.debug("FLAG: split_doc_state=1 / switch state 3 --> 5")
+ self.state = 5
+ self.split_doc_state = 1
+ if self.ctx.decl_type == 'function':
+ self.error("odd construct, gathering documentation of a function"
+ " outside of the main block?!?")
+
+ elif self.ctx.decl_type == 'function':
+ self.process_state3_function(line)
+ else:
+ self.process_state3_type(line)
+
+ def state_4(self, line):
+ u"""state: 4 - documentation block"""
+
+ if doc_block.match(line):
+ # a new DOC block arrived, dump the last section and pass the new
+ # DOC block to state 1.
+ self.dump_doc_section(self.ctx.section, self.ctx.contents)
+ self.ctx = self.ctx.new()
+ self.debug("END & START: DOC block / switch state 4 --> 1")
+ self.state = 1
+ self.state_1(line)
+
+ elif doc_end.match(line):
+ # the DOC block ends here, dump it and reset to state 0
+ self.debug("END: DOC block / dump doc section / switch state 4 --> 0")
+ self.dump_doc_section(self.ctx.section, self.ctx.contents)
+ self.ctx = self.ctx.new()
+ self.state = 0
+
+ elif doc_content.match(line):
+ cont = doc_content[0]
+ if (not cont.strip() # dismiss leading newlines
+ and not self.ctx.contents):
+ pass
+ else:
+ self.ctx.contents += doc_content[0] + "\n"
+
+ def state_5(self, line):
+ u"""state: 5 - gathering documentation outside main block"""
+
+ if (self.split_doc_state == 1
+ and doc_split_sect.match(line)):
+
+ # First line (split_doc_state 1) needs to be a @parameter
+ self.ctx.section = self.sect_title(doc_split_sect[0].strip())
+ self.ctx.contents = doc_split_sect[1].strip()
+ self.split_doc_state = 2
+ self.debug("SPLIT-DOC-START: '%(param)s' / split-state 1 --> 2"
+ , param = self.ctx.section)
+ self.info("section: %(sec)s" , sec=self.ctx.section)
+
+ elif doc_split_end.match(line):
+ # Documentation block end
+ self.debug("SPLIT-DOC-END: ...")
+
+ if not self.ctx.contents.strip():
+ self.debug("SPLIT-DOC-END: ... no description to dump")
+
+ else:
+ self.dump_section(self.ctx.section, self.ctx.contents)
+ self.ctx.section = self.section_default
+ self.ctx.contents = ""
+
+ self.debug("SPLIT-DOC-END: ... split-state --> 0 / state = 3")
+ self.state = 3
+ self.split_doc_state = 0
+
+ elif doc_content.match(line):
+ # Regular text
+ if self.split_doc_state == 2:
+ self.ctx.contents += doc_content[0] + "\n"
+
+ elif self.split_doc_state == 1:
+ self.split_doc_state = 4
+ self.error("Comment without header was found split-state --> 4")
+ self.warn("Incorrect use of kernel-doc format: %(line)s"
+ , line = line)
+
+ # ------------------------------------------------------------
+ # helper to parse special objects
+ # ------------------------------------------------------------
+
+ def process_state3_function(self, line):
+
+ self.debug("PROCESS-FUNCTION: %(line)s", line=line)
+ line = C99_comments.sub("", line) # strip C99-style comments to end of line
+ line = line.strip()
+
+ stripProto = RE(r"([^\{]*)")
+
+ # ?!?!? MACDOC does not (no more?) exists
+ # if ($x =~ m#\s*/\*\s+MACDOC\s*#io || ($x =~ /^#/ && $x !~ /^#\s*define/)) {
+ # do nothing
+ # }
+
+ if line.startswith("#") and not MACRO_define.search(line):
+ # do nothing
+ pass
+ elif stripProto.match(line):
+ self.ctx.prototype += " " + stripProto[0]
+
+ if (MACRO_define.search(line)
+ or "{" in line
+ or ";" in line ):
+
+ # strip cr&nl, strip C89 comments, strip leading whitespaces
+ self.ctx.prototype = C89_comments.sub(
+ "", CR_NL.sub(" ", self.ctx.prototype)).lstrip()
+
+ if SYSCALL_DEFINE.search(self.ctx.prototype):
+ self.ctx.prototype = self.syscall_munge(self.ctx.prototype)
+
+ if (TRACE_EVENT.search(line)
+ or DEFINE_EVENT.search(line)
+ or DEFINE_SINGLE_EVENT.search(line) ):
+ self.ctx.prototype = self.tracepoint_munge(self.ctx.prototype)
+
+ self.ctx.prototype = self.ctx.prototype.strip()
+ self.info("prototype --> '%(proto)s'", proto=self.ctx.prototype)
+ self.dump_function(self.ctx.prototype)
+ self.reset_state()
+
+ def syscall_munge(self, prototype):
+ self.debug("syscall munge: '%(prototype)s'" , prototype=prototype)
+ void = False
+
+ # strip needles whitespaces
+ prototype = normalize_ws(prototype)
+
+ if SYSCALL_DEFINE0.search(prototype):
+ void = True
+ prototype = SYSCALL_DEFINE.sub("long sys_", prototype)
+
+ if re.search(r"long (sys_.*?),", prototype):
+ prototype = prototype.replace(",", "(", 1)
+ elif void:
+ prototype = prototype.replace(")","(void)",1)
+
+ # now delete all of the odd-number commas in $prototype
+ # so that arg types & arg names don't have a comma between them
+
+ retVal = prototype
+ if not void:
+ x = prototype.split(",")
+ y = []
+ while x:
+ y.append(x.pop(0) + x.pop(0))
+ retVal = ",".join(y)
+ self.debug("syscall munge: retVal '%(retVal)s'" , retVal=retVal)
+ return retVal
+
+ def tracepoint_munge(self, prototype):
+ self.debug("tracepoint munge: %(prototype)s" , prototype=prototype)
+
+ retVal = prototype
+ tp_name = ""
+ tp_args = ""
+
+ if TRACE_EVENT_name.match(prototype):
+ tp_name = TRACE_EVENT_name[0]
+
+ elif DEFINE_SINGLE_EVENT_name.match(prototype):
+ tp_name = DEFINE_SINGLE_EVENT_name[0]
+
+ elif DEFINE_EVENT_name.match(prototype):
+ tp_name = DEFINE_EVENT_name[1]
+
+ tp_name = tp_name.lstrip()
+
+ if TP_PROTO.match(prototype):
+ tp_args = TP_PROTO[0]
+
+ if not (tp_name.strip() and tp_args.strip()):
+ self.warn("Unrecognized tracepoint format: %(prototype)s"
+ , prototype=prototype)
+ else:
+ retVal = ("static inline void trace_%s(%s)"
+ % (tp_name, tp_args))
+ return retVal
+
+ def process_state3_type(self, line):
+ self.debug("PROCESS-TYPE: %(line)s", line=line)
+
+ # strip cr&nl, strip C99 comments, strip leading&trailing whitespaces
+ line = C99_comments.sub("", CR_NL.sub(" ", line)).strip()
+
+ if MACRO.match(line):
+ # To distinguish preprocessor directive from regular declaration
+ # later.
+ line += ";"
+
+ m = RE(r"([^{};]*)([{};])(.*)")
+
+ while True:
+ if m.search(line):
+ self.ctx.prototype += m[0] + m[1]
+ if m[1] == "{":
+ self.brcount += 1
+ if m[1] == "}":
+ self.brcount -= 1
+ if m[1] == ";" and self.brcount == 0:
+ self.info("prototype --> '%(proto)s'", proto=self.ctx.prototype)
+ self.debug("decl_type: %(decl_type)s", decl_type=self.ctx.decl_type)
+ if self.ctx.decl_type == "union":
+ self.dump_union(self.ctx.prototype)
+ elif self.ctx.decl_type == "struct":
+ self.dump_struct(self.ctx.prototype)
+ elif self.ctx.decl_type == "enum":
+ self.dump_enum(self.ctx.prototype)
+ elif self.ctx.decl_type == "typedef":
+ self.dump_typedef(self.ctx.prototype)
+ else:
+ raise ParserBuggy(
+ self, "unknown decl_type: %s" % self.ctx.decl_type)
+
+ self.reset_state()
+ break
+ line = m[2]
+ else:
+ self.ctx.prototype += line
+ break
+
+ # ------------------------------------------------------------
+ # selective translation
+ # ------------------------------------------------------------
+
+ def mark_as_translated(self, name):
+ u"""Test if content should be translated"""
+ retVal = False
+ if name in self.options.skip_names:
+ retVal = False
+ elif not self.options.use_names:
+ retVal = True
+ elif name in self.options.use_names:
+ retVal = True
+ if retVal:
+ self.ctx.translated_names.append(name)
+ return retVal
+
+ # ------------------------------------------------------------
+ # dump objects
+ # ------------------------------------------------------------
+
+ def dump_preamble(self):
+ if not self.options.skip_preamble:
+ self.translator.output_preamble()
+
+ def dump_epilog(self):
+ if not self.options.skip_epilog:
+ self.translator.output_epilog()
+
+ def dump_section(self, name, cont):
+ u"""Store section's *content* under it's name.
+
+ :param str name: name of the section
+ :param str cont: content of the section
+
+ Stores the *content* under section's *name* in one of the *container*. A
+ container is a hash object, the section name is the *key* and the
+ content is the *value*.
+
+ Container:
+
+ * self.ctx.constants: holds constant's descriptions
+ * self.ctx.parameterdescs: holds parameter's descriptions
+ * self.ctx.sections: holds common sections like "Return:"
+
+ There are the following contai
+ """
+ self.debug("dump_section(): %(name)s", name = name)
+ name = name.strip()
+ cont = cont.rstrip() # dismiss trailing whitespace
+ if type_constant.match(name): # '%CONST' - name of a constant.
+ name = type_constant[0]
+ self.debug("constant section '%(name)s'", name = name)
+ if self.ctx.constants.get(name, None):
+ self.error("duplicate constant definition '%(name)s'"
+ , name = name)
+ self.ctx.constants[name] = cont
+
+ elif type_param.match(name): # '@parameter' - name of a parameter
+ name = type_param[0]
+ self.debug("parameter definition '%(name)s'", name = name)
+ if self.ctx.parameterdescs.get(name, None):
+ self.error("duplicate parameter definition '%(name)s'", name = name)
+ self.ctx.parameterdescs[name] = cont
+ self.ctx.sectcheck.append(name)
+
+ elif name == "@...":
+ self.debug("parameter definiton '...'")
+ name = "..."
+ if self.ctx.parameterdescs.get(name, None):
+ self.error("parameter definiton '...'")
+ self.ctx.parameterdescs[name] = cont
+ self.ctx.sectcheck.append(name)
+ else:
+ self.debug("other section '%(name)s'", name = name)
+ if self.ctx.sections.get(name, None):
+ self.error("duplicate section name '%(name)s'", name = name)
+ self.ctx.sections[name] = cont
+
+ def dump_function(self, proto):
+ self.debug("dump_function(): (1) '%(proto)s'", proto=proto)
+ hasRetVal = True
+ proto = re.sub( r"^static +" , "", proto )
+ proto = re.sub( r"^extern +" , "", proto )
+ proto = re.sub( r"^asmlinkage +" , "", proto )
+ proto = re.sub( r"^inline +" , "", proto )
+ proto = re.sub( r"^__inline__ +" , "", proto )
+ proto = re.sub( r"^__inline +" , "", proto )
+ proto = re.sub( r"^__always_inline +", "", proto )
+ proto = re.sub( r"^noinline +" , "", proto )
+ proto = re.sub( r"__init +" , "", proto )
+ proto = re.sub( r"__init_or_module +", "", proto )
+ proto = re.sub( r"__meminit +" , "", proto )
+ proto = re.sub( r"__must_check +" , "", proto )
+ proto = re.sub( r"__weak +" , "", proto )
+
+ define = bool(MACRO_define.match(proto))
+ proto = MACRO_define.sub("", proto )
+
+ proto = re.sub( r"__attribute__\s*\(\([a-z,]*\)\)" , "", proto )
+
+ # Yes, this truly is vile. We are looking for:
+ # 1. Return type (may be nothing if we're looking at a macro)
+ # 2. Function name
+ # 3. Function parameters.
+ #
+ # All the while we have to watch out for function pointer parameters
+ # (which IIRC is what the two sections are for), C types (these
+ # regexps don't even start to express all the possibilities), and
+ # so on.
+ #
+ # If you mess with these regexps, it's a good idea to check that
+ # the following functions' documentation still comes out right:
+ # - parport_register_device (function pointer parameters)
+ # - atomic_set (macro)
+ # - pci_match_device, __copy_to_user (long return type)
+
+ self.debug("dump_function(): (2) '%(proto)s'", proto=proto)
+
+ x = RE(r"^()([a-zA-Z0-9_~:]+)\s+")
+
+ if define and x.match(proto):
+ # This is an object-like macro, it has no return type and no
+ # parameter list. Function-like macros are not allowed to have
+ # spaces between decl_name and opening parenthesis (notice
+ # the \s+).
+ self.ctx.return_type = x[0]
+ self.ctx.decl_name = x[1]
+ hasRetVal = False
+ self.debug("dump_function(): (hasRetVal = False) '%(proto)s'"
+ , proto=proto)
+ else:
+ matchExpr = None
+ for regexp in FUNC_PROTOTYPES:
+ if regexp.match(proto):
+ matchExpr = regexp
+ self.debug("dump_function(): (matchExpr = %(pattern)s) '%(proto)s'"
+ , pattern = matchExpr.pattern, proto=proto)
+ break
+
+ if matchExpr is not None:
+ self.debug("dump_function(): return_type='%(x)s'", x=matchExpr[0])
+ self.ctx.return_type = matchExpr[0]
+ self.debug("dump_function(): decl_name='%(x)s'", x=matchExpr[1])
+ self.ctx.decl_name = matchExpr[1]
+ self.create_parameterlist(matchExpr[2], ",")
+ else:
+ self.warn("can't understand function proto: '%(prototype)s'"
+ , **self.ctx)
+ return
+
+ self.check_sections(self.ctx.decl_name
+ , self.ctx.decl_type
+ , self.ctx.sectcheck
+ , self.ctx.parameterlist
+ , "")
+ if hasRetVal:
+ self.check_return_section(self.ctx.decl_name, self.ctx.return_type)
+
+ if self.mark_as_translated(self.ctx.decl_name):
+ self.translator.output_function_decl(
+ function = self.ctx.decl_name
+ , return_type = self.ctx.return_type
+ , parameterlist = self.ctx.parameterlist
+ , parameterdescs = self.ctx.parameterdescs
+ , parametertypes = self.ctx.parametertypes
+ , sections = self.ctx.sections
+ , purpose = self.ctx.decl_purpose )
+ else:
+ self.debug("skip translation of %(t)s: '%(n)s'"
+ , t=self.ctx.decl_type, n=self.ctx.decl_name)
+
+ def dump_doc_section(self, name, cont):
+ if self.mark_as_translated(name):
+ self.dump_section(name, cont)
+ self.translator.output_doc_section(sections=self.ctx.sections)
+ else:
+ self.debug("skip translation of DOC: '%(n)s'", n=name)
+
+ def dump_union(self, proto):
+ if not self.prepare_struct_union(proto):
+ self.error("can't parse union!")
+ return
+
+ if self.mark_as_translated(self.ctx.decl_name):
+ self.translator.output_union_decl(
+ decl_name = self.ctx.decl_name
+ , decl_type = self.ctx.decl_type
+ , parameterlist = self.ctx.parameterlist
+ , parameterdescs = self.ctx.parameterdescs
+ , parametertypes = self.ctx.parametertypes
+ , sections = self.ctx.sections
+ , purpose = self.ctx.decl_purpose )
+ else:
+ self.debug("skip translation of %(t)s: '%(n)s'"
+ , t=self.ctx.decl_type, n=self.ctx.decl_name)
+
+ def dump_struct(self, proto):
+ if not self.prepare_struct_union(proto):
+ self.error("can't parse struct!")
+ return
+
+ if self.mark_as_translated(self.ctx.decl_name):
+ self.translator.output_struct_decl(
+ decl_name = self.ctx.decl_name
+ , decl_type = self.ctx.decl_type
+ , parameterlist = self.ctx.parameterlist
+ , parameterdescs = self.ctx.parameterdescs
+ , parametertypes = self.ctx.parametertypes
+ , sections = self.ctx.sections
+ , purpose = self.ctx.decl_purpose )
+ else:
+ self.debug("skip translation of %(t)s: '%(n)s'"
+ , t=self.ctx.decl_type, n=self.ctx.decl_name)
+
+ def prepare_struct_union(self, proto):
+ self.debug("prepare_struct_union(): '%(proto)s'", proto=proto)
+
+ retVal = False
+ members = ""
+ nested = ""
+
+ if C_STRUCT_UNION.match(proto):
+
+ if C_STRUCT_UNION[0] != self.ctx.decl_type:
+ raise ParserBuggy(
+ self
+ , ("determine of decl_type is inconsistent: '%s' <--> '%s'"
+ "\nprototype: %s"
+ % (C_STRUCT_UNION[0], self.ctx.decl_type, proto)))
+
+ self.ctx.decl_name = C_STRUCT_UNION[1]
+ members = C_STRUCT_UNION[2]
+
+ # ignore embedded structs or unions
+ embeded_re = RE(r"({.*})")
+ if embeded_re.search(proto):
+ nested = embeded_re[0]
+ embeded_re.sub("", members)
+
+ # ignore members marked private:
+ members = re.sub(r"/\*\s*private:.*?/\*\s*public:.*?\*/", "", members, flags=re.I)
+ members = re.sub(r"/\*\s*private:.*", "", members, flags=re.I)
+
+ # strip comments:
+ members = C89_comments.sub("", members)
+ nested = C89_comments.sub("", nested)
+
+ # strip kmemcheck_bitfield_{begin,end}.*;
+ members = re.sub(r"kmemcheck_bitfield_.*?;", "", members)
+
+ # strip attributes
+ members = re.sub(r"__attribute__\s*\(\([a-z,_\*\s\(\)]*\)\)", "", members, flags=re.I)
+ members = re.sub(r"__aligned\s*\([^;]*\)", "", members)
+ members = re.sub(r"\s*CRYPTO_MINALIGN_ATTR", "", members)
+
+ # replace DECLARE_BITMAP
+ members = re.sub(r"DECLARE_BITMAP\s*\(([^,)]+), ([^,)]+)\)"
+ , r"unsigned long \1\[BITS_TO_LONGS(\2)\]"
+ , members )
+
+ self.create_parameterlist(members, ';')
+ self.check_sections(self.ctx.decl_name
+ , self.ctx.decl_type
+ , self.ctx.sectcheck
+ , self.ctx.parameterlist # self.ctx.struct_actual.split(" ")
+ , nested)
+ retVal = True
+
+ else:
+ retVal = False
+
+ return retVal
+
+ def dump_enum(self, proto):
+ self.debug("dump_enum(): '%(proto)s'", proto=proto)
+
+ proto = C89_comments.sub("", proto)
+ # strip #define macros inside enums
+ proto = re.sub(r"#\s*((define|ifdef)\s+|endif)[^;]*;", "", proto)
+
+ splitchar = ","
+ RE_NAME = RE(r"^\s*(\w+).*")
+
+ if C_ENUM.search(proto):
+ self.ctx.decl_name = C_ENUM[0]
+ members = normalize_ws(C_ENUM[1])
+
+ # drop trailing splitchar, if extists
+ if members.endswith(splitchar):
+ members = members[:-1]
+
+ for member in members.split(splitchar):
+ name = RE_NAME.sub(r"\1", member)
+ self.ctx.parameterlist.append(name)
+ if not self.ctx.parameterdescs.get(name, None):
+ self.warn(
+ "Enum value '%(name)s' not described"
+ " in enum '%(decl_name)s'"
+ , name = name, decl_name=self.ctx.decl_name )
+ self.ctx.parameterdescs[name] = Parser.undescribed
+
+ if self.mark_as_translated(self.ctx.decl_name):
+ self.translator.output_enum_decl(
+ enum = self.ctx.decl_name
+ , parameterlist = self.ctx.parameterlist
+ , parameterdescs = self.ctx.parameterdescs
+ , sections = self.ctx.sections
+ , purpose = self.ctx.decl_purpose )
+ else:
+ self.debug("skip translation of %(t)s: '%(n)s'"
+ , t=self.ctx.decl_type, n=self.ctx.decl_name)
+ else:
+ self.error("can't parse enum!")
+
+ def dump_typedef(self, proto):
+ self.debug("dump_typedef(): '%(proto)s'", proto=proto)
+
+ proto = C89_comments.sub("", proto)
+
+ if C_FUNC_TYPEDEF.search(proto):
+ # Parse function prototypes
+
+ self.ctx.return_type = C_FUNC_TYPEDEF[0]
+ self.ctx.decl_name = C_FUNC_TYPEDEF[1]
+ self.check_return_section(self.ctx.decl_name, self.ctx.return_type)
+
+ f_args = C_FUNC_TYPEDEF[2]
+ self.create_parameterlist(f_args, ',')
+
+ if self.mark_as_translated(self.ctx.decl_name):
+ self.translator.output_function_decl(
+ function = self.ctx.decl_name
+ , return_type = self.ctx.return_type
+ , parameterlist = self.ctx.parameterlist
+ , parameterdescs = self.ctx.parameterdescs
+ , parametertypes = self.ctx.parametertypes
+ , sections = self.ctx.sections
+ , purpose = self.ctx.decl_purpose)
+ else:
+ self.debug("skip translation of %(t)s: '%(n)s'"
+ , t=self.ctx.decl_type, n=self.ctx.decl_name)
+
+ else:
+ self.debug("dump_typedef(): '%(proto)s'", proto=proto)
+ x1 = RE(r"\(*.\)\s*;$")
+ x2 = RE(r"\[*.\]\s*;$")
+
+ while x1.search(proto) or x2.search(proto):
+ proto = x1.sub(";", proto)
+ proto = x2.sub(";", proto)
+
+ self.debug("dump_typedef(): '%(proto)s'", proto=proto)
+
+ if C_TYPEDEF.match(proto):
+ self.ctx.decl_name = C_TYPEDEF[0]
+
+ if self.mark_as_translated(self.ctx.decl_name):
+ self.translator.output_typedef_decl(
+ typedef = self.ctx.decl_name
+ , sections = self.ctx.sections
+ , purpose = self.ctx.decl_purpose )
+ else:
+ self.debug("skip translation of %(t)s: '%(n)s'"
+ , t=self.ctx.decl_type, n=self.ctx.decl_name)
+ else:
+ self.error("can't parse typedef!")
+
+ def create_parameterlist(self, parameter, splitchar):
+ self.debug("create_parameterlist(): splitchar='%(x)s' params='%(y)s'"
+ , x=splitchar, y=parameter)
+ parameter = normalize_ws(parameter)
+ pointer_to_func = RE(r"\(.+\)\s*\(")
+
+ # temporarily replace commas inside function pointer definition
+ m = RE(r"(\([^\),]+),")
+
+ while m.search(parameter):
+ parameter = m.sub(r"\1#", parameter)
+ # drop trailing splitchar, if extists
+ if parameter.endswith(splitchar):
+ parameter = parameter[:-1]
+
+ for c, p in enumerate(parameter.split(splitchar)):
+
+ p = C99_comments.sub("", p)
+ p = p.strip()
+
+ self.debug(" parameter#%(c)s: %(p)s", c=c, p=p)
+ p_type = None
+ p_name = None
+
+ if MACRO.match(p):
+
+ # Treat preprocessor directive as a typeless variable just to
+ # fill corresponding data structures "correctly". Catch it later
+ # in output_* subs.
+ self.debug(" parameter#%(c)s: (MACRO) %(p)s=''" , c=c, p=p)
+ self.push_parameter(p, "")
+
+ elif pointer_to_func.search(p):
+
+ # pointer-to-function
+ p = p.replace("#", ",") # reinsert temporarily removed commas
+ self.debug(" parameter#%(c)s: (pointer to function) %(p)s", c=c, p=p)
+ m = RE(r"[^\(]+\(\*?\s*(\w*)\s*\)")
+ m.match(p)
+ p_name = m[0]
+ p_type = p
+ p_type = re.sub(r"([^\(]+\(\*?)\s*"+p_name, r"\1", p_type)
+ #self.save_struct_actual(p_name)
+ self.push_parameter(p_name, p_type)
+
+ else:
+ p = re.sub(r"\s*:\s*", ":", p)
+ p = re.sub(r"\s*\[" , "[", p)
+ self.debug(" parameter#%(c)s: (common) %(p)s", c=c, p=p)
+
+ p_args = re.split(r"\s*,\s*", p)
+ if re.match(r"\s*,\s*", p_args[0]):
+ p_args[0] = re.sub(r"(\*+)\s*", r" \1", p_args[0])
+
+ self.debug(" parameter#%(c)s : (1) p_args = %(p_args)s"
+ , c=c, p_args=repr(p_args))
+
+ first_arg = []
+ m = RE(r"^(.*\s+)(.*?\[.*\].*)$")
+ if m.match(p_args[0]):
+ p_args.pop(0)
+ first_arg.extend(re.split(r"\s+", m[0]))
+ first_arg.append(m[1])
+ else:
+ first_arg.extend(re.split(r"\s+", p_args.pop(0)))
+
+ p_args = [first_arg.pop() ] + p_args
+ self.debug(" parameter#%(c)s : (2) p_args=%(p_args)s"
+ , c=c, p_args=repr(p_args))
+ p_type = " ".join(first_arg)
+
+ ma = RE(r"^(\*+)\s*(.*)")
+ mb = RE(r"(.*?):(\d+)")
+
+ for p_name in p_args:
+ self.debug(" parameter#%(c)s : (3) p_name='%(p_name)s'"
+ , c=c, p_name=p_name)
+
+ if ma.match(p_name):
+ p_type = "%s %s" % (p_type, ma[0])
+ p_name = ma[1]
+
+ elif mb.match(p_name):
+ if p_type: # skip unnamed bit-fields
+ p_name = mb[0]
+ p_type = "%s:%s" % (p_type, mb[1])
+
+ self.debug(" parameter#%(c)s : (4) p_name='%(p_name)s' / p_type='%(p_type)s'"
+ , c=c, p_name=p_name, p_type=p_type)
+ #self.save_struct_actual(p_name)
+ self.push_parameter(p_name, p_type)
+
+ def push_parameter(self, p_name, p_type):
+ self.debug(
+ "push_parameter(): p_name='%(p_name)s' / p_type='%(p_type)s'"
+ , p_name=p_name, p_type=p_type)
+
+ p_name = p_name.strip()
+ p_type = p_type.strip()
+
+ if (self.anon_struct_union
+ and not p_type
+ and p_name == "}"):
+ # ignore the ending }; from anon. struct/union
+ return
+
+ self.anon_struct_union = False
+
+ self.debug(
+ "push_parameter(): (1) p_name='%(p_name)s' / p_type='%(p_type)s'"
+ , p_name=p_name, p_type=p_type)
+
+ if not p_type and re.search(r"\.\.\.$", p_name):
+ if not self.ctx.parameterdescs.get(p_name, None):
+ self.ctx.parameterdescs[p_name] = "variable arguments"
+
+ elif not p_type and (not p_name or p_name == "void"):
+ p_name = "void"
+ self.ctx.parameterdescs[p_name] = "no arguments"
+
+ elif not p_type and (p_name == "struct" or p_name == "union"):
+ # handle unnamed (anonymous) union or struct:
+ p_type = p_name
+ p_name = "{unnamed_" + p_name + "}"
+ self.ctx.parameterdescs[p_name] = "anonymous\n"
+ self.anon_struct_union = True
+
+ self.debug(
+ "push_parameter(): (2) p_name='%(p_name)s' / p_type='%(p_type)s'"
+ , p_name=p_name, p_type=p_type)
+
+ # strip array from paramater name / e.g. p_name is "modes[]" from a
+ # parmeter defined by: "const char * const modes[]"
+
+ p_name_doc = re.sub(r"\[.*", "", p_name)
+
+ # warn if parameter has no description (but ignore ones starting with
+ # '#' as these are not parameters but inline preprocessor statements);
+ # also ignore unnamed structs/unions;
+
+ if not self.anon_struct_union:
+
+ if (not self.ctx.parameterdescs.get(p_name_doc, None)
+ and not p_name.startswith("#")):
+
+ if p_type == "function" or p_type == "enum":
+ self.warn("Function parameter or member '%(p_name)s' not "
+ "described in '%(decl_name)s'."
+ , p_name = p_name
+ , decl_name = self.ctx.decl_name)
+ else:
+ self.warn("no description found for parameter '%(p_name)s'"
+ , p_name = p_name)
+ self.ctx.parameterdescs[p_name] = Parser.undescribed
+
+ self.debug(
+ "push_parameter(): (3) p_name='%(p_name)s' / p_type='%(p_type)s'"
+ , p_name=p_name, p_type=p_type)
+
+ self.ctx.parameterlist.append(p_name)
+ self.ctx.parametertypes[p_name] = p_type.strip()
+
+ # def save_struct_actual(self, actual):
+ # # strip all spaces from the actual param so that it looks like one
+ # # string item
+ # self.debug("save_struct_actual(): actual='%(a)s'", a=actual)
+ # actual = WHITESPACE.sub("", actual)
+ # self.ctx.struct_actual += actual + " "
+ # self.debug("save_struct_actual: '%(a)s'", a=self.ctx.struct_actual)
+
+
+ def check_sections(self, decl_name, decl_type
+ , sectcheck, parameterlist, nested):
+ self.debug("check_sections(): decl_name='%(n)s' / decl_type='%(t)s' /"
+ " sectcheck=%(sc)s / parameterlist=%(pl)s / nested='%(nested)s'"
+ , n=decl_name, t=decl_type, sc=sectcheck, pl=parameterlist, nested=nested)
+
+ for sect in sectcheck:
+ err = True
+ for para in parameterlist:
+ para = re.sub(r"\[.*\]", "", para)
+ #para = re.sub(r"/__attribute__\s*\(\([A-Za-z,_\*\s\(\)]*\)\)/", "", para)
+ if para == sect:
+ err = False
+ break
+ if err:
+ if decl_type == "function":
+ self.warn("excess function parameter '%(sect)s' description in '%(decl_name)s'"
+ , sect = sect, decl_name = decl_name)
+ elif not re.search(r"\Q" + sect + r"\E", nested):
+ self.warn("excess %(decl_type)s member '%(sect)s' description in '%(decl_name)s'"
+ , decl_type = decl_type, decl_name = decl_name, sect = sect)
+ else:
+ self.debug("check_sections(): parameter '%(sect)s': description exists / OK"
+ , sect=sect)
+
+ def check_return_section(self, decl_name, return_type):
+ self.debug("check_return_section(): decl_name='%(n)s', return_type='%(t)s"
+ , n=decl_name, t=return_type)
+ # Ignore an empty return type (It's a macro) and ignore functions with a
+ # "void" return type. (But don't ignore "void *")
+
+ if (not return_type
+ or re.match(r"void\s*\w*\s*$", return_type)):
+ self.debug("check_return_section(): ignore void")
+ return
+
+ if not self.ctx.sections.get(self.section_return, None):
+ self.warn("no description found for return-value of function '%(func)s()'"
+ , func = decl_name)
+ else:
+ self.debug("check_return_section(): return-value of %(func)s() OK"
+ , func = decl_name)
+
+# ==============================================================================
+# 2cent debugging & introspection
+# ==============================================================================
+
+def CONSOLE(arround=5, frame=None):
+ import inspect, code, linecache
+ # pylint: disable=C0321
+ sys.stderr.flush()
+ sys.stdout.flush()
+
+ frame = frame or inspect.currentframe().f_back
+ fName = frame.f_code.co_filename
+ lineNo = frame.f_lineno
+
+ ns = dict(**frame.f_globals)
+ ns.update(**frame.f_locals)
+
+ histfile = os.path.join(os.path.expanduser("~"), ".kernel-doc-history")
+ try:
+ import readline, rlcompleter # pylint: disable=W0612
+ readline.set_completer(rlcompleter.Completer(namespace=ns).complete)
+ readline.parse_and_bind("tab: complete")
+ readline.set_history_length(1000)
+ if os.path.exists(histfile):
+ readline.read_history_file(histfile)
+ except ImportError:
+ readline = None
+ lines = []
+ for c in range(lineNo - arround, lineNo + arround):
+ if c > 0:
+ prefix = "%-04s|" % c
+ if c == lineNo: prefix = "---->"
+ line = linecache.getline(fName, c, frame.f_globals)
+ if line != '': lines.append(prefix + line)
+ else:
+ if lines: lines[-1] = lines[-1] + "<EOF>\n"
+ break
+ banner = "".join(lines) + "file: %s:%s\n" % (fName, lineNo)
+ try:
+ code.interact(banner=banner, local=ns)
+ finally:
+ if readline is not None:
+ readline.write_history_file(histfile)
+
+# ==============================================================================
+# run ...
+# ==============================================================================
+
+if __name__ == "__main__":
+ sys.exit(main())
+else:
+ # FIXME: just for testing
+ __builtins__["CONSOLE"] = CONSOLE
+
--
v4.7-rc-2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 6/7] kernel-doc directive: initial implementation
2016-06-06 16:32 [PATCH 0/7] add reST/sphinx-doc to linux documentation Markus Heiser
` (4 preceding siblings ...)
2016-06-06 16:32 ` [PATCH 5/7] kernel-doc parser: inital python implementation Markus Heiser
@ 2016-06-06 16:32 ` Markus Heiser
2016-06-06 16:32 ` [PATCH 7/7] flat-table " Markus Heiser
` (2 subsequent siblings)
8 siblings, 0 replies; 23+ messages in thread
From: Markus Heiser @ 2016-06-06 16:32 UTC (permalink / raw)
To: corbet
Cc: jani.nikula, daniel.vetter, grant.likely, mchehab, keithp,
linux-kernel, linux-doc, hverkuil, Heiser, Markus
From: "Heiser, Markus" <markus.heiser@darmarIT.de>
Implements the reST directive specified at
books/kernel-doc-HOWTO/kernel-doc-directive.rst [1]
Below is a short overview of the options (further reading [1]):
.. kernel-doc:: <filename>
:doc: <section title>
:export:
:internal:
:functions: <function [, functions [, ...]]>
:module: <prefix-id>
:snippets: <snippet [, snippets [, ...]]>
:language: <snippet-lang>
:linenos:
:debug:
The initial implementation was taken from the sphkerneldoc project [2]
[1] http://return42.github.io/sphkerneldoc/books/kernel-doc-HOWTO/kernel-doc-directive.html
[2] http://return42.github.io/sphkerneldoc
Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
---
scripts/site-python/linuxdoc/rstKernelDoc.py | 369 +++++++++++++++++++++++++++
1 file changed, 369 insertions(+)
create mode 100755 scripts/site-python/linuxdoc/rstKernelDoc.py
diff --git a/scripts/site-python/linuxdoc/rstKernelDoc.py b/scripts/site-python/linuxdoc/rstKernelDoc.py
new file mode 100755
index 0000000..8246a57
--- /dev/null
+++ b/scripts/site-python/linuxdoc/rstKernelDoc.py
@@ -0,0 +1,369 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8; mode: python -*-
+# pylint: disable=C0330, R0903
+
+u"""
+ rstKernelDoc
+ ~~~~~~~~~~~~
+
+ Implementation of the ``kernel-doc`` reST-directive.
+
+ :copyright: Copyright (C) 2016 Markus Heiser
+ :license: GPL V3.0, see LICENSE for details.
+
+ The ``kernel-doc`` (:py:class:`KernelDoc`) directive includes contens from
+ linux kernel source code comments.
+
+ Options:
+
+ * ``:doc: <section title>`` inserts the contents of the ``DOC:`` section
+ titled ``<section title>`` from ``<filename>``. Spaces are allowed in
+ ``<section title>``; do not quote the ``<section title>``.
+
+ * ``:export:`` inserts the documentation in ``<filename>`` of functions /
+ structs or whatever are exported using EXPORT_SYMBOL (``EXPORT_SYMBOL()``,
+ ``EXPORT_SYMBOL_GPL()`` & ``EXPORT_SYMBOL_GPL_FUTURE()``).
+
+ * ``:internal:`` is replaced by the documentation of functions, structs,
+ titles etc. that are documented, but not **not** exported using
+ EXPORT_SYMBOL.
+
+ * ``:functions: <function [, functions [, ...]]>`` is replaced by the
+ documentation of function, struct or whatever object/title is documented
+ <filename>.
+
+ * The option ``:module: <prefix-id>`` sets a module-name, this name is used
+ as prefix for automatic generated IDs (reference anchors).
+
+ The following example shows how to insert documention from the source file
+ ``/drivers/gpu/drm/drm_drv.c``. In this example the documention from the
+ ``DOC:`` section with the title "driver instance overview" and the
+ documentation of all exported symbols (EXPORT_SYMBOL) is included in the
+ reST tree.
+
+ .. code-block:: rst
+
+ .. kernel-doc:: drivers/gpu/drm/drm_drv.c
+ :export:
+ :doc: driver instance overview
+
+ An other example is to use only one function description.
+
+ .. kernel-doc:: include/media/i2c/tvp7002.h
+ :functions: tvp7002_config
+ :module: tvp7002
+
+ This will produce the follwing reST markup to include:
+
+ .. code-block:: rst
+
+ .. _`tvp514x.tvp514x_platform_data`:
+
+ struct tvp514x_platform_data
+ ============================
+
+ .. c:type:: tvp514x_platform_data
+
+
+ .. _`tvp514x.tvp514x_platform_data.definition`:
+
+ Definition
+ ----------
+
+ .. code-block:: c
+
+ struct tvp514x_platform_data {
+ bool clk_polarity;
+ bool hs_polarity;
+ bool vs_polarity;
+ }
+
+ .. _`tvp514x.tvp514x_platform_data.members`:
+
+ Members
+ -------
+
+ clk_polarity
+ Clock polarity of the current interface.
+
+ hs_polarity
+ HSYNC Polarity configuration for current interface.
+
+ vs_polarity
+ VSYNC Polarity configuration for current interface.
+
+ The last example illustrates, that the option ``:module: tvp514x`` is used
+ as a prefix for anchors. E.g. ```ref:`tvp514x.tvp514x_platform_data.members¸```
+ refers to the to the member description of ``struct tvp514x_platform_data``.
+
+
+"""
+
+# ==============================================================================
+# common globals
+# ==============================================================================
+
+# The version numbering follows numbering of the specification
+# (Documentation/books/kernel-doc-HOWTO).
+__version__ = '1.0'
+
+# ==============================================================================
+# imports
+# ==============================================================================
+
+from os import path
+from io import StringIO
+from docutils import nodes
+from docutils.parsers.rst import Directive, directives
+from docutils.utils import SystemMessage
+from docutils.statemachine import ViewList
+from sphinx.util.nodes import nested_parse_with_titles
+
+import kernel_doc as kerneldoc
+
+# ==============================================================================
+def setup(app):
+# ==============================================================================
+
+ app.add_config_value('kernel_doc_raise_error', False, 'env')
+ app.add_directive("kernel-doc", KernelDoc)
+
+# ==============================================================================
+class KernelDocParser(kerneldoc.Parser):
+# ==============================================================================
+
+ def __init__(self, app, *args, **kwargs):
+ super(KernelDocParser, self).__init__(*args, **kwargs)
+ self.app = app
+
+ # -------------------------------------------------
+ # bind the parser logging to the sphinx application
+ # -------------------------------------------------
+
+ def error(self, message, **replace):
+ self.errors += 1
+ self.app.warn(
+ message % replace
+ , location = "%s:%s [kernel-doc ERROR]" % (self.options.fname, self.ctx.line_no)
+ , prefix = "" )
+
+ def warn(self, message, **replace):
+ self.app.warn(
+ message % replace
+ , location = "%s:%s [kernel-doc WARN]" % (self.options.fname, self.ctx.line_no)
+ , prefix = "")
+
+ def info(self, message, **replace):
+ self.app.verbose(
+ "%s:%s: [kernel-doc INFO]: " %(self.options.fname, self.ctx.line_no)
+ + message % replace)
+
+ def debug(self, message, **replace):
+ if self.app.verbosity < 2:
+ return
+ replace.update(dict(fname=self.options.fname, line_no=self.ctx.line_no, logclass = "DEBUG"))
+ message = "%(fname)s:%(line_no)s [kernel-doc %(logclass)s] : " + message
+ self.app.debug(message, **replace)
+
+
+# ==============================================================================
+class KernelDoc(Directive):
+# ==============================================================================
+
+ u"""KernelDoc (``kernel-doc``) directive"""
+
+ required_arguments = 1
+ optional_arguments = 0
+ final_argument_whitespace = True
+
+ option_spec = {
+ "doc" : directives.unchanged_required # aka lines containing !P
+ , "export" : directives.flag # aka lines containing !E
+ , "internal" : directives.flag # aka lines containing !I
+ , "functions" : directives.unchanged_required # aka lines containing !F
+
+ , "debug" : directives.flag # insert generated reST as code-block
+
+ , "snippets" : directives.unchanged_required
+ , "language" : directives.unchanged_required
+ , "linenos" : directives.flag
+
+ # not yet supported:
+ #
+ # !C<filename> is replaced by nothing, but makes the tools check that
+ # all DOC: sections and documented functions, symbols, etc. are used.
+ # This makes sense to use when you use !F/!P only and want to verify
+ # that all documentation is included.
+ #
+ #, "check" : directives.flag # aka lines containing !C
+
+ # module name / used as id-prefix
+ , "module" : directives.unchanged_required
+
+ # The encoding of the source file with the kernel-doc comments. The
+ # default is the config.source_encoding from sphinx configuration and
+ # this default is utf-8-sig
+ , "encoding" : directives.encoding
+
+ }
+
+
+ def getOopsEntry(self, msg):
+ retVal = ("\n\n.. todo::"
+ "\n\n Oops: Document generation inconsistency."
+ "\n\n The template for this document tried to insert"
+ " structured comment at this point, but an error occoured."
+ " This dummy section is inserted to allow generation to continue.::"
+ "\n\n")
+
+ for l in msg.split("\n"):
+ retVal += " " + l + "\n"
+ retVal += "\n\n"
+ return retVal
+
+ def errMsg(self, msg, lev=4):
+ err = self.state_machine.reporter.severe(
+ msg
+ , nodes.literal_block(self.block_text, self.block_text)
+ , line=self.lineno )
+ return SystemMessage(err, lev)
+
+ def run(self):
+ doc = self.state.document
+ env = doc.settings.env
+
+ retVal = []
+ try:
+ retVal = self._run(doc, env)
+ except SystemMessage, exc:
+ if env.config.kernel_doc_raise_error:
+ raise
+ self.state_machine.insert_input(
+ self.getOopsEntry(unicode(exc)).split("\n")
+ , self.arguments[0])
+
+ finally:
+ pass
+ return retVal
+
+ def _run(self, document, env):
+
+ # do some checks
+
+ if not document.settings.file_insertion_enabled:
+ raise self.errMsg('File insertion disabled')
+
+ fname = self.arguments[0]
+ src_tree = kerneldoc.SRCTREE
+
+ if self.arguments[0].startswith("./"):
+ # the prefix "./" indicates a relative pathname
+ fname = self.arguments[0][2:]
+ src_tree = path.dirname(path.normpath(document.current_source))
+
+ if "internal" in self.options:
+ if "export" in self.options:
+ raise self.errMsg(
+ "Options 'export' and 'internal' are orthogonal,"
+ " can't use them togehter")
+
+ if "snippets" in self.options:
+ rest = set(self.options.keys()) - set(["snippets", "linenos", "language", "debug"])
+ if rest:
+ raise self.errMsg(
+ "kernel-doc 'snippets' has non of these options: %s"
+ % ",".join(rest))
+
+ # set parse adjustments
+
+ env.note_dependency(fname)
+ rstout = StringIO()
+ ctx = kerneldoc.ParserContext()
+ opts = kerneldoc.ParseOptions(
+ fname = fname
+ , src_tree = src_tree
+ , id_prefix = self.options.get("module", "").strip()
+ , out = rstout
+ , encoding = self.options.get("encoding", env.config.source_encoding)
+ , translator = kerneldoc.ReSTTranslator()
+ ,)
+
+ opts.set_defaults()
+ if not path.exists(opts.fname):
+ raise self.errMsg(
+ "kernel-doc refers to nonexisting document %s" % fname)
+
+ if self.options:
+ opts.skip_preamble = True
+ opts.skip_epilog = True
+
+ if "snippets" in self.options:
+ opts.translator = kerneldoc.ReSTTranslator()
+
+ if "doc" in self.options:
+ opts.use_names.append(self.options.get("doc"))
+
+ if "export" in self.options:
+ # gather exported symbols and add them to the list of names
+ kerneldoc.Parser.gather_context(kerneldoc.readFile(opts.fname), ctx)
+ opts.use_names.extend(ctx.exported_symbols)
+ opts.error_missing = False
+
+ if "internal" in self.options:
+ # gather exported symbols and add them to the ignore-list of names
+ kerneldoc.Parser.gather_context(kerneldoc.readFile(opts.fname), ctx)
+ opts.skip_names.extend(ctx.exported_symbols)
+
+ if "functions" in self.options:
+ opts.error_missing = True
+ opts.use_names.extend(
+ self.options["functions"].replace(","," ").split())
+
+ parser = KernelDocParser(env.app, opts)
+ env.app.info("parse kernel-doc comments from: %s" % fname)
+ parser.parse()
+
+ lines = rstout.getvalue().split("\n")
+
+ if "functions" in self.options:
+ selected = self.options["functions"].replace(","," ").split()
+ names = parser.ctx.translated_names
+ not_found = [ s for s in selected if s not in names]
+ if not_found:
+ raise self.errMsg(
+ "selected section(s) not found: %s" % ", ".join(not_found))
+
+ if "snippets" in self.options:
+ selected = self.options["snippets"].replace(","," ").split()
+ names = parser.ctx.snippets.keys()
+ not_found = [ s for s in selected if s not in names]
+ if not_found:
+ raise self.errMsg(
+ "selected snippets(s) not found: %s" % ", ".join(not_found))
+
+ lines = ["", ".. code-block:: %s"
+ % self.options.get("language", "c"), ]
+ if "linenos" in self.options:
+ lines.append(" :linenos:")
+ lines.append("")
+
+ while selected:
+ snippet = parser.ctx.snippets[selected.pop(0)].split("\n")
+ lines.extend([" " + l for l in snippet])
+ if selected:
+ # delemit snippets with two newlines
+ lines.extend(["",""])
+
+ if "debug" in self.options:
+ code_block = "\n.. code-block:: rst\n :linenos:\n\n".split("\n")
+ for l in lines:
+ code_block.append(" " + l)
+ lines = code_block
+
+ content = ViewList(lines)
+ node = nodes.section()
+ node.document = self.state.document
+ nested_parse_with_titles(self.state, content, node)
+
+ return node.children
+
--
v4.7-rc-2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 7/7] flat-table directive: initial implementation
2016-06-06 16:32 [PATCH 0/7] add reST/sphinx-doc to linux documentation Markus Heiser
` (5 preceding siblings ...)
2016-06-06 16:32 ` [PATCH 6/7] kernel-doc directive: initial implementation Markus Heiser
@ 2016-06-06 16:32 ` Markus Heiser
2016-06-07 7:54 ` [PATCH 0/7] add reST/sphinx-doc to linux documentation Daniel Vetter
2016-06-07 13:47 ` Markus Heiser
8 siblings, 0 replies; 23+ messages in thread
From: Markus Heiser @ 2016-06-06 16:32 UTC (permalink / raw)
To: corbet
Cc: jani.nikula, daniel.vetter, grant.likely, mchehab, keithp,
linux-kernel, linux-doc, hverkuil, Heiser, Markus
From: "Heiser, Markus" <markus.heiser@darmarIT.de>
Implements the reST flat-table directive specified at
books/kernel-doc-HOWTO/table-markup.rst [1]
The 'flat-table' directive belongs to the group of *list tables* (see
below), it is a double-stage list similar to the 'list-table' [2] with
some additional features:
* column-span: with the role 'cspan' a cell can be extended through
additional columns
* row-span: with the role 'rspan' a cell can be extended through
additional rows
* auto-span rightmost cell of a table row over the missing cells on the right
side of that table-row. With Option ``:fill-cells:`` this behavior can
changed from *auto span* to *auto fill*, which automaticly inserts (empty)
cells instead of spanning the last cell.
list tables
The *list tables* formats are double stage lists, compared to the
ASCII-art they migth not be as comfortable for readers of the
text-files. Their advantage is, that they are easy to create/modify
and that the diff of a modification is much more meaningfull, because
it is limited to the modified content.
The initial implementation was taken from the sphkerneldoc project [3]
[1] http://return42.github.io/sphkerneldoc/books/kernel-doc-HOWTO/table-markup.html#flat-table
[2] http://docutils.sourceforge.net/docs/ref/rst/directives.html#list-table
[3] http://return42.github.io/sphkerneldoc
Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
---
scripts/site-python/linuxdoc/rstFlatTable.py | 356 +++++++++++++++++++++++++++
1 file changed, 356 insertions(+)
create mode 100755 scripts/site-python/linuxdoc/rstFlatTable.py
diff --git a/scripts/site-python/linuxdoc/rstFlatTable.py b/scripts/site-python/linuxdoc/rstFlatTable.py
new file mode 100755
index 0000000..096f54a
--- /dev/null
+++ b/scripts/site-python/linuxdoc/rstFlatTable.py
@@ -0,0 +1,356 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8; mode: python -*-
+
+u"""
+ flat-table
+ ~~~~~~~~~~
+
+ Implementation of the ``flat-table`` reST-directive.
+
+ :copyright: Copyright (C) 2016 Markus Heiser
+ :license: GPL V3.0, see LICENSE for details.
+
+ The ``flat-table`` (:py:class:`FlatTable`) is a double-stage list similar
+ to the ``list-table`` with some additional features:
+
+ * *column-span*: with the role ``cspan`` a cell can be extended through
+ additional columns
+
+ * *row-span*: with the role ``rspan`` a cell can be extended through
+ additional rows
+
+ * Autospan last cell over missing cell on the right side of the table-row.
+ With Option ``:fill-cells:`` this behavior can changed to insert empty
+ cells instead of spanning.
+
+ * *auto span* rightmost cell of a table row over the missing cells on the
+ right side of that table-row. With Option ``:fill-cells:`` this behavior
+ can changed from *auto span* to *auto fill*, which automaticly inserts
+ (empty) cells instead of spanning the last cell.
+
+ Options:
+
+ * \:header-rows: [int] count of header rows
+ * \:stub-columns: [int] count of stub columns
+ * \:widths: [[int] [int] ... ] widths of columns
+ * \:fill-cells: instead of autospann missing cells, insert missing cells
+
+ roles:
+
+ * \:cspan: [int] additionale columns (*morecols*)
+ * \:rspan: [int] additionale rows (*morerows*)
+"""
+
+# ==============================================================================
+# common globals
+# ==============================================================================
+
+# The version numbering follows numbering of the specification
+# (Documentation/books/kernel-doc-HOWTO).
+__version__ = '1.0'
+
+# ==============================================================================
+# imports
+# ==============================================================================
+
+from docutils import io, nodes, statemachine, utils
+from docutils.parsers.rst import directives, roles
+from docutils.parsers.rst.directives.tables import Table
+from docutils.utils import SystemMessagePropagation
+
+# ==============================================================================
+def setup(app):
+# ==============================================================================
+
+ app.add_directive("flat-table", FlatTable)
+ roles.register_local_role('cspan', cspan)
+ roles.register_local_role('rspan', rspan)
+
+# ==============================================================================
+def cspan(name, rawtext, text, lineno, inliner, options=None, content=None):
+# ==============================================================================
+
+ options = options if options is not None else {}
+ content = content if content is not None else []
+ nodelist = [colSpan(span=int(text))]
+ msglist = []
+ return nodelist, msglist
+
+# ==============================================================================
+def rspan(name, rawtext, text, lineno, inliner, options=None, content=None):
+# ==============================================================================
+
+ options = options if options is not None else {}
+ content = content if content is not None else []
+ nodelist = [rowSpan(span=int(text))]
+ msglist = []
+ return nodelist, msglist
+
+
+# ==============================================================================
+class rowSpan(nodes.General, nodes.Element): pass
+class colSpan(nodes.General, nodes.Element): pass
+# ==============================================================================
+
+# ==============================================================================
+class FlatTable(Table):
+# ==============================================================================
+
+ u"""FlatTable (``flat-table``) directive"""
+
+ option_spec = {
+ 'name': directives.unchanged
+ , 'class': directives.class_option
+ , 'header-rows': directives.nonnegative_int
+ , 'stub-columns': directives.nonnegative_int
+ , 'widths': directives.positive_int_list
+ , 'fill-cells' : directives.flag }
+
+ def run(self):
+
+ if not self.content:
+ error = self.state_machine.reporter.error(
+ 'The "%s" directive is empty; content required.' % self.name,
+ nodes.literal_block(self.block_text, self.block_text),
+ line=self.lineno)
+ return [error]
+
+ title, messages = self.make_title()
+ node = nodes.Element() # anonymous container for parsing
+ self.state.nested_parse(self.content, self.content_offset, node)
+
+ tableBuilder = ListTableBuilder(self)
+ tableBuilder.parseFlatTableNode(node)
+ tableNode = tableBuilder.buildTableNode()
+ # SDK.CONSOLE() # print --> tableNode.asdom().toprettyxml()
+ if title:
+ tableNode.insert(0, title)
+ return [tableNode] + messages
+
+
+# ==============================================================================
+class ListTableBuilder(object):
+# ==============================================================================
+
+ u"""Builds a table from a double-stage list"""
+
+ def __init__(self, directive):
+ self.directive = directive
+ self.rows = []
+ self.max_cols = 0
+
+ def buildTableNode(self):
+
+ colwidths = self.directive.get_column_widths(self.max_cols)
+ stub_columns = self.directive.options.get('stub-columns', 0)
+ header_rows = self.directive.options.get('header-rows', 0)
+
+ table = nodes.table()
+ tgroup = nodes.tgroup(cols=len(colwidths))
+ table += tgroup
+
+
+ for colwidth in colwidths:
+ colspec = nodes.colspec(colwidth=colwidth)
+ # FIXME: It seems, that the stub method only works well in the
+ # absence of rowspan (observed by the html builer, the docutils-xml
+ # build seems OK). This is not extraordinary, because there exists
+ # no table directive (except *this* flat-table) which allows to
+ # define coexistent of rowspan and stubs (there was no use-case
+ # before flat-table). This should be reviewed (later).
+ if stub_columns:
+ colspec.attributes['stub'] = 1
+ stub_columns -= 1
+ tgroup += colspec
+ stub_columns = self.directive.options.get('stub-columns', 0)
+
+ if header_rows:
+ thead = nodes.thead()
+ tgroup += thead
+ for row in self.rows[:header_rows]:
+ thead += self.buildTableRowNode(row)
+
+ tbody = nodes.tbody()
+ tgroup += tbody
+
+ for row in self.rows[header_rows:]:
+ tbody += self.buildTableRowNode(row)
+ return table
+
+ def buildTableRowNode(self, row_data, classes=None):
+ classes = [] if classes is None else classes
+ row = nodes.row()
+ for cell in row_data:
+ if cell is None:
+ continue
+ cspan, rspan, cellElements = cell
+
+ attributes = {"classes" : classes}
+ if rspan:
+ attributes['morerows'] = rspan
+ if cspan:
+ attributes['morecols'] = cspan
+ entry = nodes.entry(**attributes)
+ entry.extend(cellElements)
+ row += entry
+ return row
+
+ def raiseError(self, msg):
+ error = self.directive.state_machine.reporter.error(
+ msg
+ , nodes.literal_block(self.directive.block_text
+ , self.directive.block_text)
+ , line = self.directive.lineno )
+ raise SystemMessagePropagation(error)
+
+ def parseFlatTableNode(self, node):
+ u"""parses the node from a :py:class:`FlatTable` directive's body"""
+
+ if len(node) != 1 or not isinstance(node[0], nodes.bullet_list):
+ self.raiseError(
+ 'Error parsing content block for the "%s" directive: '
+ 'exactly one bullet list expected.' % self.directive.name )
+
+ for rowNum, rowItem in enumerate(node[0]):
+ row = self.parseRowItem(rowItem, rowNum)
+ self.rows.append(row)
+ self.roundOffTableDefinition()
+
+ def roundOffTableDefinition(self):
+ u"""Round off the table definition.
+
+ This method rounds off the table definition in :py:member:`rows`.
+
+ * This method inserts the needed ``None`` values for the missing cells
+ arising from spanning cells over rows and/or columns.
+
+ * recount the :py:member:`max_cols`
+
+ * Autospan or fill (option ``fill-cells``) missing cells on the right
+ side of the table-row
+ """
+
+ y = 0
+ while y < len(self.rows):
+ x = 0
+
+ while x < len(self.rows[y]):
+ cell = self.rows[y][x]
+ if cell is None:
+ x += 1
+ continue
+ cspan, rspan = cell[:2]
+ # handle colspan in current row
+ for c in range(cspan):
+ try:
+ self.rows[y].insert(x+c+1, None)
+ except:
+ # the user sets ambiguous rowspans
+ pass # SDK.CONSOLE()
+ # handle colspan in spanned rows
+ for r in range(rspan):
+ for c in range(cspan + 1):
+ try:
+ self.rows[y+r+1].insert(x+c, None)
+ except:
+ # the user sets ambiguous rowspans
+ pass # SDK.CONSOLE()
+ x += 1
+ y += 1
+
+ # Insert the missing cells on the right side. For this, first
+ # re-calculate the max columns.
+
+ for row in self.rows:
+ if self.max_cols < len(row):
+ self.max_cols = len(row)
+
+ # fill with empty cells or cellspan?
+
+ fill_cells = False
+ if 'fill-cells' in self.directive.options:
+ fill_cells = True
+
+ for row in self.rows:
+ x = self.max_cols - len(row)
+ if x and not fill_cells:
+ if row[-1] is None:
+ row.append( ( x - 1, 0, []) )
+ else:
+ cspan, rspan, content = row[-1]
+ row[-1] = (cspan + x, rspan, content)
+ elif x and fill_cells:
+ for i in range(x):
+ row.append( (0, 0, nodes.comment()) )
+
+ def pprint(self):
+ # for debugging
+ retVal = "[ "
+ for row in self.rows:
+ retVal += "[ "
+ for col in row:
+ if col is None:
+ retVal += ('%r' % col)
+ retVal += "\n , "
+ else:
+ content = col[2][0].astext()
+ if 30 < len (content):
+ content = content[:30] + "..."
+ retVal += ('(cspan=%s, rspan=%s, %r)'
+ % (col[0], col[1], content))
+ retVal += "]\n , "
+ retVal = retVal[:-2]
+ retVal += "]\n , "
+ retVal = retVal[:-2]
+ return retVal + "]"
+
+ def parseRowItem(self, rowItem, rowNum):
+ row = []
+ childNo = 0
+ error = False
+ cell = None
+ target = None
+
+ for child in rowItem:
+ if (isinstance(child , nodes.comment)
+ or isinstance(child, nodes.system_message)):
+ pass
+ elif isinstance(child , nodes.target):
+ target = child
+ elif isinstance(child, nodes.bullet_list):
+ childNo += 1
+ cell = child
+ else:
+ error = True
+ break
+
+ if childNo != 1 or error:
+ self.raiseError(
+ 'Error parsing content block for the "%s" directive: '
+ 'two-level bullet list expected, but row %s does not '
+ 'contain a second-level bullet list.'
+ % (self.directive.name, rowNum + 1))
+
+ for cellItem in cell:
+ cspan, rspan, cellElements = self.parseCellItem(cellItem)
+ if target is not None:
+ cellElements.insert(0, target)
+ row.append( (cspan, rspan, cellElements) )
+ return row
+
+ def parseCellItem(self, cellItem):
+ # search and remove cspan, rspan colspec from the first element in
+ # this listItem (field).
+ cspan = rspan = 0
+ if not len(cellItem):
+ return cspan, rspan, []
+ for elem in cellItem[0]:
+ if isinstance(elem, colSpan):
+ cspan = elem.get("span")
+ elem.parent.remove(elem)
+ continue
+ if isinstance(elem, rowSpan):
+ rspan = elem.get("span")
+ elem.parent.remove(elem)
+ continue
+ return cspan, rspan, cellItem[:]
--
v4.7-rc-2
^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [PATCH 0/7] add reST/sphinx-doc to linux documentation
2016-06-06 16:32 [PATCH 0/7] add reST/sphinx-doc to linux documentation Markus Heiser
` (6 preceding siblings ...)
2016-06-06 16:32 ` [PATCH 7/7] flat-table " Markus Heiser
@ 2016-06-07 7:54 ` Daniel Vetter
2016-06-07 8:59 ` Jani Nikula
` (3 more replies)
2016-06-07 13:47 ` Markus Heiser
8 siblings, 4 replies; 23+ messages in thread
From: Daniel Vetter @ 2016-06-07 7:54 UTC (permalink / raw)
To: Markus Heiser
Cc: Jonathan Corbet, Jani Nikula, Grant Likely,
Mauro Carvalho Chehab, Keith Packard, Linux Kernel Mailing List,
linux-doc, Hans Verkuil
On Mon, Jun 6, 2016 at 6:32 PM, Markus Heiser <markus.heiser@darmarit.de> wrote:
> From: "Heiser, Markus" <markus.heiser@darmarIT.de>
>
> Hi Jonathan, Jani, all -
>
> I merged the work from sphkerneldoc POC [1], into branch (on v4.7-rc2)
>
> git://github.com/return42/linux.git linux-doc-reST
>
> The specification of the implementation is the (new) kernel-doc-HOWTO book which
> is a part of the patch series (also online avaliable at [2]).
>
> The kernel-doc-HOWTO book is a rewrite of the kernel-doc-nano-HOWTO.txt,
> taking reST extension into account. It serves several purposes:
>
> * user guide for kernel developers
> * specification of the kernel-doc parser
> * test case
>
> The kernel_doc.py parser is a python implementation of the kernel-doc-HOWTO, the
> kernel-doc perl script is not needed for reST builds. The python variant of the
> kernel-doc parser supports the markup modes:
>
> * kernel-doc (vintage)
> * reST
>
> This series also includes the reST directives describe in the kernel-doc-HOWTO:
>
> * kernel-doc (uses/imports the parser from kernel_doc.py)
> * flat-table (table notation with meaningfull diffs)
>
> The Python/Sphinx extensions placed in the scripts/site-packages folder. With
> this folder a minimal python infrastructure is provided.
>
> The basic sphinx-build infrastructure consists of:
>
> * Documentation/Makefile.reST & Documentation/conf.py
>
> Makefile and basic 'sphinx config' file to build the various reST
> documents and output formats. Provides the basic sphinx-doc build
> infrastructure including the *sphinx-subprojects* feature. With this
> feature each book can be build and distributed stand-alone.
>
> * Documentation/books
>
> In this folder, the books with reST markup are placed. To
> provide *sphinx-subprojects*, each book has its one folder and
> a (optional) Documentation/books/{book-name}.conf file
> which *overwrites* the basic configuration from Documentation/conf.py
>
> Any comments are welcome, but please read [2] first.
I think getting the flat-table directive landed would be great. We
have a similarly annoying table in gpu.tmpl where the ascii-art tables
fall short and are annoying. We want to split it up, but that's not a
short-term solution suitable for conversion. Atm it's licensed under
gplv3, which is incompatible with the kernel, so need to fix that.
I'm still not sold on the vintage-kerneldoc idea. At least in my
experience (after first converting to asciidoc and now to sphinx) this
is a non-issue. Yes, there's the oddball misrendering, but that's no
worse than the oddball typo. And a really good way for
drive-by-contributors and newbies to send in a few useful patches.
Personally I'm totally happy to have that bit of ugliness, same way I
don't reject patches just because they trip over checkpatch.pl in some
minor detail. The downside otoh means we'll have to forever maintain 2
versions of kernel-doc, and I don't want that. Jani&me have already
ripped out some of the more bonghits features of kernel-doc (like
anything with a \w: is a heading and other nonsense like that). The
only thing we really need to keep as special is the overall kerneldoc
layout, and the special referencing using &, %, @ and friends.
Reimplementing kernel-doc in python is also something Jani&I
discussed. But I think it only makes sense if we go ahead and also
modernize the parser, i.e. use something like python-clang to parse C,
and build a modern parser-combinators thing for the kernel-doc itself.
The regex state machinery is a horror show, whether it's written in
cargo-culted perl or python.
I agree that we need to update the kernel-doc howto (and Jani's
working on that already too), but I think there's no need to split up
that finely. Maybe that's a bit against sphinx best practices, but
right now we have rather large documents (both plain text and
docbook). Splitting definitely makes sense in some cases (e.g.
gpu.tmpl is only this large because we couldn't do cross-template
referencing), but by-and-large I think the current splitting is mostly
reasonable. A better demonstration vehicle is imo converting all
existing .tmpl to .rst. And also making sure the new kernel-doc is
still able to parse all existing kernel-doc comments, even when
they're not pulled into .tmpl files. Jani has done that (using
hacked-up versions of the sphinx-lint script) for his tree, making
sure there's no unexpected changes anywhere.
I think next steps would be:
- rebase flat-table onto Jani's work and relicense under gplv2
- look into rewriting kernel-doc in python more as a long-term project
- start converting docs instead - I really want to start reaping
benefits of all this work as soon as possible.
Thanks, Daniel
>
> With regards
>
> -- Markus --
>
> [1] https://return42.github.io/sphkerneldoc
> [2] https://return42.github.io/sphkerneldoc/books/kernel-doc-HOWTO
>
>
>
> Heiser, Markus (7):
> python: add scripts/site-packages
> sphinx-doc: add basic sphinx-build infrastructure
> kernel-doc-HOWTO: add kernel-doc specification
> linuxdoc: add python package linuxdoc
> kernel-doc parser: inital python implementation
> kernel-doc directive: initial implementation
> flat-table directive: initial implementation
>
> Documentation/.gitignore | 2 +
> Documentation/Makefile.reST | 111 +
> Documentation/books/.gitignore | 0
> Documentation/books/kernel-doc-HOWTO.conf | 200 ++
> .../books/kernel-doc-HOWTO/all-in-a-tumble-src.rst | 12 +
> .../books/kernel-doc-HOWTO/all-in-a-tumble.h | 271 ++
> .../books/kernel-doc-HOWTO/all-in-a-tumble.rst | 11 +
> Documentation/books/kernel-doc-HOWTO/csv_table.txt | 6 +
> Documentation/books/kernel-doc-HOWTO/index.rst | 46 +
> .../kernel-doc-HOWTO/kernel-doc-components.rst | 35 +
> .../kernel-doc-HOWTO/kernel-doc-directive.rst | 199 ++
> .../books/kernel-doc-HOWTO/kernel-doc-examples.rst | 16 +
> .../books/kernel-doc-HOWTO/kernel-doc-intro.rst | 85 +
> .../books/kernel-doc-HOWTO/kernel-doc-syntax.rst | 171 ++
> .../kernel-doc-HOWTO/reST-kernel-doc-mode.rst | 120 +
> Documentation/books/kernel-doc-HOWTO/refs.txt | 15 +
> .../books/kernel-doc-HOWTO/table-markup.rst | 499 ++++
> .../books/kernel-doc-HOWTO/test/parser_test.h | 332 +++
> .../kernel-doc-HOWTO/vintage-kernel-doc-mode.rst | 100 +
> Documentation/conf.py | 357 +++
> Documentation/sphinx-static/theme_overrides.css | 45 +
> Documentation/sphinx-tex/Makefile | 88 +
> scripts/site-python/.gitignore | 1 +
> scripts/site-python/README | 14 +
> scripts/site-python/linuxdoc/__init__.py | 8 +
> scripts/site-python/linuxdoc/kernel_doc.py | 2764 ++++++++++++++++++++
> scripts/site-python/linuxdoc/rstFlatTable.py | 356 +++
> scripts/site-python/linuxdoc/rstKernelDoc.py | 369 +++
> 28 files changed, 6233 insertions(+)
> create mode 100644 Documentation/.gitignore
> create mode 100644 Documentation/Makefile.reST
> create mode 100644 Documentation/books/.gitignore
> create mode 100644 Documentation/books/kernel-doc-HOWTO.conf
> create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble-src.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.h
> create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/csv_table.txt
> create mode 100644 Documentation/books/kernel-doc-HOWTO/index.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-components.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-directive.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-examples.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-intro.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-syntax.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/reST-kernel-doc-mode.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/refs.txt
> create mode 100644 Documentation/books/kernel-doc-HOWTO/table-markup.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/test/parser_test.h
> create mode 100644 Documentation/books/kernel-doc-HOWTO/vintage-kernel-doc-mode.rst
> create mode 100644 Documentation/conf.py
> create mode 100644 Documentation/sphinx-static/theme_overrides.css
> create mode 100644 Documentation/sphinx-tex/Makefile
> create mode 100644 scripts/site-python/.gitignore
> create mode 100644 scripts/site-python/README
> create mode 100644 scripts/site-python/linuxdoc/__init__.py
> create mode 100755 scripts/site-python/linuxdoc/kernel_doc.py
> create mode 100755 scripts/site-python/linuxdoc/rstFlatTable.py
> create mode 100755 scripts/site-python/linuxdoc/rstKernelDoc.py
>
> --
> v4.7-rc-2
>
--
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 0/7] add reST/sphinx-doc to linux documentation
2016-06-07 7:54 ` [PATCH 0/7] add reST/sphinx-doc to linux documentation Daniel Vetter
@ 2016-06-07 8:59 ` Jani Nikula
2016-06-07 9:36 ` Markus Heiser
2016-06-07 9:14 ` Markus Heiser
` (2 subsequent siblings)
3 siblings, 1 reply; 23+ messages in thread
From: Jani Nikula @ 2016-06-07 8:59 UTC (permalink / raw)
To: Daniel Vetter, Markus Heiser
Cc: Jonathan Corbet, Grant Likely, Mauro Carvalho Chehab,
Keith Packard, Linux Kernel Mailing List, linux-doc,
Hans Verkuil
On Tue, 07 Jun 2016, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> I think getting the flat-table directive landed would be great. We
> have a similarly annoying table in gpu.tmpl where the ascii-art tables
> fall short and are annoying. We want to split it up, but that's not a
> short-term solution suitable for conversion. Atm it's licensed under
> gplv3, which is incompatible with the kernel, so need to fix that.
Agreed.
> I'm still not sold on the vintage-kerneldoc idea. At least in my
> experience (after first converting to asciidoc and now to sphinx) this
> is a non-issue. Yes, there's the oddball misrendering, but that's no
> worse than the oddball typo. And a really good way for
> drive-by-contributors and newbies to send in a few useful patches.
> Personally I'm totally happy to have that bit of ugliness, same way I
> don't reject patches just because they trip over checkpatch.pl in some
> minor detail. The downside otoh means we'll have to forever maintain 2
> versions of kernel-doc, and I don't want that. Jani&me have already
> ripped out some of the more bonghits features of kernel-doc (like
> anything with a \w: is a heading and other nonsense like that). The
> only thing we really need to keep as special is the overall kerneldoc
> layout, and the special referencing using &, %, @ and friends.
Agreed.
> Reimplementing kernel-doc in python is also something Jani&I
> discussed. But I think it only makes sense if we go ahead and also
> modernize the parser, i.e. use something like python-clang to parse C,
> and build a modern parser-combinators thing for the kernel-doc itself.
> The regex state machinery is a horror show, whether it's written in
> cargo-culted perl or python.
I think the only sensible way to change the parser is to get everything
working using the perl version first, and then you can diff the results
against the python version. That's about the only validation of the new
parser that actually matters in the real world.
At this time, I'd put this way down in the list of priorities.
> I agree that we need to update the kernel-doc howto (and Jani's
> working on that already too), but I think there's no need to split up
> that finely. Maybe that's a bit against sphinx best practices, but
> right now we have rather large documents (both plain text and
> docbook). Splitting definitely makes sense in some cases (e.g.
> gpu.tmpl is only this large because we couldn't do cross-template
> referencing), but by-and-large I think the current splitting is mostly
> reasonable.
One of the key arguments against too much splitting that hasn't been
mentioned is that despite all the fine output Sphinx can produce, we
have plenty of people who couldn't care less about running Sphinx to get
readable documentation. They will grep and read the plain text files
directly, and that's a large part of the appeal of any lightweight
markup.
When you split up a file into snippets that no longer tell a coherent
story independently, you've failed.
For the .txt files under Documentation, we mostly do not want to split
them up any more if and when they're converted to rst. For the .tmpl
files under Documentation/DocBook, each rst file split off from there
should still be a sensible document on its own, with the filename
telling what it's about. This will be the main benefit of this whole
exercise for the people who do not care about Sphinx - instead of
reading (read: ignoring) DocBook XML, they can now read the rst files.
BR,
Jani.
--
Jani Nikula, Intel Open Source Technology Center
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 0/7] add reST/sphinx-doc to linux documentation
2016-06-07 7:54 ` [PATCH 0/7] add reST/sphinx-doc to linux documentation Daniel Vetter
2016-06-07 8:59 ` Jani Nikula
@ 2016-06-07 9:14 ` Markus Heiser
2016-06-08 19:49 ` Jonathan Corbet
2016-06-15 13:01 ` Markus Heiser
3 siblings, 0 replies; 23+ messages in thread
From: Markus Heiser @ 2016-06-07 9:14 UTC (permalink / raw)
To: Daniel Vetter
Cc: Jonathan Corbet, Jani Nikula, Grant Likely,
Mauro Carvalho Chehab, Keith Packard, Linux Kernel Mailing List,
linux-doc, Hans Verkuil
Am 07.06.2016 um 09:54 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>:
> On Mon, Jun 6, 2016 at 6:32 PM, Markus Heiser <markus.heiser@darmarit.de> wrote:
>> From: "Heiser, Markus" <markus.heiser@darmarIT.de>
>>
>> Hi Jonathan, Jani, all -
>>
>> I merged the work from sphkerneldoc POC [1], into branch (on v4.7-rc2)
>>
>> git://github.com/return42/linux.git linux-doc-reST
>>
>> The specification of the implementation is the (new) kernel-doc-HOWTO book which
>> is a part of the patch series (also online avaliable at [2]).
>>
>> The kernel-doc-HOWTO book is a rewrite of the kernel-doc-nano-HOWTO.txt,
>> taking reST extension into account. It serves several purposes:
>>
>> * user guide for kernel developers
>> * specification of the kernel-doc parser
>> * test case
>>
>> The kernel_doc.py parser is a python implementation of the kernel-doc-HOWTO, the
>> kernel-doc perl script is not needed for reST builds. The python variant of the
>> kernel-doc parser supports the markup modes:
>>
>> * kernel-doc (vintage)
>> * reST
>>
>> This series also includes the reST directives describe in the kernel-doc-HOWTO:
>>
>> * kernel-doc (uses/imports the parser from kernel_doc.py)
>> * flat-table (table notation with meaningfull diffs)
>>
>> The Python/Sphinx extensions placed in the scripts/site-packages folder. With
>> this folder a minimal python infrastructure is provided.
>>
>> The basic sphinx-build infrastructure consists of:
>>
>> * Documentation/Makefile.reST & Documentation/conf.py
>>
>> Makefile and basic 'sphinx config' file to build the various reST
>> documents and output formats. Provides the basic sphinx-doc build
>> infrastructure including the *sphinx-subprojects* feature. With this
>> feature each book can be build and distributed stand-alone.
>>
>> * Documentation/books
>>
>> In this folder, the books with reST markup are placed. To
>> provide *sphinx-subprojects*, each book has its one folder and
>> a (optional) Documentation/books/{book-name}.conf file
>> which *overwrites* the basic configuration from Documentation/conf.py
>>
>> Any comments are welcome, but please read [2] first.
>
> I think getting the flat-table directive landed would be great. We
> have a similarly annoying table in gpu.tmpl where the ascii-art tables
> fall short and are annoying. We want to split it up, but that's not a
> short-term solution suitable for conversion. Atm it's licensed under
> gplv3, which is incompatible with the kernel, so need to fix that.
No problem, changed to gplv2
https://github.com/return42/linux/commit/97ee23e74384200fe79fe4557954a897725949e3
is this OK?
> I'm still not sold on the vintage-kerneldoc idea. At least in my
> experience (after first converting to asciidoc and now to sphinx) this
> is a non-issue. Yes, there's the oddball misrendering, but that's no
> worse than the oddball typo. And a really good way for
> drive-by-contributors and newbies to send in a few useful patches.
> Personally I'm totally happy to have that bit of ugliness, same way I
> don't reject patches just because they trip over checkpatch.pl in some
> minor detail. The downside otoh means we'll have to forever maintain 2
> versions of kernel-doc, and I don't want that.
This can be stopped by a reST lint process similar to what Jani mentioned.
> Jani&me have already
> ripped out some of the more bonghits features of kernel-doc (like
> anything with a \w: is a heading and other nonsense like that).
Yes, I removed it in the reST mode of the parser. This is also
a reason to differentiate between *vintage* and reST or you will
break tons of kernel-doc comments, which are valid in the term of
the *vintage* style.
> The only thing we really need to keep as special is the overall kerneldoc
> layout, and the special referencing using &, %, @ and friends.
Ok, I will try to add this highlighting feature to the reST mode.
I left it out because I feared that there are any collision with
reST markup, but with a closer look the fear might be unfounded.
> Reimplementing kernel-doc in python is also something Jani&I
> discussed. But I think it only makes sense if we go ahead and also
> modernize the parser, i.e. use something like python-clang to parse C,
> and build a modern parser-combinators thing for the kernel-doc itself.
> The regex state machinery is a horror show, whether it's written in
> cargo-culted perl or python.
Yes clang might be better solution and be more flexible on parsing
C, but it brings also much more decencies, but no one has tested
it against the kernel tree (I can't judge it, I have no experience
with) ... the regular expressions are proofed.
But anyway, if someone implements a C parser based on clang we can
use it, but before the python implementation is the best we have ;-)
> I agree that we need to update the kernel-doc howto (and Jani's
> working on that already too), but I think there's no need to split up
> that finely. Maybe that's a bit against sphinx best practices, but
> right now we have rather large documents (both plain text and
> docbook). Splitting definitely makes sense in some cases (e.g.
> gpu.tmpl is only this large because we couldn't do cross-template
> referencing), but by-and-large I think the current splitting is mostly
> reasonable.
You should be getting used to split the documents :-)
> A better demonstration vehicle is imo converting all
> existing .tmpl to .rst. And also making sure the new kernel-doc is
> still able to parse all existing kernel-doc comments, even when
> they're not pulled into .tmpl files.
This was my test case. Current migration of all .tmpl files
is based on v4.7-rc2 you will find the migrated reST books at [1]
and the rendered HTML at [2]
[1] https://github.com/return42/sphkerneldoc/tree/master/doc/books
[2] http://return42.github.io/sphkerneldoc/articles/books.html
It also finds the "Oops: Document generation inconsistency."
and mark it as to ToDo, e.g. see bottom of [3] or [4]
[3] http://return42.github.io/sphkerneldoc/books/gpu/index.html
[4] http://return42.github.io/sphkerneldoc/books/iio/index.html
> Jani has done that (using
> hacked-up versions of the sphinx-lint script) for his tree, making
> sure there's no unexpected changes anywhere.
> I think next steps would be:
> - rebase flat-table onto Jani's work and relicense under gplv2
> - look into rewriting kernel-doc in python more as a long-term project
I can't stop you ;-) .. but why not using my patch series. IMO at least
the python kernel-doc parser with deep and easy extendable
sphinx integration (compared to the pipe interface of the perl version),
the flat-table directive and the infrastructure with sphinx sub-projects
to compile and distribute each book separate are reasons to take it.
And yes, if you have any requirements, inform me, I will implement it.
Merging the highlighting feature to the reST mode ... seems the first ;-)
> - start converting docs instead - I really want to start reaping
> benefits of all this work as soon as possible.
As said, you can get all migrated .tmpl files from [1].
If you need something other migrated then v4.7-rc2 show me, where i can pull it,
I will send you a patch with migrated reST back. Or read [5]
[5] http://return42.github.io/sphkerneldoc/articles/dbtools.html
-- Markus --
>
> Thanks, Daniel
>
>
>>
>> With regards
>>
>> -- Markus --
>>
>> [1] https://return42.github.io/sphkerneldoc
>> [2] https://return42.github.io/sphkerneldoc/books/kernel-doc-HOWTO
>>
>>
>>
>> Heiser, Markus (7):
>> python: add scripts/site-packages
>> sphinx-doc: add basic sphinx-build infrastructure
>> kernel-doc-HOWTO: add kernel-doc specification
>> linuxdoc: add python package linuxdoc
>> kernel-doc parser: inital python implementation
>> kernel-doc directive: initial implementation
>> flat-table directive: initial implementation
>>
>> Documentation/.gitignore | 2 +
>> Documentation/Makefile.reST | 111 +
>> Documentation/books/.gitignore | 0
>> Documentation/books/kernel-doc-HOWTO.conf | 200 ++
>> .../books/kernel-doc-HOWTO/all-in-a-tumble-src.rst | 12 +
>> .../books/kernel-doc-HOWTO/all-in-a-tumble.h | 271 ++
>> .../books/kernel-doc-HOWTO/all-in-a-tumble.rst | 11 +
>> Documentation/books/kernel-doc-HOWTO/csv_table.txt | 6 +
>> Documentation/books/kernel-doc-HOWTO/index.rst | 46 +
>> .../kernel-doc-HOWTO/kernel-doc-components.rst | 35 +
>> .../kernel-doc-HOWTO/kernel-doc-directive.rst | 199 ++
>> .../books/kernel-doc-HOWTO/kernel-doc-examples.rst | 16 +
>> .../books/kernel-doc-HOWTO/kernel-doc-intro.rst | 85 +
>> .../books/kernel-doc-HOWTO/kernel-doc-syntax.rst | 171 ++
>> .../kernel-doc-HOWTO/reST-kernel-doc-mode.rst | 120 +
>> Documentation/books/kernel-doc-HOWTO/refs.txt | 15 +
>> .../books/kernel-doc-HOWTO/table-markup.rst | 499 ++++
>> .../books/kernel-doc-HOWTO/test/parser_test.h | 332 +++
>> .../kernel-doc-HOWTO/vintage-kernel-doc-mode.rst | 100 +
>> Documentation/conf.py | 357 +++
>> Documentation/sphinx-static/theme_overrides.css | 45 +
>> Documentation/sphinx-tex/Makefile | 88 +
>> scripts/site-python/.gitignore | 1 +
>> scripts/site-python/README | 14 +
>> scripts/site-python/linuxdoc/__init__.py | 8 +
>> scripts/site-python/linuxdoc/kernel_doc.py | 2764 ++++++++++++++++++++
>> scripts/site-python/linuxdoc/rstFlatTable.py | 356 +++
>> scripts/site-python/linuxdoc/rstKernelDoc.py | 369 +++
>> 28 files changed, 6233 insertions(+)
>> create mode 100644 Documentation/.gitignore
>> create mode 100644 Documentation/Makefile.reST
>> create mode 100644 Documentation/books/.gitignore
>> create mode 100644 Documentation/books/kernel-doc-HOWTO.conf
>> create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble-src.rst
>> create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.h
>> create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.rst
>> create mode 100644 Documentation/books/kernel-doc-HOWTO/csv_table.txt
>> create mode 100644 Documentation/books/kernel-doc-HOWTO/index.rst
>> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-components.rst
>> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-directive.rst
>> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-examples.rst
>> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-intro.rst
>> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-syntax.rst
>> create mode 100644 Documentation/books/kernel-doc-HOWTO/reST-kernel-doc-mode.rst
>> create mode 100644 Documentation/books/kernel-doc-HOWTO/refs.txt
>> create mode 100644 Documentation/books/kernel-doc-HOWTO/table-markup.rst
>> create mode 100644 Documentation/books/kernel-doc-HOWTO/test/parser_test.h
>> create mode 100644 Documentation/books/kernel-doc-HOWTO/vintage-kernel-doc-mode.rst
>> create mode 100644 Documentation/conf.py
>> create mode 100644 Documentation/sphinx-static/theme_overrides.css
>> create mode 100644 Documentation/sphinx-tex/Makefile
>> create mode 100644 scripts/site-python/.gitignore
>> create mode 100644 scripts/site-python/README
>> create mode 100644 scripts/site-python/linuxdoc/__init__.py
>> create mode 100755 scripts/site-python/linuxdoc/kernel_doc.py
>> create mode 100755 scripts/site-python/linuxdoc/rstFlatTable.py
>> create mode 100755 scripts/site-python/linuxdoc/rstKernelDoc.py
>>
>> --
>> v4.7-rc-2
>>
>
>
>
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> +41 (0) 79 365 57 48 - http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 0/7] add reST/sphinx-doc to linux documentation
2016-06-07 8:59 ` Jani Nikula
@ 2016-06-07 9:36 ` Markus Heiser
2016-06-07 11:09 ` Jani Nikula
0 siblings, 1 reply; 23+ messages in thread
From: Markus Heiser @ 2016-06-07 9:36 UTC (permalink / raw)
To: Jani Nikula
Cc: Daniel Vetter, Jonathan Corbet, Grant Likely,
Mauro Carvalho Chehab, Keith Packard, Linux Kernel Mailing List,
linux-doc, Hans Verkuil
Am 07.06.2016 um 10:59 schrieb Jani Nikula <jani.nikula@intel.com>:
> On Tue, 07 Jun 2016, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>> I think getting the flat-table directive landed would be great. We
>> have a similarly annoying table in gpu.tmpl where the ascii-art tables
>> fall short and are annoying. We want to split it up, but that's not a
>> short-term solution suitable for conversion. Atm it's licensed under
>> gplv3, which is incompatible with the kernel, so need to fix that.
>
> Agreed.
see last mail
>> I'm still not sold on the vintage-kerneldoc idea. At least in my
>> experience (after first converting to asciidoc and now to sphinx) this
>> is a non-issue. Yes, there's the oddball misrendering, but that's no
>> worse than the oddball typo. And a really good way for
>> drive-by-contributors and newbies to send in a few useful patches.
>> Personally I'm totally happy to have that bit of ugliness, same way I
>> don't reject patches just because they trip over checkpatch.pl in some
>> minor detail. The downside otoh means we'll have to forever maintain 2
>> versions of kernel-doc, and I don't want that. Jani&me have already
>> ripped out some of the more bonghits features of kernel-doc (like
>> anything with a \w: is a heading and other nonsense like that). The
>> only thing we really need to keep as special is the overall kerneldoc
>> layout, and the special referencing using &, %, @ and friends.
>
> Agreed.
see last mail
>> Reimplementing kernel-doc in python is also something Jani&I
>> discussed. But I think it only makes sense if we go ahead and also
>> modernize the parser, i.e. use something like python-clang to parse C,
>> and build a modern parser-combinators thing for the kernel-doc itself.
>> The regex state machinery is a horror show, whether it's written in
>> cargo-culted perl or python.
>
> I think the only sensible way to change the parser is to get everything
> working using the perl version first, and then you can diff the results
> against the python version. That's about the only validation of the new
> parser that actually matters in the real world.
>
> At this time, I'd put this way down in the list of priorities.
I tested the python version of the parser by running it over nearly
the whole kernel-tree .. yes this is not a side by side test, but I also
tested it by partial side by side comparing the output of [1]
[1] https://github.com/return42/sphkerneldoc/blob/master/scripts/kernel-doc
I spend over a week in testing ... IMO in the meantime the quality
of the python parser is a bit over the perl one. But at the end,
I have never seen a "error free implementation".
>> I agree that we need to update the kernel-doc howto (and Jani's
>> working on that already too), but I think there's no need to split up
>> that finely. Maybe that's a bit against sphinx best practices, but
>> right now we have rather large documents (both plain text and
>> docbook). Splitting definitely makes sense in some cases (e.g.
>> gpu.tmpl is only this large because we couldn't do cross-template
>> referencing), but by-and-large I think the current splitting is mostly
>> reasonable.
>
> One of the key arguments against too much splitting that hasn't been
> mentioned is that despite all the fine output Sphinx can produce, we
> have plenty of people who couldn't care less about running Sphinx to get
> readable documentation. They will grep and read the plain text files
> directly, and that's a large part of the appeal of any lightweight
> markup.
But they have read XML or compiled DocBook XML? ... Sphinx brings a
search engine with its html.
> When you split up a file into snippets that no longer tell a coherent
> story independently, you've failed.
Chapters are breaking stories?
> For the .txt files under Documentation, we mostly do not want to split
> them up any more if and when they're converted to rst. For the .tmpl
> files under Documentation/DocBook, each rst file split off from there
> should still be a sensible document on its own, with the filename
> telling what it's about. This will be the main benefit of this whole
> exercise for the people who do not care about Sphinx - instead of
> reading (read: ignoring) DocBook XML, they can now read the rst files.
Sorry but in IMO this suggestion is backward, if someone don't be able
to build HTML documents he should at least be able to use the
internet [1] :-o
[1] http://return42.github.io/sphkerneldoc/articles/books.html
>
> BR,
> Jani.
>
> --
> Jani Nikula, Intel Open Source Technology Center
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 0/7] add reST/sphinx-doc to linux documentation
2016-06-07 9:36 ` Markus Heiser
@ 2016-06-07 11:09 ` Jani Nikula
2016-06-07 15:13 ` Markus Heiser
0 siblings, 1 reply; 23+ messages in thread
From: Jani Nikula @ 2016-06-07 11:09 UTC (permalink / raw)
To: Markus Heiser
Cc: Daniel Vetter, Jonathan Corbet, Grant Likely,
Mauro Carvalho Chehab, Keith Packard, Linux Kernel Mailing List,
linux-doc, Hans Verkuil
On Tue, 07 Jun 2016, Markus Heiser <markus.heiser@darmarit.de> wrote:
> Am 07.06.2016 um 10:59 schrieb Jani Nikula <jani.nikula@intel.com>:
>> One of the key arguments against too much splitting that hasn't been
>> mentioned is that despite all the fine output Sphinx can produce, we
>> have plenty of people who couldn't care less about running Sphinx to get
>> readable documentation. They will grep and read the plain text files
>> directly, and that's a large part of the appeal of any lightweight
>> markup.
>
> But they have read XML or compiled DocBook XML? ... Sphinx brings a
> search engine with its html.
>
>> When you split up a file into snippets that no longer tell a coherent
>> story independently, you've failed.
>
> Chapters are breaking stories?
>
>> For the .txt files under Documentation, we mostly do not want to split
>> them up any more if and when they're converted to rst. For the .tmpl
>> files under Documentation/DocBook, each rst file split off from there
>> should still be a sensible document on its own, with the filename
>> telling what it's about. This will be the main benefit of this whole
>> exercise for the people who do not care about Sphinx - instead of
>> reading (read: ignoring) DocBook XML, they can now read the rst files.
>
> Sorry but in IMO this suggestion is backward, if someone don't be able
> to build HTML documents he should at least be able to use the
> internet [1] :-o
>
> [1] http://return42.github.io/sphkerneldoc/articles/books.html
I don't think you understand the target audience very well.
BR,
Jani.
--
Jani Nikula, Intel Open Source Technology Center
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 0/7] add reST/sphinx-doc to linux documentation
2016-06-06 16:32 [PATCH 0/7] add reST/sphinx-doc to linux documentation Markus Heiser
` (7 preceding siblings ...)
2016-06-07 7:54 ` [PATCH 0/7] add reST/sphinx-doc to linux documentation Daniel Vetter
@ 2016-06-07 13:47 ` Markus Heiser
8 siblings, 0 replies; 23+ messages in thread
From: Markus Heiser @ 2016-06-07 13:47 UTC (permalink / raw)
To: corbet
Cc: corbet, jani.nikula, daniel.vetter, grant.likely, mchehab,
keithp, linux-kernel, linux-doc, hverkuil
Hi all,
sorry I forgot to mentioning that I have not yet add the Makefile.reST
to the root Makefile. To test, please give the srctree environment in the
command line. E.g:
cd /share/linux/Documentation
srctree=/share/linux make -f Makefile.reST books/kernel-doc-HOWTO.html
-- Markus --
Am 06.06.2016 um 18:32 schrieb Markus Heiser <markus.heiser@darmarIT.de>:
> From: "Heiser, Markus" <markus.heiser@darmarIT.de>
>
> Hi Jonathan, Jani, all -
>
> I merged the work from sphkerneldoc POC [1], into branch (on v4.7-rc2)
>
> git://github.com/return42/linux.git linux-doc-reST
>
> The specification of the implementation is the (new) kernel-doc-HOWTO book which
> is a part of the patch series (also online avaliable at [2]).
>
> The kernel-doc-HOWTO book is a rewrite of the kernel-doc-nano-HOWTO.txt,
> taking reST extension into account. It serves several purposes:
>
> * user guide for kernel developers
> * specification of the kernel-doc parser
> * test case
>
> The kernel_doc.py parser is a python implementation of the kernel-doc-HOWTO, the
> kernel-doc perl script is not needed for reST builds. The python variant of the
> kernel-doc parser supports the markup modes:
>
> * kernel-doc (vintage)
> * reST
>
> This series also includes the reST directives describe in the kernel-doc-HOWTO:
>
> * kernel-doc (uses/imports the parser from kernel_doc.py)
> * flat-table (table notation with meaningfull diffs)
>
> The Python/Sphinx extensions placed in the scripts/site-packages folder. With
> this folder a minimal python infrastructure is provided.
>
> The basic sphinx-build infrastructure consists of:
>
> * Documentation/Makefile.reST & Documentation/conf.py
>
> Makefile and basic 'sphinx config' file to build the various reST
> documents and output formats. Provides the basic sphinx-doc build
> infrastructure including the *sphinx-subprojects* feature. With this
> feature each book can be build and distributed stand-alone.
>
> * Documentation/books
>
> In this folder, the books with reST markup are placed. To
> provide *sphinx-subprojects*, each book has its one folder and
> a (optional) Documentation/books/{book-name}.conf file
> which *overwrites* the basic configuration from Documentation/conf.py
>
> Any comments are welcome, but please read [2] first.
>
> With regards
>
> -- Markus --
>
> [1] https://return42.github.io/sphkerneldoc
> [2] https://return42.github.io/sphkerneldoc/books/kernel-doc-HOWTO
>
>
>
> Heiser, Markus (7):
> python: add scripts/site-packages
> sphinx-doc: add basic sphinx-build infrastructure
> kernel-doc-HOWTO: add kernel-doc specification
> linuxdoc: add python package linuxdoc
> kernel-doc parser: inital python implementation
> kernel-doc directive: initial implementation
> flat-table directive: initial implementation
>
> Documentation/.gitignore | 2 +
> Documentation/Makefile.reST | 111 +
> Documentation/books/.gitignore | 0
> Documentation/books/kernel-doc-HOWTO.conf | 200 ++
> .../books/kernel-doc-HOWTO/all-in-a-tumble-src.rst | 12 +
> .../books/kernel-doc-HOWTO/all-in-a-tumble.h | 271 ++
> .../books/kernel-doc-HOWTO/all-in-a-tumble.rst | 11 +
> Documentation/books/kernel-doc-HOWTO/csv_table.txt | 6 +
> Documentation/books/kernel-doc-HOWTO/index.rst | 46 +
> .../kernel-doc-HOWTO/kernel-doc-components.rst | 35 +
> .../kernel-doc-HOWTO/kernel-doc-directive.rst | 199 ++
> .../books/kernel-doc-HOWTO/kernel-doc-examples.rst | 16 +
> .../books/kernel-doc-HOWTO/kernel-doc-intro.rst | 85 +
> .../books/kernel-doc-HOWTO/kernel-doc-syntax.rst | 171 ++
> .../kernel-doc-HOWTO/reST-kernel-doc-mode.rst | 120 +
> Documentation/books/kernel-doc-HOWTO/refs.txt | 15 +
> .../books/kernel-doc-HOWTO/table-markup.rst | 499 ++++
> .../books/kernel-doc-HOWTO/test/parser_test.h | 332 +++
> .../kernel-doc-HOWTO/vintage-kernel-doc-mode.rst | 100 +
> Documentation/conf.py | 357 +++
> Documentation/sphinx-static/theme_overrides.css | 45 +
> Documentation/sphinx-tex/Makefile | 88 +
> scripts/site-python/.gitignore | 1 +
> scripts/site-python/README | 14 +
> scripts/site-python/linuxdoc/__init__.py | 8 +
> scripts/site-python/linuxdoc/kernel_doc.py | 2764 ++++++++++++++++++++
> scripts/site-python/linuxdoc/rstFlatTable.py | 356 +++
> scripts/site-python/linuxdoc/rstKernelDoc.py | 369 +++
> 28 files changed, 6233 insertions(+)
> create mode 100644 Documentation/.gitignore
> create mode 100644 Documentation/Makefile.reST
> create mode 100644 Documentation/books/.gitignore
> create mode 100644 Documentation/books/kernel-doc-HOWTO.conf
> create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble-src.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.h
> create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/csv_table.txt
> create mode 100644 Documentation/books/kernel-doc-HOWTO/index.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-components.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-directive.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-examples.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-intro.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-syntax.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/reST-kernel-doc-mode.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/refs.txt
> create mode 100644 Documentation/books/kernel-doc-HOWTO/table-markup.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/test/parser_test.h
> create mode 100644 Documentation/books/kernel-doc-HOWTO/vintage-kernel-doc-mode.rst
> create mode 100644 Documentation/conf.py
> create mode 100644 Documentation/sphinx-static/theme_overrides.css
> create mode 100644 Documentation/sphinx-tex/Makefile
> create mode 100644 scripts/site-python/.gitignore
> create mode 100644 scripts/site-python/README
> create mode 100644 scripts/site-python/linuxdoc/__init__.py
> create mode 100755 scripts/site-python/linuxdoc/kernel_doc.py
> create mode 100755 scripts/site-python/linuxdoc/rstFlatTable.py
> create mode 100755 scripts/site-python/linuxdoc/rstKernelDoc.py
>
> --
> v4.7-rc-2
>
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 0/7] add reST/sphinx-doc to linux documentation
2016-06-07 11:09 ` Jani Nikula
@ 2016-06-07 15:13 ` Markus Heiser
0 siblings, 0 replies; 23+ messages in thread
From: Markus Heiser @ 2016-06-07 15:13 UTC (permalink / raw)
To: Jani Nikula
Cc: Daniel Vetter, Jonathan Corbet, Grant Likely,
Mauro Carvalho Chehab, Keith Packard, Linux Kernel Mailing List,
linux-doc, Hans Verkuil
Am 07.06.2016 um 13:09 schrieb Jani Nikula <jani.nikula@intel.com>:
> On Tue, 07 Jun 2016, Markus Heiser <markus.heiser@darmarit.de> wrote:
>> Am 07.06.2016 um 10:59 schrieb Jani Nikula <jani.nikula@intel.com>:
>>> One of the key arguments against too much splitting that hasn't been
>>> mentioned is that despite all the fine output Sphinx can produce, we
>>> have plenty of people who couldn't care less about running Sphinx to get
>>> readable documentation. They will grep and read the plain text files
>>> directly, and that's a large part of the appeal of any lightweight
>>> markup.
>>
>> But they have read XML or compiled DocBook XML? ... Sphinx brings a
>> search engine with its html.
>>
>>> When you split up a file into snippets that no longer tell a coherent
>>> story independently, you've failed.
>>
>> Chapters are breaking stories?
>>
>>> For the .txt files under Documentation, we mostly do not want to split
>>> them up any more if and when they're converted to rst. For the .tmpl
>>> files under Documentation/DocBook, each rst file split off from there
>>> should still be a sensible document on its own, with the filename
>>> telling what it's about. This will be the main benefit of this whole
>>> exercise for the people who do not care about Sphinx - instead of
>>> reading (read: ignoring) DocBook XML, they can now read the rst files.
>>
>> Sorry but in IMO this suggestion is backward, if someone don't be able
>> to build HTML documents he should at least be able to use the
>> internet [1] :-o
>>
>> [1] http://return42.github.io/sphkerneldoc/articles/books.html
>
> I don't think you understand the target audience very well.
May be, help me .. I can't see the use case ... is emac's incremental search
your use case? ... possibly we are talking past each other
* chunking is my recommendation on books, e.g. chapters is a good point
to split a book .. beside: books which make use of the kernel-doc directive
are not *well grep-able* in their reST format ..
* not every .txt file should be chunked ... and folders with the 00-INDEX
are already chunked.
The only thing I want to say is, that I recommend chunking, but
at the end it should be an author decision, not a handicap of the
build system.
-- Markus --
>
> BR,
> Jani.
>
>
> --
> Jani Nikula, Intel Open Source Technology Center
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 0/7] add reST/sphinx-doc to linux documentation
2016-06-07 7:54 ` [PATCH 0/7] add reST/sphinx-doc to linux documentation Daniel Vetter
2016-06-07 8:59 ` Jani Nikula
2016-06-07 9:14 ` Markus Heiser
@ 2016-06-08 19:49 ` Jonathan Corbet
2016-06-10 15:25 ` Markus Heiser
2016-06-15 13:01 ` Markus Heiser
3 siblings, 1 reply; 23+ messages in thread
From: Jonathan Corbet @ 2016-06-08 19:49 UTC (permalink / raw)
To: Daniel Vetter
Cc: Markus Heiser, Jani Nikula, Grant Likely, Mauro Carvalho Chehab,
Keith Packard, Linux Kernel Mailing List, linux-doc,
Hans Verkuil
So I've finally gotten a chance to make another pass over this stuff.
Markus, your enthusiasm is great; I'm hoping you'll do great things
helping us to improve the kernel's documentation toolchain. But please,
at this point, let's build on Jani's work and go from there. Things have
waited for long enough while we've gone around on this; I think what we
have is a good starting point.
On the specifics, Daniel already covered most of it pretty well.
On Tue, 7 Jun 2016 09:54:21 +0200
Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> I think next steps would be:
> - rebase flat-table onto Jani's work and relicense under gplv2
This I would really like to see.
> - look into rewriting kernel-doc in python more as a long-term project
There is nobody who would like to dump the Perl kernel-doc more than I
would; it wasn't pretty to begin with and hasn't improved over the years.
I, too, had thought about redoing it, but I, too, concluded that it wasn't
the highest of priorities.
Please do keep this around, we may want it before too long. I have some
sympathy for Daniel's suggestion to look into using LLVM; we could also
maybe stay a little closer to our roots and use the sparse library. But
there might also be value in a Python version that doesn't add more
dependencies to the docs toolchain. We need to think about this, but I
don't think we need to answer it now.
> - start converting docs instead - I really want to start reaping
> benefits of all this work as soon as possible.
Absolutely.
Along these lines, I don't currently have a strong opinion on the
big-files vs. little-files question. I *do*, however, like the idea of
trying to create one coherent kernel document rather than perpetuation our
current collection of independent book silos. Initially it will certainly
look like the LDP-based books that people used to duct-tape together back
in the 90's, but it should improve over time.
Thanks,
jon
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 3/7] kernel-doc-HOWTO: add kernel-doc specification
2016-06-06 16:32 ` [PATCH 3/7] kernel-doc-HOWTO: add kernel-doc specification Markus Heiser
@ 2016-06-09 18:05 ` Randy Dunlap
2016-06-09 18:42 ` Jani Nikula
2016-06-10 16:00 ` Markus Heiser
0 siblings, 2 replies; 23+ messages in thread
From: Randy Dunlap @ 2016-06-09 18:05 UTC (permalink / raw)
To: Markus Heiser, corbet
Cc: jani.nikula, daniel.vetter, grant.likely, mchehab, keithp,
linux-kernel, linux-doc, hverkuil
Hi,
Some spellos and a few questions...
On 06/06/16 09:32, Markus Heiser wrote:
> From: "Heiser, Markus" <markus.heiser@darmarIT.de>
>
> The kernel-doc-HOWTO book is a rewrite of the kernel-doc-nano-HOWTO.txt,
> taking reST extension into account.
>
> The kernel-doc-HOWTO is a user guide for kernel developers that also
> serves as a specification.
>
> The format for this documentation is called the *kernel-doc* format. With kernel
> version 4.x the restructuredText (reST_) markup was added to the *kernel-doc*
> format. The kernel-doc parser supports the markup modes:
>
> * kernel-doc (vintage)
> * reST
>
> This document gives hints on how to use these markups and how to refer
> and extract documentation from source files.
>
> A online kernel-doc-HOWTO is available at [1]
>
> [1] http://return42.github.io/sphkerneldoc/books/kernel-doc-HOWTO
>
> Signed-off-by: Markus Heiser <markus.heiser@darmarIT.de>
> ---
> Documentation/books/kernel-doc-HOWTO.conf | 200 +++++++++
> .../books/kernel-doc-HOWTO/all-in-a-tumble-src.rst | 12 +
> .../books/kernel-doc-HOWTO/all-in-a-tumble.h | 271 +++++++++++
> .../books/kernel-doc-HOWTO/all-in-a-tumble.rst | 11 +
> Documentation/books/kernel-doc-HOWTO/csv_table.txt | 6 +
> Documentation/books/kernel-doc-HOWTO/index.rst | 46 ++
> .../kernel-doc-HOWTO/kernel-doc-components.rst | 35 ++
> .../kernel-doc-HOWTO/kernel-doc-directive.rst | 199 ++++++++
> .../books/kernel-doc-HOWTO/kernel-doc-examples.rst | 16 +
> .../books/kernel-doc-HOWTO/kernel-doc-intro.rst | 85 ++++
> .../books/kernel-doc-HOWTO/kernel-doc-syntax.rst | 171 +++++++
> .../kernel-doc-HOWTO/reST-kernel-doc-mode.rst | 120 +++++
> Documentation/books/kernel-doc-HOWTO/refs.txt | 15 +
> .../books/kernel-doc-HOWTO/table-markup.rst | 499 +++++++++++++++++++++
> .../books/kernel-doc-HOWTO/test/parser_test.h | 332 ++++++++++++++
> .../kernel-doc-HOWTO/vintage-kernel-doc-mode.rst | 100 +++++
> 16 files changed, 2118 insertions(+)
> create mode 100644 Documentation/books/kernel-doc-HOWTO.conf
> create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble-src.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.h
> create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/csv_table.txt
> create mode 100644 Documentation/books/kernel-doc-HOWTO/index.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-components.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-directive.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-examples.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-intro.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-syntax.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/reST-kernel-doc-mode.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/refs.txt
> create mode 100644 Documentation/books/kernel-doc-HOWTO/table-markup.rst
> create mode 100644 Documentation/books/kernel-doc-HOWTO/test/parser_test.h
> create mode 100644 Documentation/books/kernel-doc-HOWTO/vintage-kernel-doc-mode.rst
>
> diff --git a/Documentation/books/kernel-doc-HOWTO.conf b/Documentation/books/kernel-doc-HOWTO.conf
> new file mode 100644
> index 0000000..4e8c32a
> --- /dev/null
> +++ b/Documentation/books/kernel-doc-HOWTO.conf
> @@ -0,0 +1,200 @@
> +# -*- coding: utf-8; mode: python -*-
> +#
> +# This is the project specific sphinx-build configuration, which is loaded from
> +# the base configuration file (``../conf.py``). About config values consult:
> +#
> +# * http://www.sphinx-doc.org/en/stable/config.html
> +#
> +# While setting values here, please take care to not overwrite common needed
> +# configurations. This means, do not *overwrite* composite values (e.g. the
> +# list- or dictionary-value of "latex_elements" resp. "extensions") by
> +# thoughtless assignments. Manipulate composite values always by *update*
> +# (dict-values) or extend (list-values). Nevertheless, if you know what you are
> +# doing, you are free to *overwrite* values to your needs.
> +#
> +# useful preseted names:
preset
> +#
> +# * BASE_FOLDER: the folder where the top conf.py is located
> +# * main_name: the basename of this project-folder
> +
> +# ------------------------------------------------------------------------------
> +# General configuration
> +# ------------------------------------------------------------------------------
> +
> +project = u'kernel-doc HOWTO'
> +copyright = u'2016, Linux documentation authors'
> +author = u'Linux contributors'
> +
> +extlinks.update({ })
> +
> +intersphinx_mapping.update({ })
> +
> +extensions.extend([
> + # 'sphinx.ext.pngmath'
> + #, 'sphinx.ext.mathjax'
> +])
> +
> +# The version info for the project you're documenting, acts as replacement for
> +# |version| and |release|, also used in various other places throughout the
> +# built documents.
> +#
> +# The short X.Y version.
> +version = u'1.0'
> +# The full version, including alpha/beta/rc tags.
> +# release = u'0'
> +
> +# ------------------------------------------------------------------------------
> +# Options for HTML output
> +# ------------------------------------------------------------------------------
> +
> +# The name for this set of Sphinx documents. If None, it defaults to
> +# "<project> v<release> documentation".
> +#html_title = None
> +
> +# A shorter title for the navigation bar. Default is the same as html_title.
> +#html_short_title = None
> +
> +# The name of an image file (relative to this directory) to place at the top
> +# of the sidebar.
> +#html_logo = pathjoin(BASE_FOLDER, "_tex", "logo.png")
> +
> +# The name of an image file (within the static path) to use as favicon of the
> +# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
> +# pixels large.
> +#html_favicon = None
> +
> +# Add any paths that contain custom static files (such as style sheets) here,
> +# relative to this directory. They are copied after the builtin static files,
> +# so a file named "default.css" will overwrite the builtin "default.css".
> +html_static_path.extend([])
> +
> +# Output file base name for HTML help builder.
> +htmlhelp_basename = main_name
> +
> +# ------------------------------------------------------------------------------
> +# Options for XeLaTeX output
> +# ------------------------------------------------------------------------------
> +
> +xelatex_documents = [
> + # A book with default DIN A4 and 12pt
> + dict(docname = master_doc
> + , targetname = "%s.tex" % main_name
> + , documentclass = "darmarITArticle")]
> +
> +# This value determines the topmost sectioning unit. It should be chosen from
> +# ``part``, ``chapter`` or ``section``. The default is ``None``; the topmost
> +# sectioning unit is switched by documentclass. ``section`` is used if
> +# documentclass will be ``howto``, otherwise ``chapter`` will be used.
> +
> +latex_toplevel_sectioning = 'chapter'
> +
> +# ------------------------------------------------------------------------------
> +# Options for manual page output
> +# ------------------------------------------------------------------------------
> +
> +# One entry per manual page. List of tuples
> +# (source start file, name, description, authors, manual section).
> +# man_pages = [
> +# (master_doc, 'kernel-doc', u'Kernel-Doc',
> +# [author], 1)
> +# ]
> +
> +# If true, show URL addresses after external links.
> +#man_show_urls = False
> +
> +# ------------------------------------------------------------------------------
> +# Options for Texinfo output
> +# ------------------------------------------------------------------------------
> +
> +# Grouping the document tree into Texinfo files. List of tuples
> +# (source start file, target name, title, author,
> +# dir menu entry, description, category)
> +# texinfo_documents = [
> +# (master_doc, 'Kernel-Doc', u'Kernel-Doc Documentation',
> +# author, 'Kernel-Doc', 'One line description of project.',
> +# 'Miscellaneous'),
> +# ]
> +
> +# Documents to append as an appendix to all manuals.
> +#texinfo_appendices = []
> +
> +# If false, no module index is generated.
> +#texinfo_domain_indices = True
> +
> +# How to display URL addresses: 'footnote', 'no', or 'inline'.
> +#texinfo_show_urls = 'footnote'
> +
> +# If true, do not generate a @detailmenu in the "Top" node's menu.
> +#texinfo_no_detailmenu = False
> +
> +# ------------------------------------------------------------------------------
> +# Options for Epub output
> +# ------------------------------------------------------------------------------
> +
> +# Bibliographic Dublin Core info.
> +# epub_title = project
> +# epub_author = author
> +# epub_publisher = author
> +# epub_copyright = copyright
> +
> +# The basename for the epub file. It defaults to the project name.
> +#epub_basename = project
> +
> +# The HTML theme for the epub output. Since the default themes are not
> +# optimized for small screen space, using the same theme for HTML and epub
> +# output is usually not wise. This defaults to 'epub', a theme designed to save
> +# visual space.
> +#epub_theme = 'epub'
> +
> +# The language of the text. It defaults to the language option
> +# or 'en' if the language is not set.
> +#epub_language = ''
> +
> +# The scheme of the identifier. Typical schemes are ISBN or URL.
> +#epub_scheme = ''
> +
> +# The unique identifier of the text. This can be a ISBN number
> +# or the project homepage.
> +#epub_identifier = ''
> +
> +# A unique identification for the text.
> +#epub_uid = ''
> +
> +# A tuple containing the cover image and cover page html template filenames.
> +#epub_cover = ()
> +
> +# A sequence of (type, uri, title) tuples for the guide element of content.opf.
> +#epub_guide = ()
> +
> +# HTML files that should be inserted before the pages created by sphinx.
> +# The format is a list of tuples containing the path and title.
> +#epub_pre_files.extend([])
> +
> +# HTML files that should be inserted after the pages created by sphinx.
> +# The format is a list of tuples containing the path and title.
> +#epub_post_files.extend([])
> +
> +# A list of files that should not be packed into the epub file.
> +epub_exclude_files.extend([])
> +
> +# The depth of the table of contents in toc.ncx.
> +#epub_tocdepth = 3
> +
> +# Allow duplicate toc entries.
> +#epub_tocdup = True
> +
> +# Choose between 'default' and 'includehidden'.
> +#epub_tocscope = 'default'
> +
> +# Fix unsupported image types using the Pillow.
> +#epub_fix_images = False
> +
> +# Scale large images.
> +#epub_max_image_width = 0
> +
> +# How to display URL addresses: 'footnote', 'no', or 'inline'.
> +#epub_show_urls = 'inline'
> +
> +# If false, no index is generated.
> +#epub_use_index = True
> +
> diff --git a/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble-src.rst b/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble-src.rst
> new file mode 100644
> index 0000000..9d360ae
> --- /dev/null
> +++ b/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble-src.rst
> @@ -0,0 +1,12 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +.. include:: refs.txt
> +
> +.. _all-in-a-tumble-src:
> +
> +================
> +Examples Sources
> +================
> +
> +.. literalinclude:: ./all-in-a-tumble.h
> + :linenos:
> + :language: c
> diff --git a/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.h b/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.h
> new file mode 100644
> index 0000000..d72c8bd
> --- /dev/null
> +++ b/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.h
> @@ -0,0 +1,271 @@
> +/* parse-markup: reST */
> +
> +/* parse-SNIP: intro */
> +/**
> + * DOC: kernel-doc intro
> + *
> + * This file / chapter includes all the refered examples of the kernel-doc-HOWTO
referred
> + * and additional tests.
> + *
> + * The content itself is nonsens / don’t look to close ;-)
nonsense
> + */
> +/* parse-SNAP: */
> +
> +
> +/* parse-SNIP: hello-world */
> +#include<stdio.h>
> +int main() {
> + printf("Hello World\n");
> + return 0;
> +}
> +/* parse-SNAP: */
> +
> +
> +/* parse-SNIP: my_struct */
> +/**
> + * struct my_struct - short description
> + * @a: first member
> + * @b: second member
> + *
> + * Longer description
> + */
> +struct my_struct {
> + int a;
> + int b;
> +/* private: */
> + int c;
> +};
> +/* parse-SNAP: */
> +
> +
> +/* parse-SNIP: my_long_struct */
> +/**
> + * struct my_struct - short description
> + * @a: first member
> + * @b: second member
> + *
> + * Longer description
> + */
> +struct my_long_struct {
> + int a;
> + int b;
> + /**
> + * @c: This is longer description of C.
> + *
> + * You can use paragraphs to describe arguments
> + * using this method.
> + */
> + int c;
> +};
> +/* parse-SNAP: */
> +
> +
> +/* parse-SNIP: theory-of-operation */
> +/**
> + * DOC: Theory of Operation
> + *
> + * The whizbang foobar is a dilly of a gizmo. It can do whatever you
> + * want it to do, at any time. It reads your mind. Here's how it works.
> + *
> + * foo bar splat
> + *
> + * The only drawback to this gizmo is that is can sometimes damage
> + * hardware, software, or its subject(s).
> + */
> +/* parse-SNAP: */
> +
> +
> +
> +/* parse-SNIP: user_function */
> +/**
> + * user_function - function that can only be called in user context
> + * @a: some argument
> + * Context: !in_interrupt()
> + *
> + * This function makes no sense, it is only kernel-doc demonstration.
> + *
> + * ::
> + *
> + * Example:
> + * x = user_function(22);
> + *
> + * Return:
> + * Returns first argument
> + */
> +int user_function(int a)
> +{
> + return a;
> +}
> +/* parse-SNAP: */
> +
> +
> +
> +
> +/* parse-markup: kernel-doc */
> +
> +/**
> + * vintage - short description of this function
> + * @parameter_a: first argument
> + * @parameter_b: second argument
> + * Context: in_gizmo_mode().
> + *
> + * Long description. This function has two integer arguments. The first is
> + * @parameter_a and the second is @parameter_b.
> + *
> + * Example:
> + * user_function(22);
> + *
> + * Return:
> + * Sum of @parameter_a and @parameter_b.
> + *
> + * highlighting:
> + *
> + * - vintage() : function
> + * - @parameter_a : name of a parameter
> + * - $ENVVAR : environmental variable
> + * - &my_struct : name of a structure (up to two words including ``struct``)
> + * - %CONST : name of a constant.
> + *
> + * Parser Mode: *vintage* kernel-doc mode
> + *
> + * Within the *vintage kernel-doc mode* dogged ignores any whitespace or inline
> + * markup.
> + *
> + * - Inline markup like *emphasis* or **emphasis strong**
> + * - Literals and/or block indent:
> + *
> + * a + b
> + *
> + * In kernel-doc *vintage* mode, there are no special block or inline markups
> + * available. Markups like the one above result in ambiguous reST markup which
> + * could produce error messages in the subsequently sphinx-build
> + * process. Unexpected outputs are mostly the result.
> + *
> + * This is a link https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/
> + * to the Linux kernel source tree
> + *
> + * colon markup: sectioning by colon markup in vintage mode is partial ugly. ;-)
> + *
> + */
> +int vintage(int parameter_a, char parameter_b)
> +{
> + return a + b;
> +}
> +
> +
> +/* parse-markup: reST */
> +
> +/**
> + * rst_mode - short description of this function
> + * @a: first argument
> + * @b: second argument
> + * Context: :c:func:`in_gizmo_mode`.
> + *
> + * Long description. This function has two integer arguments. The first is
> + * ``parameter_a`` and the second is ``parameter_b``.
> + *
> + * As long as the reST / sphinx-doc toolchain uses `intersphinx
> + * <http://www.sphinx-doc.org/en/stable/ext/intersphinx.html>`__ you can refer
> + * definitions *outside* like :c:type:`struct media_device <media_device>`. If
> + * the description of ``media_device`` struct is found in any of the intersphinx
> + * locations, a hyperref to this target is genarated a build time.
generated
> + *
> + * Example:
> + *
> + * .. code-block:: c
> + *
> + * user_function(22);
> + *
> + * Return:
> + * Sum of ``parameter_a`` and the second is ``parameter_b``.
> + *
> + * highlighting:
> + *
> + * Because reST markup syntax conflicts with the highlighting markup from the
> + * *vintage* mode, these *vintage* highlighting markup is not available in
> + * reST-mode. reST brings it's own markup to refer and highlight function,
> + * structs or whatever definition :
> + *
> + * - :c:func:`rst_mode` : function
> + * - ``$ENVVAR``: environmental variable
> + * - :c:type:`my_struct` : name of a structure
> + * - ``parameter_a`` : name of a parameter
> + * - ``CONST`` : name of a constant.
> + *
> + * Parser Mode:
> + *
> + * This is an example with activated reST additions, in this section you will
> + * find some common inline markups.
> + *
> + * Within the *reST mode* the kernel-doc parser pass through all markups to the
> + * reST toolchain, except the *vintage highlighting* but including any
> + * whitespace. With this, the full reST markup is available in the comments.
> + *
> + * This is a link to the `Linux kernel source tree
> + * <https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/>`_.
> + *
> + * This description is only to show some reST inline markups like *emphasise*
> + * and **emphasis strong**. The follwing is a demo of a reST list markup:
following
> + *
> + * Definition list:
> + * :def1: lorem
> + * :def2: ipsum
> + *
> + * Ordered List:
> + * - item one
> + * - item two
> + * - item three with
> + * a linebreak
> + *
> + * Literal blocks:
> + * The next example shows a literal block::
> + *
> + * +------+ +------+
> + * |\ |\ /| /|
> + * | +----+-+ +-+----+ |
> + * | | | | | | | |
> + * +-+----+ | | +----+-+
> + * \| \| |/ |/
> + * +------+ +------+
> + *
> + * Highlighted code blocks:
> + * The next example shows a code example, with highlighting C syntax in the
> + * output.
> + *
> + * .. code-block:: c
> + *
> + * // Hello World program
> + *
> + * #include<stdio.h>
> + *
> + * int main()
> + * {
> + * printf("Hello World");
> + * }
> + *
> + *
> + * reST sectioning:
> + *
> + * colon markup: sectioning by colon markup in reST mode is less ugly. ;-)
> + *
> + * A kernel-doc section like *this* section is translated into a reST
> + * *subsection*. This means, you can only use the follwing *sub-levels* within a
following
> + * kernel-doc section.
> + *
> + * a subsubsection
> + * ^^^^^^^^^^^^^^^
> + *
> + * lorem ipsum
> + *
> + * a paragraph
> + * """""""""""
> + *
> + * lorem ipsum
> + *
> + */
> +
> +int rst_mode(int a, char b)
> +{
> + return a + b;
> +}
> +
> diff --git a/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.rst b/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.rst
> new file mode 100644
> index 0000000..17b127b
> --- /dev/null
> +++ b/Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.rst
> @@ -0,0 +1,11 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +.. include:: refs.txt
> +
> +.. _all-in-a-tumble:
> +
> +=================
> +Rendered Examples
> +=================
> +
> +.. kernel-doc:: ./all-in-a-tumble.h
> + :module: example
> diff --git a/Documentation/books/kernel-doc-HOWTO/csv_table.txt b/Documentation/books/kernel-doc-HOWTO/csv_table.txt
> new file mode 100644
> index 0000000..8a14541
> --- /dev/null
> +++ b/Documentation/books/kernel-doc-HOWTO/csv_table.txt
> @@ -0,0 +1,6 @@
> +stub col row 1, column, "loremLorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy
> +eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam
> +voluptua."
> +stub col row 1, "At vero eos et accusam et justo duo dolores et ea rebum. Stet clita
> +kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.", column
> +stub col row 1, column, column
> diff --git a/Documentation/books/kernel-doc-HOWTO/index.rst b/Documentation/books/kernel-doc-HOWTO/index.rst
> new file mode 100644
> index 0000000..1893303
> --- /dev/null
> +++ b/Documentation/books/kernel-doc-HOWTO/index.rst
> @@ -0,0 +1,46 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +.. include:: refs.txt
> +
> +.. _kernel-doc-howto:
> +
> +================
> +kernel-doc HOWTO
> +================
> +
> +In order to provide embedded, 'C' friendly, easy to maintain, but consistent and
> +extractable documentation of the functions and data structures in the Linux
> +kernel, the Linux kernel has adopted a consistent style for documenting
> +functions and their parameters, and structures and their members.
> +
> +The format for this documentation is called the *kernel-doc* format. With kernel
> +version 4.x the restructuredText (reST_) markup was added to the *kernel-doc*
> +format. The kernel-doc parser supports the markup modes:
> +:ref:`vintage-kernel-doc-mode` / :ref:`reST-kernel-doc-mode`. This document
> +gives hints on how to use these markups and how to refer and extract
> +documentation from source files.
> +
> +The kernel-doc parser extracts the kernel-doc descriptions from the source files
> +and produced reST markup as base format. With reST as base format the
> +documentation building process changed also. The building process is now based
> +on sphinx-doc_ and the DocBook documents will be migrated to reST gradually.
> +
> +.. toctree::
> + :maxdepth: 1
> +
> + kernel-doc-intro
> + kernel-doc-syntax
> + vintage-kernel-doc-mode
> + reST-kernel-doc-mode
> + kernel-doc-directive
> + table-markup
> + kernel-doc-components
> + kernel-doc-examples
> +
> +The examples in this HOWTO using the kernel-doc comments from the example file
> +:ref:`all-in-a-tumble-src`. They are rendered in the
> +chapter :ref:`all-in-a-tumble`.
> +
> +.. only:: html
> +
> + * :ref:`genindex`
> +
> diff --git a/Documentation/books/kernel-doc-HOWTO/kernel-doc-components.rst b/Documentation/books/kernel-doc-HOWTO/kernel-doc-components.rst
> new file mode 100644
> index 0000000..f515f04
> --- /dev/null
> +++ b/Documentation/books/kernel-doc-HOWTO/kernel-doc-components.rst
> @@ -0,0 +1,35 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +.. include:: refs.txt
> +
> +.. _kernel-doc-components:
> +
> +===================================
> +Components of the kernel-doc system
> +===================================
> +
> +Many places in the source tree have extractable kernel-doc documentation. The
> +components of this system are:
> +
> +Documentation/Makefile.reST and Documentation/conf.py
> + Makefile and basic `sphinx config`_ file to build the various reST documents
> + and output formats. Provides the basic sphinx-doc_ build infrastructure
> + including the *sphinx-subprojects* feature. With this feature each book can be
> + build and distributed stand-alone. Cross reference between *subprojects* will
> + be ensured by `intersphinx`_.
> +
> +Documentation/sphinx-static and Documentation/sphinx-tex
> + Paths that contain sphinx-doc_ custom static files (such as style sheets).
> +
> +Documentation/books
> + In this folder, the books with reST markup are placed. To provide
> + *sphinx-subprojects*, each book has its one folder and a (optional)
> + ``Documentation/books/{book-name}.conf`` file which *overwrites* the basic
> + configuration from ``Documentation/conf.py`` (settings see `sphinx config`_)
> +
> +scripts/site-python/linuxdoc
> + This folder includes python extensions related to the linux documentation
> + processes.
> +
> +
> +
> +
> diff --git a/Documentation/books/kernel-doc-HOWTO/kernel-doc-directive.rst b/Documentation/books/kernel-doc-HOWTO/kernel-doc-directive.rst
> new file mode 100644
> index 0000000..5ae5c76
> --- /dev/null
> +++ b/Documentation/books/kernel-doc-HOWTO/kernel-doc-directive.rst
> @@ -0,0 +1,199 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +.. include:: refs.txt
> +
> +.. _kernel-doc-directive:
> +
> +================================
> +Use kernel-doc in reST documents
> +================================
> +
> +There exists a `reST-directive
> +<http://www.sphinx-doc.org/en/stable/rest.html#directives>`_ named
> +``kernel-doc`` to integrate kernel-doc comments into a reST document (e.g. in a
> +*book*). The directive comes with options to fine grain control which parts
> +should be placed into the reST document. With no options given, the complete
> +kernel-doc comments from a source file will be inserted. So, the first and very
> +simple example is:
> +
> +.. code-block:: rst
> +
> + My Media Book
> + =============
> +
> + .. kernel-doc:: include/media/media-device.h
> +
> +With this small example the kernel-doc comments from the media-device.h will be
> +inserted direct under the chapter "My Media Book". The "DOC:" sections, the function
> +and the type descriptions will be inserted in the order they appear in the source file.
> +Mostly you want to select more fine grained, read on to see how.
> +
> +kernel-doc config
> +=================
> +
> +Within the sphinx-doc config file (``config.py``) you can set the following
> +option.
> +
> +kernel_doc_raise_error: ``True``
> + If true, fatal errors (like missing function descriptions) raise an error. The
> + default is ``True``. Because this might break your build process, you can
> + change the value to ``False``.
> +
> + In this example, the documentation of definition ``no_longer_exists`` is
> + required.
> +
> + .. code-block:: rst
> +
> + .. kernel-doc:: ./all-in-a-tumble.h
> + :functions: no_longer_exist
> +
> + Since this definition not exists (anymore), the following TODO entry is
> + inserted, when ``kernel_doc_raise_error`` is ``False``.
> +
> + .. kernel-doc:: ./all-in-a-tumble.h
> + :functions: no_longer_exist
> +
> +
> +kernel-doc options
> +==================
> +
> +Here is a short overview of the options:
> +
> +.. code-block:: rst
> +
> + .. kernel-doc:: <filename>
> + :doc: <section title>
> + :export:
> + :internal:
> + :functions: <function [, functions [, ...]]>
> + :module: <prefix-id>
> + :snippets: <snippet [, snippets [, ...]]>
> + :language: <snippet-lang>
> + :linenos:
> + :debug:
> +
> +The argument ``<filename>`` is required, it points to a source file in the
> +kernel source tree. The pathname is relativ to kernel's root folder. The
> +options have the following meaning, but be aware that not all combinations of
> +these options make sense:
> +
> +``doc <section title>``
> + Inserts the contents of the ``DOC:`` section titled ``<section title>``.
> + Spaces are allowed in ``<section title>``; do not quote the ``<section
> + title>``.
> +
> +``export``
> + Inserts the documentation of function, struct or whatever definition that is
> + exported using EXPORT_SYMBOL (``EXPORT_SYMBOL()``, ``EXPORT_SYMBOL_GPL()`` &
> + ``EXPORT_SYMBOL_GPL_FUTURE()``). Assume that a the all exported symbols are
> + documented.
> +
> +``internal``
> + Inserts the documentation of function, struct or whatever definition that
> + is documented, but not **not** exported using EXPORT_SYMBOL.
> +
> +``functions <name [, names [, ...]]>``
> + Inserts the documentation of function(s), struct(s) or whatever
> + definition(s) named ``name``.
> +
> +``module <prefix-id>``
> + The option ``:module: <id-prefix>`` sets a module-name. The module-name is
> + used as a prefix for automatic generated IDs (reference anchors).
> +
> +``snippets <name [, names [, ...]]>``
> + Inserts the source-code passage(s) marked with the snippet ``name``. The
> + snippet is inserted with a `code-block:: <http://www.sphinx-doc.org/en/stable/markup/code.html>`_
> + directive.
> +
> + The next options make only sense in conjunction with option ``snippets``:
> +
> + ``:language: <highlighter>``
> + Set highlighting language of the snippet code-block.
> +
> + ``:linenos:``
> + Set line numbers in the snippet code-block.
> +
> +``debug``
> + Inserts a code-block with the generated reST source. This might somtimes
> + helpful to see how the kernel-doc parser transforms the kernel-doc markup to
> + reST markup.
> +
> +ducumentation blocks
> +====================
> +
> +The following example inserts the documentation block with the title "Theory of
> +Operation".
> +
> +.. code-block:: rst
> +
> + .. kernel-doc:: ./all-in-a-tumble.h
> + :doc: Theory of Operation
> + :module: example
> +
> +With the module name "example" the title refers by:
> +
> +.. code-block:: rst
> +
> + Rendered example: :ref:`example.theory-of-operation`
> +
> +Rendered example: :ref:`example.theory-of-operation`
> +
> +functions
> +=========
> +
> +The following example inserts the documentation of struct 'user_function'.
> +
> +.. code-block:: rst
> +
> + .. kernel-doc:: ./all-in-a-tumble.h
> + :functions: user_function
> + :module: example
> +
> +.. code-block:: rst
> +
> + * Rendered example by ID with module prefix: :ref:`example.user_function`
> + * Function reference: :c:func:`user_function`
> +
> +* Rendered example by ID with module prefix: :ref:`example.user_function`
> +* Function reference: :c:func:`user_function`
> +
> +
> +structs, unions, enums and typedefs
> +===================================
> +
> +The following example inserts the documentation of struct 'my_long_struct'.
> +
> +.. code-block:: rst
> +
> + .. kernel-doc:: ./all-in-a-tumble.h
> + :functions: my_long_struct
> + :module: example
> +
> +.. code-block:: rst
> +
> + * Rendered example by ID with module prefix: :ref:`example.my_long_struct`
> + * Type reference: :c:type:`my_long_struct` or the alternativ notation
> + with title :c:type:`struct my_long_struct <my_long_struct>`
> +
> +* Rendered example by ID with module prefix: :ref:`example.my_long_struct`
> +* Type reference: :c:type:`my_long_struct` or the alternativ notation
> + with title :c:type:`struct my_long_struct <my_long_struct>`
> +
> +Snippets
> +========
> +
> +The kernel-doc Parser supports a comment-markup for
> +:ref:`kernel-doc-syntax-snippets`. By example; The directive of the shown
> +code-snippet below is:
> +
> +.. code-block:: rst
> +
> + .. kernel-doc:: ./all-in-a-tumble.h
> + :snippets: hello-world
> + :language: c
> + :linenos:
> +
> +.. kernel-doc:: ./all-in-a-tumble.h
> + :snippets: hello-world
> + :language: c
> + :linenos:
> +
> diff --git a/Documentation/books/kernel-doc-HOWTO/kernel-doc-examples.rst b/Documentation/books/kernel-doc-HOWTO/kernel-doc-examples.rst
> new file mode 100644
> index 0000000..4c3ba3b
> --- /dev/null
> +++ b/Documentation/books/kernel-doc-HOWTO/kernel-doc-examples.rst
> @@ -0,0 +1,16 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +.. include:: refs.txt
> +
> +.. _kernel-doc-examples:
> +
> +=====================
> +Examples & test cases
> +=====================
> +
> +.. toctree::
> + :maxdepth: 2
> +
> + all-in-a-tumble-src
> + all-in-a-tumble
> +
> +
> diff --git a/Documentation/books/kernel-doc-HOWTO/kernel-doc-intro.rst b/Documentation/books/kernel-doc-HOWTO/kernel-doc-intro.rst
> new file mode 100644
> index 0000000..8cad730
> --- /dev/null
> +++ b/Documentation/books/kernel-doc-HOWTO/kernel-doc-intro.rst
> @@ -0,0 +1,85 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +.. include:: refs.txt
> +
> +.. _kernel-doc-intro:
> +
> +================
> +kernel-doc intro
> +================
> +
> +In order to provide good documentation of kernel functions and data structures,
> +please use the following conventions to format your kernel-doc comments in Linux
> +kernel source.
> +
> +We also look to provide kernel-doc formatted documentation for functions
> +externally visible to other kernel files (not marked "static"). We also
> +recommend providing kernel-doc formatted documentation for private (file
> +"static") routines, for consistency of kernel source code layout. But this is
> +lower priority and at the discretion of the MAINTAINER of that kernel source
> +file. Data structures visible in kernel include files should also be documented
> +using kernel-doc formatted comments.
> +
> +The opening comment mark ``/**`` is reserved for kernel-doc comments. Only
> +comments so marked will be considered by the kernel-doc tools, and any comment
> +so marked must be in kernel-doc format. The closing comment marker for
> +kernel-doc comments can be either ``*/`` or ``**/``, but ``*/`` is preferred in
> +the Linux kernel tree.
> +
> +.. hint::
> +
> + 1. Do not use ``/**`` to be begin a comment block unless the comment block
> + contains kernel-doc formatted comments.
> +
> + 2. We definitely need kernel-doc formatted documentation for functions that
> + are exported to loadable modules using EXPORT_SYMBOL.
> +
> + 3. Kernel-doc comments should be placed just before the function or data
> + structure being described.
> +
> +
> +Example kernel-doc function comment:
> +
> +.. code-block:: c
> +
> + /**
> + * foobar() - short function description of foobar
> + * @arg1: Describe the first argument to foobar.
> + * @arg2: Describe the second argument to foobar.
> + * One can provide multiple line descriptions
> + * for arguments.
> + *
> + * A longer description, with more discussion of the function foobar()
> + * that might be useful to those using or modifying it. Begins with
> + * empty comment line, and may include additional embedded empty
> + * comment lines.
> + *
> + * The longer description can have multiple paragraphs.
> + *
> + * Return: Describe the return value of foobar.
> + */
> +
> +The short description following the subject can span multiple lines and ends
> +with an ``@name`` description, an empty line or the end of the comment block.
> +The kernel-doc function comments describe each parameter to the function, in
> +order, with the ``@name`` lines. The ``@name`` descriptions must begin on the
> +very next line following this opening short function description line, with no
> +intervening empty comment lines. If a function parameter is ``...`` (varargs),
> +it should be listed in kernel-doc notation as::
> +
> + * @...: description
> +
> +The return value, if any, should be described in a dedicated section named
> +``Return``. Beside functions you can also write documentation for structs,
> +unions, enums and typedefs. Example kernel-doc data structure comment.::
> +
> + /**
> + * struct blah - the basic blah structure
> + * @mem1: describe the first member of struct blah
> + * @mem2: describe the second member of struct blah,
> + * perhaps with more lines and words.
> + *
> + * Longer description of this structure.
> + */
> +
> +The kernel-doc data structure comments describe each structure member in the
> +data structure, with the ``@name`` lines.
> diff --git a/Documentation/books/kernel-doc-HOWTO/kernel-doc-syntax.rst b/Documentation/books/kernel-doc-HOWTO/kernel-doc-syntax.rst
> new file mode 100644
> index 0000000..3037941
> --- /dev/null
> +++ b/Documentation/books/kernel-doc-HOWTO/kernel-doc-syntax.rst
> @@ -0,0 +1,171 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +.. include:: refs.txt
> +
> +.. _kernel-doc-syntax:
> +
> +==================
> +kernel-doc syntax
> +==================
> +
> +In the following examples,
> +
> +* ``(...)?`` signifies optional structure and
> +* ``(...)*`` signifies 0 or more structure elements
> +
> +The definition (or DOC) name is the section header. The section header names
> +must be unique per source file.
> +
> +.. _kernel-doc-syntax-functions:
> +
> +functions
> +=========
> +
> +The format of the block comment is like this::
> +
> + /**
> + * function_name(:)? (- short description)?
> + (* @parameterx: (description of parameter x)?)*
> + (* a blank line)?
> + * (Description:)? (Description of function)?
> + * (section header: (section description)? )*
> + (*)?*/
> +
> +All *description* text can span multiple lines, although the
> +``function_name`` & its short description are traditionally on a single line.
> +Description text may also contain blank lines (i.e., lines that contain only a
> +"*").
> +
> +.. ????
> +.. Avoid putting a spurious blank line after the function name, or else the
> +.. description will be repeated!
> +.. ????
> +
> +So, the trivial example would be:
> +
> +.. code-block:: c
> +
> + /**
> + * my_function
> + */
> +
> +If the Description: header tag is omitted, then there must be a blank line
> +after the last parameter specification.:
> +
> +.. code-block:: c
> +
> + /**
> + * my_function - does my stuff
> + * @my_arg: its mine damnit
> + *
> + * Does my stuff explained.
> + */
> +
> +or, could also use:
> +
> +.. code-block:: c
> +
> + /**
> + * my_function - does my stuff
> + * @my_arg: its mine damnit
> + * Description: Does my stuff explained.
> + */
> +
> +You can also add additional sections. When documenting kernel functions you
> +should document the ``Context:`` of the function, e.g. whether the functions can
> +be called form interrupts. Unlike other sections you can end it with an empty
> +line.
> +
> +A non-void function should have a ``Return:`` section describing the return
> +value(s). Example-sections should contain the string ``EXAMPLE`` so that
> +they are marked appropriately in the output format.
> +
> +.. kernel-doc:: ./all-in-a-tumble.h
> + :snippets: user_function
> + :language: c
> + :linenos:
> +
> +Rendered example: :ref:`example.user_function`
> +
> +.. _kernel-doc-syntax-misc-types:
> +
> +structs, unions, enums and typedefs
> +===================================
> +
> +Beside functions you can also write documentation for structs, unions, enums and
> +typedefs. Instead of the function name you must write the name of the
> +declaration; the ``struct``, ``union``, ``enum`` or ``typedef`` must always
> +precede the name. Nesting of declarations is not supported. Use the
> +``@argument`` mechanism to document members or constants.
> +
> +Inside a struct description, you can use the 'private:' and 'public:' comment
> +tags. Structure fields that are inside a 'private:' area are not listed in the
> +generated output documentation. The 'private:' and 'public:' tags must begin
> +immediately following a ``/*`` comment marker. They may optionally include
> +comments between the ``:`` and the ending ``*/`` marker.
> +
> +.. kernel-doc:: ./all-in-a-tumble.h
> + :snippets: my_struct
> + :language: c
> + :linenos:
> +
> +Rendered example: :ref:`example.my_struct`
> +
> +All descriptions can be multiline, except the short function description.
> +For really longs structs, you can also describe arguments inside the body of
> +the struct.
> +
> +.. kernel-doc:: ./all-in-a-tumble.h
> + :snippets: my_long_struct
> + :language: c
> + :linenos:
> +
> +Rendered example: :ref:`example.my_long_struct`
> +
> +This should be used only for struct and enum members.
> +
> +.. _kernel-doc-syntax-doc:
> +
> +ducumentation blocks
> +====================
> +
> +To facilitate having source code and comments close together, you can include
> +kernel-doc documentation blocks that are *free-form* comments instead of being
> +kernel-doc for functions, structures, unions, enums, or typedefs. This could be
> +used for something like a theory of operation for a driver or library code, for
> +example.
> +
> +This is done by using a ``DOC:`` section keyword with a section title. A small
> +example:
> +
> +.. kernel-doc:: ./all-in-a-tumble.h
> + :snippets: theory-of-operation
> + :language: c
> + :linenos:
> +
> +Rendered example: :ref:`example.theory-of-operation`
> +
> +.. _kernel-doc-syntax-snippets:
> +
> +Snippets
> +========
> +
> +The kernel-doc Parser supports a comment-markup for snippets out of the source
> +code. To start a region to snip insert::
> +
> + /* parse-SNIP: <snippet-name> */
> +
> +The snippet region stops with a new snippet region or at the next::
> +
> + /* parse-SNAP: */
> +
> +A small example:
> +
> +.. code-block:: c
> +
> + /* parse-SNIP: hello-world */
> + #include<stdio.h>
> + int main() {
> + printf("Hello World\n");
> + return 0;
> + }
> + /* parse-SNAP: */
> diff --git a/Documentation/books/kernel-doc-HOWTO/reST-kernel-doc-mode.rst b/Documentation/books/kernel-doc-HOWTO/reST-kernel-doc-mode.rst
> new file mode 100644
> index 0000000..5cfe59f
> --- /dev/null
> +++ b/Documentation/books/kernel-doc-HOWTO/reST-kernel-doc-mode.rst
> @@ -0,0 +1,120 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +.. include:: refs.txt
> +
> +.. _reST-kernel-doc-mode:
> +
> +====================
> +reST kernel-doc mode
> +====================
> +
> +To get in use of the fully reST_ add the following comment (e.g.) at the top of
> +your source code file (or at any line reST content starts).::
> +
> + /* parse-markup: reST */
> +
> +In reST mode the kernel-doc parser pass through all text / markups unchanged to
> +the reST toolchain including any whitespace. To toggle back to
> +:ref:`vintage-kernel-doc-mode` type the following line::
> +
> + /* parse-markup: kernel-doc */
> +
> +Within reST mode, most of the *vintage* kernel-doc markup -- as described in
> +:ref:`kernel-doc-syntax` -- stays unchanged, except the vintage-highlighting
> +markup and the treatment of whitespaces in description blocks. The *vintage
> +highlighting* markup is not supported in reST mode, because it conflicts with
> +the reST markup syntax.
> +
> +The reST syntax brings it's own markup to refer and highlight function,
> +structs or whatever definition e.g.:
> +
> +* functions ...
> +
> + .. code-block:: rst
> +
> + :c:func:`foo_func`
> +
> +* structs ...
> +
> + .. code-block:: rst
> +
> + :c:type:`stuct foo_struct`
> +
> +If you are familiar with the vintage style, first this might be a cons-change
> +for you, but take in account, that you get a expressive ASCII markup on the
> +pro-side.
> +
> +
> +reST section structure
> +======================
> +
> +Since a section title in reST mode needs a line break after the colon, the colon
> +handling is less ugly (:ref:`vintage-mode-quirks`). E.g.::
> +
> + prints out: hello world
> +
> +is rendered as expected in one line. If non text follows the colon, a section
> +is inserted. To avoid sectioning in any case, place a space in front of the column.::
> +
> + lorem list :
> +
> + * lorem
> + * ipsum
> +
> +On the opposite, super-short sections like::
> +
> + Return: sum of a and b
> +
> +are no longer supported, you have to enter at least one line break::
> +
> + Return:
> + sum of a and b
> +
> +Beside these *sectioning* of the kernel-doc syntax, reST has it's own chapter,
> +section etc. markup (e.g. see `Sections
> +<http://www.sphinx-doc.org/en/stable/rest.html#sections>`_). Normally, there are
> +no heading levels assigned to certain characters as the structure is determined
> +from the succession of headings. However, there is a common convention, which is
> +used by the kernel-doc parser also:
> +
> +* ``#`` with overline, for parts
> +* ``*`` with overline, for chapters
> +* ``=`` for sections
> +* ``-`` for subsections
> +* ``^`` for subsubsections
> +* ``"`` for paragraphs
> +
> +Within kernel-doc comments you should use this sectioning with care. A
> +kernel-doc section like the "Return" section above is translated into a reST
> +section with the following markup.
> +
> +.. code-block:: rst
> +
> + Return
> + ------
> +
> + sum of a and b
> +
> +As you see, a kernel-doc section is at reST *subsection* level. This means, you
> +can only use the following *sub-levels* within a kernel-doc section.
> +
> +* ``^`` for subsubsections
> +* ``"`` for paragraphs
> +
> +
> +further references
> +==================
> +
> +Here are some handy links about reST_ and the `Sphinx markup constructs`_:
> +
> +* reST_ primer, `reST (quickref)`_, `reST (spec)`_
> +* `Sphinx markup constructs`_
> +* `sphinx domains`_
> +* `sphinx cross refences`_
> +* `intersphinx`_, `sphinx.ext.intersphinx`_
> +* `sphinx-doc`_, `sphinx-doc FAQ`_
> +* `docutils`_, `docutils FAQ`_
> +
> +In absence of a more detailed C style guide for documentation, the `Python's
> +Style Guide for documentating
> +<https://docs.python.org/devguide/documenting.html#style-guide>`_ provides a
> +good orientation.
> diff --git a/Documentation/books/kernel-doc-HOWTO/refs.txt b/Documentation/books/kernel-doc-HOWTO/refs.txt
> new file mode 100644
> index 0000000..6d96765
> --- /dev/null
> +++ b/Documentation/books/kernel-doc-HOWTO/refs.txt
> @@ -0,0 +1,15 @@
> +.. _`reST (spec)`: http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html
> +.. _`reST (quickref)`: http://docutils.sourceforge.net/docs/user/rst/quickref.html
> +
> +.. _`reST`: http://www.sphinx-doc.org/en/stable/rest.html
> +.. _`Sphinx markup constructs`: http://www.sphinx-doc.org/en/stable/markup/index.html
> +.. _`sphinx-doc`: http://www.sphinx-doc.org/
> +.. _`sphinx-doc FAQ`: http://www.sphinx-doc.org/en/stable/faq.html
> +.. _`sphinx domains`: http://www.sphinx-doc.org/en/stable/domains.html
> +.. _`sphinx cross refences`: http://www.sphinx-doc.org/en/stable/markup/inline.html#cross-referencing-arbitrary-locations
> +.. _`sphinx.ext.intersphinx`: http://www.sphinx-doc.org/en/stable/ext/intersphinx.html#module-sphinx.ext.intersphinx
> +.. _`intersphinx`: http://www.sphinx-doc.org/en/stable/ext/intersphinx.html
> +.. _`sphinx config`: http://www.sphinx-doc.org/en/stable/config.html
> +
> +.. _`docutils`: http://docutils.sourceforge.net/docs/index.html
> +.. _`docutils FAQ`: http://docutils.sourceforge.net/FAQ.html
> diff --git a/Documentation/books/kernel-doc-HOWTO/table-markup.rst b/Documentation/books/kernel-doc-HOWTO/table-markup.rst
> new file mode 100644
> index 0000000..38a6de0
> --- /dev/null
> +++ b/Documentation/books/kernel-doc-HOWTO/table-markup.rst
> @@ -0,0 +1,499 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +
> +.. _`Emacs Table Mode`: https://www.emacswiki.org/emacs/TableMode
> +.. _`Online Tables Generator`: http://www.tablesgenerator.com/text_tables
> +.. _`OASIS XML Exchange Table Model`: https://www.oasis-open.org/specs/tm9901.html
> +
> +
> +.. _xref_table_concerns:
> +
> +============
> +About tables
> +============
> +
> +First: the rest-flat-table_ directive is preferred in the Linux kernel tree.
> +
> +Internaly, the docutils uses a representation according to the `OASIS XML
> +Exchange Table Model`_ (same as DocBook). The *OASIS Table Model* gives a huge
> +bandwith of possibilities to form tables. This often seduce authors to force a
> +specific layout. Misuse of tables in this manner is not recommended, because it
> +breaks the separation of *presentation from content* which most often ends in
> +problems in range of output formats. Tables (and preformated text like source
> +code listings) should be used advisedly. In a HTML output, the horizontal and
> +vertical expansion is handled by a scrollbar. On print medias (paper / pdf)
> +there is no scrollbar, automaticaly (page-) breaking a table (or line-breaking a
> +preformated text) in the layout process ends mostly in a unwanted results.
in unwanted results.
> +
> +.. hint::
> +
> + Tables and preformated text in itself violate the separation of *presentation
> + from content*, but we will never be able to entirely renounce them. Use them
> + with care, if your content should be rendered well, in wide variation of
> + output formats.
> +
> +
> +ASCII-art tables
> +================
> +
> +ASCII-art tables might be comfortable for readers of the text-files, but they
> +have huge disadvantages in the creation and modifying. First, they are hard to
> +edit. Think about adding a row or a column to a ASCII-art table or adding a
> +paraggraph in a cell, it is a nightmare on big tables. Second the diff of
> +modifing ASCII-art tables is not meaningfull, e.g. widening a cell generates a
> +diff in which also changes are included, which are only ascribable to the
> +ASCII-art (see also :ref:`list-table-directives`).
> +
> +* `Emacs Table Mode`_
> +* `Online Tables Generator`_
> +
> +
> +Simple tables
> +-------------
> +
> +simple tables allow *colspan* but not *rowspan*:
> +
> +.. code-block:: none
> +
> + ====== ====== ======
> + Inputs Output
> + ------------- ------
> + A B A or B
> + ====== ====== ======
> + False
> + --------------------
> + True
> + --------------------
> + True False True
> + ------ ------ ------
> + False True
> + ====== =============
> +
> +Rendered as:
> +
> +====== ====== ======
> + Inputs Output
> +------------- ------
> +A B A or B
> +====== ====== ======
> +False
> +--------------------
> +True
> +--------------------
> +True False True
> +------ ------ ------
> +False True
> +====== =============
> +
> +
> +Grid tables
> +-----------
> +
> +grid tables allow colspan *colspan* and *rowspan*:
> +
> +.. code-block:: rst
> +
> + +------------+------------+-----------+
> + | Header 1 | Header 2 | Header 3 |
> + +============+============+===========+
> + | body row 1 | column 2 | column 3 |
> + +------------+------------+-----------+
> + | body row 2 | Cells may span columns.|
> + +------------+------------+-----------+
> + | body row 3 | Cells may | - Cells |
> + +------------+ span rows. | - contain |
> + | body row 4 | | - blocks. |
> + +------------+------------+-----------+
> +
> +Rendered as:
> +
> ++------------+------------+-----------+
> +| Header 1 | Header 2 | Header 3 |
> ++============+============+===========+
> +| body row 1 | column 2 | column 3 |
> ++------------+------------+-----------+
> +| body row 2 | Cells may span columns.|
> ++------------+------------+-----------+
> +| body row 3 | Cells may | - Cells |
> ++------------+ span rows. | - contain |
> +| body row 4 | | - blocks. |
> ++------------+------------+-----------+
> +
> +.. _list-table-directives:
> +
> +List table directives
> +=====================
> +
> +The *list table* formats are double stage list, compared to the ASCII-art they
> +migth not be as comfortable for readers of the text-files. Their advantage is,
> +that they are easy to create/modify and that the diff of a modification is much
> +more meaningfull, because it is limited to the modified content.
> +
> +.. _rest-list-table:
> +
> +list-table
> +----------
> +
> +The ``list-tables`` has no ability to *colspan* nor *rowspan*:
> +
> +.. code-block:: rst
> +
> + .. list-table:: table title
> + :header-rows: 1
> + :stub-columns: 1
> +
> + * - ..
> + - head col 1
> + - head col 2
> +
> + * - stub col row 1
> + - column
> + - column
> +
> + * - stub col row 2
> + - column
> + - column
> +
> + * - stub col row 3
> + - column
> + - column
> +
> +
> +Rendered as:
> +
> +.. list-table:: table title
> + :header-rows: 1
> + :stub-columns: 1
> +
> + * - ..
> + - head col 1
> + - head col 2
> +
> + * - stub col row 1
> + - column
> + - column
> +
> + * - stub col row 2
> + - column
> + - column
> +
> + * - stub col row 3
> + - column
> + - column
> +
> +.. _rest-flat-table:
> +
> +flat-table
> +----------
> +
> +The ``flat-table`` (:py:class:`FlatTable`) is a double-stage list similar
> +to the ``list-table`` with some additional features:
> +
> +* *column-span*: with the role ``cspan`` a cell can be extended through
> + additional columns
> +
> +* *row-span*: with the role ``rspan`` a cell can be extended through
> + additional rows
> +
> +* *auto span* rightmost cell of a table row over the missing cells on the right
> + side of that table-row. With Option ``:fill-cells:`` this behavior can
> + changed from *auto span* to *auto fill*, which automaticly inserts (empty)
> + cells instead of spanning the last cell.
> +
> +options:
> +
> +:header-rows: [int] count of header rows
> +:stub-columns: [int] count of stub columns
> +:widths: [[int] [int] ... ] widths of columns
> +:fill-cells: instead of autospann missing cells, insert missing cells
> +
> +roles:
> +
> +:cspan: [int] additionale columns (*morecols*)
> +:rspan: [int] additionale rows (*morerows*)
> +
> +The example below shows how to use this markup. The first level of the staged
> +list is the *table-row*. In the *table-row* there is only one markup allowed,
> +the list of the cells in this *table-row*. Exception are *comments* ( ``..`` )
> +and *targets* (e.g. a ref to :ref:`row 2 of table's body <row body 2>`).
> +
> +.. code-block:: rst
> +
> + .. flat-table:: table title
> + :header-rows: 2
> + :stub-columns: 1
> + :widths: 1 1 1 1 2
> +
> + * - :rspan:`1` head / stub
> + - :cspan:`3` head 1.1-4
> +
> + * - head 2.1
> + - head 2.2
> + - head 2.3
> + - head 2.4
> +
> + * .. row body 1 / this is a comment
> +
> + - row 1
> + - :rspan:`2` cell 1-3.1
> + - cell 1.2
> + - cell 1.3
> + - cell 1.4
> +
> + * .. Comments and targets are allowed on *table-row* stage.
> + .. _`row body 2`:
> +
> + - row 2
> + - cell 2.2
> + - :rspan:`1` :cspan:`1`
> + cell 2.3 with a span over
> +
> + * col 3-4 &
> + * row 2-3
> +
> + * - row 3
> + - cell 3.2
> +
> + * - row 4
> + - cell 4.1
> + - cell 4.2
> + - cell 4.3
> + - cell 4.4
> +
> + * - row 5
> + - cell 5.1 with automatic span to rigth end
> +
> + * - row 6
> + - cell 6.1
> + - ..
> +
> +
> +Rendered as:
> +
> + .. flat-table:: table title
> + :header-rows: 2
> + :stub-columns: 1
> + :widths: 1 1 1 1 2
> +
> + * - :rspan:`1` head / stub
> + - :cspan:`3` head 1.1-4
> +
> + * - head 2.1
> + - head 2.2
> + - head 2.3
> + - head 2.4
> +
> + * .. row body 1 / this is a comment
> +
> + - row 1
> + - :rspan:`2` cell 1-3.1
> + - cell 1.2
> + - cell 1.3
> + - cell 1.4
> +
> + * .. Comments and targets are allowed on *table-row* stage.
> + .. _`row body 2`:
> +
> + - row 2
> + - cell 2.2
> + - :rspan:`1` :cspan:`1`
> + cell 2.3 with a span over
> +
> + * col 3-4 &
> + * row 2-3
> +
> + * - row 3
> + - cell 3.2
> +
> + * - row 4
> + - cell 4.1
> + - cell 4.2
> + - cell 4.3
> + - cell 4.4
> +
> + * - row 5
> + - cell 5.1 with automatic span to rigth end
> +
> + * - row 6
> + - cell 6.1
> + - ..
> +
> +
> +CSV table
> +=========
> +
> +CSV table might be the choice if you want to include CSV-data from a outstanding
> +(build) process into your documentation.
> +
> +.. code-block:: rst
> +
> + .. csv-table:: table title
> + :header: , Header1, Header2
> + :widths: 15, 10, 30
> + :stub-columns: 1
> + :file: csv_table.txt
> +
> +Content of file ``csv_table.txt``:
> +
> +.. literalinclude:: csv_table.txt
> +
> +Rendered as:
> +
> +.. csv-table:: table title
> + :header: , Header1, Header2
> + :widths: 15, 10, 30
> + :stub-columns: 1
> + :file: csv_table.txt
> +
> +
> +Nested Tables
> +=============
> +
> +Nested tables are ugly, don't use them! This part here is only to show what you
> +should never do. They are ugly because they cause huge problems in many output
> +formats and there is always no need for nested tables.
> +
> +.. code-block:: rst
> +
> + +-----------+----------------------------------------------------+
> + | W/NW cell | N/NE cell |
> + | +-------------+--------------------------+-----------+
> + | | W/NW center | N/NE center | E/SE cell |
> + | | +------------+-------------+ |
> + | | | +--------+ | E/SE center | |
> + | | | | nested | | | |
> + | | | +--------+ | | |
> + | | | | table | | | |
> + | | | +--------+ | | |
> + | +-------------+------------+ | |
> + | | S/SE center | | |
> + +-----------+--------------------------+-------------+ |
> + | S/SW cell | |
> + +----------------------------------------------------+-----------+
> +
> +Rendered as: Not supported by all sphinx-builders, don't use nested tables!!!
> +
> +
> +raw HTML tables
> +===============
> +
> +If HTML is the only format you want to render, you could use a raw-import of a
> +HTML table markup. But be aware, this breaks the separation of *presentation from
> +content*. HTML-Tables are only rendered within a HTML output.
> +
> +.. code-block:: html
> +
> + <div class="wy-table-responsive">
> + <table class="docutils">
> + <thead>
> + <tr style="font-weight: bold;">
> + <td>Owner Module/Drivers</td>
> + <td>Group</td>
> + <td>Property Name</td>
> + <td>Type</td>
> + <td>Property Values</td>
> + <td>Object attached</td>
> + <td>Description/Restrictions</td>
> + </tr>
> + </thead>
> + <tbody>
> + <tr>
> + <td rowspan="4">DRM</td>
> + <td>Generic</td>
> + <td>"rotation"</td>
> + <td>BITMASK</td>
> + <td>{ 0, "rotate-0" }, { 1, "rotate-90" }, { 2, "rotate-180" }, { 3,
> + "rotate-270" }, { 4, "reflect-x" }, { 5, "reflect-y" }</td>
> + <td>CRTC, Plane</td>
> + <td>rotate-(degrees) rotates the image by the specified amount in
> + degrees in counter clockwise direction. reflect-x and reflect-y
> + reflects the image along the specified axis prior to rotation</td>
> + </tr>
> +
> + <tr>
> + <td rowspan="3">Connector</td>
> + <td>"EDID"</td>
> + <td>BLOB | IMMUTABLE</td>
> + <td>0</td>
> + <td>Connector</td>
> + <td>Contains id of edid blob ptr object.</td>
> + </tr>
> +
> + <tr>
> + <td>"DPMS"</td>
> + <td>ENUM</td>
> + <td>{ "On", "Standby", "Suspend", "Off" }</td>
> + <td>Connector</td>
> + <td>Contains DPMS operation mode value.</td>
> + </tr>
> +
> + <tr>
> + <td>"PATH"</td>
> + <td>BLOB | IMMUTABLE</td>
> + <td>0</td>
> + <td>Connector</td>
> + <td>Contains topology path to a connector.</td>
> + </tr>
> + </tbody>
> + </table>
> + </div>
> +
> +
> +
> +.. raw:: html
> +
> + <div class="wy-table-responsive">
> + <table class="docutils">
> + <thead>
> + <tr style="font-weight: bold;">
> + <td>Owner Module/Drivers</td>
> + <td>Group</td>
> + <td>Property Name</td>
> + <td>Type</td>
> + <td>Property Values</td>
> + <td>Object attached</td>
> + <td>Description/Restrictions</td>
> + </tr>
> + </thead>
> + <tbody>
> + <tr>
> + <td rowspan="4">DRM</td>
> + <td>Generic</td>
> + <td>"rotation"</td>
> + <td>BITMASK</td>
> + <td>{ 0, "rotate-0" }, { 1, "rotate-90" }, { 2, "rotate-180" }, { 3,
> + "rotate-270" }, { 4, "reflect-x" }, { 5, "reflect-y" }</td>
> + <td>CRTC, Plane</td>
> + <td>rotate-(degrees) rotates the image by the specified amount in
> + degrees in counter clockwise direction. reflect-x and reflect-y
> + reflects the image along the specified axis prior to rotation</td>
> + </tr>
> +
> + <tr>
> + <td rowspan="3">Connector</td>
> + <td>"EDID"</td>
> + <td>BLOB | IMMUTABLE</td>
> + <td>0</td>
> + <td>Connector</td>
> + <td>Contains id of edid blob ptr object.</td>
> + </tr>
> +
> + <tr>
> + <td>"DPMS"</td>
> + <td>ENUM</td>
> + <td>{ "On", "Standby", "Suspend", "Off" }</td>
> + <td>Connector</td>
> + <td>Contains DPMS operation mode value.</td>
> + </tr>
> +
> + <tr>
> + <td>"PATH"</td>
> + <td>BLOB | IMMUTABLE</td>
> + <td>0</td>
> + <td>Connector</td>
> + <td>Contains topology path to a connector.</td>
> + </tr>
> + </tbody>
> + </table>
> + </div>
> +
> +.. include:: refs.txt
> diff --git a/Documentation/books/kernel-doc-HOWTO/test/parser_test.h b/Documentation/books/kernel-doc-HOWTO/test/parser_test.h
> new file mode 100644
> index 0000000..455b74c
> --- /dev/null
> +++ b/Documentation/books/kernel-doc-HOWTO/test/parser_test.h
> @@ -0,0 +1,332 @@
> +/* parse-markup: kernel-doc */
> +
> +/**
> + * struct v4l2_subdev_core_ops - Define core ops callbacks for subdevs
> + *
> + * @log_status: callback for VIDIOC_LOG_STATUS ioctl handler code.
> + *
> + * @s_io_pin_config: configure one or more chip I/O pins for chips that
> + * multiplex different internal signal pads out to IO pins. This function
> + * takes a pointer to an array of 'n' pin configuration entries, one for
> + * each pin being configured. This function could be called at times
> + * other than just subdevice initialization.
> + *
> + * @init: initialize the sensor registers to some sort of reasonable default
> + * values. Do not use for new drivers and should be removed in existing
> + * drivers.
> + *
> + * @load_fw: load firmware.
> + *
> + * @reset: generic reset command. The argument selects which subsystems to
> + * reset. Passing 0 will always reset the whole chip. Do not use for new
> + * drivers without discussing this first on the linux-media mailinglist.
> + * There should be no reason normally to reset a device.
> + *
> + * @s_gpio: set GPIO pins. Very simple right now, might need to be extended with
> + * a direction argument if needed.
> + *
> + * @queryctrl: callback for VIDIOC_QUERYCTL ioctl handler code.
> + *
> + * @g_ctrl: callback for VIDIOC_G_CTRL ioctl handler code.
> + *
> + * @s_ctrl: callback for VIDIOC_S_CTRL ioctl handler code.
> + *
> + * @g_ext_ctrls: callback for VIDIOC_G_EXT_CTRLS ioctl handler code.
> + *
> + * @s_ext_ctrls: callback for VIDIOC_S_EXT_CTRLS ioctl handler code.
> + *
> + * @try_ext_ctrls: callback for VIDIOC_TRY_EXT_CTRLS ioctl handler code.
> + *
> + * @querymenu: callback for VIDIOC_QUERYMENU ioctl handler code.
> + *
> + * @ioctl: called at the end of ioctl() syscall handler at the V4L2 core.
> + * used to provide support for private ioctls used on the driver.
> + *
> + * @compat_ioctl32: called when a 32 bits application uses a 64 bits Kernel,
> + * in order to fix data passed from/to userspace.
> + *
> + * @g_register: callback for VIDIOC_G_REGISTER ioctl handler code.
> + *
> + * @s_register: callback for VIDIOC_G_REGISTER ioctl handler code.
> + *
> + * @s_power: puts subdevice in power saving mode (on == 0) or normal operation
> + * mode (on == 1).
> + *
> + * @interrupt_service_routine: Called by the bridge chip's interrupt service
> + * handler, when an interrupt status has be raised due to this subdev,
> + * so that this subdev can handle the details. It may schedule work to be
> + * performed later. It must not sleep. *Called from an IRQ context*.
> + *
> + * @subscribe_event: used by the drivers to request the control framework that
> + * for it to be warned when the value of a control changes.
> + *
> + * @unsubscribe_event: remove event subscription from the control framework.
> + *
> + * @registered_async: the subdevice has been registered async.
> + *
> + * This is a copy&paste of a more complexe struct, just for testing.
complex
> + */
> +struct v4l2_subdev_core_ops {
> + int (*log_status)(struct v4l2_subdev *sd);
> + int (*s_io_pin_config)(struct v4l2_subdev *sd, size_t n,
> + struct v4l2_subdev_io_pin_config *pincfg);
> + int (*init)(struct v4l2_subdev *sd, u32 val);
> + int (*load_fw)(struct v4l2_subdev *sd);
> + int (*reset)(struct v4l2_subdev *sd, u32 val);
> + int (*s_gpio)(struct v4l2_subdev *sd, u32 val);
> + int (*queryctrl)(struct v4l2_subdev *sd, struct v4l2_queryctrl *qc);
> + int (*g_ctrl)(struct v4l2_subdev *sd, struct v4l2_control *ctrl);
> + int (*s_ctrl)(struct v4l2_subdev *sd, struct v4l2_control *ctrl);
> + int (*g_ext_ctrls)(struct v4l2_subdev *sd, struct v4l2_ext_controls *ctrls);
> + int (*s_ext_ctrls)(struct v4l2_subdev *sd, struct v4l2_ext_controls *ctrls);
> + int (*try_ext_ctrls)(struct v4l2_subdev *sd, struct v4l2_ext_controls *ctrls);
> + int (*querymenu)(struct v4l2_subdev *sd, struct v4l2_querymenu *qm);
> + long (*ioctl)(struct v4l2_subdev *sd, unsigned int cmd, void *arg);
> +#ifdef CONFIG_COMPAT
> + long (*compat_ioctl32)(struct v4l2_subdev *sd, unsigned int cmd,
> + unsigned long arg);
> +#endif
> +#ifdef CONFIG_VIDEO_ADV_DEBUG
> + int (*g_register)(struct v4l2_subdev *sd, struct v4l2_dbg_register *reg);
> + int (*s_register)(struct v4l2_subdev *sd, const struct v4l2_dbg_register *reg);
> +#endif
> + int (*s_power)(struct v4l2_subdev *sd, int on);
> + int (*interrupt_service_routine)(struct v4l2_subdev *sd,
> + u32 status, bool *handled);
> + int (*subscribe_event)(struct v4l2_subdev *sd, struct v4l2_fh *fh,
> + struct v4l2_event_subscription *sub);
> + int (*unsubscribe_event)(struct v4l2_subdev *sd, struct v4l2_fh *fh,
> + struct v4l2_event_subscription *sub);
> + int (*registered_async)(struct v4l2_subdev *sd);
> +};
> +
> +/**
> + * DOC: Media Controller
> + *
> + * The media controller userspace API is documented in DocBook format in
> + * Documentation/DocBook/media/v4l/media-controller.xml. This document focus
> + * on the kernel-side implementation of the media framework.
> + *
> + * * Abstract media device model:
> + *
> + * Discovering a device internal topology, and configuring it at runtime, is one
> + * of the goals of the media framework. To achieve this, hardware devices are
> + * modelled as an oriented graph of building blocks called entities connected
> + * through pads.
> + *
> + * An entity is a basic media hardware building block. It can correspond to
> + * a large variety of logical blocks such as physical hardware devices
> + * (CMOS sensor for instance), logical hardware devices (a building block
> + * in a System-on-Chip image processing pipeline), DMA channels or physical
> + * connectors.
> + *
> + * A pad is a connection endpoint through which an entity can interact with
> + * other entities. Data (not restricted to video) produced by an entity
> + * flows from the entity's output to one or more entity inputs. Pads should
> + * not be confused with physical pins at chip boundaries.
> + *
> + * A link is a point-to-point oriented connection between two pads, either
> + * on the same entity or on different entities. Data flows from a source
> + * pad to a sink pad.
> + *
> + *
> + * * Media device:
> + *
> + * A media device is represented by a struct &media_device instance, defined in
> + * include/media/media-device.h. Allocation of the structure is handled by the
> + * media device driver, usually by embedding the &media_device instance in a
> + * larger driver-specific structure.
> + *
> + * Drivers register media device instances by calling
> + * __media_device_register() via the macro media_device_register()
> + * and unregistered by calling
> + * media_device_unregister().
> + *
> + * * Entities, pads and links:
> + *
> + * - Entities
> + *
> + * Entities are represented by a struct &media_entity instance, defined in
> + * include/media/media-entity.h. The structure is usually embedded into a
> + * higher-level structure, such as a v4l2_subdev or video_device instance,
> + * although drivers can allocate entities directly.
> + *
> + * Drivers initialize entity pads by calling
> + * media_entity_pads_init().
> + *
> + * Drivers register entities with a media device by calling
> + * media_device_register_entity()
> + * and unregistred by calling
> + * media_device_unregister_entity().
> + *
> + * - Interfaces
> + *
> + * Interfaces are represented by a struct &media_interface instance, defined in
> + * include/media/media-entity.h. Currently, only one type of interface is
> + * defined: a device node. Such interfaces are represented by a struct
> + * &media_intf_devnode.
> + *
> + * Drivers initialize and create device node interfaces by calling
> + * media_devnode_create()
> + * and remove them by calling:
> + * media_devnode_remove().
> + *
> + * - Pads
> + *
> + * Pads are represented by a struct &media_pad instance, defined in
> + * include/media/media-entity.h. Each entity stores its pads in a pads array
> + * managed by the entity driver. Drivers usually embed the array in a
> + * driver-specific structure.
> + *
> + * Pads are identified by their entity and their 0-based index in the pads
> + * array.
> + * Both information are stored in the &media_pad structure, making the
> + * &media_pad pointer the canonical way to store and pass link references.
> + *
> + * Pads have flags that describe the pad capabilities and state.
> + *
> + * %MEDIA_PAD_FL_SINK indicates that the pad supports sinking data.
> + * %MEDIA_PAD_FL_SOURCE indicates that the pad supports sourcing data.
> + *
> + * NOTE: One and only one of %MEDIA_PAD_FL_SINK and %MEDIA_PAD_FL_SOURCE must
> + * be set for each pad.
> + *
> + * - Links
> + *
> + * Links are represented by a struct &media_link instance, defined in
> + * include/media/media-entity.h. There are two types of links:
> + *
> + * 1. pad to pad links:
> + *
> + * Associate two entities via their PADs. Each entity has a list that points
> + * to all links originating at or targeting any of its pads.
> + * A given link is thus stored twice, once in the source entity and once in
> + * the target entity.
> + *
> + * Drivers create pad to pad links by calling:
> + * media_create_pad_link() and remove with media_entity_remove_links().
> + *
> + * 2. interface to entity links:
> + *
> + * Associate one interface to a Link.
> + *
> + * Drivers create interface to entity links by calling:
> + * media_create_intf_link() and remove with media_remove_intf_links().
> + *
> + * NOTE:
> + *
> + * Links can only be created after having both ends already created.
> + *
> + * Links have flags that describe the link capabilities and state. The
> + * valid values are described at media_create_pad_link() and
> + * media_create_intf_link().
> + *
> + * Graph traversal:
> + *
> + * The media framework provides APIs to iterate over entities in a graph.
> + *
> + * To iterate over all entities belonging to a media device, drivers can use
> + * the media_device_for_each_entity macro, defined in
> + * include/media/media-device.h.
> + *
> + * struct media_entity *entity;
> + *
> + * media_device_for_each_entity(entity, mdev) {
> + * // entity will point to each entity in turn
> + * ...
> + * }
> + *
> + * Drivers might also need to iterate over all entities in a graph that can be
> + * reached only through enabled links starting at a given entity. The media
> + * framework provides a depth-first graph traversal API for that purpose.
> + *
> + * Note that graphs with cycles (whether directed or undirected) are *NOT*
> + * supported by the graph traversal API. To prevent infinite loops, the graph
> + * traversal code limits the maximum depth to MEDIA_ENTITY_ENUM_MAX_DEPTH,
> + * currently defined as 16.
> + *
> + * Drivers initiate a graph traversal by calling
> + * media_entity_graph_walk_start()
> + *
> + * The graph structure, provided by the caller, is initialized to start graph
> + * traversal at the given entity.
> + *
> + * Drivers can then retrieve the next entity by calling
> + * media_entity_graph_walk_next()
> + *
> + * When the graph traversal is complete the function will return NULL.
> + *
> + * Graph traversal can be interrupted at any moment. No cleanup function call
> + * is required and the graph structure can be freed normally.
> + *
> + * Helper functions can be used to find a link between two given pads, or a pad
> + * connected to another pad through an enabled link
> + * media_entity_find_link() and media_entity_remote_pad()
> + *
> + * Use count and power handling:
> + *
> + * Due to the wide differences between drivers regarding power management
> + * needs, the media controller does not implement power management. However,
> + * the &media_entity structure includes a use_count field that media drivers
> + * can use to track the number of users of every entity for power management
> + * needs.
> + *
> + * The &media_entity.@use_count field is owned by media drivers and must not be
> + * touched by entity drivers. Access to the field must be protected by the
> + * &media_device.@graph_mutex lock.
> + *
> + * Links setup:
> + *
> + * Link properties can be modified at runtime by calling
> + * media_entity_setup_link()
> + *
> + * Pipelines and media streams:
> + *
> + * When starting streaming, drivers must notify all entities in the pipeline to
> + * prevent link states from being modified during streaming by calling
> + * media_entity_pipeline_start().
> + *
> + * The function will mark all entities connected to the given entity through
> + * enabled links, either directly or indirectly, as streaming.
> + *
> + * The &media_pipeline instance pointed to by the pipe argument will be stored
> + * in every entity in the pipeline. Drivers should embed the &media_pipeline
> + * structure in higher-level pipeline structures and can then access the
> + * pipeline through the &media_entity pipe field.
> + *
> + * Calls to media_entity_pipeline_start() can be nested. The pipeline pointer
> + * must be identical for all nested calls to the function.
> + *
> + * media_entity_pipeline_start() may return an error. In that case, it will
> + * clean up any of the changes it did by itself.
> + *
> + * When stopping the stream, drivers must notify the entities with
> + * media_entity_pipeline_stop().
> + *
> + * If multiple calls to media_entity_pipeline_start() have been made the same
> + * number of media_entity_pipeline_stop() calls are required to stop streaming.
> + * The &media_entity pipe field is reset to NULL on the last nested stop call.
> + *
> + * Link configuration will fail with -%EBUSY by default if either end of the
> + * link is a streaming entity. Links that can be modified while streaming must
> + * be marked with the %MEDIA_LNK_FL_DYNAMIC flag.
> + *
> + * If other operations need to be disallowed on streaming entities (such as
> + * changing entities configuration parameters) drivers can explicitly check the
> + * media_entity stream_count field to find out if an entity is streaming. This
> + * operation must be done with the media_device graph_mutex held.
> + *
> + * Link validation:
> + *
> + * Link validation is performed by media_entity_pipeline_start() for any
> + * entity which has sink pads in the pipeline. The
> + * &media_entity.@link_validate() callback is used for that purpose. In
> + * @link_validate() callback, entity driver should check that the properties of
> + * the source pad of the connected entity and its own sink pad match. It is up
> + * to the type of the entity (and in the end, the properties of the hardware)
> + * what matching actually means.
> + *
> + * Subsystems should facilitate link validation by providing subsystem specific
> + * helper functions to provide easy access for commonly needed information, and
> + * in the end provide a way to use driver-specific callbacks.
> + */
> diff --git a/Documentation/books/kernel-doc-HOWTO/vintage-kernel-doc-mode.rst b/Documentation/books/kernel-doc-HOWTO/vintage-kernel-doc-mode.rst
> new file mode 100644
> index 0000000..a49bc25
> --- /dev/null
> +++ b/Documentation/books/kernel-doc-HOWTO/vintage-kernel-doc-mode.rst
> @@ -0,0 +1,100 @@
> +.. -*- coding: utf-8; mode: rst -*-
> +.. include:: refs.txt
> +
> +.. _vintage-kernel-doc-mode:
> +
> +=======================
> +Vintage kernel-doc mode
> +=======================
> +
> +All kernel-doc markup is processed as described in :ref:`kernel-doc-syntax`, all
> +descriptive text is further processed, scanning for the following special
> +patterns, which are highlighted appropriately.
> +
> +* ``funcname()`` - function
> +* ``$ENVVAR`` - environmental variable
> +* ``&struct name`` - name of a structure (up to two words including ``struct``)
> +* ``@parameter`` - name of a parameter
> +* ``%CONST`` - name of a constant.
> +
> +These highlighted patterns are not used when you are using the reST addition
> +(:ref:`reST-kernel-doc-mode`). This is, because reST brings it's own markup to
> +refer and highlight function, structs or whatever definition.
> +
> +Within the *vintage* kernel-doc mode the kernel-doc parser highlights the pattern
> +above, but he also dogged ignores any whitespace formatting/markup.
what is "dogged"??
> +
> +.. hint::
> +
> + Formatting with whitespaces is substantial for ASCII markups. By this, it's
> + recommended to use the :ref:`reST-kernel-doc-mode` on any new or changed
> + comment.
> +
> +
> +.. _vintage-mode-quirks:
> +
> +vintage mode quirks
> +===================
> +
> +In the following, you will find some quirks of the *vintage* kernel-doc mode.
> +
> +* Since a colon introduce a new section, you can't use colons. E.g. a comment
> + line like::
> +
> + prints out: hello world
> +
> + will result in a section with the title "prints out" and a paragraph with only
> + "hello world" in, this is mostly not what you expect. To avoid sectioning,
> + place a space in front of the column::
> +
> + prints out : hello world
> +
> +* The multi-line descriptive text you provide does *not* recognize
> + line breaks, so if you try to format some text nicely, as in::
> +
> + Return:
> + 0 - cool
> + 1 - invalid arg
> + 2 - out of memory
> +
> + this will all run together and produce::
> +
> + Return: 0 - cool 1 - invalid arg 2 - out of memory
> +
> +* If the descriptive text you provide has lines that begin with some phrase
> + followed by a colon, each of those phrases will be taken as a new section
> + heading, which means you should similarly try to avoid text like::
> +
> + Return:
> + 0: cool
> + 1: invalid arg
> + 2: out of memory
> +
> + every line of which would start a new section. Again, probably not what you
> + were after.
> +
> +Determined by the historical development of the kernel-doc comments, the
> +*vintage* kernel-doc comments contain characters like "*" or strings with
> +e.g. leading/trailing underscore ("_"), which are inline markups in reST. Here a
> +short example from a *vintage* comment::
> +
> + <SNIP> -----
> + * In contrast to the other drm_get_*_name functions this one here returns a
> + * const pointer and hence is threadsafe.
> + <SNAP> -----
> +
> +Within reST markup (the new bas format), the wildcard in the string
what is "bas"??
> +``drm_get_*_name`` has to be masked: ``drm_get_\\*_name``. Some more examples
> +from reST markup:
> +
> +* Emphasis "*": like ``*emphasis*`` or ``**emphasis strong**``
> +* Leading "_" : is a *anchor* in reST markup (``_foo``).
> +* Trailing "_: is a reference in reST markup (``foo_``).
> +* interpreted text: "`"
> +* inline literals: "``"
> +* substitution references: "|"
> +
> +As long as you in the *vintage* kernel-doc mode, these special strings will be
> +masked in the reST output and can't be used as *plain-text markup*.
> +
> +
>
My only "requirement" (if I may have one) is that we not introduce big
hurdles to kernel developers adding documentation.
I'm not saying that this is a big hurdle.. I haven't looked at it enough
yet.
--
~Randy
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 3/7] kernel-doc-HOWTO: add kernel-doc specification
2016-06-09 18:05 ` Randy Dunlap
@ 2016-06-09 18:42 ` Jani Nikula
2016-06-10 16:00 ` Markus Heiser
1 sibling, 0 replies; 23+ messages in thread
From: Jani Nikula @ 2016-06-09 18:42 UTC (permalink / raw)
To: Randy Dunlap, Markus Heiser, corbet
Cc: daniel.vetter, grant.likely, mchehab, keithp, linux-kernel,
linux-doc, hverkuil
On Thu, 09 Jun 2016, Randy Dunlap <rdunlap@infradead.org> wrote:
> My only "requirement" (if I may have one) is that we not introduce big
> hurdles to kernel developers adding documentation.
>
> I'm not saying that this is a big hurdle.. I haven't looked at it enough
> yet.
I suggest you look at the series starting at [1] instead. We're looking
to *remove* hurdles in the process.
BR,
Jani.
[1] http://mid.gmane.org/cover.1465031816.git.jani.nikula@intel.com
--
Jani Nikula, Intel Open Source Technology Center
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 0/7] add reST/sphinx-doc to linux documentation
2016-06-08 19:49 ` Jonathan Corbet
@ 2016-06-10 15:25 ` Markus Heiser
2016-06-10 17:19 ` Jani Nikula
0 siblings, 1 reply; 23+ messages in thread
From: Markus Heiser @ 2016-06-10 15:25 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Daniel Vetter, Jani Nikula, Grant Likely, Mauro Carvalho Chehab,
Keith Packard, Linux Kernel Mailing List, linux-doc,
Hans Verkuil
Am 08.06.2016 um 21:49 schrieb Jonathan Corbet <corbet@lwn.net>:
> So I've finally gotten a chance to make another pass over this stuff.
>
> Markus, your enthusiasm is great; I'm hoping you'll do great things
> helping us to improve the kernel's documentation toolchain.
With 7 years DocBook and 8 years reST experience this is my
opportunity to give linux something back ;-)
> But please,
> at this point, let's build on Jani's work and go from there. Things have
> waited for long enough while we've gone around on this; I think what we
> have is a good starting point.
I'am willing to contribute, but take my POV: I have finished all
including migration of **all** DocBook to reST, so why should I
throw it all away?
Pull it from:
https://github.com/return42/linux.git linux-doc-reST
The kernel-doc HOWTO [1], the Template Book [2] and
make books-help
are your friends. You will see that all requirements to get
productive are well done. Within the next days I will
add more features, which has been requested on the ML.
[1] https://return42.github.io/sphkerneldoc/books/kernel-doc-HOWTO
[2] http://return42.github.io/sphkerneldoc/books/template-book
>
> On the specifics, Daniel already covered most of it pretty well.
>
> On Tue, 7 Jun 2016 09:54:21 +0200
> Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
>> I think next steps would be:
>> - rebase flat-table onto Jani's work and relicense under gplv2
>
> This I would really like to see.
is already done.
>
>> - look into rewriting kernel-doc in python more as a long-term project
>
> There is nobody who would like to dump the Perl kernel-doc more than I
> would; it wasn't pretty to begin with and hasn't improved over the years.
> I, too, had thought about redoing it, but I, too, concluded that it wasn't
> the highest of priorities.
>
> Please do keep this around, we may want it before too long. I have some
> sympathy for Daniel's suggestion to look into using LLVM; we could also
> maybe stay a little closer to our roots and use the sparse library. But
> there might also be value in a Python version that doesn't add more
> dependencies to the docs toolchain. We need to think about this, but I
> don't think we need to answer it now.
nevertheless which kind of implementation is used, the parsers are all
exchangeable. There is only the user interface which has to be stable
and this is the ".. kernel-doc:" directive.
I implemented a python version of the kernel-doc parser with an (python)
API, so why should we fiddle with perl and pipes when implementing
a ".. kernel-doc:" directive?
>
>> - start converting docs instead - I really want to start reaping
>> benefits of all this work as soon as possible.
>
> Absolutely.
pull above, there are all converted DocBooks are in.
>
> Along these lines, I don't currently have a strong opinion on the
> big-files vs. little-files question. I *do*, however, like the idea of
> trying to create one coherent kernel document rather than perpetuation our
> current collection of independent book silos. Initially it will certainly
> look like the LDP-based books that people used to duct-tape together back
> in the 90's, but it should improve over time.
Placing all DocBooks in one huge sphinx-project does not scales well
and brings to additional dependencies. To solve this problem
I use intersphinx and a index-page where all books are referred.
Read chapter "Getting started with reST" from [1].
--Markus--
>
> Thanks,
>
> jon
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 3/7] kernel-doc-HOWTO: add kernel-doc specification
2016-06-09 18:05 ` Randy Dunlap
2016-06-09 18:42 ` Jani Nikula
@ 2016-06-10 16:00 ` Markus Heiser
2016-06-10 16:04 ` Randy Dunlap
1 sibling, 1 reply; 23+ messages in thread
From: Markus Heiser @ 2016-06-10 16:00 UTC (permalink / raw)
To: Randy Dunlap
Cc: corbet, jani.nikula, daniel.vetter, grant.likely, mchehab,
keithp, linux-kernel, linux-doc, hverkuil
Hi Randy,
thanks for your amendments / has been fixed [1]
[1] https://github.com/return42/linux/commit/98a9fc42cbd0c23b266ac28494dafe20d7920d05
Am 09.06.2016 um 20:05 schrieb Randy Dunlap <rdunlap@infradead.org>:
> Hi,
>
> Some spellos and a few questions...
>> + b/Documentation/books/kernel-doc-HOWTO/vintage-kernel-doc-mode.rst
>> +Within the *vintage* kernel-doc mode the kernel-doc parser highlights the pattern
>> +above, but he also dogged ignores any whitespace formatting/markup.
>
> what is "dogged"??
>
... "insistently" ...
>> +Within reST markup (the new bas format), the wildcard in the string
>
> what is "bas"??
>
sorry, another typo: "the new base format"
>
>> +``drm_get_*_name`` has to be masked: ``drm_get_\\*_name``. Some more examples
>> +from reST markup:
>> +
>> +* Emphasis "*": like ``*emphasis*`` or ``**emphasis strong**``
>> +* Leading "_" : is a *anchor* in reST markup (``_foo``).
>> +* Trailing "_: is a reference in reST markup (``foo_``).
>> +* interpreted text: "`"
>> +* inline literals: "``"
>> +* substitution references: "|"
>> +
>> +As long as you in the *vintage* kernel-doc mode, these special strings will be
>> +masked in the reST output and can't be used as *plain-text markup*.
>> +
>> +
>>
>
> My only "requirement" (if I may have one) is that we not introduce big
> hurdles to kernel developers adding documentation.
>
> I'm not saying that this is a big hurdle.. I haven't looked at it enough
> yet.
The parser has two modes, *vintage* "kernel-doc" and "reST". You can
switch between these modes in the source code with this two comments:
/* parse-markup: reST */
/* parse-markup: kernel-doc */
Jani says that we should prefer reST as the default mode for parsing, I
recommended kernel-doc as default, because all old source are written
with the *vintage* markup .. the odd side of this is, that you have to put
a /* parse-markup: reST */ at the top of your source file to get reST
in use ... I don't know what the best choice might bee ... I think, it
is the best to make an option in the conf.py for this.
-- Markus--
>
> --
> ~Randy
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 3/7] kernel-doc-HOWTO: add kernel-doc specification
2016-06-10 16:00 ` Markus Heiser
@ 2016-06-10 16:04 ` Randy Dunlap
0 siblings, 0 replies; 23+ messages in thread
From: Randy Dunlap @ 2016-06-10 16:04 UTC (permalink / raw)
To: Markus Heiser
Cc: corbet, jani.nikula, daniel.vetter, grant.likely, mchehab,
keithp, linux-kernel, linux-doc, hverkuil
On 06/10/16 09:00, Markus Heiser wrote:
> Hi Randy,
>
> thanks for your amendments / has been fixed [1]
>
> [1] https://github.com/return42/linux/commit/98a9fc42cbd0c23b266ac28494dafe20d7920d05
>
> Am 09.06.2016 um 20:05 schrieb Randy Dunlap <rdunlap@infradead.org>:
>
>> Hi,
>>
>> Some spellos and a few questions...
>
>>> + b/Documentation/books/kernel-doc-HOWTO/vintage-kernel-doc-mode.rst
>>> +Within the *vintage* kernel-doc mode the kernel-doc parser highlights the pattern
>>> +above, but he also dogged ignores any whitespace formatting/markup.
>>
>> what is "dogged"??
>>
>
> ... "insistently" ...
ah, that would be "doggedly" then.
>>> +Within reST markup (the new bas format), the wildcard in the string
>>
>> what is "bas"??
>>
>
> sorry, another typo: "the new base format"
>
>>
>>> +``drm_get_*_name`` has to be masked: ``drm_get_\\*_name``. Some more examples
>>> +from reST markup:
>>> +
>>> +* Emphasis "*": like ``*emphasis*`` or ``**emphasis strong**``
>>> +* Leading "_" : is a *anchor* in reST markup (``_foo``).
>>> +* Trailing "_: is a reference in reST markup (``foo_``).
>>> +* interpreted text: "`"
>>> +* inline literals: "``"
>>> +* substitution references: "|"
>>> +
>>> +As long as you in the *vintage* kernel-doc mode, these special strings will be
>>> +masked in the reST output and can't be used as *plain-text markup*.
>>> +
>>> +
>>>
>>
>> My only "requirement" (if I may have one) is that we not introduce big
>> hurdles to kernel developers adding documentation.
>>
>> I'm not saying that this is a big hurdle.. I haven't looked at it enough
>> yet.
>
> The parser has two modes, *vintage* "kernel-doc" and "reST". You can
> switch between these modes in the source code with this two comments:
>
> /* parse-markup: reST */
> /* parse-markup: kernel-doc */
>
> Jani says that we should prefer reST as the default mode for parsing, I
> recommended kernel-doc as default, because all old source are written
> with the *vintage* markup .. the odd side of this is, that you have to put
> a /* parse-markup: reST */ at the top of your source file to get reST
> in use ... I don't know what the best choice might bee ... I think, it
> is the best to make an option in the conf.py for this.
Thanks for that info.
--
~Randy
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 0/7] add reST/sphinx-doc to linux documentation
2016-06-10 15:25 ` Markus Heiser
@ 2016-06-10 17:19 ` Jani Nikula
0 siblings, 0 replies; 23+ messages in thread
From: Jani Nikula @ 2016-06-10 17:19 UTC (permalink / raw)
To: Markus Heiser, Jonathan Corbet
Cc: Daniel Vetter, Grant Likely, Mauro Carvalho Chehab,
Keith Packard, Linux Kernel Mailing List, linux-doc,
Hans Verkuil, Randy Dunlap
On Fri, 10 Jun 2016, Markus Heiser <markus.heiser@darmarit.de> wrote:
> Am 08.06.2016 um 21:49 schrieb Jonathan Corbet <corbet@lwn.net>:
>> But please,
>> at this point, let's build on Jani's work and go from there. Things have
>> waited for long enough while we've gone around on this; I think what we
>> have is a good starting point.
>
> I'am willing to contribute, but take my POV: I have finished all
> including migration of **all** DocBook to reST, so why should I
> throw it all away?
Nobody is asking you to throw it away; we're asking you to rebase that
work, with some modifications, on top of docs-next, which now has my
work merged.
> are your friends. You will see that all requirements to get
> productive are well done. Within the next days I will
> add more features, which has been requested on the ML.
OTOH, if you do keep working on a baseline that no longer applies to
docs-next, you will be wasting your efforts.
> nevertheless which kind of implementation is used, the parsers are all
> exchangeable. There is only the user interface which has to be stable
> and this is the ".. kernel-doc:" directive.
>
> I implemented a python version of the kernel-doc parser with an (python)
> API, so why should we fiddle with perl and pipes when implementing
> a ".. kernel-doc:" directive?
No matter how hacky the perl script seems, it has been used on the
kernel sources for nearly two decades. Sure, it has accumulated more
than a little cruft along the way, but also a lot of corner cases and
gotchas have been ironed out. I wouldn't dream of replacing that until
we've migrated a bunch of documents using it, and can confirm the
replacement produces the same output.
Also, there may still be users for the perl script outside of the Sphinx
pipeline. I'd like to make sure we don't end up having to maintain *two*
homebrew scripted C parsers before adding a new one.
>> Along these lines, I don't currently have a strong opinion on the
>> big-files vs. little-files question. I *do*, however, like the idea of
>> trying to create one coherent kernel document rather than perpetuation our
>> current collection of independent book silos. Initially it will certainly
>> look like the LDP-based books that people used to duct-tape together back
>> in the 90's, but it should improve over time.
>
> Placing all DocBooks in one huge sphinx-project does not scales well
> and brings to additional dependencies. To solve this problem
> I use intersphinx and a index-page where all books are referred.
On the modifications, I'll repeat that I strongly object to maintaining
that arbitrary distinction between "books" and other files. We had that
with DocBook, and we'd be more than happy to get rid of it.
I object to adding a separate Sphinx project and duplicating a
configuration file per document. I object to splitting the documents too
much into small bits and pieces, for the reasons I explained in my
earlier email [1].
If someone wants to have separate projects with dedicated configuration
for special needs, they can add them. But I maintain that most documents
do not have that need, and everything is simpler without.
[1] http://mid.gmane.org/87wpm15q7q.fsf@intel.com
> Read chapter "Getting started with reST" from [1].
> [1] https://return42.github.io/sphkerneldoc/books/kernel-doc-HOWTO
How about:
1) Write a file in rst, save it somewhere under Documentation.
2) Reference the file in Documentation/index.rst.
3) Be happy.
I'll hand it to you that you have more experience in Sphinx than I do,
but I like to think I know my fellow kernel developers better. The above
is what they want to hear. They don't want a single hurdle more.
BR,
Jani.
--
Jani Nikula, Intel Open Source Technology Center
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 0/7] add reST/sphinx-doc to linux documentation
2016-06-07 7:54 ` [PATCH 0/7] add reST/sphinx-doc to linux documentation Daniel Vetter
` (2 preceding siblings ...)
2016-06-08 19:49 ` Jonathan Corbet
@ 2016-06-15 13:01 ` Markus Heiser
3 siblings, 0 replies; 23+ messages in thread
From: Markus Heiser @ 2016-06-15 13:01 UTC (permalink / raw)
To: Daniel Vetter
Cc: Jonathan Corbet, Jani Nikula, Grant Likely,
Mauro Carvalho Chehab, Keith Packard, Linux Kernel Mailing List,
linux-doc, Hans Verkuil
FYI
Am 07.06.2016 um 09:54 schrieb Daniel Vetter <daniel.vetter@ffwll.ch>:
> On Mon, Jun 6, 2016 at 6:32 PM, Markus Heiser <markus.heiser@darmarit.de> wrote:
>> From: "Heiser, Markus" <markus.heiser@darmarIT.de>
>>
> I'm still not sold on the vintage-kerneldoc idea. At least in my
> experience (after first converting to asciidoc and now to sphinx) this
> is a non-issue. Yes, there's the oddball misrendering, but that's no
> worse than the oddball typo.
Since *vintage* and reST mode is an option of the ".. kernel-doc:"
directive it is compareabel. E.g 80211.html produce markup errors:
* 230 in "reST" mode
* 6 in *vintage* "kernel-doc" mode
95% of the errors caused by:
Emphasis “*”: like *emphasis* or **emphasis strong**
Leading “_” : is a anchor in reST markup (_foo).
Trailing “_: is a reference in reST markup (foo_).
interpreted text: “`”
inline literals: “``”
substitution references: “|”
which qouted when the parser runs in vintage "kernel-doc" mode.
--Markus--
^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2016-06-15 13:01 UTC | newest]
Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-06-06 16:32 [PATCH 0/7] add reST/sphinx-doc to linux documentation Markus Heiser
2016-06-06 16:32 ` [PATCH 1/7] python: add scripts/site-packages Markus Heiser
2016-06-06 16:32 ` [PATCH 2/7] sphinx-doc: add basic sphinx-build infrastructure Markus Heiser
2016-06-06 16:32 ` [PATCH 3/7] kernel-doc-HOWTO: add kernel-doc specification Markus Heiser
2016-06-09 18:05 ` Randy Dunlap
2016-06-09 18:42 ` Jani Nikula
2016-06-10 16:00 ` Markus Heiser
2016-06-10 16:04 ` Randy Dunlap
2016-06-06 16:32 ` [PATCH 4/7] linuxdoc: add python package linuxdoc Markus Heiser
2016-06-06 16:32 ` [PATCH 5/7] kernel-doc parser: inital python implementation Markus Heiser
2016-06-06 16:32 ` [PATCH 6/7] kernel-doc directive: initial implementation Markus Heiser
2016-06-06 16:32 ` [PATCH 7/7] flat-table " Markus Heiser
2016-06-07 7:54 ` [PATCH 0/7] add reST/sphinx-doc to linux documentation Daniel Vetter
2016-06-07 8:59 ` Jani Nikula
2016-06-07 9:36 ` Markus Heiser
2016-06-07 11:09 ` Jani Nikula
2016-06-07 15:13 ` Markus Heiser
2016-06-07 9:14 ` Markus Heiser
2016-06-08 19:49 ` Jonathan Corbet
2016-06-10 15:25 ` Markus Heiser
2016-06-10 17:19 ` Jani Nikula
2016-06-15 13:01 ` Markus Heiser
2016-06-07 13:47 ` Markus Heiser
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).