* [PATCH 0/5] Better markup for GPU DocBook
@ 2015-11-25 17:07 Daniel Vetter
2015-11-25 17:07 ` Daniel Vetter
` (4 more replies)
0 siblings, 5 replies; 23+ messages in thread
From: Daniel Vetter @ 2015-11-25 17:07 UTC (permalink / raw)
To: DRI Development; +Cc: Daniel Vetter, Intel Graphics Development
Hi all,
As you might know the markup improvements have been discussed at kernel summit:
https://lwn.net/Articles/662930/
Unfortunately it died in a bikeshed fest due to an alliance of people who think
docs are useless and you should just read code and others who didn't know how to
build the kerneldoc into something pretty. Oh well.
But I still want this, and Dave Airlie is ok with using it for drm kerneldoc as
long as I maintain the support. Plan is to merge the first patch to adjust
gpu.tmpl into drm properly, and keep everything else in topic/kerneldoc at
git://anongit.freedesktop.org/drm-intel topic/kerneldoc
If you want to build pretty docs just install asciidoc and base your drm
documentation patches on top of drm-intel-nightly. That tree also includes all
of Dave's tree. Alternatively pull in the above topic branch.
Note that asciidoc is strictly optional, but without it the docbook will look a
bit bad since beyond paragraphs there won't be any additional formatting (like
tables, quoting for sourcecode snippets or lists).
Intel also has an autobuilder that pushes latest drm-intel-nightly docs to
http://dri.freedesktop.org/docs/drm/
Cheers, Daniel
Daniel Vetter (2):
scripts/kernel-doc: Use asciidoc instead of markdown
drm: Enable markdown^Wasciidoc for gpu.tmpl
Danilo Cesar Lemes de Paula (3):
drm/doc: Convert to markdown
scripts/kernel-doc: Adding infrastructure for markdown support
scripts/kernel-doc: Improve Markdown results
Documentation/DocBook/Makefile | 25 +++++---
Documentation/DocBook/gpu.tmpl | 86 --------------------------
drivers/gpu/drm/drm_modes.c | 12 ++--
drivers/gpu/drm/drm_modeset_lock.c | 14 ++---
drivers/gpu/drm/drm_prime.c | 16 ++---
drivers/gpu/drm/i915/i915_reg.h | 48 +++++++--------
scripts/docproc.c | 54 ++++++++++++-----
scripts/kernel-doc | 120 ++++++++++++++++++++++++++++++++-----
8 files changed, 203 insertions(+), 172 deletions(-)
--
2.5.1
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 23+ messages in thread
* [PATCH 1/5] drm/doc: Convert to markdown
2015-11-25 17:07 [PATCH 0/5] Better markup for GPU DocBook Daniel Vetter
@ 2015-11-25 17:07 ` Daniel Vetter
2015-11-25 17:07 ` Daniel Vetter
` (3 subsequent siblings)
4 siblings, 0 replies; 23+ messages in thread
From: Daniel Vetter @ 2015-11-25 17:07 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Danilo Cesar Lemes de Paula,
Randy Dunlap, Daniel Vetter, Laurent Pinchart, Jonathan Corbet,
Herbert Xu, Stephan Mueller, Michal Marek, linux-kernel,
linux-doc, Daniel Vetter
From: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
DRM Docbook is now Markdown ready. This means its doc is able to
use markdown text on it.
* Documentation/DocBook/drm.tmpl: Contains a table duplicated from
drivers/gpu/drm/i915/i915_reg.h. This is not needed anymore
* drivers/gpu/drm/drm_modeset_lock.c: had a code example that used
to look pretty bad on html. Fixed by using proper code markup.
* drivers/gpu/drm/drm_prime.c: Remove spaces between lines to make
a proper markup list.
* drivers/gpu/drm/i915/i915_reg.h: Altought pandoc supports tables,
it doesn't support table cell spanning. But we can use fixed-width
for those special cases.
* include/drm/drm_vma_manager.h: Another code example that should be
proper indented with four spaces.
v2 (Daniel): Adjust name to gpu.xml due to rename.
v3 (Daniel):
Split out the actual enabling in the Makefile - this way we can merge
the conversion, while just keeping the enabling in a drm-private tree.
Signed-off-by: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk> (v1)
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Stephan Mueller <smueller@chronox.de>
Cc: Michal Marek <mmarek@suse.cz>
Cc: linux-kernel@vger.kernel.org
Cc: linux-doc@vger.kernel.org
Cc: intel-gfx <intel-gfx@lists.freedesktop.org>
Cc: dri-devel <dri-devel@lists.freedesktop.org>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
Documentation/DocBook/gpu.tmpl | 86 --------------------------------------
drivers/gpu/drm/drm_modes.c | 12 +++---
drivers/gpu/drm/drm_modeset_lock.c | 14 +++----
drivers/gpu/drm/drm_prime.c | 16 +++----
drivers/gpu/drm/i915/i915_reg.h | 48 ++++++++++-----------
5 files changed, 42 insertions(+), 134 deletions(-)
diff --git a/Documentation/DocBook/gpu.tmpl b/Documentation/DocBook/gpu.tmpl
index 201dcd3c2e9d..c05d7df3067d 100644
--- a/Documentation/DocBook/gpu.tmpl
+++ b/Documentation/DocBook/gpu.tmpl
@@ -4039,92 +4039,6 @@ int num_ioctls;</synopsis>
<sect2>
<title>DPIO</title>
!Pdrivers/gpu/drm/i915/i915_reg.h DPIO
- <table id="dpiox2">
- <title>Dual channel PHY (VLV/CHV/BXT)</title>
- <tgroup cols="8">
- <colspec colname="c0" />
- <colspec colname="c1" />
- <colspec colname="c2" />
- <colspec colname="c3" />
- <colspec colname="c4" />
- <colspec colname="c5" />
- <colspec colname="c6" />
- <colspec colname="c7" />
- <spanspec spanname="ch0" namest="c0" nameend="c3" />
- <spanspec spanname="ch1" namest="c4" nameend="c7" />
- <spanspec spanname="ch0pcs01" namest="c0" nameend="c1" />
- <spanspec spanname="ch0pcs23" namest="c2" nameend="c3" />
- <spanspec spanname="ch1pcs01" namest="c4" nameend="c5" />
- <spanspec spanname="ch1pcs23" namest="c6" nameend="c7" />
- <thead>
- <row>
- <entry spanname="ch0">CH0</entry>
- <entry spanname="ch1">CH1</entry>
- </row>
- </thead>
- <tbody valign="top" align="center">
- <row>
- <entry spanname="ch0">CMN/PLL/REF</entry>
- <entry spanname="ch1">CMN/PLL/REF</entry>
- </row>
- <row>
- <entry spanname="ch0pcs01">PCS01</entry>
- <entry spanname="ch0pcs23">PCS23</entry>
- <entry spanname="ch1pcs01">PCS01</entry>
- <entry spanname="ch1pcs23">PCS23</entry>
- </row>
- <row>
- <entry>TX0</entry>
- <entry>TX1</entry>
- <entry>TX2</entry>
- <entry>TX3</entry>
- <entry>TX0</entry>
- <entry>TX1</entry>
- <entry>TX2</entry>
- <entry>TX3</entry>
- </row>
- <row>
- <entry spanname="ch0">DDI0</entry>
- <entry spanname="ch1">DDI1</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
- <table id="dpiox1">
- <title>Single channel PHY (CHV/BXT)</title>
- <tgroup cols="4">
- <colspec colname="c0" />
- <colspec colname="c1" />
- <colspec colname="c2" />
- <colspec colname="c3" />
- <spanspec spanname="ch0" namest="c0" nameend="c3" />
- <spanspec spanname="ch0pcs01" namest="c0" nameend="c1" />
- <spanspec spanname="ch0pcs23" namest="c2" nameend="c3" />
- <thead>
- <row>
- <entry spanname="ch0">CH0</entry>
- </row>
- </thead>
- <tbody valign="top" align="center">
- <row>
- <entry spanname="ch0">CMN/PLL/REF</entry>
- </row>
- <row>
- <entry spanname="ch0pcs01">PCS01</entry>
- <entry spanname="ch0pcs23">PCS23</entry>
- </row>
- <row>
- <entry>TX0</entry>
- <entry>TX1</entry>
- <entry>TX2</entry>
- <entry>TX3</entry>
- </row>
- <row>
- <entry spanname="ch0">DDI2</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
</sect2>
<sect2>
diff --git a/drivers/gpu/drm/drm_modes.c b/drivers/gpu/drm/drm_modes.c
index cd74a0953f42..9480464434cf 100644
--- a/drivers/gpu/drm/drm_modes.c
+++ b/drivers/gpu/drm/drm_modes.c
@@ -553,10 +553,10 @@ EXPORT_SYMBOL(drm_gtf_mode_complex);
* drivers/video/fbmon.c
*
* Standard GTF parameters:
- * M = 600
- * C = 40
- * K = 128
- * J = 20
+ * M = 600
+ * C = 40
+ * K = 128
+ * J = 20
*
* Returns:
* The modeline based on the GTF algorithm stored in a drm_display_mode object.
@@ -1212,7 +1212,7 @@ EXPORT_SYMBOL(drm_mode_connector_list_update);
* This uses the same parameters as the fb modedb.c, except for an extra
* force-enable, force-enable-digital and force-disable bit at the end:
*
- * <xres>x<yres>[M][R][-<bpp>][@<refresh>][i][m][eDd]
+ * <xres>x<yres>[M][R][-<bpp>][@<refresh>][i][m][eDd]
*
* The intermediate drm_cmdline_mode structure is required to store additional
* options from the command line modline like the force-enable/disable flag.
@@ -1491,4 +1491,4 @@ int drm_mode_convert_umode(struct drm_display_mode *out,
out:
return ret;
-}
\ No newline at end of file
+}
diff --git a/drivers/gpu/drm/drm_modeset_lock.c b/drivers/gpu/drm/drm_modeset_lock.c
index 6675b1428410..7c9ca2381d78 100644
--- a/drivers/gpu/drm/drm_modeset_lock.c
+++ b/drivers/gpu/drm/drm_modeset_lock.c
@@ -40,17 +40,15 @@
* The basic usage pattern is to:
*
* drm_modeset_acquire_init(&ctx)
- * retry:
+ * retry:
* foreach (lock in random_ordered_set_of_locks) {
- * ret = drm_modeset_lock(lock, &ctx)
- * if (ret == -EDEADLK) {
- * drm_modeset_backoff(&ctx);
- * goto retry;
- * }
+ * ret = drm_modeset_lock(lock, &ctx)
+ * if (ret == -EDEADLK) {
+ * drm_modeset_backoff(&ctx);
+ * goto retry;
+ * }
* }
- *
* ... do stuff ...
- *
* drm_modeset_drop_locks(&ctx);
* drm_modeset_acquire_fini(&ctx);
*/
diff --git a/drivers/gpu/drm/drm_prime.c b/drivers/gpu/drm/drm_prime.c
index 9f935f55d74c..27aa7183b20b 100644
--- a/drivers/gpu/drm/drm_prime.c
+++ b/drivers/gpu/drm/drm_prime.c
@@ -313,19 +313,15 @@ static const struct dma_buf_ops drm_gem_prime_dmabuf_ops = {
*
* Export callbacks:
*
- * - @gem_prime_pin (optional): prepare a GEM object for exporting
- *
- * - @gem_prime_get_sg_table: provide a scatter/gather table of pinned pages
- *
- * - @gem_prime_vmap: vmap a buffer exported by your driver
- *
- * - @gem_prime_vunmap: vunmap a buffer exported by your driver
- *
- * - @gem_prime_mmap (optional): mmap a buffer exported by your driver
+ * * @gem_prime_pin (optional): prepare a GEM object for exporting
+ * * @gem_prime_get_sg_table: provide a scatter/gather table of pinned pages
+ * * @gem_prime_vmap: vmap a buffer exported by your driver
+ * * @gem_prime_vunmap: vunmap a buffer exported by your driver
+ * * @gem_prime_mmap (optional): mmap a buffer exported by your driver
*
* Import callback:
*
- * - @gem_prime_import_sg_table (import): produce a GEM object from another
+ * * @gem_prime_import_sg_table (import): produce a GEM object from another
* driver's scatter/gather table
*/
diff --git a/drivers/gpu/drm/i915/i915_reg.h b/drivers/gpu/drm/i915/i915_reg.h
index bc7b8faba84d..9ebf03230ac0 100644
--- a/drivers/gpu/drm/i915/i915_reg.h
+++ b/drivers/gpu/drm/i915/i915_reg.h
@@ -804,31 +804,31 @@ enum skl_disp_power_wells {
*
* Note: DDI0 is digital port B, DD1 is digital port C, and DDI2 is
* digital port D (CHV) or port A (BXT).
- */
-/*
- * Dual channel PHY (VLV/CHV/BXT)
- * ---------------------------------
- * | CH0 | CH1 |
- * | CMN/PLL/REF | CMN/PLL/REF |
- * |---------------|---------------| Display PHY
- * | PCS01 | PCS23 | PCS01 | PCS23 |
- * |-------|-------|-------|-------|
- * |TX0|TX1|TX2|TX3|TX0|TX1|TX2|TX3|
- * ---------------------------------
- * | DDI0 | DDI1 | DP/HDMI ports
- * ---------------------------------
*
- * Single channel PHY (CHV/BXT)
- * -----------------
- * | CH0 |
- * | CMN/PLL/REF |
- * |---------------| Display PHY
- * | PCS01 | PCS23 |
- * |-------|-------|
- * |TX0|TX1|TX2|TX3|
- * -----------------
- * | DDI2 | DP/HDMI port
- * -----------------
+ *
+ * Dual channel PHY (VLV/CHV/BXT)
+ * ---------------------------------
+ * | CH0 | CH1 |
+ * | CMN/PLL/REF | CMN/PLL/REF |
+ * |---------------|---------------| Display PHY
+ * | PCS01 | PCS23 | PCS01 | PCS23 |
+ * |-------|-------|-------|-------|
+ * |TX0|TX1|TX2|TX3|TX0|TX1|TX2|TX3|
+ * ---------------------------------
+ * | DDI0 | DDI1 | DP/HDMI ports
+ * ---------------------------------
+ *
+ * Single channel PHY (CHV/BXT)
+ * -----------------
+ * | CH0 |
+ * | CMN/PLL/REF |
+ * |---------------| Display PHY
+ * | PCS01 | PCS23 |
+ * |-------|-------|
+ * |TX0|TX1|TX2|TX3|
+ * -----------------
+ * | DDI2 | DP/HDMI port
+ * -----------------
*/
#define DPIO_DEVFN 0
--
2.5.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 1/5] drm/doc: Convert to markdown
@ 2015-11-25 17:07 ` Daniel Vetter
0 siblings, 0 replies; 23+ messages in thread
From: Daniel Vetter @ 2015-11-25 17:07 UTC (permalink / raw)
To: DRI Development
Cc: Michal Marek, Herbert Xu, Danilo Cesar Lemes de Paula,
Jonathan Corbet, Stephan Mueller, Daniel Vetter,
Intel Graphics Development, Randy Dunlap, linux-doc,
linux-kernel, Laurent Pinchart, Daniel Vetter
From: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
DRM Docbook is now Markdown ready. This means its doc is able to
use markdown text on it.
* Documentation/DocBook/drm.tmpl: Contains a table duplicated from
drivers/gpu/drm/i915/i915_reg.h. This is not needed anymore
* drivers/gpu/drm/drm_modeset_lock.c: had a code example that used
to look pretty bad on html. Fixed by using proper code markup.
* drivers/gpu/drm/drm_prime.c: Remove spaces between lines to make
a proper markup list.
* drivers/gpu/drm/i915/i915_reg.h: Altought pandoc supports tables,
it doesn't support table cell spanning. But we can use fixed-width
for those special cases.
* include/drm/drm_vma_manager.h: Another code example that should be
proper indented with four spaces.
v2 (Daniel): Adjust name to gpu.xml due to rename.
v3 (Daniel):
Split out the actual enabling in the Makefile - this way we can merge
the conversion, while just keeping the enabling in a drm-private tree.
Signed-off-by: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk> (v1)
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Stephan Mueller <smueller@chronox.de>
Cc: Michal Marek <mmarek@suse.cz>
Cc: linux-kernel@vger.kernel.org
Cc: linux-doc@vger.kernel.org
Cc: intel-gfx <intel-gfx@lists.freedesktop.org>
Cc: dri-devel <dri-devel@lists.freedesktop.org>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
Documentation/DocBook/gpu.tmpl | 86 --------------------------------------
drivers/gpu/drm/drm_modes.c | 12 +++---
drivers/gpu/drm/drm_modeset_lock.c | 14 +++----
drivers/gpu/drm/drm_prime.c | 16 +++----
drivers/gpu/drm/i915/i915_reg.h | 48 ++++++++++-----------
5 files changed, 42 insertions(+), 134 deletions(-)
diff --git a/Documentation/DocBook/gpu.tmpl b/Documentation/DocBook/gpu.tmpl
index 201dcd3c2e9d..c05d7df3067d 100644
--- a/Documentation/DocBook/gpu.tmpl
+++ b/Documentation/DocBook/gpu.tmpl
@@ -4039,92 +4039,6 @@ int num_ioctls;</synopsis>
<sect2>
<title>DPIO</title>
!Pdrivers/gpu/drm/i915/i915_reg.h DPIO
- <table id="dpiox2">
- <title>Dual channel PHY (VLV/CHV/BXT)</title>
- <tgroup cols="8">
- <colspec colname="c0" />
- <colspec colname="c1" />
- <colspec colname="c2" />
- <colspec colname="c3" />
- <colspec colname="c4" />
- <colspec colname="c5" />
- <colspec colname="c6" />
- <colspec colname="c7" />
- <spanspec spanname="ch0" namest="c0" nameend="c3" />
- <spanspec spanname="ch1" namest="c4" nameend="c7" />
- <spanspec spanname="ch0pcs01" namest="c0" nameend="c1" />
- <spanspec spanname="ch0pcs23" namest="c2" nameend="c3" />
- <spanspec spanname="ch1pcs01" namest="c4" nameend="c5" />
- <spanspec spanname="ch1pcs23" namest="c6" nameend="c7" />
- <thead>
- <row>
- <entry spanname="ch0">CH0</entry>
- <entry spanname="ch1">CH1</entry>
- </row>
- </thead>
- <tbody valign="top" align="center">
- <row>
- <entry spanname="ch0">CMN/PLL/REF</entry>
- <entry spanname="ch1">CMN/PLL/REF</entry>
- </row>
- <row>
- <entry spanname="ch0pcs01">PCS01</entry>
- <entry spanname="ch0pcs23">PCS23</entry>
- <entry spanname="ch1pcs01">PCS01</entry>
- <entry spanname="ch1pcs23">PCS23</entry>
- </row>
- <row>
- <entry>TX0</entry>
- <entry>TX1</entry>
- <entry>TX2</entry>
- <entry>TX3</entry>
- <entry>TX0</entry>
- <entry>TX1</entry>
- <entry>TX2</entry>
- <entry>TX3</entry>
- </row>
- <row>
- <entry spanname="ch0">DDI0</entry>
- <entry spanname="ch1">DDI1</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
- <table id="dpiox1">
- <title>Single channel PHY (CHV/BXT)</title>
- <tgroup cols="4">
- <colspec colname="c0" />
- <colspec colname="c1" />
- <colspec colname="c2" />
- <colspec colname="c3" />
- <spanspec spanname="ch0" namest="c0" nameend="c3" />
- <spanspec spanname="ch0pcs01" namest="c0" nameend="c1" />
- <spanspec spanname="ch0pcs23" namest="c2" nameend="c3" />
- <thead>
- <row>
- <entry spanname="ch0">CH0</entry>
- </row>
- </thead>
- <tbody valign="top" align="center">
- <row>
- <entry spanname="ch0">CMN/PLL/REF</entry>
- </row>
- <row>
- <entry spanname="ch0pcs01">PCS01</entry>
- <entry spanname="ch0pcs23">PCS23</entry>
- </row>
- <row>
- <entry>TX0</entry>
- <entry>TX1</entry>
- <entry>TX2</entry>
- <entry>TX3</entry>
- </row>
- <row>
- <entry spanname="ch0">DDI2</entry>
- </row>
- </tbody>
- </tgroup>
- </table>
</sect2>
<sect2>
diff --git a/drivers/gpu/drm/drm_modes.c b/drivers/gpu/drm/drm_modes.c
index cd74a0953f42..9480464434cf 100644
--- a/drivers/gpu/drm/drm_modes.c
+++ b/drivers/gpu/drm/drm_modes.c
@@ -553,10 +553,10 @@ EXPORT_SYMBOL(drm_gtf_mode_complex);
* drivers/video/fbmon.c
*
* Standard GTF parameters:
- * M = 600
- * C = 40
- * K = 128
- * J = 20
+ * M = 600
+ * C = 40
+ * K = 128
+ * J = 20
*
* Returns:
* The modeline based on the GTF algorithm stored in a drm_display_mode object.
@@ -1212,7 +1212,7 @@ EXPORT_SYMBOL(drm_mode_connector_list_update);
* This uses the same parameters as the fb modedb.c, except for an extra
* force-enable, force-enable-digital and force-disable bit at the end:
*
- * <xres>x<yres>[M][R][-<bpp>][@<refresh>][i][m][eDd]
+ * <xres>x<yres>[M][R][-<bpp>][@<refresh>][i][m][eDd]
*
* The intermediate drm_cmdline_mode structure is required to store additional
* options from the command line modline like the force-enable/disable flag.
@@ -1491,4 +1491,4 @@ int drm_mode_convert_umode(struct drm_display_mode *out,
out:
return ret;
-}
\ No newline at end of file
+}
diff --git a/drivers/gpu/drm/drm_modeset_lock.c b/drivers/gpu/drm/drm_modeset_lock.c
index 6675b1428410..7c9ca2381d78 100644
--- a/drivers/gpu/drm/drm_modeset_lock.c
+++ b/drivers/gpu/drm/drm_modeset_lock.c
@@ -40,17 +40,15 @@
* The basic usage pattern is to:
*
* drm_modeset_acquire_init(&ctx)
- * retry:
+ * retry:
* foreach (lock in random_ordered_set_of_locks) {
- * ret = drm_modeset_lock(lock, &ctx)
- * if (ret == -EDEADLK) {
- * drm_modeset_backoff(&ctx);
- * goto retry;
- * }
+ * ret = drm_modeset_lock(lock, &ctx)
+ * if (ret == -EDEADLK) {
+ * drm_modeset_backoff(&ctx);
+ * goto retry;
+ * }
* }
- *
* ... do stuff ...
- *
* drm_modeset_drop_locks(&ctx);
* drm_modeset_acquire_fini(&ctx);
*/
diff --git a/drivers/gpu/drm/drm_prime.c b/drivers/gpu/drm/drm_prime.c
index 9f935f55d74c..27aa7183b20b 100644
--- a/drivers/gpu/drm/drm_prime.c
+++ b/drivers/gpu/drm/drm_prime.c
@@ -313,19 +313,15 @@ static const struct dma_buf_ops drm_gem_prime_dmabuf_ops = {
*
* Export callbacks:
*
- * - @gem_prime_pin (optional): prepare a GEM object for exporting
- *
- * - @gem_prime_get_sg_table: provide a scatter/gather table of pinned pages
- *
- * - @gem_prime_vmap: vmap a buffer exported by your driver
- *
- * - @gem_prime_vunmap: vunmap a buffer exported by your driver
- *
- * - @gem_prime_mmap (optional): mmap a buffer exported by your driver
+ * * @gem_prime_pin (optional): prepare a GEM object for exporting
+ * * @gem_prime_get_sg_table: provide a scatter/gather table of pinned pages
+ * * @gem_prime_vmap: vmap a buffer exported by your driver
+ * * @gem_prime_vunmap: vunmap a buffer exported by your driver
+ * * @gem_prime_mmap (optional): mmap a buffer exported by your driver
*
* Import callback:
*
- * - @gem_prime_import_sg_table (import): produce a GEM object from another
+ * * @gem_prime_import_sg_table (import): produce a GEM object from another
* driver's scatter/gather table
*/
diff --git a/drivers/gpu/drm/i915/i915_reg.h b/drivers/gpu/drm/i915/i915_reg.h
index bc7b8faba84d..9ebf03230ac0 100644
--- a/drivers/gpu/drm/i915/i915_reg.h
+++ b/drivers/gpu/drm/i915/i915_reg.h
@@ -804,31 +804,31 @@ enum skl_disp_power_wells {
*
* Note: DDI0 is digital port B, DD1 is digital port C, and DDI2 is
* digital port D (CHV) or port A (BXT).
- */
-/*
- * Dual channel PHY (VLV/CHV/BXT)
- * ---------------------------------
- * | CH0 | CH1 |
- * | CMN/PLL/REF | CMN/PLL/REF |
- * |---------------|---------------| Display PHY
- * | PCS01 | PCS23 | PCS01 | PCS23 |
- * |-------|-------|-------|-------|
- * |TX0|TX1|TX2|TX3|TX0|TX1|TX2|TX3|
- * ---------------------------------
- * | DDI0 | DDI1 | DP/HDMI ports
- * ---------------------------------
*
- * Single channel PHY (CHV/BXT)
- * -----------------
- * | CH0 |
- * | CMN/PLL/REF |
- * |---------------| Display PHY
- * | PCS01 | PCS23 |
- * |-------|-------|
- * |TX0|TX1|TX2|TX3|
- * -----------------
- * | DDI2 | DP/HDMI port
- * -----------------
+ *
+ * Dual channel PHY (VLV/CHV/BXT)
+ * ---------------------------------
+ * | CH0 | CH1 |
+ * | CMN/PLL/REF | CMN/PLL/REF |
+ * |---------------|---------------| Display PHY
+ * | PCS01 | PCS23 | PCS01 | PCS23 |
+ * |-------|-------|-------|-------|
+ * |TX0|TX1|TX2|TX3|TX0|TX1|TX2|TX3|
+ * ---------------------------------
+ * | DDI0 | DDI1 | DP/HDMI ports
+ * ---------------------------------
+ *
+ * Single channel PHY (CHV/BXT)
+ * -----------------
+ * | CH0 |
+ * | CMN/PLL/REF |
+ * |---------------| Display PHY
+ * | PCS01 | PCS23 |
+ * |-------|-------|
+ * |TX0|TX1|TX2|TX3|
+ * -----------------
+ * | DDI2 | DP/HDMI port
+ * -----------------
*/
#define DPIO_DEVFN 0
--
2.5.1
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 2/5] scripts/kernel-doc: Adding infrastructure for markdown support
2015-11-25 17:07 [PATCH 0/5] Better markup for GPU DocBook Daniel Vetter
@ 2015-11-25 17:07 ` Daniel Vetter
2015-11-25 17:07 ` Daniel Vetter
` (3 subsequent siblings)
4 siblings, 0 replies; 23+ messages in thread
From: Daniel Vetter @ 2015-11-25 17:07 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Danilo Cesar Lemes de Paula,
Randy Dunlap, Daniel Vetter, Laurent Pinchart, Jonathan Corbet,
Herbert Xu, Stephan Mueller, Michal Marek, linux-kernel,
linux-doc
From: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Markdown support is given by calling an external tool, pandoc, for all
highlighted text on kernel-doc.
Pandoc converts Markdown text to proper Docbook tags, which will be
later translated to pdf, html or other targets.
This adds the capability of adding human-readle text highlight (bold,
underline, etc), bullet and numbered lists, simple tables, fixed-width
text (including asciiart), requiring minimal changes to current
documentation.
At this moment, pandoc is totally optional. Docbooks ready for markdown
should be added to the MARKDOWNREADY variable inside the Makefile. In
case the developer doesn't have pandoc installed, Make will throw a
warning and the documentation build will continue, generating
simple Documentation without the features brought by pandoc.
Signed-off-by: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Stephan Mueller <smueller@chronox.de>
Cc: Michal Marek <mmarek@suse.cz>
Cc: linux-kernel@vger.kernel.org
Cc: linux-doc@vger.kernel.org
Cc: intel-gfx <intel-gfx@lists.freedesktop.org>
Cc: dri-devel <dri-devel@lists.freedesktop.org>
---
Documentation/DocBook/Makefile | 25 +++++++++++-----
scripts/docproc.c | 54 ++++++++++++++++++++++++----------
scripts/kernel-doc | 66 ++++++++++++++++++++++++++++++++++++++++--
3 files changed, 119 insertions(+), 26 deletions(-)
diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile
index 91f6d89bb19f..246ad38550e5 100644
--- a/Documentation/DocBook/Makefile
+++ b/Documentation/DocBook/Makefile
@@ -17,6 +17,8 @@ DOCBOOKS := z8530book.xml device-drivers.xml \
tracepoint.xml gpu.xml media_api.xml w1.xml \
writing_musb_glue_layer.xml crypto-API.xml iio.xml
+MARKDOWNREADY :=
+
include Documentation/DocBook/media/Makefile
###
@@ -87,18 +89,23 @@ XMLTOFLAGS += --skip-validation
# The following rules are used to generate the .xml documentation
# required to generate the final targets. (ps, pdf, html).
quiet_cmd_docproc = DOCPROC $@
- cmd_docproc = SRCTREE=$(srctree)/ $(DOCPROC) doc $< >$@
+ cmd_docproc = SRCTREE=$(srctree)/ $(DOCPROC) doc $<
define rule_docproc
- set -e; \
- $(if $($(quiet)cmd_$(1)),echo ' $($(quiet)cmd_$(1))';) \
- $(cmd_$(1)); \
- ( \
- echo 'cmd_$@ := $(cmd_$(1))'; \
- echo $@: `SRCTREE=$(srctree) $(DOCPROC) depend $<`; \
+ set -e; \
+ USEMARKDOWN=""; \
+ FILE=`basename $@`; \
+ [[ "$(MARKDOWNREADY)" =~ "$${FILE}" ]] && USEMARKDOWN="-use-markdown"; \
+ $(if $($(quiet)cmd_$(1)),echo ' $($(quiet)cmd_$(1))';) \
+ $(cmd_$(1)) $$USEMARKDOWN >$@; \
+ ( \
+ echo 'cmd_$@ := $(cmd_$(1))'; \
+ echo $@: `SRCTREE=$(srctree) $(DOCPROC) depend $<`; \
) > $(dir $@).$(notdir $@).cmd
endef
%.xml: %.tmpl $(KERNELDOC) $(DOCPROC) $(KERNELDOCXMLREF) FORCE
+ @(which pandoc > /dev/null 2>&1) || \
+ (echo "*** To get propper documentation you need to install pandoc ***";)
$(call if_changed_rule,docproc)
# Tell kbuild to always build the programs
@@ -109,6 +116,10 @@ notfoundtemplate = echo "*** You have to install docbook-utils or xmlto ***"; \
db2xtemplate = db2TYPE -o $(dir $@) $<
xmltotemplate = xmlto TYPE $(XMLTOFLAGS) -o $(dir $@) $<
+ifneq ($(shell which pandoc >/dev/null 2>&1 && echo found),found)
+ MARKDOWNREADY := "";
+endif
+
# determine which methods are available
ifeq ($(shell which db2ps >/dev/null 2>&1 && echo found),found)
use-db2x = db2x
diff --git a/scripts/docproc.c b/scripts/docproc.c
index e267e621431a..7c6b225a6a50 100644
--- a/scripts/docproc.c
+++ b/scripts/docproc.c
@@ -73,12 +73,15 @@ FILELINE * docsection;
#define NOFUNCTION "-nofunction"
#define NODOCSECTIONS "-no-doc-sections"
#define SHOWNOTFOUND "-show-not-found"
+#define USEMARKDOWN "-use-markdown"
static char *srctree, *kernsrctree;
static char **all_list = NULL;
static int all_list_len = 0;
+static int use_markdown = 0;
+
static void consume_symbol(const char *sym)
{
int i;
@@ -95,10 +98,11 @@ static void consume_symbol(const char *sym)
static void usage (void)
{
- fprintf(stderr, "Usage: docproc {doc|depend} file\n");
+ fprintf(stderr, "Usage: docproc {doc|depend} [-use-markdown] file\n");
fprintf(stderr, "Input is read from file.tmpl. Output is sent to stdout\n");
fprintf(stderr, "doc: frontend when generating kernel documentation\n");
fprintf(stderr, "depend: generate list of files referenced within file\n");
+ fprintf(stderr, "-use-markdown: pass -use-markdown to kernel-doc\n");
fprintf(stderr, "Environment variable SRCTREE: absolute path to sources.\n");
fprintf(stderr, " KBUILD_SRC: absolute path to kernel source tree.\n");
}
@@ -257,12 +261,15 @@ static void docfunctions(char * filename, char * type)
for (i=0; i <= symfilecnt; i++)
symcnt += symfilelist[i].symbolcnt;
- vec = malloc((2 + 2 * symcnt + 3) * sizeof(char *));
+ vec = malloc((2 + 2 * symcnt + 4) * sizeof(char *));
if (vec == NULL) {
perror("docproc: ");
exit(1);
}
vec[idx++] = KERNELDOC;
+ if (use_markdown)
+ vec[idx++] = USEMARKDOWN;
+
vec[idx++] = DOCBOOK;
vec[idx++] = NODOCSECTIONS;
for (i=0; i < symfilecnt; i++) {
@@ -294,6 +301,9 @@ static void singfunc(char * filename, char * line)
int i, idx = 0;
int startofsym = 1;
vec[idx++] = KERNELDOC;
+ if (use_markdown)
+ vec[idx++] = USEMARKDOWN;
+
vec[idx++] = DOCBOOK;
vec[idx++] = SHOWNOTFOUND;
@@ -328,8 +338,9 @@ static void singfunc(char * filename, char * line)
static void docsect(char *filename, char *line)
{
/* kerneldoc -docbook -show-not-found -function "section" file NULL */
- char *vec[7];
+ char *vec[8];
char *s;
+ int idx = 0;
for (s = line; *s; s++)
if (*s == '\n')
@@ -342,30 +353,37 @@ static void docsect(char *filename, char *line)
consume_symbol(s);
free(s);
- vec[0] = KERNELDOC;
- vec[1] = DOCBOOK;
- vec[2] = SHOWNOTFOUND;
- vec[3] = FUNCTION;
- vec[4] = line;
- vec[5] = filename;
- vec[6] = NULL;
+ vec[idx++] = KERNELDOC;
+ if (use_markdown)
+ vec[idx++] = USEMARKDOWN;
+
+ vec[idx++] = DOCBOOK;
+ vec[idx++] = SHOWNOTFOUND;
+ vec[idx++] = FUNCTION;
+ vec[idx++] = line;
+ vec[idx++] = filename;
+ vec[idx] = NULL;
exec_kernel_doc(vec);
}
static void find_all_symbols(char *filename)
{
- char *vec[4]; /* kerneldoc -list file NULL */
+ char *vec[5]; /* kerneldoc -list file NULL */
pid_t pid;
int ret, i, count, start;
char real_filename[PATH_MAX + 1];
int pipefd[2];
char *data, *str;
size_t data_len = 0;
+ int idx = 0;
+
+ vec[idx++] = KERNELDOC;
+ if (use_markdown)
+ vec[idx++] = USEMARKDOWN;
- vec[0] = KERNELDOC;
- vec[1] = LIST;
- vec[2] = filename;
- vec[3] = NULL;
+ vec[idx++] = LIST;
+ vec[idx++] = filename;
+ vec[idx] = NULL;
if (pipe(pipefd)) {
perror("pipe");
@@ -509,7 +527,7 @@ int main(int argc, char *argv[])
kernsrctree = getenv("KBUILD_SRC");
if (!kernsrctree || !*kernsrctree)
kernsrctree = srctree;
- if (argc != 3) {
+ if (argc < 3 || argc > 4) {
usage();
exit(1);
}
@@ -521,6 +539,10 @@ int main(int argc, char *argv[])
exit(2);
}
+ if (argc == 4 && strcmp("-use-markdown", argv[3]) == 0) {
+ use_markdown = 1;
+ }
+
if (strcmp("doc", argv[1]) == 0) {
/* Need to do this in two passes.
* First pass is used to collect all symbols exported
diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 638a38e1b419..28737053395a 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1,6 +1,7 @@
#!/usr/bin/perl -w
use strict;
+use IPC::Open3;
## Copyright (c) 1998 Michael Zucchi, All Rights Reserved ##
## Copyright (C) 2000, 1 Tim Waugh <twaugh@redhat.com> ##
@@ -282,6 +283,7 @@ if ($#ARGV == -1) {
my $kernelversion;
my $dohighlight = "";
+my $use_markdown = 0;
my $verbose = 0;
my $output_mode = "man";
@@ -424,6 +426,8 @@ while ($ARGV[0] =~ m/^-(.*)/) {
$function_only = 2;
$function = shift @ARGV;
$function_table{$function} = 1;
+ } elsif ($cmd eq "-use-markdown") {
+ $use_markdown = 1;
} elsif ($cmd eq "-v") {
$verbose = 1;
} elsif (($cmd eq "-h") || ($cmd eq "--help")) {
@@ -442,6 +446,7 @@ sub usage {
print " [ -no-doc-sections ]\n";
print " [ -function funcname [ -function funcname ...] ]\n";
print " [ -nofunction funcname [ -nofunction funcname ...] ]\n";
+ print " [ -use-markdown ]\n";
print " [ -v ]\n";
print " c source file(s) > outputfile\n";
print " -v : verbose output, more warnings & other info listed\n";
@@ -515,6 +520,49 @@ sub dump_doc_section {
}
}
+sub markdown_to_docbook {
+ my $orig_content = $_[0];
+
+ my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "pandoc --columns=80 -f markdown -t docbook" );
+
+ print CHLD_IN "$orig_content";
+ close(CHLD_IN);
+
+ waitpid($pid, 0);
+
+ my $content = "";
+ chomp(my @lines = <CHLD_OUT>);
+ foreach my $line (@lines) {
+ $content .= $line . "\n";
+ }
+ close(CHLD_OUT);
+ close(CHLD_ERR);
+
+ # pandoc insists in adding Main <para></para>, we should remove them.
+ $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm;
+
+ return $content;
+}
+
+# Markdown->Docbook conversion by pandoc requires unescaped text
+# Kernel-doc converts every & to "&", we need to convert it back.
+sub markdown_unescape
+{
+ my $text = shift;
+ my @lines = split /\n/, $text;
+
+ my @result;
+ foreach my $line (@lines) {
+ if ( $line =~ m /^ /s ) {
+ $line =~ s:\&:\&:gs
+ }
+ push @result, $line;
+ }
+
+ return join "\n",@result;
+
+}
+
##
# output function
#
@@ -541,11 +589,19 @@ sub output_highlight {
$contents = local_unescape($contents);
# convert data read & converted thru xml_escape() into &xyz; format:
$contents =~ s/\\\\\\/\&/g;
+
+ if ($use_markdown) {
+ $contents = markdown_unescape($contents);
+ }
}
+
# print STDERR "contents b4:$contents\n";
eval $dohighlight;
die $@ if $@;
# print STDERR "contents af:$contents\n";
+ if ($use_markdown) {
+ $contents = markdown_to_docbook($contents);
+ }
# strip whitespaces when generating html5
if ($output_mode eq "html5") {
@@ -553,7 +609,8 @@ sub output_highlight {
$contents =~ s/\s+$//;
}
foreach $line (split "\n", $contents) {
- if (! $output_preformatted) {
+ if (! $output_preformatted &&
+ !($use_markdown && $line =~ m /^ /s)) {
$line =~ s/^\s*//;
}
if ($line eq ""){
@@ -974,7 +1031,9 @@ sub output_section_xml(%) {
print "<refsect1>\n";
print "<title>$section</title>\n";
if ($section =~ m/EXAMPLE/i) {
- print "<informalexample><programlisting>\n";
+ print "<informalexample>\n";
+ # programlisting is already included by pandoc
+ print "<programlisting>\n" unless $use_markdown;
$output_preformatted = 1;
} else {
print "<para>\n";
@@ -982,7 +1041,8 @@ sub output_section_xml(%) {
output_highlight($args{'sections'}{$section});
$output_preformatted = 0;
if ($section =~ m/EXAMPLE/i) {
- print "</programlisting></informalexample>\n";
+ print "</programlisting>\n" unless $use_markdown;
+ print "</informalexample>\n";
} else {
print "</para>\n";
}
--
2.5.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 2/5] scripts/kernel-doc: Adding infrastructure for markdown support
@ 2015-11-25 17:07 ` Daniel Vetter
0 siblings, 0 replies; 23+ messages in thread
From: Daniel Vetter @ 2015-11-25 17:07 UTC (permalink / raw)
To: DRI Development
Cc: Michal Marek, Herbert Xu, Danilo Cesar Lemes de Paula,
Jonathan Corbet, Stephan Mueller, Daniel Vetter,
Intel Graphics Development, Randy Dunlap, linux-doc,
linux-kernel, Laurent Pinchart
From: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Markdown support is given by calling an external tool, pandoc, for all
highlighted text on kernel-doc.
Pandoc converts Markdown text to proper Docbook tags, which will be
later translated to pdf, html or other targets.
This adds the capability of adding human-readle text highlight (bold,
underline, etc), bullet and numbered lists, simple tables, fixed-width
text (including asciiart), requiring minimal changes to current
documentation.
At this moment, pandoc is totally optional. Docbooks ready for markdown
should be added to the MARKDOWNREADY variable inside the Makefile. In
case the developer doesn't have pandoc installed, Make will throw a
warning and the documentation build will continue, generating
simple Documentation without the features brought by pandoc.
Signed-off-by: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Stephan Mueller <smueller@chronox.de>
Cc: Michal Marek <mmarek@suse.cz>
Cc: linux-kernel@vger.kernel.org
Cc: linux-doc@vger.kernel.org
Cc: intel-gfx <intel-gfx@lists.freedesktop.org>
Cc: dri-devel <dri-devel@lists.freedesktop.org>
---
Documentation/DocBook/Makefile | 25 +++++++++++-----
scripts/docproc.c | 54 ++++++++++++++++++++++++----------
scripts/kernel-doc | 66 ++++++++++++++++++++++++++++++++++++++++--
3 files changed, 119 insertions(+), 26 deletions(-)
diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile
index 91f6d89bb19f..246ad38550e5 100644
--- a/Documentation/DocBook/Makefile
+++ b/Documentation/DocBook/Makefile
@@ -17,6 +17,8 @@ DOCBOOKS := z8530book.xml device-drivers.xml \
tracepoint.xml gpu.xml media_api.xml w1.xml \
writing_musb_glue_layer.xml crypto-API.xml iio.xml
+MARKDOWNREADY :=
+
include Documentation/DocBook/media/Makefile
###
@@ -87,18 +89,23 @@ XMLTOFLAGS += --skip-validation
# The following rules are used to generate the .xml documentation
# required to generate the final targets. (ps, pdf, html).
quiet_cmd_docproc = DOCPROC $@
- cmd_docproc = SRCTREE=$(srctree)/ $(DOCPROC) doc $< >$@
+ cmd_docproc = SRCTREE=$(srctree)/ $(DOCPROC) doc $<
define rule_docproc
- set -e; \
- $(if $($(quiet)cmd_$(1)),echo ' $($(quiet)cmd_$(1))';) \
- $(cmd_$(1)); \
- ( \
- echo 'cmd_$@ := $(cmd_$(1))'; \
- echo $@: `SRCTREE=$(srctree) $(DOCPROC) depend $<`; \
+ set -e; \
+ USEMARKDOWN=""; \
+ FILE=`basename $@`; \
+ [[ "$(MARKDOWNREADY)" =~ "$${FILE}" ]] && USEMARKDOWN="-use-markdown"; \
+ $(if $($(quiet)cmd_$(1)),echo ' $($(quiet)cmd_$(1))';) \
+ $(cmd_$(1)) $$USEMARKDOWN >$@; \
+ ( \
+ echo 'cmd_$@ := $(cmd_$(1))'; \
+ echo $@: `SRCTREE=$(srctree) $(DOCPROC) depend $<`; \
) > $(dir $@).$(notdir $@).cmd
endef
%.xml: %.tmpl $(KERNELDOC) $(DOCPROC) $(KERNELDOCXMLREF) FORCE
+ @(which pandoc > /dev/null 2>&1) || \
+ (echo "*** To get propper documentation you need to install pandoc ***";)
$(call if_changed_rule,docproc)
# Tell kbuild to always build the programs
@@ -109,6 +116,10 @@ notfoundtemplate = echo "*** You have to install docbook-utils or xmlto ***"; \
db2xtemplate = db2TYPE -o $(dir $@) $<
xmltotemplate = xmlto TYPE $(XMLTOFLAGS) -o $(dir $@) $<
+ifneq ($(shell which pandoc >/dev/null 2>&1 && echo found),found)
+ MARKDOWNREADY := "";
+endif
+
# determine which methods are available
ifeq ($(shell which db2ps >/dev/null 2>&1 && echo found),found)
use-db2x = db2x
diff --git a/scripts/docproc.c b/scripts/docproc.c
index e267e621431a..7c6b225a6a50 100644
--- a/scripts/docproc.c
+++ b/scripts/docproc.c
@@ -73,12 +73,15 @@ FILELINE * docsection;
#define NOFUNCTION "-nofunction"
#define NODOCSECTIONS "-no-doc-sections"
#define SHOWNOTFOUND "-show-not-found"
+#define USEMARKDOWN "-use-markdown"
static char *srctree, *kernsrctree;
static char **all_list = NULL;
static int all_list_len = 0;
+static int use_markdown = 0;
+
static void consume_symbol(const char *sym)
{
int i;
@@ -95,10 +98,11 @@ static void consume_symbol(const char *sym)
static void usage (void)
{
- fprintf(stderr, "Usage: docproc {doc|depend} file\n");
+ fprintf(stderr, "Usage: docproc {doc|depend} [-use-markdown] file\n");
fprintf(stderr, "Input is read from file.tmpl. Output is sent to stdout\n");
fprintf(stderr, "doc: frontend when generating kernel documentation\n");
fprintf(stderr, "depend: generate list of files referenced within file\n");
+ fprintf(stderr, "-use-markdown: pass -use-markdown to kernel-doc\n");
fprintf(stderr, "Environment variable SRCTREE: absolute path to sources.\n");
fprintf(stderr, " KBUILD_SRC: absolute path to kernel source tree.\n");
}
@@ -257,12 +261,15 @@ static void docfunctions(char * filename, char * type)
for (i=0; i <= symfilecnt; i++)
symcnt += symfilelist[i].symbolcnt;
- vec = malloc((2 + 2 * symcnt + 3) * sizeof(char *));
+ vec = malloc((2 + 2 * symcnt + 4) * sizeof(char *));
if (vec == NULL) {
perror("docproc: ");
exit(1);
}
vec[idx++] = KERNELDOC;
+ if (use_markdown)
+ vec[idx++] = USEMARKDOWN;
+
vec[idx++] = DOCBOOK;
vec[idx++] = NODOCSECTIONS;
for (i=0; i < symfilecnt; i++) {
@@ -294,6 +301,9 @@ static void singfunc(char * filename, char * line)
int i, idx = 0;
int startofsym = 1;
vec[idx++] = KERNELDOC;
+ if (use_markdown)
+ vec[idx++] = USEMARKDOWN;
+
vec[idx++] = DOCBOOK;
vec[idx++] = SHOWNOTFOUND;
@@ -328,8 +338,9 @@ static void singfunc(char * filename, char * line)
static void docsect(char *filename, char *line)
{
/* kerneldoc -docbook -show-not-found -function "section" file NULL */
- char *vec[7];
+ char *vec[8];
char *s;
+ int idx = 0;
for (s = line; *s; s++)
if (*s == '\n')
@@ -342,30 +353,37 @@ static void docsect(char *filename, char *line)
consume_symbol(s);
free(s);
- vec[0] = KERNELDOC;
- vec[1] = DOCBOOK;
- vec[2] = SHOWNOTFOUND;
- vec[3] = FUNCTION;
- vec[4] = line;
- vec[5] = filename;
- vec[6] = NULL;
+ vec[idx++] = KERNELDOC;
+ if (use_markdown)
+ vec[idx++] = USEMARKDOWN;
+
+ vec[idx++] = DOCBOOK;
+ vec[idx++] = SHOWNOTFOUND;
+ vec[idx++] = FUNCTION;
+ vec[idx++] = line;
+ vec[idx++] = filename;
+ vec[idx] = NULL;
exec_kernel_doc(vec);
}
static void find_all_symbols(char *filename)
{
- char *vec[4]; /* kerneldoc -list file NULL */
+ char *vec[5]; /* kerneldoc -list file NULL */
pid_t pid;
int ret, i, count, start;
char real_filename[PATH_MAX + 1];
int pipefd[2];
char *data, *str;
size_t data_len = 0;
+ int idx = 0;
+
+ vec[idx++] = KERNELDOC;
+ if (use_markdown)
+ vec[idx++] = USEMARKDOWN;
- vec[0] = KERNELDOC;
- vec[1] = LIST;
- vec[2] = filename;
- vec[3] = NULL;
+ vec[idx++] = LIST;
+ vec[idx++] = filename;
+ vec[idx] = NULL;
if (pipe(pipefd)) {
perror("pipe");
@@ -509,7 +527,7 @@ int main(int argc, char *argv[])
kernsrctree = getenv("KBUILD_SRC");
if (!kernsrctree || !*kernsrctree)
kernsrctree = srctree;
- if (argc != 3) {
+ if (argc < 3 || argc > 4) {
usage();
exit(1);
}
@@ -521,6 +539,10 @@ int main(int argc, char *argv[])
exit(2);
}
+ if (argc == 4 && strcmp("-use-markdown", argv[3]) == 0) {
+ use_markdown = 1;
+ }
+
if (strcmp("doc", argv[1]) == 0) {
/* Need to do this in two passes.
* First pass is used to collect all symbols exported
diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 638a38e1b419..28737053395a 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1,6 +1,7 @@
#!/usr/bin/perl -w
use strict;
+use IPC::Open3;
## Copyright (c) 1998 Michael Zucchi, All Rights Reserved ##
## Copyright (C) 2000, 1 Tim Waugh <twaugh@redhat.com> ##
@@ -282,6 +283,7 @@ if ($#ARGV == -1) {
my $kernelversion;
my $dohighlight = "";
+my $use_markdown = 0;
my $verbose = 0;
my $output_mode = "man";
@@ -424,6 +426,8 @@ while ($ARGV[0] =~ m/^-(.*)/) {
$function_only = 2;
$function = shift @ARGV;
$function_table{$function} = 1;
+ } elsif ($cmd eq "-use-markdown") {
+ $use_markdown = 1;
} elsif ($cmd eq "-v") {
$verbose = 1;
} elsif (($cmd eq "-h") || ($cmd eq "--help")) {
@@ -442,6 +446,7 @@ sub usage {
print " [ -no-doc-sections ]\n";
print " [ -function funcname [ -function funcname ...] ]\n";
print " [ -nofunction funcname [ -nofunction funcname ...] ]\n";
+ print " [ -use-markdown ]\n";
print " [ -v ]\n";
print " c source file(s) > outputfile\n";
print " -v : verbose output, more warnings & other info listed\n";
@@ -515,6 +520,49 @@ sub dump_doc_section {
}
}
+sub markdown_to_docbook {
+ my $orig_content = $_[0];
+
+ my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "pandoc --columns=80 -f markdown -t docbook" );
+
+ print CHLD_IN "$orig_content";
+ close(CHLD_IN);
+
+ waitpid($pid, 0);
+
+ my $content = "";
+ chomp(my @lines = <CHLD_OUT>);
+ foreach my $line (@lines) {
+ $content .= $line . "\n";
+ }
+ close(CHLD_OUT);
+ close(CHLD_ERR);
+
+ # pandoc insists in adding Main <para></para>, we should remove them.
+ $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm;
+
+ return $content;
+}
+
+# Markdown->Docbook conversion by pandoc requires unescaped text
+# Kernel-doc converts every & to "&", we need to convert it back.
+sub markdown_unescape
+{
+ my $text = shift;
+ my @lines = split /\n/, $text;
+
+ my @result;
+ foreach my $line (@lines) {
+ if ( $line =~ m /^ /s ) {
+ $line =~ s:\&:\&:gs
+ }
+ push @result, $line;
+ }
+
+ return join "\n",@result;
+
+}
+
##
# output function
#
@@ -541,11 +589,19 @@ sub output_highlight {
$contents = local_unescape($contents);
# convert data read & converted thru xml_escape() into &xyz; format:
$contents =~ s/\\\\\\/\&/g;
+
+ if ($use_markdown) {
+ $contents = markdown_unescape($contents);
+ }
}
+
# print STDERR "contents b4:$contents\n";
eval $dohighlight;
die $@ if $@;
# print STDERR "contents af:$contents\n";
+ if ($use_markdown) {
+ $contents = markdown_to_docbook($contents);
+ }
# strip whitespaces when generating html5
if ($output_mode eq "html5") {
@@ -553,7 +609,8 @@ sub output_highlight {
$contents =~ s/\s+$//;
}
foreach $line (split "\n", $contents) {
- if (! $output_preformatted) {
+ if (! $output_preformatted &&
+ !($use_markdown && $line =~ m /^ /s)) {
$line =~ s/^\s*//;
}
if ($line eq ""){
@@ -974,7 +1031,9 @@ sub output_section_xml(%) {
print "<refsect1>\n";
print "<title>$section</title>\n";
if ($section =~ m/EXAMPLE/i) {
- print "<informalexample><programlisting>\n";
+ print "<informalexample>\n";
+ # programlisting is already included by pandoc
+ print "<programlisting>\n" unless $use_markdown;
$output_preformatted = 1;
} else {
print "<para>\n";
@@ -982,7 +1041,8 @@ sub output_section_xml(%) {
output_highlight($args{'sections'}{$section});
$output_preformatted = 0;
if ($section =~ m/EXAMPLE/i) {
- print "</programlisting></informalexample>\n";
+ print "</programlisting>\n" unless $use_markdown;
+ print "</informalexample>\n";
} else {
print "</para>\n";
}
--
2.5.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 3/5] scripts/kernel-doc: Improve Markdown results
2015-11-25 17:07 [PATCH 0/5] Better markup for GPU DocBook Daniel Vetter
@ 2015-11-25 17:07 ` Daniel Vetter
2015-11-25 17:07 ` Daniel Vetter
` (3 subsequent siblings)
4 siblings, 0 replies; 23+ messages in thread
From: Daniel Vetter @ 2015-11-25 17:07 UTC (permalink / raw)
To: DRI Development
Cc: Intel Graphics Development, Danilo Cesar Lemes de Paula,
Randy Dunlap, Daniel Vetter, Laurent Pinchart, Jonathan Corbet,
Herbert Xu, Stephan Mueller, Michal Marek, linux-kernel,
linux-doc, Graham Whaley
From: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Using pandoc as the Markdown engine cause some minor side effects as
pandoc includes main <para> tags for almost everything.
Original Markdown support approach removes those main tags, but it caused
some inconsistencies when that tag is not the main one, like:
<something>..</something>
<para>...</para>
As kernel-doc was already including a <para> tag, it causes the presence
of double <para> tags (<para><para>), which is not supported by DocBook
spec.
Html target gets away with it, so it causes no harm, although other
targets might not be so lucky (pdf as example).
We're now delegating the inclusion of the main <para> tag to pandoc
only, as it knows when it's necessary or not.
That behavior causes a corner case, the only situation where we're
certainly that <para> is not needed, which is the <refpurpose> content.
For those cases, we're using a $output_markdown_nopara = 1 control var.
Signed-off-by: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Stephan Mueller <smueller@chronox.de>
Cc: Michal Marek <mmarek@suse.cz>
Cc: linux-kernel@vger.kernel.org
Cc: linux-doc@vger.kernel.org
Cc: intel-gfx <intel-gfx@lists.freedesktop.org>
Cc: dri-devel <dri-devel@lists.freedesktop.org>
Cc: Graham Whaley <graham.whaley@linux.intel.com>
---
scripts/kernel-doc | 48 ++++++++++++++++++++++++++++++++++--------------
1 file changed, 34 insertions(+), 14 deletions(-)
diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 28737053395a..e01e74f15a22 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -288,6 +288,7 @@ my $use_markdown = 0;
my $verbose = 0;
my $output_mode = "man";
my $output_preformatted = 0;
+my $output_markdown_nopara = 0;
my $no_doc_sections = 0;
my @highlights = @highlights_man;
my $blankline = $blankline_man;
@@ -538,8 +539,11 @@ sub markdown_to_docbook {
close(CHLD_OUT);
close(CHLD_ERR);
- # pandoc insists in adding Main <para></para>, we should remove them.
- $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm;
+ if ($output_markdown_nopara) {
+ # pandoc insists in adding Main <para></para>, sometimes we
+ # want to remove them.
+ $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm;
+ }
return $content;
}
@@ -614,7 +618,7 @@ sub output_highlight {
$line =~ s/^\s*//;
}
if ($line eq ""){
- if (! $output_preformatted) {
+ if (! $output_preformatted && ! $use_markdown) {
print $lineprefix, local_unescape($blankline);
}
} else {
@@ -1035,7 +1039,7 @@ sub output_section_xml(%) {
# programlisting is already included by pandoc
print "<programlisting>\n" unless $use_markdown;
$output_preformatted = 1;
- } else {
+ } elsif (! $use_markdown) {
print "<para>\n";
}
output_highlight($args{'sections'}{$section});
@@ -1043,7 +1047,7 @@ sub output_section_xml(%) {
if ($section =~ m/EXAMPLE/i) {
print "</programlisting>\n" unless $use_markdown;
print "</informalexample>\n";
- } else {
+ } elsif (! $use_markdown) {
print "</para>\n";
}
print "</refsect1>\n";
@@ -1075,7 +1079,9 @@ sub output_function_xml(%) {
print " <refname>" . $args{'function'} . "</refname>\n";
print " <refpurpose>\n";
print " ";
+ $output_markdown_nopara = 1;
output_highlight ($args{'purpose'});
+ $output_markdown_nopara = 0;
print " </refpurpose>\n";
print "</refnamediv>\n";
@@ -1113,10 +1119,12 @@ sub output_function_xml(%) {
$parameter_name =~ s/\[.*//;
print " <varlistentry>\n <term><parameter>$parameter</parameter></term>\n";
- print " <listitem>\n <para>\n";
+ print " <listitem>\n";
+ print " <para>\n" unless $use_markdown;
$lineprefix=" ";
output_highlight($args{'parameterdescs'}{$parameter_name});
- print " </para>\n </listitem>\n </varlistentry>\n";
+ print " </para>\n" unless $use_markdown;
+ print " </listitem>\n </varlistentry>\n";
}
print " </variablelist>\n";
} else {
@@ -1152,7 +1160,9 @@ sub output_struct_xml(%) {
print " <refname>" . $args{'type'} . " " . $args{'struct'} . "</refname>\n";
print " <refpurpose>\n";
print " ";
+ $output_markdown_nopara = 1;
output_highlight ($args{'purpose'});
+ $output_markdown_nopara = 0;
print " </refpurpose>\n";
print "</refnamediv>\n";
@@ -1205,9 +1215,11 @@ sub output_struct_xml(%) {
($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
print " <varlistentry>";
print " <term>$parameter</term>\n";
- print " <listitem><para>\n";
+ print " <listitem>\n";
+ print " <para>\n" unless $use_markdown;
output_highlight($args{'parameterdescs'}{$parameter_name});
- print " </para></listitem>\n";
+ print " </para>\n" unless $use_markdown;
+ print " </listitem>\n";
print " </varlistentry>\n";
}
print " </variablelist>\n";
@@ -1246,7 +1258,9 @@ sub output_enum_xml(%) {
print " <refname>enum " . $args{'enum'} . "</refname>\n";
print " <refpurpose>\n";
print " ";
+ $output_markdown_nopara = 1;
output_highlight ($args{'purpose'});
+ $output_markdown_nopara = 0;
print " </refpurpose>\n";
print "</refnamediv>\n";
@@ -1276,9 +1290,11 @@ sub output_enum_xml(%) {
print " <varlistentry>";
print " <term>$parameter</term>\n";
- print " <listitem><para>\n";
+ print " <listitem>\n";
+ print " <para>\n" unless $use_markdown;
output_highlight($args{'parameterdescs'}{$parameter_name});
- print " </para></listitem>\n";
+ print " </para>\n" unless $use_markdown;
+ print " </listitem>\n";
print " </varlistentry>\n";
}
print " </variablelist>\n";
@@ -1344,14 +1360,14 @@ sub output_blockhead_xml(%) {
if ($section =~ m/EXAMPLE/i) {
print "<example><para>\n";
$output_preformatted = 1;
- } else {
+ } elsif (! $use_markdown) {
print "<para>\n";
}
output_highlight($args{'sections'}{$section});
$output_preformatted = 0;
if ($section =~ m/EXAMPLE/i) {
print "</para></example>\n";
- } else {
+ } elsif (! $use_markdown) {
print "</para>";
}
if (!$args{'content-only'}) {
@@ -2721,7 +2737,11 @@ sub process_file($) {
{
if ( $1 eq "" )
{
- $contents .= $blankline;
+ if ($use_markdown) {
+ $contents .= "\n";
+ } else {
+ $contents .= $blankline;
+ }
}
else
{
--
2.5.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 3/5] scripts/kernel-doc: Improve Markdown results
@ 2015-11-25 17:07 ` Daniel Vetter
0 siblings, 0 replies; 23+ messages in thread
From: Daniel Vetter @ 2015-11-25 17:07 UTC (permalink / raw)
To: DRI Development
Cc: Michal Marek, Herbert Xu, Danilo Cesar Lemes de Paula,
Jonathan Corbet, Stephan Mueller, Daniel Vetter,
Intel Graphics Development, Randy Dunlap, linux-doc,
linux-kernel, Graham Whaley, Laurent Pinchart
From: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Using pandoc as the Markdown engine cause some minor side effects as
pandoc includes main <para> tags for almost everything.
Original Markdown support approach removes those main tags, but it caused
some inconsistencies when that tag is not the main one, like:
<something>..</something>
<para>...</para>
As kernel-doc was already including a <para> tag, it causes the presence
of double <para> tags (<para><para>), which is not supported by DocBook
spec.
Html target gets away with it, so it causes no harm, although other
targets might not be so lucky (pdf as example).
We're now delegating the inclusion of the main <para> tag to pandoc
only, as it knows when it's necessary or not.
That behavior causes a corner case, the only situation where we're
certainly that <para> is not needed, which is the <refpurpose> content.
For those cases, we're using a $output_markdown_nopara = 1 control var.
Signed-off-by: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Stephan Mueller <smueller@chronox.de>
Cc: Michal Marek <mmarek@suse.cz>
Cc: linux-kernel@vger.kernel.org
Cc: linux-doc@vger.kernel.org
Cc: intel-gfx <intel-gfx@lists.freedesktop.org>
Cc: dri-devel <dri-devel@lists.freedesktop.org>
Cc: Graham Whaley <graham.whaley@linux.intel.com>
---
scripts/kernel-doc | 48 ++++++++++++++++++++++++++++++++++--------------
1 file changed, 34 insertions(+), 14 deletions(-)
diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 28737053395a..e01e74f15a22 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -288,6 +288,7 @@ my $use_markdown = 0;
my $verbose = 0;
my $output_mode = "man";
my $output_preformatted = 0;
+my $output_markdown_nopara = 0;
my $no_doc_sections = 0;
my @highlights = @highlights_man;
my $blankline = $blankline_man;
@@ -538,8 +539,11 @@ sub markdown_to_docbook {
close(CHLD_OUT);
close(CHLD_ERR);
- # pandoc insists in adding Main <para></para>, we should remove them.
- $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm;
+ if ($output_markdown_nopara) {
+ # pandoc insists in adding Main <para></para>, sometimes we
+ # want to remove them.
+ $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm;
+ }
return $content;
}
@@ -614,7 +618,7 @@ sub output_highlight {
$line =~ s/^\s*//;
}
if ($line eq ""){
- if (! $output_preformatted) {
+ if (! $output_preformatted && ! $use_markdown) {
print $lineprefix, local_unescape($blankline);
}
} else {
@@ -1035,7 +1039,7 @@ sub output_section_xml(%) {
# programlisting is already included by pandoc
print "<programlisting>\n" unless $use_markdown;
$output_preformatted = 1;
- } else {
+ } elsif (! $use_markdown) {
print "<para>\n";
}
output_highlight($args{'sections'}{$section});
@@ -1043,7 +1047,7 @@ sub output_section_xml(%) {
if ($section =~ m/EXAMPLE/i) {
print "</programlisting>\n" unless $use_markdown;
print "</informalexample>\n";
- } else {
+ } elsif (! $use_markdown) {
print "</para>\n";
}
print "</refsect1>\n";
@@ -1075,7 +1079,9 @@ sub output_function_xml(%) {
print " <refname>" . $args{'function'} . "</refname>\n";
print " <refpurpose>\n";
print " ";
+ $output_markdown_nopara = 1;
output_highlight ($args{'purpose'});
+ $output_markdown_nopara = 0;
print " </refpurpose>\n";
print "</refnamediv>\n";
@@ -1113,10 +1119,12 @@ sub output_function_xml(%) {
$parameter_name =~ s/\[.*//;
print " <varlistentry>\n <term><parameter>$parameter</parameter></term>\n";
- print " <listitem>\n <para>\n";
+ print " <listitem>\n";
+ print " <para>\n" unless $use_markdown;
$lineprefix=" ";
output_highlight($args{'parameterdescs'}{$parameter_name});
- print " </para>\n </listitem>\n </varlistentry>\n";
+ print " </para>\n" unless $use_markdown;
+ print " </listitem>\n </varlistentry>\n";
}
print " </variablelist>\n";
} else {
@@ -1152,7 +1160,9 @@ sub output_struct_xml(%) {
print " <refname>" . $args{'type'} . " " . $args{'struct'} . "</refname>\n";
print " <refpurpose>\n";
print " ";
+ $output_markdown_nopara = 1;
output_highlight ($args{'purpose'});
+ $output_markdown_nopara = 0;
print " </refpurpose>\n";
print "</refnamediv>\n";
@@ -1205,9 +1215,11 @@ sub output_struct_xml(%) {
($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
print " <varlistentry>";
print " <term>$parameter</term>\n";
- print " <listitem><para>\n";
+ print " <listitem>\n";
+ print " <para>\n" unless $use_markdown;
output_highlight($args{'parameterdescs'}{$parameter_name});
- print " </para></listitem>\n";
+ print " </para>\n" unless $use_markdown;
+ print " </listitem>\n";
print " </varlistentry>\n";
}
print " </variablelist>\n";
@@ -1246,7 +1258,9 @@ sub output_enum_xml(%) {
print " <refname>enum " . $args{'enum'} . "</refname>\n";
print " <refpurpose>\n";
print " ";
+ $output_markdown_nopara = 1;
output_highlight ($args{'purpose'});
+ $output_markdown_nopara = 0;
print " </refpurpose>\n";
print "</refnamediv>\n";
@@ -1276,9 +1290,11 @@ sub output_enum_xml(%) {
print " <varlistentry>";
print " <term>$parameter</term>\n";
- print " <listitem><para>\n";
+ print " <listitem>\n";
+ print " <para>\n" unless $use_markdown;
output_highlight($args{'parameterdescs'}{$parameter_name});
- print " </para></listitem>\n";
+ print " </para>\n" unless $use_markdown;
+ print " </listitem>\n";
print " </varlistentry>\n";
}
print " </variablelist>\n";
@@ -1344,14 +1360,14 @@ sub output_blockhead_xml(%) {
if ($section =~ m/EXAMPLE/i) {
print "<example><para>\n";
$output_preformatted = 1;
- } else {
+ } elsif (! $use_markdown) {
print "<para>\n";
}
output_highlight($args{'sections'}{$section});
$output_preformatted = 0;
if ($section =~ m/EXAMPLE/i) {
print "</para></example>\n";
- } else {
+ } elsif (! $use_markdown) {
print "</para>";
}
if (!$args{'content-only'}) {
@@ -2721,7 +2737,11 @@ sub process_file($) {
{
if ( $1 eq "" )
{
- $contents .= $blankline;
+ if ($use_markdown) {
+ $contents .= "\n";
+ } else {
+ $contents .= $blankline;
+ }
}
else
{
--
2.5.1
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 4/5] scripts/kernel-doc: Use asciidoc instead of markdown
2015-11-25 17:07 [PATCH 0/5] Better markup for GPU DocBook Daniel Vetter
` (2 preceding siblings ...)
2015-11-25 17:07 ` Daniel Vetter
@ 2015-11-25 17:07 ` Daniel Vetter
2015-12-02 9:34 ` [PATCH] " Daniel Vetter
2015-11-25 17:07 ` [PATCH 5/5] drm: Enable markdown^Wasciidoc for gpu.tmpl Daniel Vetter
4 siblings, 1 reply; 23+ messages in thread
From: Daniel Vetter @ 2015-11-25 17:07 UTC (permalink / raw)
To: DRI Development
Cc: Danilo Cesar Lemes de Paula, Jonathan Corbet, Daniel Vetter,
Intel Graphics Development, Thomas Wood, Daniel Vetter
By popular demand.
This needs some adjustment/fixups after feeding snippets to asciidoc
since compared to markdown asciidown escapes xml markup and doesn't
just let it through.
The other noticeable change is that build times increase a lot - we
need to launch the markup process per-snippet, there's a few thousand
of them and asciidoc (python) has a substantial higher overhead per
invocation than pandoc (haskell).
Cc: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Cc: Thomas Wood <thomas.wood@intel.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
Documentation/DocBook/Makefile | 6 +++---
scripts/kernel-doc | 12 +++++++++++-
2 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile
index 246ad38550e5..5335955c0de5 100644
--- a/Documentation/DocBook/Makefile
+++ b/Documentation/DocBook/Makefile
@@ -104,8 +104,8 @@ define rule_docproc
endef
%.xml: %.tmpl $(KERNELDOC) $(DOCPROC) $(KERNELDOCXMLREF) FORCE
- @(which pandoc > /dev/null 2>&1) || \
- (echo "*** To get propper documentation you need to install pandoc ***";)
+ @(which asciidoc > /dev/null 2>&1) || \
+ (echo "*** To get propper documentation you need to install asciidoc ***";)
$(call if_changed_rule,docproc)
# Tell kbuild to always build the programs
@@ -116,7 +116,7 @@ notfoundtemplate = echo "*** You have to install docbook-utils or xmlto ***"; \
db2xtemplate = db2TYPE -o $(dir $@) $<
xmltotemplate = xmlto TYPE $(XMLTOFLAGS) -o $(dir $@) $<
-ifneq ($(shell which pandoc >/dev/null 2>&1 && echo found),found)
+ifneq ($(shell which asciidoc >/dev/null 2>&1 && echo found),found)
MARKDOWNREADY := "";
endif
diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index e01e74f15a22..c8eed5299a4b 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -524,7 +524,7 @@ sub dump_doc_section {
sub markdown_to_docbook {
my $orig_content = $_[0];
- my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "pandoc --columns=80 -f markdown -t docbook" );
+ my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "asciidoc --no-header-footer --backend=docbook45 -" );
print CHLD_IN "$orig_content";
close(CHLD_IN);
@@ -605,6 +605,16 @@ sub output_highlight {
# print STDERR "contents af:$contents\n";
if ($use_markdown) {
$contents = markdown_to_docbook($contents);
+
+ # Compared to markdown asciidoc doesn't let through arbitrary xml
+ # markup. We need to un-escape the kerneldoc markup for functions,
+ # structures, ...
+ $contents =~ s/<quote>(\S*)<\/quote>/<quote>$1<\/quote>/g;
+ $contents =~ s/<constant>(\S*)<\/constant>/<constant>$1<\/constant>/g;
+ $contents =~ s/<structname>(\S*)<\/structname>/<structname>$1<\/structname>/g;
+ $contents =~ s/<parameter>(\S*)<\/parameter>/<parameter>$1<\/parameter>/g;
+ $contents =~ s/<function>(\S*)<\/function>/<function>$1<\/function>/g;
+ $contents =~ s/<envar>(\S*)<\/envar>/<envar>$1<\/envar>/g;
}
# strip whitespaces when generating html5
--
2.5.1
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH 5/5] drm: Enable markdown^Wasciidoc for gpu.tmpl
2015-11-25 17:07 [PATCH 0/5] Better markup for GPU DocBook Daniel Vetter
` (3 preceding siblings ...)
2015-11-25 17:07 ` [PATCH 4/5] scripts/kernel-doc: Use asciidoc instead of markdown Daniel Vetter
@ 2015-11-25 17:07 ` Daniel Vetter
2015-12-11 22:12 ` Jonathan Corbet
4 siblings, 1 reply; 23+ messages in thread
From: Daniel Vetter @ 2015-11-25 17:07 UTC (permalink / raw)
To: DRI Development
Cc: Jonathan Corbet, Daniel Vetter, Intel Graphics Development,
Thomas Wood, Dave Airlie, Daniel Vetter
Unfortunately the entire improved docbook project died at KS in a
massive bikeshed. So we need to carry this around in drm private trees
forever :(
I discussed this with Dave at kernel summit and he's ok with this.
I'll maintain the asciidoc support in topic/kerneldoc in the
drm-intel.git tree. There's an autobuilder that pushes
drm-intel-nightly (which includes all of Dave's trees) to
http://dri.freedesktop.org/docs/drm/
Thomas Wood keeps care of that autobuilder, so please ping him if
there's something amiss.
Note that asciidoc is optional - all the kerneldoc comment layout
rules are the same, those bits landed in 4.4. It will simply not quite
look as pretty if you don't have it installed.
Cc: Dave Airlie <airlied@redhat.com>
Cc: Thomas Wood <thomas.wood@intel.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Acked-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
Documentation/DocBook/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile
index 5335955c0de5..b655343631cf 100644
--- a/Documentation/DocBook/Makefile
+++ b/Documentation/DocBook/Makefile
@@ -17,7 +17,7 @@ DOCBOOKS := z8530book.xml device-drivers.xml \
tracepoint.xml gpu.xml media_api.xml w1.xml \
writing_musb_glue_layer.xml crypto-API.xml iio.xml
-MARKDOWNREADY :=
+MARKDOWNREADY := gpu.xml
include Documentation/DocBook/media/Makefile
--
2.5.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [PATCH] scripts/kernel-doc: Use asciidoc instead of markdown
2015-11-25 17:07 ` [PATCH 4/5] scripts/kernel-doc: Use asciidoc instead of markdown Daniel Vetter
@ 2015-12-02 9:34 ` Daniel Vetter
2015-12-02 13:33 ` Jani Nikula
0 siblings, 1 reply; 23+ messages in thread
From: Daniel Vetter @ 2015-12-02 9:34 UTC (permalink / raw)
To: DRI Development
Cc: Daniel Vetter, Daniel Vetter, Thomas Wood,
Danilo Cesar Lemes de Paula, Jonathan Corbet
By popular demand.
This needs some adjustment/fixups after feeding snippets to asciidoc
since compared to markdown asciidown escapes xml markup and doesn't
just let it through.
The other noticeable change is that build times increase a lot - we
need to launch the markup process per-snippet, there's a few thousand
of them and asciidoc (python) has a substantial higher overhead per
invocation than pandoc (haskell).
v2: More fine-tuning:
- Use unixe newlines, not the default dos ones. Only results in
ugliness in the intermediate gpu.xml, but still.
- Resurrect the hack to remove paragraphs for the one-line references.
Like markdown asciidoc insists to wrap everything.
Cc: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Cc: Thomas Wood <thomas.wood@intel.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
---
Documentation/DocBook/Makefile | 6 +++---
scripts/kernel-doc | 18 ++++++++++++++----
2 files changed, 17 insertions(+), 7 deletions(-)
diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile
index 246ad38550e5..5335955c0de5 100644
--- a/Documentation/DocBook/Makefile
+++ b/Documentation/DocBook/Makefile
@@ -104,8 +104,8 @@ define rule_docproc
endef
%.xml: %.tmpl $(KERNELDOC) $(DOCPROC) $(KERNELDOCXMLREF) FORCE
- @(which pandoc > /dev/null 2>&1) || \
- (echo "*** To get propper documentation you need to install pandoc ***";)
+ @(which asciidoc > /dev/null 2>&1) || \
+ (echo "*** To get propper documentation you need to install asciidoc ***";)
$(call if_changed_rule,docproc)
# Tell kbuild to always build the programs
@@ -116,7 +116,7 @@ notfoundtemplate = echo "*** You have to install docbook-utils or xmlto ***"; \
db2xtemplate = db2TYPE -o $(dir $@) $<
xmltotemplate = xmlto TYPE $(XMLTOFLAGS) -o $(dir $@) $<
-ifneq ($(shell which pandoc >/dev/null 2>&1 && echo found),found)
+ifneq ($(shell which asciidoc >/dev/null 2>&1 && echo found),found)
MARKDOWNREADY := "";
endif
diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index e01e74f15a22..cbfa4c03189e 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -524,7 +524,7 @@ sub dump_doc_section {
sub markdown_to_docbook {
my $orig_content = $_[0];
- my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "pandoc --columns=80 -f markdown -t docbook" );
+ my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "asciidoc -a 'newline=\\n' --no-header-footer --backend=docbook45 -" );
print CHLD_IN "$orig_content";
close(CHLD_IN);
@@ -540,9 +540,9 @@ sub markdown_to_docbook {
close(CHLD_ERR);
if ($output_markdown_nopara) {
- # pandoc insists in adding Main <para></para>, sometimes we
- # want to remove them.
- $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm;
+ # asciidoc insists in adding Main <simpara></simpara>, sometimes
+ # we want to remove them.
+ $content =~ s:\A\s*<simpara>(.*)</simpara>\Z:$1:egsm;
}
return $content;
@@ -605,6 +605,16 @@ sub output_highlight {
# print STDERR "contents af:$contents\n";
if ($use_markdown) {
$contents = markdown_to_docbook($contents);
+
+ # Compared to markdown asciidoc doesn't let through arbitrary xml
+ # markup. We need to un-escape the kerneldoc markup for functions,
+ # structures, ...
+ $contents =~ s/<quote>(\S*)<\/quote>/<quote>$1<\/quote>/g;
+ $contents =~ s/<constant>(\S*)<\/constant>/<constant>$1<\/constant>/g;
+ $contents =~ s/<structname>(\S*)<\/structname>/<structname>$1<\/structname>/g;
+ $contents =~ s/<parameter>(\S*)<\/parameter>/<parameter>$1<\/parameter>/g;
+ $contents =~ s/<function>(\S*)<\/function>/<function>$1<\/function>/g;
+ $contents =~ s/<envar>(\S*)<\/envar>/<envar>$1<\/envar>/g;
}
# strip whitespaces when generating html5
--
2.5.1
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [PATCH] scripts/kernel-doc: Use asciidoc instead of markdown
2015-12-02 9:34 ` [PATCH] " Daniel Vetter
@ 2015-12-02 13:33 ` Jani Nikula
2015-12-02 15:32 ` Daniel Vetter
0 siblings, 1 reply; 23+ messages in thread
From: Jani Nikula @ 2015-12-02 13:33 UTC (permalink / raw)
To: DRI Development
Cc: Daniel Vetter, Jonathan Corbet, Thomas Wood,
Danilo Cesar Lemes de Paula, Daniel Vetter
On Wed, 02 Dec 2015, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> By popular demand.
Unpopular if you ask me.
Also, I think what you're trying to say is, use asciidoc the tool
instead of pandoc the tool, and as a side effect change the markup. Or
even, use a tool written in one language instead of a tool written in
another, and as a side effect change the markup.
In the end, I guess I would just like to have the technical and factual
reasons for this change in the commit message, instead of referring to
some mythical unverifiable popular demand.
BR,
Jani.
>
> This needs some adjustment/fixups after feeding snippets to asciidoc
> since compared to markdown asciidown escapes xml markup and doesn't
> just let it through.
>
> The other noticeable change is that build times increase a lot - we
> need to launch the markup process per-snippet, there's a few thousand
> of them and asciidoc (python) has a substantial higher overhead per
> invocation than pandoc (haskell).
>
> v2: More fine-tuning:
>
> - Use unixe newlines, not the default dos ones. Only results in
> ugliness in the intermediate gpu.xml, but still.
>
> - Resurrect the hack to remove paragraphs for the one-line references.
> Like markdown asciidoc insists to wrap everything.
>
> Cc: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
> Cc: Thomas Wood <thomas.wood@intel.com>
> Cc: Jonathan Corbet <corbet@lwn.net>
> Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> ---
> Documentation/DocBook/Makefile | 6 +++---
> scripts/kernel-doc | 18 ++++++++++++++----
> 2 files changed, 17 insertions(+), 7 deletions(-)
>
> diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile
> index 246ad38550e5..5335955c0de5 100644
> --- a/Documentation/DocBook/Makefile
> +++ b/Documentation/DocBook/Makefile
> @@ -104,8 +104,8 @@ define rule_docproc
> endef
>
> %.xml: %.tmpl $(KERNELDOC) $(DOCPROC) $(KERNELDOCXMLREF) FORCE
> - @(which pandoc > /dev/null 2>&1) || \
> - (echo "*** To get propper documentation you need to install pandoc ***";)
> + @(which asciidoc > /dev/null 2>&1) || \
> + (echo "*** To get propper documentation you need to install asciidoc ***";)
> $(call if_changed_rule,docproc)
>
> # Tell kbuild to always build the programs
> @@ -116,7 +116,7 @@ notfoundtemplate = echo "*** You have to install docbook-utils or xmlto ***"; \
> db2xtemplate = db2TYPE -o $(dir $@) $<
> xmltotemplate = xmlto TYPE $(XMLTOFLAGS) -o $(dir $@) $<
>
> -ifneq ($(shell which pandoc >/dev/null 2>&1 && echo found),found)
> +ifneq ($(shell which asciidoc >/dev/null 2>&1 && echo found),found)
> MARKDOWNREADY := "";
> endif
>
> diff --git a/scripts/kernel-doc b/scripts/kernel-doc
> index e01e74f15a22..cbfa4c03189e 100755
> --- a/scripts/kernel-doc
> +++ b/scripts/kernel-doc
> @@ -524,7 +524,7 @@ sub dump_doc_section {
> sub markdown_to_docbook {
> my $orig_content = $_[0];
>
> - my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "pandoc --columns=80 -f markdown -t docbook" );
> + my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "asciidoc -a 'newline=\\n' --no-header-footer --backend=docbook45 -" );
>
> print CHLD_IN "$orig_content";
> close(CHLD_IN);
> @@ -540,9 +540,9 @@ sub markdown_to_docbook {
> close(CHLD_ERR);
>
> if ($output_markdown_nopara) {
> - # pandoc insists in adding Main <para></para>, sometimes we
> - # want to remove them.
> - $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm;
> + # asciidoc insists in adding Main <simpara></simpara>, sometimes
> + # we want to remove them.
> + $content =~ s:\A\s*<simpara>(.*)</simpara>\Z:$1:egsm;
> }
>
> return $content;
> @@ -605,6 +605,16 @@ sub output_highlight {
> # print STDERR "contents af:$contents\n";
> if ($use_markdown) {
> $contents = markdown_to_docbook($contents);
> +
> + # Compared to markdown asciidoc doesn't let through arbitrary xml
> + # markup. We need to un-escape the kerneldoc markup for functions,
> + # structures, ...
> + $contents =~ s/<quote>(\S*)<\/quote>/<quote>$1<\/quote>/g;
> + $contents =~ s/<constant>(\S*)<\/constant>/<constant>$1<\/constant>/g;
> + $contents =~ s/<structname>(\S*)<\/structname>/<structname>$1<\/structname>/g;
> + $contents =~ s/<parameter>(\S*)<\/parameter>/<parameter>$1<\/parameter>/g;
> + $contents =~ s/<function>(\S*)<\/function>/<function>$1<\/function>/g;
> + $contents =~ s/<envar>(\S*)<\/envar>/<envar>$1<\/envar>/g;
> }
>
> # strip whitespaces when generating html5
--
Jani Nikula, Intel Open Source Technology Center
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] scripts/kernel-doc: Use asciidoc instead of markdown
2015-12-02 13:33 ` Jani Nikula
@ 2015-12-02 15:32 ` Daniel Vetter
2015-12-02 15:57 ` Jani Nikula
0 siblings, 1 reply; 23+ messages in thread
From: Daniel Vetter @ 2015-12-02 15:32 UTC (permalink / raw)
To: Jani Nikula
Cc: Danilo Cesar Lemes de Paula, Jonathan Corbet, Daniel Vetter,
DRI Development, Thomas Wood, Daniel Vetter
On Wed, Dec 02, 2015 at 03:33:43PM +0200, Jani Nikula wrote:
> On Wed, 02 Dec 2015, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> > By popular demand.
>
> Unpopular if you ask me.
>
> Also, I think what you're trying to say is, use asciidoc the tool
> instead of pandoc the tool, and as a side effect change the markup. Or
> even, use a tool written in one language instead of a tool written in
> another, and as a side effect change the markup.
>
> In the end, I guess I would just like to have the technical and factual
> reasons for this change in the commit message, instead of referring to
> some mythical unverifiable popular demand.
When discussing with Dave at KS whether we could do some kind of markup
for drm only he asked me to use asciidoc instead of pandoc. So popular =
100% of the relevant maintainer (which is just Dave).
Suprisingly the markup doesn't really change much since mostly asciidoc is
just a (massive) extension of the markdown markup we've used. The changes
are really just in how it's glued into kernel-doc. At least I didn't
notice anything else yet. So yep, it's really just a tooling exchange that
resulted in markup language changes.
What I probably should mention is that asciidoc has better table support,
which is something we've run into a bit with markdown already (our
property tables ain't pretty).
-Daniel
>
> BR,
> Jani.
>
>
> >
> > This needs some adjustment/fixups after feeding snippets to asciidoc
> > since compared to markdown asciidown escapes xml markup and doesn't
> > just let it through.
> >
> > The other noticeable change is that build times increase a lot - we
> > need to launch the markup process per-snippet, there's a few thousand
> > of them and asciidoc (python) has a substantial higher overhead per
> > invocation than pandoc (haskell).
> >
> > v2: More fine-tuning:
> >
> > - Use unixe newlines, not the default dos ones. Only results in
> > ugliness in the intermediate gpu.xml, but still.
> >
> > - Resurrect the hack to remove paragraphs for the one-line references.
> > Like markdown asciidoc insists to wrap everything.
> >
> > Cc: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
> > Cc: Thomas Wood <thomas.wood@intel.com>
> > Cc: Jonathan Corbet <corbet@lwn.net>
> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
> > ---
> > Documentation/DocBook/Makefile | 6 +++---
> > scripts/kernel-doc | 18 ++++++++++++++----
> > 2 files changed, 17 insertions(+), 7 deletions(-)
> >
> > diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile
> > index 246ad38550e5..5335955c0de5 100644
> > --- a/Documentation/DocBook/Makefile
> > +++ b/Documentation/DocBook/Makefile
> > @@ -104,8 +104,8 @@ define rule_docproc
> > endef
> >
> > %.xml: %.tmpl $(KERNELDOC) $(DOCPROC) $(KERNELDOCXMLREF) FORCE
> > - @(which pandoc > /dev/null 2>&1) || \
> > - (echo "*** To get propper documentation you need to install pandoc ***";)
> > + @(which asciidoc > /dev/null 2>&1) || \
> > + (echo "*** To get propper documentation you need to install asciidoc ***";)
> > $(call if_changed_rule,docproc)
> >
> > # Tell kbuild to always build the programs
> > @@ -116,7 +116,7 @@ notfoundtemplate = echo "*** You have to install docbook-utils or xmlto ***"; \
> > db2xtemplate = db2TYPE -o $(dir $@) $<
> > xmltotemplate = xmlto TYPE $(XMLTOFLAGS) -o $(dir $@) $<
> >
> > -ifneq ($(shell which pandoc >/dev/null 2>&1 && echo found),found)
> > +ifneq ($(shell which asciidoc >/dev/null 2>&1 && echo found),found)
> > MARKDOWNREADY := "";
> > endif
> >
> > diff --git a/scripts/kernel-doc b/scripts/kernel-doc
> > index e01e74f15a22..cbfa4c03189e 100755
> > --- a/scripts/kernel-doc
> > +++ b/scripts/kernel-doc
> > @@ -524,7 +524,7 @@ sub dump_doc_section {
> > sub markdown_to_docbook {
> > my $orig_content = $_[0];
> >
> > - my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "pandoc --columns=80 -f markdown -t docbook" );
> > + my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "asciidoc -a 'newline=\\n' --no-header-footer --backend=docbook45 -" );
> >
> > print CHLD_IN "$orig_content";
> > close(CHLD_IN);
> > @@ -540,9 +540,9 @@ sub markdown_to_docbook {
> > close(CHLD_ERR);
> >
> > if ($output_markdown_nopara) {
> > - # pandoc insists in adding Main <para></para>, sometimes we
> > - # want to remove them.
> > - $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm;
> > + # asciidoc insists in adding Main <simpara></simpara>, sometimes
> > + # we want to remove them.
> > + $content =~ s:\A\s*<simpara>(.*)</simpara>\Z:$1:egsm;
> > }
> >
> > return $content;
> > @@ -605,6 +605,16 @@ sub output_highlight {
> > # print STDERR "contents af:$contents\n";
> > if ($use_markdown) {
> > $contents = markdown_to_docbook($contents);
> > +
> > + # Compared to markdown asciidoc doesn't let through arbitrary xml
> > + # markup. We need to un-escape the kerneldoc markup for functions,
> > + # structures, ...
> > + $contents =~ s/<quote>(\S*)<\/quote>/<quote>$1<\/quote>/g;
> > + $contents =~ s/<constant>(\S*)<\/constant>/<constant>$1<\/constant>/g;
> > + $contents =~ s/<structname>(\S*)<\/structname>/<structname>$1<\/structname>/g;
> > + $contents =~ s/<parameter>(\S*)<\/parameter>/<parameter>$1<\/parameter>/g;
> > + $contents =~ s/<function>(\S*)<\/function>/<function>$1<\/function>/g;
> > + $contents =~ s/<envar>(\S*)<\/envar>/<envar>$1<\/envar>/g;
> > }
> >
> > # strip whitespaces when generating html5
>
> --
> Jani Nikula, Intel Open Source Technology Center
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH] scripts/kernel-doc: Use asciidoc instead of markdown
2015-12-02 15:32 ` Daniel Vetter
@ 2015-12-02 15:57 ` Jani Nikula
0 siblings, 0 replies; 23+ messages in thread
From: Jani Nikula @ 2015-12-02 15:57 UTC (permalink / raw)
To: Daniel Vetter
Cc: Danilo Cesar Lemes de Paula, Jonathan Corbet, Daniel Vetter,
DRI Development, Thomas Wood, Daniel Vetter
On Wed, 02 Dec 2015, Daniel Vetter <daniel@ffwll.ch> wrote:
> On Wed, Dec 02, 2015 at 03:33:43PM +0200, Jani Nikula wrote:
>> On Wed, 02 Dec 2015, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>> > By popular demand.
>>
>> Unpopular if you ask me.
>>
>> Also, I think what you're trying to say is, use asciidoc the tool
>> instead of pandoc the tool, and as a side effect change the markup. Or
>> even, use a tool written in one language instead of a tool written in
>> another, and as a side effect change the markup.
>>
>> In the end, I guess I would just like to have the technical and factual
>> reasons for this change in the commit message, instead of referring to
>> some mythical unverifiable popular demand.
>
> When discussing with Dave at KS whether we could do some kind of markup
> for drm only he asked me to use asciidoc instead of pandoc. So popular =
> 100% of the relevant maintainer (which is just Dave).
Please put that in the commit message then.
> Suprisingly the markup doesn't really change much since mostly asciidoc is
> just a (massive) extension of the markdown markup we've used. The changes
> are really just in how it's glued into kernel-doc. At least I didn't
> notice anything else yet. So yep, it's really just a tooling exchange that
> resulted in markup language changes.
>
> What I probably should mention is that asciidoc has better table support,
> which is something we've run into a bit with markdown already (our
> property tables ain't pretty).
I'll live.
BR,
Jani.
> -Daniel
>
>>
>> BR,
>> Jani.
>>
>>
>> >
>> > This needs some adjustment/fixups after feeding snippets to asciidoc
>> > since compared to markdown asciidown escapes xml markup and doesn't
>> > just let it through.
>> >
>> > The other noticeable change is that build times increase a lot - we
>> > need to launch the markup process per-snippet, there's a few thousand
>> > of them and asciidoc (python) has a substantial higher overhead per
>> > invocation than pandoc (haskell).
>> >
>> > v2: More fine-tuning:
>> >
>> > - Use unixe newlines, not the default dos ones. Only results in
>> > ugliness in the intermediate gpu.xml, but still.
>> >
>> > - Resurrect the hack to remove paragraphs for the one-line references.
>> > Like markdown asciidoc insists to wrap everything.
>> >
>> > Cc: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
>> > Cc: Thomas Wood <thomas.wood@intel.com>
>> > Cc: Jonathan Corbet <corbet@lwn.net>
>> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
>> > ---
>> > Documentation/DocBook/Makefile | 6 +++---
>> > scripts/kernel-doc | 18 ++++++++++++++----
>> > 2 files changed, 17 insertions(+), 7 deletions(-)
>> >
>> > diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile
>> > index 246ad38550e5..5335955c0de5 100644
>> > --- a/Documentation/DocBook/Makefile
>> > +++ b/Documentation/DocBook/Makefile
>> > @@ -104,8 +104,8 @@ define rule_docproc
>> > endef
>> >
>> > %.xml: %.tmpl $(KERNELDOC) $(DOCPROC) $(KERNELDOCXMLREF) FORCE
>> > - @(which pandoc > /dev/null 2>&1) || \
>> > - (echo "*** To get propper documentation you need to install pandoc ***";)
>> > + @(which asciidoc > /dev/null 2>&1) || \
>> > + (echo "*** To get propper documentation you need to install asciidoc ***";)
>> > $(call if_changed_rule,docproc)
>> >
>> > # Tell kbuild to always build the programs
>> > @@ -116,7 +116,7 @@ notfoundtemplate = echo "*** You have to install docbook-utils or xmlto ***"; \
>> > db2xtemplate = db2TYPE -o $(dir $@) $<
>> > xmltotemplate = xmlto TYPE $(XMLTOFLAGS) -o $(dir $@) $<
>> >
>> > -ifneq ($(shell which pandoc >/dev/null 2>&1 && echo found),found)
>> > +ifneq ($(shell which asciidoc >/dev/null 2>&1 && echo found),found)
>> > MARKDOWNREADY := "";
>> > endif
>> >
>> > diff --git a/scripts/kernel-doc b/scripts/kernel-doc
>> > index e01e74f15a22..cbfa4c03189e 100755
>> > --- a/scripts/kernel-doc
>> > +++ b/scripts/kernel-doc
>> > @@ -524,7 +524,7 @@ sub dump_doc_section {
>> > sub markdown_to_docbook {
>> > my $orig_content = $_[0];
>> >
>> > - my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "pandoc --columns=80 -f markdown -t docbook" );
>> > + my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "asciidoc -a 'newline=\\n' --no-header-footer --backend=docbook45 -" );
>> >
>> > print CHLD_IN "$orig_content";
>> > close(CHLD_IN);
>> > @@ -540,9 +540,9 @@ sub markdown_to_docbook {
>> > close(CHLD_ERR);
>> >
>> > if ($output_markdown_nopara) {
>> > - # pandoc insists in adding Main <para></para>, sometimes we
>> > - # want to remove them.
>> > - $content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm;
>> > + # asciidoc insists in adding Main <simpara></simpara>, sometimes
>> > + # we want to remove them.
>> > + $content =~ s:\A\s*<simpara>(.*)</simpara>\Z:$1:egsm;
>> > }
>> >
>> > return $content;
>> > @@ -605,6 +605,16 @@ sub output_highlight {
>> > # print STDERR "contents af:$contents\n";
>> > if ($use_markdown) {
>> > $contents = markdown_to_docbook($contents);
>> > +
>> > + # Compared to markdown asciidoc doesn't let through arbitrary xml
>> > + # markup. We need to un-escape the kerneldoc markup for functions,
>> > + # structures, ...
>> > + $contents =~ s/<quote>(\S*)<\/quote>/<quote>$1<\/quote>/g;
>> > + $contents =~ s/<constant>(\S*)<\/constant>/<constant>$1<\/constant>/g;
>> > + $contents =~ s/<structname>(\S*)<\/structname>/<structname>$1<\/structname>/g;
>> > + $contents =~ s/<parameter>(\S*)<\/parameter>/<parameter>$1<\/parameter>/g;
>> > + $contents =~ s/<function>(\S*)<\/function>/<function>$1<\/function>/g;
>> > + $contents =~ s/<envar>(\S*)<\/envar>/<envar>$1<\/envar>/g;
>> > }
>> >
>> > # strip whitespaces when generating html5
>>
>> --
>> Jani Nikula, Intel Open Source Technology Center
--
Jani Nikula, Intel Open Source Technology Center
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 5/5] drm: Enable markdown^Wasciidoc for gpu.tmpl
2015-11-25 17:07 ` [PATCH 5/5] drm: Enable markdown^Wasciidoc for gpu.tmpl Daniel Vetter
@ 2015-12-11 22:12 ` Jonathan Corbet
2015-12-12 11:13 ` Daniel Vetter
0 siblings, 1 reply; 23+ messages in thread
From: Jonathan Corbet @ 2015-12-11 22:12 UTC (permalink / raw)
To: Daniel Vetter
Cc: Graham Whaley, Intel Graphics Development, DRI Development,
Thomas Wood, Daniel Vetter, Dave Airlie
On Wed, 25 Nov 2015 18:07:59 +0100
Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
> Unfortunately the entire improved docbook project died at KS in a
> massive bikeshed. So we need to carry this around in drm private trees
> forever :(
I don't think that's an entirely helpful way to look at things, honestly.
Changing how the docs system works affects a lot of people, they're going
to have input on the matter.
And I sure hope it hasn't "died". The posting of these patches suggests
that perhaps it has not.
Anyway, I wanted to say that, my silence notwithstanding, I haven't just
dropped these. I had some hassles to deal with (replacing the entire LWN
server infrastructure, for example) but those are done; I hope to be able
to mess with this stuff a bit in the very near future.
Have the table-rendering issues that Graham talked about before gotten any
better with the newer scheme?
Thanks,
jon
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 5/5] drm: Enable markdown^Wasciidoc for gpu.tmpl
2015-12-11 22:12 ` Jonathan Corbet
@ 2015-12-12 11:13 ` Daniel Vetter
2016-01-12 1:12 ` Jonathan Corbet
0 siblings, 1 reply; 23+ messages in thread
From: Daniel Vetter @ 2015-12-12 11:13 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Graham Whaley, Daniel Vetter, Intel Graphics Development,
DRI Development, Thomas Wood, Dave Airlie, Daniel Vetter
On Fri, Dec 11, 2015 at 03:12:26PM -0700, Jonathan Corbet wrote:
> On Wed, 25 Nov 2015 18:07:59 +0100
> Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>
> > Unfortunately the entire improved docbook project died at KS in a
> > massive bikeshed. So we need to carry this around in drm private trees
> > forever :(
>
> I don't think that's an entirely helpful way to look at things, honestly.
> Changing how the docs system works affects a lot of people, they're going
> to have input on the matter.
>
> And I sure hope it hasn't "died". The posting of these patches suggests
> that perhaps it has not.
Hm, I think I fumbled the cc for the cover letter:
http://www.spinics.net/lists/intel-gfx/msg81351.html
In short it hasn't died, and 4.5 will have a few thousand more lines of
doc already employing asciidoc (and ofc also the other stuff that already
landed in 4.4). I just figured there's no way this could get it, and I'd
much rather improve the docs themselves than trying to convince core
kernel folks that this might be useful.
> Anyway, I wanted to say that, my silence notwithstanding, I haven't just
> dropped these. I had some hassles to deal with (replacing the entire LWN
> server infrastructure, for example) but those are done; I hope to be able
> to mess with this stuff a bit in the very near future.
>
> Have the table-rendering issues that Graham talked about before gotten any
> better with the newer scheme?
I haven't tackled the table conversion story yet, but one reason for me to
switch to asciidoc (besides that Dave thought it was a good idea) was the
better table support. I haven't tried, but it /should/ be powerful enough.
Another bonus is in-line figures as asciiart that get rendered to png. But
haven't done the integration work on that yet either.
Anyway things seem to work and I'm happy.
Thanks, Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 5/5] drm: Enable markdown^Wasciidoc for gpu.tmpl
2015-12-12 11:13 ` Daniel Vetter
@ 2016-01-12 1:12 ` Jonathan Corbet
2016-01-12 6:15 ` Jani Nikula
` (2 more replies)
0 siblings, 3 replies; 23+ messages in thread
From: Jonathan Corbet @ 2016-01-12 1:12 UTC (permalink / raw)
To: Daniel Vetter
Cc: Graham Whaley, Daniel Vetter, Intel Graphics Development,
DRI Development, Thomas Wood, Dave Airlie, Daniel Vetter
On Sat, 12 Dec 2015 12:13:45 +0100
Daniel Vetter <daniel@ffwll.ch> wrote:
> I just figured there's no way this could get it, and I'd
> much rather improve the docs themselves than trying to convince core
> kernel folks that this might be useful.
So I'm not quite sure why you figured that; I never said it, certainly.
I've been messing with it a bit, seems to work. I do still wish we could
consider alternatives, especially those that might simplify the toolchain
rather than complicating it. But it's clear that I'm not succeeding in
finding time to actually explore that idea; the contents of $EXCUSES are
good, but the end result is the same. And the patch fairy just isn't
coming through for me on this one.
In my mind, there's clearly no good that can come from (further) delaying
something that works in favor of an "it would be nice" that may never
even exist. So I'm currently thinking that I'll pull this into the docs
tree once the merge window is done, with the plan to push it for 4.6.
Then we can see if anybody screams.
That gives a couple of weeks for an updated patch set, should you have
one.
The build-time increase is painful in the extreme - about a factor of
three for a -j1 build, and that's with only one file using the feature.
It feels wrong, somehow, for the docs build to take longer than building
the kernel itself. Can we do something about that?
- How many of the comments actually use asciidoc features? Might there
be some possibility of detecting those in kernel-doc and skipping the
callout to asciidoc when it's not needed?
- Pandoc seems to do asciidoc. I still don't like the idea of depending
on it for this to work, but having the *option* to use it is fine. If
it's really that much faster (yes, Python startup is painful) then
maybe providing the option is worth it.
- All over the kernel we've seen that batching improves performance. It
would take a bit of work, but I bet kernel-doc could put together all
the snippets from one file, pass them through a single asciidoc
invocation, then split the results back apart. That would probably
eliminate the performance hit entirely.
None of that is a condition for pulling this stuff in, but can it be
looked into?
Thanks,
jon
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 5/5] drm: Enable markdown^Wasciidoc for gpu.tmpl
2016-01-12 1:12 ` Jonathan Corbet
@ 2016-01-12 6:15 ` Jani Nikula
2016-01-12 8:34 ` Daniel Vetter
2016-01-14 20:03 ` Jani Nikula
2 siblings, 0 replies; 23+ messages in thread
From: Jani Nikula @ 2016-01-12 6:15 UTC (permalink / raw)
To: Jonathan Corbet, Daniel Vetter
Cc: Graham Whaley, Daniel Vetter, Intel Graphics Development,
DRI Development, Thomas Wood, Daniel Vetter, Dave Airlie
On Tue, 12 Jan 2016, Jonathan Corbet <corbet@lwn.net> wrote:
> In my mind, there's clearly no good that can come from (further) delaying
> something that works in favor of an "it would be nice" that may never
> even exist. So I'm currently thinking that I'll pull this into the docs
> tree once the merge window is done, with the plan to push it for 4.6.
> Then we can see if anybody screams.
Must... resist... urge to bikeshed about the choice of markup...
> The build-time increase is painful in the extreme - about a factor of
> three for a -j1 build, and that's with only one file using the feature.
> It feels wrong, somehow, for the docs build to take longer than building
> the kernel itself. Can we do something about that?
"Holy big-O, batman. Asciidoc appears to be quadractically slow." [1]
Fortunately the same quote lead me to asciidoctor [2], which was maybe
twice as fast as asciidoc. An improvement, but could be much better.
BR,
Jani.
[1] https://twitter.com/marijnjh/status/473935469676216321
[2] http://asciidoctor.org/docs/asciidoc-asciidoctor-diffs/
--
Jani Nikula, Intel Open Source Technology Center
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 5/5] drm: Enable markdown^Wasciidoc for gpu.tmpl
2016-01-12 1:12 ` Jonathan Corbet
2016-01-12 6:15 ` Jani Nikula
@ 2016-01-12 8:34 ` Daniel Vetter
2016-01-12 11:06 ` Graham Whaley
2016-01-14 20:03 ` Jani Nikula
2 siblings, 1 reply; 23+ messages in thread
From: Daniel Vetter @ 2016-01-12 8:34 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Graham Whaley, Daniel Vetter, Intel Graphics Development,
DRI Development, Thomas Wood, Daniel Vetter, Dave Airlie
On Mon, Jan 11, 2016 at 06:12:12PM -0700, Jonathan Corbet wrote:
> On Sat, 12 Dec 2015 12:13:45 +0100
> Daniel Vetter <daniel@ffwll.ch> wrote:
>
> > I just figured there's no way this could get it, and I'd
> > much rather improve the docs themselves than trying to convince core
> > kernel folks that this might be useful.
>
> So I'm not quite sure why you figured that; I never said it, certainly.
To clarify this wasn't really my impression of your stance, but of the
overall room opinion when we had the discussion at KS. And then my main
goal here is to write great docs for drm (we have about 3k lines more docs
in 4.5 already), so that's why I dropped the ball on upstreaming. It
seemed unlikely to succeed, at least without some really seriuos effort at
convincing everyone, all while the drm docs for atomic haven't been in
good shape yet. Since then we had a few contributors of new atomic drivers
note on irc already that "oh cool, this is documented now". Overall really
just boils down to what I see as the most important things for drm ;-)
> I've been messing with it a bit, seems to work. I do still wish we could
> consider alternatives, especially those that might simplify the toolchain
> rather than complicating it. But it's clear that I'm not succeeding in
> finding time to actually explore that idea; the contents of $EXCUSES are
> good, but the end result is the same. And the patch fairy just isn't
> coming through for me on this one.
>
> In my mind, there's clearly no good that can come from (further) delaying
> something that works in favor of an "it would be nice" that may never
> even exist. So I'm currently thinking that I'll pull this into the docs
> tree once the merge window is done, with the plan to push it for 4.6.
> Then we can see if anybody screams.
>
> That gives a couple of weeks for an updated patch set, should you have
> one.
>
> The build-time increase is painful in the extreme - about a factor of
> three for a -j1 build, and that's with only one file using the feature.
> It feels wrong, somehow, for the docs build to take longer than building
> the kernel itself. Can we do something about that?
>
> - How many of the comments actually use asciidoc features? Might there
> be some possibility of detecting those in kernel-doc and skipping the
> callout to asciidoc when it's not needed?
I think that amounts to writing a partial parser (we use asciidoc for
tables, lists, links, formatting, code snippets by now already, someone
even thought of using the asciiart->png feature it has but it's not yet
wired up). I don't think it's feasible.
> - Pandoc seems to do asciidoc. I still don't like the idea of depending
> on it for this to work, but having the *option* to use it is fine. If
> it's really that much faster (yes, Python startup is painful) then
> maybe providing the option is worth it.
Hm, Dave asked me to convert to use python-based asciidoc insted of
haskell-based pandoc.
> - All over the kernel we've seen that batching improves performance. It
> would take a bit of work, but I bet kernel-doc could put together all
> the snippets from one file, pass them through a single asciidoc
> invocation, then split the results back apart. That would probably
> eliminate the performance hit entirely.
>
> None of that is a condition for pulling this stuff in, but can it be
> looked into?
Besides what Jani mention that asciidoctor should be a drop-in replacement
if installed it also seems possible to parallelize the call-out to
kernel-doc from docproc.c without too much effort. I hoped Jani would get
around to implement the asciidoctor support, and I'm hoping I can snipe
away some free sometimes the next few months to look at docproc.c more
seriously. This would kinda be a cool intern project, but atm we throw
them all at improving testing infrastructure ...
Anyway I'm of course still open to get this upstream, and I think a few
things should be polished (like the speed-up). But right now bandwidth on
my side isn't too plentiful. Maybe we should aim to have a few better
ideas (perhaps even for all of the docs stuff) for next KS and respin that
discussion?
Thanks, Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 5/5] drm: Enable markdown^Wasciidoc for gpu.tmpl
2016-01-12 8:34 ` Daniel Vetter
@ 2016-01-12 11:06 ` Graham Whaley
2016-01-12 17:43 ` Daniel Vetter
0 siblings, 1 reply; 23+ messages in thread
From: Graham Whaley @ 2016-01-12 11:06 UTC (permalink / raw)
To: Daniel Vetter, Jonathan Corbet
Cc: Daniel Vetter, Intel Graphics Development, DRI Development,
Thomas Wood, Dave Airlie, Daniel Vetter
On Tue, 2016-01-12 at 09:34 +0100, Daniel Vetter wrote:
> On Mon, Jan 11, 2016 at 06:12:12PM -0700, Jonathan Corbet wrote:
> > On Sat, 12 Dec 2015 12:13:45 +0100
> > Daniel Vetter <daniel@ffwll.ch> wrote:
> >
> > > I just figured there's no way this could get it, and I'd
> > > much rather improve the docs themselves than trying to convince
> > > core
> > > kernel folks that this might be useful.
> >
> > So I'm not quite sure why you figured that; I never said it,
> > certainly.
>
> To clarify this wasn't really my impression of your stance, but of
> the
> overall room opinion when we had the discussion at KS. And then my
> main
> goal here is to write great docs for drm (we have about 3k lines more
> docs
> in 4.5 already), so that's why I dropped the ball on upstreaming. It
> seemed unlikely to succeed, at least without some really seriuos
> effort at
> convincing everyone, all while the drm docs for atomic haven't been
> in
> good shape yet. Since then we had a few contributors of new atomic
> drivers
> note on irc already that "oh cool, this is documented now". Overall
> really
> just boils down to what I see as the most important things for drm ;
> -)
>
> > I've been messing with it a bit, seems to work. I do still wish we
> > could
> > consider alternatives, especially those that might simplify the
> > toolchain
> > rather than complicating it. But it's clear that I'm not
> > succeeding in
> > finding time to actually explore that idea; the contents of
> > $EXCUSES are
> > good, but the end result is the same. And the patch fairy just
> > isn't
> > coming through for me on this one.
> >
> > In my mind, there's clearly no good that can come from (further)
> > delaying
> > something that works in favor of an "it would be nice" that may
> > never
> > even exist. So I'm currently thinking that I'll pull this into the
> > docs
> > tree once the merge window is done, with the plan to push it for
> > 4.6.
> > Then we can see if anybody screams.
> >
> > That gives a couple of weeks for an updated patch set, should you
> > have
> > one.
> >
> > The build-time increase is painful in the extreme - about a factor
> > of
> > three for a -j1 build, and that's with only one file using the
> > feature.
> > It feels wrong, somehow, for the docs build to take longer than
> > building
> > the kernel itself. Can we do something about that?
> >
> > - How many of the comments actually use asciidoc features? Might
> > there
> > be some possibility of detecting those in kernel-doc and
> > skipping the
> > callout to asciidoc when it's not needed?
>
> I think that amounts to writing a partial parser (we use asciidoc for
> tables, lists, links, formatting, code snippets by now already,
> someone
> even thought of using the asciiart->png feature it has but it's not
> yet
> wired up). I don't think it's feasible.
>
> > - Pandoc seems to do asciidoc. I still don't like the idea of
> > depending
> > on it for this to work, but having the *option* to use it is
> > fine. If
> > it's really that much faster (yes, Python startup is painful)
> > then
> > maybe providing the option is worth it.
>
> Hm, Dave asked me to convert to use python-based asciidoc insted of
> haskell-based pandoc.
>
> > - All over the kernel we've seen that batching improves
> > performance. It
> > would take a bit of work, but I bet kernel-doc could put
> > together all
> > the snippets from one file, pass them through a single asciidoc
> > invocation, then split the results back apart. That would
> > probably
> > eliminate the performance hit entirely.
> >
> > None of that is a condition for pulling this stuff in, but can it
> > be
> > looked into?
>
> Besides what Jani mention that asciidoctor should be a drop-in
> replacement
> if installed it also seems possible to parallelize the call-out to
> kernel-doc from docproc.c without too much effort. I hoped Jani would
> get
> around to implement the asciidoctor support, and I'm hoping I can
> snipe
> away some free sometimes the next few months to look at docproc.c
> more
> seriously. This would kinda be a cool intern project, but atm we
> throw
> them all at improving testing infrastructure ...
>
> Anyway I'm of course still open to get this upstream, and I think a
> few
> things should be polished (like the speed-up). But right now
> bandwidth on
> my side isn't too plentiful. Maybe we should aim to have a few better
> ideas (perhaps even for all of the docs stuff) for next KS and respin
> that
> discussion?
I was just about to reply to the thread.... looking at the
linux.conf.au schedule it would seem that you are both attending and
presenting, and there appears to be some sort of Documentation mini
-summit on the Monday as well (not sure if that is the place for a
discussion though). I will be at LCA for the Wed-Fri. You may not have
to wait until the next KS?
Graham
>
> Thanks, Daniel
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 5/5] drm: Enable markdown^Wasciidoc for gpu.tmpl
2016-01-12 11:06 ` Graham Whaley
@ 2016-01-12 17:43 ` Daniel Vetter
0 siblings, 0 replies; 23+ messages in thread
From: Daniel Vetter @ 2016-01-12 17:43 UTC (permalink / raw)
To: Graham Whaley
Cc: Jonathan Corbet, Daniel Vetter, Intel Graphics Development,
DRI Development, Thomas Wood, Daniel Vetter, Dave Airlie
On Tue, Jan 12, 2016 at 11:06:17AM +0000, Graham Whaley wrote:
> On Tue, 2016-01-12 at 09:34 +0100, Daniel Vetter wrote:
> > On Mon, Jan 11, 2016 at 06:12:12PM -0700, Jonathan Corbet wrote:
> > > On Sat, 12 Dec 2015 12:13:45 +0100
> > > Daniel Vetter <daniel@ffwll.ch> wrote:
> > >
> > > > I just figured there's no way this could get it, and I'd
> > > > much rather improve the docs themselves than trying to convince
> > > > core
> > > > kernel folks that this might be useful.
> > >
> > > So I'm not quite sure why you figured that; I never said it,
> > > certainly.
> >
> > To clarify this wasn't really my impression of your stance, but of
> > the
> > overall room opinion when we had the discussion at KS. And then my
> > main
> > goal here is to write great docs for drm (we have about 3k lines more
> > docs
> > in 4.5 already), so that's why I dropped the ball on upstreaming. It
> > seemed unlikely to succeed, at least without some really seriuos
> > effort at
> > convincing everyone, all while the drm docs for atomic haven't been
> > in
> > good shape yet. Since then we had a few contributors of new atomic
> > drivers
> > note on irc already that "oh cool, this is documented now". Overall
> > really
> > just boils down to what I see as the most important things for drm ;
> > -)
> >
> > > I've been messing with it a bit, seems to work. I do still wish we
> > > could
> > > consider alternatives, especially those that might simplify the
> > > toolchain
> > > rather than complicating it. But it's clear that I'm not
> > > succeeding in
> > > finding time to actually explore that idea; the contents of
> > > $EXCUSES are
> > > good, but the end result is the same. And the patch fairy just
> > > isn't
> > > coming through for me on this one.
> > >
> > > In my mind, there's clearly no good that can come from (further)
> > > delaying
> > > something that works in favor of an "it would be nice" that may
> > > never
> > > even exist. So I'm currently thinking that I'll pull this into the
> > > docs
> > > tree once the merge window is done, with the plan to push it for
> > > 4.6.
> > > Then we can see if anybody screams.
> > >
> > > That gives a couple of weeks for an updated patch set, should you
> > > have
> > > one.
> > >
> > > The build-time increase is painful in the extreme - about a factor
> > > of
> > > three for a -j1 build, and that's with only one file using the
> > > feature.
> > > It feels wrong, somehow, for the docs build to take longer than
> > > building
> > > the kernel itself. Can we do something about that?
> > >
> > > - How many of the comments actually use asciidoc features? Might
> > > there
> > > be some possibility of detecting those in kernel-doc and
> > > skipping the
> > > callout to asciidoc when it's not needed?
> >
> > I think that amounts to writing a partial parser (we use asciidoc for
> > tables, lists, links, formatting, code snippets by now already,
> > someone
> > even thought of using the asciiart->png feature it has but it's not
> > yet
> > wired up). I don't think it's feasible.
> >
> > > - Pandoc seems to do asciidoc. I still don't like the idea of
> > > depending
> > > on it for this to work, but having the *option* to use it is
> > > fine. If
> > > it's really that much faster (yes, Python startup is painful)
> > > then
> > > maybe providing the option is worth it.
> >
> > Hm, Dave asked me to convert to use python-based asciidoc insted of
> > haskell-based pandoc.
> >
> > > - All over the kernel we've seen that batching improves
> > > performance. It
> > > would take a bit of work, but I bet kernel-doc could put
> > > together all
> > > the snippets from one file, pass them through a single asciidoc
> > > invocation, then split the results back apart. That would
> > > probably
> > > eliminate the performance hit entirely.
> > >
> > > None of that is a condition for pulling this stuff in, but can it
> > > be
> > > looked into?
> >
> > Besides what Jani mention that asciidoctor should be a drop-in
> > replacement
> > if installed it also seems possible to parallelize the call-out to
> > kernel-doc from docproc.c without too much effort. I hoped Jani would
> > get
> > around to implement the asciidoctor support, and I'm hoping I can
> > snipe
> > away some free sometimes the next few months to look at docproc.c
> > more
> > seriously. This would kinda be a cool intern project, but atm we
> > throw
> > them all at improving testing infrastructure ...
> >
> > Anyway I'm of course still open to get this upstream, and I think a
> > few
> > things should be polished (like the speed-up). But right now
> > bandwidth on
> > my side isn't too plentiful. Maybe we should aim to have a few better
> > ideas (perhaps even for all of the docs stuff) for next KS and respin
> > that
> > discussion?
>
> I was just about to reply to the thread.... looking at the
> linux.conf.au schedule it would seem that you are both attending and
> presenting, and there appears to be some sort of Documentation mini
> -summit on the Monday as well (not sure if that is the place for a
> discussion though). I will be at LCA for the Wed-Fri. You may not have
> to wait until the next KS?
Sounds like a great idea to pick this up at lca and toss around for some.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 5/5] drm: Enable markdown^Wasciidoc for gpu.tmpl
2016-01-12 1:12 ` Jonathan Corbet
2016-01-12 6:15 ` Jani Nikula
2016-01-12 8:34 ` Daniel Vetter
@ 2016-01-14 20:03 ` Jani Nikula
2016-01-14 20:18 ` Jonathan Corbet
2 siblings, 1 reply; 23+ messages in thread
From: Jani Nikula @ 2016-01-14 20:03 UTC (permalink / raw)
To: Jonathan Corbet, Daniel Vetter
Cc: Graham Whaley, Daniel Vetter, Intel Graphics Development,
DRI Development, Thomas Wood, Daniel Vetter, Dave Airlie
On Tue, 12 Jan 2016, Jonathan Corbet <corbet@lwn.net> wrote:
> On Sat, 12 Dec 2015 12:13:45 +0100
> Daniel Vetter <daniel@ffwll.ch> wrote:
>
>> I just figured there's no way this could get it, and I'd
>> much rather improve the docs themselves than trying to convince core
>> kernel folks that this might be useful.
>
> So I'm not quite sure why you figured that; I never said it, certainly.
>
> I've been messing with it a bit, seems to work. I do still wish we could
> consider alternatives, especially those that might simplify the toolchain
> rather than complicating it. But it's clear that I'm not succeeding in
> finding time to actually explore that idea; the contents of $EXCUSES are
> good, but the end result is the same. And the patch fairy just isn't
> coming through for me on this one.
>
> In my mind, there's clearly no good that can come from (further) delaying
> something that works in favor of an "it would be nice" that may never
> even exist. So I'm currently thinking that I'll pull this into the docs
> tree once the merge window is done, with the plan to push it for 4.6.
> Then we can see if anybody screams.
>
> That gives a couple of weeks for an updated patch set, should you have
> one.
>
> The build-time increase is painful in the extreme - about a factor of
> three for a -j1 build, and that's with only one file using the feature.
> It feels wrong, somehow, for the docs build to take longer than building
> the kernel itself. Can we do something about that?
>
> - How many of the comments actually use asciidoc features? Might there
> be some possibility of detecting those in kernel-doc and skipping the
> callout to asciidoc when it's not needed?
>
> - Pandoc seems to do asciidoc. I still don't like the idea of depending
> on it for this to work, but having the *option* to use it is fine. If
> it's really that much faster (yes, Python startup is painful) then
> maybe providing the option is worth it.
>
> - All over the kernel we've seen that batching improves performance. It
> would take a bit of work, but I bet kernel-doc could put together all
> the snippets from one file, pass them through a single asciidoc
> invocation, then split the results back apart. That would probably
> eliminate the performance hit entirely.
I had another look at the whole thing, inspired by your lwn article [1].
I am guessing the whole design of calling the markup tool (asciidoc or
pandoc or whatever) from kernel-doc for every documentation comment
comes from trying really hard to minimize changes to the rest of the
documentation system. From trying not to touch the DocBook parts so
much. And thus the documentation build works really hard to convert
thousands of markup snippets to DocBook and then again back to a bunch
of other formats like html or pdf.
What if we added support for some markup language as an alternative to
DocBook for the high level documentation? What if we taught kernel-doc
to output said markup natively, and included those generated pieces into
the high level documentation using the markup's own include directives?
At least AsciiDoc and reStructuredText support this.
kernel-doc already supports several output formats, DocBook, HTML, plain
text, man, whatnot. It should not be a big deal to add another. Even
though I don't do perl (and that's the main reason I've generally
steered clear anything kernel-doc) I toyed with adding reStructuredText
support, and I got some sensible output surprisingly quickly.
This would mean *one* kernel-doc invocation per source file included,
and *one* markup tool invocation per high level document. Additionally
there would have to be dependency generation from the high level
document to the included files.
As a bonus, one could write (and *read*!) the high level documentation
in a markup that's meant for humans. If needed, at least AsciiDoc
supports generating DocBook, which I suppose could be fed back to the
existing documentation pipeline.
> None of that is a condition for pulling this stuff in, but can it be
> looked into?
To be clear, I also don't want this idea to block the code that we have
now from being merged. This is a longer term thing anyway. But I'd like
to explore the alternatives.
Thoughts?
BR,
Jani.
[1] https://lwn.net/Articles/671496/
--
Jani Nikula, Intel Open Source Technology Center
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 5/5] drm: Enable markdown^Wasciidoc for gpu.tmpl
2016-01-14 20:03 ` Jani Nikula
@ 2016-01-14 20:18 ` Jonathan Corbet
2016-01-15 7:14 ` Jani Nikula
0 siblings, 1 reply; 23+ messages in thread
From: Jonathan Corbet @ 2016-01-14 20:18 UTC (permalink / raw)
To: Jani Nikula
Cc: Graham Whaley, Daniel Vetter, Intel Graphics Development,
DRI Development, Thomas Wood, Daniel Vetter, Dave Airlie
On Thu, 14 Jan 2016 22:03:26 +0200
Jani Nikula <jani.nikula@linux.intel.com> wrote:
> What if we added support for some markup language as an alternative to
> DocBook for the high level documentation? What if we taught kernel-doc
> to output said markup natively, and included those generated pieces into
> the high level documentation using the markup's own include directives?
> At least AsciiDoc and reStructuredText support this.
That is kind of what I've been thinking. I think we could dispense with
docbook entirely, simplifying the toolchain and making the template files
easier to read and edit. Something like that would also make it easy to
keep the regular .txt documents in a structured form and to integrate them
into the "formatted" documents if it makes sense.
Jani, want to join us at LCA and talk about it? :)
jon
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [PATCH 5/5] drm: Enable markdown^Wasciidoc for gpu.tmpl
2016-01-14 20:18 ` Jonathan Corbet
@ 2016-01-15 7:14 ` Jani Nikula
0 siblings, 0 replies; 23+ messages in thread
From: Jani Nikula @ 2016-01-15 7:14 UTC (permalink / raw)
To: Jonathan Corbet
Cc: Graham Whaley, Daniel Vetter, Intel Graphics Development,
DRI Development, Thomas Wood, Daniel Vetter, Dave Airlie
On Thu, 14 Jan 2016, Jonathan Corbet <corbet@lwn.net> wrote:
> On Thu, 14 Jan 2016 22:03:26 +0200
> Jani Nikula <jani.nikula@linux.intel.com> wrote:
>
>> What if we added support for some markup language as an alternative to
>> DocBook for the high level documentation? What if we taught kernel-doc
>> to output said markup natively, and included those generated pieces into
>> the high level documentation using the markup's own include directives?
>> At least AsciiDoc and reStructuredText support this.
>
> That is kind of what I've been thinking. I think we could dispense with
> docbook entirely, simplifying the toolchain and making the template files
> easier to read and edit. Something like that would also make it easy to
> keep the regular .txt documents in a structured form and to integrate them
> into the "formatted" documents if it makes sense.
Thanks, this is enough encouragement that maybe I'll toy around with
this a little more in my, uh, copious free time.
> Jani, want to join us at LCA and talk about it? :)
Heh, sure I'd *want* to...
BR,
Jani.
--
Jani Nikula, Intel Open Source Technology Center
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2016-01-15 7:14 UTC | newest]
Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-11-25 17:07 [PATCH 0/5] Better markup for GPU DocBook Daniel Vetter
2015-11-25 17:07 ` [PATCH 1/5] drm/doc: Convert to markdown Daniel Vetter
2015-11-25 17:07 ` Daniel Vetter
2015-11-25 17:07 ` [PATCH 2/5] scripts/kernel-doc: Adding infrastructure for markdown support Daniel Vetter
2015-11-25 17:07 ` Daniel Vetter
2015-11-25 17:07 ` [PATCH 3/5] scripts/kernel-doc: Improve Markdown results Daniel Vetter
2015-11-25 17:07 ` Daniel Vetter
2015-11-25 17:07 ` [PATCH 4/5] scripts/kernel-doc: Use asciidoc instead of markdown Daniel Vetter
2015-12-02 9:34 ` [PATCH] " Daniel Vetter
2015-12-02 13:33 ` Jani Nikula
2015-12-02 15:32 ` Daniel Vetter
2015-12-02 15:57 ` Jani Nikula
2015-11-25 17:07 ` [PATCH 5/5] drm: Enable markdown^Wasciidoc for gpu.tmpl Daniel Vetter
2015-12-11 22:12 ` Jonathan Corbet
2015-12-12 11:13 ` Daniel Vetter
2016-01-12 1:12 ` Jonathan Corbet
2016-01-12 6:15 ` Jani Nikula
2016-01-12 8:34 ` Daniel Vetter
2016-01-12 11:06 ` Graham Whaley
2016-01-12 17:43 ` Daniel Vetter
2016-01-14 20:03 ` Jani Nikula
2016-01-14 20:18 ` Jonathan Corbet
2016-01-15 7:14 ` Jani Nikula
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.