Re: [PATCH] scripts/kernel-doc: Use asciidoc instead of markdown

From: Jani Nikula <jani.nikula@linux.intel.com>
To: Daniel Vetter <daniel@ffwll.ch>
Cc: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>,
	Jonathan Corbet <corbet@lwn.net>,
	Daniel Vetter <daniel.vetter@ffwll.ch>,
	DRI Development <dri-devel@lists.freedesktop.org>,
	Thomas Wood <thomas.wood@intel.com>,
	Daniel Vetter <daniel.vetter@intel.com>
Subject: Re: [PATCH] scripts/kernel-doc: Use asciidoc instead of markdown
Date: Wed, 02 Dec 2015 17:57:29 +0200	[thread overview]
Message-ID: <87610g3kt2.fsf@intel.com> (raw)
In-Reply-To: <20151202153220.GD17050@phenom.ffwll.local>

On Wed, 02 Dec 2015, Daniel Vetter <daniel@ffwll.ch> wrote:
> On Wed, Dec 02, 2015 at 03:33:43PM +0200, Jani Nikula wrote:
>> On Wed, 02 Dec 2015, Daniel Vetter <daniel.vetter@ffwll.ch> wrote:
>> > By popular demand.
>> 
>> Unpopular if you ask me.
>> 
>> Also, I think what you're trying to say is, use asciidoc the tool
>> instead of pandoc the tool, and as a side effect change the markup. Or
>> even, use a tool written in one language instead of a tool written in
>> another, and as a side effect change the markup.
>> 
>> In the end, I guess I would just like to have the technical and factual
>> reasons for this change in the commit message, instead of referring to
>> some mythical unverifiable popular demand.
>
> When discussing with Dave at KS whether we could do some kind of markup
> for drm only he asked me to use asciidoc instead of pandoc. So popular =
> 100% of the relevant maintainer (which is just Dave).

Please put that in the commit message then.

> Suprisingly the markup doesn't really change much since mostly asciidoc is
> just a (massive) extension of the markdown markup we've used. The changes
> are really just in how it's glued into kernel-doc. At least I didn't
> notice anything else yet. So yep, it's really just a tooling exchange that
> resulted in markup language changes.
>
> What I probably should mention is that asciidoc has better table support,
> which is something we've run into a bit with markdown already (our
> property tables ain't pretty).

I'll live.

BR,
Jani.

> -Daniel
>
>> 
>> BR,
>> Jani.
>> 
>> 
>> >
>> > This needs some adjustment/fixups after feeding snippets to asciidoc
>> > since compared to markdown asciidown escapes xml markup and doesn't
>> > just let it through.
>> >
>> > The other noticeable change is that build times increase a lot - we
>> > need to launch the markup process per-snippet, there's a few thousand
>> > of them and asciidoc (python) has a substantial higher overhead per
>> > invocation than pandoc (haskell).
>> >
>> > v2: More fine-tuning:
>> >
>> > - Use unixe newlines, not the default dos ones. Only results in
>> >   ugliness in the intermediate gpu.xml, but still.
>> >
>> > - Resurrect the hack to remove paragraphs for the one-line references.
>> >   Like markdown asciidoc insists to wrap everything.
>> >
>> > Cc: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
>> > Cc: Thomas Wood <thomas.wood@intel.com>
>> > Cc: Jonathan Corbet <corbet@lwn.net>
>> > Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
>> > ---
>> >  Documentation/DocBook/Makefile |  6 +++---
>> >  scripts/kernel-doc             | 18 ++++++++++++++----
>> >  2 files changed, 17 insertions(+), 7 deletions(-)
>> >
>> > diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile
>> > index 246ad38550e5..5335955c0de5 100644
>> > --- a/Documentation/DocBook/Makefile
>> > +++ b/Documentation/DocBook/Makefile
>> > @@ -104,8 +104,8 @@ define rule_docproc
>> >  endef
>> >  
>> >  %.xml: %.tmpl $(KERNELDOC) $(DOCPROC) $(KERNELDOCXMLREF) FORCE
>> > -	@(which pandoc > /dev/null 2>&1) || \
>> > -	(echo "*** To get propper documentation you need to install pandoc ***";)
>> > +	@(which asciidoc > /dev/null 2>&1) || \
>> > +	(echo "*** To get propper documentation you need to install asciidoc ***";)
>> >  	$(call if_changed_rule,docproc)
>> >  
>> >  # Tell kbuild to always build the programs
>> > @@ -116,7 +116,7 @@ notfoundtemplate = echo "*** You have to install docbook-utils or xmlto ***"; \
>> >  db2xtemplate = db2TYPE -o $(dir $@) $<
>> >  xmltotemplate = xmlto TYPE $(XMLTOFLAGS) -o $(dir $@) $<
>> >  
>> > -ifneq ($(shell which pandoc >/dev/null 2>&1 && echo found),found)
>> > +ifneq ($(shell which asciidoc >/dev/null 2>&1 && echo found),found)
>> >  	MARKDOWNREADY := "";
>> >  endif
>> >  
>> > diff --git a/scripts/kernel-doc b/scripts/kernel-doc
>> > index e01e74f15a22..cbfa4c03189e 100755
>> > --- a/scripts/kernel-doc
>> > +++ b/scripts/kernel-doc
>> > @@ -524,7 +524,7 @@ sub dump_doc_section {
>> >  sub markdown_to_docbook {
>> >  	my $orig_content = $_[0];
>> >  
>> > -	my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "pandoc  --columns=80 -f markdown -t docbook" );
>> > +	my $pid = open3( \*CHLD_IN, \*CHLD_OUT, \*CHLD_ERR, "asciidoc  -a 'newline=\\n' --no-header-footer --backend=docbook45 -" );
>> >  
>> >  	print CHLD_IN "$orig_content";
>> >  	close(CHLD_IN);
>> > @@ -540,9 +540,9 @@ sub markdown_to_docbook {
>> >  	close(CHLD_ERR);
>> >  
>> >  	if ($output_markdown_nopara) {
>> > -		# pandoc insists in adding Main <para></para>, sometimes we
>> > -		# want to remove them.
>> > -		$content =~ s:\A\s*<para>\s*\n(.*)\n</para>\Z$:$1:egsm;
>> > +		# asciidoc insists in adding Main <simpara></simpara>, sometimes
>> > +		# we want to remove them.
>> > +		$content =~ s:\A\s*<simpara>(.*)</simpara>\Z:$1:egsm;
>> >  	}
>> >  
>> >  	return $content;
>> > @@ -605,6 +605,16 @@ sub output_highlight {
>> >  #   print STDERR "contents af:$contents\n";
>> >      if ($use_markdown) {
>> >          $contents = markdown_to_docbook($contents);
>> > +
>> > +	# Compared to markdown asciidoc doesn't let through arbitrary xml
>> > +	# markup. We need to un-escape the kerneldoc markup for functions,
>> > +	# structures, ...
>> > +	$contents =~ s/&lt;quote&gt;(\S*)&lt;\/quote&gt;/<quote>$1<\/quote>/g;
>> > +	$contents =~ s/&lt;constant&gt;(\S*)&lt;\/constant&gt;/<constant>$1<\/constant>/g;
>> > +	$contents =~ s/&lt;structname&gt;(\S*)&lt;\/structname&gt;/<structname>$1<\/structname>/g;
>> > +	$contents =~ s/&lt;parameter&gt;(\S*)&lt;\/parameter&gt;/<parameter>$1<\/parameter>/g;
>> > +	$contents =~ s/&lt;function&gt;(\S*)&lt;\/function&gt;/<function>$1<\/function>/g;
>> > +	$contents =~ s/&lt;envar&gt;(\S*)&lt;\/envar&gt;/<envar>$1<\/envar>/g;
>> >      }
>> >  
>> >  #   strip whitespaces when generating html5
>> 
>> -- 
>> Jani Nikula, Intel Open Source Technology Center

-- 
Jani Nikula, Intel Open Source Technology Center
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel