[RFC PATCH 1/2] scripts/kernel-doc: Auto-detect common code-blocks

[RFC PATCH 1/2] scripts/kernel-doc: Auto-detect common code-blocks

[Mauro Carvalho Chehab]

Comments that end with a comma and have certain keywords
should be presented as code-blocks

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
 scripts/kernel-doc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 0057d8eafcc1..8ce2d0f54369 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -803,7 +803,8 @@ sub output_highlight_rst {
 	#
 	if (! $in_literal) {
 	    $block .= $line . "\n";
-	    if (($line =~ /$sphinx_literal/) || ($line =~ /$sphinx_cblock/)) {
+	    if (($block =~ s/(code|example|artwork|flow|diagram)([^\:]*:)\n/$1$2:\n/) ||
+	        ($line =~ /$sphinx_literal/) || ($line =~ /$sphinx_cblock/)) {
 		$in_literal = 1;
 		$litprefix = "";
 		$output .= highlight_block($block);
-- 
2.17.0

[RFC PATCH 1/2] scripts/kernel-doc: Auto-detect common code-blocks

[Mauro Carvalho Chehab]

Comments that end with a comma and have certain keywords
should be presented as code-blocks

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
 scripts/kernel-doc | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 0057d8eafcc1..8ce2d0f54369 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -803,7 +803,8 @@ sub output_highlight_rst {
 	#
 	if (! $in_literal) {
 	    $block .= $line . "\n";
-	    if (($line =~ /$sphinx_literal/) || ($line =~ /$sphinx_cblock/)) {
+	    if (($block =~ s/(code|example|artwork|flow|diagram)([^\:]*:)\n/$1$2:\n/) ||
+	        ($line =~ /$sphinx_literal/) || ($line =~ /$sphinx_cblock/)) {
 		$in_literal = 1;
 		$litprefix = "";
 		$output .= highlight_block($block);
-- 
2.17.0


--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC PATCH 2/2] wait: add a keyword to indicate a diagram code block on a kernel-doc

[Mauro Carvalho Chehab]

When handling code-blocks, usual parsing should be disabled.
Indicate it by using the keyword disagram.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
 include/linux/wait.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/wait.h b/include/linux/wait.h
index d9f131ecf708..653814475ee9 100644
--- a/include/linux/wait.h
+++ b/include/linux/wait.h
@@ -101,7 +101,7 @@ init_waitqueue_func_entry(struct wait_queue_entry *wq_entry, wait_queue_func_t f
  * lead to sporadic and non-obvious failure.
  *
  * Use either while holding wait_queue_head::lock or when used for wakeups
- * with an extra smp_mb() like:
+ * with an extra smp_mb() as shown in this diagram:
  *
  *      CPU0 - waker                    CPU1 - waiter
  *
-- 
2.17.0

[RFC PATCH 2/2] wait: add a keyword to indicate a diagram code block on a kernel-doc

[Mauro Carvalho Chehab]

When handling code-blocks, usual parsing should be disabled.
Indicate it by using the keyword disagram.

Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
---
 include/linux/wait.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/linux/wait.h b/include/linux/wait.h
index d9f131ecf708..653814475ee9 100644
--- a/include/linux/wait.h
+++ b/include/linux/wait.h
@@ -101,7 +101,7 @@ init_waitqueue_func_entry(struct wait_queue_entry *wq_entry, wait_queue_func_t f
  * lead to sporadic and non-obvious failure.
  *
  * Use either while holding wait_queue_head::lock or when used for wakeups
- * with an extra smp_mb() like:
+ * with an extra smp_mb() as shown in this diagram:
  *
  *      CPU0 - waker                    CPU1 - waiter
  *
-- 
2.17.0


--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH 2/2] wait: add a keyword to indicate a diagram code block on a kernel-doc

[Randy Dunlap]

On 05/10/2018 07:38 AM, Mauro Carvalho Chehab wrote:
> When handling code-blocks, usual parsing should be disabled.
> Indicate it by using the keyword disagram.

                                   diagram.

Please add such keyword(s) to one of Documentation/doc-guide/{kernel-doc,sphinx}.rst.

> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> ---
>  include/linux/wait.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/linux/wait.h b/include/linux/wait.h
> index d9f131ecf708..653814475ee9 100644
> --- a/include/linux/wait.h
> +++ b/include/linux/wait.h
> @@ -101,7 +101,7 @@ init_waitqueue_func_entry(struct wait_queue_entry *wq_entry, wait_queue_func_t f
>   * lead to sporadic and non-obvious failure.
>   *
>   * Use either while holding wait_queue_head::lock or when used for wakeups
> - * with an extra smp_mb() like:
> + * with an extra smp_mb() as shown in this diagram:
>   *
>   *      CPU0 - waker                    CPU1 - waiter
>   *
> 


-- 
~Randy

Re: [RFC PATCH 2/2] wait: add a keyword to indicate a diagram code block on a kernel-doc

[Randy Dunlap]

On 05/10/2018 07:38 AM, Mauro Carvalho Chehab wrote:
> When handling code-blocks, usual parsing should be disabled.
> Indicate it by using the keyword disagram.

                                   diagram.

Please add such keyword(s) to one of Documentation/doc-guide/{kernel-doc,sphinx}.rst.

> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
> ---
>  include/linux/wait.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/linux/wait.h b/include/linux/wait.h
> index d9f131ecf708..653814475ee9 100644
> --- a/include/linux/wait.h
> +++ b/include/linux/wait.h
> @@ -101,7 +101,7 @@ init_waitqueue_func_entry(struct wait_queue_entry *wq_entry, wait_queue_func_t f
>   * lead to sporadic and non-obvious failure.
>   *
>   * Use either while holding wait_queue_head::lock or when used for wakeups
> - * with an extra smp_mb() like:
> + * with an extra smp_mb() as shown in this diagram:
>   *
>   *      CPU0 - waker                    CPU1 - waiter
>   *
> 


-- 
~Randy
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH 1/2] scripts/kernel-doc: Auto-detect common code-blocks

[Matthew Wilcox]

On Thu, May 10, 2018 at 10:38:16AM -0400, Mauro Carvalho Chehab wrote:
> Comments that end with a comma and have certain keywords
> should be presented as code-blocks

I think this is a bit fragile.  Why not just search for ':\n'?  Is
there ever a case where we want to write:

/**
 * foo is a bar:
 * wibble
 */
and have wibble not be a code-block?

Re: [RFC PATCH 1/2] scripts/kernel-doc: Auto-detect common code-blocks

[Matthew Wilcox]

On Thu, May 10, 2018 at 10:38:16AM -0400, Mauro Carvalho Chehab wrote:
> Comments that end with a comma and have certain keywords
> should be presented as code-blocks

I think this is a bit fragile.  Why not just search for ':\n'?  Is
there ever a case where we want to write:

/**
 * foo is a bar:
 * wibble
 */
and have wibble not be a code-block?

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH 1/2] scripts/kernel-doc: Auto-detect common code-blocks

[Jonathan Corbet]

On Thu, 10 May 2018 09:34:56 -0700
Matthew Wilcox <willy@infradead.org> wrote:

> I think this is a bit fragile.  Why not just search for ':\n'?  Is
> there ever a case where we want to write:
> 
> /**
>  * foo is a bar:
>  * wibble
>  */
> and have wibble not be a code-block?

Yeah, we might want to write something like:

 - Leading off a bulleted list

 1) or a numbered list

for example.  That's why I was thinking of looking for explicit markers
for such lists.

It'll take some playing around with to have a hope of getting right,
methinks.

jon

Re: [RFC PATCH 1/2] scripts/kernel-doc: Auto-detect common code-blocks

[Jonathan Corbet]

On Thu, 10 May 2018 09:34:56 -0700
Matthew Wilcox <willy@infradead.org> wrote:

> I think this is a bit fragile.  Why not just search for ':\n'?  Is
> there ever a case where we want to write:
> 
> /**
>  * foo is a bar:
>  * wibble
>  */
> and have wibble not be a code-block?

Yeah, we might want to write something like:

 - Leading off a bulleted list

 1) or a numbered list

for example.  That's why I was thinking of looking for explicit markers
for such lists.

It'll take some playing around with to have a hope of getting right,
methinks.

jon
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH 1/2] scripts/kernel-doc: Auto-detect common code-blocks

[Jani Nikula]

On Thu, 10 May 2018, Jonathan Corbet <corbet@lwn.net> wrote:
> On Thu, 10 May 2018 09:34:56 -0700
> Matthew Wilcox <willy@infradead.org> wrote:
>
>> I think this is a bit fragile.  Why not just search for ':\n'?  Is
>> there ever a case where we want to write:
>> 
>> /**
>>  * foo is a bar:
>>  * wibble
>>  */
>> and have wibble not be a code-block?
>
> Yeah, we might want to write something like:
>
>  - Leading off a bulleted list
>
>  1) or a numbered list
>
> for example.  That's why I was thinking of looking for explicit markers
> for such lists.
>
> It'll take some playing around with to have a hope of getting right,
> methinks.

We had serious trouble with the old DocBook toolchain because the tool
pipeline was so long, and each step had its own expectations and quirks.
For example, remember the escaping needed for xml in kernel-doc? Or tmpl
xml files being invalid xml because of the docproc directives? One of
the big benefits of the current toolchain is minimizing the amount of
special casing magic required.

The more we add custom syntax sugar in kernel-doc, the more we run the
risk of running into hard problems later on in the Sphinx phase. For
example, our own syntax preventing the use of legitimate rst syntax. And
now we get somewhat reasonable error messages from Sphinx when things go
wrong; we didn't get that when the impedance mismatches caused issues
with the old toolchain. They were hard to debug and mostly nobody even
bothered. We should work to reduce the amount of special processing in
kernel-doc, not the other way round.

The use of "::" is a valid and arguably rather non-invasive rst token
for indicating the following indented block is a literal block. Adding
heuristics (especially ones based on English language magic words) will
lead to bigger problems than it's trying to solve.


BR,
Jani.


-- 
Jani Nikula, Intel Open Source Technology Center

Re: [RFC PATCH 1/2] scripts/kernel-doc: Auto-detect common code-blocks

[Jani Nikula]

On Thu, 10 May 2018, Jonathan Corbet <corbet@lwn.net> wrote:
> On Thu, 10 May 2018 09:34:56 -0700
> Matthew Wilcox <willy@infradead.org> wrote:
>
>> I think this is a bit fragile.  Why not just search for ':\n'?  Is
>> there ever a case where we want to write:
>> 
>> /**
>>  * foo is a bar:
>>  * wibble
>>  */
>> and have wibble not be a code-block?
>
> Yeah, we might want to write something like:
>
>  - Leading off a bulleted list
>
>  1) or a numbered list
>
> for example.  That's why I was thinking of looking for explicit markers
> for such lists.
>
> It'll take some playing around with to have a hope of getting right,
> methinks.

We had serious trouble with the old DocBook toolchain because the tool
pipeline was so long, and each step had its own expectations and quirks.
For example, remember the escaping needed for xml in kernel-doc? Or tmpl
xml files being invalid xml because of the docproc directives? One of
the big benefits of the current toolchain is minimizing the amount of
special casing magic required.

The more we add custom syntax sugar in kernel-doc, the more we run the
risk of running into hard problems later on in the Sphinx phase. For
example, our own syntax preventing the use of legitimate rst syntax. And
now we get somewhat reasonable error messages from Sphinx when things go
wrong; we didn't get that when the impedance mismatches caused issues
with the old toolchain. They were hard to debug and mostly nobody even
bothered. We should work to reduce the amount of special processing in
kernel-doc, not the other way round.

The use of "::" is a valid and arguably rather non-invasive rst token
for indicating the following indented block is a literal block. Adding
heuristics (especially ones based on English language magic words) will
lead to bigger problems than it's trying to solve.


BR,
Jani.


-- 
Jani Nikula, Intel Open Source Technology Center
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH 1/2] scripts/kernel-doc: Auto-detect common code-blocks

[Daniel Vetter]

On Mon, May 14, 2018 at 1:03 PM, Jani Nikula
<jani.nikula@linux.intel.com> wrote:
> On Thu, 10 May 2018, Jonathan Corbet <corbet@lwn.net> wrote:
>> On Thu, 10 May 2018 09:34:56 -0700
>> Matthew Wilcox <willy@infradead.org> wrote:
>>
>>> I think this is a bit fragile.  Why not just search for ':\n'?  Is
>>> there ever a case where we want to write:
>>>
>>> /**
>>>  * foo is a bar:
>>>  * wibble
>>>  */
>>> and have wibble not be a code-block?
>>
>> Yeah, we might want to write something like:
>>
>>  - Leading off a bulleted list
>>
>>  1) or a numbered list
>>
>> for example.  That's why I was thinking of looking for explicit markers
>> for such lists.
>>
>> It'll take some playing around with to have a hope of getting right,
>> methinks.
>
> We had serious trouble with the old DocBook toolchain because the tool
> pipeline was so long, and each step had its own expectations and quirks.
> For example, remember the escaping needed for xml in kernel-doc? Or tmpl
> xml files being invalid xml because of the docproc directives? One of
> the big benefits of the current toolchain is minimizing the amount of
> special casing magic required.
>
> The more we add custom syntax sugar in kernel-doc, the more we run the
> risk of running into hard problems later on in the Sphinx phase. For
> example, our own syntax preventing the use of legitimate rst syntax. And
> now we get somewhat reasonable error messages from Sphinx when things go
> wrong; we didn't get that when the impedance mismatches caused issues
> with the old toolchain. They were hard to debug and mostly nobody even
> bothered. We should work to reduce the amount of special processing in
> kernel-doc, not the other way round.
>
> The use of "::" is a valid and arguably rather non-invasive rst token
> for indicating the following indented block is a literal block. Adding
> heuristics (especially ones based on English language magic words) will
> lead to bigger problems than it's trying to solve.

Very late +1 on this. I think :: plus a 4 char indent is a very
non-intrusive way to shut up sphinx, and most reasonable maintainers
can be convinced of the same. Some insists on making docs worse for
everyone for some ideal of plain text purity in comments
unfortunately.

And I also fully agree with Jani on maintainability headaches with
custom syntax and heuristics. It just makes debugging toolchain issues
extremely painful. What we have right now is about the bare minimu
that was required for a smooth transition, and no more.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: [RFC PATCH 1/2] scripts/kernel-doc: Auto-detect common code-blocks

[Daniel Vetter]

On Mon, May 14, 2018 at 1:03 PM, Jani Nikula
<jani.nikula@linux.intel.com> wrote:
> On Thu, 10 May 2018, Jonathan Corbet <corbet@lwn.net> wrote:
>> On Thu, 10 May 2018 09:34:56 -0700
>> Matthew Wilcox <willy@infradead.org> wrote:
>>
>>> I think this is a bit fragile.  Why not just search for ':\n'?  Is
>>> there ever a case where we want to write:
>>>
>>> /**
>>>  * foo is a bar:
>>>  * wibble
>>>  */
>>> and have wibble not be a code-block?
>>
>> Yeah, we might want to write something like:
>>
>>  - Leading off a bulleted list
>>
>>  1) or a numbered list
>>
>> for example.  That's why I was thinking of looking for explicit markers
>> for such lists.
>>
>> It'll take some playing around with to have a hope of getting right,
>> methinks.
>
> We had serious trouble with the old DocBook toolchain because the tool
> pipeline was so long, and each step had its own expectations and quirks.
> For example, remember the escaping needed for xml in kernel-doc? Or tmpl
> xml files being invalid xml because of the docproc directives? One of
> the big benefits of the current toolchain is minimizing the amount of
> special casing magic required.
>
> The more we add custom syntax sugar in kernel-doc, the more we run the
> risk of running into hard problems later on in the Sphinx phase. For
> example, our own syntax preventing the use of legitimate rst syntax. And
> now we get somewhat reasonable error messages from Sphinx when things go
> wrong; we didn't get that when the impedance mismatches caused issues
> with the old toolchain. They were hard to debug and mostly nobody even
> bothered. We should work to reduce the amount of special processing in
> kernel-doc, not the other way round.
>
> The use of "::" is a valid and arguably rather non-invasive rst token
> for indicating the following indented block is a literal block. Adding
> heuristics (especially ones based on English language magic words) will
> lead to bigger problems than it's trying to solve.

Very late +1 on this. I think :: plus a 4 char indent is a very
non-intrusive way to shut up sphinx, and most reasonable maintainers
can be convinced of the same. Some insists on making docs worse for
everyone for some ideal of plain text purity in comments
unfortunately.

And I also fully agree with Jani on maintainability headaches with
custom syntax and heuristics. It just makes debugging toolchain issues
extremely painful. What we have right now is about the bare minimu
that was required for a smooth transition, and no more.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html