[PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

[PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

[Danilo Cesar Lemes de Paula]

Describing arguments at top of a struct definition works fine
for small/medium size structs, but it definitely doesn't work well
for struct with a huge list of elements.

Keeping the arguments list inside the struct body makes it easier
to maintain the documentation.
ie:
/**
 * struct my_struct - short description
 * @a: first member
 * @b: second member
 *
 * Longer description
 */
struct my_struct {
    int a;
    int b;
    /**
     * @c: This is longer description of C
     *
     * You can use paragraphs to describe arguments
     * using this method.
     */
    int c;
};

This patch allows the use of this kind of syntax. Only one argument
per comment and user can use how many paragraphs he needs. It should
start with /**, which is already being used by kernel-doc. If those
comment doesn't follow those rules, it will be ignored.

Signed-off-by: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Stephan Mueller <smueller@chronox.de>
Cc: Michal Marek <mmarek@suse.cz>
Cc: linux-kernel@vger.kernel.org
Cc: linux-doc@vger.kernel.org
Cc: intel-gfx <intel-gfx@lists.freedesktop.org>
Cc: dri-devel <dri-devel@lists.freedesktop.org>
---
 scripts/kernel-doc | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 78 insertions(+), 2 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 9922e66..9bfa8b9 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -133,6 +133,30 @@ use strict;
 #
 # All descriptions can be multiline, except the short function description.
 #
+# For really longs structs, you can also describe arguments inside the
+# body of the struct.
+# eg.
+# /**
+#  * struct my_struct - short description
+#  * @a: first member
+#  * @b: second member
+#  *
+#  * Longer description
+#  */
+# struct my_struct {
+#     int a;
+#     int b;
+#     /**
+#      * @c: This is longer description of C
+#      *
+#      * You can use paragraphs to describe arguments
+#      * using this method.
+#      */
+#     int c;
+# };
+#
+# This should be use for arguments only.
+#
 # You can also add additional sections. When documenting kernel functions you
 # should document the "Context:" of the function, e.g. whether the functions
 # can be called form interrupts. Unlike other sections you can end it with an
@@ -287,9 +311,19 @@ my $lineprefix="";
 # 2 - scanning field start.
 # 3 - scanning prototype.
 # 4 - documentation block
+# 5 - gathering documentation outside main block
 my $state;
 my $in_doc_sect;
 
+# Split Doc State
+# 0 - Invalid (Before start or after finish)
+# 1 - Is started (the /** was found inside a struct)
+# 2 - The @parameter header was found, start accepting multi paragraph text.
+# 3 - Finished (the */ was found)
+# 4 - Error - Comment without header was found. Spit a warning as it's not
+#     proper kernel-doc and ignore the rest.
+my $split_doc_state;
+
 #declaration types: can be
 # 'function', 'struct', 'union', 'enum', 'typedef'
 my $decl_type;
@@ -304,6 +338,9 @@ my $doc_decl = $doc_com . '(\w+)';
 my $doc_sect = $doc_com . '([' . $doc_special . ']?[\w\s]+):(.*)';
 my $doc_content = $doc_com_body . '(.*)';
 my $doc_block = $doc_com . 'DOC:\s*(.*)?';
+my $doc_split_start = '^\s*/\*\*\s*$';
+my $doc_split_sect = '\s*\*\s*(@[\w\s]+):(.*)';
+my $doc_split_end = '^\s*\*/\s*$';
 
 my %constants;
 my %parameterdescs;
@@ -2181,6 +2218,7 @@ sub reset_state {
     $prototype = "";
 
     $state = 0;
+    $split_doc_state = 0;
 }
 
 sub tracepoint_munge($) {
@@ -2453,7 +2491,6 @@ sub process_file($) {
 		}
 		$section = $newsection;
 	    } elsif (/$doc_end/) {
-
 		if (($contents ne "") && ($contents ne "\n")) {
 		    dump_section($file, $section, xml_escape($contents));
 		    $section = $section_default;
@@ -2494,8 +2531,47 @@ sub process_file($) {
 		print STDERR "Warning(${file}:$.): bad line: $_";
 		++$warnings;
 	    }
+	} elsif ($state == 5) { # scanning for split parameters
+
+	    # First line (state 1) needs to be a @parameter
+	    if ($split_doc_state == 1 && /$doc_split_sect/o) {
+	        $section = $1;
+	        $contents = $2;
+	        if ($contents ne "") {
+	    	    while ((substr($contents, 0, 1) eq " ") ||
+	                    substr($contents, 0, 1) eq "\t") {
+	    	        $contents = substr($contents, 1);
+	            }
+	            $contents .= "\n";
+	        }
+	        $split_doc_state = 2;
+
+	    # End commend */
+	    } elsif (/$doc_split_end/) {
+	        if (($contents ne "") && ($contents ne "\n")) {
+	            dump_section($file, $section, xml_escape($contents));
+	            $section = $section_default;
+	            $contents = "";
+	        }
+	        $state = 3;
+	        $split_doc_state = 0;
+
+	    # Regular text
+	    } elsif (/$doc_content/) {
+	    	if ($split_doc_state == 2) {
+	    	    $contents .= $1 . "\n";
+	    	} elsif ($split_doc_state == 1) {
+	    	    $split_doc_state = 4;
+	    	    print STDERR "Warning(${file}:$.): ";
+	    	    print STDERR "Incorrect use of kernel-doc format: $_";
+	    	    ++$warnings;
+	    	}
+	    }
 	} elsif ($state == 3) {	# scanning for function '{' (end of prototype)
-	    if ($decl_type eq 'function') {
+	    if (/$doc_split_start/) {
+	    	    $state = 5;
+	    	    $split_doc_state = 1;
+	    } elsif ($decl_type eq 'function') {
 		process_state3_function($_, $file);
 	    } else {
 		process_state3_type($_, $file);
-- 
2.4.6

Re: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

[Jonathan Corbet]

On Fri, 31 Jul 2015 18:06:45 -0300
Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk> wrote:

> Describing arguments at top of a struct definition works fine
> for small/medium size structs, but it definitely doesn't work well
> for struct with a huge list of elements.
> 
> Keeping the arguments list inside the struct body makes it easier
> to maintain the documentation.

Interesting approach.  I think it could make sense, but I fear pushback
from a subset of maintainers refusing to accept this mode.  I wonder what
it would take to get a consensus on allowing these in-struct comments?

I'm wondering if we need a kernel summit session on commenting
conventions, markdown-in-kerneldoc, etc?  Maybe I'll stick a proposal out
there.

Thanks,

jon

Re: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

[Laurent Pinchart]

Hi Jon,

On Saturday 01 August 2015 13:22:10 Jonathan Corbet wrote:
> On Fri, 31 Jul 2015 18:06:45 -0300 Danilo Cesar Lemes de Paula wrote:
> > Describing arguments at top of a struct definition works fine
> > for small/medium size structs, but it definitely doesn't work well
> > for struct with a huge list of elements.
> > 
> > Keeping the arguments list inside the struct body makes it easier
> > to maintain the documentation.
> 
> Interesting approach.  I think it could make sense, but I fear pushback
> from a subset of maintainers refusing to accept this mode.  I wonder what
> it would take to get a consensus on allowing these in-struct comments?
> 
> I'm wondering if we need a kernel summit session on commenting
> conventions, markdown-in-kerneldoc, etc?  Maybe I'll stick a proposal out
> there.

The topic of documentation has been raised in the mail thread of the kernel 
recruitment proposal for the kernel summit. I believe it's an important one, 
and I will be very interested in contributing if it is discussed as a separate 
topic.

-- 
Regards,

Laurent Pinchart

Re: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

[Daniel Vetter]

On Sat, Aug 01, 2015 at 01:22:10PM +0200, Jonathan Corbet wrote:
> On Fri, 31 Jul 2015 18:06:45 -0300
> Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk> wrote:
> 
> > Describing arguments at top of a struct definition works fine
> > for small/medium size structs, but it definitely doesn't work well
> > for struct with a huge list of elements.
> > 
> > Keeping the arguments list inside the struct body makes it easier
> > to maintain the documentation.
> 
> Interesting approach.  I think it could make sense, but I fear pushback
> from a subset of maintainers refusing to accept this mode.  I wonder what
> it would take to get a consensus on allowing these in-struct comments?

At least in drm we have a lot of such comments (as non-kerneldoc) right
above struct members to explain some details. Common examples are:
- locks, with a description of what they protect and maybe also how they
  nest.
- vfunc ops structs, with a per-function description of what each hook
  does.
- tricky stuff which can't be described in one sentence only.

So it's not just huge structs by number of members, but huge by number of
comment lines. Trying to stuff that all into the top kerneldoc comment
means that it's much harder to jump to the right comment, and it's also
easier to ignore the comments (since it e.g. won't show up in the diff
context).

The current approach at least in drm is to duplicate comments and that
just results in inconsistency.
 
> I'm wondering if we need a kernel summit session on commenting
> conventions, markdown-in-kerneldoc, etc?  Maybe I'll stick a proposal out
> there.

Might be useful, but I'm not sure how many people really would actively
work on improving the tooling. The only comment I've seen is to maybe use
gtkdoc, but that would be a pain since it's slightly incompatible with
kerneldoc.

And it's the improved tooling I really need for my long-term plan to get
solid docs for drm & drm/i915. Next step is to start building a proper doc
writer team to make all the bits we already have into a consistent hole
(and nag developers to fill in the areas still undocumented). For that
I've already pulled Danilo's patches into the drm-intel integration tree
and I plan to use them for any further drm kerneldoc I write since we
really need them.

Thanks, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

[Jonathan Corbet]

On Mon, 3 Aug 2015 10:23:19 +0200
Daniel Vetter <daniel@ffwll.ch> wrote:

> > I'm wondering if we need a kernel summit session on commenting
> > conventions, markdown-in-kerneldoc, etc?  Maybe I'll stick a proposal out
> > there.  
> 
> Might be useful, but I'm not sure how many people really would actively
> work on improving the tooling. The only comment I've seen is to maybe use
> gtkdoc, but that would be a pain since it's slightly incompatible with
> kerneldoc.

The idea was to get a sense for what sort of improvements would be
useful, to begin with.  But my attempt to start a discussion on the
kernel summit list appears to have hit the ground pretty hard; I guess
that means I have free rein :)

I expect I'll apply the struct-args doc patch in the fairly near future.
Then we'll see if others complain when patches using it start to show up,
but the feature itself shouldn't break anything.  I'm *really* hoping to
take a hard look at Danilo's stuff for a 4.3 merge as well.  It should be
possible, but there's real-world obnoxiousness that is doing its best to
get in the way.

jon

Re: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

[Daniel Vetter]

On Mon, Aug 03, 2015 at 08:37:41AM -0600, Jonathan Corbet wrote:
> On Mon, 3 Aug 2015 10:23:19 +0200
> Daniel Vetter <daniel@ffwll.ch> wrote:
> 
> > > I'm wondering if we need a kernel summit session on commenting
> > > conventions, markdown-in-kerneldoc, etc?  Maybe I'll stick a proposal out
> > > there.  
> > 
> > Might be useful, but I'm not sure how many people really would actively
> > work on improving the tooling. The only comment I've seen is to maybe use
> > gtkdoc, but that would be a pain since it's slightly incompatible with
> > kerneldoc.
> 
> The idea was to get a sense for what sort of improvements would be
> useful, to begin with.  But my attempt to start a discussion on the
> kernel summit list appears to have hit the ground pretty hard; I guess
> that means I have free rein :)

Wrt feature wishlists the 3 things Danilo has worked on thus far
(hyperlinks, markdown and inline struct member kerneldoc) are really the
things I'd like to have. Of course there's room for some more
prettification, but I think that would better fit as improvements to
pandoc. One example is more flexible table handling with row/column
spanning - currently pandoc doesn't handle that in the docbook converter.

> I expect I'll apply the struct-args doc patch in the fairly near future.
> Then we'll see if others complain when patches using it start to show up,
> but the feature itself shouldn't break anything.  I'm *really* hoping to
> take a hard look at Danilo's stuff for a 4.3 merge as well.  It should be
> possible, but there's real-world obnoxiousness that is doing its best to
> get in the way.

Awesome. Missing 4.3 wouldn't be a big deal for i915 really since drm
feature freeze should happen around -rc5 anyway, so everything new I pull
in will be for 4.4 only. But getting it in early always helps, just in
case there's something unexpected.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

[Randy Dunlap]

On 07/31/15 14:06, Danilo Cesar Lemes de Paula wrote:
> Describing arguments at top of a struct definition works fine
> for small/medium size structs, but it definitely doesn't work well
> for struct with a huge list of elements.
> 
> Keeping the arguments list inside the struct body makes it easier
> to maintain the documentation.
> ie:
> /**
>  * struct my_struct - short description
>  * @a: first member
>  * @b: second member
>  *
>  * Longer description
>  */
> struct my_struct {
>     int a;
>     int b;
>     /**
>      * @c: This is longer description of C
>      *
>      * You can use paragraphs to describe arguments
>      * using this method.
>      */
>     int c;
> };
> 
> This patch allows the use of this kind of syntax. Only one argument
> per comment and user can use how many paragraphs he needs. It should
> start with /**, which is already being used by kernel-doc. If those
> comment doesn't follow those rules, it will be ignored.
> 
> Signed-off-by: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
> Cc: Randy Dunlap <rdunlap@infradead.org>
> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: Herbert Xu <herbert@gondor.apana.org.au>
> Cc: Stephan Mueller <smueller@chronox.de>
> Cc: Michal Marek <mmarek@suse.cz>
> Cc: linux-kernel@vger.kernel.org
> Cc: linux-doc@vger.kernel.org
> Cc: intel-gfx <intel-gfx@lists.freedesktop.org>
> Cc: dri-devel <dri-devel@lists.freedesktop.org>
> ---
>  scripts/kernel-doc | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 78 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/kernel-doc b/scripts/kernel-doc
> index 9922e66..9bfa8b9 100755
> --- a/scripts/kernel-doc
> +++ b/scripts/kernel-doc
> @@ -133,6 +133,30 @@ use strict;
>  #
>  # All descriptions can be multiline, except the short function description.
>  #
> +# For really longs structs, you can also describe arguments inside the
> +# body of the struct.
> +# eg.
> +# /**
> +#  * struct my_struct - short description
> +#  * @a: first member
> +#  * @b: second member
> +#  *
> +#  * Longer description
> +#  */
> +# struct my_struct {
> +#     int a;
> +#     int b;
> +#     /**
> +#      * @c: This is longer description of C
> +#      *
> +#      * You can use paragraphs to describe arguments
> +#      * using this method.
> +#      */
> +#     int c;
> +# };
> +#
> +# This should be use for arguments only.

                    used

What are "arguments" in this context?  do you mean struct fields or members?

> +#
>  # You can also add additional sections. When documenting kernel functions you
>  # should document the "Context:" of the function, e.g. whether the functions
>  # can be called form interrupts. Unlike other sections you can end it with an
> @@ -287,9 +311,19 @@ my $lineprefix="";
>  # 2 - scanning field start.
>  # 3 - scanning prototype.
>  # 4 - documentation block
> +# 5 - gathering documentation outside main block
>  my $state;
>  my $in_doc_sect;
>  
> +# Split Doc State
> +# 0 - Invalid (Before start or after finish)
> +# 1 - Is started (the /** was found inside a struct)
> +# 2 - The @parameter header was found, start accepting multi paragraph text.
> +# 3 - Finished (the */ was found)
> +# 4 - Error - Comment without header was found. Spit a warning as it's not
> +#     proper kernel-doc and ignore the rest.
> +my $split_doc_state;
> +
>  #declaration types: can be
>  # 'function', 'struct', 'union', 'enum', 'typedef'
>  my $decl_type;
> @@ -304,6 +338,9 @@ my $doc_decl = $doc_com . '(\w+)';
>  my $doc_sect = $doc_com . '([' . $doc_special . ']?[\w\s]+):(.*)';
>  my $doc_content = $doc_com_body . '(.*)';
>  my $doc_block = $doc_com . 'DOC:\s*(.*)?';
> +my $doc_split_start = '^\s*/\*\*\s*$';
> +my $doc_split_sect = '\s*\*\s*(@[\w\s]+):(.*)';
> +my $doc_split_end = '^\s*\*/\s*$';
>  
>  my %constants;
>  my %parameterdescs;
> @@ -2181,6 +2218,7 @@ sub reset_state {
>      $prototype = "";
>  
>      $state = 0;
> +    $split_doc_state = 0;
>  }
>  
>  sub tracepoint_munge($) {
> @@ -2453,7 +2491,6 @@ sub process_file($) {
>  		}
>  		$section = $newsection;
>  	    } elsif (/$doc_end/) {
> -
>  		if (($contents ne "") && ($contents ne "\n")) {
>  		    dump_section($file, $section, xml_escape($contents));
>  		    $section = $section_default;
> @@ -2494,8 +2531,47 @@ sub process_file($) {
>  		print STDERR "Warning(${file}:$.): bad line: $_";
>  		++$warnings;
>  	    }
> +	} elsif ($state == 5) { # scanning for split parameters
> +
> +	    # First line (state 1) needs to be a @parameter
> +	    if ($split_doc_state == 1 && /$doc_split_sect/o) {
> +	        $section = $1;
> +	        $contents = $2;
> +	        if ($contents ne "") {
> +	    	    while ((substr($contents, 0, 1) eq " ") ||
> +	                    substr($contents, 0, 1) eq "\t") {
> +	    	        $contents = substr($contents, 1);
> +	            }
> +	            $contents .= "\n";
> +	        }
> +	        $split_doc_state = 2;
> +
> +	    # End commend */

	          comment

> +	    } elsif (/$doc_split_end/) {
> +	        if (($contents ne "") && ($contents ne "\n")) {
> +	            dump_section($file, $section, xml_escape($contents));
> +	            $section = $section_default;
> +	            $contents = "";
> +	        }
> +	        $state = 3;
> +	        $split_doc_state = 0;
> +

-- 
~Randy

Re: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

[Danilo Cesar Lemes de Paula]

On 08/03/2015 12:59 PM, Randy Dunlap wrote:
> On 07/31/15 14:06, Danilo Cesar Lemes de Paula wrote:
>> Describing arguments at top of a struct definition works fine
>> for small/medium size structs, but it definitely doesn't work well
>> for struct with a huge list of elements.
>>
>> Keeping the arguments list inside the struct body makes it easier
>> to maintain the documentation.
>> ie:
>> /**
>>  * struct my_struct - short description
>>  * @a: first member
>>  * @b: second member
>>  *
>>  * Longer description
>>  */
>> struct my_struct {
>>     int a;
>>     int b;
>>     /**
>>      * @c: This is longer description of C
>>      *
>>      * You can use paragraphs to describe arguments
>>      * using this method.
>>      */
>>     int c;
>> };
>>
>> This patch allows the use of this kind of syntax. Only one argument
>> per comment and user can use how many paragraphs he needs. It should
>> start with /**, which is already being used by kernel-doc. If those
>> comment doesn't follow those rules, it will be ignored.
>>
>> Signed-off-by: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
>> Cc: Randy Dunlap <rdunlap@infradead.org>
>> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
>> Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
>> Cc: Jonathan Corbet <corbet@lwn.net>
>> Cc: Herbert Xu <herbert@gondor.apana.org.au>
>> Cc: Stephan Mueller <smueller@chronox.de>
>> Cc: Michal Marek <mmarek@suse.cz>
>> Cc: linux-kernel@vger.kernel.org
>> Cc: linux-doc@vger.kernel.org
>> Cc: intel-gfx <intel-gfx@lists.freedesktop.org>
>> Cc: dri-devel <dri-devel@lists.freedesktop.org>
>> ---
>>  scripts/kernel-doc | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++--
>>  1 file changed, 78 insertions(+), 2 deletions(-)
>>
>> diff --git a/scripts/kernel-doc b/scripts/kernel-doc
>> index 9922e66..9bfa8b9 100755
>> --- a/scripts/kernel-doc
>> +++ b/scripts/kernel-doc
>> @@ -133,6 +133,30 @@ use strict;
>>  #
>>  # All descriptions can be multiline, except the short function description.
>>  #
>> +# For really longs structs, you can also describe arguments inside the
>> +# body of the struct.
>> +# eg.
>> +# /**
>> +#  * struct my_struct - short description
>> +#  * @a: first member
>> +#  * @b: second member
>> +#  *
>> +#  * Longer description
>> +#  */
>> +# struct my_struct {
>> +#     int a;
>> +#     int b;
>> +#     /**
>> +#      * @c: This is longer description of C
>> +#      *
>> +#      * You can use paragraphs to describe arguments
>> +#      * using this method.
>> +#      */
>> +#     int c;
>> +# };
>> +#
>> +# This should be use for arguments only.
> 
>                     used
> 
> What are "arguments" in this context?  do you mean struct fields or members?

Yes. I can change the text if you want to.

> 
>> +#
>>  # You can also add additional sections. When documenting kernel functions you
>>  # should document the "Context:" of the function, e.g. whether the functions
>>  # can be called form interrupts. Unlike other sections you can end it with an
>> @@ -287,9 +311,19 @@ my $lineprefix="";
>>  # 2 - scanning field start.
>>  # 3 - scanning prototype.
>>  # 4 - documentation block
>> +# 5 - gathering documentation outside main block
>>  my $state;
>>  my $in_doc_sect;
>>  
>> +# Split Doc State
>> +# 0 - Invalid (Before start or after finish)
>> +# 1 - Is started (the /** was found inside a struct)
>> +# 2 - The @parameter header was found, start accepting multi paragraph text.
>> +# 3 - Finished (the */ was found)
>> +# 4 - Error - Comment without header was found. Spit a warning as it's not
>> +#     proper kernel-doc and ignore the rest.
>> +my $split_doc_state;
>> +
>>  #declaration types: can be
>>  # 'function', 'struct', 'union', 'enum', 'typedef'
>>  my $decl_type;
>> @@ -304,6 +338,9 @@ my $doc_decl = $doc_com . '(\w+)';
>>  my $doc_sect = $doc_com . '([' . $doc_special . ']?[\w\s]+):(.*)';
>>  my $doc_content = $doc_com_body . '(.*)';
>>  my $doc_block = $doc_com . 'DOC:\s*(.*)?';
>> +my $doc_split_start = '^\s*/\*\*\s*$';
>> +my $doc_split_sect = '\s*\*\s*(@[\w\s]+):(.*)';
>> +my $doc_split_end = '^\s*\*/\s*$';
>>  
>>  my %constants;
>>  my %parameterdescs;
>> @@ -2181,6 +2218,7 @@ sub reset_state {
>>      $prototype = "";
>>  
>>      $state = 0;
>> +    $split_doc_state = 0;
>>  }
>>  
>>  sub tracepoint_munge($) {
>> @@ -2453,7 +2491,6 @@ sub process_file($) {
>>  		}
>>  		$section = $newsection;
>>  	    } elsif (/$doc_end/) {
>> -
>>  		if (($contents ne "") && ($contents ne "\n")) {
>>  		    dump_section($file, $section, xml_escape($contents));
>>  		    $section = $section_default;
>> @@ -2494,8 +2531,47 @@ sub process_file($) {
>>  		print STDERR "Warning(${file}:$.): bad line: $_";
>>  		++$warnings;
>>  	    }
>> +	} elsif ($state == 5) { # scanning for split parameters
>> +
>> +	    # First line (state 1) needs to be a @parameter
>> +	    if ($split_doc_state == 1 && /$doc_split_sect/o) {
>> +	        $section = $1;
>> +	        $contents = $2;
>> +	        if ($contents ne "") {
>> +	    	    while ((substr($contents, 0, 1) eq " ") ||
>> +	                    substr($contents, 0, 1) eq "\t") {
>> +	    	        $contents = substr($contents, 1);
>> +	            }
>> +	            $contents .= "\n";
>> +	        }
>> +	        $split_doc_state = 2;
>> +
>> +	    # End commend */
> 
> 	          comment

Fixed in my next v2.

Thanks

> 
>> +	    } elsif (/$doc_split_end/) {
>> +	        if (($contents ne "") && ($contents ne "\n")) {
>> +	            dump_section($file, $section, xml_escape($contents));
>> +	            $section = $section_default;
>> +	            $contents = "";
>> +	        }
>> +	        $state = 3;
>> +	        $split_doc_state = 0;
>> +
> 

Re: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

[Daniel Vetter]

On Fri, Jul 31, 2015 at 06:06:45PM -0300, Danilo Cesar Lemes de Paula wrote:
> Describing arguments at top of a struct definition works fine
> for small/medium size structs, but it definitely doesn't work well
> for struct with a huge list of elements.
> 
> Keeping the arguments list inside the struct body makes it easier
> to maintain the documentation.
> ie:
> /**
>  * struct my_struct - short description
>  * @a: first member
>  * @b: second member
>  *
>  * Longer description
>  */
> struct my_struct {
>     int a;
>     int b;
>     /**
>      * @c: This is longer description of C
>      *
>      * You can use paragraphs to describe arguments
>      * using this method.
>      */
>     int c;
> };
> 
> This patch allows the use of this kind of syntax. Only one argument
> per comment and user can use how many paragraphs he needs. It should
> start with /**, which is already being used by kernel-doc. If those
> comment doesn't follow those rules, it will be ignored.
> 
> Signed-off-by: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
> Cc: Randy Dunlap <rdunlap@infradead.org>
> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: Herbert Xu <herbert@gondor.apana.org.au>
> Cc: Stephan Mueller <smueller@chronox.de>
> Cc: Michal Marek <mmarek@suse.cz>
> Cc: linux-kernel@vger.kernel.org
> Cc: linux-doc@vger.kernel.org
> Cc: intel-gfx <intel-gfx@lists.freedesktop.org>
> Cc: dri-devel <dri-devel@lists.freedesktop.org>
> ---
>  scripts/kernel-doc | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 78 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/kernel-doc b/scripts/kernel-doc
> index 9922e66..9bfa8b9 100755
> --- a/scripts/kernel-doc
> +++ b/scripts/kernel-doc
> @@ -133,6 +133,30 @@ use strict;
>  #
>  # All descriptions can be multiline, except the short function description.
>  #
> +# For really longs structs, you can also describe arguments inside the
> +# body of the struct.
> +# eg.
> +# /**
> +#  * struct my_struct - short description
> +#  * @a: first member
> +#  * @b: second member
> +#  *
> +#  * Longer description
> +#  */
> +# struct my_struct {
> +#     int a;
> +#     int b;
> +#     /**
> +#      * @c: This is longer description of C
> +#      *
> +#      * You can use paragraphs to describe arguments
> +#      * using this method.
> +#      */
> +#     int c;
> +# };
> +#
> +# This should be use for arguments only.
> +#
>  # You can also add additional sections. When documenting kernel functions you
>  # should document the "Context:" of the function, e.g. whether the functions
>  # can be called form interrupts. Unlike other sections you can end it with an
> @@ -287,9 +311,19 @@ my $lineprefix="";
>  # 2 - scanning field start.
>  # 3 - scanning prototype.
>  # 4 - documentation block
> +# 5 - gathering documentation outside main block
>  my $state;
>  my $in_doc_sect;
>  
> +# Split Doc State
> +# 0 - Invalid (Before start or after finish)
> +# 1 - Is started (the /** was found inside a struct)
> +# 2 - The @parameter header was found, start accepting multi paragraph text.
> +# 3 - Finished (the */ was found)
> +# 4 - Error - Comment without header was found. Spit a warning as it's not
> +#     proper kernel-doc and ignore the rest.
> +my $split_doc_state;
> +
>  #declaration types: can be
>  # 'function', 'struct', 'union', 'enum', 'typedef'
>  my $decl_type;
> @@ -304,6 +338,9 @@ my $doc_decl = $doc_com . '(\w+)';
>  my $doc_sect = $doc_com . '([' . $doc_special . ']?[\w\s]+):(.*)';
>  my $doc_content = $doc_com_body . '(.*)';
>  my $doc_block = $doc_com . 'DOC:\s*(.*)?';
> +my $doc_split_start = '^\s*/\*\*\s*$';
> +my $doc_split_sect = '\s*\*\s*(@[\w\s]+):(.*)';
> +my $doc_split_end = '^\s*\*/\s*$';
>  
>  my %constants;
>  my %parameterdescs;
> @@ -2181,6 +2218,7 @@ sub reset_state {
>      $prototype = "";
>  
>      $state = 0;
> +    $split_doc_state = 0;
>  }
>  
>  sub tracepoint_munge($) {
> @@ -2453,7 +2491,6 @@ sub process_file($) {
>  		}
>  		$section = $newsection;
>  	    } elsif (/$doc_end/) {
> -
>  		if (($contents ne "") && ($contents ne "\n")) {
>  		    dump_section($file, $section, xml_escape($contents));
>  		    $section = $section_default;
> @@ -2494,8 +2531,47 @@ sub process_file($) {
>  		print STDERR "Warning(${file}:$.): bad line: $_";
>  		++$warnings;
>  	    }
> +	} elsif ($state == 5) { # scanning for split parameters
> +
> +	    # First line (state 1) needs to be a @parameter
> +	    if ($split_doc_state == 1 && /$doc_split_sect/o) {
> +	        $section = $1;
> +	        $contents = $2;

You're using a mix of tabs and spaces here to indent. Ofc we need 4 spaces
for odd indent levels, but for others it shouldn't be required.
-Daniel

> +	        if ($contents ne "") {
> +	    	    while ((substr($contents, 0, 1) eq " ") ||
> +	                    substr($contents, 0, 1) eq "\t") {
> +	    	        $contents = substr($contents, 1);
> +	            }
> +	            $contents .= "\n";
> +	        }
> +	        $split_doc_state = 2;
> +
> +	    # End commend */
> +	    } elsif (/$doc_split_end/) {
> +	        if (($contents ne "") && ($contents ne "\n")) {
> +	            dump_section($file, $section, xml_escape($contents));
> +	            $section = $section_default;
> +	            $contents = "";
> +	        }
> +	        $state = 3;
> +	        $split_doc_state = 0;
> +
> +	    # Regular text
> +	    } elsif (/$doc_content/) {
> +	    	if ($split_doc_state == 2) {
> +	    	    $contents .= $1 . "\n";
> +	    	} elsif ($split_doc_state == 1) {
> +	    	    $split_doc_state = 4;
> +	    	    print STDERR "Warning(${file}:$.): ";
> +	    	    print STDERR "Incorrect use of kernel-doc format: $_";
> +	    	    ++$warnings;
> +	    	}
> +	    }
>  	} elsif ($state == 3) {	# scanning for function '{' (end of prototype)
> -	    if ($decl_type eq 'function') {
> +	    if (/$doc_split_start/) {
> +	    	    $state = 5;
> +	    	    $split_doc_state = 1;
> +	    } elsif ($decl_type eq 'function') {
>  		process_state3_function($_, $file);
>  	    } else {
>  		process_state3_type($_, $file);
> -- 
> 2.4.6
> 

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

[PATCH v2] scripts/kernel-doc Allow struct arguments documentation in struct body

[Danilo Cesar Lemes de Paula]

Describing arguments at top of a struct definition works fine
for small/medium size structs, but it definitely doesn't work well
for struct with a huge list of elements.

Keeping the arguments list inside the struct body makes it easier
to maintain the documentation.
ie:
/**
 * struct my_struct - short description
 * @a: first member
 * @b: second member
 *
 * Longer description
 */
struct my_struct {
    int a;
    int b;
    /**
     * @c: This is longer description of C
     *
     * You can use paragraphs to describe arguments
     * using this method.
     */
    int c;
};

This patch allows the use of this kind of syntax. Only one argument
per comment and user can use how many paragraphs he needs. It should
start with /**, which is already being used by kernel-doc. If those
comment doesn't follow those rules, it will be ignored.

Signed-off-by: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Stephan Mueller <smueller@chronox.de>
Cc: Michal Marek <mmarek@suse.cz>
Cc: linux-kernel@vger.kernel.org
Cc: linux-doc@vger.kernel.org
Cc: intel-gfx <intel-gfx@lists.freedesktop.org>
Cc: dri-devel <dri-devel@lists.freedesktop.org>
---
 Changelog:
  v2:
     - Fixed comment typo
     - Fixed identantion by following the rest of the kernel-doc script pattern.
     - Improved usage comment.

 scripts/kernel-doc | 77 ++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 75 insertions(+), 2 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 9922e66..5adc7ee 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -133,6 +133,30 @@ use strict;
 #
 # All descriptions can be multiline, except the short function description.
 #
+# For really longs structs, you can also describe arguments inside the
+# body of the struct.
+# eg.
+# /**
+#  * struct my_struct - short description
+#  * @a: first member
+#  * @b: second member
+#  *
+#  * Longer description
+#  */
+# struct my_struct {
+#     int a;
+#     int b;
+#     /**
+#      * @c: This is longer description of C
+#      *
+#      * You can use paragraphs to describe arguments
+#      * using this method.
+#      */
+#     int c;
+# };
+#
+# This should be use only for struct/enum members.
+#
 # You can also add additional sections. When documenting kernel functions you
 # should document the "Context:" of the function, e.g. whether the functions
 # can be called form interrupts. Unlike other sections you can end it with an
@@ -287,9 +311,19 @@ my $lineprefix="";
 # 2 - scanning field start.
 # 3 - scanning prototype.
 # 4 - documentation block
+# 5 - gathering documentation outside main block
 my $state;
 my $in_doc_sect;
 
+# Split Doc State
+# 0 - Invalid (Before start or after finish)
+# 1 - Is started (the /** was found inside a struct)
+# 2 - The @parameter header was found, start accepting multi paragraph text.
+# 3 - Finished (the */ was found)
+# 4 - Error - Comment without header was found. Spit a warning as it's not
+#     proper kernel-doc and ignore the rest.
+my $split_doc_state;
+
 #declaration types: can be
 # 'function', 'struct', 'union', 'enum', 'typedef'
 my $decl_type;
@@ -304,6 +338,9 @@ my $doc_decl = $doc_com . '(\w+)';
 my $doc_sect = $doc_com . '([' . $doc_special . ']?[\w\s]+):(.*)';
 my $doc_content = $doc_com_body . '(.*)';
 my $doc_block = $doc_com . 'DOC:\s*(.*)?';
+my $doc_split_start = '^\s*/\*\*\s*$';
+my $doc_split_sect = '\s*\*\s*(@[\w\s]+):(.*)';
+my $doc_split_end = '^\s*\*/\s*$';
 
 my %constants;
 my %parameterdescs;
@@ -2181,6 +2218,7 @@ sub reset_state {
     $prototype = "";
 
     $state = 0;
+    $split_doc_state = 0;
 }
 
 sub tracepoint_munge($) {
@@ -2453,7 +2491,6 @@ sub process_file($) {
 		}
 		$section = $newsection;
 	    } elsif (/$doc_end/) {
-
 		if (($contents ne "") && ($contents ne "\n")) {
 		    dump_section($file, $section, xml_escape($contents));
 		    $section = $section_default;
@@ -2494,8 +2531,44 @@ sub process_file($) {
 		print STDERR "Warning(${file}:$.): bad line: $_";
 		++$warnings;
 	    }
+	} elsif ($state == 5) { # scanning for split parameters
+	    # First line (state 1) needs to be a @parameter
+	    if ($split_doc_state == 1 && /$doc_split_sect/o) {
+		$section = $1;
+		$contents = $2;
+		if ($contents ne "") {
+		    while ((substr($contents, 0, 1) eq " ") ||
+		           substr($contents, 0, 1) eq "\t") {
+			$contents = substr($contents, 1);
+		    }
+		$contents .= "\n";
+		}
+		$split_doc_state = 2;
+	    # Documentation block end */
+	    } elsif (/$doc_split_end/) {
+		if (($contents ne "") && ($contents ne "\n")) {
+		    dump_section($file, $section, xml_escape($contents));
+		    $section = $section_default;
+		    $contents = "";
+		}
+		$state = 3;
+		$split_doc_state = 0;
+	    # Regular text
+	    } elsif (/$doc_content/) {
+		if ($split_doc_state == 2) {
+		    $contents .= $1 . "\n";
+		} elsif ($split_doc_state == 1) {
+		    $split_doc_state = 4;
+		    print STDERR "Warning(${file}:$.): ";
+		    print STDERR "Incorrect use of kernel-doc format: $_";
+		    ++$warnings;
+		}
+	    }
 	} elsif ($state == 3) {	# scanning for function '{' (end of prototype)
-	    if ($decl_type eq 'function') {
+	    if (/$doc_split_start/) {
+		$state = 5;
+		$split_doc_state = 1;
+	    } elsif ($decl_type eq 'function') {
 		process_state3_function($_, $file);
 	    } else {
 		process_state3_type($_, $file);
-- 
2.4.6

Re: [PATCH v2] scripts/kernel-doc Allow struct arguments documentation in struct body

[Jonathan Corbet]

On Tue,  4 Aug 2015 09:04:08 -0300
Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk> wrote:

> Describing arguments at top of a struct definition works fine
> for small/medium size structs, but it definitely doesn't work well
> for struct with a huge list of elements.
> 
> Keeping the arguments list inside the struct body makes it easier
> to maintain the documentation.

OK, this change has been merged into the docs tree.

Thanks,

jon