All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2 00/13] kernel-doc: add supported to document nested structs/$
@ 2017-09-27 21:10 Mauro Carvalho Chehab
  2017-09-27 21:10 ` [PATCH v2 01/13] scripts: kernel-doc: get rid of unused output formats Mauro Carvalho Chehab
                   ` (12 more replies)
  0 siblings, 13 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-27 21:10 UTC (permalink / raw)
  To: Linux Media Mailing List, Jonathan Corbet
  Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, Linux Doc Mailing List

Right now, it is not possible to document nested struct and nested unions.
kernel-doc simply ignore them.

Add support to document them.

Patch 1 gets rid of the now unused output formats for kernel-doc: since 
we got rid of all DocBook stuff, we should not need them anymore. The
reason for dropping it (despite cleaning up), is that it doesn't make 
sense to invest time on adding new features for formats that aren't
used anymore.

Patches 2 to 7 improve kernel-doc documentation to reflect what
kernel-doc currently supports and import some stuff from the
old kernel-doc-nano-HOWTO.txt.

Patch 8 gets rid of the old documentation (kernel-doc-nano-HOWTO.txt).

Patch 9 is the most interesting one in this series: it adds support for
nested structures and unions.

Patch 10 does a cleanup, removing $nexted parameter at the
routines that handle structs.

Patch 11 Improves warning output by printing the identifier where
the warning occurred.

Patch 12 complements patch 9, by adding support for function
definitions inside nested structures. It is needed by some media
docs with use such kind of things.

Patch 13 is just an example from a random header with kernel-doc
markups. There's no special reason for selecting this file, and the
comments there are likely wrong. So, please use it only as a way to test
the new parser logic from patch 9. The real usage (for me, will
be at the media patches, but this is on a 30+ patch series, together
with a bunch of other stuff.

--

v2:
  - solved some issues that Randy pointed;
  - added patch 10 as suggested by Markus;
  - Fixed some bugs on patch 9, after parsing nested structs
   on media subsystem;
  - added patch 11 with a warning improvement fixup;
  - added patch 12 in order to handle function parameters
   on nested structures, due to DVB demux kAPI.

Mauro Carvalho Chehab (13):
  scripts: kernel-doc: get rid of unused output formats
  docs: kernel-doc.rst: better describe kernel-doc arguments
  docs: kernel-doc.rst: improve private members description
  docs: kernel-doc.rst: improve function documentation section
  docs: kernel-doc.rst: improve structs chapter
  docs: kernel-doc.rst: improve typedef documentation
  docs: kernel-doc.rst: add documentation about man pages
  docs: get rid of kernel-doc-nano-HOWTO.txt
  scripts: kernel-doc: parse next structs/unions
  scripts: kernel-doc: get rid of $nested parameter
  scripts: kernel-doc: print the declaration name on warnings
  scripts: kernel-doc: handle nested struct function arguments
  [RFC] w1_netlink.h: add support for nested structs

 Documentation/00-INDEX                  |    2 -
 Documentation/doc-guide/kernel-doc.rst  |  360 +++++---
 Documentation/kernel-doc-nano-HOWTO.txt |  322 --------
 drivers/w1/w1_netlink.h                 |    4 +
 scripts/kernel-doc                      | 1376 +++----------------------------
 5 files changed, 369 insertions(+), 1695 deletions(-)
 delete mode 100644 Documentation/kernel-doc-nano-HOWTO.txt

-- 
2.13.5

^ permalink raw reply	[flat|nested] 28+ messages in thread

* [PATCH v2 01/13] scripts: kernel-doc: get rid of unused output formats
  2017-09-27 21:10 [PATCH v2 00/13] kernel-doc: add supported to document nested structs/$ Mauro Carvalho Chehab
@ 2017-09-27 21:10 ` Mauro Carvalho Chehab
  2017-09-27 21:10 ` [PATCH v2 02/13] docs: kernel-doc.rst: better describe kernel-doc arguments Mauro Carvalho Chehab
                   ` (11 subsequent siblings)
  12 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-27 21:10 UTC (permalink / raw)
  To: Linux Media Mailing List, Jonathan Corbet
  Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, Linux Doc Mailing List

Since there isn't any docbook code anymore upstream,
we can get rid of several output formats:

- docbook/xml, html, html5 and list formats were used by
  the old build system;
- As ReST is text, there's not much sense on outputting
  on a different text format.

After this patch, only man and rst output formats are
supported.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Acked-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 1182 +---------------------------------------------------
 1 file changed, 4 insertions(+), 1178 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 9d3eafea58f0..69757ee9db4c 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -51,13 +51,8 @@ The documentation comments are identified by "/**" opening comment mark. See
 Documentation/kernel-doc-nano-HOWTO.txt for the documentation comment syntax.
 
 Output format selection (mutually exclusive):
-  -docbook		Output DocBook format.
-  -html			Output HTML format.
-  -html5		Output HTML5 format.
-  -list			Output symbol list format. This is for use by docproc.
   -man			Output troff manual page format. This is the default.
   -rst			Output reStructuredText format.
-  -text			Output plain text format.
 
 Output selection (mutually exclusive):
   -export		Only output documentation for symbols that have been
@@ -224,84 +219,11 @@ my $type_typedef = '\&(typedef\s*([_\w]+))';
 my $type_union = '\&(union\s*([_\w]+))';
 my $type_member = '\&([_\w]+)(\.|->)([_\w]+)';
 my $type_fallback = '\&([_\w]+)';
-my $type_enum_xml = '\&amp;(enum\s*([_\w]+))';
-my $type_struct_xml = '\&amp;(struct\s*([_\w]+))';
-my $type_typedef_xml = '\&amp;(typedef\s*([_\w]+))';
-my $type_union_xml = '\&amp;(union\s*([_\w]+))';
-my $type_member_xml = '\&amp;([_\w]+)(\.|-\&gt;)([_\w]+)';
-my $type_fallback_xml = '\&amp([_\w]+)';
 my $type_member_func = $type_member . '\(\)';
 
 # Output conversion substitutions.
 #  One for each output format
 
-# these work fairly well
-my @highlights_html = (
-                       [$type_constant, "<i>\$1</i>"],
-                       [$type_constant2, "<i>\$1</i>"],
-                       [$type_func, "<b>\$1</b>"],
-                       [$type_enum_xml, "<i>\$1</i>"],
-                       [$type_struct_xml, "<i>\$1</i>"],
-                       [$type_typedef_xml, "<i>\$1</i>"],
-                       [$type_union_xml, "<i>\$1</i>"],
-                       [$type_env, "<b><i>\$1</i></b>"],
-                       [$type_param, "<tt><b>\$1</b></tt>"],
-                       [$type_member_xml, "<tt><i>\$1</i>\$2\$3</tt>"],
-                       [$type_fallback_xml, "<i>\$1</i>"]
-                      );
-my $local_lt = "\\\\\\\\lt:";
-my $local_gt = "\\\\\\\\gt:";
-my $blankline_html = $local_lt . "p" . $local_gt;	# was "<p>"
-
-# html version 5
-my @highlights_html5 = (
-                        [$type_constant, "<span class=\"const\">\$1</span>"],
-                        [$type_constant2, "<span class=\"const\">\$1</span>"],
-                        [$type_func, "<span class=\"func\">\$1</span>"],
-                        [$type_enum_xml, "<span class=\"enum\">\$1</span>"],
-                        [$type_struct_xml, "<span class=\"struct\">\$1</span>"],
-                        [$type_typedef_xml, "<span class=\"typedef\">\$1</span>"],
-                        [$type_union_xml, "<span class=\"union\">\$1</span>"],
-                        [$type_env, "<span class=\"env\">\$1</span>"],
-                        [$type_param, "<span class=\"param\">\$1</span>]"],
-                        [$type_member_xml, "<span class=\"literal\"><span class=\"struct\">\$1</span>\$2<span class=\"member\">\$3</span></span>"],
-                        [$type_fallback_xml, "<span class=\"struct\">\$1</span>"]
-		       );
-my $blankline_html5 = $local_lt . "br /" . $local_gt;
-
-# XML, docbook format
-my @highlights_xml = (
-                      ["([^=])\\\"([^\\\"<]+)\\\"", "\$1<quote>\$2</quote>"],
-                      [$type_constant, "<constant>\$1</constant>"],
-                      [$type_constant2, "<constant>\$1</constant>"],
-                      [$type_enum_xml, "<type>\$1</type>"],
-                      [$type_struct_xml, "<structname>\$1</structname>"],
-                      [$type_typedef_xml, "<type>\$1</type>"],
-                      [$type_union_xml, "<structname>\$1</structname>"],
-                      [$type_param, "<parameter>\$1</parameter>"],
-                      [$type_func, "<function>\$1</function>"],
-                      [$type_env, "<envar>\$1</envar>"],
-                      [$type_member_xml, "<literal><structname>\$1</structname>\$2<structfield>\$3</structfield></literal>"],
-                      [$type_fallback_xml, "<structname>\$1</structname>"]
-		     );
-my $blankline_xml = $local_lt . "/para" . $local_gt . $local_lt . "para" . $local_gt . "\n";
-
-# gnome, docbook format
-my @highlights_gnome = (
-                        [$type_constant, "<replaceable class=\"option\">\$1</replaceable>"],
-                        [$type_constant2, "<replaceable class=\"option\">\$1</replaceable>"],
-                        [$type_func, "<function>\$1</function>"],
-                        [$type_enum, "<type>\$1</type>"],
-                        [$type_struct, "<structname>\$1</structname>"],
-                        [$type_typedef, "<type>\$1</type>"],
-                        [$type_union, "<structname>\$1</structname>"],
-                        [$type_env, "<envar>\$1</envar>"],
-                        [$type_param, "<parameter>\$1</parameter>" ],
-                        [$type_member, "<literal><structname>\$1</structname>\$2<structfield>\$3</structfield></literal>"],
-                        [$type_fallback, "<structname>\$1</structname>"]
-		       );
-my $blankline_gnome = "</para><para>\n";
-
 # these are pretty rough
 my @highlights_man = (
                       [$type_constant, "\$1"],
@@ -317,21 +239,6 @@ my @highlights_man = (
 		     );
 my $blankline_man = "";
 
-# text-mode
-my @highlights_text = (
-                       [$type_constant, "\$1"],
-                       [$type_constant2, "\$1"],
-                       [$type_func, "\$1"],
-                       [$type_enum, "\$1"],
-                       [$type_struct, "\$1"],
-                       [$type_typedef, "\$1"],
-                       [$type_union, "\$1"],
-                       [$type_param, "\$1"],
-                       [$type_member, "\$1\$2\$3"],
-                       [$type_fallback, "\$1"]
-		      );
-my $blankline_text = "";
-
 # rst-mode
 my @highlights_rst = (
                        [$type_constant, "``\$1``"],
@@ -351,21 +258,6 @@ my @highlights_rst = (
 		      );
 my $blankline_rst = "\n";
 
-# list mode
-my @highlights_list = (
-                       [$type_constant, "\$1"],
-                       [$type_constant2, "\$1"],
-                       [$type_func, "\$1"],
-                       [$type_enum, "\$1"],
-                       [$type_struct, "\$1"],
-                       [$type_typedef, "\$1"],
-                       [$type_union, "\$1"],
-                       [$type_param, "\$1"],
-                       [$type_member, "\$1"],
-                       [$type_fallback, "\$1"]
-		      );
-my $blankline_list = "";
-
 # read arguments
 if ($#ARGV == -1) {
     usage();
@@ -500,38 +392,14 @@ reset_state();
 
 while ($ARGV[0] =~ m/^-(.*)/) {
     my $cmd = shift @ARGV;
-    if ($cmd eq "-html") {
-	$output_mode = "html";
-	@highlights = @highlights_html;
-	$blankline = $blankline_html;
-    } elsif ($cmd eq "-html5") {
-	$output_mode = "html5";
-	@highlights = @highlights_html5;
-	$blankline = $blankline_html5;
-    } elsif ($cmd eq "-man") {
+    if ($cmd eq "-man") {
 	$output_mode = "man";
 	@highlights = @highlights_man;
 	$blankline = $blankline_man;
-    } elsif ($cmd eq "-text") {
-	$output_mode = "text";
-	@highlights = @highlights_text;
-	$blankline = $blankline_text;
     } elsif ($cmd eq "-rst") {
 	$output_mode = "rst";
 	@highlights = @highlights_rst;
 	$blankline = $blankline_rst;
-    } elsif ($cmd eq "-docbook") {
-	$output_mode = "xml";
-	@highlights = @highlights_xml;
-	$blankline = $blankline_xml;
-    } elsif ($cmd eq "-list") {
-	$output_mode = "list";
-	@highlights = @highlights_list;
-	$blankline = $blankline_list;
-    } elsif ($cmd eq "-gnome") {
-	$output_mode = "gnome";
-	@highlights = @highlights_gnome;
-	$blankline = $blankline_gnome;
     } elsif ($cmd eq "-module") { # not needed for XML, inherits from calling document
 	$modulename = shift @ARGV;
     } elsif ($cmd eq "-function") { # to only output specific functions
@@ -667,22 +535,11 @@ sub output_highlight {
 #	confess "output_highlight got called with no args?\n";
 #   }
 
-    if ($output_mode eq "html" || $output_mode eq "html5" ||
-	$output_mode eq "xml") {
-	$contents = local_unescape($contents);
-	# convert data read & converted thru xml_escape() into &xyz; format:
-	$contents =~ s/\\\\\\/\&/g;
-    }
 #   print STDERR "contents b4:$contents\n";
     eval $dohighlight;
     die $@ if $@;
 #   print STDERR "contents af:$contents\n";
 
-#   strip whitespaces when generating html5
-    if ($output_mode eq "html5") {
-	$contents =~ s/^\s+//;
-	$contents =~ s/\s+$//;
-    }
     foreach $line (split "\n", $contents) {
 	if (! $output_preformatted) {
 	    $line =~ s/^\s*//;
@@ -703,817 +560,6 @@ sub output_highlight {
     }
 }
 
-# output sections in html
-sub output_section_html(%) {
-    my %args = %{$_[0]};
-    my $section;
-
-    foreach $section (@{$args{'sectionlist'}}) {
-	print "<h3>$section</h3>\n";
-	print "<blockquote>\n";
-	output_highlight($args{'sections'}{$section});
-	print "</blockquote>\n";
-    }
-}
-
-# output enum in html
-sub output_enum_html(%) {
-    my %args = %{$_[0]};
-    my ($parameter);
-    my $count;
-    print "<h2>enum " . $args{'enum'} . "</h2>\n";
-
-    print "<b>enum " . $args{'enum'} . "</b> {<br>\n";
-    $count = 0;
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	print " <b>" . $parameter . "</b>";
-	if ($count != $#{$args{'parameterlist'}}) {
-	    $count++;
-	    print ",\n";
-	}
-	print "<br>";
-    }
-    print "};<br>\n";
-
-    print "<h3>Constants</h3>\n";
-    print "<dl>\n";
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	print "<dt><b>" . $parameter . "</b>\n";
-	print "<dd>";
-	output_highlight($args{'parameterdescs'}{$parameter});
-    }
-    print "</dl>\n";
-    output_section_html(@_);
-    print "<hr>\n";
-}
-
-# output typedef in html
-sub output_typedef_html(%) {
-    my %args = %{$_[0]};
-    my ($parameter);
-    my $count;
-    print "<h2>typedef " . $args{'typedef'} . "</h2>\n";
-
-    print "<b>typedef " . $args{'typedef'} . "</b>\n";
-    output_section_html(@_);
-    print "<hr>\n";
-}
-
-# output struct in html
-sub output_struct_html(%) {
-    my %args = %{$_[0]};
-    my ($parameter);
-
-    print "<h2>" . $args{'type'} . " " . $args{'struct'} . " - " . $args{'purpose'} . "</h2>\n";
-    print "<b>" . $args{'type'} . " " . $args{'struct'} . "</b> {<br>\n";
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	if ($parameter =~ /^#/) {
-		print "$parameter<br>\n";
-		next;
-	}
-	my $parameter_name = $parameter;
-	$parameter_name =~ s/\[.*//;
-
-	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
-	$type = $args{'parametertypes'}{$parameter};
-	if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
-	    # pointer-to-function
-	    print "&nbsp; &nbsp; <i>$1</i><b>$parameter</b>) <i>($2)</i>;<br>\n";
-	} elsif ($type =~ m/^(.*?)\s*(:.*)/) {
-	    # bitfield
-	    print "&nbsp; &nbsp; <i>$1</i> <b>$parameter</b>$2;<br>\n";
-	} else {
-	    print "&nbsp; &nbsp; <i>$type</i> <b>$parameter</b>;<br>\n";
-	}
-    }
-    print "};<br>\n";
-
-    print "<h3>Members</h3>\n";
-    print "<dl>\n";
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	($parameter =~ /^#/) && next;
-
-	my $parameter_name = $parameter;
-	$parameter_name =~ s/\[.*//;
-
-	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
-	print "<dt><b>" . $parameter . "</b>\n";
-	print "<dd>";
-	output_highlight($args{'parameterdescs'}{$parameter_name});
-    }
-    print "</dl>\n";
-    output_section_html(@_);
-    print "<hr>\n";
-}
-
-# output function in html
-sub output_function_html(%) {
-    my %args = %{$_[0]};
-    my ($parameter, $section);
-    my $count;
-
-    print "<h2>" . $args{'function'} . " - " . $args{'purpose'} . "</h2>\n";
-    print "<i>" . $args{'functiontype'} . "</i>\n";
-    print "<b>" . $args{'function'} . "</b>\n";
-    print "(";
-    $count = 0;
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	$type = $args{'parametertypes'}{$parameter};
-	if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
-	    # pointer-to-function
-	    print "<i>$1</i><b>$parameter</b>) <i>($2)</i>";
-	} else {
-	    print "<i>" . $type . "</i> <b>" . $parameter . "</b>";
-	}
-	if ($count != $#{$args{'parameterlist'}}) {
-	    $count++;
-	    print ",\n";
-	}
-    }
-    print ")\n";
-
-    print "<h3>Arguments</h3>\n";
-    print "<dl>\n";
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	my $parameter_name = $parameter;
-	$parameter_name =~ s/\[.*//;
-
-	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
-	print "<dt><b>" . $parameter . "</b>\n";
-	print "<dd>";
-	output_highlight($args{'parameterdescs'}{$parameter_name});
-    }
-    print "</dl>\n";
-    output_section_html(@_);
-    print "<hr>\n";
-}
-
-# output DOC: block header in html
-sub output_blockhead_html(%) {
-    my %args = %{$_[0]};
-    my ($parameter, $section);
-    my $count;
-
-    foreach $section (@{$args{'sectionlist'}}) {
-	print "<h3>$section</h3>\n";
-	print "<ul>\n";
-	output_highlight($args{'sections'}{$section});
-	print "</ul>\n";
-    }
-    print "<hr>\n";
-}
-
-# output sections in html5
-sub output_section_html5(%) {
-    my %args = %{$_[0]};
-    my $section;
-
-    foreach $section (@{$args{'sectionlist'}}) {
-	print "<section>\n";
-	print "<h1>$section</h1>\n";
-	print "<p>\n";
-	output_highlight($args{'sections'}{$section});
-	print "</p>\n";
-	print "</section>\n";
-    }
-}
-
-# output enum in html5
-sub output_enum_html5(%) {
-    my %args = %{$_[0]};
-    my ($parameter);
-    my $count;
-    my $html5id;
-
-    $html5id = $args{'enum'};
-    $html5id =~ s/[^a-zA-Z0-9\-]+/_/g;
-    print "<article class=\"enum\" id=\"enum:". $html5id . "\">";
-    print "<h1>enum " . $args{'enum'} . "</h1>\n";
-    print "<ol class=\"code\">\n";
-    print "<li>";
-    print "<span class=\"keyword\">enum</span> ";
-    print "<span class=\"identifier\">" . $args{'enum'} . "</span> {";
-    print "</li>\n";
-    $count = 0;
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	print "<li class=\"indent\">";
-	print "<span class=\"param\">" . $parameter . "</span>";
-	if ($count != $#{$args{'parameterlist'}}) {
-	    $count++;
-	    print ",";
-	}
-	print "</li>\n";
-    }
-    print "<li>};</li>\n";
-    print "</ol>\n";
-
-    print "<section>\n";
-    print "<h1>Constants</h1>\n";
-    print "<dl>\n";
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	print "<dt>" . $parameter . "</dt>\n";
-	print "<dd>";
-	output_highlight($args{'parameterdescs'}{$parameter});
-	print "</dd>\n";
-    }
-    print "</dl>\n";
-    print "</section>\n";
-    output_section_html5(@_);
-    print "</article>\n";
-}
-
-# output typedef in html5
-sub output_typedef_html5(%) {
-    my %args = %{$_[0]};
-    my ($parameter);
-    my $count;
-    my $html5id;
-
-    $html5id = $args{'typedef'};
-    $html5id =~ s/[^a-zA-Z0-9\-]+/_/g;
-    print "<article class=\"typedef\" id=\"typedef:" . $html5id . "\">\n";
-    print "<h1>typedef " . $args{'typedef'} . "</h1>\n";
-
-    print "<ol class=\"code\">\n";
-    print "<li>";
-    print "<span class=\"keyword\">typedef</span> ";
-    print "<span class=\"identifier\">" . $args{'typedef'} . "</span>";
-    print "</li>\n";
-    print "</ol>\n";
-    output_section_html5(@_);
-    print "</article>\n";
-}
-
-# output struct in html5
-sub output_struct_html5(%) {
-    my %args = %{$_[0]};
-    my ($parameter);
-    my $html5id;
-
-    $html5id = $args{'struct'};
-    $html5id =~ s/[^a-zA-Z0-9\-]+/_/g;
-    print "<article class=\"struct\" id=\"struct:" . $html5id . "\">\n";
-    print "<hgroup>\n";
-    print "<h1>" . $args{'type'} . " " . $args{'struct'} . "</h1>";
-    print "<h2>". $args{'purpose'} . "</h2>\n";
-    print "</hgroup>\n";
-    print "<ol class=\"code\">\n";
-    print "<li>";
-    print "<span class=\"type\">" . $args{'type'} . "</span> ";
-    print "<span class=\"identifier\">" . $args{'struct'} . "</span> {";
-    print "</li>\n";
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	print "<li class=\"indent\">";
-	if ($parameter =~ /^#/) {
-		print "<span class=\"param\">" . $parameter ."</span>\n";
-		print "</li>\n";
-		next;
-	}
-	my $parameter_name = $parameter;
-	$parameter_name =~ s/\[.*//;
-
-	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
-	$type = $args{'parametertypes'}{$parameter};
-	if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
-	    # pointer-to-function
-	    print "<span class=\"type\">$1</span> ";
-	    print "<span class=\"param\">$parameter</span>";
-	    print "<span class=\"type\">)</span> ";
-	    print "(<span class=\"args\">$2</span>);";
-	} elsif ($type =~ m/^(.*?)\s*(:.*)/) {
-	    # bitfield
-	    print "<span class=\"type\">$1</span> ";
-	    print "<span class=\"param\">$parameter</span>";
-	    print "<span class=\"bits\">$2</span>;";
-	} else {
-	    print "<span class=\"type\">$type</span> ";
-	    print "<span class=\"param\">$parameter</span>;";
-	}
-	print "</li>\n";
-    }
-    print "<li>};</li>\n";
-    print "</ol>\n";
-
-    print "<section>\n";
-    print "<h1>Members</h1>\n";
-    print "<dl>\n";
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	($parameter =~ /^#/) && next;
-
-	my $parameter_name = $parameter;
-	$parameter_name =~ s/\[.*//;
-
-	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
-	print "<dt>" . $parameter . "</dt>\n";
-	print "<dd>";
-	output_highlight($args{'parameterdescs'}{$parameter_name});
-	print "</dd>\n";
-    }
-    print "</dl>\n";
-    print "</section>\n";
-    output_section_html5(@_);
-    print "</article>\n";
-}
-
-# output function in html5
-sub output_function_html5(%) {
-    my %args = %{$_[0]};
-    my ($parameter, $section);
-    my $count;
-    my $html5id;
-
-    $html5id = $args{'function'};
-    $html5id =~ s/[^a-zA-Z0-9\-]+/_/g;
-    print "<article class=\"function\" id=\"func:". $html5id . "\">\n";
-    print "<hgroup>\n";
-    print "<h1>" . $args{'function'} . "</h1>";
-    print "<h2>" . $args{'purpose'} . "</h2>\n";
-    print "</hgroup>\n";
-    print "<ol class=\"code\">\n";
-    print "<li>";
-    print "<span class=\"type\">" . $args{'functiontype'} . "</span> ";
-    print "<span class=\"identifier\">" . $args{'function'} . "</span> (";
-    print "</li>";
-    $count = 0;
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	print "<li class=\"indent\">";
-	$type = $args{'parametertypes'}{$parameter};
-	if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
-	    # pointer-to-function
-	    print "<span class=\"type\">$1</span> ";
-	    print "<span class=\"param\">$parameter</span>";
-	    print "<span class=\"type\">)</span> ";
-	    print "(<span class=\"args\">$2</span>)";
-	} else {
-	    print "<span class=\"type\">$type</span> ";
-	    print "<span class=\"param\">$parameter</span>";
-	}
-	if ($count != $#{$args{'parameterlist'}}) {
-	    $count++;
-	    print ",";
-	}
-	print "</li>\n";
-    }
-    print "<li>)</li>\n";
-    print "</ol>\n";
-
-    print "<section>\n";
-    print "<h1>Arguments</h1>\n";
-    print "<p>\n";
-    print "<dl>\n";
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	my $parameter_name = $parameter;
-	$parameter_name =~ s/\[.*//;
-
-	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
-	print "<dt>" . $parameter . "</dt>\n";
-	print "<dd>";
-	output_highlight($args{'parameterdescs'}{$parameter_name});
-	print "</dd>\n";
-    }
-    print "</dl>\n";
-    print "</section>\n";
-    output_section_html5(@_);
-    print "</article>\n";
-}
-
-# output DOC: block header in html5
-sub output_blockhead_html5(%) {
-    my %args = %{$_[0]};
-    my ($parameter, $section);
-    my $count;
-    my $html5id;
-
-    foreach $section (@{$args{'sectionlist'}}) {
-	$html5id = $section;
-	$html5id =~ s/[^a-zA-Z0-9\-]+/_/g;
-	print "<article class=\"doc\" id=\"doc:". $html5id . "\">\n";
-	print "<h1>$section</h1>\n";
-	print "<p>\n";
-	output_highlight($args{'sections'}{$section});
-	print "</p>\n";
-    }
-    print "</article>\n";
-}
-
-sub output_section_xml(%) {
-    my %args = %{$_[0]};
-    my $section;
-    # print out each section
-    $lineprefix="   ";
-    foreach $section (@{$args{'sectionlist'}}) {
-	print "<refsect1>\n";
-	print "<title>$section</title>\n";
-	if ($section =~ m/EXAMPLE/i) {
-	    print "<informalexample><programlisting>\n";
-	    $output_preformatted = 1;
-	} else {
-	    print "<para>\n";
-	}
-	output_highlight($args{'sections'}{$section});
-	$output_preformatted = 0;
-	if ($section =~ m/EXAMPLE/i) {
-	    print "</programlisting></informalexample>\n";
-	} else {
-	    print "</para>\n";
-	}
-	print "</refsect1>\n";
-    }
-}
-
-# output function in XML DocBook
-sub output_function_xml(%) {
-    my %args = %{$_[0]};
-    my ($parameter, $section);
-    my $count;
-    my $id;
-
-    $id = "API-" . $args{'function'};
-    $id =~ s/[^A-Za-z0-9]/-/g;
-
-    print "<refentry id=\"$id\">\n";
-    print "<refentryinfo>\n";
-    print " <title>LINUX</title>\n";
-    print " <productname>Kernel Hackers Manual</productname>\n";
-    print " <date>$man_date</date>\n";
-    print "</refentryinfo>\n";
-    print "<refmeta>\n";
-    print " <refentrytitle><phrase>" . $args{'function'} . "</phrase></refentrytitle>\n";
-    print " <manvolnum>9</manvolnum>\n";
-    print " <refmiscinfo class=\"version\">" . $kernelversion . "</refmiscinfo>\n";
-    print "</refmeta>\n";
-    print "<refnamediv>\n";
-    print " <refname>" . $args{'function'} . "</refname>\n";
-    print " <refpurpose>\n";
-    print "  ";
-    output_highlight ($args{'purpose'});
-    print " </refpurpose>\n";
-    print "</refnamediv>\n";
-
-    print "<refsynopsisdiv>\n";
-    print " <title>Synopsis</title>\n";
-    print "  <funcsynopsis><funcprototype>\n";
-    print "   <funcdef>" . $args{'functiontype'} . " ";
-    print "<function>" . $args{'function'} . " </function></funcdef>\n";
-
-    $count = 0;
-    if ($#{$args{'parameterlist'}} >= 0) {
-	foreach $parameter (@{$args{'parameterlist'}}) {
-	    $type = $args{'parametertypes'}{$parameter};
-	    if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
-		# pointer-to-function
-		print "   <paramdef>$1<parameter>$parameter</parameter>)\n";
-		print "     <funcparams>$2</funcparams></paramdef>\n";
-	    } else {
-		print "   <paramdef>" . $type;
-		print " <parameter>$parameter</parameter></paramdef>\n";
-	    }
-	}
-    } else {
-	print "  <void/>\n";
-    }
-    print "  </funcprototype></funcsynopsis>\n";
-    print "</refsynopsisdiv>\n";
-
-    # print parameters
-    print "<refsect1>\n <title>Arguments</title>\n";
-    if ($#{$args{'parameterlist'}} >= 0) {
-	print " <variablelist>\n";
-	foreach $parameter (@{$args{'parameterlist'}}) {
-	    my $parameter_name = $parameter;
-	    $parameter_name =~ s/\[.*//;
-	    $type = $args{'parametertypes'}{$parameter};
-
-	    print "  <varlistentry>\n   <term><parameter>$type $parameter</parameter></term>\n";
-	    print "   <listitem>\n    <para>\n";
-	    $lineprefix="     ";
-	    output_highlight($args{'parameterdescs'}{$parameter_name});
-	    print "    </para>\n   </listitem>\n  </varlistentry>\n";
-	}
-	print " </variablelist>\n";
-    } else {
-	print " <para>\n  None\n </para>\n";
-    }
-    print "</refsect1>\n";
-
-    output_section_xml(@_);
-    print "</refentry>\n\n";
-}
-
-# output struct in XML DocBook
-sub output_struct_xml(%) {
-    my %args = %{$_[0]};
-    my ($parameter, $section);
-    my $id;
-
-    $id = "API-struct-" . $args{'struct'};
-    $id =~ s/[^A-Za-z0-9]/-/g;
-
-    print "<refentry id=\"$id\">\n";
-    print "<refentryinfo>\n";
-    print " <title>LINUX</title>\n";
-    print " <productname>Kernel Hackers Manual</productname>\n";
-    print " <date>$man_date</date>\n";
-    print "</refentryinfo>\n";
-    print "<refmeta>\n";
-    print " <refentrytitle><phrase>" . $args{'type'} . " " . $args{'struct'} . "</phrase></refentrytitle>\n";
-    print " <manvolnum>9</manvolnum>\n";
-    print " <refmiscinfo class=\"version\">" . $kernelversion . "</refmiscinfo>\n";
-    print "</refmeta>\n";
-    print "<refnamediv>\n";
-    print " <refname>" . $args{'type'} . " " . $args{'struct'} . "</refname>\n";
-    print " <refpurpose>\n";
-    print "  ";
-    output_highlight ($args{'purpose'});
-    print " </refpurpose>\n";
-    print "</refnamediv>\n";
-
-    print "<refsynopsisdiv>\n";
-    print " <title>Synopsis</title>\n";
-    print "  <programlisting>\n";
-    print $args{'type'} . " " . $args{'struct'} . " {\n";
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	if ($parameter =~ /^#/) {
-	    my $prm = $parameter;
-	    # convert data read & converted thru xml_escape() into &xyz; format:
-	    # This allows us to have #define macros interspersed in a struct.
-	    $prm =~ s/\\\\\\/\&/g;
-	    print "$prm\n";
-	    next;
-	}
-
-	my $parameter_name = $parameter;
-	$parameter_name =~ s/\[.*//;
-
-	defined($args{'parameterdescs'}{$parameter_name}) || next;
-	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
-	$type = $args{'parametertypes'}{$parameter};
-	if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
-	    # pointer-to-function
-	    print "  $1 $parameter) ($2);\n";
-	} elsif ($type =~ m/^(.*?)\s*(:.*)/) {
-	    # bitfield
-	    print "  $1 $parameter$2;\n";
-	} else {
-	    print "  " . $type . " " . $parameter . ";\n";
-	}
-    }
-    print "};";
-    print "  </programlisting>\n";
-    print "</refsynopsisdiv>\n";
-
-    print " <refsect1>\n";
-    print "  <title>Members</title>\n";
-
-    if ($#{$args{'parameterlist'}} >= 0) {
-    print "  <variablelist>\n";
-    foreach $parameter (@{$args{'parameterlist'}}) {
-      ($parameter =~ /^#/) && next;
-
-      my $parameter_name = $parameter;
-      $parameter_name =~ s/\[.*//;
-
-      defined($args{'parameterdescs'}{$parameter_name}) || next;
-      ($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
-      $type = $args{'parametertypes'}{$parameter};
-      print "    <varlistentry>";
-      print "      <term><literal>$type $parameter</literal></term>\n";
-      print "      <listitem><para>\n";
-      output_highlight($args{'parameterdescs'}{$parameter_name});
-      print "      </para></listitem>\n";
-      print "    </varlistentry>\n";
-    }
-    print "  </variablelist>\n";
-    } else {
-	print " <para>\n  None\n </para>\n";
-    }
-    print " </refsect1>\n";
-
-    output_section_xml(@_);
-
-    print "</refentry>\n\n";
-}
-
-# output enum in XML DocBook
-sub output_enum_xml(%) {
-    my %args = %{$_[0]};
-    my ($parameter, $section);
-    my $count;
-    my $id;
-
-    $id = "API-enum-" . $args{'enum'};
-    $id =~ s/[^A-Za-z0-9]/-/g;
-
-    print "<refentry id=\"$id\">\n";
-    print "<refentryinfo>\n";
-    print " <title>LINUX</title>\n";
-    print " <productname>Kernel Hackers Manual</productname>\n";
-    print " <date>$man_date</date>\n";
-    print "</refentryinfo>\n";
-    print "<refmeta>\n";
-    print " <refentrytitle><phrase>enum " . $args{'enum'} . "</phrase></refentrytitle>\n";
-    print " <manvolnum>9</manvolnum>\n";
-    print " <refmiscinfo class=\"version\">" . $kernelversion . "</refmiscinfo>\n";
-    print "</refmeta>\n";
-    print "<refnamediv>\n";
-    print " <refname>enum " . $args{'enum'} . "</refname>\n";
-    print " <refpurpose>\n";
-    print "  ";
-    output_highlight ($args{'purpose'});
-    print " </refpurpose>\n";
-    print "</refnamediv>\n";
-
-    print "<refsynopsisdiv>\n";
-    print " <title>Synopsis</title>\n";
-    print "  <programlisting>\n";
-    print "enum " . $args{'enum'} . " {\n";
-    $count = 0;
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	print "  $parameter";
-	if ($count != $#{$args{'parameterlist'}}) {
-	    $count++;
-	    print ",";
-	}
-	print "\n";
-    }
-    print "};";
-    print "  </programlisting>\n";
-    print "</refsynopsisdiv>\n";
-
-    print "<refsect1>\n";
-    print " <title>Constants</title>\n";
-    print "  <variablelist>\n";
-    foreach $parameter (@{$args{'parameterlist'}}) {
-      my $parameter_name = $parameter;
-      $parameter_name =~ s/\[.*//;
-
-      print "    <varlistentry>";
-      print "      <term>$parameter</term>\n";
-      print "      <listitem><para>\n";
-      output_highlight($args{'parameterdescs'}{$parameter_name});
-      print "      </para></listitem>\n";
-      print "    </varlistentry>\n";
-    }
-    print "  </variablelist>\n";
-    print "</refsect1>\n";
-
-    output_section_xml(@_);
-
-    print "</refentry>\n\n";
-}
-
-# output typedef in XML DocBook
-sub output_typedef_xml(%) {
-    my %args = %{$_[0]};
-    my ($parameter, $section);
-    my $id;
-
-    $id = "API-typedef-" . $args{'typedef'};
-    $id =~ s/[^A-Za-z0-9]/-/g;
-
-    print "<refentry id=\"$id\">\n";
-    print "<refentryinfo>\n";
-    print " <title>LINUX</title>\n";
-    print " <productname>Kernel Hackers Manual</productname>\n";
-    print " <date>$man_date</date>\n";
-    print "</refentryinfo>\n";
-    print "<refmeta>\n";
-    print " <refentrytitle><phrase>typedef " . $args{'typedef'} . "</phrase></refentrytitle>\n";
-    print " <manvolnum>9</manvolnum>\n";
-    print "</refmeta>\n";
-    print "<refnamediv>\n";
-    print " <refname>typedef " . $args{'typedef'} . "</refname>\n";
-    print " <refpurpose>\n";
-    print "  ";
-    output_highlight ($args{'purpose'});
-    print " </refpurpose>\n";
-    print "</refnamediv>\n";
-
-    print "<refsynopsisdiv>\n";
-    print " <title>Synopsis</title>\n";
-    print "  <synopsis>typedef " . $args{'typedef'} . ";</synopsis>\n";
-    print "</refsynopsisdiv>\n";
-
-    output_section_xml(@_);
-
-    print "</refentry>\n\n";
-}
-
-# output in XML DocBook
-sub output_blockhead_xml(%) {
-    my %args = %{$_[0]};
-    my ($parameter, $section);
-    my $count;
-
-    my $id = $args{'module'};
-    $id =~ s/[^A-Za-z0-9]/-/g;
-
-    # print out each section
-    $lineprefix="   ";
-    foreach $section (@{$args{'sectionlist'}}) {
-	if (!$args{'content-only'}) {
-		print "<refsect1>\n <title>$section</title>\n";
-	}
-	if ($section =~ m/EXAMPLE/i) {
-	    print "<example><para>\n";
-	    $output_preformatted = 1;
-	} else {
-	    print "<para>\n";
-	}
-	output_highlight($args{'sections'}{$section});
-	$output_preformatted = 0;
-	if ($section =~ m/EXAMPLE/i) {
-	    print "</para></example>\n";
-	} else {
-	    print "</para>";
-	}
-	if (!$args{'content-only'}) {
-		print "\n</refsect1>\n";
-	}
-    }
-
-    print "\n\n";
-}
-
-# output in XML DocBook
-sub output_function_gnome {
-    my %args = %{$_[0]};
-    my ($parameter, $section);
-    my $count;
-    my $id;
-
-    $id = $args{'module'} . "-" . $args{'function'};
-    $id =~ s/[^A-Za-z0-9]/-/g;
-
-    print "<sect2>\n";
-    print " <title id=\"$id\">" . $args{'function'} . "</title>\n";
-
-    print "  <funcsynopsis>\n";
-    print "   <funcdef>" . $args{'functiontype'} . " ";
-    print "<function>" . $args{'function'} . " ";
-    print "</function></funcdef>\n";
-
-    $count = 0;
-    if ($#{$args{'parameterlist'}} >= 0) {
-	foreach $parameter (@{$args{'parameterlist'}}) {
-	    $type = $args{'parametertypes'}{$parameter};
-	    if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
-		# pointer-to-function
-		print "   <paramdef>$1 <parameter>$parameter</parameter>)\n";
-		print "     <funcparams>$2</funcparams></paramdef>\n";
-	    } else {
-		print "   <paramdef>" . $type;
-		print " <parameter>$parameter</parameter></paramdef>\n";
-	    }
-	}
-    } else {
-	print "  <void>\n";
-    }
-    print "  </funcsynopsis>\n";
-    if ($#{$args{'parameterlist'}} >= 0) {
-	print " <informaltable pgwide=\"1\" frame=\"none\" role=\"params\">\n";
-	print "<tgroup cols=\"2\">\n";
-	print "<colspec colwidth=\"2*\">\n";
-	print "<colspec colwidth=\"8*\">\n";
-	print "<tbody>\n";
-	foreach $parameter (@{$args{'parameterlist'}}) {
-	    my $parameter_name = $parameter;
-	    $parameter_name =~ s/\[.*//;
-
-	    print "  <row><entry align=\"right\"><parameter>$parameter</parameter></entry>\n";
-	    print "   <entry>\n";
-	    $lineprefix="     ";
-	    output_highlight($args{'parameterdescs'}{$parameter_name});
-	    print "    </entry></row>\n";
-	}
-	print " </tbody></tgroup></informaltable>\n";
-    } else {
-	print " <para>\n  None\n </para>\n";
-    }
-
-    # print out each section
-    $lineprefix="   ";
-    foreach $section (@{$args{'sectionlist'}}) {
-	print "<simplesect>\n <title>$section</title>\n";
-	if ($section =~ m/EXAMPLE/i) {
-	    print "<example><programlisting>\n";
-	    $output_preformatted = 1;
-	} else {
-	}
-	print "<para>\n";
-	output_highlight($args{'sections'}{$section});
-	$output_preformatted = 0;
-	print "</para>\n";
-	if ($section =~ m/EXAMPLE/i) {
-	    print "</programlisting></example>\n";
-	} else {
-	}
-	print " </simplesect>\n";
-    }
-
-    print "</sect2>\n\n";
-}
-
 ##
 # output function in man
 sub output_function_man(%) {
@@ -1692,161 +738,6 @@ sub output_blockhead_man(%) {
 }
 
 ##
-# output in text
-sub output_function_text(%) {
-    my %args = %{$_[0]};
-    my ($parameter, $section);
-    my $start;
-
-    print "Name:\n\n";
-    print $args{'function'} . " - " . $args{'purpose'} . "\n";
-
-    print "\nSynopsis:\n\n";
-    if ($args{'functiontype'} ne "") {
-	$start = $args{'functiontype'} . " " . $args{'function'} . " (";
-    } else {
-	$start = $args{'function'} . " (";
-    }
-    print $start;
-
-    my $count = 0;
-    foreach my $parameter (@{$args{'parameterlist'}}) {
-	$type = $args{'parametertypes'}{$parameter};
-	if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
-	    # pointer-to-function
-	    print $1 . $parameter . ") (" . $2;
-	} else {
-	    print $type . " " . $parameter;
-	}
-	if ($count != $#{$args{'parameterlist'}}) {
-	    $count++;
-	    print ",\n";
-	    print " " x length($start);
-	} else {
-	    print ");\n\n";
-	}
-    }
-
-    print "Arguments:\n\n";
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	my $parameter_name = $parameter;
-	$parameter_name =~ s/\[.*//;
-
-	print $parameter . "\n\t" . $args{'parameterdescs'}{$parameter_name} . "\n";
-    }
-    output_section_text(@_);
-}
-
-#output sections in text
-sub output_section_text(%) {
-    my %args = %{$_[0]};
-    my $section;
-
-    print "\n";
-    foreach $section (@{$args{'sectionlist'}}) {
-	print "$section:\n\n";
-	output_highlight($args{'sections'}{$section});
-    }
-    print "\n\n";
-}
-
-# output enum in text
-sub output_enum_text(%) {
-    my %args = %{$_[0]};
-    my ($parameter);
-    my $count;
-    print "Enum:\n\n";
-
-    print "enum " . $args{'enum'} . " - " . $args{'purpose'} . "\n\n";
-    print "enum " . $args{'enum'} . " {\n";
-    $count = 0;
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	print "\t$parameter";
-	if ($count != $#{$args{'parameterlist'}}) {
-	    $count++;
-	    print ",";
-	}
-	print "\n";
-    }
-    print "};\n\n";
-
-    print "Constants:\n\n";
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	print "$parameter\n\t";
-	print $args{'parameterdescs'}{$parameter} . "\n";
-    }
-
-    output_section_text(@_);
-}
-
-# output typedef in text
-sub output_typedef_text(%) {
-    my %args = %{$_[0]};
-    my ($parameter);
-    my $count;
-    print "Typedef:\n\n";
-
-    print "typedef " . $args{'typedef'} . " - " . $args{'purpose'} . "\n";
-    output_section_text(@_);
-}
-
-# output struct as text
-sub output_struct_text(%) {
-    my %args = %{$_[0]};
-    my ($parameter);
-
-    print $args{'type'} . " " . $args{'struct'} . " - " . $args{'purpose'} . "\n\n";
-    print $args{'type'} . " " . $args{'struct'} . " {\n";
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	if ($parameter =~ /^#/) {
-	    print "$parameter\n";
-	    next;
-	}
-
-	my $parameter_name = $parameter;
-	$parameter_name =~ s/\[.*//;
-
-	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
-	$type = $args{'parametertypes'}{$parameter};
-	if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
-	    # pointer-to-function
-	    print "\t$1 $parameter) ($2);\n";
-	} elsif ($type =~ m/^(.*?)\s*(:.*)/) {
-	    # bitfield
-	    print "\t$1 $parameter$2;\n";
-	} else {
-	    print "\t" . $type . " " . $parameter . ";\n";
-	}
-    }
-    print "};\n\n";
-
-    print "Members:\n\n";
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	($parameter =~ /^#/) && next;
-
-	my $parameter_name = $parameter;
-	$parameter_name =~ s/\[.*//;
-
-	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
-	print "$parameter\n\t";
-	print $args{'parameterdescs'}{$parameter_name} . "\n";
-    }
-    print "\n";
-    output_section_text(@_);
-}
-
-sub output_blockhead_text(%) {
-    my %args = %{$_[0]};
-    my ($parameter, $section);
-
-    foreach $section (@{$args{'sectionlist'}}) {
-	print " $section:\n";
-	print "    -> ";
-	output_highlight($args{'sections'}{$section});
-    }
-}
-
-##
 # output in restructured text
 #
 
@@ -2080,43 +971,6 @@ sub output_struct_rst(%) {
     output_section_rst(@_);
 }
 
-
-## list mode output functions
-
-sub output_function_list(%) {
-    my %args = %{$_[0]};
-
-    print $args{'function'} . "\n";
-}
-
-# output enum in list
-sub output_enum_list(%) {
-    my %args = %{$_[0]};
-    print $args{'enum'} . "\n";
-}
-
-# output typedef in list
-sub output_typedef_list(%) {
-    my %args = %{$_[0]};
-    print $args{'typedef'} . "\n";
-}
-
-# output struct as list
-sub output_struct_list(%) {
-    my %args = %{$_[0]};
-
-    print $args{'struct'} . "\n";
-}
-
-sub output_blockhead_list(%) {
-    my %args = %{$_[0]};
-    my ($parameter, $section);
-
-    foreach $section (@{$args{'sectionlist'}}) {
-	print "DOC: $section\n";
-    }
-}
-
 ##
 # generic output function for all types (function, struct/union, typedef, enum);
 # calls the generated, variable output_ function name based on
@@ -2795,7 +1649,7 @@ sub process_proto_type($$) {
 # just before actual output; (this is done by local_unescape())
 sub xml_escape($) {
 	my $text = shift;
-	if (($output_mode eq "text") || ($output_mode eq "man")) {
+	if ($output_mode eq "man") {
 		return $text;
 	}
 	$text =~ s/\&/\\\\\\amp;/g;
@@ -2807,7 +1661,7 @@ sub xml_escape($) {
 # xml_unescape: reverse the effects of xml_escape
 sub xml_unescape($) {
 	my $text = shift;
-	if (($output_mode eq "text") || ($output_mode eq "man")) {
+	if ($output_mode eq "man") {
 		return $text;
 	}
 	$text =~ s/\\\\\\amp;/\&/g;
@@ -2820,7 +1674,7 @@ sub xml_unescape($) {
 # local escape strings look like:  '\\\\menmonic:' (that's 4 backslashes)
 sub local_unescape($) {
 	my $text = shift;
-	if (($output_mode eq "text") || ($output_mode eq "man")) {
+	if ($output_mode eq "man") {
 		return $text;
 	}
 	$text =~ s/\\\\\\\\lt:/</g;
@@ -3140,34 +1994,6 @@ sub process_file($) {
 	if (($output_selection == OUTPUT_INCLUDE) && ($show_not_found == 1)) {
 	    print STDERR "    Was looking for '$_'.\n" for keys %function_table;
 	}
-	if ($output_mode eq "xml") {
-	    # The template wants at least one RefEntry here; make one.
-	    print "<refentry>\n";
-	    print " <refnamediv>\n";
-	    print "  <refname>\n";
-	    print "   ${orig_file}\n";
-	    print "  </refname>\n";
-	    print "  <refpurpose>\n";
-	    print "   Document generation inconsistency\n";
-	    print "  </refpurpose>\n";
-	    print " </refnamediv>\n";
-	    print " <refsect1>\n";
-	    print "  <title>\n";
-	    print "   Oops\n";
-	    print "  </title>\n";
-	    print "  <warning>\n";
-	    print "   <para>\n";
-	    print "    The template for this document tried to insert\n";
-	    print "    the structured comment from the file\n";
-	    print "    <filename>${orig_file}</filename> at this point,\n";
-	    print "    but none was found.\n";
-	    print "    This dummy section is inserted to allow\n";
-	    print "    generation to continue.\n";
-	    print "   </para>\n";
-	    print "  </warning>\n";
-	    print " </refsect1>\n";
-	    print "</refentry>\n";
-	}
     }
 }
 
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v2 02/13] docs: kernel-doc.rst: better describe kernel-doc arguments
  2017-09-27 21:10 [PATCH v2 00/13] kernel-doc: add supported to document nested structs/$ Mauro Carvalho Chehab
  2017-09-27 21:10 ` [PATCH v2 01/13] scripts: kernel-doc: get rid of unused output formats Mauro Carvalho Chehab
@ 2017-09-27 21:10 ` Mauro Carvalho Chehab
  2017-09-27 21:10 ` [PATCH v2 03/13] docs: kernel-doc.rst: improve private members description Mauro Carvalho Chehab
                   ` (10 subsequent siblings)
  12 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-27 21:10 UTC (permalink / raw)
  To: Linux Media Mailing List, Jonathan Corbet
  Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter

Add a new section to describe kernel-doc arguments,
adding examples about how identation should happen, as failing
to do that causes Sphinx to do the wrong thing.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 Documentation/doc-guide/kernel-doc.rst | 44 +++++++++++++++++++++++++++++++---
 1 file changed, 41 insertions(+), 3 deletions(-)

diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst
index b24854b5d6be..662755f830d5 100644
--- a/Documentation/doc-guide/kernel-doc.rst
+++ b/Documentation/doc-guide/kernel-doc.rst
@@ -112,16 +112,17 @@ Example kernel-doc function comment::
 
   /**
    * foobar() - Brief description of foobar.
-   * @arg: Description of argument of foobar.
+   * @argument1: Description of parameter argument1 of foobar.
+   * @argument2: Description of parameter argument2 of foobar.
    *
    * Longer description of foobar.
    *
    * Return: Description of return value of foobar.
    */
-  int foobar(int arg)
+  int foobar(int argument1, char *argument2)
 
 The format is similar for documentation for structures, enums, paragraphs,
-etc. See the sections below for details.
+etc. See the sections below for specific details of each type.
 
 The kernel-doc structure is extracted from the comments, and proper `Sphinx C
 Domain`_ function and type descriptions with anchors are generated for them. The
@@ -130,6 +131,43 @@ cross-references. See below for details.
 
 .. _Sphinx C Domain: http://www.sphinx-doc.org/en/stable/domains.html
 
+
+Parameters and member arguments
+-------------------------------
+
+The kernel-doc function comments describe each parameter to the function and
+function typedefs or each member of struct/union, in order, with the
+``@argument:`` descriptions. For each non-private member argument, one
+``@argument`` definition is needed.
+
+The ``@argument:`` descriptions begin on the very next line following
+the opening brief function description line, with no intervening blank
+comment lines.
+
+The ``@argument:`` descriptions may span multiple lines.
+
+.. note::
+
+   If the ``@argument`` description has multiple lines, the continuation
+   of the description should be starting exactly at the same column as
+   the previous line, e. g.::
+
+      * @argument: some long description
+      *       that continues on next lines
+
+   or::
+
+      * @argument:
+      *		some long description
+      *		that continues on next lines
+
+If a function or typedef parameter argument is ``...`` (e. g. a variable
+number of arguments), its description should be listed in kernel-doc
+notation as::
+
+      * @...: description
+
+
 Highlights and cross-references
 -------------------------------
 
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v2 03/13] docs: kernel-doc.rst: improve private members description
  2017-09-27 21:10 [PATCH v2 00/13] kernel-doc: add supported to document nested structs/$ Mauro Carvalho Chehab
  2017-09-27 21:10 ` [PATCH v2 01/13] scripts: kernel-doc: get rid of unused output formats Mauro Carvalho Chehab
  2017-09-27 21:10 ` [PATCH v2 02/13] docs: kernel-doc.rst: better describe kernel-doc arguments Mauro Carvalho Chehab
@ 2017-09-27 21:10 ` Mauro Carvalho Chehab
  2017-09-27 21:10 ` [PATCH v2 04/13] docs: kernel-doc.rst: improve function documentation section Mauro Carvalho Chehab
                   ` (9 subsequent siblings)
  12 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-27 21:10 UTC (permalink / raw)
  To: Linux Media Mailing List, Jonathan Corbet
  Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter

The private members section can now be moved to be together
with the arguments section. Move it there and add an example
about the usage of public:

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 Documentation/doc-guide/kernel-doc.rst | 56 ++++++++++++++++++----------------
 1 file changed, 30 insertions(+), 26 deletions(-)

diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst
index 662755f830d5..146041fc494a 100644
--- a/Documentation/doc-guide/kernel-doc.rst
+++ b/Documentation/doc-guide/kernel-doc.rst
@@ -167,6 +167,36 @@ notation as::
 
       * @...: description
 
+Private members
+~~~~~~~~~~~~~~~
+
+Inside a struct or union description, you can use the ``private:`` and
+``public:`` comment tags. Structure fields that are inside a ``private:``
+area are not listed in the generated output documentation.
+
+The ``private:`` and ``public:`` tags must begin immediately following a
+``/*`` comment marker.  They may optionally include comments between the
+``:`` and the ending ``*/`` marker.
+
+Example::
+
+  /**
+   * struct my_struct - short description
+   * @a: first member
+   * @b: second member
+   * @d: fourth member
+   *
+   * Longer description
+   */
+  struct my_struct {
+      int a;
+      int b;
+  /* private: internal use only */
+      int c;
+  /* public: the next one is public */
+      int d;
+  };
+
 
 Highlights and cross-references
 -------------------------------
@@ -332,32 +362,6 @@ on a line of their own, like all other kernel-doc comments::
         int foobar;
   }
 
-Private members
-~~~~~~~~~~~~~~~
-
-Inside a struct description, you can use the "private:" and "public:" comment
-tags. Structure fields that are inside a "private:" area are not listed in the
-generated output documentation.  The "private:" and "public:" tags must begin
-immediately following a ``/*`` comment marker.  They may optionally include
-comments between the ``:`` and the ending ``*/`` marker.
-
-Example::
-
-  /**
-   * struct my_struct - short description
-   * @a: first member
-   * @b: second member
-   *
-   * Longer description
-   */
-  struct my_struct {
-      int a;
-      int b;
-  /* private: internal use only */
-      int c;
-  };
-
-
 Typedef documentation
 ---------------------
 
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v2 04/13] docs: kernel-doc.rst: improve function documentation section
  2017-09-27 21:10 [PATCH v2 00/13] kernel-doc: add supported to document nested structs/$ Mauro Carvalho Chehab
                   ` (2 preceding siblings ...)
  2017-09-27 21:10 ` [PATCH v2 03/13] docs: kernel-doc.rst: improve private members description Mauro Carvalho Chehab
@ 2017-09-27 21:10 ` Mauro Carvalho Chehab
  2017-09-27 21:10 ` [PATCH v2 05/13] docs: kernel-doc.rst: improve structs chapter Mauro Carvalho Chehab
                   ` (8 subsequent siblings)
  12 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-27 21:10 UTC (permalink / raw)
  To: Linux Media Mailing List, Jonathan Corbet
  Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter

Move its contents to happen earlier and improve the description
of return values, adding a subsection to it. Most of the contents
there came from kernel-doc-nano-HOWTO.txt.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 Documentation/doc-guide/kernel-doc.rst | 100 ++++++++++++++++++++-------------
 1 file changed, 61 insertions(+), 39 deletions(-)

diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst
index 146041fc494a..ad0fb487e423 100644
--- a/Documentation/doc-guide/kernel-doc.rst
+++ b/Documentation/doc-guide/kernel-doc.rst
@@ -197,6 +197,67 @@ Example::
       int d;
   };
 
+Function documentation
+----------------------
+
+The general format of a function and function-like macro kernel-doc comment is::
+
+  /**
+   * function_name() - Brief description of function.
+   * @arg1: Describe the first argument.
+   * @arg2: Describe the second argument.
+   *        One can provide multiple line descriptions
+   *        for arguments.
+   *
+   * A longer description, with more discussion of the function function_name()
+   * that might be useful to those using or modifying it. Begins with an
+   * empty comment line, and may include additional embedded empty
+   * comment lines.
+   *
+   * The longer description may have multiple paragraphs.
+   *
+   * Return: Describe the return value of foobar.
+   *
+   * The return value description can also have multiple paragraphs, and should
+   * be placed at the end of the comment block.
+   */
+
+The brief description following the function name may span multiple lines, and
+ends with an argument description, a blank comment line, or the end of the
+comment block.
+
+Return values
+~~~~~~~~~~~~~
+
+The return value, if any, should be described in a dedicated section
+named ``Return``.
+
+.. note::
+
+  #) The multi-line descriptive text you provide does *not* recognize
+     line breaks, so if you try to format some text nicely, as in::
+
+	* Return:
+	* 0 - OK
+	* -EINVAL - invalid argument
+	* -ENOMEM - out of memory
+
+     this will all run together and produce::
+
+	Return: 0 - OK -EINVAL - invalid argument -ENOMEM - out of memory
+
+     So, in order to produce the desired line breaks, you need to use a
+     ReST list, e. g.::
+
+      * Return:
+      * * 0		- OK to runtime suspend the device
+      * * -EBUSY	- Device should not be runtime suspended
+
+  #) If the descriptive text you provide has lines that begin with
+     some phrase followed by a colon, each of those phrases will be taken
+     as a new section heading, with probably won't produce the desired
+     effect.
+
 
 Highlights and cross-references
 -------------------------------
@@ -269,45 +330,6 @@ cross-references.
 
 For further details, please refer to the `Sphinx C Domain`_ documentation.
 
-Function documentation
-----------------------
-
-The general format of a function and function-like macro kernel-doc comment is::
-
-  /**
-   * function_name() - Brief description of function.
-   * @arg1: Describe the first argument.
-   * @arg2: Describe the second argument.
-   *        One can provide multiple line descriptions
-   *        for arguments.
-   *
-   * A longer description, with more discussion of the function function_name()
-   * that might be useful to those using or modifying it. Begins with an
-   * empty comment line, and may include additional embedded empty
-   * comment lines.
-   *
-   * The longer description may have multiple paragraphs.
-   *
-   * Return: Describe the return value of foobar.
-   *
-   * The return value description can also have multiple paragraphs, and should
-   * be placed at the end of the comment block.
-   */
-
-The brief description following the function name may span multiple lines, and
-ends with an ``@argument:`` description, a blank comment line, or the end of the
-comment block.
-
-The kernel-doc function comments describe each parameter to the function, in
-order, with the ``@argument:`` descriptions. The ``@argument:`` descriptions
-must begin on the very next line following the opening brief function
-description line, with no intervening blank comment lines. The ``@argument:``
-descriptions may span multiple lines. The continuation lines may contain
-indentation. If a function parameter is ``...`` (varargs), it should be listed
-in kernel-doc notation as: ``@...:``.
-
-The return value, if any, should be described in a dedicated section at the end
-of the comment starting with "Return:".
 
 Structure, union, and enumeration documentation
 -----------------------------------------------
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v2 05/13] docs: kernel-doc.rst: improve structs chapter
  2017-09-27 21:10 [PATCH v2 00/13] kernel-doc: add supported to document nested structs/$ Mauro Carvalho Chehab
                   ` (3 preceding siblings ...)
  2017-09-27 21:10 ` [PATCH v2 04/13] docs: kernel-doc.rst: improve function documentation section Mauro Carvalho Chehab
@ 2017-09-27 21:10 ` Mauro Carvalho Chehab
  2017-09-27 21:10 ` [PATCH v2 06/13] docs: kernel-doc.rst: improve typedef documentation Mauro Carvalho Chehab
                   ` (7 subsequent siblings)
  12 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-27 21:10 UTC (permalink / raw)
  To: Linux Media Mailing List, Jonathan Corbet
  Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter

There is a mess on this chapter: it suggests that even
enums and unions should be documented with "struct". That's
not the way it should be ;-)

Fix it and move it to happen earlier.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 Documentation/doc-guide/kernel-doc.rst | 48 +++++++++++++++++-----------------
 1 file changed, 24 insertions(+), 24 deletions(-)

diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst
index ad0fb487e423..b330234a3601 100644
--- a/Documentation/doc-guide/kernel-doc.rst
+++ b/Documentation/doc-guide/kernel-doc.rst
@@ -258,6 +258,30 @@ named ``Return``.
      as a new section heading, with probably won't produce the desired
      effect.
 
+Structure, union, and enumeration documentation
+-----------------------------------------------
+
+The general format of a struct, union, and enum kernel-doc comment is::
+
+  /**
+   * struct struct_name - Brief description.
+   * @argument: Description of member member_name.
+   *
+   * Description of the structure.
+   */
+
+On the above, ``struct`` is used to mean structs. You can also use ``union``
+and ``enum``  to describe unions and enums. ``argument`` is used
+to mean struct and union member names as well as enumerations in an enum.
+
+The brief description following the structure name may span multiple lines, and
+ends with a member description, a blank comment line, or the end of the
+comment block.
+
+The kernel-doc data structure comments describe each member of the structure,
+in order, with the member descriptions.
+
+
 
 Highlights and cross-references
 -------------------------------
@@ -331,30 +355,6 @@ cross-references.
 For further details, please refer to the `Sphinx C Domain`_ documentation.
 
 
-Structure, union, and enumeration documentation
------------------------------------------------
-
-The general format of a struct, union, and enum kernel-doc comment is::
-
-  /**
-   * struct struct_name - Brief description.
-   * @member_name: Description of member member_name.
-   *
-   * Description of the structure.
-   */
-
-Below, "struct" is used to mean structs, unions and enums, and "member" is used
-to mean struct and union members as well as enumerations in an enum.
-
-The brief description following the structure name may span multiple lines, and
-ends with a ``@member:`` description, a blank comment line, or the end of the
-comment block.
-
-The kernel-doc data structure comments describe each member of the structure, in
-order, with the ``@member:`` descriptions. The ``@member:`` descriptions must
-begin on the very next line following the opening brief function description
-line, with no intervening blank comment lines. The ``@member:`` descriptions may
-span multiple lines. The continuation lines may contain indentation.
 
 In-line member documentation comments
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v2 06/13] docs: kernel-doc.rst: improve typedef documentation
  2017-09-27 21:10 [PATCH v2 00/13] kernel-doc: add supported to document nested structs/$ Mauro Carvalho Chehab
                   ` (4 preceding siblings ...)
  2017-09-27 21:10 ` [PATCH v2 05/13] docs: kernel-doc.rst: improve structs chapter Mauro Carvalho Chehab
@ 2017-09-27 21:10 ` Mauro Carvalho Chehab
  2017-09-27 21:10 ` [PATCH v2 07/13] docs: kernel-doc.rst: add documentation about man pages Mauro Carvalho Chehab
                   ` (6 subsequent siblings)
  12 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-27 21:10 UTC (permalink / raw)
  To: Linux Media Mailing List, Jonathan Corbet
  Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter

Add documentation about typedefs for function prototypes and
move it to happen earlier.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 Documentation/doc-guide/kernel-doc.rst | 32 ++++++++++++++++++++++----------
 1 file changed, 22 insertions(+), 10 deletions(-)

diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst
index b330234a3601..0923c8bd5769 100644
--- a/Documentation/doc-guide/kernel-doc.rst
+++ b/Documentation/doc-guide/kernel-doc.rst
@@ -282,6 +282,28 @@ The kernel-doc data structure comments describe each member of the structure,
 in order, with the member descriptions.
 
 
+Typedef documentation
+---------------------
+
+The general format of a typedef kernel-doc comment is::
+
+  /**
+   * typedef type_name - Brief description.
+   *
+   * Description of the type.
+   */
+
+Typedefs with function prototypes can also be documented::
+
+  /**
+   * typedef type_name - Brief description.
+   * @arg1: description of arg1
+   * @arg2: description of arg2
+   *
+   * Description of the type.
+   */
+   typedef void (*type_name)(struct v4l2_ctrl *arg1, void *arg2);
+
 
 Highlights and cross-references
 -------------------------------
@@ -384,16 +406,6 @@ on a line of their own, like all other kernel-doc comments::
         int foobar;
   }
 
-Typedef documentation
----------------------
-
-The general format of a typedef kernel-doc comment is::
-
-  /**
-   * typedef type_name - Brief description.
-   *
-   * Description of the type.
-   */
 
 Overview documentation comments
 -------------------------------
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v2 07/13] docs: kernel-doc.rst: add documentation about man pages
  2017-09-27 21:10 [PATCH v2 00/13] kernel-doc: add supported to document nested structs/$ Mauro Carvalho Chehab
                   ` (5 preceding siblings ...)
  2017-09-27 21:10 ` [PATCH v2 06/13] docs: kernel-doc.rst: improve typedef documentation Mauro Carvalho Chehab
@ 2017-09-27 21:10 ` Mauro Carvalho Chehab
  2017-09-27 21:20   ` Randy Dunlap
  2017-09-27 21:10 ` [PATCH v2 08/13] docs: get rid of kernel-doc-nano-HOWTO.txt Mauro Carvalho Chehab
                   ` (5 subsequent siblings)
  12 siblings, 1 reply; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-27 21:10 UTC (permalink / raw)
  To: Linux Media Mailing List, Jonathan Corbet
  Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter

kernel-doc-nano-HOWTO.txt has a chapter about man pages
production. While we don't have a working  "make manpages"
target, add it.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 Documentation/doc-guide/kernel-doc.rst | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst
index 0923c8bd5769..96012f9e314d 100644
--- a/Documentation/doc-guide/kernel-doc.rst
+++ b/Documentation/doc-guide/kernel-doc.rst
@@ -452,3 +452,37 @@ file.
 
 Data structures visible in kernel include files should also be documented using
 kernel-doc formatted comments.
+
+How to use kernel-doc to generate man pages
+-------------------------------------------
+
+If you just want to use kernel-doc to generate man pages you can do this
+from the Kernel git tree::
+
+  $ scripts/kernel-doc -man $(git grep -l '/\*\*' |grep -v Documentation/) | ./split-man.pl /tmp/man
+
+Using the small ``split-man.pl`` script below::
+
+
+  #!/usr/bin/perl
+
+  if ($#ARGV < 0) {
+     die "where do I put the results?\n";
+  }
+
+  mkdir $ARGV[0],0777;
+  $state = 0;
+  while (<STDIN>) {
+      if (/^\.TH \"[^\"]*\" 9 \"([^\"]*)\"/) {
+	if ($state == 1) { close OUT }
+	$state = 1;
+	$fn = "$ARGV[0]/$1.9";
+	print STDERR "Creating $fn\n";
+	open OUT, ">$fn" or die "can't open $fn: $!\n";
+	print OUT $_;
+      } elsif ($state != 0) {
+	print OUT $_;
+      }
+  }
+
+  close OUT;
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v2 08/13] docs: get rid of kernel-doc-nano-HOWTO.txt
  2017-09-27 21:10 [PATCH v2 00/13] kernel-doc: add supported to document nested structs/$ Mauro Carvalho Chehab
                   ` (6 preceding siblings ...)
  2017-09-27 21:10 ` [PATCH v2 07/13] docs: kernel-doc.rst: add documentation about man pages Mauro Carvalho Chehab
@ 2017-09-27 21:10 ` Mauro Carvalho Chehab
  2017-09-27 21:10 ` [PATCH v2 09/13] scripts: kernel-doc: parse next structs/unions Mauro Carvalho Chehab
                   ` (4 subsequent siblings)
  12 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-27 21:10 UTC (permalink / raw)
  To: Linux Media Mailing List, Jonathan Corbet
  Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, Linux Doc Mailing List

Everything there is already described at
Documentation/doc-guide/kernel-doc.rst. So, there's no reason why
to keep it anymore.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 Documentation/00-INDEX                  |   2 -
 Documentation/kernel-doc-nano-HOWTO.txt | 322 --------------------------------
 scripts/kernel-doc                      |   2 +-
 3 files changed, 1 insertion(+), 325 deletions(-)
 delete mode 100644 Documentation/kernel-doc-nano-HOWTO.txt

diff --git a/Documentation/00-INDEX b/Documentation/00-INDEX
index 3bec49c33bbb..aca4f00ec69b 100644
--- a/Documentation/00-INDEX
+++ b/Documentation/00-INDEX
@@ -228,8 +228,6 @@ isdn/
 	- directory with info on the Linux ISDN support, and supported cards.
 kbuild/
 	- directory with info about the kernel build process.
-kernel-doc-nano-HOWTO.txt
-	- outdated info about kernel-doc documentation.
 kdump/
 	- directory with mini HowTo on getting the crash dump code to work.
 doc-guide/
diff --git a/Documentation/kernel-doc-nano-HOWTO.txt b/Documentation/kernel-doc-nano-HOWTO.txt
deleted file mode 100644
index c23e2c5ab80d..000000000000
--- a/Documentation/kernel-doc-nano-HOWTO.txt
+++ /dev/null
@@ -1,322 +0,0 @@
-NOTE: this document is outdated and will eventually be removed.  See
-Documentation/doc-guide/ for current information.
-
-kernel-doc nano-HOWTO
-=====================
-
-How to format kernel-doc comments
----------------------------------
-
-In order to provide embedded, 'C' friendly, easy to maintain,
-but consistent and extractable documentation of the functions and
-data structures in the Linux kernel, the Linux kernel has adopted
-a consistent style for documenting functions and their parameters,
-and structures and their members.
-
-The format for this documentation is called the kernel-doc format.
-It is documented in this Documentation/kernel-doc-nano-HOWTO.txt file.
-
-This style embeds the documentation within the source files, using
-a few simple conventions.  The scripts/kernel-doc perl script, the
-Documentation/sphinx/kerneldoc.py Sphinx extension and other tools understand
-these conventions, and are used to extract this embedded documentation
-into various documents.
-
-In order to provide good documentation of kernel functions and data
-structures, please use the following conventions to format your
-kernel-doc comments in Linux kernel source.
-
-We definitely need kernel-doc formatted documentation for functions
-that are exported to loadable modules using EXPORT_SYMBOL.
-
-We also look to provide kernel-doc formatted documentation for
-functions externally visible to other kernel files (not marked
-"static").
-
-We also recommend providing kernel-doc formatted documentation
-for private (file "static") routines, for consistency of kernel
-source code layout.  But this is lower priority and at the
-discretion of the MAINTAINER of that kernel source file.
-
-Data structures visible in kernel include files should also be
-documented using kernel-doc formatted comments.
-
-The opening comment mark "/**" is reserved for kernel-doc comments.
-Only comments so marked will be considered by the kernel-doc scripts,
-and any comment so marked must be in kernel-doc format.  Do not use
-"/**" to be begin a comment block unless the comment block contains
-kernel-doc formatted comments.  The closing comment marker for
-kernel-doc comments can be either "*/" or "**/", but "*/" is
-preferred in the Linux kernel tree.
-
-Kernel-doc comments should be placed just before the function
-or data structure being described.
-
-Example kernel-doc function comment:
-
-/**
- * foobar() - short function description of foobar
- * @arg1:	Describe the first argument to foobar.
- * @arg2:	Describe the second argument to foobar.
- *		One can provide multiple line descriptions
- *		for arguments.
- *
- * A longer description, with more discussion of the function foobar()
- * that might be useful to those using or modifying it.  Begins with
- * empty comment line, and may include additional embedded empty
- * comment lines.
- *
- * The longer description can have multiple paragraphs.
- *
- * Return: Describe the return value of foobar.
- */
-
-The short description following the subject can span multiple lines
-and ends with an @argument description, an empty line or the end of
-the comment block.
-
-The @argument descriptions must begin on the very next line following
-this opening short function description line, with no intervening
-empty comment lines.
-
-If a function parameter is "..." (varargs), it should be listed in
-kernel-doc notation as:
- * @...: description
-
-The return value, if any, should be described in a dedicated section
-named "Return".
-
-Example kernel-doc data structure comment.
-
-/**
- * struct blah - the basic blah structure
- * @mem1:	describe the first member of struct blah
- * @mem2:	describe the second member of struct blah,
- *		perhaps with more lines and words.
- *
- * Longer description of this structure.
- */
-
-The kernel-doc function comments describe each parameter to the
-function, in order, with the @name lines.
-
-The kernel-doc data structure comments describe each structure member
-in the data structure, with the @name lines.
-
-The longer description formatting is "reflowed", losing your line
-breaks.  So presenting carefully formatted lists within these
-descriptions won't work so well; derived documentation will lose
-the formatting.
-
-See the section below "How to add extractable documentation to your
-source files" for more details and notes on how to format kernel-doc
-comments.
-
-Components of the kernel-doc system
------------------------------------
-
-Many places in the source tree have extractable documentation in the
-form of block comments above functions.  The components of this system
-are:
-
-- scripts/kernel-doc
-
-  This is a perl script that hunts for the block comments and can mark
-  them up directly into DocBook, ReST, man, text, and HTML. (No, not
-  texinfo.)
-
-- scripts/docproc.c
-
-  This is a program for converting SGML template files into SGML
-  files. When a file is referenced it is searched for symbols
-  exported (EXPORT_SYMBOL), to be able to distinguish between internal
-  and external functions.
-  It invokes kernel-doc, giving it the list of functions that
-  are to be documented.
-  Additionally it is used to scan the SGML template files to locate
-  all the files referenced herein. This is used to generate dependency
-  information as used by make.
-
-- Makefile
-
-  The targets 'xmldocs', 'latexdocs', 'pdfdocs', 'epubdocs'and 'htmldocs'
-  are used to build XML DocBook files, LaTeX files, PDF files,
-  ePub files and html files in Documentation/.
-
-How to extract the documentation
---------------------------------
-
-If you just want to read the ready-made books on the various
-subsystems, just type 'make epubdocs', or 'make pdfdocs', or 'make htmldocs',
-depending on your preference.  If you would rather read a different format,
-you can type 'make xmldocs' and then use DocBook tools to convert
-Documentation/output/*.xml to a format of your choice (for example,
-'db2html ...' if 'make htmldocs' was not defined).
-
-If you want to see man pages instead, you can do this:
-
-$ cd linux
-$ scripts/kernel-doc -man $(find -name '*.c') | split-man.pl /tmp/man
-$ scripts/kernel-doc -man $(find -name '*.h') | split-man.pl /tmp/man
-
-Here is split-man.pl:
-
--->
-#!/usr/bin/perl
-
-if ($#ARGV < 0) {
-   die "where do I put the results?\n";
-}
-
-mkdir $ARGV[0],0777;
-$state = 0;
-while (<STDIN>) {
-    if (/^\.TH \"[^\"]*\" 9 \"([^\"]*)\"/) {
-	if ($state == 1) { close OUT }
-	$state = 1;
-	$fn = "$ARGV[0]/$1.9";
-	print STDERR "Creating $fn\n";
-	open OUT, ">$fn" or die "can't open $fn: $!\n";
-	print OUT $_;
-    } elsif ($state != 0) {
-	print OUT $_;
-    }
-}
-
-close OUT;
-<--
-
-If you just want to view the documentation for one function in one
-file, you can do this:
-
-$ scripts/kernel-doc -man -function fn file | nroff -man | less
-
-or this:
-
-$ scripts/kernel-doc -text -function fn file
-
-
-How to add extractable documentation to your source files
----------------------------------------------------------
-
-The format of the block comment is like this:
-
-/**
- * function_name(:)? (- short description)?
-(* @parameterx(space)*: (description of parameter x)?)*
-(* a blank line)?
- * (Description:)? (Description of function)?
- * (section header: (section description)? )*
-(*)?*/
-
-All "description" text can span multiple lines, although the
-function_name & its short description are traditionally on a single line.
-Description text may also contain blank lines (i.e., lines that contain
-only a "*").
-
-"section header:" names must be unique per function (or struct,
-union, typedef, enum).
-
-Use the section header "Return" for sections describing the return value
-of a function.
-
-Avoid putting a spurious blank line after the function name, or else the
-description will be repeated!
-
-All descriptive text is further processed, scanning for the following special
-patterns, which are highlighted appropriately.
-
-'funcname()' - function
-'$ENVVAR' - environment variable
-'&struct_name' - name of a structure (up to two words including 'struct')
-'@parameter' - name of a parameter
-'%CONST' - name of a constant.
-
-NOTE 1:  The multi-line descriptive text you provide does *not* recognize
-line breaks, so if you try to format some text nicely, as in:
-
-  Return:
-    0 - cool
-    1 - invalid arg
-    2 - out of memory
-
-this will all run together and produce:
-
-  Return: 0 - cool 1 - invalid arg 2 - out of memory
-
-NOTE 2:  If the descriptive text you provide has lines that begin with
-some phrase followed by a colon, each of those phrases will be taken as
-a new section heading, which means you should similarly try to avoid text
-like:
-
-  Return:
-    0: cool
-    1: invalid arg
-    2: out of memory
-
-every line of which would start a new section.  Again, probably not
-what you were after.
-
-Take a look around the source tree for examples.
-
-
-kernel-doc for structs, unions, enums, and typedefs
----------------------------------------------------
-
-Beside functions you can also write documentation for structs, unions,
-enums and typedefs. Instead of the function name you must write the name
-of the declaration;  the struct/union/enum/typedef must always precede
-the name. Nesting of declarations is not supported.
-Use the argument mechanism to document members or constants.
-
-Inside a struct description, you can use the "private:" and "public:"
-comment tags.  Structure fields that are inside a "private:" area
-are not listed in the generated output documentation.  The "private:"
-and "public:" tags must begin immediately following a "/*" comment
-marker.  They may optionally include comments between the ":" and the
-ending "*/" marker.
-
-Example:
-
-/**
- * struct my_struct - short description
- * @a: first member
- * @b: second member
- *
- * Longer description
- */
-struct my_struct {
-    int a;
-    int b;
-/* private: internal use only */
-    int c;
-};
-
-
-Including documentation blocks in source files
-----------------------------------------------
-
-To facilitate having source code and comments close together, you can
-include kernel-doc documentation blocks that are free-form comments
-instead of being kernel-doc for functions, structures, unions,
-enums, or typedefs.  This could be used for something like a
-theory of operation for a driver or library code, for example.
-
-This is done by using a DOC: section keyword with a section title.  E.g.:
-
-/**
- * DOC: Theory of Operation
- *
- * The whizbang foobar is a dilly of a gizmo.  It can do whatever you
- * want it to do, at any time.  It reads your mind.  Here's how it works.
- *
- * foo bar splat
- *
- * The only drawback to this gizmo is that is can sometimes damage
- * hardware, software, or its subject(s).
- */
-
-DOC: sections are used in ReST files.
-
-Tim.
-*/ <twaugh@redhat.com>
diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 69757ee9db4c..b6f3f6962897 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -48,7 +48,7 @@ Read C language source or header FILEs, extract embedded documentation comments,
 and print formatted documentation to standard output.
 
 The documentation comments are identified by "/**" opening comment mark. See
-Documentation/kernel-doc-nano-HOWTO.txt for the documentation comment syntax.
+Documentation/doc-guide/kernel-doc.rst for the documentation comment syntax.
 
 Output format selection (mutually exclusive):
   -man			Output troff manual page format. This is the default.
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v2 09/13] scripts: kernel-doc: parse next structs/unions
  2017-09-27 21:10 [PATCH v2 00/13] kernel-doc: add supported to document nested structs/$ Mauro Carvalho Chehab
                   ` (7 preceding siblings ...)
  2017-09-27 21:10 ` [PATCH v2 08/13] docs: get rid of kernel-doc-nano-HOWTO.txt Mauro Carvalho Chehab
@ 2017-09-27 21:10 ` Mauro Carvalho Chehab
  2017-09-28 16:28   ` Markus Heiser
  2017-09-27 21:10 ` [PATCH v2 10/13] scripts: kernel-doc: get rid of $nested parameter Mauro Carvalho Chehab
                   ` (3 subsequent siblings)
  12 siblings, 1 reply; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-27 21:10 UTC (permalink / raw)
  To: Linux Media Mailing List, Jonathan Corbet
  Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter

There are several places within the Kernel tree with nested
structs/unions, like this one:

  struct ingenic_cgu_clk_info {
    const char *name;
    enum {
      CGU_CLK_NONE = 0,
      CGU_CLK_EXT = BIT(0),
      CGU_CLK_PLL = BIT(1),
      CGU_CLK_GATE = BIT(2),
      CGU_CLK_MUX = BIT(3),
      CGU_CLK_MUX_GLITCHFREE = BIT(4),
      CGU_CLK_DIV = BIT(5),
      CGU_CLK_FIXDIV = BIT(6),
      CGU_CLK_CUSTOM = BIT(7),
    } type;
    int parents[4];
    union {
      struct ingenic_cgu_pll_info pll;
      struct {
        struct ingenic_cgu_gate_info gate;
        struct ingenic_cgu_mux_info mux;
        struct ingenic_cgu_div_info div;
        struct ingenic_cgu_fixdiv_info fixdiv;
      };
      struct ingenic_cgu_custom_info custom;
    };
  };

Currently, such struct is documented as:

	**Definition**

	::
	struct ingenic_cgu_clk_info {
	    const char * name;
	};

	**Members**

	``name``
	  name of the clock

With is obvioulsy wrong. It also generates an error:
	drivers/clk/ingenic/cgu.h:169: warning: No description found for parameter 'enum'

However, there's nothing wrong with this kernel-doc markup: everything
is documented there.

It makes sense to document all fields there. So, add a
way for the core to parse those structs.

With this patch, all documented fields will properly generate
documentation.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 Documentation/doc-guide/kernel-doc.rst |  46 ++++++++++++
 scripts/kernel-doc                     | 123 ++++++++++++++++++---------------
 2 files changed, 115 insertions(+), 54 deletions(-)

diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst
index 96012f9e314d..9e63a18cceea 100644
--- a/Documentation/doc-guide/kernel-doc.rst
+++ b/Documentation/doc-guide/kernel-doc.rst
@@ -281,6 +281,52 @@ comment block.
 The kernel-doc data structure comments describe each member of the structure,
 in order, with the member descriptions.
 
+Nested structs/unions
+~~~~~~~~~~~~~~~~~~~~~
+
+It is possible to document nested structs unions, like::
+
+      /**
+       * struct nested_foobar - a struct with nested unions and structs
+       * @arg1: - first argument of anonymous union/anonymous struct
+       * @arg2: - second argument of anonymous union/anonymous struct
+       * @arg3: - third argument of anonymous union/anonymous struct
+       * @arg4: - fourth argument of anonymous union/anonymous struct
+       * @bar.st1.arg1 - first argument of struct st1 on union bar
+       * @bar.st1.arg2 - second argument of struct st1 on union bar
+       * @bar.st2.arg1 - first argument of struct st2 on union bar
+       * @bar.st2.arg2 - second argument of struct st2 on union bar
+      struct nested_foobar {
+        /* Anonymous union/struct*/
+        union {
+          struct {
+            int arg1;
+            int arg2;
+	  }
+          struct {
+            void *arg3;
+            int arg4;
+	  }
+	}
+	union {
+          struct {
+            int arg1;
+            int arg2;
+	  } st1;
+          struct {
+            void *arg1;
+            int arg2;
+	  } st2;
+	} bar;
+      };
+
+.. note::
+
+   #) When documenting nested structs or unions, if the struct/union ``foo``
+      is named, the argument ``bar`` inside it should be documented as
+      ``@foo.bar:``
+   #) When the nested struct/union is anonymous, the argument ``bar`` on it
+      should be documented as ``@bar:``
 
 Typedef documentation
 ---------------------
diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index b6f3f6962897..63aa9f85d635 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -210,7 +210,7 @@ my $anon_struct_union = 0;
 my $type_constant = '\b``([^\`]+)``\b';
 my $type_constant2 = '\%([-_\w]+)';
 my $type_func = '(\w+)\(\)';
-my $type_param = '\@(\w+(\.\.\.)?)';
+my $type_param = '\@(\w*(\.\w+)*(\.\.\.)?)';
 my $type_fp_param = '\@(\w+)\(\)';  # Special RST handling for func ptr params
 my $type_env = '(\$\w+)';
 my $type_enum = '\&(enum\s*([_\w]+))';
@@ -663,32 +663,12 @@ sub output_struct_man(%) {
     print ".SH NAME\n";
     print $args{'type'} . " " . $args{'struct'} . " \\- " . $args{'purpose'} . "\n";
 
+    my $declaration = $args{'definition'};
+    $declaration =~ s/\t/  /g;
+    $declaration =~ s/\n/"\n.br\n.BI \"/g;
     print ".SH SYNOPSIS\n";
     print $args{'type'} . " " . $args{'struct'} . " {\n.br\n";
-
-    foreach my $parameter (@{$args{'parameterlist'}}) {
-	if ($parameter =~ /^#/) {
-	    print ".BI \"$parameter\"\n.br\n";
-	    next;
-	}
-	my $parameter_name = $parameter;
-	$parameter_name =~ s/\[.*//;
-
-	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
-	$type = $args{'parametertypes'}{$parameter};
-	if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
-	    # pointer-to-function
-	    print ".BI \"    " . $1 . "\" " . $parameter . " \") (" . $2 . ")" . "\"\n;\n";
-	} elsif ($type =~ m/^(.*?)\s*(:.*)/) {
-	    # bitfield
-	    print ".BI \"    " . $1 . "\ \" " . $parameter . $2 . " \"" . "\"\n;\n";
-	} else {
-	    $type =~ s/([^\*])$/$1 /;
-	    print ".BI \"    " . $type . "\" " . $parameter . " \"" . "\"\n;\n";
-	}
-	print "\n.br\n";
-    }
-    print "};\n.br\n";
+    print ".BI \"$declaration\n};\n.br\n\n";
 
     print ".SH Members\n";
     foreach $parameter (@{$args{'parameterlist'}}) {
@@ -926,29 +906,9 @@ sub output_struct_rst(%) {
 
     print "**Definition**\n\n";
     print "::\n\n";
-    print "  " . $args{'type'} . " " . $args{'struct'} . " {\n";
-    foreach $parameter (@{$args{'parameterlist'}}) {
-	if ($parameter =~ /^#/) {
-	    print "  " . "$parameter\n";
-	    next;
-	}
-
-	my $parameter_name = $parameter;
-	$parameter_name =~ s/\[.*//;
-
-	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
-	$type = $args{'parametertypes'}{$parameter};
-	if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
-	    # pointer-to-function
-	    print "    $1 $parameter) ($2);\n";
-	} elsif ($type =~ m/^(.*?)\s*(:.*)/) {
-	    # bitfield
-	    print "    $1 $parameter$2;\n";
-	} else {
-	    print "    " . $type . " " . $parameter . ";\n";
-	}
-    }
-    print "  };\n\n";
+    my $declaration = $args{'definition'};
+    $declaration =~ s/\t/  /g;
+    print "  " . $args{'type'} . " " . $args{'struct'} . " {\n$declaration  };\n\n";
 
     print "**Members**\n\n";
     $lineprefix = "  ";
@@ -1022,20 +982,15 @@ sub dump_struct($$) {
     my $nested;
 
     if ($x =~ /(struct|union)\s+(\w+)\s*{(.*)}/) {
-	#my $decl_type = $1;
+	my $decl_type = $1;
 	$declaration_name = $2;
 	my $members = $3;
 
-	# ignore embedded structs or unions
-	$members =~ s/({.*})//g;
-	$nested = $1;
-
 	# ignore members marked private:
 	$members =~ s/\/\*\s*private:.*?\/\*\s*public:.*?\*\///gosi;
 	$members =~ s/\/\*\s*private:.*//gosi;
 	# strip comments:
 	$members =~ s/\/\*.*?\*\///gos;
-	$nested =~ s/\/\*.*?\*\///gos;
 	# strip kmemcheck_bitfield_{begin,end}.*;
 	$members =~ s/kmemcheck_bitfield_.*?;//gos;
 	# strip attributes
@@ -1047,13 +1002,73 @@ sub dump_struct($$) {
 	# replace DECLARE_HASHTABLE
 	$members =~ s/DECLARE_HASHTABLE\s*\(([^,)]+), ([^,)]+)\)/unsigned long $1\[1 << (($2) - 1)\]/gos;
 
+	my $declaration = $members;
+
+	# Split nested struct/union elements as newer ones
+	my $cont = 1;
+	while ($cont) {
+		$cont = 0;
+		while ($members =~ m/(struct|union)([^{};]+){([^{}]*)}([^{}\;]*)\;/) {
+			my $newmember = "$1 $4;";
+			my $id = $4;
+			my $content = $3;
+			$id =~ s/[:\[].*//;
+			$id =~ s/^\*+//;
+			foreach my $arg (split /;/, $content) {
+				next if ($arg =~ m/^\s*$/);
+				my $type = $arg;
+				my $name = $arg;
+				$type =~ s/\s\S+$//;
+				$name =~ s/.*\s//;
+				$name =~ s/[:\[].*//;
+				$name =~ s/^\*+//;
+				next if (($name =~ m/^\s*$/));
+				if ($id =~ m/^\s*$/) {
+					# anonymous struct/union
+					$newmember .= "$type $name;";
+				} else {
+					$newmember .= "$type $id.$name;";
+				}
+			}
+			$members =~ s/(struct|union)([^{};]+){([^{}]*)}([^{}\;]*)\;/$newmember/;
+			$cont = 1;
+		};
+	};
+
+	# Ignore other nested elements, like enums
+	$members =~ s/({[^\{\}]*})//g;
+	$nested = $decl_type;
+	$nested =~ s/\/\*.*?\*\///gos;
+
 	create_parameterlist($members, ';', $file);
 	check_sections($file, $declaration_name, "struct", $sectcheck, $struct_actual, $nested);
 
+	# Adjust declaration for better display
+	$declaration =~ s/([{;])/$1\n/g;
+	$declaration =~ s/}\s+;/};/g;
+	# Better handle inlined enums
+	do {} while ($declaration =~ s/(enum\s+{[^}]+),([^\n])/$1,\n$2/);
+
+	my @def_args = split /\n/, $declaration;
+	my $level = 1;
+	$declaration = "";
+	foreach my $clause (@def_args) {
+		$clause =~ s/^\s+//;
+		$clause =~ s/\s+$//;
+		$clause =~ s/\s+/ /;
+		next if (!$clause);
+		$level-- if ($clause =~ m/(})/ && $level > 1);
+		if (!($clause =~ m/^\s*#/)) {
+			$declaration .= "\t" x $level;
+		}
+		$declaration .= "\t" . $clause . "\n";
+		$level++ if ($clause =~ m/({)/ && !($clause =~m/}/));
+	}
 	output_declaration($declaration_name,
 			   'struct',
 			   {'struct' => $declaration_name,
 			    'module' => $modulename,
+			    'definition' => $declaration,
 			    'parameterlist' => \@parameterlist,
 			    'parameterdescs' => \%parameterdescs,
 			    'parametertypes' => \%parametertypes,
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v2 10/13] scripts: kernel-doc: get rid of $nested parameter
  2017-09-27 21:10 [PATCH v2 00/13] kernel-doc: add supported to document nested structs/$ Mauro Carvalho Chehab
                   ` (8 preceding siblings ...)
  2017-09-27 21:10 ` [PATCH v2 09/13] scripts: kernel-doc: parse next structs/unions Mauro Carvalho Chehab
@ 2017-09-27 21:10 ` Mauro Carvalho Chehab
  2017-09-27 21:10 ` [PATCH v2 11/13] scripts: kernel-doc: print the declaration name on warnings Mauro Carvalho Chehab
                   ` (2 subsequent siblings)
  12 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-27 21:10 UTC (permalink / raw)
  To: Linux Media Mailing List, Jonathan Corbet
  Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, Linux Doc Mailing List

The check_sections() function has a $nested parameter, meant
to identify when a nested struct is present. As we now have
a logic that handles it, get rid of such parameter.

Suggested-by: Markus Heiser <markus.heiser@darmarit.de>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 scripts/kernel-doc | 19 ++++---------------
 1 file changed, 4 insertions(+), 15 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 63aa9f85d635..28c5fe63fb58 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -979,7 +979,6 @@ sub dump_union($$) {
 sub dump_struct($$) {
     my $x = shift;
     my $file = shift;
-    my $nested;
 
     if ($x =~ /(struct|union)\s+(\w+)\s*{(.*)}/) {
 	my $decl_type = $1;
@@ -1037,11 +1036,9 @@ sub dump_struct($$) {
 
 	# Ignore other nested elements, like enums
 	$members =~ s/({[^\{\}]*})//g;
-	$nested = $decl_type;
-	$nested =~ s/\/\*.*?\*\///gos;
 
 	create_parameterlist($members, ';', $file);
-	check_sections($file, $declaration_name, "struct", $sectcheck, $struct_actual, $nested);
+	check_sections($file, $declaration_name, "struct", $sectcheck, $struct_actual);
 
 	# Adjust declaration for better display
 	$declaration =~ s/([{;])/$1\n/g;
@@ -1337,8 +1334,8 @@ sub push_parameter($$$) {
 	$parametertypes{$param} = $type;
 }
 
-sub check_sections($$$$$$) {
-	my ($file, $decl_name, $decl_type, $sectcheck, $prmscheck, $nested) = @_;
+sub check_sections($$$$$) {
+	my ($file, $decl_name, $decl_type, $sectcheck, $prmscheck) = @_;
 	my @sects = split ' ', $sectcheck;
 	my @prms = split ' ', $prmscheck;
 	my $err;
@@ -1372,14 +1369,6 @@ sub check_sections($$$$$$) {
 					"'$sects[$sx]' " .
 					"description in '$decl_name'\n";
 				++$warnings;
-			} else {
-				if ($nested !~ m/\Q$sects[$sx]\E/) {
-				    print STDERR "${file}:$.: warning: " .
-					"Excess struct/union/enum/typedef member " .
-					"'$sects[$sx]' " .
-					"description in '$decl_name'\n";
-				    ++$warnings;
-				}
 			}
 		}
 	}
@@ -1490,7 +1479,7 @@ sub dump_function($$) {
     }
 
 	my $prms = join " ", @parameterlist;
-	check_sections($file, $declaration_name, "function", $sectcheck, $prms, "");
+	check_sections($file, $declaration_name, "function", $sectcheck, $prms);
 
         # This check emits a lot of warnings at the moment, because many
         # functions don't have a 'Return' doc section. So until the number
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v2 11/13] scripts: kernel-doc: print the declaration name on warnings
  2017-09-27 21:10 [PATCH v2 00/13] kernel-doc: add supported to document nested structs/$ Mauro Carvalho Chehab
                   ` (9 preceding siblings ...)
  2017-09-27 21:10 ` [PATCH v2 10/13] scripts: kernel-doc: get rid of $nested parameter Mauro Carvalho Chehab
@ 2017-09-27 21:10 ` Mauro Carvalho Chehab
  2017-09-27 21:10 ` [PATCH v2 12/13] scripts: kernel-doc: handle nested struct function arguments Mauro Carvalho Chehab
  2017-09-27 21:10 ` [PATCH v2 13/13] [RFC] w1_netlink.h: add support for nested structs Mauro Carvalho Chehab
  12 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-27 21:10 UTC (permalink / raw)
  To: Linux Media Mailing List, Jonathan Corbet
  Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, Linux Doc Mailing List

The logic at create_parameterlist()'s ancillary push_parameter()
function has already a way to output the declaration name, with
would help to discover what declaration is missing.

However, currently, the logic is utterly broken, as it uses
the var $type with a wrong meaning. With the current code,
it will never print anything. I suspect that originally
it was using the second argument of output_declaration().

I opted to not rely on a globally defined $declaration_name,
but, instead, to pass it explicitly as a parameter.

While here, I removed a unaligned check for !$anon_struct_union.
This is not needed, as, if $anon_struct_union is not zero,
$parameterdescs{$param} will be defined.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 scripts/kernel-doc | 38 ++++++++++++++++----------------------
 1 file changed, 16 insertions(+), 22 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 28c5fe63fb58..713608046d3a 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1037,7 +1037,7 @@ sub dump_struct($$) {
 	# Ignore other nested elements, like enums
 	$members =~ s/({[^\{\}]*})//g;
 
-	create_parameterlist($members, ';', $file);
+	create_parameterlist($members, ';', $file, $declaration_name);
 	check_sections($file, $declaration_name, "struct", $sectcheck, $struct_actual);
 
 	# Adjust declaration for better display
@@ -1137,7 +1137,7 @@ sub dump_typedef($$) {
 	$declaration_name = $2;
 	my $args = $3;
 
-	create_parameterlist($args, ',', $file);
+	create_parameterlist($args, ',', $file, $declaration_name);
 
 	output_declaration($declaration_name,
 			   'function',
@@ -1186,10 +1186,11 @@ sub save_struct_actual($) {
     $struct_actual = $struct_actual . $actual . " ";
 }
 
-sub create_parameterlist($$$) {
+sub create_parameterlist($$$$) {
     my $args = shift;
     my $splitter = shift;
     my $file = shift;
+    my $declaration_name = shift;
     my $type;
     my $param;
 
@@ -1219,7 +1220,7 @@ sub create_parameterlist($$$) {
 	    $type = $arg;
 	    $type =~ s/([^\(]+\(\*?)\s*$param/$1/;
 	    save_struct_actual($param);
-	    push_parameter($param, $type, $file);
+	    push_parameter($param, $type, $file, $declaration_name);
 	} elsif ($arg) {
 	    $arg =~ s/\s*:\s*/:/g;
 	    $arg =~ s/\s*\[/\[/g;
@@ -1244,27 +1245,28 @@ sub create_parameterlist($$$) {
 	    foreach $param (@args) {
 		if ($param =~ m/^(\*+)\s*(.*)/) {
 		    save_struct_actual($2);
-		    push_parameter($2, "$type $1", $file);
+		    push_parameter($2, "$type $1", $file, $declaration_name);
 		}
 		elsif ($param =~ m/(.*?):(\d+)/) {
 		    if ($type ne "") { # skip unnamed bit-fields
 			save_struct_actual($1);
-			push_parameter($1, "$type:$2", $file)
+			push_parameter($1, "$type:$2", $file, $declaration_name)
 		    }
 		}
 		else {
 		    save_struct_actual($param);
-		    push_parameter($param, $type, $file);
+		    push_parameter($param, $type, $file, $declaration_name);
 		}
 	    }
 	}
     }
 }
 
-sub push_parameter($$$) {
+sub push_parameter($$$$) {
 	my $param = shift;
 	my $type = shift;
 	my $file = shift;
+	my $declaration_name = shift;
 
 	if (($anon_struct_union == 1) && ($type eq "") &&
 	    ($param eq "}")) {
@@ -1301,21 +1303,13 @@ sub push_parameter($$$) {
 	# warn if parameter has no description
 	# (but ignore ones starting with # as these are not parameters
 	# but inline preprocessor statements);
-	# also ignore unnamed structs/unions;
-	if (!$anon_struct_union) {
+	# Note: It will also ignore void params and unnamed structs/unions
 	if (!defined $parameterdescs{$param} && $param !~ /^#/) {
+		$parameterdescs{$param} = $undescribed;
 
-	    $parameterdescs{$param} = $undescribed;
-
-	    if (($type eq 'function') || ($type eq 'enum')) {
-		print STDERR "${file}:$.: warning: Function parameter ".
-		    "or member '$param' not " .
-		    "described in '$declaration_name'\n";
-	    }
-	    print STDERR "${file}:$.: warning:" .
-			 " No description found for parameter '$param'\n";
-	    ++$warnings;
-	}
+		print STDERR
+		      "${file}:$.: warning: Function parameter or member '$param' not described in '$declaration_name'\n";
+		++$warnings;
 	}
 
 	$param = xml_escape($param);
@@ -1472,7 +1466,7 @@ sub dump_function($$) {
 	$declaration_name = $2;
 	my $args = $3;
 
-	create_parameterlist($args, ',', $file);
+	create_parameterlist($args, ',', $file, $declaration_name);
     } else {
 	print STDERR "${file}:$.: warning: cannot understand function prototype: '$prototype'\n";
 	return;
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v2 12/13] scripts: kernel-doc: handle nested struct function arguments
  2017-09-27 21:10 [PATCH v2 00/13] kernel-doc: add supported to document nested structs/$ Mauro Carvalho Chehab
                   ` (10 preceding siblings ...)
  2017-09-27 21:10 ` [PATCH v2 11/13] scripts: kernel-doc: print the declaration name on warnings Mauro Carvalho Chehab
@ 2017-09-27 21:10 ` Mauro Carvalho Chehab
  2017-09-28 16:32   ` Markus Heiser
  2017-09-27 21:10 ` [PATCH v2 13/13] [RFC] w1_netlink.h: add support for nested structs Mauro Carvalho Chehab
  12 siblings, 1 reply; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-27 21:10 UTC (permalink / raw)
  To: Linux Media Mailing List, Jonathan Corbet
  Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab, Linux Doc Mailing List

Function arguments are different than usual ones. So, an
special logic is needed in order to handle such arguments
on nested structs.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 scripts/kernel-doc | 38 ++++++++++++++++++++++++++------------
 1 file changed, 26 insertions(+), 12 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 713608046d3a..376365d41718 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1015,18 +1015,32 @@ sub dump_struct($$) {
 			$id =~ s/^\*+//;
 			foreach my $arg (split /;/, $content) {
 				next if ($arg =~ m/^\s*$/);
-				my $type = $arg;
-				my $name = $arg;
-				$type =~ s/\s\S+$//;
-				$name =~ s/.*\s//;
-				$name =~ s/[:\[].*//;
-				$name =~ s/^\*+//;
-				next if (($name =~ m/^\s*$/));
-				if ($id =~ m/^\s*$/) {
-					# anonymous struct/union
-					$newmember .= "$type $name;";
+				if ($arg =~ m/^([^\(]+\(\*?\s*)([\w\.]*)(\s*\).*)/) {
+					# pointer-to-function
+					my $type = $1;
+					my $name = $2;
+					my $extra = $3;
+					next if (!$name);
+					if ($id =~ m/^\s*$/) {
+						# anonymous struct/union
+						$newmember .= "$type$name$extra;";
+					} else {
+						$newmember .= "$type$id.$name$extra;";
+					}
 				} else {
-					$newmember .= "$type $id.$name;";
+					my $type = $arg;
+					my $name = $arg;
+					$type =~ s/\s\S+$//;
+					$name =~ s/.*\s+//;
+					$name =~ s/[:\[].*//;
+					$name =~ s/^\*+//;
+					next if (($name =~ m/^\s*$/));
+					if ($id =~ m/^\s*$/) {
+						# anonymous struct/union
+						$newmember .= "$type $name;";
+					} else {
+						$newmember .= "$type $id.$name;";
+					}
 				}
 			}
 			$members =~ s/(struct|union)([^{};]+){([^{}]*)}([^{}\;]*)\;/$newmember/;
@@ -1215,7 +1229,7 @@ sub create_parameterlist($$$$) {
 	} elsif ($arg =~ m/\(.+\)\s*\(/) {
 	    # pointer-to-function
 	    $arg =~ tr/#/,/;
-	    $arg =~ m/[^\(]+\(\*?\s*(\w*)\s*\)/;
+	    $arg =~ m/[^\(]+\(\*?\s*([\w\.]*)\s*\)/;
 	    $param = $1;
 	    $type = $arg;
 	    $type =~ s/([^\(]+\(\*?)\s*$param/$1/;
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v2 13/13] [RFC] w1_netlink.h: add support for nested structs
  2017-09-27 21:10 [PATCH v2 00/13] kernel-doc: add supported to document nested structs/$ Mauro Carvalho Chehab
                   ` (11 preceding siblings ...)
  2017-09-27 21:10 ` [PATCH v2 12/13] scripts: kernel-doc: handle nested struct function arguments Mauro Carvalho Chehab
@ 2017-09-27 21:10 ` Mauro Carvalho Chehab
  12 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-27 21:10 UTC (permalink / raw)
  To: Linux Media Mailing List, Jonathan Corbet
  Cc: Mauro Carvalho Chehab, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Evgeniy Polyakov

Describe nested struct/union fields

NOTE: This is a pure test patch, meant to validate if the
parsing logic for nested structs is working properly.

I've no idea if the random text I added there is correct!

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
---
 drivers/w1/w1_netlink.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/w1/w1_netlink.h b/drivers/w1/w1_netlink.h
index a36661cd1f05..e781d1109cd7 100644
--- a/drivers/w1/w1_netlink.h
+++ b/drivers/w1/w1_netlink.h
@@ -60,6 +60,10 @@ enum w1_netlink_message_types {
  * @status: kernel feedback for success 0 or errno failure value
  * @len: length of data following w1_netlink_msg
  * @id: union holding master bus id (msg.id) and slave device id (id[8]).
+ * @id.id: Slave ID (8 bytes)
+ * @id.mst: master bus identification
+ * @id.mst.id: master bus ID
+ * @id.mst.res: master bus reserved
  * @data: start address of any following data
  *
  * The base message structure for w1 messages over netlink.
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* Re: [PATCH v2 07/13] docs: kernel-doc.rst: add documentation about man pages
  2017-09-27 21:10 ` [PATCH v2 07/13] docs: kernel-doc.rst: add documentation about man pages Mauro Carvalho Chehab
@ 2017-09-27 21:20   ` Randy Dunlap
  2017-09-28  1:12     ` Mauro Carvalho Chehab
  0 siblings, 1 reply; 28+ messages in thread
From: Randy Dunlap @ 2017-09-27 21:20 UTC (permalink / raw)
  To: Mauro Carvalho Chehab, Linux Media Mailing List, Jonathan Corbet
  Cc: Mauro Carvalho Chehab, Linux Doc Mailing List, Daniel Vetter

On 09/27/17 14:10, Mauro Carvalho Chehab wrote:
> kernel-doc-nano-HOWTO.txt has a chapter about man pages

  kernel-doc.rst has a chapter (or section)

> production. While we don't have a working  "make manpages"
> target, add it.
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> ---
>  Documentation/doc-guide/kernel-doc.rst | 34 ++++++++++++++++++++++++++++++++++
>  1 file changed, 34 insertions(+)
> 
> diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst
> index 0923c8bd5769..96012f9e314d 100644
> --- a/Documentation/doc-guide/kernel-doc.rst
> +++ b/Documentation/doc-guide/kernel-doc.rst


-- 
~Randy

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v2 07/13] docs: kernel-doc.rst: add documentation about man pages
  2017-09-27 21:20   ` Randy Dunlap
@ 2017-09-28  1:12     ` Mauro Carvalho Chehab
  0 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-28  1:12 UTC (permalink / raw)
  To: Randy Dunlap
  Cc: Linux Media Mailing List, Jonathan Corbet, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter

Em Wed, 27 Sep 2017 14:20:03 -0700
Randy Dunlap <rdunlap@infradead.org> escreveu:

> On 09/27/17 14:10, Mauro Carvalho Chehab wrote:
> > kernel-doc-nano-HOWTO.txt has a chapter about man pages  
> 
>   kernel-doc.rst has a chapter (or section)

I actually meant to say that kernel-doc-nano-HOWTO.txt has a chapter
about man pages, but such chapter is missing at kernel-doc.rst.

I'll make it clearer at the patch's description.

Thanks!
Mauro

> 
> > production. While we don't have a working  "make manpages"
> > target, add it.
> > 
> > Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> > ---
> >  Documentation/doc-guide/kernel-doc.rst | 34 ++++++++++++++++++++++++++++++++++
> >  1 file changed, 34 insertions(+)
> > 
> > diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst
> > index 0923c8bd5769..96012f9e314d 100644
> > --- a/Documentation/doc-guide/kernel-doc.rst
> > +++ b/Documentation/doc-guide/kernel-doc.rst  
> 
> 



Thanks,
Mauro

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v2 09/13] scripts: kernel-doc: parse next structs/unions
  2017-09-27 21:10 ` [PATCH v2 09/13] scripts: kernel-doc: parse next structs/unions Mauro Carvalho Chehab
@ 2017-09-28 16:28   ` Markus Heiser
  2017-09-29 12:08     ` Mauro Carvalho Chehab
                       ` (2 more replies)
  0 siblings, 3 replies; 28+ messages in thread
From: Markus Heiser @ 2017-09-28 16:28 UTC (permalink / raw)
  To: Mauro Carvalho Chehab
  Cc: Linux Media Mailing List, Jonathan Corbet, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter

Hi Mauro,

> Am 27.09.2017 um 23:10 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> 
> 
> 
> diff --git a/Documentation/doc-guide/kernel-doc.rst b/Documentation/doc-guide/kernel-doc.rst
> index 96012f9e314d..9e63a18cceea 100644
> --- a/Documentation/doc-guide/kernel-doc.rst
> +++ b/Documentation/doc-guide/kernel-doc.rst
> @@ -281,6 +281,52 @@ comment block.
> The kernel-doc data structure comments describe each member of the structure,
> in order, with the member descriptions.
> 
> +Nested structs/unions
> +~~~~~~~~~~~~~~~~~~~~~
> +
> +It is possible to document nested structs unions, like::
> +
> +      /**
> +       * struct nested_foobar - a struct with nested unions and structs
> +       * @arg1: - first argument of anonymous union/anonymous struct
> +       * @arg2: - second argument of anonymous union/anonymous struct
> +       * @arg3: - third argument of anonymous union/anonymous struct
> +       * @arg4: - fourth argument of anonymous union/anonymous struct
> +       * @bar.st1.arg1 - first argument of struct st1 on union bar
> +       * @bar.st1.arg2 - second argument of struct st1 on union bar
> +       * @bar.st2.arg1 - first argument of struct st2 on union bar
> +       * @bar.st2.arg2 - second argument of struct st2 on union bar

Sorry, this example is totally broken --> below I attached a more
elaborate example. I also untabified the example since tabs in reST are
a nightmare, especially in code blocks ... tabulators are the source
of all evil [1] ...

  Please, never use tabs in markups or programming languages
  where indentation is a part of the markup respectively the 
  language!!

> diff --git a/scripts/kernel-doc b/scripts/kernel-doc
> index b6f3f6962897..63aa9f85d635 100755
> --- a/scripts/kernel-doc
> +++ b/scripts/kernel-doc
> @@ -210,7 +210,7 @@ my $anon_struct_union = 0;
> my $type_constant = '\b``([^\`]+)``\b';
> my $type_constant2 = '\%([-_\w]+)';
> my $type_func = '(\w+)\(\)';
> -my $type_param = '\@(\w+(\.\.\.)?)';
> +my $type_param = '\@(\w*(\.\w+)*(\.\.\.)?)';
> my $type_fp_param = '\@(\w+)\(\)';  # Special RST handling for func ptr params
> my $type_env = '(\$\w+)';
> my $type_enum = '\&(enum\s*([_\w]+))';
> @@ -663,32 +663,12 @@ sub output_struct_man(%) {
>    print ".SH NAME\n";
>    print $args{'type'} . " " . $args{'struct'} . " \\- " . $args{'purpose'} . "\n";
> 
> +    my $declaration = $args{'definition'};
> +    $declaration =~ s/\t/  /g;
> +    $declaration =~ s/\n/"\n.br\n.BI \"/g;
>    print ".SH SYNOPSIS\n";
>    print $args{'type'} . " " . $args{'struct'} . " {\n.br\n";
> -
> -    foreach my $parameter (@{$args{'parameterlist'}}) {
> -	if ($parameter =~ /^#/) {
> -	    print ".BI \"$parameter\"\n.br\n";
> -	    next;
> -	}
> -	my $parameter_name = $parameter;
> -	$parameter_name =~ s/\[.*//;
> -
> -	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
> -	$type = $args{'parametertypes'}{$parameter};
> -	if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
> -	    # pointer-to-function
> -	    print ".BI \"    " . $1 . "\" " . $parameter . " \") (" . $2 . ")" . "\"\n;\n";
> -	} elsif ($type =~ m/^(.*?)\s*(:.*)/) {
> -	    # bitfield
> -	    print ".BI \"    " . $1 . "\ \" " . $parameter . $2 . " \"" . "\"\n;\n";
> -	} else {
> -	    $type =~ s/([^\*])$/$1 /;
> -	    print ".BI \"    " . $type . "\" " . $parameter . " \"" . "\"\n;\n";
> -	}
> -	print "\n.br\n";
> -    }
> -    print "};\n.br\n";
> +    print ".BI \"$declaration\n};\n.br\n\n";
> 
>    print ".SH Members\n";
>    foreach $parameter (@{$args{'parameterlist'}}) {
> @@ -926,29 +906,9 @@ sub output_struct_rst(%) {
> 
>    print "**Definition**\n\n";
>    print "::\n\n";
> -    print "  " . $args{'type'} . " " . $args{'struct'} . " {\n";
> -    foreach $parameter (@{$args{'parameterlist'}}) {
> -	if ($parameter =~ /^#/) {
> -	    print "  " . "$parameter\n";
> -	    next;
> -	}
> -
> -	my $parameter_name = $parameter;
> -	$parameter_name =~ s/\[.*//;
> -
> -	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
> -	$type = $args{'parametertypes'}{$parameter};
> -	if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
> -	    # pointer-to-function
> -	    print "    $1 $parameter) ($2);\n";
> -	} elsif ($type =~ m/^(.*?)\s*(:.*)/) {
> -	    # bitfield
> -	    print "    $1 $parameter$2;\n";
> -	} else {
> -	    print "    " . $type . " " . $parameter . ";\n";
> -	}
> -    }
> -    print "  };\n\n";
> +    my $declaration = $args{'definition'};
> +    $declaration =~ s/\t/  /g;
> +    print "  " . $args{'type'} . " " . $args{'struct'} . " {\n$declaration  };\n\n";
> 
>    print "**Members**\n\n";
>    $lineprefix = "  ";
> @@ -1022,20 +982,15 @@ sub dump_struct($$) {
>    my $nested;
> 
>    if ($x =~ /(struct|union)\s+(\w+)\s*{(.*)}/) {
> -	#my $decl_type = $1;
> +	my $decl_type = $1;
> 	$declaration_name = $2;
> 	my $members = $3;
> 
> -	# ignore embedded structs or unions
> -	$members =~ s/({.*})//g;
> -	$nested = $1;
> -
> 	# ignore members marked private:
> 	$members =~ s/\/\*\s*private:.*?\/\*\s*public:.*?\*\///gosi;
> 	$members =~ s/\/\*\s*private:.*//gosi;
> 	# strip comments:
> 	$members =~ s/\/\*.*?\*\///gos;
> -	$nested =~ s/\/\*.*?\*\///gos;
> 	# strip kmemcheck_bitfield_{begin,end}.*;
> 	$members =~ s/kmemcheck_bitfield_.*?;//gos;
> 	# strip attributes
> @@ -1047,13 +1002,73 @@ sub dump_struct($$) {
> 	# replace DECLARE_HASHTABLE
> 	$members =~ s/DECLARE_HASHTABLE\s*\(([^,)]+), ([^,)]+)\)/unsigned long $1\[1 << (($2) - 1)\]/gos;
> 
> +	my $declaration = $members;
> +
> +	# Split nested struct/union elements as newer ones
> +	my $cont = 1;
> +	while ($cont) {
> +		$cont = 0;


You ignored the remarks I made to v1: 

 this outer loop is not needed!


> +		while ($members =~ m/(struct|union)([^{};]+){([^{}]*)}([^{}\;]*)\;/) {


The inner loop is enough.

> +			my $newmember = "$1 $4;";
> +			my $id = $4;


> Am 26.09.2017 um 21:16 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> 
>> This won't work if  you have e.g.
>> 
>>  union car {int foo;} bar1, bar2, *bbar3;
>> 
>> where $id will be "bar1, bar2, *bbar3", you need to split
>> this string and iterate over the id's.
> 
> That's true, if we have cases like that. I hope not!

We have! I can't remember where, but I hit such constructs while
parsing the whole sources.

> 
> Seriously, in the above case, the developer should very likely declare
> the union outside the main struct and have its own kernel-doc markup.
> 
> My personal taste would be to not explicitly support the above. If it
> hits some code, try to argue with the developer first, before patching
> kernel-doc to such weird stuff.

Why is it wired? Anyway we have it and this kernel-doc fails with.

It also fails with bit types e.g.

       struct {
	    __u8 arg1 : 1;
	    __u8 arg2 : 3;
       };


May it help, if you inspect the py-version of this loop:

   https://github.com/return42/linuxdoc/blob/master/linuxdoc/kernel_doc.py#L2499


> +			my $content = $3;
> +			$id =~ s/[:\[].*//;
> +			$id =~ s/^\*+//;
> +			foreach my $arg (split /;/, $content) {
> +				next if ($arg =~ m/^\s*$/);
> +				my $type = $arg;
> +				my $name = $arg;
> +				$type =~ s/\s\S+$//;
> +				$name =~ s/.*\s//;
> +				$name =~ s/[:\[].*//;
> +				$name =~ s/^\*+//;
> +				next if (($name =~ m/^\s*$/));
> +				if ($id =~ m/^\s*$/) {
> +					# anonymous struct/union
> +					$newmember .= "$type $name;";
> +				} else {
> +					$newmember .= "$type $id.$name;";
> +				}
> +			}
> +			$members =~ s/(struct|union)([^{};]+){([^{}]*)}([^{}\;]*)\;/$newmember/;
> +			$cont = 1;
> +		};
> +	};
> +
> +	# Ignore other nested elements, like enums
> +	$members =~ s/({[^\{\}]*})//g;
> +	$nested = $decl_type;
> +	$nested =~ s/\/\*.*?\*\///gos;
> +
> 	create_parameterlist($members, ';', $file);
> 	check_sections($file, $declaration_name, "struct", $sectcheck, $struct_actual, $nested);
> 
> +	# Adjust declaration for better display
> +	$declaration =~ s/([{;])/$1\n/g;
> +	$declaration =~ s/}\s+;/};/g;
> +	# Better handle inlined enums
> +	do {} while ($declaration =~ s/(enum\s+{[^}]+),([^\n])/$1,\n$2/);
> +
> +	my @def_args = split /\n/, $declaration;
> +	my $level = 1;
> +	$declaration = "";
> +	foreach my $clause (@def_args) {
> +		$clause =~ s/^\s+//;
> +		$clause =~ s/\s+$//;
> +		$clause =~ s/\s+/ /;
> +		next if (!$clause);
> +		$level-- if ($clause =~ m/(})/ && $level > 1);
> +		if (!($clause =~ m/^\s*#/)) {
> +			$declaration .= "\t" x $level;
> +		}
> +		$declaration .= "\t" . $clause . "\n";
> +		$level++ if ($clause =~ m/({)/ && !($clause =~m/}/));
> +	}
> 	output_declaration($declaration_name,
> 			   'struct',
> 			   {'struct' => $declaration_name,
> 			    'module' => $modulename,
> +			    'definition' => $declaration,
> 			    'parameterlist' => \@parameterlist,
> 			    'parameterdescs' => \%parameterdescs,
> 			    'parametertypes' => \%parametertypes,
> -- 
> 2.13.5


-- Markus --


[1] https://www.emacswiki.org/emacs/TabsAreEvil


---

/* parse-SNIP: my_struct */
/**
* struct my_struct - a struct with nested unions and structs
* @arg1: first argument of anonymous union/anonymous struct
* @arg2: second argument of anonymous union/anonymous struct
* @arg3: third argument of anonymous union/anonymous struct
* @arg4: fourth argument of anonymous union/anonymous struct
* @bar.st1.arg1: first argument of struct st1 on union bar
* @bar.st1.arg2: second argument of struct st1 on union bar
* @bar.st2.arg1: first argument of struct st2 on union bar
* @bar.st2.arg2: second argument of struct st2 on union bar
* @bar.st3.arg2: second argument of struct st3 on union bar
*/
struct my_struct {
   /* Anonymous union/struct*/
   union {
	struct {
	    __u8 arg1 : 1;
	    __u8 arg2 : 3;
	};
       struct {
           int arg1;
           int arg2;
       };
       struct {
           void *arg3;
           int arg4;
       };
   };
   union {
       struct {
           int arg1;
           int arg2;
       } st1;           /* bar.st1 is undocumented, cause a warning */
       struct {
           void *arg1;  /* bar.st3.arg1 is undocumented, cause a warning */
	    int arg2;
       } st2, st3;
   } bar;               /* bar is undocumented, cause a warning */

   /* private: */
   int undoc_privat;    /* is undocumented but private, no warning */

   /* public: */
   int undoc_public;    /* is undocumented, cause a warning */

};

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v2 12/13] scripts: kernel-doc: handle nested struct function arguments
  2017-09-27 21:10 ` [PATCH v2 12/13] scripts: kernel-doc: handle nested struct function arguments Mauro Carvalho Chehab
@ 2017-09-28 16:32   ` Markus Heiser
  2017-10-01 11:18     ` Mauro Carvalho Chehab
  0 siblings, 1 reply; 28+ messages in thread
From: Markus Heiser @ 2017-09-28 16:32 UTC (permalink / raw)
  To: Mauro Carvalho Chehab
  Cc: Linux Media Mailing List, Jonathan Corbet, Mauro Carvalho Chehab,
	Linux Doc Mailing List

Hi Mauro,

this 'else' addition seems a bit spooky to me. As I commented in patch 09/13
may it helps when you look at 

  https://github.com/return42/linuxdoc/blob/master/linuxdoc/kernel_doc.py#L2499

which is IMO a bit more clear.

-- Markus --

> Am 27.09.2017 um 23:10 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> 
> Function arguments are different than usual ones. So, an
> special logic is needed in order to handle such arguments
> on nested structs.
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> ---
> scripts/kernel-doc | 38 ++++++++++++++++++++++++++------------
> 1 file changed, 26 insertions(+), 12 deletions(-)
> 
> diff --git a/scripts/kernel-doc b/scripts/kernel-doc
> index 713608046d3a..376365d41718 100755
> --- a/scripts/kernel-doc
> +++ b/scripts/kernel-doc
> @@ -1015,18 +1015,32 @@ sub dump_struct($$) {
> 			$id =~ s/^\*+//;
> 			foreach my $arg (split /;/, $content) {
> 				next if ($arg =~ m/^\s*$/);
> -				my $type = $arg;
> -				my $name = $arg;
> -				$type =~ s/\s\S+$//;
> -				$name =~ s/.*\s//;
> -				$name =~ s/[:\[].*//;
> -				$name =~ s/^\*+//;
> -				next if (($name =~ m/^\s*$/));
> -				if ($id =~ m/^\s*$/) {
> -					# anonymous struct/union
> -					$newmember .= "$type $name;";
> +				if ($arg =~ m/^([^\(]+\(\*?\s*)([\w\.]*)(\s*\).*)/) {
> +					# pointer-to-function
> +					my $type = $1;
> +					my $name = $2;
> +					my $extra = $3;
> +					next if (!$name);
> +					if ($id =~ m/^\s*$/) {
> +						# anonymous struct/union
> +						$newmember .= "$type$name$extra;";
> +					} else {
> +						$newmember .= "$type$id.$name$extra;";
> +					}
> 				} else {
> -					$newmember .= "$type $id.$name;";
> +					my $type = $arg;
> +					my $name = $arg;
> +					$type =~ s/\s\S+$//;
> +					$name =~ s/.*\s+//;
> +					$name =~ s/[:\[].*//;
> +					$name =~ s/^\*+//;
> +					next if (($name =~ m/^\s*$/));
> +					if ($id =~ m/^\s*$/) {
> +						# anonymous struct/union
> +						$newmember .= "$type $name;";
> +					} else {
> +						$newmember .= "$type $id.$name;";
> +					}
> 				}
> 			}
> 			$members =~ s/(struct|union)([^{};]+){([^{}]*)}([^{}\;]*)\;/$newmember/;
> @@ -1215,7 +1229,7 @@ sub create_parameterlist($$$$) {
> 	} elsif ($arg =~ m/\(.+\)\s*\(/) {
> 	    # pointer-to-function
> 	    $arg =~ tr/#/,/;
> -	    $arg =~ m/[^\(]+\(\*?\s*(\w*)\s*\)/;
> +	    $arg =~ m/[^\(]+\(\*?\s*([\w\.]*)\s*\)/;
> 	    $param = $1;
> 	    $type = $arg;
> 	    $type =~ s/([^\(]+\(\*?)\s*$param/$1/;
> -- 
> 2.13.5
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-doc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v2 09/13] scripts: kernel-doc: parse next structs/unions
  2017-09-28 16:28   ` Markus Heiser
@ 2017-09-29 12:08     ` Mauro Carvalho Chehab
  2017-09-29 13:07       ` Markus Heiser
  2017-09-29 15:29     ` Mauro Carvalho Chehab
  2017-10-01 11:02     ` Mauro Carvalho Chehab
  2 siblings, 1 reply; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-29 12:08 UTC (permalink / raw)
  To: Markus Heiser
  Cc: Linux Media Mailing List, Jonathan Corbet, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter

Em Thu, 28 Sep 2017 18:28:32 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:

> Hi Mauro,
> 
> > Am 27.09.2017 um 23:10 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> > 
> I also untabified the example since tabs in reST are
> a nightmare, especially in code blocks ... tabulators are the source
> of all evil [1] ...
> 
>   Please, never use tabs in markups or programming languages
>   where indentation is a part of the markup respectively the 
>   language!!

Tabs will exist at the sources, as Kernel coding style recommends its
usage. There's nothing that can be done to avoid. So, whatever scripts
we use, it should handle it.

Thankfully, solving this issue is a one line perl patch, as explained at:
	http://perldoc.perl.org/perlfaq4.html#How-do-I-expand-tabs-in-a-string?

Something like the enclosed (untested) patch.

Thanks,
Mauro

[PATCH] kernel-doc: replace tabs by spaces

Sphinx has a hard time dealing with tabs, causing it to
misinterpret paragraph continuation.

As we're now mainly focused on supporting ReST output,
replace tabs by spaces, in order to avoid troubles when
the output is parsed by Sphinx.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 69757ee9db4c..7bc139184177 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1544,7 +1544,7 @@ sub tracepoint_munge($) {
 sub syscall_munge() {
 	my $void = 0;
 
-	$prototype =~ s@[\r\n\t]+@ @gos; # strip newlines/CR's/tabs
+	$prototype =~ s@[\r\n]+@ @gos; # strip newlines/CR's
 ##	if ($prototype =~ m/SYSCALL_DEFINE0\s*\(\s*(a-zA-Z0-9_)*\s*\)/) {
 	if ($prototype =~ m/SYSCALL_DEFINE0/) {
 		$void = 1;
@@ -1743,6 +1743,8 @@ sub process_file($) {
 	while (s/\\\s*$//) {
 	    $_ .= <IN>;
 	}
+	# Replace tabs by spaces
+        while ($_ =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {};
 	if ($state == STATE_NORMAL) {
 	    if (/$doc_start/o) {
 		$state = STATE_NAME;	# next line is always the function name
@@ -1842,8 +1844,7 @@ sub process_file($) {
 		$in_purpose = 0;
 		$contents = $newcontents;
                 $new_start_line = $.;
-		while ((substr($contents, 0, 1) eq " ") ||
-		       substr($contents, 0, 1) eq "\t") {
+		while (substr($contents, 0, 1) eq " ") {
 		    $contents = substr($contents, 1);
 		}
 		if ($contents ne "") {
@@ -1912,8 +1913,7 @@ sub process_file($) {
 		$contents = $2;
                 $new_start_line = $.;
 		if ($contents ne "") {
-		    while ((substr($contents, 0, 1) eq " ") ||
-		           substr($contents, 0, 1) eq "\t") {
+		    while (substr($contents, 0, 1) eq " ") {
 			$contents = substr($contents, 1);
 		    }
 		    $contents .= "\n";

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* Re: [PATCH v2 09/13] scripts: kernel-doc: parse next structs/unions
  2017-09-29 12:08     ` Mauro Carvalho Chehab
@ 2017-09-29 13:07       ` Markus Heiser
  2017-09-29 13:33         ` Mauro Carvalho Chehab
  0 siblings, 1 reply; 28+ messages in thread
From: Markus Heiser @ 2017-09-29 13:07 UTC (permalink / raw)
  To: Mauro Carvalho Chehab
  Cc: Linux Media Mailing List, Jonathan Corbet, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter


> Am 29.09.2017 um 14:08 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> 
> Em Thu, 28 Sep 2017 18:28:32 +0200
> Markus Heiser <markus.heiser@darmarit.de> escreveu:
> 
>> Hi Mauro,
>> 
>>> Am 27.09.2017 um 23:10 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
>>> 
>> I also untabified the example since tabs in reST are
>> a nightmare, especially in code blocks ... tabulators are the source
>> of all evil [1] ...
>> 
>>  Please, never use tabs in markups or programming languages
>>  where indentation is a part of the markup respectively the 
>>  language!!
> 
> Tabs will exist at the sources, as Kernel coding style recommends its
> usage. There's nothing that can be done to avoid. So, whatever scripts
> we use, it should handle it.

Sorry if I was unclear. I mean we should not use tabs in reST or in py.
In python the indentation is a part of the language syntax, same in
reST; the indentation is the markup. It's not only me who recommend to
avoid tabs:

- reST: http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#whitespace
- python: https://www.python.org/dev/peps/pep-0008/#tabs-or-spaces

both can handle tabs well (with the cost of confusing when I look at a diff), but
in python 3 it gets worse ...

  """Python 3 disallows mixing the use of tabs and spaces for indentation."""
 
If we are looking at C sources, there are no such problems since the
indentation is not a part of the syntax, so what the Kernel coding style
recommends is also correct.

  """Outside of comments, documentation and except in Kconfig, spaces are
     never used for indentation"""

Anyway, as long as it works with tabs I can't stop you ;)

> 
> Thankfully, solving this issue is a one line perl patch, as explained at:
> 	http://perldoc.perl.org/perlfaq4.html#How-do-I-expand-tabs-in-a-string?
> 
> Something like the enclosed (untested) patch.

Hm, as far as I see, this will make no sense, since the kernel-doc
parser flats prototypes to a one-liner and the reST output is independent
(e.g. output_struct_rst) from the origin source code.

-- Markus --

> 
> Thanks,
> Mauro
> 
> [PATCH] kernel-doc: replace tabs by spaces

As I said, you misunderstood me, do not apply this
patch. Replacing tabs when reading source-code is
not needed / even since we flatten  

> 
> Sphinx has a hard time dealing with tabs, causing it to
> misinterpret paragraph continuation.
> 
> As we're now mainly focused on supporting ReST output,
> replace tabs by spaces, in order to avoid troubles when
> the output is parsed by Sphinx.
> 
> Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
> 
> diff --git a/scripts/kernel-doc b/scripts/kernel-doc
> index 69757ee9db4c..7bc139184177 100755
> --- a/scripts/kernel-doc
> +++ b/scripts/kernel-doc
> @@ -1544,7 +1544,7 @@ sub tracepoint_munge($) {
> sub syscall_munge() {
> 	my $void = 0;
> 
> -	$prototype =~ s@[\r\n\t]+@ @gos; # strip newlines/CR's/tabs
> +	$prototype =~ s@[\r\n]+@ @gos; # strip newlines/CR's
> ##	if ($prototype =~ m/SYSCALL_DEFINE0\s*\(\s*(a-zA-Z0-9_)*\s*\)/) {
> 	if ($prototype =~ m/SYSCALL_DEFINE0/) {
> 		$void = 1;
> @@ -1743,6 +1743,8 @@ sub process_file($) {
> 	while (s/\\\s*$//) {
> 	    $_ .= <IN>;
> 	}
> +	# Replace tabs by spaces
> +        while ($_ =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {};
> 	if ($state == STATE_NORMAL) {
> 	    if (/$doc_start/o) {
> 		$state = STATE_NAME;	# next line is always the function name
> @@ -1842,8 +1844,7 @@ sub process_file($) {
> 		$in_purpose = 0;
> 		$contents = $newcontents;
>                 $new_start_line = $.;
> -		while ((substr($contents, 0, 1) eq " ") ||
> -		       substr($contents, 0, 1) eq "\t") {
> +		while (substr($contents, 0, 1) eq " ") {
> 		    $contents = substr($contents, 1);
> 		}
> 		if ($contents ne "") {
> @@ -1912,8 +1913,7 @@ sub process_file($) {
> 		$contents = $2;
>                 $new_start_line = $.;
> 		if ($contents ne "") {
> -		    while ((substr($contents, 0, 1) eq " ") ||
> -		           substr($contents, 0, 1) eq "\t") {
> +		    while (substr($contents, 0, 1) eq " ") {
> 			$contents = substr($contents, 1);
> 		    }
> 		    $contents .= "\n";
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-doc" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v2 09/13] scripts: kernel-doc: parse next structs/unions
  2017-09-29 13:07       ` Markus Heiser
@ 2017-09-29 13:33         ` Mauro Carvalho Chehab
  2017-09-29 15:00           ` Markus Heiser
  0 siblings, 1 reply; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-29 13:33 UTC (permalink / raw)
  To: Markus Heiser
  Cc: Linux Media Mailing List, Jonathan Corbet, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter

Em Fri, 29 Sep 2017 15:07:05 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:

> > Am 29.09.2017 um 14:08 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> > 
> > Em Thu, 28 Sep 2017 18:28:32 +0200
> > Markus Heiser <markus.heiser@darmarit.de> escreveu:
> >   
> >> Hi Mauro,
> >>   
> >>> Am 27.09.2017 um 23:10 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> >>>   
> >> I also untabified the example since tabs in reST are
> >> a nightmare, especially in code blocks ... tabulators are the source
> >> of all evil [1] ...
> >> 
> >>  Please, never use tabs in markups or programming languages
> >>  where indentation is a part of the markup respectively the 
> >>  language!!  
> > 
> > Tabs will exist at the sources, as Kernel coding style recommends its
> > usage. There's nothing that can be done to avoid. So, whatever scripts
> > we use, it should handle it.  
> 
> Sorry if I was unclear. I mean we should not use tabs in reST or in py.

Ah, OK.

> In python the indentation is a part of the language syntax, same in
> reST; the indentation is the markup. It's not only me who recommend to
> avoid tabs:
> 
> - reST: http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#whitespace

What's said there is that tabs can be used, if tab stops at 8th column:

	"Spaces are recommended for indentation, but tabs may also be used. 
	 Tabs will be converted to spaces. Tab stops are at every 8th column."

Basically, it should be doing exactly the same thing as the patch I just
for kernel-doc script :-)

> - python: https://www.python.org/dev/peps/pep-0008/#tabs-or-spaces
> 
> both can handle tabs well (with the cost of confusing when I look at a diff), but
> in python 3 it gets worse ...
>   """Python 3 disallows mixing the use of tabs and spaces for indentation."""

None to argue here. If it is part of the language, either comply or use
some other language that it isn't position oriented.

Btw, Python reminds me about Fortran... I remember I lost some compilation
jobs at the University's mainframe (that used to take an entire day to be
processed, due to student's jobs being processed only at late night) because,
on a couple of punched cards there was a missing space, causing the line to
be considered part of the previous statement...

Perhaps that's the reason why I'm not a big fan of using tabs/spaces
to split statements: it reminds me the old days of Fortran 77 on punched
cards.

> If we are looking at C sources, there are no such problems since the
> indentation is not a part of the syntax, so what the Kernel coding style
> recommends is also correct.
> 
>   """Outside of comments, documentation and except in Kconfig, spaces are
>      never used for indentation"""
> 
> Anyway, as long as it works with tabs I can't stop you ;)
> 
> > 
> > Thankfully, solving this issue is a one line perl patch, as explained at:
> > 	http://perldoc.perl.org/perlfaq4.html#How-do-I-expand-tabs-in-a-string?
> > 
> > Something like the enclosed (untested) patch.  
> 
> Hm, as far as I see, this will make no sense, since the kernel-doc
> parser flats prototypes to a one-liner and the reST output is independent
> (e.g. output_struct_rst) from the origin source code.


I tested with the following test tag (a modified version of the one you
pointed:


<code>
/**
 * struct voltagedomain - omap voltage domain global structure.
 * @name: Name of the voltage domain which can be used as a unique identifier.
 * @scalable: Whether or not this voltage domain is scalable
 * @node: list_head linking all voltage domains
 * @vc: pointer to VC channel associated with this voltagedomain
 * @vp: pointer to VP associated with this voltagedomain
 * @read: read a VC/VP register
 * @write: write a VC/VP register
 * @read: read-modify-write a VC/VP register
 * @sys_clk: system clock name/frequency, used for various timing calculations
 * @sys_clk.rate: lorem ipsum
 * @sys_clk.foo.boo: foo boo. can't explain @sys_clk.foo.boo.
 * @scale: function used to scale the voltage of the voltagedomain
 * @nominal_volt: current nominal voltage for this voltage domain
 * @volt_data: voltage table having the distinct voltages supported
 *             by the domain and other associated per voltage data.
 * 	       cont with spaces _+ tabs
 */
struct voltagedomain {
	char *name;
	bool scalable;
	struct list_head node;
	struct omap_vc_channel *vc;
	const struct omap_vfsm_instance *vfsm;
	struct omap_vp_instance *vp;
	struct omap_voltdm_pmic *pmic;
	struct omap_vp_param *vp_param;
	struct omap_vc_param *vc_param;

	/* VC/VP register access functions: SoC specific */
	u32 (*read) (u8 offset);
	void (*write) (u32 val, u8 offset);
	u32 (*rmw)(u32 mask, u32 bits, u8 offset);

	union {
		const char *name;
		u32 rate;
		union {
			int boo;
		} foo;
	} sys_clk;

	int (*scale) (struct voltagedomain *voltdm,
		      unsigned long target_volt);

	u32 nominal_volt;
	struct omap_volt_data *volt_data;
};
</code>

By purpose, the last line of @volt_data contains:
	<space><tab><spaces>cont line

That's obviously something that people should avoid doing, as an
space before a tab like that is really ugly. 

Before this patch, kernel-doc script produces:

	``volt_data``
	  voltage table having the distinct voltages supported
	  by the domain and other associated per voltage data.
	  	       cont with spaces _+ tabs


With this patch, it is now outputs fine:


	``volt_data``
	  voltage table having the distinct voltages supported
	  by the domain and other associated per voltage data.
	  cont with spaces _+ tabs

So, I guess we should apply it anyway, as I found several bugs 
already due to tabs usage on some kernel-doc markups.


Thanks,
Mauro

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v2 09/13] scripts: kernel-doc: parse next structs/unions
  2017-09-29 13:33         ` Mauro Carvalho Chehab
@ 2017-09-29 15:00           ` Markus Heiser
  2017-09-29 15:19             ` Mauro Carvalho Chehab
  0 siblings, 1 reply; 28+ messages in thread
From: Markus Heiser @ 2017-09-29 15:00 UTC (permalink / raw)
  To: Mauro Carvalho Chehab
  Cc: Linux Media Mailing List, Jonathan Corbet, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter


> Am 29.09.2017 um 15:33 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
>> Sorry if I was unclear. I mean we should not use tabs in reST or in py.
> 
> Ah, OK.
> 
>> In python the indentation is a part of the language syntax, same in
>> reST; the indentation is the markup. It's not only me who recommend to
>> avoid tabs:
>> 
>> - reST: http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#whitespace
> 
> What's said there is that tabs can be used, if tab stops at 8th column:
> 
> 	"Spaces are recommended for indentation, but tabs may also be used. 
> 	 Tabs will be converted to spaces. Tab stops are at every 8th column."
> 
> Basically, it should be doing exactly the same thing as the patch I just
> for kernel-doc script :-)

There is no need to replace tabs in the *prototype* but you are right; in the
kernel-doc comments it is needed. See below where I make a suggestion 
*how to achieve*.

>> - python: https://www.python.org/dev/peps/pep-0008/#tabs-or-spaces
>> 
>> both can handle tabs well (with the cost of confusing when I look at a diff), but
>> in python 3 it gets worse ...
>>  """Python 3 disallows mixing the use of tabs and spaces for indentation."""
> 
> None to argue here. If it is part of the language, either comply or use
> some other language that it isn't position oriented.

Just for info; when Guido van Rossum created python he thought, that
readability counts. With indentation as a part of the syntax, python
forces the developer to write readable code.

> Btw, Python reminds me about Fortran... I remember I lost some compilation
> jobs at the University's mainframe (that used to take an entire day to be
> processed, due to student's jobs being processed only at late night) because,
> on a couple of punched cards there was a missing space, causing the line to
> be considered part of the previous statement...
> 
> Perhaps that's the reason why I'm not a big fan of using tabs/spaces
> to split statements: it reminds me the old days of Fortran 77 on punched
> cards.

I have stack of punched card right here: Nowadays they are excellent
cigarettes filter ;)

> 
>> If we are looking at C sources, there are no such problems since the
>> indentation is not a part of the syntax, so what the Kernel coding style
>> recommends is also correct.
>> 
>>  """Outside of comments, documentation and except in Kconfig, spaces are
>>     never used for indentation"""
>> 
>> Anyway, as long as it works with tabs I can't stop you ;)
>> 
>>> 
>>> Thankfully, solving this issue is a one line perl patch, as explained at:
>>> 	http://perldoc.perl.org/perlfaq4.html#How-do-I-expand-tabs-in-a-string?
>>> 
>>> Something like the enclosed (untested) patch.  
>> 
>> Hm, as far as I see, this will make no sense, since the kernel-doc
>> parser flats prototypes to a one-liner and the reST output is independent
>> (e.g. output_struct_rst) from the origin source code.
> 
> 
> I tested with the following test tag (a modified version of the one you
> pointed:
> 
> 
> <code>
> /**
> * struct voltagedomain - omap voltage domain global structure.
> * @name: Name of the voltage domain which can be used as a unique identifier.
> * @scalable: Whether or not this voltage domain is scalable
> * @node: list_head linking all voltage domains
> * @vc: pointer to VC channel associated with this voltagedomain
> * @vp: pointer to VP associated with this voltagedomain
> * @read: read a VC/VP register
> * @write: write a VC/VP register
> * @read: read-modify-write a VC/VP register
> * @sys_clk: system clock name/frequency, used for various timing calculations
> * @sys_clk.rate: lorem ipsum
> * @sys_clk.foo.boo: foo boo. can't explain @sys_clk.foo.boo.
> * @scale: function used to scale the voltage of the voltagedomain
> * @nominal_volt: current nominal voltage for this voltage domain
> * @volt_data: voltage table having the distinct voltages supported
> *             by the domain and other associated per voltage data.
> * 	       cont with spaces _+ tabs
> */
> struct voltagedomain {
> 	char *name;
> 	bool scalable;
> 	struct list_head node;
> 	struct omap_vc_channel *vc;
> 	const struct omap_vfsm_instance *vfsm;
> 	struct omap_vp_instance *vp;
> 	struct omap_voltdm_pmic *pmic;
> 	struct omap_vp_param *vp_param;
> 	struct omap_vc_param *vc_param;
> 
> 	/* VC/VP register access functions: SoC specific */
> 	u32 (*read) (u8 offset);
> 	void (*write) (u32 val, u8 offset);
> 	u32 (*rmw)(u32 mask, u32 bits, u8 offset);
> 
> 	union {
> 		const char *name;
> 		u32 rate;
> 		union {
> 			int boo;
> 		} foo;
> 	} sys_clk;
> 
> 	int (*scale) (struct voltagedomain *voltdm,
> 		      unsigned long target_volt);
> 
> 	u32 nominal_volt;
> 	struct omap_volt_data *volt_data;
> };
> </code>
> 
> By purpose, the last line of @volt_data contains:
> 	<space><tab><spaces>cont line
> 
> That's obviously something that people should avoid doing, as an
> space before a tab like that is really ugly. 
> 
> Before this patch, kernel-doc script produces:
> 
> 	``volt_data``
> 	  voltage table having the distinct voltages supported
> 	  by the domain and other associated per voltage data.
> 	  	       cont with spaces _+ tabs
> 
> 
> With this patch, it is now outputs fine:
> 
> 
> 	``volt_data``
> 	  voltage table having the distinct voltages supported
> 	  by the domain and other associated per voltage data.
> 	  cont with spaces _+ tabs
> 
> So, I guess we should apply it anyway, as I found several bugs 
> already due to tabs usage on some kernel-doc markups.

I do expand tabs in the feed method of the parser

  https://github.com/return42/linuxdoc/blob/master/linuxdoc/kernel_doc.py#L1713

I guess we should do the same in sub process_file($) with <IN>. My
perl is to bad, may you could take a look at? / Thanks!


 -- Markus --

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v2 09/13] scripts: kernel-doc: parse next structs/unions
  2017-09-29 15:00           ` Markus Heiser
@ 2017-09-29 15:19             ` Mauro Carvalho Chehab
  0 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-29 15:19 UTC (permalink / raw)
  To: Markus Heiser
  Cc: Linux Media Mailing List, Jonathan Corbet, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter

Em Fri, 29 Sep 2017 17:00:36 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:

> > None to argue here. If it is part of the language, either comply or use
> > some other language that it isn't position oriented.  
> 
> Just for info; when Guido van Rossum created python he thought, that
> readability counts. With indentation as a part of the syntax, python
> forces the developer to write readable code.

I'm sympathetic with the idea of having a more readable code, but a similar
effect could be produced by noisy warnings if the indentation is not
correct. It also wouldn't force the others about a personal tabs usage
preference.

For me, the issue that bothers most on Python is that, if I need to comment
out part of the code (for example, to remove an IF clause) to do some tests
or add new prints, the code inside it needs to be re-indented, with slows
down my tests ;)

On all other languages, I just comment out the lines, and, if I want to
place additional prints, I start from column 1, as this is very easy to
notice and strip before pushing the final version.

> I have stack of punched card right here: Nowadays they are excellent
> cigarettes filter ;)

:-)

> 
>   https://github.com/return42/linuxdoc/blob/master/linuxdoc/kernel_doc.py#L1713
> 
> I guess we should do the same in sub process_file($) with <IN>. My
> perl is to bad, may you could take a look at? / Thanks!

Gah! I guess I forgot to attach the patch I wrote on my previous e-mail...
I did exactly what you suggested.

I opted to use this:

    while (<IN>) {
	while (s/\\\s*$//) {
	    $_ .= <IN>;
	}
	# Replace tabs by spaces
        while ($_ =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {};

To replace them. I could, instead, use a perl extension, but that
would add an extra dependency for something that can be done on a
single Perl regex statement.

Thanks,
Mauro

-

>From 7c4ef302b855aeb68773621f379fb6e8813c886c Mon Sep 17 00:00:00 2001
From: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Date: Fri, 29 Sep 2017 08:53:40 -0300
Subject: [PATCH] kernel-doc: replace tabs by spaces

Sphinx has a hard time dealing with tabs, causing it to
misinterpret paragraph continuation.

As we're now mainly focused on supporting ReST output,
replace tabs by spaces, in order to avoid troubles when
the output is parsed by Sphinx.

Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 69757ee9db4c..7bc139184177 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1544,7 +1544,7 @@ sub tracepoint_munge($) {
 sub syscall_munge() {
 	my $void = 0;
 
-	$prototype =~ s@[\r\n\t]+@ @gos; # strip newlines/CR's/tabs
+	$prototype =~ s@[\r\n]+@ @gos; # strip newlines/CR's
 ##	if ($prototype =~ m/SYSCALL_DEFINE0\s*\(\s*(a-zA-Z0-9_)*\s*\)/) {
 	if ($prototype =~ m/SYSCALL_DEFINE0/) {
 		$void = 1;
@@ -1743,6 +1743,8 @@ sub process_file($) {
 	while (s/\\\s*$//) {
 	    $_ .= <IN>;
 	}
+	# Replace tabs by spaces
+        while ($_ =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {};
 	if ($state == STATE_NORMAL) {
 	    if (/$doc_start/o) {
 		$state = STATE_NAME;	# next line is always the function name
@@ -1842,8 +1844,7 @@ sub process_file($) {
 		$in_purpose = 0;
 		$contents = $newcontents;
                 $new_start_line = $.;
-		while ((substr($contents, 0, 1) eq " ") ||
-		       substr($contents, 0, 1) eq "\t") {
+		while (substr($contents, 0, 1) eq " ") {
 		    $contents = substr($contents, 1);
 		}
 		if ($contents ne "") {
@@ -1912,8 +1913,7 @@ sub process_file($) {
 		$contents = $2;
                 $new_start_line = $.;
 		if ($contents ne "") {
-		    while ((substr($contents, 0, 1) eq " ") ||
-		           substr($contents, 0, 1) eq "\t") {
+		    while (substr($contents, 0, 1) eq " ") {
 			$contents = substr($contents, 1);
 		    }
 		    $contents .= "\n";

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* Re: [PATCH v2 09/13] scripts: kernel-doc: parse next structs/unions
  2017-09-28 16:28   ` Markus Heiser
  2017-09-29 12:08     ` Mauro Carvalho Chehab
@ 2017-09-29 15:29     ` Mauro Carvalho Chehab
  2017-09-29 15:44       ` Mauro Carvalho Chehab
  2017-09-29 15:47       ` Markus Heiser
  2017-10-01 11:02     ` Mauro Carvalho Chehab
  2 siblings, 2 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-29 15:29 UTC (permalink / raw)
  To: Markus Heiser
  Cc: Linux Media Mailing List, Jonathan Corbet, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter

Em Thu, 28 Sep 2017 18:28:32 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:

> Hi Mauro,
> 
> > Am 27.09.2017 um 23:10 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:

> > +It is possible to document nested structs unions, like::
> > +
> > +      /**
> > +       * struct nested_foobar - a struct with nested unions and structs
> > +       * @arg1: - first argument of anonymous union/anonymous struct
> > +       * @arg2: - second argument of anonymous union/anonymous struct
> > +       * @arg3: - third argument of anonymous union/anonymous struct
> > +       * @arg4: - fourth argument of anonymous union/anonymous struct
> > +       * @bar.st1.arg1 - first argument of struct st1 on union bar
> > +       * @bar.st1.arg2 - second argument of struct st1 on union bar
> > +       * @bar.st2.arg1 - first argument of struct st2 on union bar
> > +       * @bar.st2.arg2 - second argument of struct st2 on union bar  
> 
> Sorry, this example is totally broken --> below I attached a more
> elaborate example. 
> 
> /* parse-SNIP: my_struct */
> /**
> * struct my_struct - a struct with nested unions and structs
> * @arg1: first argument of anonymous union/anonymous struct
> * @arg2: second argument of anonymous union/anonymous struct
> * @arg3: third argument of anonymous union/anonymous struct
> * @arg4: fourth argument of anonymous union/anonymous struct
> * @bar.st1.arg1: first argument of struct st1 on union bar
> * @bar.st1.arg2: second argument of struct st1 on union bar
> * @bar.st2.arg1: first argument of struct st2 on union bar
> * @bar.st2.arg2: second argument of struct st2 on union bar
> * @bar.st3.arg2: second argument of struct st3 on union bar
> */
> struct my_struct {
>    /* Anonymous union/struct*/
>    union {
> 	struct {
> 	    __u8 arg1 : 1;
> 	    __u8 arg2 : 3;
> 	};
>        struct {
>            int arg1;
>            int arg2;
>        };

I added a:

	#define __u8 char

In order to compile the above. As reported by gcc 7, this is broken:

test2.h:22:16: error: duplicate member ‘arg1’
            int arg1;
                ^~~~
test2.h:23:16: error: duplicate member ‘arg2’
            int arg2;
                ^~~~

You can't have two symbols with the same name on different anonymous
structs.


Thanks,
Mauro

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v2 09/13] scripts: kernel-doc: parse next structs/unions
  2017-09-29 15:29     ` Mauro Carvalho Chehab
@ 2017-09-29 15:44       ` Mauro Carvalho Chehab
  2017-09-29 15:47       ` Markus Heiser
  1 sibling, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-09-29 15:44 UTC (permalink / raw)
  To: Markus Heiser
  Cc: Linux Media Mailing List, Jonathan Corbet, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter

Em Fri, 29 Sep 2017 12:29:31 -0300
Mauro Carvalho Chehab <mchehab@s-opensource.com> escreveu:

> Em Thu, 28 Sep 2017 18:28:32 +0200
> Markus Heiser <markus.heiser@darmarit.de> escreveu:
> 
> > Hi Mauro,
> >   
> > > Am 27.09.2017 um 23:10 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:  
> 
> > > +It is possible to document nested structs unions, like::
> > > +
> > > +      /**
> > > +       * struct nested_foobar - a struct with nested unions and structs
> > > +       * @arg1: - first argument of anonymous union/anonymous struct
> > > +       * @arg2: - second argument of anonymous union/anonymous struct
> > > +       * @arg3: - third argument of anonymous union/anonymous struct
> > > +       * @arg4: - fourth argument of anonymous union/anonymous struct
> > > +       * @bar.st1.arg1 - first argument of struct st1 on union bar
> > > +       * @bar.st1.arg2 - second argument of struct st1 on union bar
> > > +       * @bar.st2.arg1 - first argument of struct st2 on union bar
> > > +       * @bar.st2.arg2 - second argument of struct st2 on union bar    
> > 
> > Sorry, this example is totally broken --> below I attached a more
> > elaborate example. 
> > 
> > /* parse-SNIP: my_struct */
> > /**
> > * struct my_struct - a struct with nested unions and structs
> > * @arg1: first argument of anonymous union/anonymous struct
> > * @arg2: second argument of anonymous union/anonymous struct
> > * @arg3: third argument of anonymous union/anonymous struct
> > * @arg4: fourth argument of anonymous union/anonymous struct
> > * @bar.st1.arg1: first argument of struct st1 on union bar
> > * @bar.st1.arg2: second argument of struct st1 on union bar
> > * @bar.st2.arg1: first argument of struct st2 on union bar
> > * @bar.st2.arg2: second argument of struct st2 on union bar
> > * @bar.st3.arg2: second argument of struct st3 on union bar
> > */
> > struct my_struct {
> >    /* Anonymous union/struct*/
> >    union {
> > 	struct {
> > 	    __u8 arg1 : 1;
> > 	    __u8 arg2 : 3;
> > 	};
> >        struct {
> >            int arg1;
> >            int arg2;
> >        };  
> 
> I added a:
> 
> 	#define __u8 char
> 
> In order to compile the above. As reported by gcc 7, this is broken:
> 
> test2.h:22:16: error: duplicate member ‘arg1’
>             int arg1;
>                 ^~~~
> test2.h:23:16: error: duplicate member ‘arg2’
>             int arg2;
>                 ^~~~
> 
> You can't have two symbols with the same name on different anonymous
> structs.

I guess the enclosed nested struct covers all weirdness that we want
to be covered, and it does compile without errors/warnings.

Regards,
Mauro

- 

/* Test code to test nested structs */

/**
 * struct my_struct - a struct with nested unions and structs
 * @arg1: first argument of anonymous union/anonymous struct
 * @arg2: second argument of anonymous union/anonymous struct
 * @arg1b: first argument of anonymous union/anonymous struct
 * @arg2b: second argument of anonymous union/anonymous struct
 * @arg3: third argument of anonymous union/anonymous struct
 * @arg4: fourth argument of anonymous union/anonymous struct
 * @bar.st1.arg1: first argument of struct st1 on union bar
 * @bar.st1.arg2: second argument of struct st1 on union bar
 * @bar.st2.arg1: first argument of struct st2 on union bar
 * @bar.st2.arg2: second argument of struct st2 on union bar
 * @bar.st3.arg2: second argument of struct st3 on union bar
 * @f1: nested function on anonimous union/struct
 * @bar.st2.f2: nested function on named union/struct
 */
struct my_struct {
   /* Anonymous union/struct*/
   union {
	struct {
	    char arg1 : 1;
	    char arg2 : 3;
	};
       struct {
           int arg1b;
           int arg2b;
       };
       struct {
           void *arg3;
           int arg4;
           int (*f1)(char foo, int bar);
       };
   };
   union {
       struct {
           int arg1;
           int arg2;
       } st1;           /* bar.st1 is undocumented, cause a warning */
       struct {
           void *arg1;  /* bar.st3.arg1 is undocumented, cause a warning */
	    int arg2;
          int (*f2)(char foo, int bar); /* bar.st3.fn2 is undocumented, cause a warning */
       } st2, st3;
       int (*f3)(char foo, int bar); /* f3 is undocumented, cause a warning */
   } bar;               /* bar is undocumented, cause a warning */

   /* private: */
   int undoc_privat;    /* is undocumented but private, no warning */

   /* public: */
   int undoc_public;    /* is undocumented, cause a warning */
};

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v2 09/13] scripts: kernel-doc: parse next structs/unions
  2017-09-29 15:29     ` Mauro Carvalho Chehab
  2017-09-29 15:44       ` Mauro Carvalho Chehab
@ 2017-09-29 15:47       ` Markus Heiser
  1 sibling, 0 replies; 28+ messages in thread
From: Markus Heiser @ 2017-09-29 15:47 UTC (permalink / raw)
  To: Mauro Carvalho Chehab
  Cc: Linux Media Mailing List, Jonathan Corbet, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter


> Am 29.09.2017 um 17:29 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> 
> Em Thu, 28 Sep 2017 18:28:32 +0200
> Markus Heiser <markus.heiser@darmarit.de> escreveu:
> 
>> Hi Mauro,
>> 
>>> Am 27.09.2017 um 23:10 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> 
>>> +It is possible to document nested structs unions, like::
>>> +
>>> +      /**
>>> +       * struct nested_foobar - a struct with nested unions and structs
>>> +       * @arg1: - first argument of anonymous union/anonymous struct
>>> +       * @arg2: - second argument of anonymous union/anonymous struct
>>> +       * @arg3: - third argument of anonymous union/anonymous struct
>>> +       * @arg4: - fourth argument of anonymous union/anonymous struct
>>> +       * @bar.st1.arg1 - first argument of struct st1 on union bar
>>> +       * @bar.st1.arg2 - second argument of struct st1 on union bar
>>> +       * @bar.st2.arg1 - first argument of struct st2 on union bar
>>> +       * @bar.st2.arg2 - second argument of struct st2 on union bar

With totally broken I mean "broken in sense of kernel-doc markup".
E.g. the missing ":" at the end of "@bar.st2.arg2:"  instead, there
is a minus "-". And in the first example you gave, there also missed
a ";" at the end of some of the elements. Anyway, the example
above is obsolete.

>>>  
>> 
>> Sorry, this example is totally broken --> below I attached a more
>> elaborate example. 
>> 
>> /* parse-SNIP: my_struct */
>> /**
>> * struct my_struct - a struct with nested unions and structs
>> * @arg1: first argument of anonymous union/anonymous struct
>> * @arg2: second argument of anonymous union/anonymous struct
>> * @arg3: third argument of anonymous union/anonymous struct
>> * @arg4: fourth argument of anonymous union/anonymous struct
>> * @bar.st1.arg1: first argument of struct st1 on union bar
>> * @bar.st1.arg2: second argument of struct st1 on union bar
>> * @bar.st2.arg1: first argument of struct st2 on union bar
>> * @bar.st2.arg2: second argument of struct st2 on union bar
>> * @bar.st3.arg2: second argument of struct st3 on union bar
>> */
>> struct my_struct {
>>   /* Anonymous union/struct*/
>>   union {
>> 	struct {
>> 	    __u8 arg1 : 1;
>> 	    __u8 arg2 : 3;
>> 	};
>>       struct {
>>           int arg1;
>>           int arg2;
>>       };
> 
> I added a:
> 
> 	#define __u8 char
> 
> In order to compile the above. As reported by gcc 7, this is broken:
> 
> test2.h:22:16: error: duplicate member ‘arg1’
>            int arg1;
>                ^~~~
> test2.h:23:16: error: duplicate member ‘arg2’
>            int arg2;
>                ^~~~
> 
> You can't have two symbols with the same name on different anonymous
> structs.

Ah, ok lets fix the example.

 -- Markus --

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v2 09/13] scripts: kernel-doc: parse next structs/unions
  2017-09-28 16:28   ` Markus Heiser
  2017-09-29 12:08     ` Mauro Carvalho Chehab
  2017-09-29 15:29     ` Mauro Carvalho Chehab
@ 2017-10-01 11:02     ` Mauro Carvalho Chehab
  2 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-10-01 11:02 UTC (permalink / raw)
  To: Markus Heiser
  Cc: Linux Media Mailing List, Jonathan Corbet, Mauro Carvalho Chehab,
	Linux Doc Mailing List, Daniel Vetter

Em Thu, 28 Sep 2017 18:28:32 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:

> Hi Mauro,
> 
> > Am 27.09.2017 um 23:10 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:

> > +	# Split nested struct/union elements as newer ones
> > +	my $cont = 1;
> > +	while ($cont) {
> > +		$cont = 0;  
> 
> 
> You ignored the remarks I made to v1: 
> 
>  this outer loop is not needed!
> 
> 
> > +		while ($members =~ m/(struct|union)([^{};]+){([^{}]*)}([^{}\;]*)\;/) {  
> 
> 
> The inner loop is enough.
> 
> > +			my $newmember = "$1 $4;";
> > +			my $id = $4;  
> 
> 
> > Am 26.09.2017 um 21:16 schrieb Mauro Carvalho Chehab <mchehab@s-opensource.com>:
> >   
> >> This won't work if  you have e.g.
> >> 
> >>  union car {int foo;} bar1, bar2, *bbar3;
> >> 
> >> where $id will be "bar1, bar2, *bbar3", you need to split
> >> this string and iterate over the id's.  
> > 
> > That's true, if we have cases like that. I hope not!  
> 
> We have! I can't remember where, but I hit such constructs while
> parsing the whole sources.
> 
> > 
> > Seriously, in the above case, the developer should very likely declare
> > the union outside the main struct and have its own kernel-doc markup.
> > 
> > My personal taste would be to not explicitly support the above. If it
> > hits some code, try to argue with the developer first, before patching
> > kernel-doc to such weird stuff.  
> 
> Why is it wired? Anyway we have it and this kernel-doc fails with.
> 
> It also fails with bit types e.g.
> 
>        struct {
> 	    __u8 arg1 : 1;
> 	    __u8 arg2 : 3;
>        };
> 
> 
> May it help, if you inspect the py-version of this loop:
> 
>    https://github.com/return42/linuxdoc/blob/master/linuxdoc/kernel_doc.py#L2499

I addressed the things you pointed above.

I decided to add the new logic on a separate patch, as handling
those special cases seemed complex enough to justify placing it
in separate.

I also added the test for nested structs at the commit log, as it may
help if someone would further need to improve it to handle other
cases.

The entire patch series is at:
	https://git.linuxtv.org/mchehab/experimental.git/log/?h=nested-fix-v4


Thanks,
Mauro

[PATCH] scripts: kernel-doc: improve nested logic to handle multiple
 identifiers

It is possible to use nested structs like:

struct {
	struct {
		void *arg1;
	} st1, st2, *st3, st4;
}

Handling it requires to split each parameter. Change the logic
to allow such definitions.

In order to test the new nested logic, the following file
was used to test

<code>
struct foo { int a; }; /* Just to avoid errors if compiled */

/**
 * struct my_struct - a struct with nested unions and structs
 * @arg1: first argument of anonymous union/anonymous struct
 * @arg2: second argument of anonymous union/anonymous struct
 * @arg1b: first argument of anonymous union/anonymous struct
 * @arg2b: second argument of anonymous union/anonymous struct
 * @arg3: third argument of anonymous union/anonymous struct
 * @arg4: fourth argument of anonymous union/anonymous struct
 * @bar.st1.arg1: first argument of struct st1 on union bar
 * @bar.st1.arg2: second argument of struct st1 on union bar
 * @bar.st1.bar1: bar1 at st1
 * @bar.st1.bar2: bar2 at st1
 * @bar.st2.arg1: first argument of struct st2 on union bar
 * @bar.st2.arg2: second argument of struct st2 on union bar
 * @bar.st3.arg2: second argument of struct st3 on union bar
 * @f1: nested function on anonimous union/struct
 * @bar.st2.f2: nested function on named union/struct
 */
struct my_struct {
   /* Anonymous union/struct*/
   union {
	struct {
	    char arg1 : 1;
	    char arg2 : 3;
	};
       struct {
           int arg1b;
           int arg2b;
       };
       struct {
           void *arg3;
           int arg4;
           int (*f1)(char foo, int bar);
       };
   };
   union {
       struct {
           int arg1;
           int arg2;
	   struct foo bar1, *bar2;
       } st1;           /* bar.st1 is undocumented, cause a warning */
       struct {
           void *arg1;  /* bar.st3.arg1 is undocumented, cause a warning */
	    int arg2;
          int (*f2)(char foo, int bar); /* bar.st3.fn2 is undocumented, cause a warning */
       } st2, st3, *st4;
       int (*f3)(char foo, int bar); /* f3 is undocumented, cause a warning */
   } bar;               /* bar is undocumented, cause a warning */

   /* private: */
   int undoc_privat;    /* is undocumented but private, no warning */

   /* public: */
   int undoc_public;    /* is undocumented, cause a warning */
};
</code>

It produces the following warnings, as expected:

test2.h:57: warning: Function parameter or member 'bar' not described in 'my_struct'
test2.h:57: warning: Function parameter or member 'bar.st1' not described in 'my_struct'
test2.h:57: warning: Function parameter or member 'bar.st2' not described in 'my_struct'
test2.h:57: warning: Function parameter or member 'bar.st3' not described in 'my_struct'
test2.h:57: warning: Function parameter or member 'bar.st3.arg1' not described in 'my_struct'
test2.h:57: warning: Function parameter or member 'bar.st3.f2' not described in 'my_struct'
test2.h:57: warning: Function parameter or member 'bar.st4' not described in 'my_struct'
test2.h:57: warning: Function parameter or member 'bar.st4.arg1' not described in 'my_struct'
test2.h:57: warning: Function parameter or member 'bar.st4.arg2' not described in 'my_struct'
test2.h:57: warning: Function parameter or member 'bar.st4.f2' not described in 'my_struct'
test2.h:57: warning: Function parameter or member 'bar.f3' not described in 'my_struct'
test2.h:57: warning: Function parameter or member 'undoc_public' not described in 'my_struct'

Suggested-by: Markus Heiser <markus.heiser@darmarit.de>
Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 2dd36c2256fa..0037be99d44d 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1004,15 +1004,16 @@ sub dump_struct($$) {
 	my $declaration = $members;
 
 	# Split nested struct/union elements as newer ones
-	my $cont = 1;
-	while ($cont) {
-		$cont = 0;
-		while ($members =~ m/(struct|union)([^{};]+){([^{}]*)}([^{}\;]*)\;/) {
-			my $newmember = "$1 $4;";
-			my $id = $4;
-			my $content = $3;
+	while ($members =~ m/(struct|union)([^{};]+){([^{}]*)}([^{}\;]*)\;/) {
+		my $newmember;
+		my $maintype = $1;
+		my $ids = $4;
+		my $content = $3;
+		foreach my $id(split /,/, $ids) {
+			$newmember .= "$maintype $id; ";
+
 			$id =~ s/[:\[].*//;
-			$id =~ s/^\*+//;
+			$id =~ s/^\s*\**(\S+)\s*/$1/;
 			foreach my $arg (split /;/, $content) {
 				next if ($arg =~ m/^\s*$/);
 				if ($arg =~ m/^([^\(]+\(\*?\s*)([\w\.]*)(\s*\).*)/) {
@@ -1023,30 +1024,48 @@ sub dump_struct($$) {
 					next if (!$name);
 					if ($id =~ m/^\s*$/) {
 						# anonymous struct/union
-						$newmember .= "$type$name$extra;";
+						$newmember .= "$type$name$extra; ";
 					} else {
-						$newmember .= "$type$id.$name$extra;";
+						$newmember .= "$type$id.$name$extra; ";
 					}
 				} else {
-					my $type = $arg;
-					my $name = $arg;
-					$type =~ s/\s\S+$//;
-					$name =~ s/.*\s+//;
-					$name =~ s/[:\[].*//;
-					$name =~ s/^\*+//;
-					next if (($name =~ m/^\s*$/));
-					if ($id =~ m/^\s*$/) {
-						# anonymous struct/union
-						$newmember .= "$type $name;";
+					my $type;
+					my $names;
+					$arg =~ s/^\s+//;
+					$arg =~ s/\s+$//;
+					# Handle bitmaps
+					$arg =~ s/:\s*\d+\s*//g;
+					# Handle arrays
+					$arg =~ s/\[\S+\]//g;
+					# The type may have multiple words,
+					# and multiple IDs can be defined, like:
+					#	const struct foo, *bar, foobar
+					# So, we remove spaces when parsing the
+					# names, in order to match just names
+					# and commas for the names
+					$arg =~ s/\s*,\s*/,/g;
+					if ($arg =~ m/(.*)\s+([\S+,]+)/) {
+						$type = $1;
+						$names = $2;
 					} else {
-						$newmember .= "$type $id.$name;";
+						$newmember .= "$arg; ";
+						next;
+					}
+					foreach my $name (split /,/, $names) {
+						$name =~ s/^\s*\**(\S+)\s*/$1/;
+						next if (($name =~ m/^\s*$/));
+						if ($id =~ m/^\s*$/) {
+							# anonymous struct/union
+							$newmember .= "$type $name; ";
+						} else {
+							$newmember .= "$type $id.$name; ";
+						}
 					}
 				}
 			}
-			$members =~ s/(struct|union)([^{};]+){([^{}]*)}([^{}\;]*)\;/$newmember/;
-			$cont = 1;
-		};
-	};
+		}
+		$members =~ s/(struct|union)([^{};]+){([^{}]*)}([^{}\;]*)\;/$newmember/;
+	}
 
 	# Ignore other nested elements, like enums
 	$members =~ s/({[^\{\}]*})//g;

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* Re: [PATCH v2 12/13] scripts: kernel-doc: handle nested struct function arguments
  2017-09-28 16:32   ` Markus Heiser
@ 2017-10-01 11:18     ` Mauro Carvalho Chehab
  0 siblings, 0 replies; 28+ messages in thread
From: Mauro Carvalho Chehab @ 2017-10-01 11:18 UTC (permalink / raw)
  To: Markus Heiser
  Cc: Linux Media Mailing List, Jonathan Corbet, Mauro Carvalho Chehab,
	Linux Doc Mailing List

Em Thu, 28 Sep 2017 18:32:30 +0200
Markus Heiser <markus.heiser@darmarit.de> escreveu:

> Hi Mauro,
> 
> this 'else' addition seems a bit spooky to me. As I commented in patch 09/13
> may it helps when you look at 
> 
>   https://github.com/return42/linuxdoc/blob/master/linuxdoc/kernel_doc.py#L2499
> 
> which is IMO a bit more clear.

Please don't top post. It makes really hard to understand what you meant.

Anyway, as I answered to patc 9/13, I opted to add a separate patch in
order to solve the remaining issues.

If Jon prefers, I can just fold the three patches into one, although
IMHO it is better to keep them in separate.

Regards,
Mauro

^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2017-10-01 11:19 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-09-27 21:10 [PATCH v2 00/13] kernel-doc: add supported to document nested structs/$ Mauro Carvalho Chehab
2017-09-27 21:10 ` [PATCH v2 01/13] scripts: kernel-doc: get rid of unused output formats Mauro Carvalho Chehab
2017-09-27 21:10 ` [PATCH v2 02/13] docs: kernel-doc.rst: better describe kernel-doc arguments Mauro Carvalho Chehab
2017-09-27 21:10 ` [PATCH v2 03/13] docs: kernel-doc.rst: improve private members description Mauro Carvalho Chehab
2017-09-27 21:10 ` [PATCH v2 04/13] docs: kernel-doc.rst: improve function documentation section Mauro Carvalho Chehab
2017-09-27 21:10 ` [PATCH v2 05/13] docs: kernel-doc.rst: improve structs chapter Mauro Carvalho Chehab
2017-09-27 21:10 ` [PATCH v2 06/13] docs: kernel-doc.rst: improve typedef documentation Mauro Carvalho Chehab
2017-09-27 21:10 ` [PATCH v2 07/13] docs: kernel-doc.rst: add documentation about man pages Mauro Carvalho Chehab
2017-09-27 21:20   ` Randy Dunlap
2017-09-28  1:12     ` Mauro Carvalho Chehab
2017-09-27 21:10 ` [PATCH v2 08/13] docs: get rid of kernel-doc-nano-HOWTO.txt Mauro Carvalho Chehab
2017-09-27 21:10 ` [PATCH v2 09/13] scripts: kernel-doc: parse next structs/unions Mauro Carvalho Chehab
2017-09-28 16:28   ` Markus Heiser
2017-09-29 12:08     ` Mauro Carvalho Chehab
2017-09-29 13:07       ` Markus Heiser
2017-09-29 13:33         ` Mauro Carvalho Chehab
2017-09-29 15:00           ` Markus Heiser
2017-09-29 15:19             ` Mauro Carvalho Chehab
2017-09-29 15:29     ` Mauro Carvalho Chehab
2017-09-29 15:44       ` Mauro Carvalho Chehab
2017-09-29 15:47       ` Markus Heiser
2017-10-01 11:02     ` Mauro Carvalho Chehab
2017-09-27 21:10 ` [PATCH v2 10/13] scripts: kernel-doc: get rid of $nested parameter Mauro Carvalho Chehab
2017-09-27 21:10 ` [PATCH v2 11/13] scripts: kernel-doc: print the declaration name on warnings Mauro Carvalho Chehab
2017-09-27 21:10 ` [PATCH v2 12/13] scripts: kernel-doc: handle nested struct function arguments Mauro Carvalho Chehab
2017-09-28 16:32   ` Markus Heiser
2017-10-01 11:18     ` Mauro Carvalho Chehab
2017-09-27 21:10 ` [PATCH v2 13/13] [RFC] w1_netlink.h: add support for nested structs Mauro Carvalho Chehab

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.