[PATCH 0/9] kernel-doc/docproc prep work for reStructuredText

[PATCH 0/9] kernel-doc/docproc prep work for reStructuredText

[Jani Nikula]

Jon, I was hoping we could consider nudging things forward a bit in the
kernel-doc and docproc reStructuredText front already in 4.7. I know
it's a bit close to the merge window, but this should not interfere with
anything else, and some of it are just trivial cleanups that I've been
carrying around locally.

Obviously this doesn't actually add anything that uses them yet, but I
think it would be helpful to have some common base in to ease
collaboration.

BR,
Jani.


Jani Nikula (8):
  kernel-doc: rewrite usage description, remove duplicated comments
  kernel-doc: use rst C domain directives and references for types
  docproc: add variables for subcommand and filename
  docproc: reduce unnecessary indentation
  docproc: abstract docproc directive detection
  docproc: abstract terminating lines at first space
  docproc: add support for reStructuredText format via --rst option
  docproc: print a comment about autogeneration for rst output

Jonathan Corbet (1):
  kernel-doc: produce RestructuredText output

 scripts/docproc.c  | 221 +++++++++++++++++++++++++++----------
 scripts/kernel-doc | 315 +++++++++++++++++++++++++++++++++++++++++++++--------
 2 files changed, 430 insertions(+), 106 deletions(-)

-- 
2.1.4

[PATCH 1/9] kernel-doc: rewrite usage description, remove duplicated comments

[Jani Nikula]

Instead of having the kernel-doc usage in both comments and in output to
the user, merge them all to one here document. While at it, imrove the
text and make it pretty. Give shoemaker's children some shoes.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 83 ++++++++++++++++++++++++------------------------------
 1 file changed, 37 insertions(+), 46 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index c37255bb620d..29fd5cabb657 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -39,41 +39,43 @@ use strict;
 # 25/07/2012 - Added support for HTML5
 # -- Dan Luedtke <mail@danrl.de>
 
-#
-# This will read a 'c' file and scan for embedded comments in the
-# style of gnome comments (+minor extensions - see below).
-#
-
-# Note: This only supports 'c'.
-
-# usage:
-# kernel-doc [ -docbook | -html | -html5 | -text | -man | -list ]
-#            [ -no-doc-sections ]
-#            [ -function funcname [ -function funcname ...] ]
-#            c file(s)s > outputfile
-# or
-#            [ -nofunction funcname [ -function funcname ...] ]
-#            c file(s)s > outputfile
-#
-#  Set output format using one of -docbook -html -html5 -text or -man.
-#  Default is man.
-#  The -list format is for internal use by docproc.
-#
-#  -no-doc-sections
-#	Do not output DOC: sections
-#
-#  -function funcname
-#	If set, then only generate documentation for the given function(s) or
-#	DOC: section titles.  All other functions and DOC: sections are ignored.
-#
-#  -nofunction funcname
-#	If set, then only generate documentation for the other function(s)/DOC:
-#	sections. Cannot be used together with -function (yes, that's a bug --
-#	perl hackers can fix it 8))
-#
-#  c files - list of 'c' files to process
-#
-#  All output goes to stdout, with errors to stderr.
+sub usage {
+    my $message = <<"EOF";
+Usage: $0 [OPTION ...] FILE ...
+
+Read C language source or header FILEs, extract embedded documentation comments,
+and print formatted documentation to standard output.
+
+The documentation comments are identified by "/**" opening comment mark. See
+Documentation/kernel-doc-nano-HOWTO.txt for the documentation comment syntax.
+
+Output format selection (mutually exclusive):
+  -docbook		Output DocBook format.
+  -html			Output HTML format.
+  -html5		Output HTML5 format.
+  -list			Output symbol list format. This is for use by docproc.
+  -man			Output troff manual page format. This is the default.
+  -text			Output plain text format.
+
+Output selection (mutually exclusive):
+  -function NAME	Only output documentation for the given function(s)
+			or DOC: section title(s). All other functions and DOC:
+			sections are ignored. May be specified multiple times.
+  -nofunction NAME	Do NOT output documentation for the given function(s);
+			only output documentation for the other functions and
+			DOC: sections. May be specified multiple times.
+
+Output selection modifiers:
+  -no-doc-sections	Do not output DOC: sections.
+
+Other parameters:
+  -v			Verbose output, more warnings and other information.
+  -h			Print this help.
+
+EOF
+    print $message;
+    exit 1;
+}
 
 #
 # format of comments.
@@ -437,17 +439,6 @@ while ($ARGV[0] =~ m/^-(.*)/) {
 
 # continue execution near EOF;
 
-sub usage {
-    print "Usage: $0 [ -docbook | -html | -html5 | -text | -man | -list ]\n";
-    print "         [ -no-doc-sections ]\n";
-    print "         [ -function funcname [ -function funcname ...] ]\n";
-    print "         [ -nofunction funcname [ -nofunction funcname ...] ]\n";
-    print "         [ -v ]\n";
-    print "         c source file(s) > outputfile\n";
-    print "         -v : verbose output, more warnings & other info listed\n";
-    exit 1;
-}
-
 # get kernel version from env
 sub get_kernel_version() {
     my $version = 'unknown kernel version';
-- 
2.1.4

[PATCH 2/9] kernel-doc: produce RestructuredText output

[Jani Nikula]

From: Jonathan Corbet <corbet@lwn.net>

If given the -rst flag, output will now be in RestructuredText.  Various
glitches to be worked out yet.

In the end I decided not to use RST section headings within the kerneldoc
comments.  gpu.tmpl already has headings five levels deep; adding more is
not going to bring clarity.

This is really just Jani Nikula's asciidoc change with the serial numbers
filed off.  It's a hack job that doubtless needs a lot of cleaning up.

Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/kernel-doc | 233 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 233 insertions(+)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 29fd5cabb657..0ad1fb0e3031 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -55,6 +55,7 @@ Output format selection (mutually exclusive):
   -html5		Output HTML5 format.
   -list			Output symbol list format. This is for use by docproc.
   -man			Output troff manual page format. This is the default.
+  -rst			Output reStructuredText format.
   -text			Output plain text format.
 
 Output selection (mutually exclusive):
@@ -203,6 +204,8 @@ my $type_param = '\@(\w+)';
 my $type_struct = '\&((struct\s*)*[_\w]+)';
 my $type_struct_xml = '\\&amp;((struct\s*)*[_\w]+)';
 my $type_env = '(\$\w+)';
+my $type_enum_full = '\&(enum)\s*([_\w]+)';
+my $type_struct_full = '\&(struct)\s*([_\w]+)';
 
 # Output conversion substitutions.
 #  One for each output format
@@ -268,6 +271,17 @@ my @highlights_text = (
 		      );
 my $blankline_text = "";
 
+# rst-mode
+my @highlights_rst = (
+                       [$type_constant, "``\$1``"],
+                       [$type_func, "\\:c\\:func\\:`\$1`"],
+                       [$type_struct_full, "\\:ref\\:`\$1 \$2`"],
+                       [$type_enum_full, "\\:ref\\:`\$1 \$2`"],
+                       [$type_struct, "\\:ref\\:`struct \$1`"],
+                       [$type_param, "**\$1**"]
+		      );
+my $blankline_rst = "\n";
+
 # list mode
 my @highlights_list = (
                        [$type_constant, "\$1"],
@@ -404,6 +418,10 @@ while ($ARGV[0] =~ m/^-(.*)/) {
 	$output_mode = "text";
 	@highlights = @highlights_text;
 	$blankline = $blankline_text;
+    } elsif ($cmd eq "-rst") {
+	$output_mode = "rst";
+	@highlights = @highlights_rst;
+	$blankline = $blankline_rst;
     } elsif ($cmd eq "-docbook") {
 	$output_mode = "xml";
 	@highlights = @highlights_xml;
@@ -1704,6 +1722,209 @@ sub output_blockhead_text(%) {
     }
 }
 
+##
+# output in restructured text
+#
+
+#
+# This could use some work; it's used to output the DOC: sections, and
+# starts by putting out the name of the doc section itself, but that tends
+# to duplicate a header already in the template file.
+#
+sub output_blockhead_rst(%) {
+    my %args = %{$_[0]};
+    my ($parameter, $section);
+
+    foreach $section (@{$args{'sectionlist'}}) {
+	print "**$section**\n\n";
+	output_highlight_rst($args{'sections'}{$section});
+	print "\n";
+    }
+}
+
+sub output_highlight_rst {
+    my $contents = join "\n",@_;
+    my $line;
+
+    # undo the evil effects of xml_escape() earlier
+    $contents = xml_unescape($contents);
+
+    eval $dohighlight;
+    die $@ if $@;
+
+    foreach $line (split "\n", $contents) {
+	if ($line eq "") {
+	    print $lineprefix, $blankline;
+	} else {
+	    $line =~ s/\\\\\\/\&/g;
+	    print $lineprefix, $line;
+	}
+	print "\n";
+    }
+}
+
+sub output_function_rst(%) {
+    my %args = %{$_[0]};
+    my ($parameter, $section);
+    my $start;
+
+    print ".. c:function:: ";
+    if ($args{'functiontype'} ne "") {
+	$start = $args{'functiontype'} . " " . $args{'function'} . " (";
+    } else {
+	$start = $args{'function'} . " (";
+    }
+    print $start;
+
+    my $count = 0;
+    foreach my $parameter (@{$args{'parameterlist'}}) {
+	if ($count ne 0) {
+	    print ", ";
+	}
+	$count++;
+	$type = $args{'parametertypes'}{$parameter};
+	if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
+	    # pointer-to-function
+	    print $1 . $parameter . ") (" . $2;
+	} else {
+	    print $type . " " . $parameter;
+	}
+    }
+    print ")\n\n    " . $args{'purpose'} . "\n\n";
+
+    print ":Parameters:\n\n";
+    foreach $parameter (@{$args{'parameterlist'}}) {
+	my $parameter_name = $parameter;
+	#$parameter_name =~ s/\[.*//;
+	$type = $args{'parametertypes'}{$parameter};
+
+	if ($type ne "") {
+	    print "      ``$type $parameter``\n";
+	} else {
+	    print "      ``$parameter``\n";
+	}
+	if ($args{'parameterdescs'}{$parameter_name} ne $undescribed) {
+	    my $oldprefix = $lineprefix;
+	    $lineprefix = "        ";
+	    output_highlight_rst($args{'parameterdescs'}{$parameter_name});
+	    $lineprefix = $oldprefix;
+	} else {
+	    print "\n        _undescribed_\n";
+	}
+	print "\n";
+    }
+    output_section_rst(@_);
+}
+
+sub output_section_rst(%) {
+    my %args = %{$_[0]};
+    my $section;
+    my $oldprefix = $lineprefix;
+    $lineprefix = "        ";
+
+    foreach $section (@{$args{'sectionlist'}}) {
+	print ":$section:\n\n";
+	output_highlight_rst($args{'sections'}{$section});
+	print "\n";
+    }
+    print "\n";
+    $lineprefix = $oldprefix;
+}
+
+sub output_enum_rst(%) {
+    my %args = %{$_[0]};
+    my ($parameter);
+    my $count;
+
+    my $name = "enum " . $args{'enum'};
+    print ".. _" . $name . ":\n\n";
+    print "**$name**\n\n";
+    print "    " . $args{'purpose'} . "\n\n";
+
+    print "..\n\n:Constants:\n\n";
+    my $oldprefix = $lineprefix;
+    $lineprefix = "    ";
+    foreach $parameter (@{$args{'parameterlist'}}) {
+	print "  `$parameter`\n";
+	if ($args{'parameterdescs'}{$parameter} ne $undescribed) {
+	    output_highlight_rst($args{'parameterdescs'}{$parameter});
+	} else {
+	    print "    undescribed\n";
+	}
+	print "\n";
+    }
+    $lineprefix = $oldprefix;
+    output_section_rst(@_);
+}
+
+sub output_typedef_rst(%) {
+    my %args = %{$_[0]};
+    my ($parameter);
+    my $count;
+    my $name = "typedef " . $args{'typedef'};
+
+    print "**$name**\n\n";
+    print $args{'purpose'} . "\n\n";
+
+    output_section_rst(@_);
+}
+
+sub output_struct_rst(%) {
+    my %args = %{$_[0]};
+    my ($parameter);
+    my $name = $args{'type'} . " " . $args{'struct'};
+
+    print ".. _" . $name . ":\n\n";
+    print "**$name**\n\n";
+    print "    " . $args{'purpose'} . "\n\n";
+
+    print ":Definition:\n\n";
+    print " ::\n\n";
+    print "  " . $args{'type'} . " " . $args{'struct'} . " {\n";
+    foreach $parameter (@{$args{'parameterlist'}}) {
+	if ($parameter =~ /^#/) {
+	    print "    " . "$parameter\n";
+	    next;
+	}
+
+	my $parameter_name = $parameter;
+	$parameter_name =~ s/\[.*//;
+
+	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
+	$type = $args{'parametertypes'}{$parameter};
+	if ($type =~ m/([^\(]*\(\*)\s*\)\s*\(([^\)]*)\)/) {
+	    # pointer-to-function
+	    print "    $1 $parameter) ($2);\n";
+	} elsif ($type =~ m/^(.*?)\s*(:.*)/) {
+	    # bitfield
+	    print "    $1 $parameter$2;\n";
+	} else {
+	    print "    " . $type . " " . $parameter . ";\n";
+	}
+    }
+    print "  };\n\n";
+
+    print ":Members:\n\n";
+    foreach $parameter (@{$args{'parameterlist'}}) {
+	($parameter =~ /^#/) && next;
+
+	my $parameter_name = $parameter;
+	$parameter_name =~ s/\[.*//;
+
+	($args{'parameterdescs'}{$parameter_name} ne $undescribed) || next;
+	$type = $args{'parametertypes'}{$parameter};
+	print "      `$type $parameter`" . "\n";
+	my $oldprefix = $lineprefix;
+	$lineprefix = "        ";
+	output_highlight_rst($args{'parameterdescs'}{$parameter_name});
+	$lineprefix = $oldprefix;
+	print "\n";
+    }
+    print "\n";
+    output_section_rst(@_);
+}
+
+
 ## list mode output functions
 
 sub output_function_list(%) {
@@ -2405,6 +2626,18 @@ sub xml_escape($) {
 	return $text;
 }
 
+# xml_unescape: reverse the effects of xml_escape
+sub xml_unescape($) {
+	my $text = shift;
+	if (($output_mode eq "text") || ($output_mode eq "man")) {
+		return $text;
+	}
+	$text =~ s/\\\\\\amp;/\&/g;
+	$text =~ s/\\\\\\lt;/</g;
+	$text =~ s/\\\\\\gt;/>/g;
+	return $text;
+}
+
 # convert local escape strings to html
 # local escape strings look like:  '\\\\menmonic:' (that's 4 backslashes)
 sub local_unescape($) {
-- 
2.1.4

[PATCH 3/9] kernel-doc: use rst C domain directives and references for types

[Jani Nikula]

First, the headings for structs, enums and typedefs will be similar to
functions. Second, this provides a kind of namespace for cross
references. Third, and most importantly, the return and parameter types
from .. c:function:: definitions will automagically become cross
references to the documented types.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
---
 scripts/kernel-doc | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 0ad1fb0e3031..2fc8fad5195e 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -275,9 +275,9 @@ my $blankline_text = "";
 my @highlights_rst = (
                        [$type_constant, "``\$1``"],
                        [$type_func, "\\:c\\:func\\:`\$1`"],
-                       [$type_struct_full, "\\:ref\\:`\$1 \$2`"],
-                       [$type_enum_full, "\\:ref\\:`\$1 \$2`"],
-                       [$type_struct, "\\:ref\\:`struct \$1`"],
+                       [$type_struct_full, "\\:c\\:type\\:`\$1 \$2 <\$2>`"],
+                       [$type_enum_full, "\\:c\\:type\\:`\$1 \$2 <\$2>`"],
+                       [$type_struct, "\\:c\\:type\\:`struct \$1 <\$1>`"],
                        [$type_param, "**\$1**"]
 		      );
 my $blankline_rst = "\n";
@@ -1835,10 +1835,9 @@ sub output_enum_rst(%) {
     my %args = %{$_[0]};
     my ($parameter);
     my $count;
-
     my $name = "enum " . $args{'enum'};
-    print ".. _" . $name . ":\n\n";
-    print "**$name**\n\n";
+
+    print "\n\n.. c:type:: " . $name . "\n\n";
     print "    " . $args{'purpose'} . "\n\n";
 
     print "..\n\n:Constants:\n\n";
@@ -1863,8 +1862,9 @@ sub output_typedef_rst(%) {
     my $count;
     my $name = "typedef " . $args{'typedef'};
 
-    print "**$name**\n\n";
-    print $args{'purpose'} . "\n\n";
+    ### FIXME: should the name below contain "typedef" or not?
+    print "\n\n.. c:type:: " . $name . "\n\n";
+    print "    " . $args{'purpose'} . "\n\n";
 
     output_section_rst(@_);
 }
@@ -1874,8 +1874,7 @@ sub output_struct_rst(%) {
     my ($parameter);
     my $name = $args{'type'} . " " . $args{'struct'};
 
-    print ".. _" . $name . ":\n\n";
-    print "**$name**\n\n";
+    print "\n\n.. c:type:: " . $name . "\n\n";
     print "    " . $args{'purpose'} . "\n\n";
 
     print ":Definition:\n\n";
-- 
2.1.4

[PATCH 4/9] docproc: add variables for subcommand and filename

[Jani Nikula]

Improves clarity. No functional changes.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/docproc.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/scripts/docproc.c b/scripts/docproc.c
index e267e621431a..48abc01a920e 100644
--- a/scripts/docproc.c
+++ b/scripts/docproc.c
@@ -500,6 +500,7 @@ static void parse_file(FILE *infile)
 
 int main(int argc, char *argv[])
 {
+	const char *subcommand, *filename;
 	FILE * infile;
 	int i;
 
@@ -513,15 +514,19 @@ int main(int argc, char *argv[])
 		usage();
 		exit(1);
 	}
+
+	subcommand = argv[1];
+	filename = argv[2];
+
 	/* Open file, exit on error */
-	infile = fopen(argv[2], "r");
+	infile = fopen(filename, "r");
 	if (infile == NULL) {
 		fprintf(stderr, "docproc: ");
-		perror(argv[2]);
+		perror(filename);
 		exit(2);
 	}
 
-	if (strcmp("doc", argv[1]) == 0) {
+	if (strcmp("doc", subcommand) == 0) {
 		/* Need to do this in two passes.
 		 * First pass is used to collect all symbols exported
 		 * in the various files;
@@ -557,10 +562,10 @@ int main(int argc, char *argv[])
 			fprintf(stderr, "Warning: didn't use docs for %s\n",
 				all_list[i]);
 		}
-	} else if (strcmp("depend", argv[1]) == 0) {
+	} else if (strcmp("depend", subcommand) == 0) {
 		/* Create first part of dependency chain
 		 * file.tmpl */
-		printf("%s\t", argv[2]);
+		printf("%s\t", filename);
 		defaultline       = noaction;
 		internalfunctions = adddep;
 		externalfunctions = adddep;
@@ -571,7 +576,7 @@ int main(int argc, char *argv[])
 		parse_file(infile);
 		printf("\n");
 	} else {
-		fprintf(stderr, "Unknown option: %s\n", argv[1]);
+		fprintf(stderr, "Unknown option: %s\n", subcommand);
 		exit(1);
 	}
 	fclose(infile);
-- 
2.1.4

[PATCH 5/9] docproc: reduce unnecessary indentation

[Jani Nikula]

Improves clarity. No functional changes.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/docproc.c | 93 ++++++++++++++++++++++++++++---------------------------
 1 file changed, 47 insertions(+), 46 deletions(-)

diff --git a/scripts/docproc.c b/scripts/docproc.c
index 48abc01a920e..fb195f0ed0ef 100644
--- a/scripts/docproc.c
+++ b/scripts/docproc.c
@@ -445,52 +445,53 @@ static void parse_file(FILE *infile)
 	char line[MAXLINESZ];
 	char * s;
 	while (fgets(line, MAXLINESZ, infile)) {
-		if (line[0] == '!') {
-			s = line + 2;
-			switch (line[1]) {
-				case 'E':
-					while (*s && !isspace(*s)) s++;
-					*s = '\0';
-					externalfunctions(line+2);
-					break;
-				case 'I':
-					while (*s && !isspace(*s)) s++;
-					*s = '\0';
-					internalfunctions(line+2);
-					break;
-				case 'D':
-					while (*s && !isspace(*s)) s++;
-					*s = '\0';
-					symbolsonly(line+2);
-					break;
-				case 'F':
-					/* filename */
-					while (*s && !isspace(*s)) s++;
-					*s++ = '\0';
-					/* function names */
-					while (isspace(*s))
-						s++;
-					singlefunctions(line +2, s);
-					break;
-				case 'P':
-					/* filename */
-					while (*s && !isspace(*s)) s++;
-					*s++ = '\0';
-					/* DOC: section name */
-					while (isspace(*s))
-						s++;
-					docsection(line + 2, s);
-					break;
-				case 'C':
-					while (*s && !isspace(*s)) s++;
-					*s = '\0';
-					if (findall)
-						findall(line+2);
-					break;
-				default:
-					defaultline(line);
-			}
-		} else {
+		if (line[0] != '!') {
+			defaultline(line);
+			continue;
+		}
+
+		s = line + 2;
+		switch (line[1]) {
+		case 'E':
+			while (*s && !isspace(*s)) s++;
+			*s = '\0';
+			externalfunctions(line+2);
+			break;
+		case 'I':
+			while (*s && !isspace(*s)) s++;
+			*s = '\0';
+			internalfunctions(line+2);
+			break;
+		case 'D':
+			while (*s && !isspace(*s)) s++;
+			*s = '\0';
+			symbolsonly(line+2);
+			break;
+		case 'F':
+			/* filename */
+			while (*s && !isspace(*s)) s++;
+			*s++ = '\0';
+			/* function names */
+			while (isspace(*s))
+				s++;
+			singlefunctions(line +2, s);
+			break;
+		case 'P':
+			/* filename */
+			while (*s && !isspace(*s)) s++;
+			*s++ = '\0';
+			/* DOC: section name */
+			while (isspace(*s))
+				s++;
+			docsection(line + 2, s);
+			break;
+		case 'C':
+			while (*s && !isspace(*s)) s++;
+			*s = '\0';
+			if (findall)
+				findall(line+2);
+			break;
+		default:
 			defaultline(line);
 		}
 	}
-- 
2.1.4

[PATCH 6/9] docproc: abstract docproc directive detection

[Jani Nikula]

Helps follow-up work. No functional changes.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/docproc.c | 30 ++++++++++++++++++++----------
 1 file changed, 20 insertions(+), 10 deletions(-)

diff --git a/scripts/docproc.c b/scripts/docproc.c
index fb195f0ed0ef..bc900310b431 100644
--- a/scripts/docproc.c
+++ b/scripts/docproc.c
@@ -430,6 +430,15 @@ static void find_all_symbols(char *filename)
 	}
 }
 
+/* Return pointer to directive content, or NULL if not a directive. */
+static char *is_directive(char *line)
+{
+	if (line[0] == '!')
+		return line + 1;
+
+	return NULL;
+}
+
 /*
  * Parse file, calling action specific functions for:
  * 1) Lines containing !E
@@ -443,29 +452,30 @@ static void find_all_symbols(char *filename)
 static void parse_file(FILE *infile)
 {
 	char line[MAXLINESZ];
-	char * s;
+	char *p, *s;
 	while (fgets(line, MAXLINESZ, infile)) {
-		if (line[0] != '!') {
+		p = is_directive(line);
+		if (!p) {
 			defaultline(line);
 			continue;
 		}
 
-		s = line + 2;
-		switch (line[1]) {
+		s = p + 1;
+		switch (*p++) {
 		case 'E':
 			while (*s && !isspace(*s)) s++;
 			*s = '\0';
-			externalfunctions(line+2);
+			externalfunctions(p);
 			break;
 		case 'I':
 			while (*s && !isspace(*s)) s++;
 			*s = '\0';
-			internalfunctions(line+2);
+			internalfunctions(p);
 			break;
 		case 'D':
 			while (*s && !isspace(*s)) s++;
 			*s = '\0';
-			symbolsonly(line+2);
+			symbolsonly(p);
 			break;
 		case 'F':
 			/* filename */
@@ -474,7 +484,7 @@ static void parse_file(FILE *infile)
 			/* function names */
 			while (isspace(*s))
 				s++;
-			singlefunctions(line +2, s);
+			singlefunctions(p, s);
 			break;
 		case 'P':
 			/* filename */
@@ -483,13 +493,13 @@ static void parse_file(FILE *infile)
 			/* DOC: section name */
 			while (isspace(*s))
 				s++;
-			docsection(line + 2, s);
+			docsection(p, s);
 			break;
 		case 'C':
 			while (*s && !isspace(*s)) s++;
 			*s = '\0';
 			if (findall)
-				findall(line+2);
+				findall(p);
 			break;
 		default:
 			defaultline(line);
-- 
2.1.4

[PATCH 7/9] docproc: abstract terminating lines at first space

[Jani Nikula]

Cleaner code. Also fixes a bug when F or P directives didn't in fact
have space.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/docproc.c | 34 +++++++++++++++++++++-------------
 1 file changed, 21 insertions(+), 13 deletions(-)

diff --git a/scripts/docproc.c b/scripts/docproc.c
index bc900310b431..a933e054402d 100644
--- a/scripts/docproc.c
+++ b/scripts/docproc.c
@@ -430,6 +430,21 @@ static void find_all_symbols(char *filename)
 	}
 }
 
+/*
+ * Terminate s at first space, if any. If there was a space, return pointer to
+ * the character after that. Otherwise, return pointer to the terminating NUL.
+ */
+static char *chomp(char *s)
+{
+	while (*s && !isspace(*s))
+		s++;
+
+	if (*s)
+		*s++ = '\0';
+
+	return s;
+}
+
 /* Return pointer to directive content, or NULL if not a directive. */
 static char *is_directive(char *line)
 {
@@ -460,27 +475,22 @@ static void parse_file(FILE *infile)
 			continue;
 		}
 
-		s = p + 1;
 		switch (*p++) {
 		case 'E':
-			while (*s && !isspace(*s)) s++;
-			*s = '\0';
+			chomp(p);
 			externalfunctions(p);
 			break;
 		case 'I':
-			while (*s && !isspace(*s)) s++;
-			*s = '\0';
+			chomp(p);
 			internalfunctions(p);
 			break;
 		case 'D':
-			while (*s && !isspace(*s)) s++;
-			*s = '\0';
+			chomp(p);
 			symbolsonly(p);
 			break;
 		case 'F':
 			/* filename */
-			while (*s && !isspace(*s)) s++;
-			*s++ = '\0';
+			s = chomp(p);
 			/* function names */
 			while (isspace(*s))
 				s++;
@@ -488,16 +498,14 @@ static void parse_file(FILE *infile)
 			break;
 		case 'P':
 			/* filename */
-			while (*s && !isspace(*s)) s++;
-			*s++ = '\0';
+			s = chomp(p);
 			/* DOC: section name */
 			while (isspace(*s))
 				s++;
 			docsection(p, s);
 			break;
 		case 'C':
-			while (*s && !isspace(*s)) s++;
-			*s = '\0';
+			chomp(p);
 			if (findall)
 				findall(p);
 			break;
-- 
2.1.4

[PATCH 8/9] docproc: add support for reStructuredText format via --rst option

[Jani Nikula]

Expect reStructuredText input and have kernel-doc produce
reStructuredText output with the new --rst option. Also add --docbook
option for completeness. If no option is given, default to
reStructuredText if the input file has ".rst" extension, DocBook
otherwise.

Directives for reStructuredText use .. ! instead of just !, to make them
reStructuredText comments.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/docproc.c | 90 ++++++++++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 80 insertions(+), 10 deletions(-)

diff --git a/scripts/docproc.c b/scripts/docproc.c
index a933e054402d..9babfd108d2e 100644
--- a/scripts/docproc.c
+++ b/scripts/docproc.c
@@ -42,6 +42,7 @@
 #include <unistd.h>
 #include <limits.h>
 #include <errno.h>
+#include <getopt.h>
 #include <sys/types.h>
 #include <sys/wait.h>
 
@@ -68,12 +69,23 @@ FILELINE * docsection;
 #define KERNELDOCPATH "scripts/"
 #define KERNELDOC     "kernel-doc"
 #define DOCBOOK       "-docbook"
+#define RST           "-rst"
 #define LIST          "-list"
 #define FUNCTION      "-function"
 #define NOFUNCTION    "-nofunction"
 #define NODOCSECTIONS "-no-doc-sections"
 #define SHOWNOTFOUND  "-show-not-found"
 
+enum file_format {
+	FORMAT_AUTO,
+	FORMAT_DOCBOOK,
+	FORMAT_RST,
+};
+
+static enum file_format file_format = FORMAT_AUTO;
+
+#define KERNELDOC_FORMAT	(file_format == FORMAT_RST ? RST : DOCBOOK)
+
 static char *srctree, *kernsrctree;
 
 static char **all_list = NULL;
@@ -95,7 +107,7 @@ static void consume_symbol(const char *sym)
 
 static void usage (void)
 {
-	fprintf(stderr, "Usage: docproc {doc|depend} file\n");
+	fprintf(stderr, "Usage: docproc [{--docbook|--rst}] {doc|depend} file\n");
 	fprintf(stderr, "Input is read from file.tmpl. Output is sent to stdout\n");
 	fprintf(stderr, "doc: frontend when generating kernel documentation\n");
 	fprintf(stderr, "depend: generate list of files referenced within file\n");
@@ -242,7 +254,7 @@ static void find_export_symbols(char * filename)
 /*
  * Document all external or internal functions in a file.
  * Call kernel-doc with following parameters:
- * kernel-doc -docbook -nofunction function_name1 filename
+ * kernel-doc [-docbook|-rst] -nofunction function_name1 filename
  * Function names are obtained from all the src files
  * by find_export_symbols.
  * intfunc uses -nofunction
@@ -263,7 +275,7 @@ static void docfunctions(char * filename, char * type)
 		exit(1);
 	}
 	vec[idx++] = KERNELDOC;
-	vec[idx++] = DOCBOOK;
+	vec[idx++] = KERNELDOC_FORMAT;
 	vec[idx++] = NODOCSECTIONS;
 	for (i=0; i < symfilecnt; i++) {
 		struct symfile * sym = &symfilelist[i];
@@ -275,7 +287,10 @@ static void docfunctions(char * filename, char * type)
 	}
 	vec[idx++]     = filename;
 	vec[idx] = NULL;
-	printf("<!-- %s -->\n", filename);
+	if (file_format == FORMAT_RST)
+		printf(".. %s\n", filename);
+	else
+		printf("<!-- %s -->\n", filename);
 	exec_kernel_doc(vec);
 	fflush(stdout);
 	free(vec);
@@ -294,7 +309,7 @@ static void singfunc(char * filename, char * line)
 	int i, idx = 0;
 	int startofsym = 1;
 	vec[idx++] = KERNELDOC;
-	vec[idx++] = DOCBOOK;
+	vec[idx++] = KERNELDOC_FORMAT;
 	vec[idx++] = SHOWNOTFOUND;
 
 	/* Split line up in individual parameters preceded by FUNCTION */
@@ -343,7 +358,7 @@ static void docsect(char *filename, char *line)
 	free(s);
 
 	vec[0] = KERNELDOC;
-	vec[1] = DOCBOOK;
+	vec[1] = KERNELDOC_FORMAT;
 	vec[2] = SHOWNOTFOUND;
 	vec[3] = FUNCTION;
 	vec[4] = line;
@@ -448,8 +463,10 @@ static char *chomp(char *s)
 /* Return pointer to directive content, or NULL if not a directive. */
 static char *is_directive(char *line)
 {
-	if (line[0] == '!')
+	if (file_format == FORMAT_DOCBOOK && line[0] == '!')
 		return line + 1;
+	else if (file_format == FORMAT_RST && !strncmp(line, ".. !", 4))
+		return line + 4;
 
 	return NULL;
 }
@@ -516,6 +533,22 @@ static void parse_file(FILE *infile)
 	fflush(stdout);
 }
 
+/*
+ * Is this a RestructuredText template?  Answer the question by seeing if its
+ * name ends in ".rst".
+ */
+static int is_rst(const char *file)
+{
+	char *dot = strrchr(file, '.');
+
+	return dot && !strcmp(dot + 1, "rst");
+}
+
+enum opts {
+	OPT_DOCBOOK,
+	OPT_RST,
+	OPT_HELP,
+};
 
 int main(int argc, char *argv[])
 {
@@ -529,13 +562,50 @@ int main(int argc, char *argv[])
 	kernsrctree = getenv("KBUILD_SRC");
 	if (!kernsrctree || !*kernsrctree)
 		kernsrctree = srctree;
-	if (argc != 3) {
+
+	for (;;) {
+		int c;
+		struct option opts[] = {
+			{ "docbook",	no_argument, NULL, OPT_DOCBOOK },
+			{ "rst",	no_argument, NULL, OPT_RST },
+			{ "help",	no_argument, NULL, OPT_HELP },
+			{}
+		};
+
+		c = getopt_long_only(argc, argv, "", opts, NULL);
+		if (c == -1)
+			break;
+
+		switch (c) {
+		case OPT_DOCBOOK:
+			file_format = FORMAT_DOCBOOK;
+			break;
+		case OPT_RST:
+			file_format = FORMAT_RST;
+			break;
+		case OPT_HELP:
+			usage();
+			return 0;
+		default:
+		case '?':
+			usage();
+			return 1;
+		}
+	}
+
+	argc -= optind;
+	argv += optind;
+
+	if (argc != 2) {
 		usage();
 		exit(1);
 	}
 
-	subcommand = argv[1];
-	filename = argv[2];
+	subcommand = argv[0];
+	filename = argv[1];
+
+	if (file_format == FORMAT_AUTO)
+		file_format = is_rst(filename) ? FORMAT_RST : FORMAT_DOCBOOK;
 
 	/* Open file, exit on error */
 	infile = fopen(filename, "r");
-- 
2.1.4

[PATCH 9/9] docproc: print a comment about autogeneration for rst output

[Jani Nikula]

Leave a hint to folks which file to edit instead.

Signed-off-by: Jani Nikula <jani.nikula@intel.com>
---
 scripts/docproc.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/scripts/docproc.c b/scripts/docproc.c
index 9babfd108d2e..0a12593b9041 100644
--- a/scripts/docproc.c
+++ b/scripts/docproc.c
@@ -45,6 +45,7 @@
 #include <getopt.h>
 #include <sys/types.h>
 #include <sys/wait.h>
+#include <time.h>
 
 /* exitstatus is used to keep track of any failing calls to kernel-doc,
  * but execution continues. */
@@ -616,6 +617,12 @@ int main(int argc, char *argv[])
 	}
 
 	if (strcmp("doc", subcommand) == 0) {
+		if (file_format == FORMAT_RST) {
+			time_t t = time(NULL);
+			printf(".. generated from %s by docproc %s\n",
+			       filename, ctime(&t));
+		}
+
 		/* Need to do this in two passes.
 		 * First pass is used to collect all symbols exported
 		 * in the various files;
-- 
2.1.4

Re: [PATCH 0/9] kernel-doc/docproc prep work for reStructuredText

[Jonathan Corbet]

On Thu, 12 May 2016 16:15:35 +0300
Jani Nikula <jani.nikula@intel.com> wrote:

> Jon, I was hoping we could consider nudging things forward a bit in the
> kernel-doc and docproc reStructuredText front already in 4.7. I know
> it's a bit close to the merge window, but this should not interfere with
> anything else, and some of it are just trivial cleanups that I've been
> carrying around locally.
> 
> Obviously this doesn't actually add anything that uses them yet, but I
> think it would be helpful to have some common base in to ease
> collaboration.

Yeah, I think I agree; it's really time to make something actually happen
here.  I'll push these into linux-next and, almost certainly, into 4.7 as
well.  Thanks for putting the series together.

jon