[PATCH] docbook: Added support for generating man files

[PATCH] docbook: Added support for generating man files

[Sam Ravnborg]

Hi Linus - please apply. Only docbook relevant changes.

Originally by Michael Still <mikal@stillhq.com>
 
This patch adds two new targets to the docbook makefile -- mandocs, and
installmandocs. The targets require two new perl scripts in the scripts/
directory, but in return we get a series of man pages for kernel
functions, which are installed in man section 9. This is a good thing, as
many programmers expect documentation to be available with man, and
hunting through various PS or PDF documents to find the documentation for
the function you want can be quite frustrating.

The man pages are just extracted from the various existing DocBook SGML
documents, which are generated by kernel-doc. You also need to have
docbook2man installed on your machine.

Please note the formatting is not perfect, but I will tweak
other stuff later with further patches -- this is just an initial
implementation.

Sample output (HTMLised) can be found at
http://www.stillhq.com/linux/mandocs/2.5.68/ and
http://www.stillhq.com/linux/mandocs/2.5.70/

Re: [PATCH] docbook: Added support for generating man files

[Sam Ravnborg]

Hi Linus - please apply. Only docbook relevant changes [with patch this time].

Originally by Michael Still <mikal@stillhq.com>
 
This patch adds two new targets to the docbook makefile -- mandocs, and
installmandocs. The targets require two new perl scripts in the scripts/
directory, but in return we get a series of man pages for kernel
functions, which are installed in man section 9. This is a good thing, as
many programmers expect documentation to be available with man, and
hunting through various PS or PDF documents to find the documentation for
the function you want can be quite frustrating.

The man pages are just extracted from the various existing DocBook SGML
documents, which are generated by kernel-doc. You also need to have
docbook2man installed on your machine.

Please note the formatting is not perfect, but I will tweak
other stuff later with further patches -- this is just an initial
implementation.

Sample output (HTMLised) can be found at
http://www.stillhq.com/linux/mandocs/2.5.68/ and
http://www.stillhq.com/linux/mandocs/2.5.70/



 CREDITS                        |    8 ++
 Documentation/DocBook/Makefile |   30 +++++++++-
 Makefile                       |    4 -
 scripts/makeman                |   46 ++++++++++++++++
 scripts/split-man              |  112 +++++++++++++++++++++++++++++++++++++++++
 5 files changed, 193 insertions(+), 7 deletions(-)

diff -Nru a/CREDITS b/CREDITS
--- a/CREDITS	Fri Jul 18 23:21:44 2003
+++ b/CREDITS	Fri Jul 18 23:21:44 2003
@@ -2981,6 +2981,14 @@
 S: Sunyvale, CA 94086
 S: USA
 
+N: Michael Still
+E: mikal@stillhq.com
+W: http://www.stillhq.com
+D: Various janitorial patches
+D: mandocs and mandocs_install build targets
+S: (Email me and ask)
+S: Australia
+
 N: Henrik Storner
 E: storner@image.dk
 W: http://www.image.dk/~storner/
diff -Nru a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile
--- a/Documentation/DocBook/Makefile	Fri Jul 18 23:21:44 2003
+++ b/Documentation/DocBook/Makefile	Fri Jul 18 23:21:44 2003
@@ -20,10 +20,11 @@
 # file.tmpl --> file.sgml +--> file.ps  (psdocs)
 #                         +--> file.pdf  (pdfdocs)
 #                         +--> DIR=file  (htmldocs)
+#                         +--> man/      (mandocs)
 
 ###
 # The targets that may be used.
-.PHONY:	sgmldocs psdocs pdfdocs htmldocs clean mrproper
+.PHONY:	sgmldocs psdocs pdfdocs htmldocs mandocs installmandocs
 
 BOOKS := $(addprefix $(obj)/,$(DOCBOOKS))
 sgmldocs: $(BOOKS)
@@ -37,10 +38,18 @@
 HTML := $(patsubst %.sgml, %.html, $(BOOKS))
 htmldocs: $(HTML)
 
+MAN := $(patsubst %.sgml, %.9, $(BOOKS))
+mandocs: $(MAN)
+
+installmandocs: mandocs
+	$(MAKEMAN) install Documentation/DocBook/man
+
 ###
 #External programs used
 KERNELDOC = scripts/kernel-doc
 DOCPROC   = scripts/docproc
+SPLITMAN  = $(PERL) $(srctree)/scripts/split-man
+MAKEMAN   = $(PERL) $(srctree)/scripts/makeman
 
 ###
 # DOCPROC is used for two purposes:
@@ -128,6 +137,13 @@
             cp $(PNG-$(basename $(notdir $@))) $(patsubst %.html,%,$@); fi
 
 ###
+# Rule to generate man files - output is placed in the man subdirectory
+
+%.9:	%.sgml
+	$(SPLITMAN) $< $(objtree)/Documentation/DocBook/man "$(VERSION).$(PATCHLEVEL).$(SUBLEVEL)"
+	$(MAKEMAN) convert $(objtree)/Documentation/DocBook/man $<
+
+###
 # Rules to generate postscripts and PNG imgages from .fig format files
 quiet_cmd_fig2eps = FIG2EPS $@
       cmd_fig2eps = fig2dev -Leps $< $@
@@ -157,10 +173,10 @@
 # Help targets as used by the top-level makefile
 dochelp:
 	@echo  '  Linux kernel internal documentation in different formats:'
-	@echo  '  sgmldocs (SGML), psdocs (Postscript), pdfdocs (PDF), htmldocs (HTML)'
+	@echo  '  sgmldocs (SGML), psdocs (Postscript), pdfdocs (PDF)'
+	@echo  '  htmldocs (HTML), mandocs (man pages, use installmandocs to install)'
 
 ###
-# clean and mrproper as used by the top-level makefile 
 # Temporary files left by various tools
 clean-files := $(DOCBOOKS) \
 	$(patsubst %.sgml, %.dvi,  $(DOCBOOKS)) \
@@ -171,10 +187,14 @@
 	$(patsubst %.sgml, %.ps,   $(DOCBOOKS)) \
 	$(patsubst %.sgml, %.pdf,  $(DOCBOOKS)) \
 	$(patsubst %.sgml, %.html, $(DOCBOOKS)) \
-	$(patsubst %.fig,%.eps, $(IMG-parportbook)) \
-	$(patsubst %.fig,%.png, $(IMG-parportbook)) \
+	$(patsubst %.sgml, %.9,    $(DOCBOOKS)) \
+	$(patsubst %.fig,%.eps,    $(IMG-parportbook)) \
+	$(patsubst %.fig,%.png,    $(IMG-parportbook)) \
 	$(C-procfs-example)
 
 ifneq ($(wildcard $(patsubst %.html,%,$(HTML))),)
 clean-rule := rm -rf $(wildcard $(patsubst %.html,%,$(HTML)))
 endif
+
+#man put files in man subdir - traverse down
+subdir- := man/
diff -Nru a/Makefile b/Makefile
--- a/Makefile	Fri Jul 18 23:21:44 2003
+++ b/Makefile	Fri Jul 18 23:21:44 2003
@@ -236,7 +236,7 @@
 noconfig_targets := xconfig gconfig menuconfig config oldconfig randconfig \
 		    defconfig allyesconfig allnoconfig allmodconfig \
 		    clean mrproper distclean rpm \
-		    help tags TAGS cscope sgmldocs psdocs pdfdocs htmldocs \
+		    help tags TAGS cscope %docs \
 		    checkconfig checkhelp checkincludes
 
 RCS_FIND_IGNORE := \( -name SCCS -o -name BitKeeper -o -name .svn -o -name CVS \) -prune -o
@@ -830,7 +830,7 @@
 
 # Documentation targets
 # ---------------------------------------------------------------------------
-sgmldocs psdocs pdfdocs htmldocs: scripts/docproc FORCE
+%docs: scripts/docproc FORCE
 	$(Q)$(MAKE) $(build)=Documentation/DocBook $@
 
 # Scripts to check various things for consistency
diff -Nru a/scripts/makeman b/scripts/makeman
--- /dev/null	Wed Dec 31 16:00:00 1969
+++ b/scripts/makeman	Fri Jul 18 23:21:44 2003
@@ -0,0 +1,46 @@
+#!/usr/bin/perl
+
+use strict;
+
+## Copyright (C) Michael Still (mikal@stillhq.com)
+## Released under the terms of the GNU GPL
+##
+## A script to make or install the manpages extracted by split-man
+##
+## Arguements: $1 -- the word "convert" or "install"
+##             $2 -- the directory containing the SGML files for the manpages
+##             $3 -- the filename which contained the sgmldoc output
+##                     (I need this so I know which manpages to convert)
+
+my($LISTING);
+
+if($ARGV[0] eq ""){
+  die "Usage: makeman [convert | install] <dir> <file>\n";
+}
+
+if( ! -d "$ARGV[1]" ){
+  die "Output directory \"$ARGV[1]\" does not exist\n";
+}
+
+if($ARGV[0] eq "convert"){
+  open LISTING, "grep \"<refentrytitle>\" $ARGV[2] |";
+  while(<LISTING>){
+    s/<\/.*$//;
+    s/^.*>//;
+    s/\.sgml//;
+    s/struct //;
+    s/typedef //;
+
+    chomp;
+    print "Processing $_\n";
+    system("cd $ARGV[1]; docbook2man $_.sgml; gzip -f $_.9\n");
+  }
+}
+elsif($ARGV[0] eq "install"){
+  system("mkdir -p /usr/local/man/man9/; install $ARGV[1]/*.9.gz /usr/local/man/man9/");
+}
+else{
+  die "Usage: makeman [convert | install] <dir> <file>\n";
+}
+
+print "Done\n";
diff -Nru a/scripts/split-man b/scripts/split-man
--- /dev/null	Wed Dec 31 16:00:00 1969
+++ b/scripts/split-man	Fri Jul 18 23:21:44 2003
@@ -0,0 +1,112 @@
+#!/usr/bin/perl
+
+use strict;
+
+## Copyright (C) Michael Still (mikal@stillhq.com)
+## Released under the terms of the GNU GPL
+##
+## Hoon through the specified DocBook SGML file, and split out the
+## man pages. These can then be processed into groff format, and
+## installed if desired...
+##
+## Arguements: $1 -- the name of the sgml file
+##             $2 -- the directory to put the generated SGML files in
+##             $3 -- kernel version
+
+my($SGML, $REF, $front, $refdata, $mode, $filename);
+
+if(($ARGV[0] eq "") || ($ARGV[1] eq "") || ($ARGV[2] eq "")){
+  die "Usage: split-man <sgml file> <output dir> <kernel version>\n";
+}
+
+open SGML, "< $ARGV[0]" or die "Could not open input file \"$ARGV[0]\"\n";
+if( ! -d "$ARGV[1]" ){
+  die "Output directory \"$ARGV[1]\" does not exist\n";
+}
+
+# Possible modes:
+#   0: Looking for input I care about
+#   1: Inside book front matter
+#   2: Inside a refentry
+#   3: Inside a refentry, and we know the filename
+
+$mode = 0;
+$refdata = "";
+$front = "";
+while(<SGML>){
+  # Starting modes
+  if(/<legalnotice>/){
+    $mode = 1;
+  }
+  elsif(/<refentry>/){
+    $mode = 2;
+  }
+  elsif(/<refentrytitle><phrase[^>]*>([^<]*)<.*$/){
+    $mode = 3;
+    $filename = $1;
+
+    $filename =~ s/struct //;
+    $filename =~ s/typedef //;
+
+    print "Found manpage for $filename\n";
+    open REF, "> $ARGV[1]/$filename.sgml" or
+      die "Couldn't open output file \"$ARGV[1]/$filename.sgml\": $!\n";
+    print REF "<!DOCTYPE refentry PUBLIC \"-//Davenport//DTD DocBook V3.0//EN\">\n\n";
+    print REF "$refdata";
+    $refdata = "";
+  }
+
+  # Extraction
+  if($mode == 1){
+    $front = "$front$_";
+  }
+  elsif($mode == 2){
+    $refdata = "$refdata$_";
+  }
+  elsif($mode == 3){
+    # There are some fixups which need to be applied
+    if(/<\/refmeta>/){
+      print REF "<manvolnum>9</manvolnum>\n";
+    }
+    if(/<\/refentry>/){
+      $front =~ s/<legalnotice>//;
+      $front =~ s/<\/legalnotice>//;
+      print REF <<EOF;
+<refsect1><title>About this document</title>
+$front
+<para>
+If you have comments on the formatting of this manpage, then please contact
+Michael Still (mikal\@stillhq.com).
+</para>
+
+<para>
+This documentation was generated with kernel version $ARGV[2].
+</para>
+</refsect1>
+EOF
+    }
+
+    # For some reason, we title the synopsis twice in the main DocBook
+    if(! /<title>Synopsis<\/title>/){
+      if(/<refentrytitle>/){
+	s/struct //;
+	s/typedef //;
+      }
+
+      print REF "$_";
+    }
+  }
+
+  # Ending modes
+  if(/<\/legalnotice>/){
+    $mode = 0;
+  }
+  elsif(/<\/refentry>/){
+    $mode = 0;
+    close REF;
+  }
+}
+
+# And make sure we don't process this unnessesarily
+$ARGV[0] =~ s/\.sgml/.9/;
+`touch $ARGV[0]`;

Re: [PATCH] docbook: Added support for generating man files

[Alan Cox]

On Gwe, 2003-07-18 at 22:23, Sam Ravnborg wrote:
> Hi Linus - please apply. Only docbook relevant changes [with patch this time].
> 
> Originally by Michael Still <mikal@stillhq.com>
>  
> This patch adds two new targets to the docbook makefile -- mandocs, and

IS there any chance it could incorporate the GPL by a slightly smaller
reference or even a link for the HTML one, it looks great except that
90% of the manual page is a GPL each time 8)

Re: [PATCH] docbook: Added support for generating man files

[Michael Still]

On 18 Jul 2003, Alan Cox wrote:

Hey,

thanks to Sam, Andries and yourself for showing an interest in the patch.

> On Gwe, 2003-07-18 at 22:23, Sam Ravnborg wrote:
> > Hi Linus - please apply. Only docbook relevant changes [with patch this time].
> > 
> > Originally by Michael Still <mikal@stillhq.com>
> >  
> > This patch adds two new targets to the docbook makefile -- mandocs, and
> 
> IS there any chance it could incorporate the GPL by a slightly smaller
> reference or even a link for the HTML one, it looks great except that
> 90% of the manual page is a GPL each time 8)

Well, the only part which comes from this patch is:

<para>
If you have comments on the formatting of this manpage, then please contact
Michael Still (mikal\@stillhq.com).
</para>

<para>
This documentation was generated with kernel version $ARGV[2].
</para>

I can shorten this if people would like.

The GPL bit people have commented on is actually extracted from the front
matter of the SGML file being converted inside
<legalnotice></legalnotice>. Therefore, if the SGML kerneldoc output we
already have includes the GPL, then so does the man page. I have not
imposed new license conditions on the documentation.

If people are comfortable with dropping the legal notice, or perhaps 
inserting a line saying "refer to file X for the license on this 
documentation", then I'll do that and send a new patch.

Cheers,
Mikal

-- 

Michael Still (mikal@stillhq.com) | Stage 1: Steal underpants
http://www.stillhq.com            | Stage 2: ????
UTC + 10                          | Stage 3: Profit

Re: [PATCH] docbook: Added support for generating man files

[Andries Brouwer]

On Sat, Jul 19, 2003 at 08:36:27AM +1000, Michael Still wrote:

> > IS there any chance it could incorporate the GPL by a slightly smaller
> > reference or even a link for the HTML one, it looks great except that
> > 90% of the manual page is a GPL each time 8)

> Well, the only part which comes from this patch is:
> ... Michael Still (mikal\@stillhq.com) ...
> ... This documentation was generated with kernel version $ARGV[2]. ...
> I can shorten this if people would like.

Commenting on the man page (nroff) version:

Please put the mikal\@stillhq.com in a comment.
Please leave the kernel version.

> The GPL bit people have commented on is actually extracted from the front
> matter of the SGML file being converted inside
> <legalnotice></legalnotice>. Therefore, if the SGML kerneldoc output we
> already have includes the GPL, then so does the man page. I have not
> imposed new license conditions on the documentation.
> 
> If people are comfortable with dropping the legal notice, or perhaps 
> inserting a line saying "refer to file X for the license on this 
> documentation", then I'll do that and send a new patch.

Please put all legalities in comments - behind .\" we do not have to read
them, but they are there if anyone cares.

Andries

Re: [PATCH] docbook: Added support for generating man files

[Michael Still]

On Sat, 19 Jul 2003, Andries Brouwer wrote:

> Please put all legalities in comments - behind .\" we do not have to read
> them, but they are there if anyone cares.

Ok. After having done some more poking, I can't see a way of doing this 
with docbook2man -- there doesn't appear to be any way of emitting 
comments. There strike me as two options here: write a script to convert 
SGML to man pages, or perhaps just insert a single sentence into the 
"About this Document" section which explains where to look for copyright 
information.

Any thoughts?

Cheers,
Mikal

-- 

Michael Still (mikal@stillhq.com) | Stage 1: Steal underpants
http://www.stillhq.com            | Stage 2: ????
UTC + 10                          | Stage 3: Profit

Re: [PATCH] docbook: Added support for generating man files

[Andries Brouwer]

On Sat, Jul 19, 2003 at 12:47:31PM +1000, Michael Still wrote:
> On Sat, 19 Jul 2003, Andries Brouwer wrote:
> 
> > Please put all legalities in comments - behind .\" we do not have to read
> > them, but they are there if anyone cares.
> 
> Ok. After having done some more poking, I can't see a way of doing this 
> with docbook2man -- there doesn't appear to be any way of emitting 
> comments. There strike me as two options here: write a script to convert 
> SGML to man pages, or perhaps just insert a single sentence into the 
> "About this Document" section which explains where to look for copyright 
> information.
> 
> Any thoughts?

You do this in two steps: first construct an sgml document, then invoke
docbook2man.

So, first you have to figure out what the proper docbook markup is
for such material. According to me it is <docinfo>...</docinfo>,
to be placed inside <refentry> before <refmeta>.
So you do that - example script fragments appended below.

Next, the docbook2man stage. My presently installed docbook2man says

# IGNORE.
sgml('<DOCINFO>', sub { push_output('nul'); });
sgml('</DOCINFO>', sub { pop_output(); });

That is, it does not produce any output for a docinfo section.
I consider that a bug in docbook2man - it should output nroff comments.
Probably this bug will be fixed sooner or later.
(I'll cc Steve Cheng who is listed as maintainer.)
In the meantime I do not worry too much about this missing copyright.

By the way, you start copying at <legalnotice>, but I consider that
the least interesting part. The fact that these docs were written
by Alan Cox is much more interesting. In other words, it might be
a good idea to enlarge this <docinfo> section a bit.

Andries


In split-man:

  elsif($mode == 2){
    if(/<refmeta>/){
      $refdata = "$refdata\n<docinfo>\n$front\n$blurb\n</docinfo>\n"
    }
    $refdata = "$refdata$_";
  }

Here $front is the original one - do not delete <legalnotice>.
And $blurb is your text:

  <authorblurb><para>
    If you have comments on the formatting of this manpage, then please contact
    Michael Still (mikal\@stillhq.com).
  </para></authorblurb>

Re: [PATCH] docbook: Added support for generating man files

[Michael Still]

On Sat, 19 Jul 2003, Andries Brouwer wrote:

> On Sat, Jul 19, 2003 at 12:47:31PM +1000, Michael Still wrote:
>
> You do this in two steps: first construct an sgml document, then invoke
> docbook2man.

Which is what I currently do.

> So, first you have to figure out what the proper docbook markup is
> for such material. According to me it is <docinfo>...</docinfo>,
> to be placed inside <refentry> before <refmeta>.
> So you do that - example script fragments appended below.

I tried this...

> Next, the docbook2man stage. My presently installed docbook2man says
> 
> # IGNORE.
> sgml('<DOCINFO>', sub { push_output('nul'); });
> sgml('</DOCINFO>', sub { pop_output(); });
> 
> That is, it does not produce any output for a docinfo section.

...and hit this. Another option is that we could ship a tweaked version of 
docbook2man with the kernel sources, and call that copy. That could then 
turn docinfo's into comments -- this was what I was getting at with my 
previous message.

> In the meantime I do not worry too much about this missing copyright.

Okie, that's fair enough.

> By the way, you start copying at <legalnotice>, but I consider that
> the least interesting part. The fact that these docs were written
> by Alan Cox is much more interesting. In other words, it might be
> a good idea to enlarge this <docinfo> section a bit.

Cool. I like your comments. I've got people coming over for dinner 
tonight, but I'll see what I can do about whipping up a new patch.

Cheers,
Mikal

-- 

Michael Still (mikal@stillhq.com) | Stage 1: Steal underpants
http://www.stillhq.com            | Stage 2: ????
UTC + 10                          | Stage 3: Profit

Re: [PATCH] docbook: Added support for generating man files

[Sam Ravnborg]

On Sat, Jul 19, 2003 at 05:25:32PM +1000, Michael Still wrote:
> 
> ...and hit this. Another option is that we could ship a tweaked version of 
> docbook2man with the kernel sources, and call that copy. That could then 
> turn docinfo's into comments -- this was what I was getting at with my 
> previous message.

No, we do want to start adding tailored programs as part of
the kernel source.

	Sam