Re: PDF book of unreleased pages (was: strncpy clarify result may not be null terminated)

[Alejandro Colomar <alx@kernel.org>]

[-- Attachment #1: Type: text/plain, Size: 3480 bytes --]

Hi Deri!

On Sun, Nov 19, 2023 at 12:22:56AM +0000, Deri wrote:
> Hi Alex,
> 
> I assume you are thinking this will make production more efficient (quicker). 

Not necessarily.  The main reason is that I want to be able to inspect
and understand every little step of the groff pipeline.  See for example
how I build a pdf from a single page:

	$ touch man2/membarrier.2
	$ make build-pdf   
	PRECONV	.tmp/man/man2/membarrier.2.tbl
	TBL	.tmp/man/man2/membarrier.2.eqn
	EQN	.tmp/man/man2/membarrier.2.pdf.troff
	TROFF	.tmp/man/man2/membarrier.2.pdf.set
	GROPDF	.tmp/man/man2/membarrier.2.pdf

That helps debug the pipeline, and also learn about it.

If that helps parallelize some tasks, then that'll be welcome.

> The time saved would be absolutely minimal. It is obvious that to produce a 
> pdf containing all the man pages then all the man pages have to be consumed by 
> groff, not just the page which has changed.

But do you need to run the entire pipeline, or can you reuse most of it?
I can process in parallel much faster, with `make -jN ...`.  I guess
the .pdf.troff files can be reused; maybe even the .pdf.set ones?

Could you change the script at least to produce intermediary files as in
the pipeline shown above?  As many as possible would be excellent.

> On my system this takes about 18 
> seconds to produce the 2800+ pages of the book. Of this, a quarter of a second 
> is consumed by the "magic" part of the script, the rest of the 18 seconds is 
> consumed by calls to groff and gropdf.

But how much of that work needs to be on a single process?  I bought a
new CPU with 24 cores.  Gotta use them all  :D

> So any splitting of the perl script is 
> only going to have an effect on the quarter of a second!
> 
> I don't understand why the perl script can't be included in your make file as 
> part of build-pdf target.

It can.  I just prefer to be strict about the Makefile having "one rule
per each file", while currently the script generates 4 files (T, two
.Z's, and the .pdf).

> Presumably it would be dependent on running after 
> the scripts which add the revision label and date to each man page.

I only set the revision and date on dist tarballs.  For the git HEAD
book, I'd keep the (unreleased) version and (date).  So, no worries
there.

> 
> > Since I don't understand Perl, and don't know much of gropdf(1) either,
> > I need help.
> > 
> > Maybe Deri or Branden can help with that.  If anyone else understands it
> > and can also help, that's very welcome too!
> 
> You are probably better placed to add the necessaries to your makefile. You 
> would then just need to remember to make build-pdf any time you alter one of 
> the source man pages. Since you are manually running my script to produce the 
> pdf, it should not be difficult to automate it in a makefile.
> 
> > Then I could install a hook in my server that runs
> > 
> > 	$ make build-pdf docdir=/srv/www/...
> 
> And wait 18s each time the hook is actioned!! Or, set the build to place the 
> generated pdf somewhere in /srv/www/... and include the build in your normal 
> workflow when a man page is changed.

Hmm.  I still hope some of it can be parallelized, but 18s could be
reasonable, if the server does that in the background after pushing.
My old raspberry pi would burn, but the new computer should handle that
just fine.

Cheers,
Alex

-- 
<https://www.alejandro-colomar.es/>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]