man page rendering speed (was: Playground pager lsp(1))

["G. Branden Robinson" <g.branden.robinson@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 4511 bytes --]

[adding Colin Watson to CC to solicit his man(1) implementation
knowledge; adding the groff list as this sub-discussion is relevant to
its interests]

At 2023-04-07T09:36:10+0300, Eli Zaretskii wrote:
> > From: "G. Branden Robinson" <g.branden.robinson@gmail.com>
[re-running *roff when a viewing a man page and resizing the terminal]
> > Seems like it shouldn't be impossible to me, but what I imagine
> > would require a little reëngineering of man(1), perhaps to spawn a
> > little custom program to manage zcat/nroff pipeline it constructs.
> > This little program's sole job could be to be aware of this pipeline
> > and listen for SIGWINCH; if it happens, kill the rest of the
> > pipeline and reëxecute it.
> 
> This should be possible, but it flies in the face of the feature
> whereby formatted man pages are kept for future perusal, which is
> therefore faster:

You're referring to cat pages.  As far as I know, these are on their way
out if not already gone.  Colin Watson, who has maintained the man-db
implementation of man(1)[1] for something like 20 years, can speak more
authoritatively to this, but as I understand it, the advent of resizable
xterm windows started to kill the utility of cat pages decades ago and
the increasing importance of desktop environments accelerated their
demise.  If a cat page wasn't pre-rendered at the width of your
terminal, or for your terminal type[1], man pages were formatted from
scratch for you anyway.  You could of course cache pages at a variety of
widths (and for multiple terminal types), but doing so for any width
encountered was a space concern--or even a DoS vector if some
undergraduate rapscallion decided to try rendering every page on the
system at every terminal width from 1 to INT_MAX--in the years when man
page rendering time was also noticeable.

...which brings me to the other factor, of which I'm more confident: man
page rendering times are much lower than they were in Unix's early days.

On my system, all groff man pages but one render in between a tenth and
a fortieth of a second.  The really huge pages like groff(7),
groff_char(7), and groff_diff(7) are toward the upper end of this range,
because they are long, at ~20-25 U.S. letter pages when formatted for
PostScript or PDF, or have many large tables so the tbl(1) preprocessor
produces a lot of output.

The outlier is groff_mdoc(7) at just over one-third of a second.  It is
written in its own macro language, not man(7), and also a lengthy
document (31 U.S.  letter pages).  mdoc has always been slow; its
original implementers warned of this.  (I believe this is mainly due to
an aspect of its design: the typical mdoc(7) document has a large number
of recursive macro calls arising from a decision to help the document
author avoid having to start new control lines to call them.)

While not statistically rigorous, mainly because I didn't undertake a
large number of trials under various system loads, I attempted fair
measurements by (A) always running the 3 preprocessors pic(1), eqn(1),
and tbl(1) on _all_ input documents even though this is pointless most
of the time (only tbl(1) sees use more than rarely), and (B) formatting
both with and without operation of the output driver grotty(1) in the
pipeline, in case "cheating" by having groff(1) discard its standard
output stream artificially deflated the time consumption.  It appears
not to have.

The bottom line is that, even on BSD systems (where mdoc(7) is preferred
to man(7)), a user can expect a man page to render from *roff source in
less than, say, half a second in the worst case, and the median
GNU/Linux user can expect to start reading a man page "instantaneously":

  Human subjects need a minimum of about 0.1 second of visual experience
  or about .01 to .02 second of auditory experience to perceive
  duration; any shorter experiences are called instantaneous.
  -- Encyclopædia Britannica[2]

My findings are attached.

I'll respond to the "uninstall-info" topic in a separate subthread.

Regards,
Branden

[1] Once upon a time, Unix time-sharing systems had to support shell
    sessions originating from a wide variety of terminals; at Purdue, I
    never saw a real DEC VT in use (to my regret), but plenty of Zenith
    Z29s, Wyse 50s, Sun SPARC IPCs in console mode, and the occasional
    really retro Lear Siegler ADM-5.

[2] https://www.britannica.com/science/time-perception/Perceived-duration

[-- Attachment #1.2: TIMING --]
[-- Type: text/plain, Size: 4186 bytes --]

for m in $(find -name "*.[157]" | sort); do echo; echo $m; time ./test-groff -Ez -pet -mandoc $m; done

./contrib/chem/chem.1

real	0m0.039s
user	0m0.043s
sys	0m0.000s

./contrib/eqn2graph/eqn2graph.1

real	0m0.025s
user	0m0.028s
sys	0m0.000s

./contrib/gdiffmk/gdiffmk.1

real	0m0.026s
user	0m0.023s
sys	0m0.007s

./contrib/glilypond/glilypond.1

real	0m0.032s
user	0m0.036s
sys	0m0.000s

./contrib/gperl/gperl.1

real	0m0.028s
user	0m0.031s
sys	0m0.000s

./contrib/gpinyin/gpinyin.1

real	0m0.027s
user	0m0.028s
sys	0m0.002s

./contrib/grap2graph/grap2graph.1

real	0m0.025s
user	0m0.026s
sys	0m0.002s

./contrib/hdtbl/groff_hdtbl.7

real	0m0.035s
user	0m0.032s
sys	0m0.006s

./contrib/mm/groff_mm.7

real	0m0.087s
user	0m0.092s
sys	0m0.009s

./contrib/mm/groff_mmse.7

real	0m0.025s
user	0m0.028s
sys	0m0.000s

./contrib/mm/mmroff.1

real	0m0.024s
user	0m0.018s
sys	0m0.010s

./contrib/mom/groff_mom.7

real	0m0.058s
user	0m0.053s
sys	0m0.010s

./contrib/pdfmark/pdfroff.1

real	0m0.033s
user	0m0.036s
sys	0m0.000s

./contrib/pic2graph/pic2graph.1

real	0m0.026s
user	0m0.029s
sys	0m0.000s

./contrib/rfc1345/groff_rfc1345.7

real	0m0.026s
user	0m0.026s
sys	0m0.004s

./man/groff.7

real	0m0.099s
user	0m0.110s
sys	0m0.000s

./man/groff_char.7

real	0m0.086s
user	0m0.109s
sys	0m0.000s

./man/groff_diff.7

real	0m0.082s
user	0m0.081s
sys	0m0.010s

./man/groff_font.5

real	0m0.033s
user	0m0.037s
sys	0m0.000s

./man/groff_out.5

real	0m0.042s
user	0m0.041s
sys	0m0.005s

./man/groff_tmac.5

real	0m0.037s
user	0m0.035s
sys	0m0.006s

./man/roff.7

real	0m0.047s
user	0m0.052s
sys	0m0.000s

./src/devices/grodvi/grodvi.1

real	0m0.029s
user	0m0.030s
sys	0m0.002s

./src/devices/grohtml/grohtml.1

real	0m0.030s
user	0m0.029s
sys	0m0.004s

./src/devices/grolbp/grolbp.1

real	0m0.029s
user	0m0.027s
sys	0m0.006s

./src/devices/grolj4/grolj4.1

real	0m0.033s
user	0m0.036s
sys	0m0.000s

./src/devices/gropdf/gropdf.1

real	0m0.041s
user	0m0.045s
sys	0m0.000s

./src/devices/gropdf/pdfmom.1

real	0m0.025s
user	0m0.028s
sys	0m0.000s

./src/devices/grops/grops.1

real	0m0.045s
user	0m0.049s
sys	0m0.000s

./src/devices/grotty/grotty.1

real	0m0.031s
user	0m0.032s
sys	0m0.002s

./src/devices/xditview/gxditview.1

real	0m0.035s
user	0m0.036s
sys	0m0.002s

./src/preproc/eqn/eqn.1

real	0m0.047s
user	0m0.052s
sys	0m0.000s

./src/preproc/eqn/neqn.1

real	0m0.024s
user	0m0.025s
sys	0m0.002s

./src/preproc/grn/grn.1

real	0m0.036s
user	0m0.030s
sys	0m0.010s

./src/preproc/pic/pic.1

real	0m0.036s
user	0m0.040s
sys	0m0.000s

./src/preproc/preconv/preconv.1

real	0m0.028s
user	0m0.031s
sys	0m0.000s

./src/preproc/refer/refer.1

real	0m0.051s
user	0m0.047s
sys	0m0.008s

./src/preproc/soelim/soelim.1

real	0m0.026s
user	0m0.030s
sys	0m0.000s

./src/preproc/tbl/tbl.1

real	0m0.043s
user	0m0.047s
sys	0m0.002s

./src/roff/groff/groff.1

real	0m0.050s
user	0m0.053s
sys	0m0.002s

./src/roff/nroff/nroff.1

real	0m0.026s
user	0m0.025s
sys	0m0.004s

./src/roff/troff/troff.1

real	0m0.035s
user	0m0.037s
sys	0m0.002s

./src/utils/addftinfo/addftinfo.1

real	0m0.025s
user	0m0.028s
sys	0m0.000s

./src/utils/afmtodit/afmtodit.1

real	0m0.029s
user	0m0.030s
sys	0m0.002s

./src/utils/grog/grog.1

real	0m0.028s
user	0m0.028s
sys	0m0.004s

./src/utils/hpftodit/hpftodit.1

real	0m0.030s
user	0m0.033s
sys	0m0.000s

./src/utils/indxbib/indxbib.1

real	0m0.029s
user	0m0.026s
sys	0m0.006s

./src/utils/lkbib/lkbib.1

real	0m0.027s
user	0m0.030s
sys	0m0.000s

./src/utils/lookbib/lookbib.1

real	0m0.026s
user	0m0.028s
sys	0m0.002s

./src/utils/pfbtops/pfbtops.1

real	0m0.025s
user	0m0.017s
sys	0m0.011s

./src/utils/tfmtodit/tfmtodit.1

real	0m0.027s
user	0m0.026s
sys	0m0.004s

./src/utils/xtotroff/xtotroff.1

real	0m0.025s
user	0m0.022s
sys	0m0.006s

./tmac/groff_man.7

real	0m0.049s
user	0m0.043s
sys	0m0.012s

./tmac/groff_man_style.7

real	0m0.066s
user	0m0.070s
sys	0m0.004s

./tmac/groff_mdoc.7

real	0m0.379s
user	0m0.379s
sys	0m0.010s

./tmac/groff_me.7

real	0m0.044s
user	0m0.039s
sys	0m0.013s

./tmac/groff_ms.7

real	0m0.065s
user	0m0.060s
sys	0m0.013s

./tmac/groff_trace.7

real	0m0.027s
user	0m0.026s
sys	0m0.004s

./tmac/groff_www.7

real	0m0.035s
user	0m0.030s
sys	0m0.009s

[-- Attachment #1.3: TIMING2 --]
[-- Type: text/plain, Size: 4203 bytes --]

for m in $(find -name "*.[157]" | sort); do echo; echo $m; time ./test-groff -E -pet -mandoc -Tutf8 $m >/dev/null; done

./contrib/chem/chem.1

real	0m0.051s
user	0m0.062s
sys	0m0.008s

./contrib/eqn2graph/eqn2graph.1

real	0m0.018s
user	0m0.019s
sys	0m0.006s

./contrib/gdiffmk/gdiffmk.1

real	0m0.018s
user	0m0.026s
sys	0m0.000s

./contrib/glilypond/glilypond.1

real	0m0.027s
user	0m0.038s
sys	0m0.000s

./contrib/gperl/gperl.1

real	0m0.021s
user	0m0.021s
sys	0m0.009s

./contrib/gpinyin/gpinyin.1

real	0m0.020s
user	0m0.026s
sys	0m0.002s

./contrib/grap2graph/grap2graph.1

real	0m0.018s
user	0m0.025s
sys	0m0.000s

./contrib/hdtbl/groff_hdtbl.7

real	0m0.031s
user	0m0.044s
sys	0m0.002s

./contrib/mm/groff_mm.7

real	0m0.089s
user	0m0.129s
sys	0m0.004s

./contrib/mm/groff_mmse.7

real	0m0.018s
user	0m0.026s
sys	0m0.000s

./contrib/mm/mmroff.1

real	0m0.023s
user	0m0.029s
sys	0m0.002s

./contrib/mom/groff_mom.7

real	0m0.067s
user	0m0.093s
sys	0m0.000s

./contrib/pdfmark/pdfroff.1

real	0m0.033s
user	0m0.040s
sys	0m0.006s

./contrib/pic2graph/pic2graph.1

real	0m0.020s
user	0m0.022s
sys	0m0.007s

./contrib/rfc1345/groff_rfc1345.7

real	0m0.021s
user	0m0.028s
sys	0m0.001s

./man/groff.7

real	0m0.116s
user	0m0.169s
sys	0m0.000s

./man/groff_char.7

real	0m0.069s
user	0m0.111s
sys	0m0.000s

./man/groff_diff.7

real	0m0.093s
user	0m0.134s
sys	0m0.002s

./man/groff_font.5

real	0m0.029s
user	0m0.031s
sys	0m0.011s

./man/groff_out.5

real	0m0.042s
user	0m0.058s
sys	0m0.002s

./man/groff_tmac.5

real	0m0.034s
user	0m0.044s
sys	0m0.005s

./man/roff.7

real	0m0.049s
user	0m0.075s
sys	0m0.000s

./src/devices/grodvi/grodvi.1

real	0m0.023s
user	0m0.031s
sys	0m0.002s

./src/devices/grohtml/grohtml.1

real	0m0.025s
user	0m0.025s
sys	0m0.011s

./src/devices/grolbp/grolbp.1

real	0m0.022s
user	0m0.030s
sys	0m0.002s

./src/devices/grolj4/grolj4.1

real	0m0.027s
user	0m0.033s
sys	0m0.006s

./src/devices/gropdf/gropdf.1

real	0m0.046s
user	0m0.063s
sys	0m0.002s

./src/devices/gropdf/pdfmom.1

real	0m0.018s
user	0m0.021s
sys	0m0.004s

./src/devices/grops/grops.1

real	0m0.038s
user	0m0.055s
sys	0m0.000s

./src/devices/grotty/grotty.1

real	0m0.025s
user	0m0.033s
sys	0m0.004s

./src/devices/xditview/gxditview.1

real	0m0.026s
user	0m0.028s
sys	0m0.009s

./src/preproc/eqn/eqn.1

real	0m0.039s
user	0m0.055s
sys	0m0.000s

./src/preproc/eqn/neqn.1

real	0m0.018s
user	0m0.021s
sys	0m0.002s

./src/preproc/grn/grn.1

real	0m0.032s
user	0m0.042s
sys	0m0.004s

./src/preproc/pic/pic.1

real	0m0.032s
user	0m0.046s
sys	0m0.000s

./src/preproc/preconv/preconv.1

real	0m0.028s
user	0m0.039s
sys	0m0.001s

./src/preproc/refer/refer.1

real	0m0.050s
user	0m0.067s
sys	0m0.004s

./src/preproc/soelim/soelim.1

real	0m0.019s
user	0m0.028s
sys	0m0.000s

./src/preproc/tbl/tbl.1

real	0m0.040s
user	0m0.061s
sys	0m0.000s

./src/roff/groff/groff.1

real	0m0.048s
user	0m0.067s
sys	0m0.002s

./src/roff/nroff/nroff.1

real	0m0.020s
user	0m0.028s
sys	0m0.000s

./src/roff/troff/troff.1

real	0m0.032s
user	0m0.044s
sys	0m0.000s

./src/utils/addftinfo/addftinfo.1

real	0m0.019s
user	0m0.024s
sys	0m0.002s

./src/utils/afmtodit/afmtodit.1

real	0m0.023s
user	0m0.021s
sys	0m0.012s

./src/utils/grog/grog.1

real	0m0.026s
user	0m0.031s
sys	0m0.005s

./src/utils/hpftodit/hpftodit.1

real	0m0.026s
user	0m0.036s
sys	0m0.000s

./src/utils/indxbib/indxbib.1

real	0m0.021s
user	0m0.029s
sys	0m0.000s

./src/utils/lkbib/lkbib.1

real	0m0.019s
user	0m0.020s
sys	0m0.006s

./src/utils/lookbib/lookbib.1

real	0m0.019s
user	0m0.025s
sys	0m0.000s

./src/utils/pfbtops/pfbtops.1

real	0m0.019s
user	0m0.021s
sys	0m0.004s

./src/utils/tfmtodit/tfmtodit.1

real	0m0.023s
user	0m0.028s
sys	0m0.004s

./src/utils/xtotroff/xtotroff.1

real	0m0.020s
user	0m0.021s
sys	0m0.007s

./tmac/groff_man.7

real	0m0.044s
user	0m0.061s
sys	0m0.002s

./tmac/groff_man_style.7

real	0m0.068s
user	0m0.098s
sys	0m0.004s

./tmac/groff_mdoc.7

real	0m0.383s
user	0m0.418s
sys	0m0.006s

./tmac/groff_me.7

real	0m0.031s
user	0m0.033s
sys	0m0.013s

./tmac/groff_ms.7

real	0m0.059s
user	0m0.082s
sys	0m0.005s

./tmac/groff_trace.7

real	0m0.019s
user	0m0.023s
sys	0m0.005s

./tmac/groff_www.7

real	0m0.026s
user	0m0.036s
sys	0m0.002s

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]