docs: requirements.txt has stopped working again

docs: requirements.txt has stopped working again

[Akira Yokosawa]

Hi all,

Annoyingly, installing Sphinx 2.4.4 using requirements.txt of v6.8-rc1
ends up in a run-time error which looks similar to the one Vegard
reported in commit f4cac0f74658 ("Documentation: constrain alabaster
package to older versions").

The new error is from sphinxcontrib.applehelp which reads:

    Sphinx version error:
    The sphinxcontrib.applehelp extension used by this project needs
    at least Sphinx v5.0; it therefore cannot be built with this version.

Do we want to continue whack-a-mole update for Sphinx 2.4.4?

        Thanks, Akira

Re: docs: requirements.txt has stopped working again

[Vegard Nossum]

On 23/01/2024 05:14, Akira Yokosawa wrote:
> Hi all,
> 
> Annoyingly, installing Sphinx 2.4.4 using requirements.txt of v6.8-rc1
> ends up in a run-time error which looks similar to the one Vegard
> reported in commit f4cac0f74658 ("Documentation: constrain alabaster
> package to older versions").
> 
> The new error is from sphinxcontrib.applehelp which reads:
> 
>      Sphinx version error:
>      The sphinxcontrib.applehelp extension used by this project needs
>      at least Sphinx v5.0; it therefore cannot be built with this version.
> 
> Do we want to continue whack-a-mole update for Sphinx 2.4.4?
> 
>          Thanks, Akira

Can we have requirements(-latest).txt and requirements-2.4.4.txt?

The thing is, we are not using new features of Sphinx in the kernel
docs. We don't fundamentally require a new version in any of our .rst
files or markup, these aren't security issues being fixed or bugs that
we've run into, the problem is purely that we are using a software
ecosystem that apparently has no restraint when it comes to breaking
their users.

My issue with _requiring_ newer versions of Sphinx is the fact that they
have performance regressions: 4.3.2 takes 3x longer to run than 2.4.4 on
my laptop. You need to go all the way up to 7.x in order to back to
reasonable performance -- and that will probably be too new to support
most of the distro-packaged Sphinxes.

If we have two or more requirements*.txt files, we could just freeze ALL
the dependencies for 2.4.4 at versions that we know to work. That should
stop the whack-a-mole for that version. And then we don't have to force
everybody on to newer/slower versions.

I mean, I can also maintain my own requirements-2.4.4.txt on my local
filesystem but I think it's better to do the right thing out of the box
for all users, no? Maybe we should also add in warnings about the
known-slow Sphinx versions.


Vegard

Re: docs: requirements.txt has stopped working again

[Jani Nikula]

On Tue, 23 Jan 2024, Vegard Nossum <vegard.nossum@oracle.com> wrote:
> On 23/01/2024 05:14, Akira Yokosawa wrote:
>> Hi all,
>> 
>> Annoyingly, installing Sphinx 2.4.4 using requirements.txt of v6.8-rc1
>> ends up in a run-time error which looks similar to the one Vegard
>> reported in commit f4cac0f74658 ("Documentation: constrain alabaster
>> package to older versions").
>> 
>> The new error is from sphinxcontrib.applehelp which reads:
>> 
>>      Sphinx version error:
>>      The sphinxcontrib.applehelp extension used by this project needs
>>      at least Sphinx v5.0; it therefore cannot be built with this version.
>> 
>> Do we want to continue whack-a-mole update for Sphinx 2.4.4?
>> 
>>          Thanks, Akira
>
> Can we have requirements(-latest).txt and requirements-2.4.4.txt?
>
> The thing is, we are not using new features of Sphinx in the kernel
> docs. We don't fundamentally require a new version in any of our .rst
> files or markup, these aren't security issues being fixed or bugs that
> we've run into, the problem is purely that we are using a software
> ecosystem that apparently has no restraint when it comes to breaking
> their users.

We do use at least namespacing which was new to Sphinx 3.1. And we could
use more if we weren't so conservative about the versions.

> My issue with _requiring_ newer versions of Sphinx is the fact that they
> have performance regressions: 4.3.2 takes 3x longer to run than 2.4.4 on
> my laptop. You need to go all the way up to 7.x in order to back to
> reasonable performance -- and that will probably be too new to support
> most of the distro-packaged Sphinxes.
>
> If we have two or more requirements*.txt files, we could just freeze ALL
> the dependencies for 2.4.4 at versions that we know to work. That should
> stop the whack-a-mole for that version. And then we don't have to force
> everybody on to newer/slower versions.
>
> I mean, I can also maintain my own requirements-2.4.4.txt on my local
> filesystem but I think it's better to do the right thing out of the box
> for all users, no? Maybe we should also add in warnings about the
> known-slow Sphinx versions.

If you're using distro-packaged Sphinx, or use pip system/user install,
specifying exact package requirements like 2.4.4 does not really make
sense to me. I don't want kernel requirements to interfere with whatever
else I'm doing. You should just specify the minimum Sphinx version using
needs_sphinx in conf.py, maybe based on [1].

On the other hand, if you're using a virtual environment, what's the
point in holding back to a version as old as 2.4.4? You might just as
well go for latest, specifying only the top level dependencies,
i.e. sphinx and pyyaml. Or you could pip freeze all the requirements at
a relatively new known good configuration. That's kind of the idea with
the virtual environment.

That's really the only two (or three) approaches that make sense to
me. Using a virtual environment to use ancient versions is just weird.


BR,
Jani.


[1] https://lore.kernel.org/r/a445d391-4cc9-4d6d-85ad-02d23aa57ebb@gmail.com

-- 
Jani Nikula, Intel

Re: docs: requirements.txt has stopped working again

[Vegard Nossum]

On 23/01/2024 13:30, Jani Nikula wrote:
> On the other hand, if you're using a virtual environment, what's the
> point in holding back to a version as old as 2.4.4? You might just as
> well go for latest, specifying only the top level dependencies,

Performance... Specifying exact package requirements like 2.4.4 is
useful since 2.4.4 is by far the fastest Sphinx version that builds our
documentation correctly (AFAICT) and build speed matters a lot when the
difference is 10 minutes vs 30 minutes.

> i.e. sphinx and pyyaml. Or you could pip freeze all the requirements at
> a relatively new known good configuration. That's kind of the idea with
> the virtual environment.
> 
> That's really the only two (or three) approaches that make sense to
> me. Using a virtual environment to use ancient versions is just weird.
It makes sense when those ancient versions build our docs just fine and
run MUCH faster too. Here was my proposal, more specifically:

1) requirements.txt : take out all the version constraints so it will
just use the latest versions of everything (unless there are issues with
those) -- this is what I think Akira/Jon/you really want

2) requirements-2.4.4.txt : create this file and add and freeze ALL the
sphinx dependencies at specific versions that make 2.4.4 work --
freezing everything means we should never really need to touch this file
again

3) add a warning when building using slow sphinx versions, perhaps
encouraging people with these slow versions to use
requirements-2.4.4.txt with a virtualenv.

I think this ticks all the boxes:

- No more whack-a-mole (since requirements.txt would have no bounds to
update, and requirements-2.4.4.txt would have everything frozen)

- Doesn't raise the minimum version unnecessarily for people who would
still like to use the older and faster version.


Vegard

Re: docs: requirements.txt has stopped working again

[Jonathan Corbet]

Vegard Nossum <vegard.nossum@oracle.com> writes:

> It makes sense when those ancient versions build our docs just fine and
> run MUCH faster too. Here was my proposal, more specifically:
>
> 1) requirements.txt : take out all the version constraints so it will
> just use the latest versions of everything (unless there are issues with
> those) -- this is what I think Akira/Jon/you really want

Yes, this is what I would do.

> 2) requirements-2.4.4.txt : create this file and add and freeze ALL the
> sphinx dependencies at specific versions that make 2.4.4 work --
> freezing everything means we should never really need to touch this file
> again

I don't believe you'll ever get to a point where it never breaks; that's
a *lot* of dependencies to track down.

I don't think it's worth it.  The number of people who *really* want to
build with a 2.4.4 system is, I believe, quite small; they may all be
copied on this email.  It is enough of a niche thing at this point that,
I think, we should not go to great lengths to support it.  Perhaps
somebody can contribute a document for folks who really want to do it.

> 3) add a warning when building using slow sphinx versions, perhaps
> encouraging people with these slow versions to use
> requirements-2.4.4.txt with a virtualenv.
>
> I think this ticks all the boxes:
>
> - No more whack-a-mole (since requirements.txt would have no bounds to
> update, and requirements-2.4.4.txt would have everything frozen)
>
> - Doesn't raise the minimum version unnecessarily for people who would
> still like to use the older and faster version.

I'm happy to not break 2.4.4 for now, though I suspect that day may
come.  But I think directing people toward such an old version is not
going to lead to long-term joy; for better or for worse, Sphinx has
moved on.

jon

Re: docs: requirements.txt has stopped working again

[Jani Nikula]

On Tue, 23 Jan 2024, Akira Yokosawa <akiyks@gmail.com> wrote:
> Hi all,
>
> Annoyingly, installing Sphinx 2.4.4 using requirements.txt of v6.8-rc1
> ends up in a run-time error which looks similar to the one Vegard
> reported in commit f4cac0f74658 ("Documentation: constrain alabaster
> package to older versions").
>
> The new error is from sphinxcontrib.applehelp which reads:
>
>     Sphinx version error:
>     The sphinxcontrib.applehelp extension used by this project needs
>     at least Sphinx v5.0; it therefore cannot be built with this version.
>
> Do we want to continue whack-a-mole update for Sphinx 2.4.4?

I looked into the root cause here. Sphinx since early versions
(predating even 2.4.4) depends on packages such as

- sphinxcontrib-applehelp
- sphinxcontrib-devhelp
- sphinxcontrib-htmlhelp
- sphinxcontrib-qthelp
- sphinxcontrib-serializinghtml

but does so without any version constraints. Recently, all of those
*packages* started first depending on Sphinx >= 5. Apparently this
prevented installation of incompatible combos, but resulted in circular
dependency issues in pip.

To fix the circular dependencies, all of those *packages* made the
dependency on Sphinx >= 5 optional, but made the *extensions* in them
app.require_sphinx('5.0') runtime. This happened in the past two weeks.

What this means is that installing any Sphinx < 5 succeeds, it'll pull
in the latest versions of the above mentioned packages because they meet
the package dependencies, but all of them will fail runtime for any
Sphinx version before 5.0. Even if we don't actually use or need any of
those extensions.

I ran git blame and git tag --contains on them to figure out the
packages compatible with Sphinx < 5. These should do it:

diff --git a/Documentation/sphinx/requirements.txt b/Documentation/sphinx/requirements.txt
index 5d47ed443949..0aa4fdb84632 100644
--- a/Documentation/sphinx/requirements.txt
+++ b/Documentation/sphinx/requirements.txt
@@ -2,5 +2,10 @@
 jinja2<3.1
 # alabaster>=0.7.14 is not compatible with Sphinx<=3.3
 alabaster<0.7.14
+sphinxcontrib-applehelp==1.0.4
+sphinxcontrib-devhelp==1.0.2
+sphinxcontrib-htmlhelp==2.0.1
+sphinxcontrib-qthelp==1.0.3
+sphinxcontrib-serializinghtml==1.1.5
 Sphinx==2.4.4
 pyyaml

Conclusions:

- Distro packages probably work, because they maintain deps separately.

- Installing latest from pip probably works. There may be occasional
  hiccups, but they'll likely get noticed and fixed quickly.

- Installing known good requirements from pip freeze works.

- Installing some random version from pip is likely to fail.

- Installing a mix of distro and pip packages is likely to fail.

So maybe we'll need a minimum Sphinx version for distro packages, and
three separate requirements. One with just the top level deps (sphinx
and pyyaml, no versions specified), one which is basically a pip freeze
of that with know good config, updated regularly, and then a pip freeze
of the conservative (2.4.4?) versions, rarely updated.


BR,
Jani.


-- 
Jani Nikula, Intel

Re: docs: requirements.txt has stopped working again

[Jani Nikula]

On Tue, 23 Jan 2024, Jani Nikula <jani.nikula@linux.intel.com> wrote:
> I looked into the root cause here. Sphinx since early versions
> (predating even 2.4.4) depends on packages such as
>
> - sphinxcontrib-applehelp
> - sphinxcontrib-devhelp
> - sphinxcontrib-htmlhelp
> - sphinxcontrib-qthelp
> - sphinxcontrib-serializinghtml
>
> but does so without any version constraints. Recently, all of those
> *packages* started first depending on Sphinx >= 5. Apparently this
> prevented installation of incompatible combos, but resulted in circular
> dependency issues in pip.

See: https://github.com/sphinx-doc/sphinx/issues/11567

> To fix the circular dependencies, all of those *packages* made the
> dependency on Sphinx >= 5 optional, but made the *extensions* in them
> app.require_sphinx('5.0') runtime. This happened in the past two weeks.

See: https://github.com/sphinx-doc/sphinx/issues/11890


BR,
Jani.


-- 
Jani Nikula, Intel

Re: docs: requirements.txt has stopped working again

[Akira Yokosawa]

On Tue, 23 Jan 2024 14:21:32 +0100, Vegard Nossum wrote:
> On 23/01/2024 13:30, Jani Nikula wrote:
>> On the other hand, if you're using a virtual environment, what's the
>> point in holding back to a version as old as 2.4.4? You might just as
>> well go for latest, specifying only the top level dependencies,
> 
> Performance... Specifying exact package requirements like 2.4.4 is
> useful since 2.4.4 is by far the fastest Sphinx version that builds our
> documentation correctly (AFAICT) and build speed matters a lot when the
> difference is 10 minutes vs 30 minutes.
> 

I've never observed such a huge difference, probably because I almost
always do clean build of the document, i.e., run "make cleandocs" before
running "make htmldocs".

So I assumed the performance regression Vegard are emphasizing should
be in incremental builds.

Here are some of the results comparing v2.4.5, v4.3.2 (of ubuntu jammy),
and v7.2.6.  (v2.4.5 needs "make -j2 htmldocs" to avoid OOM.)
Incremental builds are done after moving from v6.7 to v6.8-rc1.

VM spec used: memory: 8GB, threads: 4, ubuntu jammy

data in each cell: elapsed time, max resident memory

                                    v2.4.5        v4.3.2        v7.2.6
  =============================  ============  ============  ============
  clean build at v6.7            10m08s 3.3GB  10m31s 1.1GB  10m14s 1.2GB
  incremental build at v6.8-rc1  11m22s 3.3GB  18m55s 1.2GB  19m50s 1.4GB
  clean build at v6.8-rc1        10m45s 3.2GB  10m32s 1.2GB  10m13s 1.3GB

  empty make at v6.8-rc1         3.3s          6.6s          7.0s
  =============================  ============  ============  ============

I took only one sample for each run, so these numbers should not be
taken as representative, but they can still show tendencies.

This means the slowness in incremental build is not improved in v7.2.6,
which contradicts what Vegard said earlier in this thread.

I think incremental build is what Jon uses in his merging.
So I'm afraid the performance regression might be troublesome in Jon's
workflow.

HTH, Akira

Re: docs: requirements.txt has stopped working again

[Jonathan Corbet]

Akira Yokosawa <akiyks@gmail.com> writes:

> On Tue, 23 Jan 2024 14:21:32 +0100, Vegard Nossum wrote:
>> On 23/01/2024 13:30, Jani Nikula wrote:
>>> On the other hand, if you're using a virtual environment, what's the
>>> point in holding back to a version as old as 2.4.4? You might just as
>>> well go for latest, specifying only the top level dependencies,
>> 
>> Performance... Specifying exact package requirements like 2.4.4 is
>> useful since 2.4.4 is by far the fastest Sphinx version that builds our
>> documentation correctly (AFAICT) and build speed matters a lot when the
>> difference is 10 minutes vs 30 minutes.
>> 
>
> I've never observed such a huge difference, probably because I almost
> always do clean build of the document, i.e., run "make cleandocs" before
> running "make htmldocs".
>
> So I assumed the performance regression Vegard are emphasizing should
> be in incremental builds.
>
> Here are some of the results comparing v2.4.5, v4.3.2 (of ubuntu jammy),
> and v7.2.6.  (v2.4.5 needs "make -j2 htmldocs" to avoid OOM.)
> Incremental builds are done after moving from v6.7 to v6.8-rc1.
>
> VM spec used: memory: 8GB, threads: 4, ubuntu jammy
>
> data in each cell: elapsed time, max resident memory
>
>                                     v2.4.5        v4.3.2        v7.2.6
>   =============================  ============  ============  ============
>   clean build at v6.7            10m08s 3.3GB  10m31s 1.1GB  10m14s 1.2GB
>   incremental build at v6.8-rc1  11m22s 3.3GB  18m55s 1.2GB  19m50s 1.4GB
>   clean build at v6.8-rc1        10m45s 3.2GB  10m32s 1.2GB  10m13s 1.3GB
>
>   empty make at v6.8-rc1         3.3s          6.6s          7.0s
>   =============================  ============  ============  ============

So that is quite different from my experience.  For me, full builds got
way slower starting with 3.x and haven't improved much since, though
I've not played much with 7.x yet.

It's weird that incremental builds take longer than full for you.

> I took only one sample for each run, so these numbers should not be
> taken as representative, but they can still show tendencies.
>
> This means the slowness in incremental build is not improved in v7.2.6,
> which contradicts what Vegard said earlier in this thread.
>
> I think incremental build is what Jon uses in his merging.
> So I'm afraid the performance regression might be troublesome in Jon's
> workflow.

I sometimes do incrementals between simple patches (and they tend to be
pretty fast), but I do a lot of full builds and look for changes in
output as well.

jon

Re: docs: requirements.txt has stopped working again

[Akira Yokosawa]

On Wed, 24 Jan 2024 08:25:57 -0700, Jonathan Corbet wrote:
> Akira Yokosawa <akiyks@gmail.com> writes:
> 
>> On Tue, 23 Jan 2024 14:21:32 +0100, Vegard Nossum wrote:
>>> On 23/01/2024 13:30, Jani Nikula wrote:
>>>> On the other hand, if you're using a virtual environment, what's the
>>>> point in holding back to a version as old as 2.4.4? You might just as
>>>> well go for latest, specifying only the top level dependencies,
>>>
>>> Performance... Specifying exact package requirements like 2.4.4 is
>>> useful since 2.4.4 is by far the fastest Sphinx version that builds our
>>> documentation correctly (AFAICT) and build speed matters a lot when the
>>> difference is 10 minutes vs 30 minutes.
>>>
>>
>> I've never observed such a huge difference, probably because I almost
>> always do clean build of the document, i.e., run "make cleandocs" before
>> running "make htmldocs".
>>
>> So I assumed the performance regression Vegard are emphasizing should
>> be in incremental builds.
>>
>> Here are some of the results comparing v2.4.5, v4.3.2 (of ubuntu jammy),
>> and v7.2.6.  (v2.4.5 needs "make -j2 htmldocs" to avoid OOM.)
>> Incremental builds are done after moving from v6.7 to v6.8-rc1.
>>
>> VM spec used: memory: 8GB, threads: 4, ubuntu jammy
>>
>> data in each cell: elapsed time, max resident memory
>>
>>                                     v2.4.5        v4.3.2        v7.2.6
>>   =============================  ============  ============  ============
>>   clean build at v6.7            10m08s 3.3GB  10m31s 1.1GB  10m14s 1.2GB
>>   incremental build at v6.8-rc1  11m22s 3.3GB  18m55s 1.2GB  19m50s 1.4GB
>>   clean build at v6.8-rc1        10m45s 3.2GB  10m32s 1.2GB  10m13s 1.3GB
>>
>>   empty make at v6.8-rc1         3.3s          6.6s          7.0s
>>   =============================  ============  ============  ============
> 
> So that is quite different from my experience.  For me, full builds got
> way slower starting with 3.x and haven't improved much since, though
> I've not played much with 7.x yet.

One of the reasons I can think of why 2.4.5 is not faster is
the "make -j2" I need to use.  2.4.x is way more eager to use
more parallel slots than >=3.1 in later stages of its processing.
I think you have a memory rich system that allows a lot of parallel
slots.  On a machine with 16GB memory, I can say -j4 (or -j5 if
I am lucky).

> 
> It's weird that incremental builds take longer than full for you.
> 

Incremental builds of small differences is faster than full for me
as well.

I used v6.7 --> v6.8-rc1 (full merge window) to emphasize the slowness.

But yes, it's strange to see incremental build becomes slower
than full build even if the diff is a lot.

        Thanks, Akira

Re: docs: requirements.txt has stopped working again

[Vegard Nossum]

On 24/01/2024 16:02, Akira Yokosawa wrote:
> On Tue, 23 Jan 2024 14:21:32 +0100, Vegard Nossum wrote:
>> On 23/01/2024 13:30, Jani Nikula wrote:
>>> On the other hand, if you're using a virtual environment, what's the
>>> point in holding back to a version as old as 2.4.4? You might just as
>>> well go for latest, specifying only the top level dependencies,
>>
>> Performance... Specifying exact package requirements like 2.4.4 is
>> useful since 2.4.4 is by far the fastest Sphinx version that builds our
>> documentation correctly (AFAICT) and build speed matters a lot when the
>> difference is 10 minutes vs 30 minutes.
>>
> 
> I've never observed such a huge difference, probably because I almost
> always do clean build of the document, i.e., run "make cleandocs" before
> running "make htmldocs".
> 
> So I assumed the performance regression Vegard are emphasizing should
> be in incremental builds.

I was actually referring to the full builds, but see below...

> Here are some of the results comparing v2.4.5, v4.3.2 (of ubuntu jammy),
> and v7.2.6.  (v2.4.5 needs "make -j2 htmldocs" to avoid OOM.)
> Incremental builds are done after moving from v6.7 to v6.8-rc1.
> 
> VM spec used: memory: 8GB, threads: 4, ubuntu jammy
> 
> data in each cell: elapsed time, max resident memory
> 
>                                      v2.4.5        v4.3.2        v7.2.6
>    =============================  ============  ============  ============
>    clean build at v6.7            10m08s 3.3GB  10m31s 1.1GB  10m14s 1.2GB
>    incremental build at v6.8-rc1  11m22s 3.3GB  18m55s 1.2GB  19m50s 1.4GB
>    clean build at v6.8-rc1        10m45s 3.2GB  10m32s 1.2GB  10m13s 1.3GB
> 
>    empty make at v6.8-rc1         3.3s          6.6s          7.0s
>    =============================  ============  ============  ============
> 
> I took only one sample for each run, so these numbers should not be
> taken as representative, but they can still show tendencies.

I've done a new test here with numbers that aren't quite as bad as the
ones I gave above:

2.4.4: 12m46,609s
4.3.2: 23m50,909s

I also did some tests the other day on the same machine:

7.2.6: 21m31,522s
7.3.0+/91ed62272712: 23m23,220s

These are all full clean builds.

I am quite sure I actually saw 30m+ before on 4.3.2, but maybe it's
because I didn't run 'make cleandocs' before and I'm running into the
same slowness that you see with your incremental build.

> This means the slowness in incremental build is not improved in v7.2.6,
> which contradicts what Vegard said earlier in this thread.

I think I've been confusing "100% clean full build" and "incremental
build where everything's changed" -- I didn't imagine the incremental
build could be that much slower than a full clean build.

Thank you for bringing clarity and more precise numbers. I'll know to
do a 'make cleandocs' now after switching branches if the bases are
not basically the same.


Vegard

Re: docs: requirements.txt has stopped working again

[Akira Yokosawa]

On Thu, 25 Jan 2024 00:43:51 +0900, Akira Yokosawa wrote:
> On Wed, 24 Jan 2024 08:25:57 -0700, Jonathan Corbet wrote:
>> Akira Yokosawa <akiyks@gmail.com> writes:
[...]
>>> VM spec used: memory: 8GB, threads: 4, ubuntu jammy
>>>
>>> data in each cell: elapsed time, max resident memory
>>>
>>>                                     v2.4.5        v4.3.2        v7.2.6
>>>   =============================  ============  ============  ============
>>>   clean build at v6.7            10m08s 3.3GB  10m31s 1.1GB  10m14s 1.2GB
>>>   incremental build at v6.8-rc1  11m22s 3.3GB  18m55s 1.2GB  19m50s 1.4GB
>>>   clean build at v6.8-rc1        10m45s 3.2GB  10m32s 1.2GB  10m13s 1.3GB
>>>
>>>   empty make at v6.8-rc1         3.3s          6.6s          7.0s
>>>   =============================  ============  ============  ============
>>
>> So that is quite different from my experience.  For me, full builds got
>> way slower starting with 3.x and haven't improved much since, though
>> I've not played much with 7.x yet.
> 
> One of the reasons I can think of why 2.4.5 is not faster is
> the "make -j2" I need to use.  2.4.x is way more eager to use
> more parallel slots than >=3.1 in later stages of its processing.
> I think you have a memory rich system that allows a lot of parallel
> slots.  On a machine with 16GB memory, I can say -j4 (or -j5 if
> I am lucky).
> 

So, I managed to run v2.4.5 with 4 threads.
Also, I ran incremental builds of small diffs at the docs-6.8-fixes
tag relative to v6.8-rc1.

Here is the updated table.

-----------------------------------------
VM spec used: memory: 10GB, threads: 4, ubuntu jammy
data in each cell: elapsed time, max resident memory

                                    v2.4.5       v4.3.2        v7.2.6
 ============================   ===========  ============  ============
 full build at v6.7             6m14s 2.3GB  10m31s 1.1GB  10m14s 1.2GB
 incr build at v6.8-rc1         7m26s 2.5GB  18m55s 1.2GB  19m50s 1.4GB
 full build at v6.8-rc1         6m45s 2.4GB  10m32s 1.2GB  10m13s 1.3GB
 incr build at docs-6.8-fixes   4m58s 2.2GB   6m50s 1.2GB   6m45s 1.3GB

 empty make at v6.8-rc1         3.3s         6.6s          7.0s
 ============================   ===========  ============  ============

This should be close to the reality.

HTH, Akira