* docs: requirements.txt has stopped working again @ 2024-01-23 4:14 Akira Yokosawa 2024-01-23 7:43 ` Vegard Nossum 2024-01-23 16:53 ` Jani Nikula 0 siblings, 2 replies; 12+ messages in thread From: Akira Yokosawa @ 2024-01-23 4:14 UTC (permalink / raw) To: Jonathan Corbet, Vegard Nossum Cc: linux-doc, Mauro Carvalho Chehab, Akira Yokosawa Hi all, Annoyingly, installing Sphinx 2.4.4 using requirements.txt of v6.8-rc1 ends up in a run-time error which looks similar to the one Vegard reported in commit f4cac0f74658 ("Documentation: constrain alabaster package to older versions"). The new error is from sphinxcontrib.applehelp which reads: Sphinx version error: The sphinxcontrib.applehelp extension used by this project needs at least Sphinx v5.0; it therefore cannot be built with this version. Do we want to continue whack-a-mole update for Sphinx 2.4.4? Thanks, Akira ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: docs: requirements.txt has stopped working again 2024-01-23 4:14 docs: requirements.txt has stopped working again Akira Yokosawa @ 2024-01-23 7:43 ` Vegard Nossum 2024-01-23 12:30 ` Jani Nikula 2024-01-23 16:53 ` Jani Nikula 1 sibling, 1 reply; 12+ messages in thread From: Vegard Nossum @ 2024-01-23 7:43 UTC (permalink / raw) To: Akira Yokosawa, Jonathan Corbet; +Cc: linux-doc, Mauro Carvalho Chehab On 23/01/2024 05:14, Akira Yokosawa wrote: > Hi all, > > Annoyingly, installing Sphinx 2.4.4 using requirements.txt of v6.8-rc1 > ends up in a run-time error which looks similar to the one Vegard > reported in commit f4cac0f74658 ("Documentation: constrain alabaster > package to older versions"). > > The new error is from sphinxcontrib.applehelp which reads: > > Sphinx version error: > The sphinxcontrib.applehelp extension used by this project needs > at least Sphinx v5.0; it therefore cannot be built with this version. > > Do we want to continue whack-a-mole update for Sphinx 2.4.4? > > Thanks, Akira Can we have requirements(-latest).txt and requirements-2.4.4.txt? The thing is, we are not using new features of Sphinx in the kernel docs. We don't fundamentally require a new version in any of our .rst files or markup, these aren't security issues being fixed or bugs that we've run into, the problem is purely that we are using a software ecosystem that apparently has no restraint when it comes to breaking their users. My issue with _requiring_ newer versions of Sphinx is the fact that they have performance regressions: 4.3.2 takes 3x longer to run than 2.4.4 on my laptop. You need to go all the way up to 7.x in order to back to reasonable performance -- and that will probably be too new to support most of the distro-packaged Sphinxes. If we have two or more requirements*.txt files, we could just freeze ALL the dependencies for 2.4.4 at versions that we know to work. That should stop the whack-a-mole for that version. And then we don't have to force everybody on to newer/slower versions. I mean, I can also maintain my own requirements-2.4.4.txt on my local filesystem but I think it's better to do the right thing out of the box for all users, no? Maybe we should also add in warnings about the known-slow Sphinx versions. Vegard ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: docs: requirements.txt has stopped working again 2024-01-23 7:43 ` Vegard Nossum @ 2024-01-23 12:30 ` Jani Nikula 2024-01-23 13:21 ` Vegard Nossum 0 siblings, 1 reply; 12+ messages in thread From: Jani Nikula @ 2024-01-23 12:30 UTC (permalink / raw) To: Vegard Nossum, Akira Yokosawa, Jonathan Corbet Cc: linux-doc, Mauro Carvalho Chehab On Tue, 23 Jan 2024, Vegard Nossum <vegard.nossum@oracle.com> wrote: > On 23/01/2024 05:14, Akira Yokosawa wrote: >> Hi all, >> >> Annoyingly, installing Sphinx 2.4.4 using requirements.txt of v6.8-rc1 >> ends up in a run-time error which looks similar to the one Vegard >> reported in commit f4cac0f74658 ("Documentation: constrain alabaster >> package to older versions"). >> >> The new error is from sphinxcontrib.applehelp which reads: >> >> Sphinx version error: >> The sphinxcontrib.applehelp extension used by this project needs >> at least Sphinx v5.0; it therefore cannot be built with this version. >> >> Do we want to continue whack-a-mole update for Sphinx 2.4.4? >> >> Thanks, Akira > > Can we have requirements(-latest).txt and requirements-2.4.4.txt? > > The thing is, we are not using new features of Sphinx in the kernel > docs. We don't fundamentally require a new version in any of our .rst > files or markup, these aren't security issues being fixed or bugs that > we've run into, the problem is purely that we are using a software > ecosystem that apparently has no restraint when it comes to breaking > their users. We do use at least namespacing which was new to Sphinx 3.1. And we could use more if we weren't so conservative about the versions. > My issue with _requiring_ newer versions of Sphinx is the fact that they > have performance regressions: 4.3.2 takes 3x longer to run than 2.4.4 on > my laptop. You need to go all the way up to 7.x in order to back to > reasonable performance -- and that will probably be too new to support > most of the distro-packaged Sphinxes. > > If we have two or more requirements*.txt files, we could just freeze ALL > the dependencies for 2.4.4 at versions that we know to work. That should > stop the whack-a-mole for that version. And then we don't have to force > everybody on to newer/slower versions. > > I mean, I can also maintain my own requirements-2.4.4.txt on my local > filesystem but I think it's better to do the right thing out of the box > for all users, no? Maybe we should also add in warnings about the > known-slow Sphinx versions. If you're using distro-packaged Sphinx, or use pip system/user install, specifying exact package requirements like 2.4.4 does not really make sense to me. I don't want kernel requirements to interfere with whatever else I'm doing. You should just specify the minimum Sphinx version using needs_sphinx in conf.py, maybe based on [1]. On the other hand, if you're using a virtual environment, what's the point in holding back to a version as old as 2.4.4? You might just as well go for latest, specifying only the top level dependencies, i.e. sphinx and pyyaml. Or you could pip freeze all the requirements at a relatively new known good configuration. That's kind of the idea with the virtual environment. That's really the only two (or three) approaches that make sense to me. Using a virtual environment to use ancient versions is just weird. BR, Jani. [1] https://lore.kernel.org/r/a445d391-4cc9-4d6d-85ad-02d23aa57ebb@gmail.com -- Jani Nikula, Intel ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: docs: requirements.txt has stopped working again 2024-01-23 12:30 ` Jani Nikula @ 2024-01-23 13:21 ` Vegard Nossum 2024-01-23 16:19 ` Jonathan Corbet 2024-01-24 15:02 ` Akira Yokosawa 0 siblings, 2 replies; 12+ messages in thread From: Vegard Nossum @ 2024-01-23 13:21 UTC (permalink / raw) To: Jani Nikula, Akira Yokosawa, Jonathan Corbet Cc: linux-doc, Mauro Carvalho Chehab On 23/01/2024 13:30, Jani Nikula wrote: > On the other hand, if you're using a virtual environment, what's the > point in holding back to a version as old as 2.4.4? You might just as > well go for latest, specifying only the top level dependencies, Performance... Specifying exact package requirements like 2.4.4 is useful since 2.4.4 is by far the fastest Sphinx version that builds our documentation correctly (AFAICT) and build speed matters a lot when the difference is 10 minutes vs 30 minutes. > i.e. sphinx and pyyaml. Or you could pip freeze all the requirements at > a relatively new known good configuration. That's kind of the idea with > the virtual environment. > > That's really the only two (or three) approaches that make sense to > me. Using a virtual environment to use ancient versions is just weird. It makes sense when those ancient versions build our docs just fine and run MUCH faster too. Here was my proposal, more specifically: 1) requirements.txt : take out all the version constraints so it will just use the latest versions of everything (unless there are issues with those) -- this is what I think Akira/Jon/you really want 2) requirements-2.4.4.txt : create this file and add and freeze ALL the sphinx dependencies at specific versions that make 2.4.4 work -- freezing everything means we should never really need to touch this file again 3) add a warning when building using slow sphinx versions, perhaps encouraging people with these slow versions to use requirements-2.4.4.txt with a virtualenv. I think this ticks all the boxes: - No more whack-a-mole (since requirements.txt would have no bounds to update, and requirements-2.4.4.txt would have everything frozen) - Doesn't raise the minimum version unnecessarily for people who would still like to use the older and faster version. Vegard ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: docs: requirements.txt has stopped working again 2024-01-23 13:21 ` Vegard Nossum @ 2024-01-23 16:19 ` Jonathan Corbet 2024-01-24 15:02 ` Akira Yokosawa 1 sibling, 0 replies; 12+ messages in thread From: Jonathan Corbet @ 2024-01-23 16:19 UTC (permalink / raw) To: Vegard Nossum, Jani Nikula, Akira Yokosawa Cc: linux-doc, Mauro Carvalho Chehab Vegard Nossum <vegard.nossum@oracle.com> writes: > It makes sense when those ancient versions build our docs just fine and > run MUCH faster too. Here was my proposal, more specifically: > > 1) requirements.txt : take out all the version constraints so it will > just use the latest versions of everything (unless there are issues with > those) -- this is what I think Akira/Jon/you really want Yes, this is what I would do. > 2) requirements-2.4.4.txt : create this file and add and freeze ALL the > sphinx dependencies at specific versions that make 2.4.4 work -- > freezing everything means we should never really need to touch this file > again I don't believe you'll ever get to a point where it never breaks; that's a *lot* of dependencies to track down. I don't think it's worth it. The number of people who *really* want to build with a 2.4.4 system is, I believe, quite small; they may all be copied on this email. It is enough of a niche thing at this point that, I think, we should not go to great lengths to support it. Perhaps somebody can contribute a document for folks who really want to do it. > 3) add a warning when building using slow sphinx versions, perhaps > encouraging people with these slow versions to use > requirements-2.4.4.txt with a virtualenv. > > I think this ticks all the boxes: > > - No more whack-a-mole (since requirements.txt would have no bounds to > update, and requirements-2.4.4.txt would have everything frozen) > > - Doesn't raise the minimum version unnecessarily for people who would > still like to use the older and faster version. I'm happy to not break 2.4.4 for now, though I suspect that day may come. But I think directing people toward such an old version is not going to lead to long-term joy; for better or for worse, Sphinx has moved on. jon ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: docs: requirements.txt has stopped working again 2024-01-23 13:21 ` Vegard Nossum 2024-01-23 16:19 ` Jonathan Corbet @ 2024-01-24 15:02 ` Akira Yokosawa 2024-01-24 15:25 ` Jonathan Corbet 2024-01-24 19:56 ` Vegard Nossum 1 sibling, 2 replies; 12+ messages in thread From: Akira Yokosawa @ 2024-01-24 15:02 UTC (permalink / raw) To: Vegard Nossum, Jani Nikula, Jonathan Corbet Cc: linux-doc, Mauro Carvalho Chehab, Akira Yokosawa On Tue, 23 Jan 2024 14:21:32 +0100, Vegard Nossum wrote: > On 23/01/2024 13:30, Jani Nikula wrote: >> On the other hand, if you're using a virtual environment, what's the >> point in holding back to a version as old as 2.4.4? You might just as >> well go for latest, specifying only the top level dependencies, > > Performance... Specifying exact package requirements like 2.4.4 is > useful since 2.4.4 is by far the fastest Sphinx version that builds our > documentation correctly (AFAICT) and build speed matters a lot when the > difference is 10 minutes vs 30 minutes. > I've never observed such a huge difference, probably because I almost always do clean build of the document, i.e., run "make cleandocs" before running "make htmldocs". So I assumed the performance regression Vegard are emphasizing should be in incremental builds. Here are some of the results comparing v2.4.5, v4.3.2 (of ubuntu jammy), and v7.2.6. (v2.4.5 needs "make -j2 htmldocs" to avoid OOM.) Incremental builds are done after moving from v6.7 to v6.8-rc1. VM spec used: memory: 8GB, threads: 4, ubuntu jammy data in each cell: elapsed time, max resident memory v2.4.5 v4.3.2 v7.2.6 ============================= ============ ============ ============ clean build at v6.7 10m08s 3.3GB 10m31s 1.1GB 10m14s 1.2GB incremental build at v6.8-rc1 11m22s 3.3GB 18m55s 1.2GB 19m50s 1.4GB clean build at v6.8-rc1 10m45s 3.2GB 10m32s 1.2GB 10m13s 1.3GB empty make at v6.8-rc1 3.3s 6.6s 7.0s ============================= ============ ============ ============ I took only one sample for each run, so these numbers should not be taken as representative, but they can still show tendencies. This means the slowness in incremental build is not improved in v7.2.6, which contradicts what Vegard said earlier in this thread. I think incremental build is what Jon uses in his merging. So I'm afraid the performance regression might be troublesome in Jon's workflow. HTH, Akira ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: docs: requirements.txt has stopped working again 2024-01-24 15:02 ` Akira Yokosawa @ 2024-01-24 15:25 ` Jonathan Corbet 2024-01-24 15:43 ` Akira Yokosawa 2024-01-24 19:56 ` Vegard Nossum 1 sibling, 1 reply; 12+ messages in thread From: Jonathan Corbet @ 2024-01-24 15:25 UTC (permalink / raw) To: Akira Yokosawa, Vegard Nossum, Jani Nikula Cc: linux-doc, Mauro Carvalho Chehab, Akira Yokosawa Akira Yokosawa <akiyks@gmail.com> writes: > On Tue, 23 Jan 2024 14:21:32 +0100, Vegard Nossum wrote: >> On 23/01/2024 13:30, Jani Nikula wrote: >>> On the other hand, if you're using a virtual environment, what's the >>> point in holding back to a version as old as 2.4.4? You might just as >>> well go for latest, specifying only the top level dependencies, >> >> Performance... Specifying exact package requirements like 2.4.4 is >> useful since 2.4.4 is by far the fastest Sphinx version that builds our >> documentation correctly (AFAICT) and build speed matters a lot when the >> difference is 10 minutes vs 30 minutes. >> > > I've never observed such a huge difference, probably because I almost > always do clean build of the document, i.e., run "make cleandocs" before > running "make htmldocs". > > So I assumed the performance regression Vegard are emphasizing should > be in incremental builds. > > Here are some of the results comparing v2.4.5, v4.3.2 (of ubuntu jammy), > and v7.2.6. (v2.4.5 needs "make -j2 htmldocs" to avoid OOM.) > Incremental builds are done after moving from v6.7 to v6.8-rc1. > > VM spec used: memory: 8GB, threads: 4, ubuntu jammy > > data in each cell: elapsed time, max resident memory > > v2.4.5 v4.3.2 v7.2.6 > ============================= ============ ============ ============ > clean build at v6.7 10m08s 3.3GB 10m31s 1.1GB 10m14s 1.2GB > incremental build at v6.8-rc1 11m22s 3.3GB 18m55s 1.2GB 19m50s 1.4GB > clean build at v6.8-rc1 10m45s 3.2GB 10m32s 1.2GB 10m13s 1.3GB > > empty make at v6.8-rc1 3.3s 6.6s 7.0s > ============================= ============ ============ ============ So that is quite different from my experience. For me, full builds got way slower starting with 3.x and haven't improved much since, though I've not played much with 7.x yet. It's weird that incremental builds take longer than full for you. > I took only one sample for each run, so these numbers should not be > taken as representative, but they can still show tendencies. > > This means the slowness in incremental build is not improved in v7.2.6, > which contradicts what Vegard said earlier in this thread. > > I think incremental build is what Jon uses in his merging. > So I'm afraid the performance regression might be troublesome in Jon's > workflow. I sometimes do incrementals between simple patches (and they tend to be pretty fast), but I do a lot of full builds and look for changes in output as well. jon ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: docs: requirements.txt has stopped working again 2024-01-24 15:25 ` Jonathan Corbet @ 2024-01-24 15:43 ` Akira Yokosawa 2024-01-26 14:42 ` Akira Yokosawa 0 siblings, 1 reply; 12+ messages in thread From: Akira Yokosawa @ 2024-01-24 15:43 UTC (permalink / raw) To: Jonathan Corbet, Vegard Nossum, Jani Nikula Cc: linux-doc, Mauro Carvalho Chehab, Akira Yokosawa On Wed, 24 Jan 2024 08:25:57 -0700, Jonathan Corbet wrote: > Akira Yokosawa <akiyks@gmail.com> writes: > >> On Tue, 23 Jan 2024 14:21:32 +0100, Vegard Nossum wrote: >>> On 23/01/2024 13:30, Jani Nikula wrote: >>>> On the other hand, if you're using a virtual environment, what's the >>>> point in holding back to a version as old as 2.4.4? You might just as >>>> well go for latest, specifying only the top level dependencies, >>> >>> Performance... Specifying exact package requirements like 2.4.4 is >>> useful since 2.4.4 is by far the fastest Sphinx version that builds our >>> documentation correctly (AFAICT) and build speed matters a lot when the >>> difference is 10 minutes vs 30 minutes. >>> >> >> I've never observed such a huge difference, probably because I almost >> always do clean build of the document, i.e., run "make cleandocs" before >> running "make htmldocs". >> >> So I assumed the performance regression Vegard are emphasizing should >> be in incremental builds. >> >> Here are some of the results comparing v2.4.5, v4.3.2 (of ubuntu jammy), >> and v7.2.6. (v2.4.5 needs "make -j2 htmldocs" to avoid OOM.) >> Incremental builds are done after moving from v6.7 to v6.8-rc1. >> >> VM spec used: memory: 8GB, threads: 4, ubuntu jammy >> >> data in each cell: elapsed time, max resident memory >> >> v2.4.5 v4.3.2 v7.2.6 >> ============================= ============ ============ ============ >> clean build at v6.7 10m08s 3.3GB 10m31s 1.1GB 10m14s 1.2GB >> incremental build at v6.8-rc1 11m22s 3.3GB 18m55s 1.2GB 19m50s 1.4GB >> clean build at v6.8-rc1 10m45s 3.2GB 10m32s 1.2GB 10m13s 1.3GB >> >> empty make at v6.8-rc1 3.3s 6.6s 7.0s >> ============================= ============ ============ ============ > > So that is quite different from my experience. For me, full builds got > way slower starting with 3.x and haven't improved much since, though > I've not played much with 7.x yet. One of the reasons I can think of why 2.4.5 is not faster is the "make -j2" I need to use. 2.4.x is way more eager to use more parallel slots than >=3.1 in later stages of its processing. I think you have a memory rich system that allows a lot of parallel slots. On a machine with 16GB memory, I can say -j4 (or -j5 if I am lucky). > > It's weird that incremental builds take longer than full for you. > Incremental builds of small differences is faster than full for me as well. I used v6.7 --> v6.8-rc1 (full merge window) to emphasize the slowness. But yes, it's strange to see incremental build becomes slower than full build even if the diff is a lot. Thanks, Akira ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: docs: requirements.txt has stopped working again 2024-01-24 15:43 ` Akira Yokosawa @ 2024-01-26 14:42 ` Akira Yokosawa 0 siblings, 0 replies; 12+ messages in thread From: Akira Yokosawa @ 2024-01-26 14:42 UTC (permalink / raw) To: Jonathan Corbet, Vegard Nossum, Jani Nikula Cc: linux-doc, Mauro Carvalho Chehab On Thu, 25 Jan 2024 00:43:51 +0900, Akira Yokosawa wrote: > On Wed, 24 Jan 2024 08:25:57 -0700, Jonathan Corbet wrote: >> Akira Yokosawa <akiyks@gmail.com> writes: [...] >>> VM spec used: memory: 8GB, threads: 4, ubuntu jammy >>> >>> data in each cell: elapsed time, max resident memory >>> >>> v2.4.5 v4.3.2 v7.2.6 >>> ============================= ============ ============ ============ >>> clean build at v6.7 10m08s 3.3GB 10m31s 1.1GB 10m14s 1.2GB >>> incremental build at v6.8-rc1 11m22s 3.3GB 18m55s 1.2GB 19m50s 1.4GB >>> clean build at v6.8-rc1 10m45s 3.2GB 10m32s 1.2GB 10m13s 1.3GB >>> >>> empty make at v6.8-rc1 3.3s 6.6s 7.0s >>> ============================= ============ ============ ============ >> >> So that is quite different from my experience. For me, full builds got >> way slower starting with 3.x and haven't improved much since, though >> I've not played much with 7.x yet. > > One of the reasons I can think of why 2.4.5 is not faster is > the "make -j2" I need to use. 2.4.x is way more eager to use > more parallel slots than >=3.1 in later stages of its processing. > I think you have a memory rich system that allows a lot of parallel > slots. On a machine with 16GB memory, I can say -j4 (or -j5 if > I am lucky). > So, I managed to run v2.4.5 with 4 threads. Also, I ran incremental builds of small diffs at the docs-6.8-fixes tag relative to v6.8-rc1. Here is the updated table. ----------------------------------------- VM spec used: memory: 10GB, threads: 4, ubuntu jammy data in each cell: elapsed time, max resident memory v2.4.5 v4.3.2 v7.2.6 ============================ =========== ============ ============ full build at v6.7 6m14s 2.3GB 10m31s 1.1GB 10m14s 1.2GB incr build at v6.8-rc1 7m26s 2.5GB 18m55s 1.2GB 19m50s 1.4GB full build at v6.8-rc1 6m45s 2.4GB 10m32s 1.2GB 10m13s 1.3GB incr build at docs-6.8-fixes 4m58s 2.2GB 6m50s 1.2GB 6m45s 1.3GB empty make at v6.8-rc1 3.3s 6.6s 7.0s ============================ =========== ============ ============ This should be close to the reality. HTH, Akira ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: docs: requirements.txt has stopped working again 2024-01-24 15:02 ` Akira Yokosawa 2024-01-24 15:25 ` Jonathan Corbet @ 2024-01-24 19:56 ` Vegard Nossum 1 sibling, 0 replies; 12+ messages in thread From: Vegard Nossum @ 2024-01-24 19:56 UTC (permalink / raw) To: Akira Yokosawa, Jani Nikula, Jonathan Corbet Cc: linux-doc, Mauro Carvalho Chehab On 24/01/2024 16:02, Akira Yokosawa wrote: > On Tue, 23 Jan 2024 14:21:32 +0100, Vegard Nossum wrote: >> On 23/01/2024 13:30, Jani Nikula wrote: >>> On the other hand, if you're using a virtual environment, what's the >>> point in holding back to a version as old as 2.4.4? You might just as >>> well go for latest, specifying only the top level dependencies, >> >> Performance... Specifying exact package requirements like 2.4.4 is >> useful since 2.4.4 is by far the fastest Sphinx version that builds our >> documentation correctly (AFAICT) and build speed matters a lot when the >> difference is 10 minutes vs 30 minutes. >> > > I've never observed such a huge difference, probably because I almost > always do clean build of the document, i.e., run "make cleandocs" before > running "make htmldocs". > > So I assumed the performance regression Vegard are emphasizing should > be in incremental builds. I was actually referring to the full builds, but see below... > Here are some of the results comparing v2.4.5, v4.3.2 (of ubuntu jammy), > and v7.2.6. (v2.4.5 needs "make -j2 htmldocs" to avoid OOM.) > Incremental builds are done after moving from v6.7 to v6.8-rc1. > > VM spec used: memory: 8GB, threads: 4, ubuntu jammy > > data in each cell: elapsed time, max resident memory > > v2.4.5 v4.3.2 v7.2.6 > ============================= ============ ============ ============ > clean build at v6.7 10m08s 3.3GB 10m31s 1.1GB 10m14s 1.2GB > incremental build at v6.8-rc1 11m22s 3.3GB 18m55s 1.2GB 19m50s 1.4GB > clean build at v6.8-rc1 10m45s 3.2GB 10m32s 1.2GB 10m13s 1.3GB > > empty make at v6.8-rc1 3.3s 6.6s 7.0s > ============================= ============ ============ ============ > > I took only one sample for each run, so these numbers should not be > taken as representative, but they can still show tendencies. I've done a new test here with numbers that aren't quite as bad as the ones I gave above: 2.4.4: 12m46,609s 4.3.2: 23m50,909s I also did some tests the other day on the same machine: 7.2.6: 21m31,522s 7.3.0+/91ed62272712: 23m23,220s These are all full clean builds. I am quite sure I actually saw 30m+ before on 4.3.2, but maybe it's because I didn't run 'make cleandocs' before and I'm running into the same slowness that you see with your incremental build. > This means the slowness in incremental build is not improved in v7.2.6, > which contradicts what Vegard said earlier in this thread. I think I've been confusing "100% clean full build" and "incremental build where everything's changed" -- I didn't imagine the incremental build could be that much slower than a full clean build. Thank you for bringing clarity and more precise numbers. I'll know to do a 'make cleandocs' now after switching branches if the bases are not basically the same. Vegard ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: docs: requirements.txt has stopped working again 2024-01-23 4:14 docs: requirements.txt has stopped working again Akira Yokosawa 2024-01-23 7:43 ` Vegard Nossum @ 2024-01-23 16:53 ` Jani Nikula 2024-01-23 18:11 ` Jani Nikula 1 sibling, 1 reply; 12+ messages in thread From: Jani Nikula @ 2024-01-23 16:53 UTC (permalink / raw) To: Akira Yokosawa, Jonathan Corbet, Vegard Nossum Cc: linux-doc, Mauro Carvalho Chehab, Akira Yokosawa On Tue, 23 Jan 2024, Akira Yokosawa <akiyks@gmail.com> wrote: > Hi all, > > Annoyingly, installing Sphinx 2.4.4 using requirements.txt of v6.8-rc1 > ends up in a run-time error which looks similar to the one Vegard > reported in commit f4cac0f74658 ("Documentation: constrain alabaster > package to older versions"). > > The new error is from sphinxcontrib.applehelp which reads: > > Sphinx version error: > The sphinxcontrib.applehelp extension used by this project needs > at least Sphinx v5.0; it therefore cannot be built with this version. > > Do we want to continue whack-a-mole update for Sphinx 2.4.4? I looked into the root cause here. Sphinx since early versions (predating even 2.4.4) depends on packages such as - sphinxcontrib-applehelp - sphinxcontrib-devhelp - sphinxcontrib-htmlhelp - sphinxcontrib-qthelp - sphinxcontrib-serializinghtml but does so without any version constraints. Recently, all of those *packages* started first depending on Sphinx >= 5. Apparently this prevented installation of incompatible combos, but resulted in circular dependency issues in pip. To fix the circular dependencies, all of those *packages* made the dependency on Sphinx >= 5 optional, but made the *extensions* in them app.require_sphinx('5.0') runtime. This happened in the past two weeks. What this means is that installing any Sphinx < 5 succeeds, it'll pull in the latest versions of the above mentioned packages because they meet the package dependencies, but all of them will fail runtime for any Sphinx version before 5.0. Even if we don't actually use or need any of those extensions. I ran git blame and git tag --contains on them to figure out the packages compatible with Sphinx < 5. These should do it: diff --git a/Documentation/sphinx/requirements.txt b/Documentation/sphinx/requirements.txt index 5d47ed443949..0aa4fdb84632 100644 --- a/Documentation/sphinx/requirements.txt +++ b/Documentation/sphinx/requirements.txt @@ -2,5 +2,10 @@ jinja2<3.1 # alabaster>=0.7.14 is not compatible with Sphinx<=3.3 alabaster<0.7.14 +sphinxcontrib-applehelp==1.0.4 +sphinxcontrib-devhelp==1.0.2 +sphinxcontrib-htmlhelp==2.0.1 +sphinxcontrib-qthelp==1.0.3 +sphinxcontrib-serializinghtml==1.1.5 Sphinx==2.4.4 pyyaml Conclusions: - Distro packages probably work, because they maintain deps separately. - Installing latest from pip probably works. There may be occasional hiccups, but they'll likely get noticed and fixed quickly. - Installing known good requirements from pip freeze works. - Installing some random version from pip is likely to fail. - Installing a mix of distro and pip packages is likely to fail. So maybe we'll need a minimum Sphinx version for distro packages, and three separate requirements. One with just the top level deps (sphinx and pyyaml, no versions specified), one which is basically a pip freeze of that with know good config, updated regularly, and then a pip freeze of the conservative (2.4.4?) versions, rarely updated. BR, Jani. -- Jani Nikula, Intel ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: docs: requirements.txt has stopped working again 2024-01-23 16:53 ` Jani Nikula @ 2024-01-23 18:11 ` Jani Nikula 0 siblings, 0 replies; 12+ messages in thread From: Jani Nikula @ 2024-01-23 18:11 UTC (permalink / raw) To: Akira Yokosawa, Jonathan Corbet, Vegard Nossum Cc: linux-doc, Mauro Carvalho Chehab, Akira Yokosawa On Tue, 23 Jan 2024, Jani Nikula <jani.nikula@linux.intel.com> wrote: > I looked into the root cause here. Sphinx since early versions > (predating even 2.4.4) depends on packages such as > > - sphinxcontrib-applehelp > - sphinxcontrib-devhelp > - sphinxcontrib-htmlhelp > - sphinxcontrib-qthelp > - sphinxcontrib-serializinghtml > > but does so without any version constraints. Recently, all of those > *packages* started first depending on Sphinx >= 5. Apparently this > prevented installation of incompatible combos, but resulted in circular > dependency issues in pip. See: https://github.com/sphinx-doc/sphinx/issues/11567 > To fix the circular dependencies, all of those *packages* made the > dependency on Sphinx >= 5 optional, but made the *extensions* in them > app.require_sphinx('5.0') runtime. This happened in the past two weeks. See: https://github.com/sphinx-doc/sphinx/issues/11890 BR, Jani. -- Jani Nikula, Intel ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2024-01-26 14:42 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-01-23 4:14 docs: requirements.txt has stopped working again Akira Yokosawa 2024-01-23 7:43 ` Vegard Nossum 2024-01-23 12:30 ` Jani Nikula 2024-01-23 13:21 ` Vegard Nossum 2024-01-23 16:19 ` Jonathan Corbet 2024-01-24 15:02 ` Akira Yokosawa 2024-01-24 15:25 ` Jonathan Corbet 2024-01-24 15:43 ` Akira Yokosawa 2024-01-26 14:42 ` Akira Yokosawa 2024-01-24 19:56 ` Vegard Nossum 2024-01-23 16:53 ` Jani Nikula 2024-01-23 18:11 ` Jani Nikula
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).