Re: Python 2.7 support and automarkup.py - Was: Re: [PATCH v2 1/5] docs: automarkup.py: Use new C roles in Sphinx 3

Re: Python 2.7 support and automarkup.py - Was: Re: [PATCH v2 1/5] docs: automarkup.py: Use new C roles in Sphinx 3

[Nícolas F. R. A. Prado]

On Fri Oct 30, 2020 at 11:39 AM -03, Matthew Wilcox wrote:
>
> On Fri, Oct 30, 2020 at 08:14:40AM -0600, Jonathan Corbet wrote:
> > On Fri, 30 Oct 2020 15:10:26 +0100
> > Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> >
> > > I see a few alternatives:
> > >
> > > 1) fix automarkup.py for it to work again with python 2.7;
> > >
> > > 2) conf.py could gain some logic to disable automarkup with
> > >    Python < 3;
> > >
> > > 3) scripts/sphinx-pre-install already detects Python version.
> > >    It should likely be easy to ask the user to use python 3.x,
> > >    if an older version is detected.
> > >
> > > Doing (1) or (2) will require an additional step when we raise
> > > the bar for Python version.
> >
> > We haven't dropped support for Python 2 yet, so this constitutes a
> > regression.  My own approach would be something like this at the top of
> > automarkup.py:
> >
> > 	if python2:
> > 	    ascii = 0
> > 	else:
> > 	    ascii = re.ASCII
> >
> > ...then s/re.ASCII/ascii/ throughout.  I can probably put together
> > something later this morning.
>
> Could we have a warning somewhere that python 2.7 is going to produce
> inferior docs?

But I don't expect the docs to have inferior quality using python 2.7.
The fix proposed by Jon should be enough to solve the issue.

>
> Alternatively, https://docs.python.org/2/library/re.html suggests
> using "The third-party regex module".

I still think Jon's solution would be better, as it doesn't add any additional
dependency.
In the end the issue is just python 2 defaults to ASCII while python 3 defaults
to unicode.

Thanks,
Nícolas

Re: Python 2.7 support and automarkup.py - Was: Re: [PATCH v2 1/5] docs: automarkup.py: Use new C roles in Sphinx 3

[Matthew Wilcox]

On Fri, Oct 30, 2020 at 08:14:40AM -0600, Jonathan Corbet wrote:
> On Fri, 30 Oct 2020 15:10:26 +0100
> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> 
> > I see a few alternatives:
> > 
> > 1) fix automarkup.py for it to work again with python 2.7;
> > 
> > 2) conf.py could gain some logic to disable automarkup with
> >    Python < 3;
> > 
> > 3) scripts/sphinx-pre-install already detects Python version. 
> >    It should likely be easy to ask the user to use python 3.x,
> >    if an older version is detected.
> > 
> > Doing (1) or (2) will require an additional step when we raise
> > the bar for Python version.
> 
> We haven't dropped support for Python 2 yet, so this constitutes a
> regression.  My own approach would be something like this at the top of
> automarkup.py:
> 
> 	if python2:
> 	    ascii = 0
> 	else:
> 	    ascii = re.ASCII
> 
> ...then s/re.ASCII/ascii/ throughout.  I can probably put together
> something later this morning.

Could we have a warning somewhere that python 2.7 is going to produce
inferior docs?

Alternatively, https://docs.python.org/2/library/re.html suggests
using "The third-party regex module".

Re: Python 2.7 support and automarkup.py - Was: Re: [PATCH v2 1/5] docs: automarkup.py: Use new C roles in Sphinx 3

[Mauro Carvalho Chehab]

Em Fri, 30 Oct 2020 08:14:40 -0600
Jonathan Corbet <corbet@lwn.net> escreveu:

> On Fri, 30 Oct 2020 15:10:26 +0100
> Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:
> 
> > I see a few alternatives:
> > 
> > 1) fix automarkup.py for it to work again with python 2.7;
> > 
> > 2) conf.py could gain some logic to disable automarkup with
> >    Python < 3;
> > 
> > 3) scripts/sphinx-pre-install already detects Python version. 
> >    It should likely be easy to ask the user to use python 3.x,
> >    if an older version is detected.
> > 
> > Doing (1) or (2) will require an additional step when we raise
> > the bar for Python version.  
> 
> We haven't dropped support for Python 2 yet, so this constitutes a
> regression.  My own approach would be something like this at the top of
> automarkup.py:
> 
> 	if python2:
> 	    ascii = 0
> 	else:
> 	    ascii = re.ASCII
> 
> ...then s/re.ASCII/ascii/ throughout.  I can probably put together
> something later this morning.

Makes sense.

> 
> jon



Thanks,
Mauro

Re: Python 2.7 support and automarkup.py - Was: Re: [PATCH v2 1/5] docs: automarkup.py: Use new C roles in Sphinx 3

[Jonathan Corbet]

On Fri, 30 Oct 2020 15:10:26 +0100
Mauro Carvalho Chehab <mchehab+huawei@kernel.org> wrote:

> I see a few alternatives:
> 
> 1) fix automarkup.py for it to work again with python 2.7;
> 
> 2) conf.py could gain some logic to disable automarkup with
>    Python < 3;
> 
> 3) scripts/sphinx-pre-install already detects Python version. 
>    It should likely be easy to ask the user to use python 3.x,
>    if an older version is detected.
> 
> Doing (1) or (2) will require an additional step when we raise
> the bar for Python version.

We haven't dropped support for Python 2 yet, so this constitutes a
regression.  My own approach would be something like this at the top of
automarkup.py:

	if python2:
	    ascii = 0
	else:
	    ascii = re.ASCII

...then s/re.ASCII/ascii/ throughout.  I can probably put together
something later this morning.

jon

Python 2.7 support and automarkup.py - Was: Re: [PATCH v2 1/5] docs: automarkup.py: Use new C roles in Sphinx 3

[Mauro Carvalho Chehab]

Hi Dafna,

Em Fri, 30 Oct 2020 14:33:52 +0100
Dafna Hirschfeld <dafna.hirschfeld@collabora.com> escreveu:

> Hi
> 
> Am 14.10.20 um 01:13 schrieb Nícolas F. R. A. Prado:
> > While Sphinx 2 used a single c:type role for struct, union, enum and
> > typedef, Sphinx 3 uses a specific role for each one.
> > To keep backward compatibility, detect the Sphinx version and use the
> > correct roles for that version.
> > 
> > Signed-off-by: Nícolas F. R. A. Prado <nfraprado@protonmail.com>
> > ---
> >   Documentation/sphinx/automarkup.py | 55 ++++++++++++++++++++++++++----
> >   1 file changed, 49 insertions(+), 6 deletions(-)
> > 
> > diff --git a/Documentation/sphinx/automarkup.py b/Documentation/sphinx/automarkup.py
> > index a1b0f554cd82..db13fb15cedc 100644
> > --- a/Documentation/sphinx/automarkup.py
> > +++ b/Documentation/sphinx/automarkup.py
> > @@ -23,7 +23,21 @@ from itertools import chain
> >   # bit tries to restrict matches to things that won't create trouble.
> >   #
> >   RE_function = re.compile(r'(([\w_][\w\d_]+)\(\))')
> > -RE_type = re.compile(r'(struct|union|enum|typedef)\s+([\w_][\w\d_]+)')
> > +
> > +#
> > +# Sphinx 2 uses the same :c:type role for struct, union, enum and typedef
> > +#
> > +RE_generic_type = re.compile(r'(struct|union|enum|typedef)\s+([\w_][\w\d_]+)')
> > +
> > +#
> > +# Sphinx 3 uses a different C role for each one of struct, union, enum and
> > +# typedef
> > +#
> > +RE_struct = re.compile(r'\b(struct)\s+([a-zA-Z_]\w+)', flags=re.ASCII)
> > +RE_union = re.compile(r'\b(union)\s+([a-zA-Z_]\w+)', flags=re.ASCII)
> > +RE_enum = re.compile(r'\b(enum)\s+([a-zA-Z_]\w+)', flags=re.ASCII)
> > +RE_typedef = re.compile(r'\b(typedef)\s+([a-zA-Z_]\w+)', flags=re.ASCII)  
> 
> I use ubuntu 18.04, my default python is 2.7,
> when running 'make htmldocs' with that fix I get:
> 
> AttributeError: 'module' object has no attribute 'ASCII'

FYI, there's a discussion at kernel-doc ML about dropping support for
python 2.7 at Kernel 5.11. While not explicitly mentioned at the
discussion, this is the rationale:

	https://www.python.org/doc/sunset-python-2/

As this is currently broken with Python 2.7, then perhaps we can
do that for 5.10 :-)

Jon,

What do you think? 

I see a few alternatives:

1) fix automarkup.py for it to work again with python 2.7;

2) conf.py could gain some logic to disable automarkup with
   Python < 3;

3) scripts/sphinx-pre-install already detects Python version. 
   It should likely be easy to ask the user to use python 3.x,
   if an older version is detected.

Doing (1) or (2) will require an additional step when we raise
the bar for Python version.

Thanks,
Mauro