[PATCH] docs: sphinx-pre-install: Reword warning on installing cjk font

[PATCH] docs: sphinx-pre-install: Reword warning on installing cjk font

[Akira Yokosawa]

Installing a ckj font as recommended by the warning message causes
generated latex code to have:

	% This is needed for translations
	\usepackage{xeCJK}
	\setCJKmainfont{Noto Sans CJK SC}

in its preamble even for an English document.

The package "xeCJK" changes wide characters' appearance including
apostrophe (single quote) and double quotes, and it changes line-break
behavior with regard to the boundary of narrow and wide characters.

This greatly degrades readability of English PDFs typeset by xelatex.

As a band-aid help, reword the message and let the user at least be
aware of the degradation.

Signed-off-by: Akira Yokosawa <akiyks@gmail.com>
---
Hello,

I'm a newcomer to kernel-doc who started trying sphinx this week.
I have both sphinx 2.4.4 and 1.7.9 installed.
For pdfdocs, I use 2.4.4.

"make htmldocs" was fairly easy, but I struggled a while to get
PDFs as I expected.

The culprit turned out to be the "xeCJK" package mentioned above
in the change log.  It appears in all the generated latex files
due to the "Noto Sans CJK" font I have installed seeing the warning
message from sphinx-pre-install.

By uninstalling the font, I can now build the PDF as I expect.

The reworded message is just a band-aid help.

In the long term, I see that rst2pdf tool is your hope to avoid
LaTeX altogether.  But it would be nice if you can enable xeCJK only
for the translations document.  Can sphinx permit such language
choice per subdirectory (or hopefully per .rst file)?

        Thanks, Akira
--
 scripts/sphinx-pre-install | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index fe92020d67e3..b3c7da2b8ad2 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -101,7 +101,12 @@ sub check_missing(%)
 		}
 
 		if ($is_optional) {
-			print "Warning: better to also install \"$prog\".\n";
+			if ($prog =~ /cjk/ ) {
+				print "Warning: For translations PDF, better to install \"$prog\".\n";
+				print "However, doing so will *degrade* English PDF typesetting.\n";
+			} else {
+				print "Warning: better to also install \"$prog\".\n";
+			}
 		} else {
 			print "ERROR: please install \"$prog\", otherwise, build won't work.\n";
 		}
-- 
2.17.1

Re: [PATCH] docs: sphinx-pre-install: Reword warning on installing cjk font

[Mauro Carvalho Chehab]

HI Akira,

Em Fri, 21 May 2021 16:14:19 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:

> Installing a ckj font as recommended by the warning message causes
> generated latex code to have:
> 
> 	% This is needed for translations
> 	\usepackage{xeCJK}
> 	\setCJKmainfont{Noto Sans CJK SC}
> 
> in its preamble even for an English document.

Yes. The same LaTeX configuration is applied to all documents.

While the standard Sphinx logic allows just one conf.py, there's
a logic on Linux that allows a per-directory configuration.
Perhaps it would be possible to set the font just for translations.

Yet, this can't be easily done per-translation - Italian
translation for instance doesn't need CJK fonts.

> The package "xeCJK" changes wide characters' appearance including
> apostrophe (single quote) and double quotes, and it changes line-break
> behavior with regard to the boundary of narrow and wide characters.
> 
> This greatly degrades readability of English PDFs typeset by xelatex.

Hmm... could you give an example where it looks ugly?

At least on the documents I use to check the PDF output, I was unable
to see any big issue.

Regards,
Mauro

> 
> As a band-aid help, reword the message and let the user at least be
> aware of the degradation.
> 
> Signed-off-by: Akira Yokosawa <akiyks@gmail.com>
> ---
> Hello,
> 
> I'm a newcomer to kernel-doc who started trying sphinx this week.
> I have both sphinx 2.4.4 and 1.7.9 installed.
> For pdfdocs, I use 2.4.4.
> 
> "make htmldocs" was fairly easy, but I struggled a while to get
> PDFs as I expected.
> 
> The culprit turned out to be the "xeCJK" package mentioned above
> in the change log.  It appears in all the generated latex files
> due to the "Noto Sans CJK" font I have installed seeing the warning
> message from sphinx-pre-install.
> 
> By uninstalling the font, I can now build the PDF as I expect.
> 
> The reworded message is just a band-aid help.
> 
> In the long term, I see that rst2pdf tool is your hope to avoid
> LaTeX altogether.  But it would be nice if you can enable xeCJK only
> for the translations document.  Can sphinx permit such language
> choice per subdirectory (or hopefully per .rst file)?
> 
>         Thanks, Akira
> --
>  scripts/sphinx-pre-install | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
> index fe92020d67e3..b3c7da2b8ad2 100755
> --- a/scripts/sphinx-pre-install
> +++ b/scripts/sphinx-pre-install
> @@ -101,7 +101,12 @@ sub check_missing(%)
>  		}
>  
>  		if ($is_optional) {
> -			print "Warning: better to also install \"$prog\".\n";
> +			if ($prog =~ /cjk/ ) {
> +				print "Warning: For translations PDF, better to install \"$prog\".\n";
> +				print "However, doing so will *degrade* English PDF typesetting.\n";
> +			} else {
> +				print "Warning: better to also install \"$prog\".\n";
> +			}
>  		} else {
>  			print "ERROR: please install \"$prog\", otherwise, build won't work.\n";
>  		}



Thanks,
Mauro

Re: [PATCH] docs: sphinx-pre-install: Reword warning on installing cjk font

[Akira Yokosawa]

[-- Attachment #1: Type: text/plain, Size: 2070 bytes --]

Hi Mauro,

On Fri, 21 May 2021 09:54:42 +0200, Mauro Carvalho Chehab wrote:
> HI Akira,
> 
> Em Fri, 21 May 2021 16:14:19 +0900
> Akira Yokosawa <akiyks@gmail.com> escreveu:
> 
>> Installing a ckj font as recommended by the warning message causes
>> generated latex code to have:
>>
>> 	% This is needed for translations
>> 	\usepackage{xeCJK}
>> 	\setCJKmainfont{Noto Sans CJK SC}
>>
>> in its preamble even for an English document.
> 
> Yes. The same LaTeX configuration is applied to all documents.
> 
> While the standard Sphinx logic allows just one conf.py, there's
> a logic on Linux that allows a per-directory configuration.
> Perhaps it would be possible to set the font just for translations.
> 
> Yet, this can't be easily done per-translation - Italian
> translation for instance doesn't need CJK fonts.

Yes, the Italian part looks ugly with xeCJK.

> 
>> The package "xeCJK" changes wide characters' appearance including
>> apostrophe (single quote) and double quotes, and it changes line-break
>> behavior with regard to the boundary of narrow and wide characters.
>>
>> This greatly degrades readability of English PDFs typeset by xelatex.
> 
> Hmm... could you give an example where it looks ugly?
> 
> At least on the documents I use to check the PDF output, I was unable
> to see any big issue.

Appended are screenshots from RCU.pdf built with and without xeCJK.

They are built on Ubuntu Bionic based container with sphinx 2.4.4.

I think you can see the difference of how apostrophes are rendered.
Line-break points are also affected by the widths of apostrophes.

Can you spot the difference?

BTW, on current docs-next, wich the CJK font installed, "make pdfdocs"
stops while building s390.pdf.

I needed to manually run "make latexdocs", then run
"latexmk -xelatex RCU.tex" under Documentation/output/latex/ to get
RCU.pdf.

        Thanks, Akira

> 
> Regards,
> Mauro
> 
>>
>> As a band-aid help, reword the message and let the user at least be
>> aware of the degradation.
>>
>> Signed-off-by: Akira Yokosawa <akiyks@gmail.com>
>> ---
[...]

[-- Attachment #2: RCU_AND_LOCKDEP_CHECKING-with-xeCJK.png --]
[-- Type: image/png, Size: 64255 bytes --]

[-- Attachment #3: RCU_AND_LOCKDEP_CHECKING-no_xeCJK.png --]
[-- Type: image/png, Size: 63708 bytes --]

Re: [PATCH] docs: sphinx-pre-install: Reword warning on installing cjk font

[Mauro Carvalho Chehab]

Em Fri, 21 May 2021 18:00:00 +0900
Akira Yokosawa <akiyks@gmail.com> escreveu:

> Hi Mauro,
> 
> On Fri, 21 May 2021 09:54:42 +0200, Mauro Carvalho Chehab wrote:
> > HI Akira,
> > 
> > Em Fri, 21 May 2021 16:14:19 +0900
> > Akira Yokosawa <akiyks@gmail.com> escreveu:
> >   
> >> Installing a ckj font as recommended by the warning message causes
> >> generated latex code to have:
> >>
> >> 	% This is needed for translations
> >> 	\usepackage{xeCJK}
> >> 	\setCJKmainfont{Noto Sans CJK SC}
> >>
> >> in its preamble even for an English document.  
> > 
> > Yes. The same LaTeX configuration is applied to all documents.
> > 
> > While the standard Sphinx logic allows just one conf.py, there's
> > a logic on Linux that allows a per-directory configuration.
> > Perhaps it would be possible to set the font just for translations.
> > 
> > Yet, this can't be easily done per-translation - Italian
> > translation for instance doesn't need CJK fonts.  
> 
> Yes, the Italian part looks ugly with xeCJK.
> 
> >   
> >> The package "xeCJK" changes wide characters' appearance including
> >> apostrophe (single quote) and double quotes, and it changes line-break
> >> behavior with regard to the boundary of narrow and wide characters.
> >>
> >> This greatly degrades readability of English PDFs typeset by xelatex.  
> > 
> > Hmm... could you give an example where it looks ugly?
> > 
> > At least on the documents I use to check the PDF output, I was unable
> > to see any big issue.  
> 
> Appended are screenshots from RCU.pdf built with and without xeCJK.
> 
> They are built on Ubuntu Bionic based container with sphinx 2.4.4.
> 
> I think you can see the difference of how apostrophes are rendered.
> Line-break points are also affected by the widths of apostrophes.
> 
> Can you spot the difference?

Ok, now I understand what you're meaning. We need to double check
what's wrong there, as it doesn't make much sense to have a
"`  " character instead of "`" on those places, nor to change
the word's hyphenation logic.

> 
> BTW, on current docs-next, wich the CJK font installed, "make pdfdocs"
> stops while building s390.pdf.

That's weird.

> I needed to manually run "make latexdocs", then run
> "latexmk -xelatex RCU.tex" under Documentation/output/latex/ to get
> RCU.pdf.

Well, you can pass some options to latexmk when building a
pdf via an environment var (LATEXMKOPTS), like (untested):

	LATEXMKOPTS="-interaction=nonstopmode" make pdfdocs.

or change it to interactive mode, in order to show what part
of the s390.tex is causing the issue.

Thanks,
Mauro

Re: [PATCH] docs: sphinx-pre-install: Reword warning on installing cjk font

[Akira Yokosawa]

On Fri, 21 May 2021 14:19:52 +0200, Mauro Carvalho Chehab wrote:
> Em Fri, 21 May 2021 18:00:00 +0900
> Akira Yokosawa <akiyks@gmail.com> escreveu:
> 
>> Hi Mauro,
>>
>> On Fri, 21 May 2021 09:54:42 +0200, Mauro Carvalho Chehab wrote:
>>> HI Akira,
>>>
>>> Em Fri, 21 May 2021 16:14:19 +0900
>>> Akira Yokosawa <akiyks@gmail.com> escreveu:
>>>   
>>>> Installing a ckj font as recommended by the warning message causes
>>>> generated latex code to have:
>>>>
>>>> 	% This is needed for translations
>>>> 	\usepackage{xeCJK}
>>>> 	\setCJKmainfont{Noto Sans CJK SC}
>>>>
>>>> in its preamble even for an English document.  
>>>
>>> Yes. The same LaTeX configuration is applied to all documents.
>>>
>>> While the standard Sphinx logic allows just one conf.py, there's
>>> a logic on Linux that allows a per-directory configuration.
>>> Perhaps it would be possible to set the font just for translations.
>>>
>>> Yet, this can't be easily done per-translation - Italian
>>> translation for instance doesn't need CJK fonts.  
>>
>> Yes, the Italian part looks ugly with xeCJK.
>>
>>>   
>>>> The package "xeCJK" changes wide characters' appearance including
>>>> apostrophe (single quote) and double quotes, and it changes line-break
>>>> behavior with regard to the boundary of narrow and wide characters.
>>>>
>>>> This greatly degrades readability of English PDFs typeset by xelatex.  
>>>
>>> Hmm... could you give an example where it looks ugly?
>>>
>>> At least on the documents I use to check the PDF output, I was unable
>>> to see any big issue.  
>>
>> Appended are screenshots from RCU.pdf built with and without xeCJK.
>>
>> They are built on Ubuntu Bionic based container with sphinx 2.4.4.
>>
>> I think you can see the difference of how apostrophes are rendered.
>> Line-break points are also affected by the widths of apostrophes.
>>
>> Can you spot the difference?
> 
> Ok, now I understand what you're meaning. We need to double check
> what's wrong there, as it doesn't make much sense to have a
> "`  " character instead of "`" on those places, nor to change
> the word's hyphenation logic.

No, I'm afraid you don't get the point yet.

The point is, just commenting out the lines:

 	% This is needed for translations
% 	\usepackage{xeCJK}
% 	\setCJKmainfont{Noto Sans CJK SC}

, xelatex renders both UTF8 and ASCII apostrophe characters in the
same way.

On the contrary, with xeCJK, UTF and ASCII code are handled
differently.
The reason is that in CJK typesetting, line breaks are permitted almost
anywhere in the text, with a few exceptions specific to each language.

So, sphinx is doing nothing wrong when it converts ASCII apostrophe in
.rst into the UTF8 in latex code.

Enabling xeCJK in English document is the very wrong thing to do in the
first place.

I hope I have made my point clear enough.

        Thanks, Akira

> 
>>
>> BTW, on current docs-next, wich the CJK font installed, "make pdfdocs"
>> stops while building s390.pdf.
> 
> That's weird.
> 
>> I needed to manually run "make latexdocs", then run
>> "latexmk -xelatex RCU.tex" under Documentation/output/latex/ to get
>> RCU.pdf.
> 
> Well, you can pass some options to latexmk when building a
> pdf via an environment var (LATEXMKOPTS), like (untested):
> 
> 	LATEXMKOPTS="-interaction=nonstopmode" make pdfdocs.
> 
> or change it to interactive mode, in order to show what part
> of the s390.tex is causing the issue.
> 
> Thanks,
> Mauro
>