From: Jiang Xin <worldhello.net@gmail.com>
To: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
Cc: "Junio C Hamano" <gitster@pobox.com>,
"Git List" <git@vger.kernel.org>,
"Jiang Xin" <zhiyou.jx@alibaba-inc.com>,
"Alexander Shopov" <ash@kambanaria.org>,
"Jordi Mas" <jmas@softcatala.org>,
"Matthias Rüster" <matthias.ruester@gmail.com>,
"Jimmy Angelakos" <vyruss@hellug.gr>,
"Christopher Díaz" <christopher.diaz.riv@gmail.com>,
"Jean-Noël Avila" <jn.avila@free.fr>,
"Bagas Sanjaya" <bagasdotme@gmail.com>,
"Alessandro Menti" <alessandro.menti@alessandromenti.it>,
"Gwan-gyeong Mun" <elongbug@gmail.com>, Arusekk <arek_koz@o2.pl>,
"Daniel Santos" <dacs.git@brilhante.top>,
"Dimitriy Ryazantcev" <DJm00n@mail.ru>,
"Peter Krefting" <peter@softwolves.pp.se>,
"Emir SARI" <bitigchi@me.com>,
"Trần Ngọc Quân" <vnwildman@gmail.com>,
"Fangyi Zhou" <me@fangyi.io>, "Yi-Jyun Pan" <pan93412@gmail.com>
Subject: Re: [PATCH v3 0/9] Incremental po/git.pot update and new l10n workflow
Date: Tue, 24 May 2022 00:13:05 +0800 [thread overview]
Message-ID: <CANYiYbGozxEw6i-dGPPD_Y2CNttnpsa30akstb=C1qzXrq-dfA@mail.gmail.com> (raw)
In-Reply-To: <220523.86sfp0xpoz.gmgdl@evledraar.gmail.com>
On Mon, May 23, 2022 at 10:56 PM Ævar Arnfjörð Bjarmason
<avarab@gmail.com> wrote:
>
>
> On Mon, May 23 2022, Jiang Xin wrote:
>
> > On Mon, May 23, 2022 at 4:19 PM Ævar Arnfjörð Bjarmason
> >> $(LOCALIZED_SH_GEN_PO): .build/pot/po/%.po: %
> >> $(call mkdir_p_parent_template)
> >> @@ -2786,11 +2780,24 @@ sed -e 's|charset=CHARSET|charset=UTF-8|' \
> >> echo '"Plural-Forms: nplurals=INTEGER; plural=EXPRESSION;\\n"' >>$@
> >> endef
> >>
> >> -.build/pot/git.header: $(LOCALIZED_ALL_GEN_PO)
> >> +.build/pot/git.header:
> >
> > No. We should rebuild the pot header if any po file need to be update,
> > because we want to refresh the timestamp in the "POT-Creation-Date:"
> > filed of the pot header.
>
> Okey, I did leave a question about this in an earlier E-Mail though,
> i.e. does anything actually rely on this, or the header at all, or is
> this just cargo-culting?
>
> I haven't found anything in our toolchain that cares about the header at
> all (for the *.pot, not *.po!) let alone the update timestamp.
When creating a new po/XX.po manually using msginit from POT file with
or without a header, the new generated po/XX.po has different header.
$ msginit -i po/git.pot -o po/XX-with-header.po \
--locale=ja --no-translator
$ msginit -i po/git-headless.pot -o po/XX-without-header.po \
--locale=ja --no-translator
$ diff po/XX-with-header.po po/XX-without-header.po
1,5d0
< # Japanese translations for Git package.
< # Copyright (C) 2022 THE Git'S COPYRIGHT HOLDER
< # This file is distributed under the same license as the Git package.
< # Automatically generated, 2022.
< #
8,11c3
< "Project-Id-Version: Git\n"
< "Report-Msgid-Bugs-To: Git Mailing List <git@vger.kernel.org>\n"
< "POT-Creation-Date: 2022-05-23 23:27+0800\n"
< "PO-Revision-Date: 2022-05-23 23:27+0800\n"
---
> "Project-Id-Version: git 2.36.0.7.g31429651cf.dirty\n"
16c8
< "Content-Type: text/plain; charset=UTF-8\n"
---
> "Content-Type: text/plain; charset=ASCII\n"
Should we ignore this change?
>
> >> $(call mkdir_p_parent_template)
> >> $(QUIET_GEN)$(gen_pot_header)
> >>
> >> -po/git.pot: .build/pot/git.header $(LOCALIZED_ALL_GEN_PO)
> >> +# We go through this dance of having a prepared
> >> +# e.g. .build/pot/po/grep.c.po and copying it to
> >> +# .build/pot/to-cat/grep.c only because some IDEs (e.g. VSCode) pick
> >> +# up on the "real" extension for the purposes of auto-completion, even
> >> +# if the .build directiory is in .gitignore.
> >> +LOCALIZED_ALL_GEN_TO_CAT = $(LOCALIZED_ALL_GEN_PO:.build/pot/po/%.po=.build/pot/to-cat/%)
> >> +ifdef AGGRESSIVE_INTERMEDIATE
> >> +.INTERMEDIATE: $(LOCALIZED_ALL_GEN_TO_CAT)
> >> +endif
> >> +$(LOCALIZED_ALL_GEN_TO_CAT): .build/pot/to-cat/%: .build/pot/po/%.po
> >> + $(call mkdir_p_parent_template)
> >> + $(QUIET_GEN)cat $< >$@
> >
> > Copy each po file in ".build/pot/po/" to another location
> > ".build/pot/to-cat/", but without the ".po" extension.
> >
> > Let's take "date.c" as an example:
> >
> > 1. Copy "date.c" to an intermediate C source file
> > ".build/pot/po-munged/date.c" and replace PRItime with PRIuMAX in it.
> >
> > 2. Call xgettext to create ".build/pot/po/date.c.po" from the
> > intermediate C source file ".build/pot/po-munged/date.c".
> >
> > 3. Optionally remove intermediate C source files like
> > ".build/pot/po-munged/date.c". To have two identical C source files in
> > the same worktree is not good, some software may break. So I choose to
> > remove them.
> >
> > 4. Copy the po file (".build/pot/po/date.c.po") created in step 2 to
> > an intermediate fake C source file ".build/pot/to-cat/date.c" which is
> > a file without the ".po" extension. Please note this intermediate fake
> > C source file ".build/pot/to-cat/date.c" is not a valid C file, but a
> > PO file.
> >
> > 5. Call msgcat to create "po/git.pot" from all the intermediate fake C
> > source files including ".build/pot/to-cat/date.c".
> >
> > 6. Optionally remove all the intermediate fake C source files in
> > ".build/pot/to-cat/". I choose to remove them, because leave lots of
> > invalid C source files in worktree is not good.
> >
> > For example, ".build/pot/po/date.c.po" was created from
> >> +
> >> +po/git.pot: .build/pot/git.header $(LOCALIZED_ALL_GEN_TO_CAT)
> >> $(QUIET_GEN)$(MSGCAT) $(MSGCAT_FLAGS) $^ >$@
> >
> > 7. "po/git.pot" depends on the intermediate fake C source files. If
> > any single C source file has been changed, will run step 6 to copy all
> > po files in ".build/pot/po" to corresponding fake C source files in
> > ".build/pot/to-cat/", if we choose to remove these intermediate fake C
> > source files.
> >
> > This implementation is too heavy to solve a trivial issue. I think we
> > can push forward this patch series and leave these comments in
> > "po/git.pot":
>
> If you find it too "heavy" & are trying to optimize it for some reason
> then that whole extra special-dance can be made conditional on
> MAKE_AVOID_REAL_EXTENSIONS_IN_GITIGNORED_FILES.
>
> But really, it's 15MB of .build/pot in my local HEAD with this fix-up,
> it's 1.4MB without it, but this whole thing just seems like premature
> optimization. Especially given:
>
> $ git hyperfine -r 3 -L rev origin/master,HEAD~,HEAD,avar/Makefile-incremental-po-git-pot-rule~,avar/Makefile-incremental-po-git-pot-rule -p 'git clean -dxf; git reset --hard' 'make pot' --warmup 1
> Benchmark 1: make pot' in 'origin/master
> Time (mean ± σ): 1.970 s ± 0.014 s [User: 1.683 s, System: 0.353 s]
> Range (min … max): 1.955 s … 1.982 s 3 runs
>
> Benchmark 2: make pot' in 'HEAD~
> Time (mean ± σ): 931.3 ms ± 4.7 ms [User: 3358.5 ms, System: 1088.7 ms]
> Range (min … max): 927.0 ms … 936.3 ms 3 runs
>
> Benchmark 3: make pot' in 'HEAD
> Time (mean ± σ): 1.506 s ± 0.389 s [User: 4.655 s, System: 1.363 s]
> Range (min … max): 1.257 s … 1.955 s 3 runs
>
> Benchmark 4: make pot' in 'avar/Makefile-incremental-po-git-pot-rule~
> Time (mean ± σ): 1.015 s ± 0.002 s [User: 3.615 s, System: 1.224 s]
> Range (min … max): 1.013 s … 1.017 s 3 runs
>
> Benchmark 5: make pot' in 'avar/Makefile-incremental-po-git-pot-rule
> Time (mean ± σ): 1.014 s ± 0.008 s [User: 3.540 s, System: 1.068 s]
> Range (min … max): 1.007 s … 1.023 s 3 runs
>
> Summary
> 'make pot' in 'HEAD~' ran
> 1.09 ± 0.01 times faster than 'make pot' in 'avar/Makefile-incremental-po-git-pot-rule'
> 1.09 ± 0.01 times faster than 'make pot' in 'avar/Makefile-incremental-po-git-pot-rule~'
> 1.62 ± 0.42 times faster than 'make pot' in 'HEAD'
> 2.12 ± 0.02 times faster than 'make pot' in 'origin/master'
>
> I.e. all of this is much faster than what we have on "master" now. My
> 22434ef36ae (Makefile: avoid "sed" on C files that don't need it,
> 2022-04-08) (avar/Makefile-incremental-po-git-pot-rule) is then just 10%
> slower than the "grep or xgettext", its "~" is the corresponding
> unoptimized.
>
> The HEAD here is with my fix-up, and HEAD~ is your series here.
>
> Anyway, if you really feel strongly about it let's go with your way of
> doing it.
>
> It just sounded like you weren't actually trying top optimize anything,
> but to work around your editor. So if we had a method to do that....
But for users who prefer to delete all the intermediate files in
".build/pot/to-cat" and ".build/pot/po-munged/", then they will get
performance penalty.
> ...except it seems you also care about making it much faster than
> "master" (or care about <20MB of disk space), which to be blunt seems a
> bit crazy to me :) Last I checked "make test" ended up creating ~1GB of
> data (not all at once, but in parallel testing a lot more than 10MB is
> often in play at once).
>
> As this was a pretty obscure target that I only expect CI, you,
> translators & me to run in practice a small difference in the initial
> run didn't seem to matter, especially as it's all an improvement over
> "master".
>
> Anyway, you do whatever you think is best with that :)
>
> >> $ grep '#-#' po/git.pot
> >> #. #-#-#-#-# git-add--interactive.perl.po #-#-#-#-#
> >> #. #-#-#-#-# add-patch.c.po #-#-#-#-#
> >> #. #-#-#-#-# git-add--interactive.perl.po #-#-#-#-#
> >> #. #-#-#-#-# branch.c.po #-#-#-#-#
> >> #. #-#-#-#-# object-name.c.po #-#-#-#-#
> >> #. #-#-#-#-# grep.c.po #-#-#-#-#
>
next prev parent reply other threads:[~2022-05-23 16:13 UTC|newest]
Thread overview: 110+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-05-03 13:23 [PATCH 0/9] Incremental po/git.pot update and new l10n workflow Jiang Xin
2022-05-03 13:23 ` [PATCH 1/9] Makefile: sort "po/git.pot" by file location Jiang Xin
2022-05-03 13:23 ` [PATCH 2/9] Makefile: generate "po/git.pot" from stable LOCALIZED_C Jiang Xin
2022-05-03 13:23 ` [PATCH 3/9] Makefile: have "make pot" not "reset --hard" Jiang Xin
2022-05-03 13:23 ` [PATCH 4/9] i18n CI: stop allowing non-ASCII source messages in po/git.pot Jiang Xin
2022-05-03 13:23 ` [PATCH 5/9] po/git.pot: don't check in result of "make pot" Jiang Xin
2022-05-03 13:23 ` [PATCH 6/9] po/git.pot: remove this now generated file, see preceding commit Jiang Xin
2022-05-03 13:23 ` [PATCH 7/9] Makefile: add "po-update" rule to update po/XX.po Jiang Xin
2022-05-03 13:23 ` [PATCH 8/9] Makefile: add "po-init" rule to initialize po/XX.po Jiang Xin
2022-05-03 13:23 ` [PATCH 9/9] l10n: Document the new l10n workflow Jiang Xin
2022-05-03 14:07 ` [PATCH 0/9] Incremental po/git.pot update and " Peter Krefting
2022-05-04 12:41 ` Jiang Xin
2022-05-04 14:35 ` Junio C Hamano
2022-05-04 14:51 ` Daniel Santos
2022-05-05 0:20 ` Jiang Xin
2022-05-05 22:00 ` Daniel Santos
2022-05-05 22:49 ` Junio C Hamano
2022-05-06 0:50 ` Jiang Xin
2022-05-05 0:07 ` Jiang Xin
2022-05-04 17:58 ` Junio C Hamano
2022-05-19 8:15 ` [PATCH v2 " Jiang Xin
2022-05-19 10:28 ` Ævar Arnfjörð Bjarmason
2022-05-19 14:32 ` Jiang Xin
2022-05-19 14:41 ` Ævar Arnfjörð Bjarmason
2022-05-23 1:25 ` [PATCH v3 " Jiang Xin
2022-05-23 7:15 ` Ævar Arnfjörð Bjarmason
2022-05-23 8:12 ` Ævar Arnfjörð Bjarmason
2022-05-23 13:42 ` Jiang Xin
2022-05-23 14:38 ` Ævar Arnfjörð Bjarmason
2022-05-23 16:13 ` Jiang Xin [this message]
2022-05-23 8:26 ` Jiang Xin
2022-05-23 15:21 ` [PATCH v4 " Jiang Xin
2022-05-23 18:19 ` Junio C Hamano
2022-05-26 14:50 ` [PATCH v5 00/10] " Jiang Xin
2022-05-26 14:50 ` [PATCH v5 01/10] Makefile: sort source files before feeding to xgettext Jiang Xin
2022-05-26 14:50 ` [PATCH v5 02/10] Makefile: generate "po/git.pot" from stable LOCALIZED_C Jiang Xin
2022-05-26 14:50 ` [PATCH v5 03/10] Makefile: have "make pot" not "reset --hard" Jiang Xin
2022-05-26 14:50 ` [PATCH v5 04/10] i18n CI: stop allowing non-ASCII source messages in po/git.pot Jiang Xin
2022-05-26 14:50 ` [PATCH v5 05/10] Makefile: remove duplicate and unwanted files in FOUND_SOURCE_FILES Jiang Xin
2022-05-26 14:50 ` [PATCH v5 06/10] po/git.pot: this is now a generated file Jiang Xin
2022-05-26 17:32 ` Junio C Hamano
2022-05-26 14:50 ` [PATCH v5 07/10] po/git.pot: don't check in result of "make pot" Jiang Xin
2022-05-26 14:50 ` [PATCH v5 08/10] Makefile: add "po-update" rule to update po/XX.po Jiang Xin
2022-05-26 14:50 ` [PATCH v5 09/10] Makefile: add "po-init" rule to initialize po/XX.po Jiang Xin
2022-05-26 14:50 ` [PATCH v5 10/10] l10n: Document the new l10n workflow Jiang Xin
2022-05-23 15:21 ` [PATCH v4 1/9] Makefile: sort "po/git.pot" by file location Jiang Xin
2022-05-23 15:21 ` [PATCH v4 2/9] Makefile: generate "po/git.pot" from stable LOCALIZED_C Jiang Xin
2022-05-23 15:21 ` [PATCH v4 3/9] Makefile: have "make pot" not "reset --hard" Jiang Xin
2022-05-25 22:19 ` Junio C Hamano
2022-05-25 22:24 ` Junio C Hamano
2022-05-26 1:10 ` Jiang Xin
2022-05-26 2:15 ` [PATCH] Makefile: dedup git-ls-files output to prevent duplicate targets Jiang Xin
2022-05-26 4:02 ` Junio C Hamano
2022-05-26 6:06 ` Jiang Xin
2022-05-26 6:23 ` Junio C Hamano
2022-05-26 7:04 ` Jiang Xin
2022-05-26 10:00 ` Ævar Arnfjörð Bjarmason
2022-05-26 11:06 ` Jiang Xin
2022-05-26 17:18 ` Junio C Hamano
2022-05-26 18:25 ` Ævar Arnfjörð Bjarmason
2022-05-26 19:00 ` Junio C Hamano
2022-05-26 19:17 ` Ævar Arnfjörð Bjarmason
2022-05-23 15:21 ` [PATCH v4 4/9] i18n CI: stop allowing non-ASCII source messages in po/git.pot Jiang Xin
2022-05-23 15:21 ` [PATCH v4 5/9] po/git.pot: this is now a generated file Jiang Xin
2022-05-23 15:21 ` [PATCH v4 6/9] po/git.pot: don't check in result of "make pot" Jiang Xin
2022-05-23 15:21 ` [PATCH v4 7/9] Makefile: add "po-update" rule to update po/XX.po Jiang Xin
2022-05-23 15:21 ` [PATCH v4 8/9] Makefile: add "po-init" rule to initialize po/XX.po Jiang Xin
2022-05-23 15:21 ` [PATCH v4 9/9] l10n: Document the new l10n workflow Jiang Xin
2022-05-23 1:25 ` [PATCH v3 1/9] Makefile: sort "po/git.pot" by file location Jiang Xin
2022-05-23 8:05 ` Junio C Hamano
2022-05-23 8:50 ` Jiang Xin
2022-05-23 1:25 ` [PATCH v3 2/9] Makefile: generate "po/git.pot" from stable LOCALIZED_C Jiang Xin
2022-05-23 8:05 ` Junio C Hamano
2022-05-23 1:25 ` [PATCH v3 3/9] Makefile: have "make pot" not "reset --hard" Jiang Xin
2022-05-23 7:28 ` Ævar Arnfjörð Bjarmason
2022-05-23 15:00 ` Jiang Xin
2022-05-24 0:56 ` Jiang Xin
2022-05-23 8:15 ` Junio C Hamano
2022-05-23 9:37 ` Jiang Xin
2022-05-23 1:25 ` [PATCH v3 4/9] i18n CI: stop allowing non-ASCII source messages in po/git.pot Jiang Xin
2022-05-23 1:25 ` [PATCH v3 5/9] po/git.pot: this is now a generated file Jiang Xin
2022-05-23 1:25 ` [PATCH v3 6/9] po/git.pot: don't check in result of "make pot" Jiang Xin
2022-05-23 7:26 ` Ævar Arnfjörð Bjarmason
2022-05-23 8:30 ` Jiang Xin
2022-05-23 8:35 ` Jiang Xin
2022-05-23 9:28 ` Ævar Arnfjörð Bjarmason
2022-05-23 1:25 ` [PATCH v3 7/9] Makefile: add "po-update" rule to update po/XX.po Jiang Xin
2022-05-23 1:25 ` [PATCH v3 8/9] Makefile: add "po-init" rule to initialize po/XX.po Jiang Xin
2022-05-23 1:25 ` [PATCH v3 9/9] l10n: Document the new l10n workflow Jiang Xin
2022-05-19 8:15 ` [PATCH v2 1/9] Makefile: sort "po/git.pot" by file location Jiang Xin
2022-05-19 8:53 ` Ævar Arnfjörð Bjarmason
2022-05-19 12:41 ` Jiang Xin
2022-05-19 8:15 ` [PATCH v2 2/9] Makefile: generate "po/git.pot" from stable LOCALIZED_C Jiang Xin
2022-05-19 9:18 ` Ævar Arnfjörð Bjarmason
2022-05-19 12:48 ` Jiang Xin
2022-05-19 8:15 ` [PATCH v2 3/9] Makefile: have "make pot" not "reset --hard" Jiang Xin
2022-05-19 9:43 ` Ævar Arnfjörð Bjarmason
2022-05-19 13:19 ` Jiang Xin
2022-05-19 14:06 ` Ævar Arnfjörð Bjarmason
2022-05-19 8:15 ` [PATCH v2 4/9] i18n CI: stop allowing non-ASCII source messages in po/git.pot Jiang Xin
2022-05-19 10:02 ` Ævar Arnfjörð Bjarmason
2022-05-19 8:15 ` [PATCH v2 5/9] po/git.pot: this is now a generated file Jiang Xin
2022-05-19 8:15 ` [PATCH v2 6/9] po/git.pot: don't check in result of "make pot" Jiang Xin
2022-05-19 8:15 ` [PATCH v2 7/9] Makefile: add "po-update" rule to update po/XX.po Jiang Xin
2022-05-19 10:07 ` Ævar Arnfjörð Bjarmason
2022-05-19 8:15 ` [PATCH v2 8/9] Makefile: add "po-init" rule to initialize po/XX.po Jiang Xin
2022-05-19 10:22 ` Ævar Arnfjörð Bjarmason
2022-05-19 8:15 ` [PATCH v2 9/9] l10n: Document the new l10n workflow Jiang Xin
2022-05-19 17:18 ` Junio C Hamano
2022-05-21 15:06 ` Jiang Xin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CANYiYbGozxEw6i-dGPPD_Y2CNttnpsa30akstb=C1qzXrq-dfA@mail.gmail.com' \
--to=worldhello.net@gmail.com \
--cc=DJm00n@mail.ru \
--cc=alessandro.menti@alessandromenti.it \
--cc=arek_koz@o2.pl \
--cc=ash@kambanaria.org \
--cc=avarab@gmail.com \
--cc=bagasdotme@gmail.com \
--cc=bitigchi@me.com \
--cc=christopher.diaz.riv@gmail.com \
--cc=dacs.git@brilhante.top \
--cc=elongbug@gmail.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=jmas@softcatala.org \
--cc=jn.avila@free.fr \
--cc=matthias.ruester@gmail.com \
--cc=me@fangyi.io \
--cc=pan93412@gmail.com \
--cc=peter@softwolves.pp.se \
--cc=vnwildman@gmail.com \
--cc=vyruss@hellug.gr \
--cc=zhiyou.jx@alibaba-inc.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).