All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Ævar Arnfjörð Bjarmason" <avarab@gmail.com>
To: Jiang Xin <worldhello.net@gmail.com>
Cc: "Junio C Hamano" <gitster@pobox.com>,
	"Git List" <git@vger.kernel.org>,
	"Alexander Shopov" <ash@kambanaria.org>,
	"Jordi Mas" <jmas@softcatala.org>,
	"Matthias Rüster" <matthias.ruester@gmail.com>,
	"Jimmy Angelakos" <vyruss@hellug.gr>,
	"Christopher Díaz" <christopher.diaz.riv@gmail.com>,
	"Jean-Noël Avila" <jn.avila@free.fr>,
	"Bagas Sanjaya" <bagasdotme@gmail.com>,
	"Alessandro Menti" <alessandro.menti@alessandromenti.it>,
	"Gwan-gyeong Mun" <elongbug@gmail.com>, Arusekk <arek_koz@o2.pl>,
	"Daniel Santos" <dacs.git@brilhante.top>,
	"Dimitriy Ryazantcev" <DJm00n@mail.ru>,
	"Peter Krefting" <peter@softwolves.pp.se>,
	"Emir SARI" <bitigchi@me.com>,
	"Trần Ngọc Quân" <vnwildman@gmail.com>,
	"Fangyi Zhou" <me@fangyi.io>, "Yi-Jyun Pan" <pan93412@gmail.com>,
	"Jiang Xin" <zhiyou.jx@alibaba-inc.com>
Subject: Re: [PATCH v2 3/9] Makefile: have "make pot" not "reset --hard"
Date: Thu, 19 May 2022 11:43:54 +0200	[thread overview]
Message-ID: <220519.86leux6e82.gmgdl@evledraar.gmail.com> (raw)
In-Reply-To: <20220519081548.3380-4-worldhello.net@gmail.com>


On Thu, May 19 2022, Jiang Xin wrote:

> From: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
>
> Before commit fc0fd5b23b (Makefile: help gettext tools to cope with our
> custom PRItime format, 2017-07-20) we'd consider source files as-is
> with gettext, but because we need to understand PRItime in the same way
> that gettext itself understands PRIuMAX we'd first check if we had a
> clean checkout, then munge all of the processed files in-place with
> "sed", generate "po/git.pot", and then finally "reset --hard" to undo
> our changes.
>
> By generating "pot" snippets in ".build/pot/po" for each source file
> and rewriting certain source files with PRItime macros to temporary
> files in ".build/pot/po", we can avoid running "make pot" by altering
> files in place and doing a "reset --hard" afterwards.
>
> This speed of "make pot" is slower than before on an initial run,
> because we run "xgettext" many times (once per source file), but it
> can be boosted by parallelization. It is *much* faster for incremental
> runs, and will allow us to implement related targets in subsequent
> commits.
>
> When the "pot" target was originally added in cd5513a7168 (i18n:
> Makefile: "pot" target to extract messages marked for translation,
> 2011-02-22) it behaved like a "normal" target. I.e. we'd skip the
> re-generation of the po/git.pot if nothing had to be done.
>
> Then after po/git.pot was checked in in dce37b66fb0 (l10n: initial
> git.pot for 1.7.10 upcoming release, 2012-02-13) the target was broken
> until 1f31963e921 (i18n: treat "make pot" as an explicitly-invoked
> target, 2014-08-22) when it was made to depend on "FORCE". I.e. the
> Makefile's dependency resolution inherently can't handle incremental
> building when the target file may be updated by git (or something else
> external to "make"). But this case no longer applies, so FORCE is no
> longer needed.
>
> That out of the way, the main logic change here is getting rid of the
> "reset --hard":
>
> We'll generate intermediate .build/pot/po/%.po files from %, which is
> handy to see at a glance what strings (if any) in a given file are
> marked for translation:
>
> 	$ make .build/pot/po/pretty.c.po
> 	[...]
> 	$ cat .build/pot/po/pretty.c.po
> 	#: pretty.c:1051
> 	msgid "unable to parse --pretty format"
> 	msgstr ""
> 	$
>
> For these C source files which contain the PRItime macros, we will
> create temporary munged *.c files in a tree in ".build/pot/po"
> corresponding to our source tree, and have "xgettext" consider those.
> The rule needs to be careful to "(cd .build/pot/po && ...)", because
> otherwise the comments in the po/git.pot file wouldn't refer to the
> correct source locations (they'd be prefixed with ".build/pot/po").
>
> This changes the output of the generated po/git.pot file in one minor
> way: Because we're using msgcat(1) instead of xgettext(1) to
> concatenate the output we'll now disambiguate where "TRANSLATORS"
> comments come from, in cases where a message is the same in N files,
> and either only one has a "TRANSLATORS" comment, or they're
> different. E.g. for the "Your edited hunk[...]" message we'll now
> apply this change (comment content elided):
>
> 	+#. #-#-#-#-#  add-patch.c.po  #-#-#-#-#
> 	 #. TRANSLATORS: do not translate [y/n]
> 	[...]
> 	+#. #-#-#-#-#  git-add--interactive.perl.po  #-#-#-#-#
> 	 #. TRANSLATORS: do not translate [y/n]
> 	[...]
> 	 #: add-patch.c:1253 git-add--interactive.perl:1244
> 	 msgid ""
> 	 "Your edited hunk does not apply. Edit again (saying \"no\" discards!) [y/n]? "
> 	 msgstr ""
>
> There are six such changes, and they all make the context more
> understandable, as msgcat(1) is better at handling these edge cases
> than xgettext(1)'s previously used "--join-existing" flag.
>
> While we could rename the "pot" snippets without the ".po" extention
> to use more intuitive filenames in the comments, but that will
> confuse the IDE with lots of invalid C or perl source files in
> ".build/pot/po" directory.
>
> The addition of "--omit-header" option for xgettext makes the "pot"
> snippets in ".build/pot/po/*.po" smaller. For the pot header of
> "po/git.pot", we use xgettext to generate a "pot" header file
> ".build/pot/git.header" from an empty file at runtime, and use this
> header to assemble "po/git.pot".
>
> But as we'll see in a subsequent commit this header behavior has been
> hiding an encoding-related bug from us, so let's carry it forward
> instead of re-generating it with xgettext(1).
>
> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
> Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
>  .gitignore |  1 +
>  Makefile   | 73 ++++++++++++++++++++++++++++++++++++++----------------
>  2 files changed, 53 insertions(+), 21 deletions(-)
>
> diff --git a/.gitignore b/.gitignore
> index e81de1063a..a452215764 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -200,6 +200,7 @@
>  *.[aos]
>  *.o.json
>  *.py[co]
> +.build/
>  .depend/
>  *.gcda
>  *.gcno
> diff --git a/Makefile b/Makefile
> index 60ca42c268..c32ac4ca30 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -569,6 +569,7 @@ INSTALL = install
>  TCL_PATH = tclsh
>  TCLTK_PATH = wish
>  XGETTEXT = xgettext
> +MSGCAT = msgcat
>  MSGFMT = msgfmt
>  CURL_CONFIG = curl-config
>  GCOV = gcov
> @@ -2706,6 +2707,7 @@ XGETTEXT_FLAGS = \
>  	--force-po \
>  	--add-comments=TRANSLATORS: \
>  	--msgid-bugs-address="Git Mailing List <git@vger.kernel.org>" \
> +	--package-name=Git \
>  	--sort-by-file \
>  	--from-code=UTF-8
>  XGETTEXT_FLAGS_C = $(XGETTEXT_FLAGS) --language=C \
> @@ -2714,6 +2716,7 @@ XGETTEXT_FLAGS_SH = $(XGETTEXT_FLAGS) --language=Shell \
>  	--keyword=gettextln --keyword=eval_gettextln
>  XGETTEXT_FLAGS_PERL = $(XGETTEXT_FLAGS) --language=Perl \
>  	--keyword=__ --keyword=N__ --keyword="__n:1,2"
> +MSGCAT_FLAGS = --sort-by-file
>  LOCALIZED_C = $(FOUND_C_SOURCES) $(SCALAR_SOURCES) \
>  	      $(FOUND_H_SOURCES) $(GENERATED_H)
>  LOCALIZED_SH = $(SCRIPT_SH)
> @@ -2726,34 +2729,61 @@ LOCALIZED_SH += t/t0200/test.sh
>  LOCALIZED_PERL += t/t0200/test.perl
>  endif
>  
> -## Note that this is meant to be run only by the localization coordinator
> -## under a very controlled condition, i.e. (1) it is to be run in a
> -## Git repository (not a tarball extract), (2) any local modifications
> -## will be lost.
> +## We generate intermediate .build/pot/po/%.po files containing a
> +## extract of the translations we find in each file in the source
> +## tree. We will assemble them using msgcat to create the final
> +## "po/git.pot" file.
> +LOCALIZED_ALL_GEN_PO =
> +
> +LOCALIZED_C_GEN_PO = $(LOCALIZED_C:%=.build/pot/po/%.po)
> +LOCALIZED_ALL_GEN_PO += $(LOCALIZED_C_GEN_PO)
> +
> +LOCALIZED_SH_GEN_PO = $(LOCALIZED_SH:%=.build/pot/po/%.po)
> +LOCALIZED_ALL_GEN_PO += $(LOCALIZED_SH_GEN_PO)
> +
> +LOCALIZED_PERL_GEN_PO = $(LOCALIZED_PERL:%=.build/pot/po/%.po)
> +LOCALIZED_ALL_GEN_PO += $(LOCALIZED_PERL_GEN_PO)
> +
>  ## Gettext tools cannot work with our own custom PRItime type, so
>  ## we replace PRItime with PRIuMAX.  We need to update this to
>  ## PRIdMAX if we switch to a signed type later.
> +$(LOCALIZED_C_GEN_PO): .build/pot/po/%.po: %
> +	$(call mkdir_p_parent_template)
> +	$(QUIET_XGETTEXT) \
> +	    if grep -q PRItime $<; then \
> +		(\
> +			sed -e 's|PRItime|PRIuMAX|g' <$< \
> +				>.build/pot/po/$< && \
> +			cd .build/pot/po && \
> +			$(XGETTEXT) --omit-header \
> +				-o $(@:.build/pot/po/%=%) \
> +				$(XGETTEXT_FLAGS_C) $< && \
> +			rm $<; \
> +		); \
> +	    else \
> +		$(XGETTEXT) --omit-header \
> +			-o $@ $(XGETTEXT_FLAGS_C) $<; \
> +	    fi
>  
> -po/git.pot: $(GENERATED_H) FORCE
> -	# All modifications will be reverted at the end, so we do not
> -	# want to have any local change.
> -	git diff --quiet HEAD && git diff --quiet --cached
> +$(LOCALIZED_SH_GEN_PO): .build/pot/po/%.po: %
> +	$(call mkdir_p_parent_template)
> +	$(QUIET_XGETTEXT)$(XGETTEXT) --omit-header \
> +		-o$@ $(XGETTEXT_FLAGS_SH) $<
>  
> -	@for s in $(LOCALIZED_C) $(LOCALIZED_SH) $(LOCALIZED_PERL); \
> -	do \
> -		sed -e 's|PRItime|PRIuMAX|g' <"$$s" >"$$s+" && \
> -		cat "$$s+" >"$$s" && rm "$$s+"; \
> -	done
> +$(LOCALIZED_PERL_GEN_PO): .build/pot/po/%.po: %
> +	$(call mkdir_p_parent_template)
> +	$(QUIET_XGETTEXT)$(XGETTEXT) --omit-header \
> +		-o$@ $(XGETTEXT_FLAGS_PERL) $<
>  
> -	$(QUIET_XGETTEXT)$(XGETTEXT) -o$@+ $(XGETTEXT_FLAGS_C) $(LOCALIZED_C)
> -	$(QUIET_XGETTEXT)$(XGETTEXT) -o$@+ --join-existing $(XGETTEXT_FLAGS_SH) \
> -		$(LOCALIZED_SH)
> -	$(QUIET_XGETTEXT)$(XGETTEXT) -o$@+ --join-existing $(XGETTEXT_FLAGS_PERL) \
> -		$(LOCALIZED_PERL)
> +.build/pot/git.header: $(LOCALIZED_ALL_GEN_PO)
> +	$(call mkdir_p_parent_template)
> +	$(QUIET_XGETTEXT)$(XGETTEXT) $(XGETTEXT_FLAGS_C) \
> +		-o - /dev/null | \
> +	sed -e 's|charset=CHARSET|charset=UTF-8|g' >$@ && \
> +	echo '"Plural-Forms: nplurals=INTEGER; plural=EXPRESSION;\\n"' >>$@
>  
> -	# Reverting the munged source, leaving only the updated $@
> -	git reset --hard
> -	mv $@+ $@
> +po/git.pot: .build/pot/git.header $(LOCALIZED_ALL_GEN_PO)
> +	$(QUIET_GEN)$(MSGCAT) $(MSGCAT_FLAGS) $^ >$@
>  
>  .PHONY: pot
>  pot: po/git.pot
> @@ -3292,6 +3322,7 @@ cocciclean:
>  	$(RM) contrib/coccinelle/*.cocci.patch*
>  
>  clean: profile-clean coverage-clean cocciclean
> +	$(RM) -r .build
>  	$(RM) *.res
>  	$(RM) $(OBJECTS)
>  	$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB)

In the previous discussion of kicking things around I lost track of what
version of mine this is picked up from, but I range-diff'd it to my
6cf9c1f7022 (Makefile: have "make pot" not "reset --hard", 2022-04-02),
which is the latest I had in avar/Makefile-incremental-po-git-pot-rule
on my branch.

A range-diff of the two follows below (yours being the RHS). Some
comments:

 * There's a bug here where you're creating .build/pot/po/pretty.c.po
   files, not .build/pot/po/pretty.c, i.e. you add a *.po extension.

   As the commit message diff shows that results in us adding comments
   to po/git.pot that reference the filename as e.g. add-patch.c.po
   instead of the correct add-patch.c.

   So while it sucks to have a pretty.c file in .build/ that's really
   just a partial *.po instead of one with a *.po extension the whole
   excercise of us sub-shelling into .build/pot/ etc. is to work around
   msgcat's and xgettext's lack specifying a custom "trust me, this is
   the filename", so we should make it *.c again.

   (We could also post-munge the file, but that would suck more)

 * We went a bit back & forth on the "if grep -q PRItime" part on the GH
   ticket. FWIW I still think just skipping that work is a better
   choice. Yes we'll have ~10MB of redundant files in .build, and it's
   marginally slower, but "make pot" isn't a hot target, better to
   optimize for simplicity.

   But if you're really set on having it I don't mind...

 * You add a "MSGCAT_FLAGS = --sort-by-file" here, maybe worth having
   some "common" flags variable in the earlier commit we can use here?
   I.e. share --sort-by-file with xgettext.

 * Your version is missing FORCE on po/git.pot, which is a bug. We can't
   omit it on any file that's checked in. We're about to "git rm" it
   anyway, so maybe we shouldn't worry about it though...

 * This commit breaks the non-ASCII msgid, but 4/9 is about to fix it,
   so likewise maybe not worth worrying about re-arranging them...

1:  6cf9c1f7022 ! 1:  c569d1cd688 Makefile: have "make pot" not "reset --hard"
    @@ Metadata
      ## Commit message ##
         Makefile: have "make pot" not "reset --hard"
     
    -    Change "make pot" to work on copies of the working tree files in a new
    -    ".build/pot/" directory, instead of altering them in place and doing a
    -    "reset --hard" afterwards.
    -
    -    This both speeds up the "make pot" target on an initial run, and makes
    -    it *much* faster for incremental runs, and will allow us to implement
    -    related targets in subsequent commits.
    +    Before commit fc0fd5b23b (Makefile: help gettext tools to cope with our
    +    custom PRItime format, 2017-07-20) we'd consider source files as-is
    +    with gettext, but because we need to understand PRItime in the same way
    +    that gettext itself understands PRIuMAX we'd first check if we had a
    +    clean checkout, then munge all of the processed files in-place with
    +    "sed", generate "po/git.pot", and then finally "reset --hard" to undo
    +    our changes.
    +
    +    By generating "pot" snippets in ".build/pot/po" for each source file
    +    and rewriting certain source files with PRItime macros to temporary
    +    files in ".build/pot/po", we can avoid running "make pot" by altering
    +    files in place and doing a "reset --hard" afterwards.
    +
    +    This speed of "make pot" is slower than before on an initial run,
    +    because we run "xgettext" many times (once per source file), but it
    +    can be boosted by parallelization. It is *much* faster for incremental
    +    runs, and will allow us to implement related targets in subsequent
    +    commits.
     
         When the "pot" target was originally added in cd5513a7168 (i18n:
         Makefile: "pot" target to extract messages marked for translation,
    @@ Commit message
         target, 2014-08-22) when it was made to depend on "FORCE". I.e. the
         Makefile's dependency resolution inherently can't handle incremental
         building when the target file may be updated by git (or something else
    -    external to "make").
    -
    -    So we still need a FORCE on po/git.pot, but now we'll have that target
    -    copy ".build/pot/git.pot" to "po/git.pot", and the former doesn't need
    -    FORCE.
    -
    -    I think 1f31963e921 was confused in the rationale for adding "FORCE"
    -    though, it's needed for the reasons noted above, but it's got nothing
    -    to do with how we do C compilation or find headers, which it suggest
    -    is the case. It was also made as a preparatory change for
    -    d85b0dff729 (Makefile: use `find` to determine static header
    -    dependencies, 2014-08-25).
    -
    -    Unlike the compiler, when we generate the "pot "file we won't recurse
    -    from one header file to another, we only consider the specific targets
    -    we're given, so if we miss a bar.h included via foo.h $(LOCALIZED_C)
    -    the FORCE won't help us.
    +    external to "make"). But this case no longer applies, so FORCE is no
    +    longer needed.
     
         That out of the way, the main logic change here is getting rid of the
         "reset --hard":
     
    -    Before fc0fd5b23bc we'd consider source files as-is with gettext, but
    -    because we need to understand PRItime in the same way that gettext
    -    itself understands PRIuMAX we'd first check if we had a clean
    -    checkout, then munge all of the processed files in-place with "sed",
    -    generate "po/git.pot, and then finally "reset --hard" to undo our
    -    changes.
    -
    -    We didn't need to do this for the *.perl and *.sh files, so let's
    -    skip that and only do it for the *.[ch] files.
    -
    -    Let's then create these munged *.[ch] files in a tree in .build/pot/"
    -    corresponding to our source tree, and have "xgettext" consider
    -    those. The rule needs to be careful to "(cd .build/pot && ...)",
    -    because otherwise the comments in the po/git.pot file wouldn't refer
    -    to the correct source locations (they'd be prefixed with
    -    ".build/git/").
    -
    -    We'll then generate intermediate .build/pot/po/% files from %, which
    -    is handy to see at a glance what strings (if any) in a given file are
    +    We'll generate intermediate .build/pot/po/%.po files from %, which is
    +    handy to see at a glance what strings (if any) in a given file are
         marked for translation:
     
    -            $ make .build/pot/po/pretty.c
    +            $ make .build/pot/po/pretty.c.po
                 [...]
    -            $ cat .build/pot/po/pretty.c
    +            $ cat .build/pot/po/pretty.c.po
                 #: pretty.c:1051
                 msgid "unable to parse --pretty format"
                 msgstr ""
                 $
     
    +    For these C source files which contain the PRItime macros, we will
    +    create temporary munged *.c files in a tree in ".build/pot/po"
    +    corresponding to our source tree, and have "xgettext" consider those.
    +    The rule needs to be careful to "(cd .build/pot/po && ...)", because
    +    otherwise the comments in the po/git.pot file wouldn't refer to the
    +    correct source locations (they'd be prefixed with ".build/pot/po").
    +
         This changes the output of the generated po/git.pot file in one minor
         way: Because we're using msgcat(1) instead of xgettext(1) to
         concatenate the output we'll now disambiguate where "TRANSLATORS"
    @@ Commit message
         different. E.g. for the "Your edited hunk[...]" message we'll now
         apply this change (comment content elided):
     
    -            +#. #-#-#-#-#  add-patch.c  #-#-#-#-#
    +            +#. #-#-#-#-#  add-patch.c.po  #-#-#-#-#
                  #. TRANSLATORS: do not translate [y/n]
                 [...]
    -            +#. #-#-#-#-#  git-add--interactive.perl  #-#-#-#-#
    +            +#. #-#-#-#-#  git-add--interactive.perl.po  #-#-#-#-#
                  #. TRANSLATORS: do not translate [y/n]
                 [...]
                  #: add-patch.c:1253 git-add--interactive.perl:1244
    @@ Commit message
                  "Your edited hunk does not apply. Edit again (saying \"no\" discards!) [y/n]? "
                  msgstr ""
     
    -    There are three such changes, and they all make the context more
    +    There are six such changes, and they all make the context more
         understandable, as msgcat(1) is better at handling these edge cases
         than xgettext(1)'s previously used "--join-existing" flag.
     
    -    The removal of the "--msgid-bugs-address" here and the addition of
    -    "--omit-header" and ".build/pot/pot.header" is because we'll now
    -    always carry forward the existing po/git.pot header.
    +    While we could rename the "pot" snippets without the ".po" extention
    +    to use more intuitive filenames in the comments, but that will
    +    confuse the IDE with lots of invalid C or perl source files in
    +    ".build/pot/po" directory.
     
    -    We'd previously generate the header with xgettext(1), but now we'll be
    -    carrying forward whatever header we find in po/git.pot. Right now this
    -    doesn't matter, we'll even update the timestamp in the header as
    -    before when running "make pot".
    +    The addition of "--omit-header" option for xgettext makes the "pot"
    +    snippets in ".build/pot/po/*.po" smaller. For the pot header of
    +    "po/git.pot", we use xgettext to generate a "pot" header file
    +    ".build/pot/git.header" from an empty file at runtime, and use this
    +    header to assemble "po/git.pot".
     
    -    But as we'll see in a subsequent commit our previous header behavior
    -    has been hiding an encoding-related bug from us, so let's carry it
    -    forward instead of re-generating it with xgettext(1).
    +    But as we'll see in a subsequent commit this header behavior has been
    +    hiding an encoding-related bug from us, so let's carry it forward
    +    instead of re-generating it with xgettext(1).
     
         Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
    +    Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com>
    +    Signed-off-by: Junio C Hamano <gitster@pobox.com>
     
      ## .gitignore ##
     @@
    @@ .gitignore
      *.gcno
     
      ## Makefile ##
    -@@ Makefile: pdf:
    - XGETTEXT_FLAGS = \
    +@@ Makefile: INSTALL = install
    + TCL_PATH = tclsh
    + TCLTK_PATH = wish
    + XGETTEXT = xgettext
    ++MSGCAT = msgcat
    + MSGFMT = msgfmt
    + CURL_CONFIG = curl-config
    + GCOV = gcov
    +@@ Makefile: XGETTEXT_FLAGS = \
      	--force-po \
      	--add-comments=TRANSLATORS: \
    --	--msgid-bugs-address="Git Mailing List <git@vger.kernel.org>" \
    --	--from-code=UTF-8
    -+	--from-code=UTF-8 \
    -+	--omit-header
    -+
    + 	--msgid-bugs-address="Git Mailing List <git@vger.kernel.org>" \
    ++	--package-name=Git \
    + 	--sort-by-file \
    + 	--from-code=UTF-8
      XGETTEXT_FLAGS_C = $(XGETTEXT_FLAGS) --language=C \
    - 	--keyword=_ --keyword=N_ --keyword="Q_:1,2"
    - XGETTEXT_FLAGS_SH = $(XGETTEXT_FLAGS) --language=Shell \
    +@@ Makefile: XGETTEXT_FLAGS_SH = $(XGETTEXT_FLAGS) --language=Shell \
    + 	--keyword=gettextln --keyword=eval_gettextln
    + XGETTEXT_FLAGS_PERL = $(XGETTEXT_FLAGS) --language=Perl \
    + 	--keyword=__ --keyword=N__ --keyword="__n:1,2"
    ++MSGCAT_FLAGS = --sort-by-file
    + LOCALIZED_C = $(FOUND_C_SOURCES) $(SCALAR_SOURCES) \
    + 	      $(FOUND_H_SOURCES) $(GENERATED_H)
    + LOCALIZED_SH = $(SCRIPT_SH)
     @@ Makefile: LOCALIZED_SH += t/t0200/test.sh
      LOCALIZED_PERL += t/t0200/test.perl
      endif
    @@ Makefile: LOCALIZED_SH += t/t0200/test.sh
     -## under a very controlled condition, i.e. (1) it is to be run in a
     -## Git repository (not a tarball extract), (2) any local modifications
     -## will be lost.
    -+## We generate intermediate .build/pot/po/% files containing a extract
    -+## of the translations we find in each file in the source tree. The
    -+## files have the same basename as the source due xgettext(1) not
    -+## having a way to override the basename inserted into comments.
    ++## We generate intermediate .build/pot/po/%.po files containing a
    ++## extract of the translations we find in each file in the source
    ++## tree. We will assemble them using msgcat to create the final
    ++## "po/git.pot" file.
     +LOCALIZED_ALL_GEN_PO =
     +
    -+LOCALIZED_C_GEN_PO = $(LOCALIZED_C:%=.build/pot/po/%)
    ++LOCALIZED_C_GEN_PO = $(LOCALIZED_C:%=.build/pot/po/%.po)
     +LOCALIZED_ALL_GEN_PO += $(LOCALIZED_C_GEN_PO)
     +
    -+LOCALIZED_SH_GEN_PO = $(LOCALIZED_SH:%=.build/pot/po/%)
    ++LOCALIZED_SH_GEN_PO = $(LOCALIZED_SH:%=.build/pot/po/%.po)
     +LOCALIZED_ALL_GEN_PO += $(LOCALIZED_SH_GEN_PO)
     +
    -+LOCALIZED_PERL_GEN_PO = $(LOCALIZED_PERL:%=.build/pot/po/%)
    ++LOCALIZED_PERL_GEN_PO = $(LOCALIZED_PERL:%=.build/pot/po/%.po)
     +LOCALIZED_ALL_GEN_PO += $(LOCALIZED_PERL_GEN_PO)
     +
      ## Gettext tools cannot work with our own custom PRItime type, so
      ## we replace PRItime with PRIuMAX.  We need to update this to
      ## PRIdMAX if we switch to a signed type later.
    -+LOCALIZED_C_GEN	= $(LOCALIZED_C:%=.build/pot/in/%)
    -+$(LOCALIZED_C_GEN): .build/pot/in/%: %
    ++$(LOCALIZED_C_GEN_PO): .build/pot/po/%.po: %
     +	$(call mkdir_p_parent_template)
    -+	$(QUIET_GEN)sed -e 's|PRItime|PRIuMAX|g' <$< >$@
    -+
    -+$(LOCALIZED_C_GEN_PO): .build/pot/po/%: .build/pot/in/%
    -+	$(call mkdir_p_parent_template)
    -+	$(QUIET_XGETTEXT)(\
    -+		cd .build/pot/in && \
    -+		$(XGETTEXT) -o $(@:.build/pot/po/%=../po/%) \
    -+			$(XGETTEXT_FLAGS_C) \
    -+			$(<:.build/pot/in/%=%) \
    -+	)
    ++	$(QUIET_XGETTEXT) \
    ++	    if grep -q PRItime $<; then \
    ++		(\
    ++			sed -e 's|PRItime|PRIuMAX|g' <$< \
    ++				>.build/pot/po/$< && \
    ++			cd .build/pot/po && \
    ++			$(XGETTEXT) --omit-header \
    ++				-o $(@:.build/pot/po/%=%) \
    ++				$(XGETTEXT_FLAGS_C) $< && \
    ++			rm $<; \
    ++		); \
    ++	    else \
    ++		$(XGETTEXT) --omit-header \
    ++			-o $@ $(XGETTEXT_FLAGS_C) $<; \
    ++	    fi
      
     -po/git.pot: $(GENERATED_H) FORCE
     -	# All modifications will be reverted at the end, so we do not
     -	# want to have any local change.
     -	git diff --quiet HEAD && git diff --quiet --cached
    -+$(LOCALIZED_SH_GEN_PO): .build/pot/po/%: %
    ++$(LOCALIZED_SH_GEN_PO): .build/pot/po/%.po: %
     +	$(call mkdir_p_parent_template)
    -+	$(QUIET_XGETTEXT)$(XGETTEXT) -o$@ $(XGETTEXT_FLAGS_SH) $<
    ++	$(QUIET_XGETTEXT)$(XGETTEXT) --omit-header \
    ++		-o$@ $(XGETTEXT_FLAGS_SH) $<
      
     -	@for s in $(LOCALIZED_C) $(LOCALIZED_SH) $(LOCALIZED_PERL); \
     -	do \
     -		sed -e 's|PRItime|PRIuMAX|g' <"$$s" >"$$s+" && \
     -		cat "$$s+" >"$$s" && rm "$$s+"; \
     -	done
    -+$(LOCALIZED_PERL_GEN_PO): .build/pot/po/%: %
    -+	$(call mkdir_p_parent_template)
    -+	$(QUIET_XGETTEXT)$(XGETTEXT) -o$@ $(XGETTEXT_FLAGS_PERL) $<
    -+
    -+.build/pot/pot.header:
    ++$(LOCALIZED_PERL_GEN_PO): .build/pot/po/%.po: %
     +	$(call mkdir_p_parent_template)
    -+	$(QUIET_GEN)sed -n -e '/^$$/q' -e 'p' <po/git.pot >$@
    ++	$(QUIET_XGETTEXT)$(XGETTEXT) --omit-header \
    ++		-o$@ $(XGETTEXT_FLAGS_PERL) $<
      
     -	$(QUIET_XGETTEXT)$(XGETTEXT) -o$@+ $(XGETTEXT_FLAGS_C) $(LOCALIZED_C)
     -	$(QUIET_XGETTEXT)$(XGETTEXT) -o$@+ --join-existing $(XGETTEXT_FLAGS_SH) \
     -		$(LOCALIZED_SH)
     -	$(QUIET_XGETTEXT)$(XGETTEXT) -o$@+ --join-existing $(XGETTEXT_FLAGS_PERL) \
     -		$(LOCALIZED_PERL)
    -+.build/pot/git.pot: .build/pot/pot.header $(LOCALIZED_ALL_GEN_PO)
    -+	$(QUIET_GEN)msgcat $^ >$@
    ++.build/pot/git.header: $(LOCALIZED_ALL_GEN_PO)
    ++	$(call mkdir_p_parent_template)
    ++	$(QUIET_XGETTEXT)$(XGETTEXT) $(XGETTEXT_FLAGS_C) \
    ++		-o - /dev/null | \
    ++	sed -e 's|charset=CHARSET|charset=UTF-8|g' >$@ && \
    ++	echo '"Plural-Forms: nplurals=INTEGER; plural=EXPRESSION;\\n"' >>$@
      
     -	# Reverting the munged source, leaving only the updated $@
     -	git reset --hard
     -	mv $@+ $@
    -+po/git.pot: .build/pot/git.pot FORCE
    -+	$(QUIET_CP)cp $< $@
    ++po/git.pot: .build/pot/git.header $(LOCALIZED_ALL_GEN_PO)
    ++	$(QUIET_GEN)$(MSGCAT) $(MSGCAT_FLAGS) $^ >$@
      
      .PHONY: pot
      pot: po/git.pot
    @@ Makefile: cocciclean:
      	$(RM) *.res
      	$(RM) $(OBJECTS)
      	$(RM) $(LIB_FILE) $(XDIFF_LIB) $(REFTABLE_LIB) $(REFTABLE_TEST_LIB)
    -
    - ## shared.mak ##
    -@@ shared.mak: ifndef V
    - 	QUIET_AR       = @echo '   ' AR $@;
    - 	QUIET_LINK     = @echo '   ' LINK $@;
    - 	QUIET_BUILT_IN = @echo '   ' BUILTIN $@;
    -+	QUIET_CP       = @echo '   ' CP $@;
    - 	QUIET_LNCP     = @echo '   ' LN/CP $@;
    - 	QUIET_XGETTEXT = @echo '   ' XGETTEXT $@;
    - 	QUIET_MSGFMT   = @echo '   ' MSGFMT $@;

  reply	other threads:[~2022-05-19  9:53 UTC|newest]

Thread overview: 110+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-05-03 13:23 [PATCH 0/9] Incremental po/git.pot update and new l10n workflow Jiang Xin
2022-05-03 13:23 ` [PATCH 1/9] Makefile: sort "po/git.pot" by file location Jiang Xin
2022-05-03 13:23 ` [PATCH 2/9] Makefile: generate "po/git.pot" from stable LOCALIZED_C Jiang Xin
2022-05-03 13:23 ` [PATCH 3/9] Makefile: have "make pot" not "reset --hard" Jiang Xin
2022-05-03 13:23 ` [PATCH 4/9] i18n CI: stop allowing non-ASCII source messages in po/git.pot Jiang Xin
2022-05-03 13:23 ` [PATCH 5/9] po/git.pot: don't check in result of "make pot" Jiang Xin
2022-05-03 13:23 ` [PATCH 6/9] po/git.pot: remove this now generated file, see preceding commit Jiang Xin
2022-05-03 13:23 ` [PATCH 7/9] Makefile: add "po-update" rule to update po/XX.po Jiang Xin
2022-05-03 13:23 ` [PATCH 8/9] Makefile: add "po-init" rule to initialize po/XX.po Jiang Xin
2022-05-03 13:23 ` [PATCH 9/9] l10n: Document the new l10n workflow Jiang Xin
2022-05-03 14:07 ` [PATCH 0/9] Incremental po/git.pot update and " Peter Krefting
2022-05-04 12:41   ` Jiang Xin
2022-05-04 14:35 ` Junio C Hamano
2022-05-04 14:51   ` Daniel Santos
2022-05-05  0:20     ` Jiang Xin
2022-05-05 22:00       ` Daniel Santos
2022-05-05 22:49         ` Junio C Hamano
2022-05-06  0:50         ` Jiang Xin
2022-05-05  0:07   ` Jiang Xin
2022-05-04 17:58 ` Junio C Hamano
2022-05-19  8:15 ` [PATCH v2 " Jiang Xin
2022-05-19 10:28   ` Ævar Arnfjörð Bjarmason
2022-05-19 14:32     ` Jiang Xin
2022-05-19 14:41       ` Ævar Arnfjörð Bjarmason
2022-05-23  1:25   ` [PATCH v3 " Jiang Xin
2022-05-23  7:15     ` Ævar Arnfjörð Bjarmason
2022-05-23  8:12       ` Ævar Arnfjörð Bjarmason
2022-05-23 13:42         ` Jiang Xin
2022-05-23 14:38           ` Ævar Arnfjörð Bjarmason
2022-05-23 16:13             ` Jiang Xin
2022-05-23  8:26       ` Jiang Xin
2022-05-23 15:21     ` [PATCH v4 " Jiang Xin
2022-05-23 18:19       ` Junio C Hamano
2022-05-26 14:50       ` [PATCH v5 00/10] " Jiang Xin
2022-05-26 14:50       ` [PATCH v5 01/10] Makefile: sort source files before feeding to xgettext Jiang Xin
2022-05-26 14:50       ` [PATCH v5 02/10] Makefile: generate "po/git.pot" from stable LOCALIZED_C Jiang Xin
2022-05-26 14:50       ` [PATCH v5 03/10] Makefile: have "make pot" not "reset --hard" Jiang Xin
2022-05-26 14:50       ` [PATCH v5 04/10] i18n CI: stop allowing non-ASCII source messages in po/git.pot Jiang Xin
2022-05-26 14:50       ` [PATCH v5 05/10] Makefile: remove duplicate and unwanted files in FOUND_SOURCE_FILES Jiang Xin
2022-05-26 14:50       ` [PATCH v5 06/10] po/git.pot: this is now a generated file Jiang Xin
2022-05-26 17:32         ` Junio C Hamano
2022-05-26 14:50       ` [PATCH v5 07/10] po/git.pot: don't check in result of "make pot" Jiang Xin
2022-05-26 14:50       ` [PATCH v5 08/10] Makefile: add "po-update" rule to update po/XX.po Jiang Xin
2022-05-26 14:50       ` [PATCH v5 09/10] Makefile: add "po-init" rule to initialize po/XX.po Jiang Xin
2022-05-26 14:50       ` [PATCH v5 10/10] l10n: Document the new l10n workflow Jiang Xin
2022-05-23 15:21     ` [PATCH v4 1/9] Makefile: sort "po/git.pot" by file location Jiang Xin
2022-05-23 15:21     ` [PATCH v4 2/9] Makefile: generate "po/git.pot" from stable LOCALIZED_C Jiang Xin
2022-05-23 15:21     ` [PATCH v4 3/9] Makefile: have "make pot" not "reset --hard" Jiang Xin
2022-05-25 22:19       ` Junio C Hamano
2022-05-25 22:24         ` Junio C Hamano
2022-05-26  1:10           ` Jiang Xin
2022-05-26  2:15           ` [PATCH] Makefile: dedup git-ls-files output to prevent duplicate targets Jiang Xin
2022-05-26  4:02             ` Junio C Hamano
2022-05-26  6:06               ` Jiang Xin
2022-05-26  6:23                 ` Junio C Hamano
2022-05-26  7:04                   ` Jiang Xin
2022-05-26 10:00                     ` Ævar Arnfjörð Bjarmason
2022-05-26 11:06                       ` Jiang Xin
2022-05-26 17:18                       ` Junio C Hamano
2022-05-26 18:25                         ` Ævar Arnfjörð Bjarmason
2022-05-26 19:00                           ` Junio C Hamano
2022-05-26 19:17                             ` Ævar Arnfjörð Bjarmason
2022-05-23 15:21     ` [PATCH v4 4/9] i18n CI: stop allowing non-ASCII source messages in po/git.pot Jiang Xin
2022-05-23 15:21     ` [PATCH v4 5/9] po/git.pot: this is now a generated file Jiang Xin
2022-05-23 15:21     ` [PATCH v4 6/9] po/git.pot: don't check in result of "make pot" Jiang Xin
2022-05-23 15:21     ` [PATCH v4 7/9] Makefile: add "po-update" rule to update po/XX.po Jiang Xin
2022-05-23 15:21     ` [PATCH v4 8/9] Makefile: add "po-init" rule to initialize po/XX.po Jiang Xin
2022-05-23 15:21     ` [PATCH v4 9/9] l10n: Document the new l10n workflow Jiang Xin
2022-05-23  1:25   ` [PATCH v3 1/9] Makefile: sort "po/git.pot" by file location Jiang Xin
2022-05-23  8:05     ` Junio C Hamano
2022-05-23  8:50       ` Jiang Xin
2022-05-23  1:25   ` [PATCH v3 2/9] Makefile: generate "po/git.pot" from stable LOCALIZED_C Jiang Xin
2022-05-23  8:05     ` Junio C Hamano
2022-05-23  1:25   ` [PATCH v3 3/9] Makefile: have "make pot" not "reset --hard" Jiang Xin
2022-05-23  7:28     ` Ævar Arnfjörð Bjarmason
2022-05-23 15:00       ` Jiang Xin
2022-05-24  0:56       ` Jiang Xin
2022-05-23  8:15     ` Junio C Hamano
2022-05-23  9:37       ` Jiang Xin
2022-05-23  1:25   ` [PATCH v3 4/9] i18n CI: stop allowing non-ASCII source messages in po/git.pot Jiang Xin
2022-05-23  1:25   ` [PATCH v3 5/9] po/git.pot: this is now a generated file Jiang Xin
2022-05-23  1:25   ` [PATCH v3 6/9] po/git.pot: don't check in result of "make pot" Jiang Xin
2022-05-23  7:26     ` Ævar Arnfjörð Bjarmason
2022-05-23  8:30       ` Jiang Xin
2022-05-23  8:35         ` Jiang Xin
2022-05-23  9:28         ` Ævar Arnfjörð Bjarmason
2022-05-23  1:25   ` [PATCH v3 7/9] Makefile: add "po-update" rule to update po/XX.po Jiang Xin
2022-05-23  1:25   ` [PATCH v3 8/9] Makefile: add "po-init" rule to initialize po/XX.po Jiang Xin
2022-05-23  1:25   ` [PATCH v3 9/9] l10n: Document the new l10n workflow Jiang Xin
2022-05-19  8:15 ` [PATCH v2 1/9] Makefile: sort "po/git.pot" by file location Jiang Xin
2022-05-19  8:53   ` Ævar Arnfjörð Bjarmason
2022-05-19 12:41     ` Jiang Xin
2022-05-19  8:15 ` [PATCH v2 2/9] Makefile: generate "po/git.pot" from stable LOCALIZED_C Jiang Xin
2022-05-19  9:18   ` Ævar Arnfjörð Bjarmason
2022-05-19 12:48     ` Jiang Xin
2022-05-19  8:15 ` [PATCH v2 3/9] Makefile: have "make pot" not "reset --hard" Jiang Xin
2022-05-19  9:43   ` Ævar Arnfjörð Bjarmason [this message]
2022-05-19 13:19     ` Jiang Xin
2022-05-19 14:06       ` Ævar Arnfjörð Bjarmason
2022-05-19  8:15 ` [PATCH v2 4/9] i18n CI: stop allowing non-ASCII source messages in po/git.pot Jiang Xin
2022-05-19 10:02   ` Ævar Arnfjörð Bjarmason
2022-05-19  8:15 ` [PATCH v2 5/9] po/git.pot: this is now a generated file Jiang Xin
2022-05-19  8:15 ` [PATCH v2 6/9] po/git.pot: don't check in result of "make pot" Jiang Xin
2022-05-19  8:15 ` [PATCH v2 7/9] Makefile: add "po-update" rule to update po/XX.po Jiang Xin
2022-05-19 10:07   ` Ævar Arnfjörð Bjarmason
2022-05-19  8:15 ` [PATCH v2 8/9] Makefile: add "po-init" rule to initialize po/XX.po Jiang Xin
2022-05-19 10:22   ` Ævar Arnfjörð Bjarmason
2022-05-19  8:15 ` [PATCH v2 9/9] l10n: Document the new l10n workflow Jiang Xin
2022-05-19 17:18   ` Junio C Hamano
2022-05-21 15:06     ` Jiang Xin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=220519.86leux6e82.gmgdl@evledraar.gmail.com \
    --to=avarab@gmail.com \
    --cc=DJm00n@mail.ru \
    --cc=alessandro.menti@alessandromenti.it \
    --cc=arek_koz@o2.pl \
    --cc=ash@kambanaria.org \
    --cc=bagasdotme@gmail.com \
    --cc=bitigchi@me.com \
    --cc=christopher.diaz.riv@gmail.com \
    --cc=dacs.git@brilhante.top \
    --cc=elongbug@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=jmas@softcatala.org \
    --cc=jn.avila@free.fr \
    --cc=matthias.ruester@gmail.com \
    --cc=me@fangyi.io \
    --cc=pan93412@gmail.com \
    --cc=peter@softwolves.pp.se \
    --cc=vnwildman@gmail.com \
    --cc=vyruss@hellug.gr \
    --cc=worldhello.net@gmail.com \
    --cc=zhiyou.jx@alibaba-inc.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.