* [PATCH 0/5] small doc make and lint fixes @ 2021-03-26 10:36 Ævar Arnfjörð Bjarmason 2021-03-26 10:36 ` [PATCH 1/5] Documentation/Makefile: make $(wildcard howto/*.txt) a var Ævar Arnfjörð Bjarmason ` (6 more replies) 0 siblings, 7 replies; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-03-26 10:36 UTC (permalink / raw) To: git; +Cc: Junio C Hamano, Jeff King, Ævar Arnfjörð Bjarmason A small stand-alone series of doc infrastructure fixes. 5/6 fixes an interesting bug that's been missed since doc-diff was introduced. Ævar Arnfjörð Bjarmason (5): Documentation/Makefile: make $(wildcard howto/*.txt) a var Documentation/Makefile: make $(wildcard <doc deps>) a var doc lint: Perl "strict" and "warnings" in lint-gitlink.perl doc lint: fix bugs in, simplify and improve lint script doc lint: lint and fix missing "GIT" end sections Documentation/Makefile | 31 +++++++-- Documentation/git-credential.txt | 4 ++ Documentation/git-p4.txt | 4 ++ Documentation/gitnamespaces.txt | 4 ++ Documentation/lint-gitlink.perl | 110 +++++++++++++++---------------- Documentation/lint-man-txt.perl | 24 +++++++ 6 files changed, 116 insertions(+), 61 deletions(-) create mode 100755 Documentation/lint-man-txt.perl -- 2.31.0.419.gfc6e4cae13 ^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH 1/5] Documentation/Makefile: make $(wildcard howto/*.txt) a var 2021-03-26 10:36 [PATCH 0/5] small doc make and lint fixes Ævar Arnfjörð Bjarmason @ 2021-03-26 10:36 ` Ævar Arnfjörð Bjarmason 2021-03-28 6:14 ` Junio C Hamano 2021-03-26 10:36 ` [PATCH 2/5] Documentation/Makefile: make $(wildcard <doc deps>) " Ævar Arnfjörð Bjarmason ` (5 subsequent siblings) 6 siblings, 1 reply; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-03-26 10:36 UTC (permalink / raw) To: git; +Cc: Junio C Hamano, Jeff King, Ævar Arnfjörð Bjarmason Refactor occurrences of $(wildcard howto/*.txt) into a single HOWTO_TXT variable for readability and consistency. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> --- Documentation/Makefile | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/Documentation/Makefile b/Documentation/Makefile index 81d1bf7a049..0ba7564be93 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -2,6 +2,7 @@ MAN1_TXT = MAN5_TXT = MAN7_TXT = +HOWTO_TXT = TECH_DOCS = ARTICLES = SP_ARTICLES = @@ -42,6 +43,8 @@ MAN7_TXT += gittutorial-2.txt MAN7_TXT += gittutorial.txt MAN7_TXT += gitworkflows.txt +HOWTO_TXT += $(wildcard howto/*.txt) + ifdef MAN_FILTER MAN_TXT = $(filter $(MAN_FILTER),$(MAN1_TXT) $(MAN5_TXT) $(MAN7_TXT)) else @@ -427,9 +430,9 @@ $(patsubst %.txt,%.texi,$(MAN_TXT)): %.texi : %.xml $(DOCBOOK2X_TEXI) --to-stdout $*.xml >$@+ && \ mv $@+ $@ -howto-index.txt: howto-index.sh $(wildcard howto/*.txt) +howto-index.txt: howto-index.sh $(HOWTO_TXT) $(QUIET_GEN)$(RM) $@+ $@ && \ - '$(SHELL_PATH_SQ)' ./howto-index.sh $(sort $(wildcard howto/*.txt)) >$@+ && \ + '$(SHELL_PATH_SQ)' ./howto-index.sh $(sort $(HOWTO_TXT)) >$@+ && \ mv $@+ $@ $(patsubst %,%.html,$(ARTICLES)) : %.html : %.txt @@ -438,7 +441,7 @@ $(patsubst %,%.html,$(ARTICLES)) : %.html : %.txt WEBDOC_DEST = /pub/software/scm/git/docs howto/%.html: ASCIIDOC_EXTRA += -a git-relative-html-prefix=../ -$(patsubst %.txt,%.html,$(wildcard howto/*.txt)): %.html : %.txt GIT-ASCIIDOCFLAGS +$(patsubst %.txt,%.html,$(HOWTO_TXT)): %.html : %.txt GIT-ASCIIDOCFLAGS $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \ sed -e '1,/^$$/d' $< | \ $(TXT_TO_HTML) - >$@+ && \ -- 2.31.0.419.gfc6e4cae13 ^ permalink raw reply related [flat|nested] 32+ messages in thread
* Re: [PATCH 1/5] Documentation/Makefile: make $(wildcard howto/*.txt) a var 2021-03-26 10:36 ` [PATCH 1/5] Documentation/Makefile: make $(wildcard howto/*.txt) a var Ævar Arnfjörð Bjarmason @ 2021-03-28 6:14 ` Junio C Hamano 0 siblings, 0 replies; 32+ messages in thread From: Junio C Hamano @ 2021-03-28 6:14 UTC (permalink / raw) To: Ævar Arnfjörð Bjarmason; +Cc: git, Jeff King Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes: > Refactor occurrences of $(wildcard howto/*.txt) into a single > HOWTO_TXT variable for readability and consistency. OK. ^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH 2/5] Documentation/Makefile: make $(wildcard <doc deps>) a var 2021-03-26 10:36 [PATCH 0/5] small doc make and lint fixes Ævar Arnfjörð Bjarmason 2021-03-26 10:36 ` [PATCH 1/5] Documentation/Makefile: make $(wildcard howto/*.txt) a var Ævar Arnfjörð Bjarmason @ 2021-03-26 10:36 ` Ævar Arnfjörð Bjarmason 2021-03-28 6:28 ` Junio C Hamano 2021-03-26 10:36 ` [PATCH 3/5] doc lint: Perl "strict" and "warnings" in lint-gitlink.perl Ævar Arnfjörð Bjarmason ` (4 subsequent siblings) 6 siblings, 1 reply; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-03-26 10:36 UTC (permalink / raw) To: git; +Cc: Junio C Hamano, Jeff King, Ævar Arnfjörð Bjarmason Refactor the wildcard we'll scan for "include" directives into a single INCLUDE_TARGETS_TXT variable for readability, consistency. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> --- Documentation/Makefile | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/Documentation/Makefile b/Documentation/Makefile index 0ba7564be93..7313956d73f 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -3,6 +3,7 @@ MAN1_TXT = MAN5_TXT = MAN7_TXT = HOWTO_TXT = +INCLUDE_TARGETS_TXT = TECH_DOCS = ARTICLES = SP_ARTICLES = @@ -45,6 +46,9 @@ MAN7_TXT += gitworkflows.txt HOWTO_TXT += $(wildcard howto/*.txt) +INCLUDE_TARGETS_TXT += $(wildcard *.txt) +INCLUDE_TARGETS_TXT += $(wildcard config/*.txt) + ifdef MAN_FILTER MAN_TXT = $(filter $(MAN_FILTER),$(MAN1_TXT) $(MAN5_TXT) $(MAN7_TXT)) else @@ -287,7 +291,7 @@ docdep_prereqs = \ mergetools-list.made $(mergetools_txt) \ cmd-list.made $(cmds_txt) -doc.dep : $(docdep_prereqs) $(wildcard *.txt) $(wildcard config/*.txt) build-docdep.perl +doc.dep : $(docdep_prereqs) $(INCLUDE_TARGETS_TXT) build-docdep.perl $(QUIET_GEN)$(RM) $@+ $@ && \ $(PERL_PATH) ./build-docdep.perl >$@+ $(QUIET_STDERR) && \ mv $@+ $@ -- 2.31.0.419.gfc6e4cae13 ^ permalink raw reply related [flat|nested] 32+ messages in thread
* Re: [PATCH 2/5] Documentation/Makefile: make $(wildcard <doc deps>) a var 2021-03-26 10:36 ` [PATCH 2/5] Documentation/Makefile: make $(wildcard <doc deps>) " Ævar Arnfjörð Bjarmason @ 2021-03-28 6:28 ` Junio C Hamano 0 siblings, 0 replies; 32+ messages in thread From: Junio C Hamano @ 2021-03-28 6:28 UTC (permalink / raw) To: Ævar Arnfjörð Bjarmason; +Cc: git, Jeff King Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes: > Refactor the wildcard we'll scan for "include" directives into a > single INCLUDE_TARGETS_TXT variable for readability, consistency. > > Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> > --- > Documentation/Makefile | 6 +++++- > 1 file changed, 5 insertions(+), 1 deletion(-) > > diff --git a/Documentation/Makefile b/Documentation/Makefile > index 0ba7564be93..7313956d73f 100644 > --- a/Documentation/Makefile > +++ b/Documentation/Makefile > @@ -3,6 +3,7 @@ MAN1_TXT = > MAN5_TXT = > MAN7_TXT = > HOWTO_TXT = > +INCLUDE_TARGETS_TXT = > TECH_DOCS = > ARTICLES = > SP_ARTICLES = > @@ -45,6 +46,9 @@ MAN7_TXT += gitworkflows.txt > > HOWTO_TXT += $(wildcard howto/*.txt) > > +INCLUDE_TARGETS_TXT += $(wildcard *.txt) > +INCLUDE_TARGETS_TXT += $(wildcard config/*.txt) > + > ifdef MAN_FILTER > MAN_TXT = $(filter $(MAN_FILTER),$(MAN1_TXT) $(MAN5_TXT) $(MAN7_TXT)) > else > @@ -287,7 +291,7 @@ docdep_prereqs = \ > mergetools-list.made $(mergetools_txt) \ > cmd-list.made $(cmds_txt) > > -doc.dep : $(docdep_prereqs) $(wildcard *.txt) $(wildcard config/*.txt) build-docdep.perl > +doc.dep : $(docdep_prereqs) $(INCLUDE_TARGETS_TXT) build-docdep.perl > $(QUIET_GEN)$(RM) $@+ $@ && \ > $(PERL_PATH) ./build-docdep.perl >$@+ $(QUIET_STDERR) && \ > mv $@+ $@ While this may not break anything per-se, I find the name of the new macro not quite intuitive. The build-docdep script opens $(wildcard *.txt) itself (without getting them fed from Makefile) to find what other files are used via "include::", and then it recursively scans more files that are used via "include::" to build dependencies. Makefile assumes the "include::" chain would pull in $(wildcard config/*.txt) files, but this assumption may not prevent other *.txt files from getting pulled in (not a new problem introduced by this patch). I am not sure what these *.txt files involved in the process should be called, but "include targets" sounds something else. Naïvely, I would say "we rebuild documentation dependency any time any source text file changes", so if this were called "ALL_TXT" or something like that, I wouldn't have such a reaction. Thanks. ^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH 3/5] doc lint: Perl "strict" and "warnings" in lint-gitlink.perl 2021-03-26 10:36 [PATCH 0/5] small doc make and lint fixes Ævar Arnfjörð Bjarmason 2021-03-26 10:36 ` [PATCH 1/5] Documentation/Makefile: make $(wildcard howto/*.txt) a var Ævar Arnfjörð Bjarmason 2021-03-26 10:36 ` [PATCH 2/5] Documentation/Makefile: make $(wildcard <doc deps>) " Ævar Arnfjörð Bjarmason @ 2021-03-26 10:36 ` Ævar Arnfjörð Bjarmason 2021-03-28 6:28 ` Junio C Hamano 2021-03-26 10:36 ` [PATCH 4/5] doc lint: fix bugs in, simplify and improve lint script Ævar Arnfjörð Bjarmason ` (3 subsequent siblings) 6 siblings, 1 reply; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-03-26 10:36 UTC (permalink / raw) To: git; +Cc: Junio C Hamano, Jeff King, Ævar Arnfjörð Bjarmason Amend this script added in ab81411ced (ci: validate "linkgit:" in documentation, 2016-05-04) to pass under "use strict", and add a "use warnings" for good measure. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> --- Documentation/lint-gitlink.perl | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/Documentation/lint-gitlink.perl b/Documentation/lint-gitlink.perl index 476cc30b837..35230875c24 100755 --- a/Documentation/lint-gitlink.perl +++ b/Documentation/lint-gitlink.perl @@ -1,5 +1,7 @@ #!/usr/bin/perl +use strict; +use warnings; use File::Find; use Getopt::Long; @@ -45,7 +47,7 @@ sub lint { report($where, $target, "no such source"); next; } - $real_section = grab_section($page); + my $real_section = grab_section($page); if ($real_section != $section) { report($where, $target, "wrong section (should be $real_section)"); -- 2.31.0.419.gfc6e4cae13 ^ permalink raw reply related [flat|nested] 32+ messages in thread
* Re: [PATCH 3/5] doc lint: Perl "strict" and "warnings" in lint-gitlink.perl 2021-03-26 10:36 ` [PATCH 3/5] doc lint: Perl "strict" and "warnings" in lint-gitlink.perl Ævar Arnfjörð Bjarmason @ 2021-03-28 6:28 ` Junio C Hamano 0 siblings, 0 replies; 32+ messages in thread From: Junio C Hamano @ 2021-03-28 6:28 UTC (permalink / raw) To: Ævar Arnfjörð Bjarmason; +Cc: git, Jeff King Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes: > Amend this script added in ab81411ced (ci: validate "linkgit:" in > documentation, 2016-05-04) to pass under "use strict", and add a "use > warnings" for good measure. > > Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> > --- > Documentation/lint-gitlink.perl | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/Documentation/lint-gitlink.perl b/Documentation/lint-gitlink.perl > index 476cc30b837..35230875c24 100755 > --- a/Documentation/lint-gitlink.perl > +++ b/Documentation/lint-gitlink.perl > @@ -1,5 +1,7 @@ > #!/usr/bin/perl > > +use strict; > +use warnings; Yes! Thanks. > use File::Find; > use Getopt::Long; > > @@ -45,7 +47,7 @@ sub lint { > report($where, $target, "no such source"); > next; > } > - $real_section = grab_section($page); > + my $real_section = grab_section($page); > if ($real_section != $section) { > report($where, $target, > "wrong section (should be $real_section)"); ^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH 4/5] doc lint: fix bugs in, simplify and improve lint script 2021-03-26 10:36 [PATCH 0/5] small doc make and lint fixes Ævar Arnfjörð Bjarmason ` (2 preceding siblings ...) 2021-03-26 10:36 ` [PATCH 3/5] doc lint: Perl "strict" and "warnings" in lint-gitlink.perl Ævar Arnfjörð Bjarmason @ 2021-03-26 10:36 ` Ævar Arnfjörð Bjarmason 2021-03-26 11:00 ` Jeff King ` (2 more replies) 2021-03-26 10:36 ` [PATCH 5/5] doc lint: lint and fix missing "GIT" end sections Ævar Arnfjörð Bjarmason ` (2 subsequent siblings) 6 siblings, 3 replies; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-03-26 10:36 UTC (permalink / raw) To: git; +Cc: Junio C Hamano, Jeff King, Ævar Arnfjörð Bjarmason The lint-gitlink.perl script added in ab81411ced (ci: validate "linkgit:" in documentation, 2016-05-04) was more complex than it needed to be. It: - Was using File::Find to recursively find *.txt files in Documentation/, let's instead use the Makefile as a source of truth for *.txt files, and pass it down to the script. - We now don't lint linkgit:* in RelNotes/* or technical/*, which we shouldn't have been doing in the first place anyway. - When the doc-diff script was added in beb188e22a (add a script to diff rendered documentation, 2018-08-06) we started sometimes having a "git worktree" under "documentation". This tree contains a full checkout of git.git, as a result the "lint" script would recurse into that, and lint any *.txt file found in that entire repository. In practice the only in-tree "linkgit" outside of the Documentation/ tree is contrib/contacts/git-contacts.txt and contrib/subtree/git-subtree.txt, so this wouldn't emit any errors Now we instead simply trust the Makefile to give us *.txt files, and since the Makefile also knows what sections each page should be in we don't have to open the files ourselves and try to parse that out. As a bonus this will also catch bugs with the section line in the file being incorrect. The structure of the new script is mostly based on t/check-non-portable-shell.pl. As an added bonus it will also use pos() to print where the problems it finds are, e.g. given an issue like: diff --git a/Documentation/git-cherry.txt b/Documentation/git-cherry.txt [...] and line numbers. git-cherry therefore detects when commits have been -"copied" by means of linkgit:git-cherry-pick[1], linkgit:git-am[1] or -linkgit:git-rebase[1]. +"copied" by means of linkgit:git-cherry-pick[2], linkgit:git-am[3] or +linkgit:git-rebase[4]. We'll now emit: git-cherry.txt:20: error: git-cherry-pick[2]: wrong section (should be 1), shown with 'HERE' below: git-cherry.txt:20: "copied" by means of linkgit:git-cherry-pick[2]' <-- HERE git-cherry.txt:20: error: git-am[3]: wrong section (should be 1), shown with 'HERE' below: git-cherry.txt:20: "copied" by means of linkgit:git-cherry-pick[2], linkgit:git-am[3]' <-- HERE git-cherry.txt:21: error: git-rebase[4]: wrong section (should be 1), shown with 'HERE' below: git-cherry.txt:21: linkgit:git-rebase[4]' <-- HERE Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> --- Documentation/Makefile | 14 ++++- Documentation/lint-gitlink.perl | 108 +++++++++++++++----------------- 2 files changed, 65 insertions(+), 57 deletions(-) diff --git a/Documentation/Makefile b/Documentation/Makefile index 7313956d73f..6bfd8c75772 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -4,6 +4,7 @@ MAN5_TXT = MAN7_TXT = HOWTO_TXT = INCLUDE_TARGETS_TXT = +ALL_TXT = TECH_DOCS = ARTICLES = SP_ARTICLES = @@ -49,6 +50,13 @@ HOWTO_TXT += $(wildcard howto/*.txt) INCLUDE_TARGETS_TXT += $(wildcard *.txt) INCLUDE_TARGETS_TXT += $(wildcard config/*.txt) +# For linting +ALL_TXT += $(MAN1_TXT) +ALL_TXT += $(MAN5_TXT) +ALL_TXT += $(MAN7_TXT) +ALL_TXT += $(HOWTO_TXT) +ALL_TXT += $(INCLUDE_TARGETS_TXT) + ifdef MAN_FILTER MAN_TXT = $(filter $(MAN_FILTER),$(MAN1_TXT) $(MAN5_TXT) $(MAN7_TXT)) else @@ -477,7 +485,11 @@ print-man1: @for i in $(MAN1_TXT); do echo $$i; done lint-docs:: - $(QUIET_LINT)$(PERL_PATH) lint-gitlink.perl + $(QUIET_LINT)$(PERL_PATH) lint-gitlink.perl \ + --section=1 $(MAN1_TXT) \ + --section=5 $(MAN5_TXT) \ + --section=7 $(MAN7_TXT) \ + --to-lint $(ALL_TXT) ifeq ($(wildcard po/Makefile),po/Makefile) doc-l10n install-l10n:: diff --git a/Documentation/lint-gitlink.perl b/Documentation/lint-gitlink.perl index 35230875c24..d42f154e024 100755 --- a/Documentation/lint-gitlink.perl +++ b/Documentation/lint-gitlink.perl @@ -2,72 +2,68 @@ use strict; use warnings; -use File::Find; -use Getopt::Long; -my $basedir = "."; -GetOptions("basedir=s" => \$basedir) - or die("Cannot parse command line arguments\n"); +# Parse arguments, a simple state machine for input like: +# +# --section=1 git.txt git-add.txt [...] --to-lint git-add.txt a-file.txt [...] +my %TXT; +my %SECTION; +my $section; +my $lint_these = 0; +for my $arg (@ARGV) { + if (my ($sec) = $arg =~ /^--section=(\d+)$/s) { + $section = $sec; + next; + } elsif ($arg eq '--to-lint') { + $lint_these = 1; + next; + } -my $found_errors = 0; + my ($name) = $arg =~ /^(.*?)\.txt$/s; + if ($lint_these) { + $TXT{$name} = $arg; + next; + } -sub report { - my ($where, $what, $error) = @_; - print "$where: $error: $what\n"; - $found_errors = 1; + $SECTION{$name} = $section; } -sub grab_section { - my ($page) = @_; - open my $fh, "<", "$basedir/$page.txt"; - my $firstline = <$fh>; - chomp $firstline; - close $fh; - my ($section) = ($firstline =~ /.*\((\d)\)$/); - return $section; +my $exit_code = 0; +sub report { + my ($pos, $line, $target, $msg) = @_; + substr($line, $pos) = "' <-- HERE"; + $line =~ s/^\s+//; + print "$ARGV:$.: error: $target: $msg, shown with 'HERE' below:\n"; + print "$ARGV:$.:\t$line\n"; + $exit_code = 1; } -sub lint { - my ($file) = @_; - open my $fh, "<", $file - or return; - while (<$fh>) { - my $where = "$file:$."; - while (s/linkgit:((.*?)\[(\d)\])//) { - my ($target, $page, $section) = ($1, $2, $3); +@ARGV = sort values %TXT; +while (<>) { + my $line = $_; + while ($line =~ m/linkgit:((.*?)\[(\d)\])/g) { + my $pos = pos $line; + my ($target, $page, $section) = ($1, $2, $3); - # De-AsciiDoc - $page =~ s/{litdd}/--/g; + # De-AsciiDoc + $page =~ s/{litdd}/--/g; - if ($page !~ /^git/) { - report($where, $target, "nongit link"); - next; - } - if (! -f "$basedir/$page.txt") { - report($where, $target, "no such source"); - next; - } - my $real_section = grab_section($page); - if ($real_section != $section) { - report($where, $target, - "wrong section (should be $real_section)"); - next; - } + if (!exists $TXT{$page}) { + report($pos, $line, $target, "link outside of our own docs"); + next; + } + if (!exists $SECTION{$page}) { + report($pos, $line, $target, "link outside of our sectioned docs"); + next; + } + my $real_section = $SECTION{$page}; + if ($section != $SECTION{$page}) { + report($pos, $line, $target, "wrong section (should be $real_section)"); + next; } } - close $fh; -} - -sub lint_it { - lint($File::Find::name) if -f && /\.txt$/; -} - -if (!@ARGV) { - find({ wanted => \&lint_it, no_chdir => 1 }, $basedir); -} else { - for (@ARGV) { - lint($_); - } + # this resets our $. for each file + close ARGV if eof; } -exit $found_errors; +exit $exit_code; -- 2.31.0.419.gfc6e4cae13 ^ permalink raw reply related [flat|nested] 32+ messages in thread
* Re: [PATCH 4/5] doc lint: fix bugs in, simplify and improve lint script 2021-03-26 10:36 ` [PATCH 4/5] doc lint: fix bugs in, simplify and improve lint script Ævar Arnfjörð Bjarmason @ 2021-03-26 11:00 ` Jeff King 2021-03-28 1:35 ` Junio C Hamano 2021-03-26 12:48 ` Philip Oakley 2021-03-28 6:38 ` Junio C Hamano 2 siblings, 1 reply; 32+ messages in thread From: Jeff King @ 2021-03-26 11:00 UTC (permalink / raw) To: Ævar Arnfjörð Bjarmason; +Cc: git, Junio C Hamano On Fri, Mar 26, 2021 at 11:36:49AM +0100, Ævar Arnfjörð Bjarmason wrote: > lint-docs:: > - $(QUIET_LINT)$(PERL_PATH) lint-gitlink.perl > + $(QUIET_LINT)$(PERL_PATH) lint-gitlink.perl \ > + --section=1 $(MAN1_TXT) \ > + --section=5 $(MAN5_TXT) \ > + --section=7 $(MAN7_TXT) \ > + --to-lint $(ALL_TXT) This is probably bikeshedding, but I would have expected the invocation to be: link-gitlink.perl \ $(HOWTO_TXT) $(INCLUDE_TARGETS_TXT) \ --section=1 $(MAN1_TXT) \ --section=5 $(MAN5_TXT) \ --section=7 $(MAN7_TXT) I.e., list each filename only once, with the previous --section giving the expected section (and if before any --section, then expect no section). -Peff ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH 4/5] doc lint: fix bugs in, simplify and improve lint script 2021-03-26 11:00 ` Jeff King @ 2021-03-28 1:35 ` Junio C Hamano 0 siblings, 0 replies; 32+ messages in thread From: Junio C Hamano @ 2021-03-28 1:35 UTC (permalink / raw) To: Jeff King; +Cc: Ævar Arnfjörð Bjarmason, git Jeff King <peff@peff.net> writes: > On Fri, Mar 26, 2021 at 11:36:49AM +0100, Ævar Arnfjörð Bjarmason wrote: > >> lint-docs:: >> - $(QUIET_LINT)$(PERL_PATH) lint-gitlink.perl >> + $(QUIET_LINT)$(PERL_PATH) lint-gitlink.perl \ >> + --section=1 $(MAN1_TXT) \ >> + --section=5 $(MAN5_TXT) \ >> + --section=7 $(MAN7_TXT) \ >> + --to-lint $(ALL_TXT) > > This is probably bikeshedding, but I would have expected the invocation > to be: > > link-gitlink.perl \ > $(HOWTO_TXT) $(INCLUDE_TARGETS_TXT) \ > --section=1 $(MAN1_TXT) \ > --section=5 $(MAN5_TXT) \ > --section=7 $(MAN7_TXT) > > I.e., list each filename only once, with the previous --section giving > the expected section (and if before any --section, then expect no > section). Makes more sense than the presented patch. Optionally "--section=" could be used to cancel the section that was previously given. ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH 4/5] doc lint: fix bugs in, simplify and improve lint script 2021-03-26 10:36 ` [PATCH 4/5] doc lint: fix bugs in, simplify and improve lint script Ævar Arnfjörð Bjarmason 2021-03-26 11:00 ` Jeff King @ 2021-03-26 12:48 ` Philip Oakley 2021-03-28 1:34 ` Junio C Hamano 2021-03-28 6:38 ` Junio C Hamano 2 siblings, 1 reply; 32+ messages in thread From: Philip Oakley @ 2021-03-26 12:48 UTC (permalink / raw) To: Ævar Arnfjörð Bjarmason, git; +Cc: Junio C Hamano, Jeff King Hi Ævar, minor naming comment. On 26/03/2021 10:36, Ævar Arnfjörð Bjarmason wrote: > The lint-gitlink.perl script added in ab81411ced (ci: validate > "linkgit:" in documentation, 2016-05-04) was more complex than it > needed to be. It: > > - Was using File::Find to recursively find *.txt files in > Documentation/, let's instead use the Makefile as a source of truth > for *.txt files, and pass it down to the script. > > - We now don't lint linkgit:* in RelNotes/* or technical/*, which we > shouldn't have been doing in the first place anyway. > > - When the doc-diff script was added in beb188e22a (add a script to > diff rendered documentation, 2018-08-06) we started sometimes having > a "git worktree" under "documentation". This tree contains a full > checkout of git.git, as a result the "lint" script would recurse into > that, and lint any *.txt file found in that entire repository. > > In practice the only in-tree "linkgit" outside of the > Documentation/ tree is contrib/contacts/git-contacts.txt and > contrib/subtree/git-subtree.txt, so this wouldn't emit any errors > > Now we instead simply trust the Makefile to give us *.txt files, and > since the Makefile also knows what sections each page should be in we > don't have to open the files ourselves and try to parse that out. As a > bonus this will also catch bugs with the section line in the file > being incorrect. > > The structure of the new script is mostly based on > t/check-non-portable-shell.pl. As an added bonus it will also use > pos() to print where the problems it finds are, e.g. given an issue > like: > > diff --git a/Documentation/git-cherry.txt b/Documentation/git-cherry.txt > [...] > and line numbers. git-cherry therefore detects when commits have been > -"copied" by means of linkgit:git-cherry-pick[1], linkgit:git-am[1] or > -linkgit:git-rebase[1]. > +"copied" by means of linkgit:git-cherry-pick[2], linkgit:git-am[3] or > +linkgit:git-rebase[4]. > > We'll now emit: > > git-cherry.txt:20: error: git-cherry-pick[2]: wrong section (should be 1), shown with 'HERE' below: > git-cherry.txt:20: "copied" by means of linkgit:git-cherry-pick[2]' <-- HERE > git-cherry.txt:20: error: git-am[3]: wrong section (should be 1), shown with 'HERE' below: > git-cherry.txt:20: "copied" by means of linkgit:git-cherry-pick[2], linkgit:git-am[3]' <-- HERE > git-cherry.txt:21: error: git-rebase[4]: wrong section (should be 1), shown with 'HERE' below: > git-cherry.txt:21: linkgit:git-rebase[4]' <-- HERE > > Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> > --- > Documentation/Makefile | 14 ++++- > Documentation/lint-gitlink.perl | 108 +++++++++++++++----------------- > 2 files changed, 65 insertions(+), 57 deletions(-) > > diff --git a/Documentation/Makefile b/Documentation/Makefile > index 7313956d73f..6bfd8c75772 100644 > --- a/Documentation/Makefile > +++ b/Documentation/Makefile > @@ -4,6 +4,7 @@ MAN5_TXT = > MAN7_TXT = > HOWTO_TXT = > INCLUDE_TARGETS_TXT = > +ALL_TXT = Maybe LINT_TXT would better clarify its use, rather than squatting on the generic "ALL"? -- Philip > TECH_DOCS = > ARTICLES = > SP_ARTICLES = > @@ -49,6 +50,13 @@ HOWTO_TXT += $(wildcard howto/*.txt) > INCLUDE_TARGETS_TXT += $(wildcard *.txt) > INCLUDE_TARGETS_TXT += $(wildcard config/*.txt) > > +# For linting > +ALL_TXT += $(MAN1_TXT) > +ALL_TXT += $(MAN5_TXT) > +ALL_TXT += $(MAN7_TXT) > +ALL_TXT += $(HOWTO_TXT) > +ALL_TXT += $(INCLUDE_TARGETS_TXT) > + > ifdef MAN_FILTER > MAN_TXT = $(filter $(MAN_FILTER),$(MAN1_TXT) $(MAN5_TXT) $(MAN7_TXT)) > else > @@ -477,7 +485,11 @@ print-man1: > @for i in $(MAN1_TXT); do echo $$i; done > > lint-docs:: > - $(QUIET_LINT)$(PERL_PATH) lint-gitlink.perl > + $(QUIET_LINT)$(PERL_PATH) lint-gitlink.perl \ > + --section=1 $(MAN1_TXT) \ > + --section=5 $(MAN5_TXT) \ > + --section=7 $(MAN7_TXT) \ > + --to-lint $(ALL_TXT) > > ifeq ($(wildcard po/Makefile),po/Makefile) > doc-l10n install-l10n:: > diff --git a/Documentation/lint-gitlink.perl b/Documentation/lint-gitlink.perl > index 35230875c24..d42f154e024 100755 > --- a/Documentation/lint-gitlink.perl > +++ b/Documentation/lint-gitlink.perl > @@ -2,72 +2,68 @@ > > use strict; > use warnings; > -use File::Find; > -use Getopt::Long; > > -my $basedir = "."; > -GetOptions("basedir=s" => \$basedir) > - or die("Cannot parse command line arguments\n"); > +# Parse arguments, a simple state machine for input like: > +# > +# --section=1 git.txt git-add.txt [...] --to-lint git-add.txt a-file.txt [...] > +my %TXT; > +my %SECTION; > +my $section; > +my $lint_these = 0; > +for my $arg (@ARGV) { > + if (my ($sec) = $arg =~ /^--section=(\d+)$/s) { > + $section = $sec; > + next; > + } elsif ($arg eq '--to-lint') { > + $lint_these = 1; > + next; > + } > > -my $found_errors = 0; > + my ($name) = $arg =~ /^(.*?)\.txt$/s; > + if ($lint_these) { > + $TXT{$name} = $arg; > + next; > + } > > -sub report { > - my ($where, $what, $error) = @_; > - print "$where: $error: $what\n"; > - $found_errors = 1; > + $SECTION{$name} = $section; > } > > -sub grab_section { > - my ($page) = @_; > - open my $fh, "<", "$basedir/$page.txt"; > - my $firstline = <$fh>; > - chomp $firstline; > - close $fh; > - my ($section) = ($firstline =~ /.*\((\d)\)$/); > - return $section; > +my $exit_code = 0; > +sub report { > + my ($pos, $line, $target, $msg) = @_; > + substr($line, $pos) = "' <-- HERE"; > + $line =~ s/^\s+//; > + print "$ARGV:$.: error: $target: $msg, shown with 'HERE' below:\n"; > + print "$ARGV:$.:\t$line\n"; > + $exit_code = 1; > } > > -sub lint { > - my ($file) = @_; > - open my $fh, "<", $file > - or return; > - while (<$fh>) { > - my $where = "$file:$."; > - while (s/linkgit:((.*?)\[(\d)\])//) { > - my ($target, $page, $section) = ($1, $2, $3); > +@ARGV = sort values %TXT; > +while (<>) { > + my $line = $_; > + while ($line =~ m/linkgit:((.*?)\[(\d)\])/g) { > + my $pos = pos $line; > + my ($target, $page, $section) = ($1, $2, $3); > > - # De-AsciiDoc > - $page =~ s/{litdd}/--/g; > + # De-AsciiDoc > + $page =~ s/{litdd}/--/g; > > - if ($page !~ /^git/) { > - report($where, $target, "nongit link"); > - next; > - } > - if (! -f "$basedir/$page.txt") { > - report($where, $target, "no such source"); > - next; > - } > - my $real_section = grab_section($page); > - if ($real_section != $section) { > - report($where, $target, > - "wrong section (should be $real_section)"); > - next; > - } > + if (!exists $TXT{$page}) { > + report($pos, $line, $target, "link outside of our own docs"); > + next; > + } > + if (!exists $SECTION{$page}) { > + report($pos, $line, $target, "link outside of our sectioned docs"); > + next; > + } > + my $real_section = $SECTION{$page}; > + if ($section != $SECTION{$page}) { > + report($pos, $line, $target, "wrong section (should be $real_section)"); > + next; > } > } > - close $fh; > -} > - > -sub lint_it { > - lint($File::Find::name) if -f && /\.txt$/; > -} > - > -if (!@ARGV) { > - find({ wanted => \&lint_it, no_chdir => 1 }, $basedir); > -} else { > - for (@ARGV) { > - lint($_); > - } > + # this resets our $. for each file > + close ARGV if eof; > } > > -exit $found_errors; > +exit $exit_code; ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH 4/5] doc lint: fix bugs in, simplify and improve lint script 2021-03-26 12:48 ` Philip Oakley @ 2021-03-28 1:34 ` Junio C Hamano 0 siblings, 0 replies; 32+ messages in thread From: Junio C Hamano @ 2021-03-28 1:34 UTC (permalink / raw) To: Philip Oakley; +Cc: Ævar Arnfjörð Bjarmason, git, Jeff King Philip Oakley <philipoakley@iee.email> writes: >> diff --git a/Documentation/Makefile b/Documentation/Makefile >> index 7313956d73f..6bfd8c75772 100644 >> --- a/Documentation/Makefile >> +++ b/Documentation/Makefile >> @@ -4,6 +4,7 @@ MAN5_TXT = >> MAN7_TXT = >> HOWTO_TXT = >> INCLUDE_TARGETS_TXT = >> +ALL_TXT = > > Maybe LINT_TXT would better clarify its use, rather than squatting on > the generic "ALL"? As long as it is truly "all the .txt files" and will stay that way, and the set of files we lint happens to be "everything under the sun", I am OK to have this as ALL_TXT. We may in the future want to do something other than running lint-foo scripts to all the .txt files. ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH 4/5] doc lint: fix bugs in, simplify and improve lint script 2021-03-26 10:36 ` [PATCH 4/5] doc lint: fix bugs in, simplify and improve lint script Ævar Arnfjörð Bjarmason 2021-03-26 11:00 ` Jeff King 2021-03-26 12:48 ` Philip Oakley @ 2021-03-28 6:38 ` Junio C Hamano 2 siblings, 0 replies; 32+ messages in thread From: Junio C Hamano @ 2021-03-28 6:38 UTC (permalink / raw) To: Ævar Arnfjörð Bjarmason; +Cc: git, Jeff King Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes: > - Was using File::Find to recursively find *.txt files in > Documentation/, let's instead use the Makefile as a source of truth > for *.txt files, and pass it down to the script. OK. > - We now don't lint linkgit:* in RelNotes/* or technical/*, which we > shouldn't have been doing in the first place anyway. I understand RelNotes (which I consider is plain text file), but are technical/* documentation forbidden from referring to our manual pages for the commands they talk about? > - When the doc-diff script was added in beb188e22a (add a script to > diff rendered documentation, 2018-08-06) we started sometimes having > a "git worktree" under "documentation". This tree contains a full > checkout of git.git, as a result the "lint" script would recurse into > that, and lint any *.txt file found in that entire repository. Ouch. > In practice the only in-tree "linkgit" outside of the > Documentation/ tree is contrib/contacts/git-contacts.txt and > contrib/subtree/git-subtree.txt, so this wouldn't emit any errors Hmm, the nested copy of git has Documentation/ subdirectory of its own, and we do not want to scan it, I think. That is a problem worth fixing. Thanks for noticing it. > Now we instead simply trust the Makefile to give us *.txt files, and > since the Makefile also knows what sections each page should be in we > don't have to open the files ourselves and try to parse that out. As a > bonus this will also catch bugs with the section line in the file > being incorrect. OK. > diff --git a/Documentation/Makefile b/Documentation/Makefile > index 7313956d73f..6bfd8c75772 100644 > --- a/Documentation/Makefile > +++ b/Documentation/Makefile > @@ -4,6 +4,7 @@ MAN5_TXT = > MAN7_TXT = > HOWTO_TXT = > INCLUDE_TARGETS_TXT = > +ALL_TXT = > TECH_DOCS = > ARTICLES = > SP_ARTICLES = > @@ -49,6 +50,13 @@ HOWTO_TXT += $(wildcard howto/*.txt) > INCLUDE_TARGETS_TXT += $(wildcard *.txt) > INCLUDE_TARGETS_TXT += $(wildcard config/*.txt) > > +# For linting > +ALL_TXT += $(MAN1_TXT) > +ALL_TXT += $(MAN5_TXT) > +ALL_TXT += $(MAN7_TXT) > +ALL_TXT += $(HOWTO_TXT) > +ALL_TXT += $(INCLUDE_TARGETS_TXT) The "INCLUDE_TARGETS_TXT" looks more and more like "all txt sources", which is a superset of MAN1_TXT and its friends. Listing all of them in ALL_TXT duplicated feels somewhat iffy. > ifdef MAN_FILTER > MAN_TXT = $(filter $(MAN_FILTER),$(MAN1_TXT) $(MAN5_TXT) $(MAN7_TXT)) > else > @@ -477,7 +485,11 @@ print-man1: > @for i in $(MAN1_TXT); do echo $$i; done > > lint-docs:: > - $(QUIET_LINT)$(PERL_PATH) lint-gitlink.perl > + $(QUIET_LINT)$(PERL_PATH) lint-gitlink.perl \ > + --section=1 $(MAN1_TXT) \ > + --section=5 $(MAN5_TXT) \ > + --section=7 $(MAN7_TXT) \ > + --to-lint $(ALL_TXT) ^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH 5/5] doc lint: lint and fix missing "GIT" end sections 2021-03-26 10:36 [PATCH 0/5] small doc make and lint fixes Ævar Arnfjörð Bjarmason ` (3 preceding siblings ...) 2021-03-26 10:36 ` [PATCH 4/5] doc lint: fix bugs in, simplify and improve lint script Ævar Arnfjörð Bjarmason @ 2021-03-26 10:36 ` Ævar Arnfjörð Bjarmason 2021-03-26 11:04 ` Jeff King 2021-03-28 6:42 ` Junio C Hamano 2021-03-26 11:06 ` [PATCH 0/5] small doc make and lint fixes Jeff King 2021-04-09 15:02 ` [PATCH v2 0/7] " Ævar Arnfjörð Bjarmason 6 siblings, 2 replies; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-03-26 10:36 UTC (permalink / raw) To: git; +Cc: Junio C Hamano, Jeff King, Ævar Arnfjörð Bjarmason Lint for and fix the three manual pages that were missing the standard "Part of the linkgit:git[1] suite" end section. We only do this for the man[157] section documents (we don't have anything outside those sections), not files to be included, howto *.txt files etc. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> --- Documentation/Makefile | 2 ++ Documentation/git-credential.txt | 4 ++++ Documentation/git-p4.txt | 4 ++++ Documentation/gitnamespaces.txt | 4 ++++ Documentation/lint-man-txt.perl | 24 ++++++++++++++++++++++++ 5 files changed, 38 insertions(+) create mode 100755 Documentation/lint-man-txt.perl diff --git a/Documentation/Makefile b/Documentation/Makefile index 6bfd8c75772..2b6cd0f7be2 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -490,6 +490,8 @@ lint-docs:: --section=5 $(MAN5_TXT) \ --section=7 $(MAN7_TXT) \ --to-lint $(ALL_TXT) + $(QUIET_LINT)$(PERL_PATH) lint-man-txt.perl \ + $(MAN1_TXT) $(MAN5_TXT) $(MAN7_TXT) ifeq ($(wildcard po/Makefile),po/Makefile) doc-l10n install-l10n:: diff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt index 31c81c4c026..206e3c5f407 100644 --- a/Documentation/git-credential.txt +++ b/Documentation/git-credential.txt @@ -159,3 +159,7 @@ empty string. + Components which are missing from the URL (e.g., there is no username in the example above) will be left unset. + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Documentation/git-p4.txt b/Documentation/git-p4.txt index f89e68b424c..38e5257b2a4 100644 --- a/Documentation/git-p4.txt +++ b/Documentation/git-p4.txt @@ -762,3 +762,7 @@ IMPLEMENTATION DETAILS message indicating the p4 depot location and change number. This line is used by later 'git p4 sync' operations to know which p4 changes are new. + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Documentation/gitnamespaces.txt b/Documentation/gitnamespaces.txt index b614969ad2c..1c8d2ecc358 100644 --- a/Documentation/gitnamespaces.txt +++ b/Documentation/gitnamespaces.txt @@ -62,3 +62,7 @@ git clone ext::'git --namespace=foo %s /tmp/prefixed.git' ---------- include::transfer-data-leaks.txt[] + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Documentation/lint-man-txt.perl b/Documentation/lint-man-txt.perl new file mode 100755 index 00000000000..d69312e5db5 --- /dev/null +++ b/Documentation/lint-man-txt.perl @@ -0,0 +1,24 @@ +#!/usr/bin/perl + +use strict; +use warnings; + +my $exit_code = 0; +sub report { + my ($target, $msg) = @_; + print "error: $target: $msg\n"; + $exit_code = 1; +} + +local $/; +while (my $slurp = <>) { + report($ARGV, "has no 'Part of the linkgit:git[1] suite' end blurb") + unless $slurp =~ m[ + ^GIT\n + ---\n + \QPart of the linkgit:git[1] suite\E \n + \z + ]mx; +} + +exit $exit_code; -- 2.31.0.419.gfc6e4cae13 ^ permalink raw reply related [flat|nested] 32+ messages in thread
* Re: [PATCH 5/5] doc lint: lint and fix missing "GIT" end sections 2021-03-26 10:36 ` [PATCH 5/5] doc lint: lint and fix missing "GIT" end sections Ævar Arnfjörð Bjarmason @ 2021-03-26 11:04 ` Jeff King 2021-03-26 15:32 ` Ævar Arnfjörð Bjarmason 2021-03-28 6:42 ` Junio C Hamano 1 sibling, 1 reply; 32+ messages in thread From: Jeff King @ 2021-03-26 11:04 UTC (permalink / raw) To: Ævar Arnfjörð Bjarmason; +Cc: git, Junio C Hamano On Fri, Mar 26, 2021 at 11:36:50AM +0100, Ævar Arnfjörð Bjarmason wrote: > Lint for and fix the three manual pages that were missing the standard > "Part of the linkgit:git[1] suite" end section. This is a definite improvement. Two thoughts come to mind, though: 1. Do we need a separate script for this? Couldn't the existing linter script check this while it is reading all of the files (it knows which ones are supposed to be manpages because they are annotated with the --section option). That would be more efficient, and probably a little less code. 2. Instead of linting, could we just be automatically sticking this boilerplate in as part of the build (either through some asciidoc magic, or even just a plain old "cat")? Even better than being reminded that you forgot something is making it impossible to forget it in the first place. -Peff ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH 5/5] doc lint: lint and fix missing "GIT" end sections 2021-03-26 11:04 ` Jeff King @ 2021-03-26 15:32 ` Ævar Arnfjörð Bjarmason 2021-03-27 9:50 ` Jeff King 0 siblings, 1 reply; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-03-26 15:32 UTC (permalink / raw) To: Jeff King; +Cc: git, Junio C Hamano On Fri, Mar 26 2021, Jeff King wrote: > On Fri, Mar 26, 2021 at 11:36:50AM +0100, Ævar Arnfjörð Bjarmason wrote: > >> Lint for and fix the three manual pages that were missing the standard >> "Part of the linkgit:git[1] suite" end section. > > This is a definite improvement. Two thoughts come to mind, though: > > 1. Do we need a separate script for this? Couldn't the existing linter > script check this while it is reading all of the files (it knows > which ones are supposed to be manpages because they are annotated > with the --section option). It's not needed, but I think it's better, one is iterating a line-at-time, one slurps all lines, they have different sorts of error reporting (one quotes the whole line). So I thought about joining them into one, and also make them and check-non-portable-shell.pl some general lint-line-ish checker. Obviously all of that fits in one script, but I think for something like this that's a one-off script with global variables it's much harder to follow when a large part of your script is some if/else or keeping/resetting of state simply to work around the script doing two things instead of one. I mean, the whole scaffolding is basically: use strict; use warnings; sub report { ... } my $code = 0; while (<>) { ... } exit $code; We'd spend more lines effort trying to consolidate them than just copying that around. > That would be more efficient, and probably a little less code. This thing takes ~5ms to run on my (and most) boxes, by comparison the whole asciidoc dance takes some eons... > 2. Instead of linting, could we just be automatically sticking this > boilerplate in as part of the build (either through some asciidoc > magic, or even just a plain old "cat")? Even better than being > reminded that you forgot something is making it impossible to > forget it in the first place. Whenever I take an aborted effort at the docs I end up with some aborted effort to migrte them to texinfo, so I'm sympathetic to the automatic generation part of this. But for something trivial like this I think there's more value in having a 1=1 match between WYS and WYG, not adding magic blurbs by the build system for something so trivial. That being said I wouldn't mind it much, just seemed like an obvious thing to add a lint for as it stands now... ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH 5/5] doc lint: lint and fix missing "GIT" end sections 2021-03-26 15:32 ` Ævar Arnfjörð Bjarmason @ 2021-03-27 9:50 ` Jeff King 0 siblings, 0 replies; 32+ messages in thread From: Jeff King @ 2021-03-27 9:50 UTC (permalink / raw) To: Ævar Arnfjörð Bjarmason; +Cc: git, Junio C Hamano On Fri, Mar 26, 2021 at 04:32:58PM +0100, Ævar Arnfjörð Bjarmason wrote: > > This is a definite improvement. Two thoughts come to mind, though: > > > > 1. Do we need a separate script for this? Couldn't the existing linter > > script check this while it is reading all of the files (it knows > > which ones are supposed to be manpages because they are annotated > > with the --section option). > > It's not needed, but I think it's better, one is iterating a > line-at-time, one slurps all lines, they have different sorts of error > reporting (one quotes the whole line). > [...] OK. As the person who looked at all of it much more closely, I'll trust your judgement there. > > 2. Instead of linting, could we just be automatically sticking this > > boilerplate in as part of the build (either through some asciidoc > > magic, or even just a plain old "cat")? Even better than being > > reminded that you forgot something is making it impossible to > > forget it in the first place. > > Whenever I take an aborted effort at the docs I end up with some aborted > effort to migrte them to texinfo, so I'm sympathetic to the automatic > generation part of this. > > But for something trivial like this I think there's more value in having > a 1=1 match between WYS and WYG, not adding magic blurbs by the build > system for something so trivial. > > That being said I wouldn't mind it much, just seemed like an obvious > thing to add a lint for as it stands now... Yeah, I agree that complicating the build may create its own problems. If we were already munging the *.txt files it may not be a big deal to add further munging, but it would be a jump from the current state of "we feed it directly to asciidoc". I'm OK to forget about it for now and see if other useful applications of such munging come up. -Peff ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH 5/5] doc lint: lint and fix missing "GIT" end sections 2021-03-26 10:36 ` [PATCH 5/5] doc lint: lint and fix missing "GIT" end sections Ævar Arnfjörð Bjarmason 2021-03-26 11:04 ` Jeff King @ 2021-03-28 6:42 ` Junio C Hamano 2021-03-28 17:53 ` Junio C Hamano 1 sibling, 1 reply; 32+ messages in thread From: Junio C Hamano @ 2021-03-28 6:42 UTC (permalink / raw) To: Ævar Arnfjörð Bjarmason; +Cc: git, Jeff King Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes: > Lint for and fix the three manual pages that were missing the standard > "Part of the linkgit:git[1] suite" end section. > > We only do this for the man[157] section documents (we don't have > anything outside those sections), not files to be included, > howto *.txt files etc. OK. Alternatively we could drop the footer from all pages. IIRC, we used to have another footer section to credit primary authors, which we dropped years ago. I doubt "git-foo is part of git suite" is something worth repeating over and over. Thanks. > Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> > --- > Documentation/Makefile | 2 ++ > Documentation/git-credential.txt | 4 ++++ > Documentation/git-p4.txt | 4 ++++ > Documentation/gitnamespaces.txt | 4 ++++ > Documentation/lint-man-txt.perl | 24 ++++++++++++++++++++++++ > 5 files changed, 38 insertions(+) > create mode 100755 Documentation/lint-man-txt.perl > > diff --git a/Documentation/Makefile b/Documentation/Makefile > index 6bfd8c75772..2b6cd0f7be2 100644 > --- a/Documentation/Makefile > +++ b/Documentation/Makefile > @@ -490,6 +490,8 @@ lint-docs:: > --section=5 $(MAN5_TXT) \ > --section=7 $(MAN7_TXT) \ > --to-lint $(ALL_TXT) > + $(QUIET_LINT)$(PERL_PATH) lint-man-txt.perl \ > + $(MAN1_TXT) $(MAN5_TXT) $(MAN7_TXT) > > ifeq ($(wildcard po/Makefile),po/Makefile) > doc-l10n install-l10n:: > diff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt > index 31c81c4c026..206e3c5f407 100644 > --- a/Documentation/git-credential.txt > +++ b/Documentation/git-credential.txt > @@ -159,3 +159,7 @@ empty string. > + > Components which are missing from the URL (e.g., there is no > username in the example above) will be left unset. > + > +GIT > +--- > +Part of the linkgit:git[1] suite > diff --git a/Documentation/git-p4.txt b/Documentation/git-p4.txt > index f89e68b424c..38e5257b2a4 100644 > --- a/Documentation/git-p4.txt > +++ b/Documentation/git-p4.txt > @@ -762,3 +762,7 @@ IMPLEMENTATION DETAILS > message indicating the p4 depot location and change number. This > line is used by later 'git p4 sync' operations to know which p4 > changes are new. > + > +GIT > +--- > +Part of the linkgit:git[1] suite > diff --git a/Documentation/gitnamespaces.txt b/Documentation/gitnamespaces.txt > index b614969ad2c..1c8d2ecc358 100644 > --- a/Documentation/gitnamespaces.txt > +++ b/Documentation/gitnamespaces.txt > @@ -62,3 +62,7 @@ git clone ext::'git --namespace=foo %s /tmp/prefixed.git' > ---------- > > include::transfer-data-leaks.txt[] > + > +GIT > +--- > +Part of the linkgit:git[1] suite > diff --git a/Documentation/lint-man-txt.perl b/Documentation/lint-man-txt.perl > new file mode 100755 > index 00000000000..d69312e5db5 > --- /dev/null > +++ b/Documentation/lint-man-txt.perl > @@ -0,0 +1,24 @@ > +#!/usr/bin/perl > + > +use strict; > +use warnings; > + > +my $exit_code = 0; > +sub report { > + my ($target, $msg) = @_; > + print "error: $target: $msg\n"; > + $exit_code = 1; > +} > + > +local $/; > +while (my $slurp = <>) { > + report($ARGV, "has no 'Part of the linkgit:git[1] suite' end blurb") > + unless $slurp =~ m[ > + ^GIT\n > + ---\n > + \QPart of the linkgit:git[1] suite\E \n > + \z > + ]mx; > +} > + > +exit $exit_code; ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH 5/5] doc lint: lint and fix missing "GIT" end sections 2021-03-28 6:42 ` Junio C Hamano @ 2021-03-28 17:53 ` Junio C Hamano 2021-04-09 11:49 ` Ævar Arnfjörð Bjarmason 0 siblings, 1 reply; 32+ messages in thread From: Junio C Hamano @ 2021-03-28 17:53 UTC (permalink / raw) To: Ævar Arnfjörð Bjarmason; +Cc: git, Jeff King Junio C Hamano <gitster@pobox.com> writes: > Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes: > >> Lint for and fix the three manual pages that were missing the standard >> "Part of the linkgit:git[1] suite" end section. >> >> We only do this for the man[157] section documents (we don't have >> anything outside those sections), not files to be included, >> howto *.txt files etc. > > OK. Alternatively we could drop the footer from all pages. IIRC, > we used to have another footer section to credit primary authors, > which we dropped years ago. I doubt "git-foo is part of git suite" > is something worth repeating over and over. > > Thanks. Having said that, making sure all pages consistently have them is fine, too. It is just the repetition, even though my eyes have learned to take it as perfectly normal state of affairs, did not seem to be adding that much value. ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH 5/5] doc lint: lint and fix missing "GIT" end sections 2021-03-28 17:53 ` Junio C Hamano @ 2021-04-09 11:49 ` Ævar Arnfjörð Bjarmason 2021-04-10 4:14 ` Junio C Hamano 0 siblings, 1 reply; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-04-09 11:49 UTC (permalink / raw) To: Junio C Hamano; +Cc: git, Jeff King On Sun, Mar 28 2021, Junio C Hamano wrote: > Junio C Hamano <gitster@pobox.com> writes: > >> Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes: >> >>> Lint for and fix the three manual pages that were missing the standard >>> "Part of the linkgit:git[1] suite" end section. >>> >>> We only do this for the man[157] section documents (we don't have >>> anything outside those sections), not files to be included, >>> howto *.txt files etc. >> >> OK. Alternatively we could drop the footer from all pages. IIRC, >> we used to have another footer section to credit primary authors, >> which we dropped years ago. I doubt "git-foo is part of git suite" >> is something worth repeating over and over. >> >> Thanks. > > Having said that, making sure all pages consistently have them is > fine, too. It is just the repetition, even though my eyes have > learned to take it as perfectly normal state of affairs, did not > seem to be adding that much value. It adds the value of: * If you're looking at e.g. git-subtree's manpage it doesn't have it, so we're clearly marking things that are part of git itself, also if you're doing e.g. "man git-annex" (or other non-git.git tool) somewhere. * Once you're at the bottom of a manpage in e.g. a web view it's a handy link back to the start, see e.g. https://git-scm.com/docs/git-add#_git ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH 5/5] doc lint: lint and fix missing "GIT" end sections 2021-04-09 11:49 ` Ævar Arnfjörð Bjarmason @ 2021-04-10 4:14 ` Junio C Hamano 0 siblings, 0 replies; 32+ messages in thread From: Junio C Hamano @ 2021-04-10 4:14 UTC (permalink / raw) To: Ævar Arnfjörð Bjarmason; +Cc: git, Jeff King Ævar Arnfjörð Bjarmason <avarab@gmail.com> writes: > It adds the value of: > > * If you're looking at e.g. git-subtree's manpage it doesn't have it, > so we're clearly marking things that are part of git itself, also if > you're doing e.g. "man git-annex" (or other non-git.git tool) > somewhere. > > * Once you're at the bottom of a manpage in e.g. a web view it's a > handy link back to the start, see > e.g. https://git-scm.com/docs/git-add#_git OK, that's fair enough. Thanks for a clarification. ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH 0/5] small doc make and lint fixes 2021-03-26 10:36 [PATCH 0/5] small doc make and lint fixes Ævar Arnfjörð Bjarmason ` (4 preceding siblings ...) 2021-03-26 10:36 ` [PATCH 5/5] doc lint: lint and fix missing "GIT" end sections Ævar Arnfjörð Bjarmason @ 2021-03-26 11:06 ` Jeff King 2021-03-26 15:18 ` Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 0/7] " Ævar Arnfjörð Bjarmason 6 siblings, 1 reply; 32+ messages in thread From: Jeff King @ 2021-03-26 11:06 UTC (permalink / raw) To: Ævar Arnfjörð Bjarmason; +Cc: git, Junio C Hamano On Fri, Mar 26, 2021 at 11:36:45AM +0100, Ævar Arnfjörð Bjarmason wrote: > A small stand-alone series of doc infrastructure fixes. 5/6 fixes an > interesting bug that's been missed since doc-diff was introduced. I'm not sure what you mean about doc-diff here. Patch 5 (of 5?) doesn't seem related. Patch 4 is related only tangentially, in that you might have more stuff in your Documentation directory. That said... > Documentation/Makefile: make $(wildcard howto/*.txt) a var > Documentation/Makefile: make $(wildcard <doc deps>) a var > doc lint: Perl "strict" and "warnings" in lint-gitlink.perl These three look obviously correct. > doc lint: fix bugs in, simplify and improve lint script > doc lint: lint and fix missing "GIT" end sections These two seem fine, though I left some comments for possible improvements. -Peff ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH 0/5] small doc make and lint fixes 2021-03-26 11:06 ` [PATCH 0/5] small doc make and lint fixes Jeff King @ 2021-03-26 15:18 ` Ævar Arnfjörð Bjarmason 2021-03-27 9:48 ` Jeff King 0 siblings, 1 reply; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-03-26 15:18 UTC (permalink / raw) To: Jeff King; +Cc: git, Junio C Hamano On Fri, Mar 26 2021, Jeff King wrote: > On Fri, Mar 26, 2021 at 11:36:45AM +0100, Ævar Arnfjörð Bjarmason wrote: > >> A small stand-alone series of doc infrastructure fixes. 5/6 fixes an >> interesting bug that's been missed since doc-diff was introduced. > > I'm not sure what you mean about doc-diff here. Patch 5 (of 5?) doesn't > seem related. Patch 4 is related only tangentially, in that you might > have more stuff in your Documentation directory. That's how it's related, the lint script recursively looks for all *.txt under Documentation/, before doc-diff we could safely assume this was *.txt at that version of the repository, after doc-diff our recursively search leads us to a different checkout at a different revision. I don't think it had any practical effect, just say'n. > That said... > >> Documentation/Makefile: make $(wildcard howto/*.txt) a var >> Documentation/Makefile: make $(wildcard <doc deps>) a var >> doc lint: Perl "strict" and "warnings" in lint-gitlink.perl > > These three look obviously correct. > >> doc lint: fix bugs in, simplify and improve lint script >> doc lint: lint and fix missing "GIT" end sections > > These two seem fine, though I left some comments for possible > improvements. > > -Peff ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [PATCH 0/5] small doc make and lint fixes 2021-03-26 15:18 ` Ævar Arnfjörð Bjarmason @ 2021-03-27 9:48 ` Jeff King 0 siblings, 0 replies; 32+ messages in thread From: Jeff King @ 2021-03-27 9:48 UTC (permalink / raw) To: Ævar Arnfjörð Bjarmason; +Cc: git, Junio C Hamano On Fri, Mar 26, 2021 at 04:18:50PM +0100, Ævar Arnfjörð Bjarmason wrote: > >> A small stand-alone series of doc infrastructure fixes. 5/6 fixes an > >> interesting bug that's been missed since doc-diff was introduced. > > > > I'm not sure what you mean about doc-diff here. Patch 5 (of 5?) doesn't > > seem related. Patch 4 is related only tangentially, in that you might > > have more stuff in your Documentation directory. > > That's how it's related, the lint script recursively looks for all *.txt > under Documentation/, before doc-diff we could safely assume this was > *.txt at that version of the repository, after doc-diff our recursively > search leads us to a different checkout at a different revision. > > I don't think it had any practical effect, just say'n. OK, thanks. I just wanted to make sure I wasn't missing something. -Peff ^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v2 0/7] doc make and lint fixes 2021-03-26 10:36 [PATCH 0/5] small doc make and lint fixes Ævar Arnfjörð Bjarmason ` (5 preceding siblings ...) 2021-03-26 11:06 ` [PATCH 0/5] small doc make and lint fixes Jeff King @ 2021-04-09 15:02 ` Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 1/7] Documentation/Makefile: make $(wildcard howto/*.txt) a var Ævar Arnfjörð Bjarmason ` (6 more replies) 6 siblings, 7 replies; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-04-09 15:02 UTC (permalink / raw) To: git Cc: Junio C Hamano, Jeff King, Philip Oakley, Ævar Arnfjörð Bjarmason Lint for and fix the three manual pages that were missing the standard "Part of the linkgit:git[1] suite" end section. Addresses the feedback on v1, in particular stylistic / variable naming choices. I just went with all the suggestions. I've added two new patches at the end to detect (and in 7/7 fix) issues with "standard" sections like NAME, SYNOPSIS, OPTIONS etc. that are out of order, e.g. git-mktag was one of two manual pages that OPTIONS before DESCRIPTION. Those and a few other such oddities are now fixed. I've got a follow-up series to finish the work I started a while ago of including the relevant git config variable descriptions in the respective manual pages, e.g. now we have gc.* variables in git-gc(1), but not in all manual pages. We'll hopefully consistently have that everywhere soon, and this "sections in the right order" lint was very useful to check those patches. Ævar Arnfjörð Bjarmason (7): Documentation/Makefile: make $(wildcard howto/*.txt) a var Documentation/Makefile: make doc.dep dependencies a variable again doc lint: Perl "strict" and "warnings" in lint-gitlink.perl doc lint: fix bugs in, simplify and improve lint script doc lint: lint and fix missing "GIT" end sections doc lint: lint relative section order docs: fix linting issues due to incorrect relative section order Documentation/Makefile | 23 ++++- Documentation/git-credential.txt | 4 + Documentation/git-cvsserver.txt | 24 ++--- Documentation/git-grep.txt | 64 ++++++------- Documentation/git-mktag.txt | 16 ++-- Documentation/git-p4.txt | 4 + Documentation/git-rebase.txt | 12 +-- Documentation/git-svn.txt | 38 ++++---- Documentation/gitnamespaces.txt | 4 + Documentation/lint-gitlink.perl | 108 +++++++++++----------- Documentation/lint-man-end-blurb.perl | 24 +++++ Documentation/lint-man-section-order.perl | 105 +++++++++++++++++++++ 12 files changed, 288 insertions(+), 138 deletions(-) create mode 100755 Documentation/lint-man-end-blurb.perl create mode 100755 Documentation/lint-man-section-order.perl Range-diff against v1: 1: a5ec5fe4bd = 1: 8efebc7a03 Documentation/Makefile: make $(wildcard howto/*.txt) a var 2: fdc55a86cf ! 2: 8046e7941e Documentation/Makefile: make $(wildcard <doc deps>) a var @@ Metadata Author: Ævar Arnfjörð Bjarmason <avarab@gmail.com> ## Commit message ## - Documentation/Makefile: make $(wildcard <doc deps>) a var + Documentation/Makefile: make doc.dep dependencies a variable again - Refactor the wildcard we'll scan for "include" directives into a - single INCLUDE_TARGETS_TXT variable for readability, consistency. + Re-introduce a variable to declare what *.txt files need to be + considered for the purposes of scouring files to generate a dependency + graph of includes. + + When doc.dep was introduced in a5ae8e64cf (Fix documentation + dependency generation., 2005-11-07) we had such a variable called + TEXTFILES, but it was refactored away just a few commits after that in + fb612d54c1 (Documentation: fix dependency generation., + 2005-11-07). I'm planning to add more wildcards here, so let's bring + it back. + + I'm not calling it TEXTFILES because we e.g. don't consider + Documentation/technical/*.txt when generating the graph (they don't + use includes). Let's instead call it DOC_DEP_TXT. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> @@ Documentation/Makefile: MAN1_TXT = MAN5_TXT = MAN7_TXT = HOWTO_TXT = -+INCLUDE_TARGETS_TXT = ++DOC_DEP_TXT = TECH_DOCS = ARTICLES = SP_ARTICLES = @@ Documentation/Makefile: MAN7_TXT += gitworkflows.txt HOWTO_TXT += $(wildcard howto/*.txt) -+INCLUDE_TARGETS_TXT += $(wildcard *.txt) -+INCLUDE_TARGETS_TXT += $(wildcard config/*.txt) ++DOC_DEP_TXT += $(wildcard *.txt) ++DOC_DEP_TXT += $(wildcard config/*.txt) + ifdef MAN_FILTER MAN_TXT = $(filter $(MAN_FILTER),$(MAN1_TXT) $(MAN5_TXT) $(MAN7_TXT)) @@ Documentation/Makefile: docdep_prereqs = \ cmd-list.made $(cmds_txt) -doc.dep : $(docdep_prereqs) $(wildcard *.txt) $(wildcard config/*.txt) build-docdep.perl -+doc.dep : $(docdep_prereqs) $(INCLUDE_TARGETS_TXT) build-docdep.perl ++doc.dep : $(docdep_prereqs) $(DOC_DEP_TXT) build-docdep.perl $(QUIET_GEN)$(RM) $@+ $@ && \ $(PERL_PATH) ./build-docdep.perl >$@+ $(QUIET_STDERR) && \ mv $@+ $@ 3: 12573d9028 = 3: 29545543bd doc lint: Perl "strict" and "warnings" in lint-gitlink.perl 4: 5c8e8f2149 ! 4: 5acb116fea doc lint: fix bugs in, simplify and improve lint script @@ Commit message - When the doc-diff script was added in beb188e22a (add a script to diff rendered documentation, 2018-08-06) we started sometimes having - a "git worktree" under "documentation". This tree contains a full - checkout of git.git, as a result the "lint" script would recurse into - that, and lint any *.txt file found in that entire repository. + a "git worktree" under Documentation/. + + This tree contains a full checkout of git.git, as a result the + "lint" script would recurse into that, and lint any *.txt file + found in that entire repository. In practice the only in-tree "linkgit" outside of the Documentation/ tree is contrib/contacts/git-contacts.txt and contrib/subtree/git-subtree.txt, so this wouldn't emit any errors - Now we instead simply trust the Makefile to give us *.txt files, and - since the Makefile also knows what sections each page should be in we + Now we instead simply trust the Makefile to give us *.txt files. + Since the Makefile also knows what sections each page should be in we don't have to open the files ourselves and try to parse that out. As a - bonus this will also catch bugs with the section line in the file - being incorrect. + bonus this will also catch bugs with the section line in the files + themselves being incorrect. The structure of the new script is mostly based on t/check-non-portable-shell.pl. As an added bonus it will also use @@ Commit message We'll now emit: git-cherry.txt:20: error: git-cherry-pick[2]: wrong section (should be 1), shown with 'HERE' below: - git-cherry.txt:20: "copied" by means of linkgit:git-cherry-pick[2]' <-- HERE + git-cherry.txt:20: '"copied" by means of linkgit:git-cherry-pick[2]' <-- HERE git-cherry.txt:20: error: git-am[3]: wrong section (should be 1), shown with 'HERE' below: - git-cherry.txt:20: "copied" by means of linkgit:git-cherry-pick[2], linkgit:git-am[3]' <-- HERE + git-cherry.txt:20: '"copied" by means of linkgit:git-cherry-pick[2], linkgit:git-am[3]' <-- HERE git-cherry.txt:21: error: git-rebase[4]: wrong section (should be 1), shown with 'HERE' below: - git-cherry.txt:21: linkgit:git-rebase[4]' <-- HERE + git-cherry.txt:21: 'linkgit:git-rebase[4]' <-- HERE Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> ## Documentation/Makefile ## -@@ Documentation/Makefile: MAN5_TXT = - MAN7_TXT = - HOWTO_TXT = - INCLUDE_TARGETS_TXT = -+ALL_TXT = - TECH_DOCS = - ARTICLES = - SP_ARTICLES = -@@ Documentation/Makefile: HOWTO_TXT += $(wildcard howto/*.txt) - INCLUDE_TARGETS_TXT += $(wildcard *.txt) - INCLUDE_TARGETS_TXT += $(wildcard config/*.txt) - -+# For linting -+ALL_TXT += $(MAN1_TXT) -+ALL_TXT += $(MAN5_TXT) -+ALL_TXT += $(MAN7_TXT) -+ALL_TXT += $(HOWTO_TXT) -+ALL_TXT += $(INCLUDE_TARGETS_TXT) -+ - ifdef MAN_FILTER - MAN_TXT = $(filter $(MAN_FILTER),$(MAN1_TXT) $(MAN5_TXT) $(MAN7_TXT)) - else @@ Documentation/Makefile: print-man1: @for i in $(MAN1_TXT); do echo $$i; done lint-docs:: - $(QUIET_LINT)$(PERL_PATH) lint-gitlink.perl + $(QUIET_LINT)$(PERL_PATH) lint-gitlink.perl \ ++ $(HOWTO_TXT) $(DOC_DEP_TXT) \ + --section=1 $(MAN1_TXT) \ + --section=5 $(MAN5_TXT) \ -+ --section=7 $(MAN7_TXT) \ -+ --to-lint $(ALL_TXT) ++ --section=7 $(MAN7_TXT) ifeq ($(wildcard po/Makefile),po/Makefile) doc-l10n install-l10n:: @@ Documentation/lint-gitlink.perl - or die("Cannot parse command line arguments\n"); +# Parse arguments, a simple state machine for input like: +# -+# --section=1 git.txt git-add.txt [...] --to-lint git-add.txt a-file.txt [...] ++# howto/*.txt config/*.txt --section=1 git.txt git-add.txt [...] --to-lint git-add.txt a-file.txt [...] +my %TXT; +my %SECTION; +my $section; @@ Documentation/lint-gitlink.perl + if (my ($sec) = $arg =~ /^--section=(\d+)$/s) { + $section = $sec; + next; -+ } elsif ($arg eq '--to-lint') { -+ $lint_these = 1; -+ next; + } -my $found_errors = 0; + my ($name) = $arg =~ /^(.*?)\.txt$/s; -+ if ($lint_these) { ++ unless (defined $section) { + $TXT{$name} = $arg; + next; + } @@ Documentation/lint-gitlink.perl + substr($line, $pos) = "' <-- HERE"; + $line =~ s/^\s+//; + print "$ARGV:$.: error: $target: $msg, shown with 'HERE' below:\n"; -+ print "$ARGV:$.:\t$line\n"; ++ print "$ARGV:$.:\t'$line\n"; + $exit_code = 1; } @@ Documentation/lint-gitlink.perl - while (s/linkgit:((.*?)\[(\d)\])//) { - my ($target, $page, $section) = ($1, $2, $3); +@ARGV = sort values %TXT; ++die "BUG: Nothing to process!" unless @ARGV; +while (<>) { + my $line = $_; + while ($line =~ m/linkgit:((.*?)\[(\d)\])/g) { 5: d4004b6a7c ! 5: e3af1a9405 doc lint: lint and fix missing "GIT" end sections @@ Commit message anything outside those sections), not files to be included, howto *.txt files etc. + We could also add this to the existing (and then renamed) + lint-gitlink.perl, but I'm not doing that here. + + Obviously all of that fits in one script, but I think for something + like this that's a one-off script with global variables it's much + harder to follow when a large part of your script is some if/else or + keeping/resetting of state simply to work around the script doing two + things instead of one. + + Especially because in this case this script wants to process the file + as one big string, but lint-gitlink.perl wants to look at it one line + at a time. We could also consolidate this whole thing and + t/check-non-portable-shell.pl, but that one likes to join lines as + part of its shell parsing. + + So let's just add another script, whole scaffolding is basically: + + use strict; + use warnings; + sub report { ... } + my $code = 0; + while (<>) { ... } + exit $code; + + We'd spend more lines effort trying to consolidate them than just + copying that around. + Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> ## Documentation/Makefile ## @@ Documentation/Makefile: lint-docs:: + $(HOWTO_TXT) $(DOC_DEP_TXT) \ + --section=1 $(MAN1_TXT) \ --section=5 $(MAN5_TXT) \ - --section=7 $(MAN7_TXT) \ - --to-lint $(ALL_TXT) -+ $(QUIET_LINT)$(PERL_PATH) lint-man-txt.perl \ -+ $(MAN1_TXT) $(MAN5_TXT) $(MAN7_TXT) +- --section=7 $(MAN7_TXT) ++ --section=7 $(MAN7_TXT); \ ++ $(PERL_PATH) lint-man-end-blurb.perl $(MAN_TXT) ifeq ($(wildcard po/Makefile),po/Makefile) doc-l10n install-l10n:: @@ Documentation/gitnamespaces.txt: git clone ext::'git --namespace=foo %s /tmp/pre +--- +Part of the linkgit:git[1] suite - ## Documentation/lint-man-txt.perl (new) ## + ## Documentation/lint-man-end-blurb.perl (new) ## @@ +#!/usr/bin/perl + -: ---------- > 6: 8c294afe2a doc lint: lint relative section order -: ---------- > 7: 9cb100826e docs: fix linting issues due to incorrect relative section order -- 2.31.1.622.g1b8cc22e65 ^ permalink raw reply [flat|nested] 32+ messages in thread
* [PATCH v2 1/7] Documentation/Makefile: make $(wildcard howto/*.txt) a var 2021-04-09 15:02 ` [PATCH v2 0/7] " Ævar Arnfjörð Bjarmason @ 2021-04-09 15:02 ` Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 2/7] Documentation/Makefile: make doc.dep dependencies a variable again Ævar Arnfjörð Bjarmason ` (5 subsequent siblings) 6 siblings, 0 replies; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-04-09 15:02 UTC (permalink / raw) To: git Cc: Junio C Hamano, Jeff King, Philip Oakley, Ævar Arnfjörð Bjarmason Refactor occurrences of $(wildcard howto/*.txt) into a single HOWTO_TXT variable for readability and consistency. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> --- Documentation/Makefile | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/Documentation/Makefile b/Documentation/Makefile index 874a01d7a8..f0597777b9 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -2,6 +2,7 @@ MAN1_TXT = MAN5_TXT = MAN7_TXT = +HOWTO_TXT = TECH_DOCS = ARTICLES = SP_ARTICLES = @@ -42,6 +43,8 @@ MAN7_TXT += gittutorial-2.txt MAN7_TXT += gittutorial.txt MAN7_TXT += gitworkflows.txt +HOWTO_TXT += $(wildcard howto/*.txt) + ifdef MAN_FILTER MAN_TXT = $(filter $(MAN_FILTER),$(MAN1_TXT) $(MAN5_TXT) $(MAN7_TXT)) else @@ -428,9 +431,9 @@ $(patsubst %.txt,%.texi,$(MAN_TXT)): %.texi : %.xml $(DOCBOOK2X_TEXI) --to-stdout $*.xml >$@+ && \ mv $@+ $@ -howto-index.txt: howto-index.sh $(wildcard howto/*.txt) +howto-index.txt: howto-index.sh $(HOWTO_TXT) $(QUIET_GEN)$(RM) $@+ $@ && \ - '$(SHELL_PATH_SQ)' ./howto-index.sh $(sort $(wildcard howto/*.txt)) >$@+ && \ + '$(SHELL_PATH_SQ)' ./howto-index.sh $(sort $(HOWTO_TXT)) >$@+ && \ mv $@+ $@ $(patsubst %,%.html,$(ARTICLES)) : %.html : %.txt @@ -439,7 +442,7 @@ $(patsubst %,%.html,$(ARTICLES)) : %.html : %.txt WEBDOC_DEST = /pub/software/scm/git/docs howto/%.html: ASCIIDOC_EXTRA += -a git-relative-html-prefix=../ -$(patsubst %.txt,%.html,$(wildcard howto/*.txt)): %.html : %.txt GIT-ASCIIDOCFLAGS +$(patsubst %.txt,%.html,$(HOWTO_TXT)): %.html : %.txt GIT-ASCIIDOCFLAGS $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \ sed -e '1,/^$$/d' $< | \ $(TXT_TO_HTML) - >$@+ && \ -- 2.31.1.622.g1b8cc22e65 ^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v2 2/7] Documentation/Makefile: make doc.dep dependencies a variable again 2021-04-09 15:02 ` [PATCH v2 0/7] " Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 1/7] Documentation/Makefile: make $(wildcard howto/*.txt) a var Ævar Arnfjörð Bjarmason @ 2021-04-09 15:02 ` Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 3/7] doc lint: Perl "strict" and "warnings" in lint-gitlink.perl Ævar Arnfjörð Bjarmason ` (4 subsequent siblings) 6 siblings, 0 replies; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-04-09 15:02 UTC (permalink / raw) To: git Cc: Junio C Hamano, Jeff King, Philip Oakley, Ævar Arnfjörð Bjarmason Re-introduce a variable to declare what *.txt files need to be considered for the purposes of scouring files to generate a dependency graph of includes. When doc.dep was introduced in a5ae8e64cf (Fix documentation dependency generation., 2005-11-07) we had such a variable called TEXTFILES, but it was refactored away just a few commits after that in fb612d54c1 (Documentation: fix dependency generation., 2005-11-07). I'm planning to add more wildcards here, so let's bring it back. I'm not calling it TEXTFILES because we e.g. don't consider Documentation/technical/*.txt when generating the graph (they don't use includes). Let's instead call it DOC_DEP_TXT. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> --- Documentation/Makefile | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/Documentation/Makefile b/Documentation/Makefile index f0597777b9..164d5ff2f9 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -3,6 +3,7 @@ MAN1_TXT = MAN5_TXT = MAN7_TXT = HOWTO_TXT = +DOC_DEP_TXT = TECH_DOCS = ARTICLES = SP_ARTICLES = @@ -45,6 +46,9 @@ MAN7_TXT += gitworkflows.txt HOWTO_TXT += $(wildcard howto/*.txt) +DOC_DEP_TXT += $(wildcard *.txt) +DOC_DEP_TXT += $(wildcard config/*.txt) + ifdef MAN_FILTER MAN_TXT = $(filter $(MAN_FILTER),$(MAN1_TXT) $(MAN5_TXT) $(MAN7_TXT)) else @@ -288,7 +292,7 @@ docdep_prereqs = \ mergetools-list.made $(mergetools_txt) \ cmd-list.made $(cmds_txt) -doc.dep : $(docdep_prereqs) $(wildcard *.txt) $(wildcard config/*.txt) build-docdep.perl +doc.dep : $(docdep_prereqs) $(DOC_DEP_TXT) build-docdep.perl $(QUIET_GEN)$(RM) $@+ $@ && \ $(PERL_PATH) ./build-docdep.perl >$@+ $(QUIET_STDERR) && \ mv $@+ $@ -- 2.31.1.622.g1b8cc22e65 ^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v2 3/7] doc lint: Perl "strict" and "warnings" in lint-gitlink.perl 2021-04-09 15:02 ` [PATCH v2 0/7] " Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 1/7] Documentation/Makefile: make $(wildcard howto/*.txt) a var Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 2/7] Documentation/Makefile: make doc.dep dependencies a variable again Ævar Arnfjörð Bjarmason @ 2021-04-09 15:02 ` Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 4/7] doc lint: fix bugs in, simplify and improve lint script Ævar Arnfjörð Bjarmason ` (3 subsequent siblings) 6 siblings, 0 replies; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-04-09 15:02 UTC (permalink / raw) To: git Cc: Junio C Hamano, Jeff King, Philip Oakley, Ævar Arnfjörð Bjarmason Amend this script added in ab81411ced (ci: validate "linkgit:" in documentation, 2016-05-04) to pass under "use strict", and add a "use warnings" for good measure. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> --- Documentation/lint-gitlink.perl | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/Documentation/lint-gitlink.perl b/Documentation/lint-gitlink.perl index 476cc30b83..35230875c2 100755 --- a/Documentation/lint-gitlink.perl +++ b/Documentation/lint-gitlink.perl @@ -1,5 +1,7 @@ #!/usr/bin/perl +use strict; +use warnings; use File::Find; use Getopt::Long; @@ -45,7 +47,7 @@ sub lint { report($where, $target, "no such source"); next; } - $real_section = grab_section($page); + my $real_section = grab_section($page); if ($real_section != $section) { report($where, $target, "wrong section (should be $real_section)"); -- 2.31.1.622.g1b8cc22e65 ^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v2 4/7] doc lint: fix bugs in, simplify and improve lint script 2021-04-09 15:02 ` [PATCH v2 0/7] " Ævar Arnfjörð Bjarmason ` (2 preceding siblings ...) 2021-04-09 15:02 ` [PATCH v2 3/7] doc lint: Perl "strict" and "warnings" in lint-gitlink.perl Ævar Arnfjörð Bjarmason @ 2021-04-09 15:02 ` Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 5/7] doc lint: lint and fix missing "GIT" end sections Ævar Arnfjörð Bjarmason ` (2 subsequent siblings) 6 siblings, 0 replies; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-04-09 15:02 UTC (permalink / raw) To: git Cc: Junio C Hamano, Jeff King, Philip Oakley, Ævar Arnfjörð Bjarmason The lint-gitlink.perl script added in ab81411ced (ci: validate "linkgit:" in documentation, 2016-05-04) was more complex than it needed to be. It: - Was using File::Find to recursively find *.txt files in Documentation/, let's instead use the Makefile as a source of truth for *.txt files, and pass it down to the script. - We now don't lint linkgit:* in RelNotes/* or technical/*, which we shouldn't have been doing in the first place anyway. - When the doc-diff script was added in beb188e22a (add a script to diff rendered documentation, 2018-08-06) we started sometimes having a "git worktree" under Documentation/. This tree contains a full checkout of git.git, as a result the "lint" script would recurse into that, and lint any *.txt file found in that entire repository. In practice the only in-tree "linkgit" outside of the Documentation/ tree is contrib/contacts/git-contacts.txt and contrib/subtree/git-subtree.txt, so this wouldn't emit any errors Now we instead simply trust the Makefile to give us *.txt files. Since the Makefile also knows what sections each page should be in we don't have to open the files ourselves and try to parse that out. As a bonus this will also catch bugs with the section line in the files themselves being incorrect. The structure of the new script is mostly based on t/check-non-portable-shell.pl. As an added bonus it will also use pos() to print where the problems it finds are, e.g. given an issue like: diff --git a/Documentation/git-cherry.txt b/Documentation/git-cherry.txt [...] and line numbers. git-cherry therefore detects when commits have been -"copied" by means of linkgit:git-cherry-pick[1], linkgit:git-am[1] or -linkgit:git-rebase[1]. +"copied" by means of linkgit:git-cherry-pick[2], linkgit:git-am[3] or +linkgit:git-rebase[4]. We'll now emit: git-cherry.txt:20: error: git-cherry-pick[2]: wrong section (should be 1), shown with 'HERE' below: git-cherry.txt:20: '"copied" by means of linkgit:git-cherry-pick[2]' <-- HERE git-cherry.txt:20: error: git-am[3]: wrong section (should be 1), shown with 'HERE' below: git-cherry.txt:20: '"copied" by means of linkgit:git-cherry-pick[2], linkgit:git-am[3]' <-- HERE git-cherry.txt:21: error: git-rebase[4]: wrong section (should be 1), shown with 'HERE' below: git-cherry.txt:21: 'linkgit:git-rebase[4]' <-- HERE Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> --- Documentation/Makefile | 6 +- Documentation/lint-gitlink.perl | 106 +++++++++++++++----------------- 2 files changed, 55 insertions(+), 57 deletions(-) diff --git a/Documentation/Makefile b/Documentation/Makefile index 164d5ff2f9..de55c4ecf5 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -478,7 +478,11 @@ print-man1: @for i in $(MAN1_TXT); do echo $$i; done lint-docs:: - $(QUIET_LINT)$(PERL_PATH) lint-gitlink.perl + $(QUIET_LINT)$(PERL_PATH) lint-gitlink.perl \ + $(HOWTO_TXT) $(DOC_DEP_TXT) \ + --section=1 $(MAN1_TXT) \ + --section=5 $(MAN5_TXT) \ + --section=7 $(MAN7_TXT) ifeq ($(wildcard po/Makefile),po/Makefile) doc-l10n install-l10n:: diff --git a/Documentation/lint-gitlink.perl b/Documentation/lint-gitlink.perl index 35230875c2..b22a367844 100755 --- a/Documentation/lint-gitlink.perl +++ b/Documentation/lint-gitlink.perl @@ -2,72 +2,66 @@ use strict; use warnings; -use File::Find; -use Getopt::Long; -my $basedir = "."; -GetOptions("basedir=s" => \$basedir) - or die("Cannot parse command line arguments\n"); +# Parse arguments, a simple state machine for input like: +# +# howto/*.txt config/*.txt --section=1 git.txt git-add.txt [...] --to-lint git-add.txt a-file.txt [...] +my %TXT; +my %SECTION; +my $section; +my $lint_these = 0; +for my $arg (@ARGV) { + if (my ($sec) = $arg =~ /^--section=(\d+)$/s) { + $section = $sec; + next; + } -my $found_errors = 0; + my ($name) = $arg =~ /^(.*?)\.txt$/s; + unless (defined $section) { + $TXT{$name} = $arg; + next; + } -sub report { - my ($where, $what, $error) = @_; - print "$where: $error: $what\n"; - $found_errors = 1; + $SECTION{$name} = $section; } -sub grab_section { - my ($page) = @_; - open my $fh, "<", "$basedir/$page.txt"; - my $firstline = <$fh>; - chomp $firstline; - close $fh; - my ($section) = ($firstline =~ /.*\((\d)\)$/); - return $section; +my $exit_code = 0; +sub report { + my ($pos, $line, $target, $msg) = @_; + substr($line, $pos) = "' <-- HERE"; + $line =~ s/^\s+//; + print "$ARGV:$.: error: $target: $msg, shown with 'HERE' below:\n"; + print "$ARGV:$.:\t'$line\n"; + $exit_code = 1; } -sub lint { - my ($file) = @_; - open my $fh, "<", $file - or return; - while (<$fh>) { - my $where = "$file:$."; - while (s/linkgit:((.*?)\[(\d)\])//) { - my ($target, $page, $section) = ($1, $2, $3); +@ARGV = sort values %TXT; +die "BUG: Nothing to process!" unless @ARGV; +while (<>) { + my $line = $_; + while ($line =~ m/linkgit:((.*?)\[(\d)\])/g) { + my $pos = pos $line; + my ($target, $page, $section) = ($1, $2, $3); - # De-AsciiDoc - $page =~ s/{litdd}/--/g; + # De-AsciiDoc + $page =~ s/{litdd}/--/g; - if ($page !~ /^git/) { - report($where, $target, "nongit link"); - next; - } - if (! -f "$basedir/$page.txt") { - report($where, $target, "no such source"); - next; - } - my $real_section = grab_section($page); - if ($real_section != $section) { - report($where, $target, - "wrong section (should be $real_section)"); - next; - } + if (!exists $TXT{$page}) { + report($pos, $line, $target, "link outside of our own docs"); + next; + } + if (!exists $SECTION{$page}) { + report($pos, $line, $target, "link outside of our sectioned docs"); + next; + } + my $real_section = $SECTION{$page}; + if ($section != $SECTION{$page}) { + report($pos, $line, $target, "wrong section (should be $real_section)"); + next; } } - close $fh; -} - -sub lint_it { - lint($File::Find::name) if -f && /\.txt$/; -} - -if (!@ARGV) { - find({ wanted => \&lint_it, no_chdir => 1 }, $basedir); -} else { - for (@ARGV) { - lint($_); - } + # this resets our $. for each file + close ARGV if eof; } -exit $found_errors; +exit $exit_code; -- 2.31.1.622.g1b8cc22e65 ^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v2 5/7] doc lint: lint and fix missing "GIT" end sections 2021-04-09 15:02 ` [PATCH v2 0/7] " Ævar Arnfjörð Bjarmason ` (3 preceding siblings ...) 2021-04-09 15:02 ` [PATCH v2 4/7] doc lint: fix bugs in, simplify and improve lint script Ævar Arnfjörð Bjarmason @ 2021-04-09 15:02 ` Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 6/7] doc lint: lint relative section order Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 7/7] docs: fix linting issues due to incorrect " Ævar Arnfjörð Bjarmason 6 siblings, 0 replies; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-04-09 15:02 UTC (permalink / raw) To: git Cc: Junio C Hamano, Jeff King, Philip Oakley, Ævar Arnfjörð Bjarmason Lint for and fix the three manual pages that were missing the standard "Part of the linkgit:git[1] suite" end section. We only do this for the man[157] section documents (we don't have anything outside those sections), not files to be included, howto *.txt files etc. We could also add this to the existing (and then renamed) lint-gitlink.perl, but I'm not doing that here. Obviously all of that fits in one script, but I think for something like this that's a one-off script with global variables it's much harder to follow when a large part of your script is some if/else or keeping/resetting of state simply to work around the script doing two things instead of one. Especially because in this case this script wants to process the file as one big string, but lint-gitlink.perl wants to look at it one line at a time. We could also consolidate this whole thing and t/check-non-portable-shell.pl, but that one likes to join lines as part of its shell parsing. So let's just add another script, whole scaffolding is basically: use strict; use warnings; sub report { ... } my $code = 0; while (<>) { ... } exit $code; We'd spend more lines effort trying to consolidate them than just copying that around. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> --- Documentation/Makefile | 3 ++- Documentation/git-credential.txt | 4 ++++ Documentation/git-p4.txt | 4 ++++ Documentation/gitnamespaces.txt | 4 ++++ Documentation/lint-man-end-blurb.perl | 24 ++++++++++++++++++++++++ 5 files changed, 38 insertions(+), 1 deletion(-) create mode 100755 Documentation/lint-man-end-blurb.perl diff --git a/Documentation/Makefile b/Documentation/Makefile index de55c4ecf5..34b4f5716a 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -482,7 +482,8 @@ lint-docs:: $(HOWTO_TXT) $(DOC_DEP_TXT) \ --section=1 $(MAN1_TXT) \ --section=5 $(MAN5_TXT) \ - --section=7 $(MAN7_TXT) + --section=7 $(MAN7_TXT); \ + $(PERL_PATH) lint-man-end-blurb.perl $(MAN_TXT) ifeq ($(wildcard po/Makefile),po/Makefile) doc-l10n install-l10n:: diff --git a/Documentation/git-credential.txt b/Documentation/git-credential.txt index 31c81c4c02..206e3c5f40 100644 --- a/Documentation/git-credential.txt +++ b/Documentation/git-credential.txt @@ -159,3 +159,7 @@ empty string. + Components which are missing from the URL (e.g., there is no username in the example above) will be left unset. + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Documentation/git-p4.txt b/Documentation/git-p4.txt index f89e68b424..38e5257b2a 100644 --- a/Documentation/git-p4.txt +++ b/Documentation/git-p4.txt @@ -762,3 +762,7 @@ IMPLEMENTATION DETAILS message indicating the p4 depot location and change number. This line is used by later 'git p4 sync' operations to know which p4 changes are new. + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Documentation/gitnamespaces.txt b/Documentation/gitnamespaces.txt index b614969ad2..1c8d2ecc35 100644 --- a/Documentation/gitnamespaces.txt +++ b/Documentation/gitnamespaces.txt @@ -62,3 +62,7 @@ git clone ext::'git --namespace=foo %s /tmp/prefixed.git' ---------- include::transfer-data-leaks.txt[] + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Documentation/lint-man-end-blurb.perl b/Documentation/lint-man-end-blurb.perl new file mode 100755 index 0000000000..d69312e5db --- /dev/null +++ b/Documentation/lint-man-end-blurb.perl @@ -0,0 +1,24 @@ +#!/usr/bin/perl + +use strict; +use warnings; + +my $exit_code = 0; +sub report { + my ($target, $msg) = @_; + print "error: $target: $msg\n"; + $exit_code = 1; +} + +local $/; +while (my $slurp = <>) { + report($ARGV, "has no 'Part of the linkgit:git[1] suite' end blurb") + unless $slurp =~ m[ + ^GIT\n + ---\n + \QPart of the linkgit:git[1] suite\E \n + \z + ]mx; +} + +exit $exit_code; -- 2.31.1.622.g1b8cc22e65 ^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v2 6/7] doc lint: lint relative section order 2021-04-09 15:02 ` [PATCH v2 0/7] " Ævar Arnfjörð Bjarmason ` (4 preceding siblings ...) 2021-04-09 15:02 ` [PATCH v2 5/7] doc lint: lint and fix missing "GIT" end sections Ævar Arnfjörð Bjarmason @ 2021-04-09 15:02 ` Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 7/7] docs: fix linting issues due to incorrect " Ævar Arnfjörð Bjarmason 6 siblings, 0 replies; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-04-09 15:02 UTC (permalink / raw) To: git Cc: Junio C Hamano, Jeff King, Philip Oakley, Ævar Arnfjörð Bjarmason Add a linting script to check the relative order of the sections in the documentation. We should have NAME, then SYNOPSIS, DESCRIPTION, OPTIONS etc. in that order. That holds true throughout our documentation, except for a few exceptions which are hardcoded in the linting script. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> --- Documentation/Makefile | 3 +- Documentation/lint-man-section-order.perl | 125 ++++++++++++++++++++++ 2 files changed, 127 insertions(+), 1 deletion(-) create mode 100755 Documentation/lint-man-section-order.perl diff --git a/Documentation/Makefile b/Documentation/Makefile index 34b4f5716a..5e0828869b 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -483,7 +483,8 @@ lint-docs:: --section=1 $(MAN1_TXT) \ --section=5 $(MAN5_TXT) \ --section=7 $(MAN7_TXT); \ - $(PERL_PATH) lint-man-end-blurb.perl $(MAN_TXT) + $(PERL_PATH) lint-man-end-blurb.perl $(MAN_TXT); \ + $(PERL_PATH) lint-man-section-order.perl $(MAN_TXT); ifeq ($(wildcard po/Makefile),po/Makefile) doc-l10n install-l10n:: diff --git a/Documentation/lint-man-section-order.perl b/Documentation/lint-man-section-order.perl new file mode 100755 index 0000000000..5767e7e456 --- /dev/null +++ b/Documentation/lint-man-section-order.perl @@ -0,0 +1,125 @@ +#!/usr/bin/perl + +use strict; +use warnings; + +my %SECTIONS; +{ + my $order = 0; + %SECTIONS = ( + 'NAME' => { + required => 1, + order => $order++, + }, + 'SYNOPSIS' => { + required => 1, + order => $order++, + }, + 'DESCRIPTION' => { + required => 1, + order => $order++, + bad => { + 'git-mktag.txt' => 'OPTIONS', + 'git-cvsserver.txt' => 'OPTIONS', + }, + }, + 'OPTIONS' => { + order => $order++, + required => 0, + bad => { + 'git-grep.txt' => 'CONFIGURATION', + 'git-rebase.txt' => 'CONFIGURATION', + }, + }, + 'CONFIGURATION' => { + order => $order++, + bad => { + 'git-svn.txt' => 'BUGS', + }, + }, + 'BUGS' => { + order => $order++, + }, + 'SEE ALSO' => { + order => $order++, + }, + 'GIT' => { + required => 1, + order => $order++, + }, + ); +} +my $SECTION_RX = do { + my ($names) = join "|", keys %SECTIONS; + qr/^($names)$/s; +}; + +my $exit_code = 0; +sub report { + my ($msg) = @_; + print "$ARGV:$.: $msg\n"; + $exit_code = 1; +} + +my $last_was_section; +my @actual_order; +while (my $line = <>) { + chomp $line; + if ($line =~ $SECTION_RX) { + push @actual_order => $line; + $last_was_section = 1; + # Have no "last" section yet, processing NAME + next if @actual_order == 1; + + my @expected_order = sort { + $SECTIONS{$a}->{order} <=> $SECTIONS{$b}->{order} + } @actual_order; + + my $expected_last = $expected_order[-2]; + my $actual_last = $actual_order[-2]; + my $except_last = $SECTIONS{$line}->{bad}->{$ARGV} || ''; + if (($SECTIONS{$line}->{bad}->{$ARGV} || '') eq $actual_last) { + # Either we're whitelisted, or ... + next + } elsif (exists $SECTIONS{$actual_last}->{bad}->{$ARGV}) { + # ... we're complaing about the next section + # which is out of order because this one is, + # don't complain about that one. + next; + } elsif ($actual_last ne $expected_last) { + report("section '$line' incorrectly ordered, comes after '$actual_last'"); + } + next; + } + if ($last_was_section) { + my $last_section = $actual_order[-1]; + if (length $last_section ne length $line) { + report("dashes under '$last_section' should match its length!"); + } + if ($line !~ /^-+$/) { + report("dashes under '$last_section' should be '-' dashes!"); + } + $last_was_section = 0; + } + + if (eof) { + # We have both a hash and an array to consider, for + # convenience + my %actual_sections; + @actual_sections{@actual_order} = (); + + for my $section (sort keys %SECTIONS) { + next if !$SECTIONS{$section}->{required} or exists $actual_sections{$section}; + report("has no required '$section' section!"); + } + + # Reset per-file state + { + @actual_order = (); + # this resets our $. for each file + close ARGV; + } + } +} + +exit $exit_code; -- 2.31.1.622.g1b8cc22e65 ^ permalink raw reply related [flat|nested] 32+ messages in thread
* [PATCH v2 7/7] docs: fix linting issues due to incorrect relative section order 2021-04-09 15:02 ` [PATCH v2 0/7] " Ævar Arnfjörð Bjarmason ` (5 preceding siblings ...) 2021-04-09 15:02 ` [PATCH v2 6/7] doc lint: lint relative section order Ævar Arnfjörð Bjarmason @ 2021-04-09 15:02 ` Ævar Arnfjörð Bjarmason 6 siblings, 0 replies; 32+ messages in thread From: Ævar Arnfjörð Bjarmason @ 2021-04-09 15:02 UTC (permalink / raw) To: git Cc: Junio C Hamano, Jeff King, Philip Oakley, Ævar Arnfjörð Bjarmason Re-order the sections of a few manual pages to be consistent with the entirety of the rest of our documentation. This allows us to remove the just-added whitelist of "bad" order from lint-man-section-order.perl. I'm doing that this way around so that code will be easy to dig up if we'll need it in the future. I've intentionally not added some other sections such as EXAMPLES to the list of known sections. If we were to add that we'd find some out of order. Perhaps we'll want to order those consistently as well in the future, at which point whitelisting some of them might become handy again. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> --- Documentation/git-cvsserver.txt | 24 ++++----- Documentation/git-grep.txt | 64 +++++++++++------------ Documentation/git-mktag.txt | 16 +++--- Documentation/git-rebase.txt | 12 ++--- Documentation/git-svn.txt | 38 +++++++------- Documentation/lint-man-section-order.perl | 22 +------- 6 files changed, 78 insertions(+), 98 deletions(-) diff --git a/Documentation/git-cvsserver.txt b/Documentation/git-cvsserver.txt index 1b1c71ad9d..f2e4a47ebe 100644 --- a/Documentation/git-cvsserver.txt +++ b/Documentation/git-cvsserver.txt @@ -24,6 +24,18 @@ Usage: [verse] 'git-cvsserver' [<options>] [pserver|server] [<directory> ...] +DESCRIPTION +----------- + +This application is a CVS emulation layer for Git. + +It is highly functional. However, not all methods are implemented, +and for those methods that are implemented, +not all switches are implemented. + +Testing has been done using both the CLI CVS client, and the Eclipse CVS +plugin. Most functionality works fine with both of these clients. + OPTIONS ------- @@ -57,18 +69,6 @@ access still needs to be enabled by the `gitcvs.enabled` config option unless `--export-all` was given, too. -DESCRIPTION ------------ - -This application is a CVS emulation layer for Git. - -It is highly functional. However, not all methods are implemented, -and for those methods that are implemented, -not all switches are implemented. - -Testing has been done using both the CLI CVS client, and the Eclipse CVS -plugin. Most functionality works fine with both of these clients. - LIMITATIONS ----------- diff --git a/Documentation/git-grep.txt b/Documentation/git-grep.txt index 4e0ba8234a..3d393fbac1 100644 --- a/Documentation/git-grep.txt +++ b/Documentation/git-grep.txt @@ -38,38 +38,6 @@ are lists of one or more search expressions separated by newline characters. An empty string as search expression matches all lines. -CONFIGURATION -------------- - -grep.lineNumber:: - If set to true, enable `-n` option by default. - -grep.column:: - If set to true, enable the `--column` option by default. - -grep.patternType:: - Set the default matching behavior. Using a value of 'basic', 'extended', - 'fixed', or 'perl' will enable the `--basic-regexp`, `--extended-regexp`, - `--fixed-strings`, or `--perl-regexp` option accordingly, while the - value 'default' will return to the default matching behavior. - -grep.extendedRegexp:: - If set to true, enable `--extended-regexp` option by default. This - option is ignored when the `grep.patternType` option is set to a value - other than 'default'. - -grep.threads:: - Number of grep worker threads to use. If unset (or set to 0), Git will - use as many threads as the number of logical cores available. - -grep.fullName:: - If set to true, enable `--full-name` option by default. - -grep.fallbackToNoIndex:: - If set to true, fall back to git grep --no-index if git grep - is executed outside of a git repository. Defaults to false. - - OPTIONS ------- --cached:: @@ -363,6 +331,38 @@ with multiple threads might perform slower than single threaded if `--textconv` is given and there're too many text conversions. So if you experience low performance in this case, it might be desirable to use `--threads=1`. +CONFIGURATION +------------- + +grep.lineNumber:: + If set to true, enable `-n` option by default. + +grep.column:: + If set to true, enable the `--column` option by default. + +grep.patternType:: + Set the default matching behavior. Using a value of 'basic', 'extended', + 'fixed', or 'perl' will enable the `--basic-regexp`, `--extended-regexp`, + `--fixed-strings`, or `--perl-regexp` option accordingly, while the + value 'default' will return to the default matching behavior. + +grep.extendedRegexp:: + If set to true, enable `--extended-regexp` option by default. This + option is ignored when the `grep.patternType` option is set to a value + other than 'default'. + +grep.threads:: + Number of grep worker threads to use. If unset (or set to 0), Git will + use as many threads as the number of logical cores available. + +grep.fullName:: + If set to true, enable `--full-name` option by default. + +grep.fallbackToNoIndex:: + If set to true, fall back to git grep --no-index if git grep + is executed outside of a git repository. Defaults to false. + + GIT --- Part of the linkgit:git[1] suite diff --git a/Documentation/git-mktag.txt b/Documentation/git-mktag.txt index 17a2603a60..466a697519 100644 --- a/Documentation/git-mktag.txt +++ b/Documentation/git-mktag.txt @@ -11,14 +11,6 @@ SYNOPSIS [verse] 'git mktag' -OPTIONS -------- - ---strict:: - By default mktag turns on the equivalent of - linkgit:git-fsck[1] `--strict` mode. Use `--no-strict` to - disable it. - DESCRIPTION ----------- @@ -45,6 +37,14 @@ the appropriate `fsck.<msg-id>` varible: git -c fsck.extraHeaderEntry=ignore mktag <my-tag-with-headers +OPTIONS +------- + +--strict:: + By default mktag turns on the equivalent of + linkgit:git-fsck[1] `--strict` mode. Use `--no-strict` to + disable it. + Tag Format ---------- A tag signature file, to be fed to this command's standard input, diff --git a/Documentation/git-rebase.txt b/Documentation/git-rebase.txt index f08ae27e2a..8423d46372 100644 --- a/Documentation/git-rebase.txt +++ b/Documentation/git-rebase.txt @@ -200,12 +200,6 @@ Alternatively, you can undo the 'git rebase' with git rebase --abort -CONFIGURATION -------------- - -include::config/rebase.txt[] -include::config/sequencer.txt[] - OPTIONS ------- --onto <newbase>:: @@ -1266,6 +1260,12 @@ merge tlsv1.3 merge cmake ------------ +CONFIGURATION +------------- + +include::config/rebase.txt[] +include::config/sequencer.txt[] + BUGS ---- The todo list presented by the deprecated `--preserve-merges --interactive` diff --git a/Documentation/git-svn.txt b/Documentation/git-svn.txt index 67b143cc81..d5776ffcfd 100644 --- a/Documentation/git-svn.txt +++ b/Documentation/git-svn.txt @@ -1061,25 +1061,6 @@ with different name spaces. For example: branches = stable/*:refs/remotes/svn/stable/* branches = debug/*:refs/remotes/svn/debug/* -BUGS ----- - -We ignore all SVN properties except svn:executable. Any unhandled -properties are logged to $GIT_DIR/svn/<refname>/unhandled.log - -Renamed and copied directories are not detected by Git and hence not -tracked when committing to SVN. I do not plan on adding support for -this as it's quite difficult and time-consuming to get working for all -the possible corner cases (Git doesn't do it, either). Committing -renamed and copied files is fully supported if they're similar enough -for Git to detect them. - -In SVN, it is possible (though discouraged) to commit changes to a tag -(because a tag is just a directory copy, thus technically the same as a -branch). When cloning an SVN repository, 'git svn' cannot know if such a -commit to a tag will happen in the future. Thus it acts conservatively -and imports all SVN tags as branches, prefixing the tag name with 'tags/'. - CONFIGURATION ------------- @@ -1166,6 +1147,25 @@ $GIT_DIR/svn/\**/.rev_map.*:: if it is missing or not up to date. 'git svn reset' automatically rewinds it. +BUGS +---- + +We ignore all SVN properties except svn:executable. Any unhandled +properties are logged to $GIT_DIR/svn/<refname>/unhandled.log + +Renamed and copied directories are not detected by Git and hence not +tracked when committing to SVN. I do not plan on adding support for +this as it's quite difficult and time-consuming to get working for all +the possible corner cases (Git doesn't do it, either). Committing +renamed and copied files is fully supported if they're similar enough +for Git to detect them. + +In SVN, it is possible (though discouraged) to commit changes to a tag +(because a tag is just a directory copy, thus technically the same as a +branch). When cloning an SVN repository, 'git svn' cannot know if such a +commit to a tag will happen in the future. Thus it acts conservatively +and imports all SVN tags as branches, prefixing the tag name with 'tags/'. + SEE ALSO -------- linkgit:git-rebase[1] diff --git a/Documentation/lint-man-section-order.perl b/Documentation/lint-man-section-order.perl index 5767e7e456..b05f9156dd 100755 --- a/Documentation/lint-man-section-order.perl +++ b/Documentation/lint-man-section-order.perl @@ -18,24 +18,13 @@ 'DESCRIPTION' => { required => 1, order => $order++, - bad => { - 'git-mktag.txt' => 'OPTIONS', - 'git-cvsserver.txt' => 'OPTIONS', - }, }, 'OPTIONS' => { order => $order++, required => 0, - bad => { - 'git-grep.txt' => 'CONFIGURATION', - 'git-rebase.txt' => 'CONFIGURATION', - }, }, 'CONFIGURATION' => { order => $order++, - bad => { - 'git-svn.txt' => 'BUGS', - }, }, 'BUGS' => { order => $order++, @@ -77,16 +66,7 @@ sub report { my $expected_last = $expected_order[-2]; my $actual_last = $actual_order[-2]; - my $except_last = $SECTIONS{$line}->{bad}->{$ARGV} || ''; - if (($SECTIONS{$line}->{bad}->{$ARGV} || '') eq $actual_last) { - # Either we're whitelisted, or ... - next - } elsif (exists $SECTIONS{$actual_last}->{bad}->{$ARGV}) { - # ... we're complaing about the next section - # which is out of order because this one is, - # don't complain about that one. - next; - } elsif ($actual_last ne $expected_last) { + if ($actual_last ne $expected_last) { report("section '$line' incorrectly ordered, comes after '$actual_last'"); } next; -- 2.31.1.622.g1b8cc22e65 ^ permalink raw reply related [flat|nested] 32+ messages in thread
end of thread, other threads:[~2021-04-10 4:14 UTC | newest] Thread overview: 32+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-03-26 10:36 [PATCH 0/5] small doc make and lint fixes Ævar Arnfjörð Bjarmason 2021-03-26 10:36 ` [PATCH 1/5] Documentation/Makefile: make $(wildcard howto/*.txt) a var Ævar Arnfjörð Bjarmason 2021-03-28 6:14 ` Junio C Hamano 2021-03-26 10:36 ` [PATCH 2/5] Documentation/Makefile: make $(wildcard <doc deps>) " Ævar Arnfjörð Bjarmason 2021-03-28 6:28 ` Junio C Hamano 2021-03-26 10:36 ` [PATCH 3/5] doc lint: Perl "strict" and "warnings" in lint-gitlink.perl Ævar Arnfjörð Bjarmason 2021-03-28 6:28 ` Junio C Hamano 2021-03-26 10:36 ` [PATCH 4/5] doc lint: fix bugs in, simplify and improve lint script Ævar Arnfjörð Bjarmason 2021-03-26 11:00 ` Jeff King 2021-03-28 1:35 ` Junio C Hamano 2021-03-26 12:48 ` Philip Oakley 2021-03-28 1:34 ` Junio C Hamano 2021-03-28 6:38 ` Junio C Hamano 2021-03-26 10:36 ` [PATCH 5/5] doc lint: lint and fix missing "GIT" end sections Ævar Arnfjörð Bjarmason 2021-03-26 11:04 ` Jeff King 2021-03-26 15:32 ` Ævar Arnfjörð Bjarmason 2021-03-27 9:50 ` Jeff King 2021-03-28 6:42 ` Junio C Hamano 2021-03-28 17:53 ` Junio C Hamano 2021-04-09 11:49 ` Ævar Arnfjörð Bjarmason 2021-04-10 4:14 ` Junio C Hamano 2021-03-26 11:06 ` [PATCH 0/5] small doc make and lint fixes Jeff King 2021-03-26 15:18 ` Ævar Arnfjörð Bjarmason 2021-03-27 9:48 ` Jeff King 2021-04-09 15:02 ` [PATCH v2 0/7] " Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 1/7] Documentation/Makefile: make $(wildcard howto/*.txt) a var Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 2/7] Documentation/Makefile: make doc.dep dependencies a variable again Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 3/7] doc lint: Perl "strict" and "warnings" in lint-gitlink.perl Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 4/7] doc lint: fix bugs in, simplify and improve lint script Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 5/7] doc lint: lint and fix missing "GIT" end sections Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 6/7] doc lint: lint relative section order Ævar Arnfjörð Bjarmason 2021-04-09 15:02 ` [PATCH v2 7/7] docs: fix linting issues due to incorrect " Ævar Arnfjörð Bjarmason
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.