Re: [PATCH 3/4] tests: drop here-doc check from internal chain-linter

[Eric Sunshine <sunshine@sunshineco.com>]

[-- Attachment #1: Type: text/plain, Size: 3386 bytes --]

On Tue, Mar 28, 2023 at 11:13 PM Eric Sunshine <sunshine@sunshineco.com> wrote:
> On Tue, Mar 28, 2023 at 11:04 PM Jeff King <peff@peff.net> wrote:
> > So I _think_ it's something like this:
> >
> > But I wasn't sure how to surface a clean error from here, since we're in
> > the Lexer. Maybe we just accumulate a "problems" array here, and then
> > roll those up via the TestParser? I'm not very familiar with the
> > arrangement of that part of the script.
>
> Yes, it would look something like that and you chose the correct spot
> to detect the problem, but to get a "pretty" error message properly
> positioned in the input, we need to capture the input stream position
> of the here-doc tag itself in scan_heredoc_tag(). It doesn't look too
> difficult, and I even started writing a bit of code to do it, but I'm
> not sure how soon I can get around to finishing the implementation.

The attached patch seems to do the job. Apologies for Gmail messing up
the whitespace. It's also attached to the email.

This would probably make a good preparatory patch to your [3/4]. As
mentioned earlier in the thread, the changes to scan_heredoc_tag ()
capture the input-stream position of the here-doc tag itself, which is
necessary since it would be too late to do so by the time the error is
detected by swallow_heredocs(). I don't now when I'll get time to send
this as a proper patch, so feel free to write a commit message and
incorporate it into your series if you want to use it. And, of course,
you have my sign-off already in the patch. It should be easy to add a
test, as well, in t/chainlint, perhaps as
unclosed-here-doc.{text,expect}.

--- >8 ---
From b7103da900dd843aabb17201bc0f9ef0b7a704ba Mon Sep 17 00:00:00 2001
From: Eric Sunshine <sunshine@sunshineco.com>
Date: Tue, 28 Mar 2023 23:35:33 -0400
Subject: [PATCH] chainlint: diagnose unclosed here-doc

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 t/chainlint.pl | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/t/chainlint.pl b/t/chainlint.pl
index e966412999..3dac033ace 100755
--- a/t/chainlint.pl
+++ b/t/chainlint.pl
@@ -80,7 +80,8 @@ sub scan_heredoc_tag {
     return "<<$indented" unless $token;
     my $tag = $token->[0];
     $tag =~ s/['"\\]//g;
-    push(@{$self->{heretags}}, $indented ? "\t$tag" : "$tag");
+    $$token[0] = "\t$tag" if $indented;
+    push(@{$self->{heretags}}, $token);
     return "<<$indented$tag";
 }

@@ -169,10 +170,18 @@ sub swallow_heredocs {
     my $tags = $self->{heretags};
     while (my $tag = shift @$tags) {
         my $start = pos($$b);
-        my $indent = $tag =~ s/^\t// ? '\\s*' : '';
-        $$b =~ /(?:\G|\n)$indent\Q$tag\E(?:\n|\z)/gc;
+        my $indent = $$tag[0] =~ s/^\t// ? '\\s*' : '';
+        $$b =~ /(?:\G|\n)$indent\Q$$tag[0]\E(?:\n|\z)/gc;
+        if (pos($$b) > $start) {
+            my $body = substr($$b, $start, pos($$b) - $start);
+            $self->{lineno} += () = $body =~ /\n/sg;
+            next;
+        }
+        push(@{$self->{parser}->{problems}}, ['HERE', $tag]);
+        $$b =~ /(?:\G|\n).*\z/gc; # consume rest of input
         my $body = substr($$b, $start, pos($$b) - $start);
         $self->{lineno} += () = $body =~ /\n/sg;
+        last;
     }
 }

--
2.40.0.460.g7fdda0a984
--- >8 ---

[-- Attachment #2: 0001-chainlint-diagnose-unclosed-here-doc.patch --]
[-- Type: application/octet-stream, Size: 1512 bytes --]

From b7103da900dd843aabb17201bc0f9ef0b7a704ba Mon Sep 17 00:00:00 2001
From: Eric Sunshine <sunshine@sunshineco.com>
Date: Tue, 28 Mar 2023 23:35:33 -0400
Subject: [PATCH] chainlint: diagnose unclosed here-doc

Signed-off-by: Eric Sunshine <sunshine@sunshineco.com>
---
 t/chainlint.pl | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/t/chainlint.pl b/t/chainlint.pl
index e966412999..3dac033ace 100755
--- a/t/chainlint.pl
+++ b/t/chainlint.pl
@@ -80,7 +80,8 @@ sub scan_heredoc_tag {
 	return "<<$indented" unless $token;
 	my $tag = $token->[0];
 	$tag =~ s/['"\\]//g;
-	push(@{$self->{heretags}}, $indented ? "\t$tag" : "$tag");
+	$$token[0] = "\t$tag" if $indented;
+	push(@{$self->{heretags}}, $token);
 	return "<<$indented$tag";
 }
 
@@ -169,10 +170,18 @@ sub swallow_heredocs {
 	my $tags = $self->{heretags};
 	while (my $tag = shift @$tags) {
 		my $start = pos($$b);
-		my $indent = $tag =~ s/^\t// ? '\\s*' : '';
-		$$b =~ /(?:\G|\n)$indent\Q$tag\E(?:\n|\z)/gc;
+		my $indent = $$tag[0] =~ s/^\t// ? '\\s*' : '';
+		$$b =~ /(?:\G|\n)$indent\Q$$tag[0]\E(?:\n|\z)/gc;
+		if (pos($$b) > $start) {
+			my $body = substr($$b, $start, pos($$b) - $start);
+			$self->{lineno} += () = $body =~ /\n/sg;
+			next;
+		}
+		push(@{$self->{parser}->{problems}}, ['HERE', $tag]);
+		$$b =~ /(?:\G|\n).*\z/gc; # consume rest of input
 		my $body = substr($$b, $start, pos($$b) - $start);
 		$self->{lineno} += () = $body =~ /\n/sg;
+		last;
 	}
 }
 
-- 
2.40.0.460.g7fdda0a984