[for-next][PATCH 0/8] ktest.pl: Various updates and including more content in email on failures

[for-next][PATCH 0/8] ktest.pl: Various updates and including more content in email on failures

[Steven Rostedt]

Steven Rostedt (VMware) (8):
      ktest.pl: Have config-bisect save each config used in the bisect
      ktest.pl: Always show log file location if defined even on success
      ktest.pl: Define PRE_TEST_DIE to kill the test if the PRE_TEST fails
      ktest.pl: Add a NOT operator
      ktest.pl: Just open up the log file once
      ktest.pl: Turn off buffering to the log file
      ktest.pl: Add the log of last test in email on failure
      ktest.pl: Add MAIL_MAX_SIZE to limit the amount of log emailed

----
 tools/testing/ktest/ktest.pl    | 100 ++++++++++++++++++++++++++++++++++------
 tools/testing/ktest/sample.conf |  18 ++++++++
 2 files changed, 103 insertions(+), 15 deletions(-)

[for-next][PATCH 1/8] ktest.pl: Have config-bisect save each config used in the bisect

[Steven Rostedt]

From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>

When performing a automatic config bisect via ktest.pl, it is very useful to
have a copy of each of the bisects used. This way, if a bisect were to go
wrong, it is possible to retrace the steps and continue at the location
before the error was made.

The ktest.pl will make a copy of the good and bad configs, labeled as such,
as well as a number attached to it that represents the iteration of the
bisect. These files are saved in the ktest temp directory where it currently
stores the good and bad config files.

Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 7570e36d636d..5f6f88911f5c 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -3188,6 +3188,8 @@ sub config_bisect_end {
     doprint "***************************************\n\n";
 }
 
+my $pass = 1;
+
 sub run_config_bisect {
     my ($good, $bad, $last_result) = @_;
     my $reset = "";
@@ -3210,11 +3212,15 @@ sub run_config_bisect {
 
     $ret = run_config_bisect_test $config_bisect_type;
     if ($ret) {
-        doprint "NEW GOOD CONFIG\n";
+        doprint "NEW GOOD CONFIG ($pass)\n";
+	system("cp $output_config $tmpdir/good_config.tmp.$pass");
+	$pass++;
 	# Return 3 for good config
 	return 3;
     } else {
-        doprint "NEW BAD CONFIG\n";
+        doprint "NEW BAD CONFIG ($pass)\n";
+	system("cp $output_config $tmpdir/bad_config.tmp.$pass");
+	$pass++;
 	# Return 4 for bad config
 	return 4;
     }
-- 
2.26.2

[for-next][PATCH 2/8] ktest.pl: Always show log file location if defined even on success

[Steven Rostedt]

From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>

If a log file is defined and the test were to error, a print statement is
made that shows the user where the log file is to examine it further. But
this is not done if the test were to succeed.

I find it annoying that it does not show where the log file is on success,
as I run several different tests that place their log files in various
locations, and even though the test pass, there's things I want to look at
in the log file (like warnings). It is much easier to find where the log
file is, if it is displayed at the end of a test.

Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 5f6f88911f5c..5d5cf3e1e81a 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -4441,6 +4441,10 @@ if ($opt{"POWEROFF_ON_SUCCESS"}) {
 }
 
 
+if (defined($opt{"LOG_FILE"})) {
+    print "\n See $opt{LOG_FILE} for the record of results.\n";
+}
+
 doprint "\n    $successes of $opt{NUM_TESTS} tests were successful\n\n";
 
 if ($email_when_finished) {
-- 
2.26.2

[for-next][PATCH 3/8] ktest.pl: Define PRE_TEST_DIE to kill the test if the PRE_TEST fails

[Steven Rostedt]

From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>

Currently, if a PRE_TEST is defined and ran, but fails, there's nothing
currently available to make the test fail too. Add a PRE_TEST_DIE option that
when set, if a PRE_TEST is defined and fails, the test will die too.

Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    | 8 +++++++-
 tools/testing/ktest/sample.conf | 5 +++++
 2 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 5d5cf3e1e81a..f99cf633ed84 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -98,6 +98,7 @@ my $final_post_ktest;
 my $pre_ktest;
 my $post_ktest;
 my $pre_test;
+my $pre_test_die;
 my $post_test;
 my $pre_build;
 my $post_build;
@@ -273,6 +274,7 @@ my %option_map = (
     "PRE_KTEST"			=> \$pre_ktest,
     "POST_KTEST"		=> \$post_ktest,
     "PRE_TEST"			=> \$pre_test,
+    "PRE_TEST_DIE"		=> \$pre_test_die,
     "POST_TEST"			=> \$post_test,
     "BUILD_TYPE"		=> \$build_type,
     "BUILD_OPTIONS"		=> \$build_options,
@@ -4347,7 +4349,11 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     doprint "RUNNING TEST $i of $opt{NUM_TESTS}$name with option $test_type $run_type$installme\n\n";
 
     if (defined($pre_test)) {
-	run_command $pre_test;
+	my $ret = run_command $pre_test;
+	if (!$ret && defined($pre_test_die) &&
+	    $pre_test_die) {
+	    dodie "failed to pre_test\n";
+	}
     }
 
     unlink $dmesg;
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index 27666b8007ed..cb8227fbd01e 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -557,6 +557,11 @@
 # default (undefined)
 #PRE_TEST = ${SSH} reboot_to_special_kernel
 
+# To kill the entire test if PRE_TEST is defined but fails set this
+# to 1.
+# (default 0)
+#PRE_TEST_DIE = 1
+
 # If there is a command you want to run after the individual test case
 # completes, then you can set this option.
 #
-- 
2.26.2

[for-next][PATCH 4/8] ktest.pl: Add a NOT operator

[Steven Rostedt]

From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>

There is a NOT DEFINED operator, but there is not an operator that can
negate any other expression.

 For example: NOT (${FOO} == boot || ${BAR} == run)

Add the keyword NOT to allow the ktest.pl config files to negate operators.

Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index f99cf633ed84..0d04b8a2b5a2 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -911,6 +911,12 @@ sub process_expression {
 	}
     }
 
+    if ($val =~ s/^\s*NOT\s+(.*)//) {
+	my $express = $1;
+	my $ret = process_expression($name, $express);
+	return !$ret;
+    }
+
     if ($val =~ /^\s*0\s*$/) {
 	return 0;
     } elsif ($val =~ /^\s*\d+\s*$/) {
-- 
2.26.2

[for-next][PATCH 5/8] ktest.pl: Just open up the log file once

[Steven Rostedt]

From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>

Currently, every write to the log file is done by opening the file, writing
to it, then closing the file. This rather expensive. Just open it at the
beginning and close it at the end.

Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 0d04b8a2b5a2..f20a81bb3abe 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -509,9 +509,7 @@ EOF
 
 sub _logit {
     if (defined($opt{"LOG_FILE"})) {
-	open(OUT, ">> $opt{LOG_FILE}") or die "Can't write to $opt{LOG_FILE}";
-	print OUT @_;
-	close(OUT);
+	print LOG @_;
     }
 }
 
@@ -1780,8 +1778,6 @@ sub run_command {
 	(fail "unable to exec $command" and return 0);
 
     if (defined($opt{"LOG_FILE"})) {
-	open(LOG, ">>$opt{LOG_FILE}") or
-	    dodie "failed to write to log";
 	$dolog = 1;
     }
 
@@ -1829,7 +1825,6 @@ sub run_command {
     }
 
     close(CMD);
-    close(LOG) if ($dolog);
     close(RD)  if ($dord);
 
     $end_time = time;
@@ -4091,8 +4086,11 @@ if ($#new_configs >= 0) {
     }
 }
 
-if ($opt{"CLEAR_LOG"} && defined($opt{"LOG_FILE"})) {
-    unlink $opt{"LOG_FILE"};
+if (defined($opt{"LOG_FILE"})) {
+    if ($opt{"CLEAR_LOG"}) {
+	unlink $opt{"LOG_FILE"};
+    }
+    open(LOG, ">> $opt{LOG_FILE}") or die "Can't write to $opt{LOG_FILE}";
 }
 
 doprint "\n\nSTARTING AUTOMATED TESTS\n\n";
@@ -4453,14 +4451,16 @@ if ($opt{"POWEROFF_ON_SUCCESS"}) {
 }
 
 
-if (defined($opt{"LOG_FILE"})) {
-    print "\n See $opt{LOG_FILE} for the record of results.\n";
-}
-
 doprint "\n    $successes of $opt{NUM_TESTS} tests were successful\n\n";
 
 if ($email_when_finished) {
     send_email("KTEST: Your test has finished!",
             "$successes of $opt{NUM_TESTS} tests started at $script_start_time were successful!");
 }
+
+if (defined($opt{"LOG_FILE"})) {
+    print "\n See $opt{LOG_FILE} for the record of results.\n\n";
+    close LOG;
+}
+
 exit 0;
-- 
2.26.2

[for-next][PATCH 6/8] ktest.pl: Turn off buffering to the log file

[Steven Rostedt]

From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>

The log file should be up to date to whatever is happening in ktest.
Disable buffering to the LOG output file handle.

Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index f20a81bb3abe..e90e2e7cb72c 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -11,6 +11,7 @@ use File::Path qw(mkpath);
 use File::Copy qw(cp);
 use FileHandle;
 use FindBin;
+use IO::Handle;
 
 my $VERSION = "0.2";
 
@@ -4091,6 +4092,7 @@ if (defined($opt{"LOG_FILE"})) {
 	unlink $opt{"LOG_FILE"};
     }
     open(LOG, ">> $opt{LOG_FILE}") or die "Can't write to $opt{LOG_FILE}";
+    LOG->autoflush(1);
 }
 
 doprint "\n\nSTARTING AUTOMATED TESTS\n\n";
-- 
2.26.2

[for-next][PATCH 7/8] ktest.pl: Add the log of last test in email on failure

[Steven Rostedt]

From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>

If a failure happens and an email is sent, show the contents of the log of
the last test that failed in the email.

Cc: Greg KH <gregkh@linuxfoundation.org>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl | 44 ++++++++++++++++++++++++++++++++----
 1 file changed, 40 insertions(+), 4 deletions(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index e90e2e7cb72c..945a7d8c178c 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -82,6 +82,8 @@ my %default = (
     "IGNORE_UNUSED"		=> 0,
 );
 
+my $test_log_start = 0;
+
 my $ktest_config = "ktest.conf";
 my $version;
 my $have_version = 0;
@@ -1492,8 +1494,21 @@ sub dodie {
 
     if ($email_on_error) {
 	my $name = get_test_name;
+	my $log_file;
+
+	if (defined($opt{"LOG_FILE"})) {
+	    $log_file = "$tmpdir/log";
+	    open (L, "$opt{LOG_FILE}") or die "Can't open $opt{LOG_FILE} to read)";
+	    open (O, "> $tmpdir/log") or die "Can't open $tmpdir/log\n";
+	    seek(L, $test_log_start, 0);
+	    while (<L>) {
+		print O;
+	    }
+	    close O;
+	    close L;
+	}
         send_email("KTEST: critical failure for test $i [$name]",
-                "Your test started at $script_start_time has failed with:\n@_\n");
+                "Your test started at $script_start_time has failed with:\n@_\n", $log_file);
     }
 
     if ($monitor_cnt) {
@@ -4185,7 +4200,7 @@ sub find_mailer {
 }
 
 sub do_send_mail {
-    my ($subject, $message) = @_;
+    my ($subject, $message, $file) = @_;
 
     if (!defined($mail_path)) {
 	# find the mailer
@@ -4195,22 +4210,37 @@ sub do_send_mail {
 	}
     }
 
+    my $header_file = "$tmpdir/header";
+    open (HEAD, ">$header_file") or die "Can not create $header_file\n";
+    print HEAD "To: $mailto\n";
+    print HEAD "Subject: $subject\n\n";
+    print HEAD "$message\n";
+    close HEAD;
+
     if (!defined($mail_command)) {
 	if ($mailer eq "mail" || $mailer eq "mailx") {
-	    $mail_command = "\$MAIL_PATH/\$MAILER -s \'\$SUBJECT\' \$MAILTO <<< \'\$MESSAGE\'";
+	    $mail_command = "cat \$HEADER_FILE \$BODY_FILE | \$MAIL_PATH/\$MAILER -s \'\$SUBJECT\' \$MAILTO";
 	} elsif ($mailer eq "sendmail" ) {
-	    $mail_command =  "echo \'Subject: \$SUBJECT\n\n\$MESSAGE\' | \$MAIL_PATH/\$MAILER -t \$MAILTO";
+	    $mail_command =  "cat \$HEADER_FILE \$BODY_FILE | \$MAIL_PATH/\$MAILER -t \$MAILTO";
 	} else {
 	    die "\nYour mailer: $mailer is not supported.\n";
 	}
     }
 
+    if (defined($file)) {
+	$mail_command =~ s/\$BODY_FILE/$file/g;
+    } else {
+	$mail_command =~ s/\$BODY_FILE//g;
+    }
+
+    $mail_command =~ s/\$HEADER_FILE/$header_file/g;
     $mail_command =~ s/\$MAILER/$mailer/g;
     $mail_command =~ s/\$MAIL_PATH/$mail_path/g;
     $mail_command =~ s/\$MAILTO/$mailto/g;
     $mail_command =~ s/\$SUBJECT/$subject/g;
     $mail_command =~ s/\$MESSAGE/$message/g;
 
+	    print ">$mail_command<\n";
     run_command $mail_command;
 }
 
@@ -4352,6 +4382,11 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
     }
 
     doprint "\n\n";
+
+    if (defined($opt{"LOG_FILE"})) {
+	$test_log_start = tell(LOG);
+    }
+
     doprint "RUNNING TEST $i of $opt{NUM_TESTS}$name with option $test_type $run_type$installme\n\n";
 
     if (defined($pre_test)) {
@@ -4461,6 +4496,7 @@ if ($email_when_finished) {
 }
 
 if (defined($opt{"LOG_FILE"})) {
+
     print "\n See $opt{LOG_FILE} for the record of results.\n\n";
     close LOG;
 }
-- 
2.26.2

[for-next][PATCH 8/8] ktest.pl: Add MAIL_MAX_SIZE to limit the amount of log emailed

[Steven Rostedt]

From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>

Add the ktest config option MAIL_MAX_SIZE that will limit the size of the
log file that is placed into the email on failure.

Cc: Greg KH <gregkh@linuxfoundation.org>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
---
 tools/testing/ktest/ktest.pl    | 12 +++++++++++-
 tools/testing/ktest/sample.conf | 13 +++++++++++++
 2 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 945a7d8c178c..d59a6b6bcfd5 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -227,6 +227,7 @@ my $dirname = $FindBin::Bin;
 my $mailto;
 my $mailer;
 my $mail_path;
+my $mail_max_size;
 my $mail_command;
 my $email_on_error;
 my $email_when_finished;
@@ -263,6 +264,7 @@ my %option_map = (
     "MAILTO"			=> \$mailto,
     "MAILER"			=> \$mailer,
     "MAIL_PATH"			=> \$mail_path,
+    "MAIL_MAX_SIZE"		=> \$mail_max_size,
     "MAIL_COMMAND"		=> \$mail_command,
     "EMAIL_ON_ERROR"		=> \$email_on_error,
     "EMAIL_WHEN_FINISHED"	=> \$email_when_finished,
@@ -1497,10 +1499,18 @@ sub dodie {
 	my $log_file;
 
 	if (defined($opt{"LOG_FILE"})) {
+	    my $size = 0;
+	    if (defined($mail_max_size)) {
+		my $log_size = tell LOG;
+		$log_size -= $test_log_start;
+		if ($log_size > $mail_max_size) {
+		    $size = $log_size - $mail_max_size;
+		}
+	    }
 	    $log_file = "$tmpdir/log";
 	    open (L, "$opt{LOG_FILE}") or die "Can't open $opt{LOG_FILE} to read)";
 	    open (O, "> $tmpdir/log") or die "Can't open $tmpdir/log\n";
-	    seek(L, $test_log_start, 0);
+	    seek(L, $test_log_start + $size, 0);
 	    while (<L>) {
 		print O;
 	    }
diff --git a/tools/testing/ktest/sample.conf b/tools/testing/ktest/sample.conf
index cb8227fbd01e..5e7d1d729752 100644
--- a/tools/testing/ktest/sample.conf
+++ b/tools/testing/ktest/sample.conf
@@ -442,6 +442,19 @@
 # Users can cancel the test by Ctrl^C
 # (default 0)
 #EMAIL_WHEN_CANCELED = 1
+#
+# If a test ends with an error and EMAIL_ON_ERROR is set as well
+# as a LOG_FILE is defined, then the log of the failing test will
+# be included in the email that is sent.
+# It is possible that the log may be very large, in which case,
+# only the last amount of the log should be sent. To limit how
+# much of the log is sent, set MAIL_MAX_SIZE. This will be the
+# size in bytes of the last portion of the log of the failed
+# test file. That is, if this is set to 100000, then only the
+# last 100 thousand bytes of the log file will be included in
+# the email.
+# (default undef)
+#MAIL_MAX_SIZE = 1000000
 
 # Start a test setup. If you leave this off, all options
 # will be default and the test will run once.
-- 
2.26.2

Re: [for-next][PATCH 7/8] ktest.pl: Add the log of last test in email on failure

[Steven Rostedt]

On Wed, 01 Jul 2020 19:17:24 -0400
Steven Rostedt <rostedt@goodmis.org> wrote:

> +    }
> +
> +    $mail_command =~ s/\$HEADER_FILE/$header_file/g;
>      $mail_command =~ s/\$MAILER/$mailer/g;
>      $mail_command =~ s/\$MAIL_PATH/$mail_path/g;
>      $mail_command =~ s/\$MAILTO/$mailto/g;
>      $mail_command =~ s/\$SUBJECT/$subject/g;
>      $mail_command =~ s/\$MESSAGE/$message/g;
>  
> +	    print ">$mail_command<\n";

Oops, left in this debug print statement.

I nuked it before pushing to linux-next.

-- Steve

>      run_command $mail_command;
>  }
>  
> @@ -4352,6 +4382,11 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
>      }

Re: [for-next][PATCH 8/8] ktest.pl: Add MAIL_MAX_SIZE to limit the amount of log emailed

[Greg KH]

On Wed, Jul 01, 2020 at 07:17:25PM -0400, Steven Rostedt wrote:
> From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> 
> Add the ktest config option MAIL_MAX_SIZE that will limit the size of the
> log file that is placed into the email on failure.
> 
> Cc: Greg KH <gregkh@linuxfoundation.org>
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> ---
>  tools/testing/ktest/ktest.pl    | 12 +++++++++++-
>  tools/testing/ktest/sample.conf | 13 +++++++++++++
>  2 files changed, 24 insertions(+), 1 deletion(-)

Interesting, but I like full log files for my reports :)

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Re: [for-next][PATCH 7/8] ktest.pl: Add the log of last test in email on failure

[Greg KH]

On Wed, Jul 01, 2020 at 07:17:24PM -0400, Steven Rostedt wrote:
> From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> 
> If a failure happens and an email is sent, show the contents of the log of
> the last test that failed in the email.
> 
> Cc: Greg KH <gregkh@linuxfoundation.org>
> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> ---
>  tools/testing/ktest/ktest.pl | 44 ++++++++++++++++++++++++++++++++----
>  1 file changed, 40 insertions(+), 4 deletions(-)
> 
> diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
> index e90e2e7cb72c..945a7d8c178c 100755
> --- a/tools/testing/ktest/ktest.pl
> +++ b/tools/testing/ktest/ktest.pl
> @@ -82,6 +82,8 @@ my %default = (
>      "IGNORE_UNUSED"		=> 0,
>  );
>  
> +my $test_log_start = 0;
> +
>  my $ktest_config = "ktest.conf";
>  my $version;
>  my $have_version = 0;
> @@ -1492,8 +1494,21 @@ sub dodie {
>  
>      if ($email_on_error) {
>  	my $name = get_test_name;
> +	my $log_file;
> +
> +	if (defined($opt{"LOG_FILE"})) {
> +	    $log_file = "$tmpdir/log";
> +	    open (L, "$opt{LOG_FILE}") or die "Can't open $opt{LOG_FILE} to read)";
> +	    open (O, "> $tmpdir/log") or die "Can't open $tmpdir/log\n";
> +	    seek(L, $test_log_start, 0);
> +	    while (<L>) {
> +		print O;
> +	    }
> +	    close O;
> +	    close L;
> +	}
>          send_email("KTEST: critical failure for test $i [$name]",
> -                "Your test started at $script_start_time has failed with:\n@_\n");
> +                "Your test started at $script_start_time has failed with:\n@_\n", $log_file);
>      }
>  
>      if ($monitor_cnt) {
> @@ -4185,7 +4200,7 @@ sub find_mailer {
>  }
>  
>  sub do_send_mail {
> -    my ($subject, $message) = @_;
> +    my ($subject, $message, $file) = @_;
>  
>      if (!defined($mail_path)) {
>  	# find the mailer
> @@ -4195,22 +4210,37 @@ sub do_send_mail {
>  	}
>      }
>  
> +    my $header_file = "$tmpdir/header";
> +    open (HEAD, ">$header_file") or die "Can not create $header_file\n";
> +    print HEAD "To: $mailto\n";
> +    print HEAD "Subject: $subject\n\n";
> +    print HEAD "$message\n";
> +    close HEAD;
> +
>      if (!defined($mail_command)) {
>  	if ($mailer eq "mail" || $mailer eq "mailx") {
> -	    $mail_command = "\$MAIL_PATH/\$MAILER -s \'\$SUBJECT\' \$MAILTO <<< \'\$MESSAGE\'";
> +	    $mail_command = "cat \$HEADER_FILE \$BODY_FILE | \$MAIL_PATH/\$MAILER -s \'\$SUBJECT\' \$MAILTO";
>  	} elsif ($mailer eq "sendmail" ) {
> -	    $mail_command =  "echo \'Subject: \$SUBJECT\n\n\$MESSAGE\' | \$MAIL_PATH/\$MAILER -t \$MAILTO";
> +	    $mail_command =  "cat \$HEADER_FILE \$BODY_FILE | \$MAIL_PATH/\$MAILER -t \$MAILTO";
>  	} else {
>  	    die "\nYour mailer: $mailer is not supported.\n";
>  	}
>      }
>  
> +    if (defined($file)) {
> +	$mail_command =~ s/\$BODY_FILE/$file/g;
> +    } else {
> +	$mail_command =~ s/\$BODY_FILE//g;
> +    }
> +
> +    $mail_command =~ s/\$HEADER_FILE/$header_file/g;
>      $mail_command =~ s/\$MAILER/$mailer/g;
>      $mail_command =~ s/\$MAIL_PATH/$mail_path/g;
>      $mail_command =~ s/\$MAILTO/$mailto/g;
>      $mail_command =~ s/\$SUBJECT/$subject/g;
>      $mail_command =~ s/\$MESSAGE/$message/g;
>  
> +	    print ">$mail_command<\n";
>      run_command $mail_command;
>  }
>  
> @@ -4352,6 +4382,11 @@ for (my $i = 1; $i <= $opt{"NUM_TESTS"}; $i++) {
>      }
>  
>      doprint "\n\n";
> +
> +    if (defined($opt{"LOG_FILE"})) {
> +	$test_log_start = tell(LOG);
> +    }

Nit, the mix of spaces and tabs in this file is always annoying to me,
but it's your code...

> +
>      doprint "RUNNING TEST $i of $opt{NUM_TESTS}$name with option $test_type $run_type$installme\n\n";
>  
>      if (defined($pre_test)) {
> @@ -4461,6 +4496,7 @@ if ($email_when_finished) {
>  }
>  
>  if (defined($opt{"LOG_FILE"})) {
> +
>      print "\n See $opt{LOG_FILE} for the record of results.\n\n";
>      close LOG;
>  }

Extra blank line?

Anyway, looks sane to me

Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Re: [for-next][PATCH 8/8] ktest.pl: Add MAIL_MAX_SIZE to limit the amount of log emailed

[Steven Rostedt]

On Thu, 2 Jul 2020 09:41:03 +0200
Greg KH <gregkh@linuxfoundation.org> wrote:

> On Wed, Jul 01, 2020 at 07:17:25PM -0400, Steven Rostedt wrote:
> > From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> > 
> > Add the ktest config option MAIL_MAX_SIZE that will limit the size of the
> > log file that is placed into the email on failure.
> > 
> > Cc: Greg KH <gregkh@linuxfoundation.org>
> > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > ---
> >  tools/testing/ktest/ktest.pl    | 12 +++++++++++-
> >  tools/testing/ktest/sample.conf | 13 +++++++++++++
> >  2 files changed, 24 insertions(+), 1 deletion(-)  
> 
> Interesting, but I like full log files for my reports :)

I can add an option to do that if you want. My full logs end up being a
few hundred megabytes. Perhaps I could add a compress option too.

> 
> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Thanks!

-- Steve

Re: [for-next][PATCH 7/8] ktest.pl: Add the log of last test in email on failure

[Steven Rostedt]

On Thu, 2 Jul 2020 09:42:35 +0200
Greg KH <gregkh@linuxfoundation.org> wrote:

> >      doprint "\n\n";
> > +
> > +    if (defined($opt{"LOG_FILE"})) {
> > +	$test_log_start = tell(LOG);
> > +    }  
> 
> Nit, the mix of spaces and tabs in this file is always annoying to me,
> but it's your code...
> 

It's the default way emacs does Perl code. I call it "oyster mode".

I like to keep it that way as it reminds me that this is Perl and not C. ;-)


> > +
> >      doprint "RUNNING TEST $i of $opt{NUM_TESTS}$name with option $test_type $run_type$installme\n\n";
> >  
> >      if (defined($pre_test)) {
> > @@ -4461,6 +4496,7 @@ if ($email_when_finished) {
> >  }
> >  
> >  if (defined($opt{"LOG_FILE"})) {
> > +
> >      print "\n See $opt{LOG_FILE} for the record of results.\n\n";
> >      close LOG;
> >  }  
> 
> Extra blank line?

Will nuke. (Left over from having a debug print in there).

> 
> Anyway, looks sane to me
> 
> Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Awesome. Thanks for looking this over!

-- Steve

Re: [for-next][PATCH 8/8] ktest.pl: Add MAIL_MAX_SIZE to limit the amount of log emailed

[Greg KH]

On Thu, Jul 02, 2020 at 08:19:49AM -0400, Steven Rostedt wrote:
> On Thu, 2 Jul 2020 09:41:03 +0200
> Greg KH <gregkh@linuxfoundation.org> wrote:
> 
> > On Wed, Jul 01, 2020 at 07:17:25PM -0400, Steven Rostedt wrote:
> > > From: "Steven Rostedt (VMware)" <rostedt@goodmis.org>
> > > 
> > > Add the ktest config option MAIL_MAX_SIZE that will limit the size of the
> > > log file that is placed into the email on failure.
> > > 
> > > Cc: Greg KH <gregkh@linuxfoundation.org>
> > > Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
> > > ---
> > >  tools/testing/ktest/ktest.pl    | 12 +++++++++++-
> > >  tools/testing/ktest/sample.conf | 13 +++++++++++++
> > >  2 files changed, 24 insertions(+), 1 deletion(-)  
> > 
> > Interesting, but I like full log files for my reports :)
> 
> I can add an option to do that if you want. My full logs end up being a
> few hundred megabytes. Perhaps I could add a compress option too.

It's fine, the default should be good enough for me for now.  If not,
I'll just bump the value, or add compression.

thanks,

greg k-h-

Re: [for-next][PATCH 8/8] ktest.pl: Add MAIL_MAX_SIZE to limit the amount of log emailed

[Steven Rostedt]

On Thu, 2 Jul 2020 14:34:02 +0200
Greg KH <gregkh@linuxfoundation.org> wrote:

> > I can add an option to do that if you want. My full logs end up being a
> > few hundred megabytes. Perhaps I could add a compress option too.  
> 
> It's fine, the default should be good enough for me for now.  If not,
> I'll just bump the value, or add compression.

If we compress, it would need to be an attachment. I'm guessing you are
fine with that. Do you already make it an attachment?

BTW, my test just failed and for some reason it created a 13162697 byte
log file to include in my email, which failed to send :-p 

Strange that it did that, as I had the max set to 950000. Thus, I've
changed this code to be:

diff --git a/tools/testing/ktest/ktest.pl b/tools/testing/ktest/ktest.pl
index 917810fa4c85..9363a5b27339 100755
--- a/tools/testing/ktest/ktest.pl
+++ b/tools/testing/ktest/ktest.pl
@@ -1499,18 +1499,21 @@ sub dodie {
        my $log_file;
 
        if (defined($opt{"LOG_FILE"})) {
-           my $size = 0;
+           my $whence = 0; # beginning of file
+           my $pos = $test_log_start;
+
            if (defined($mail_max_size)) {
                my $log_size = tell LOG;
                $log_size -= $test_log_start;
                if ($log_size > $mail_max_size) {
-                   $size = $log_size - $mail_max_size;
+                   $whence = 2; # end of file
+                   $pos = - $mail_max_size;
                }
            }
            $log_file = "$tmpdir/log";
            open (L, "$opt{LOG_FILE}") or die "Can't open $opt{LOG_FILE} to read)";
            open (O, "> $tmpdir/log") or die "Can't open $tmpdir/log\n";
-           seek(L, $test_log_start + $size, 0);
+           seek(L, $pos, $whence);
            while (<L>) {
                print O;
            }

Let's see if this now limits it :-/

-- Steve

Re: [for-next][PATCH 8/8] ktest.pl: Add MAIL_MAX_SIZE to limit the amount of log emailed

[Greg KH]

On Thu, Jul 02, 2020 at 08:58:49AM -0400, Steven Rostedt wrote:
> On Thu, 2 Jul 2020 14:34:02 +0200
> Greg KH <gregkh@linuxfoundation.org> wrote:
> 
> > > I can add an option to do that if you want. My full logs end up being a
> > > few hundred megabytes. Perhaps I could add a compress option too.  
> > 
> > It's fine, the default should be good enough for me for now.  If not,
> > I'll just bump the value, or add compression.
> 
> If we compress, it would need to be an attachment. I'm guessing you are
> fine with that. Do you already make it an attachment?

Yes I do, so attachments are fine.

thanks,

greg k-h