[igt-dev] [PATCH i-g-t v2 1/7] lib/runnercomms: Structured communication from tests to igt_runner

[igt-dev] [PATCH i-g-t v2 1/7] lib/runnercomms: Structured communication from tests to igt_runner

[Petri Latvala]

Instead of letting the tests output their logs as text and parsing
that in the runner during and after test execution, introduce a
structured log format that is passed through a UNIX datagram socket.

This patch only introduces the datagram format and helpers for
creating/reading them. Key points about the format:

It's binary, and has some amount of forwards and backwards
compatibility. Passing through UNIX datagram sockets makes the
communication atomic which avoids message interleaving between child
process logging. (Threaded logging already gets correct serialization
through the use of a mutex, no change there.)

Having atomic logging also gives the possibility of igt_runner
injecting messages into test logs without having to worry whether the
stdout pipe is still in the middle of printing a subtest completion
message for example.

On-disk storage of datagrams will use a canary chunk to verify
correctly sized reads of datagrams, and to ensure that the endianness
of the reader and the dump match.

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arek@hiler.eu>
Acked-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>
---
 lib/meson.build   |   1 +
 lib/runnercomms.c | 635 ++++++++++++++++++++++++++++++++++++++++++++++
 lib/runnercomms.h | 259 +++++++++++++++++++
 3 files changed, 895 insertions(+)
 create mode 100644 lib/runnercomms.c
 create mode 100644 lib/runnercomms.h

diff --git a/lib/meson.build b/lib/meson.build
index 4b6d2267..9283aa9c 100644
--- a/lib/meson.build
+++ b/lib/meson.build
@@ -67,6 +67,7 @@ lib_sources = [
 	'rendercopy_gen7.c',
 	'rendercopy_gen8.c',
 	'rendercopy_gen9.c',
+	'runnercomms.c',
 	'sw_sync.c',
 	'intel_aux_pgtable.c',
 	'intel_reg_map.c',
diff --git a/lib/runnercomms.c b/lib/runnercomms.c
new file mode 100644
index 00000000..344312bd
--- /dev/null
+++ b/lib/runnercomms.c
@@ -0,0 +1,635 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2022 Intel Corporation
+ */
+
+#include <assert.h>
+#include <signal.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/mman.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include "igt_aux.h"
+#include "runnercomms.h"
+
+/**
+ * SECTION:runnercomms
+ * @short_description: Structured communication to igt_runner
+ * @title: runnercomms
+ * @include: runnercomms.h
+ *
+ * This library provides means for the tests to communicate to
+ * igt_runner with a formally specified protocol, avoiding
+ * shortcomings and pain points of text-based communication.
+ */
+
+static sig_atomic_t runner_socket_fd = -1;
+
+/**
+ * set_runner_socket:
+ * @fd: socket connected to runner
+ *
+ * If the passed fd is a valid socket, globally sets it to be the fd
+ * to use to talk to igt_runner.
+ */
+void set_runner_socket(int fd)
+{
+	struct stat sb;
+
+	if (fstat(fd, &sb))
+		return;
+
+	if (!S_ISSOCK(sb.st_mode))
+		return;
+
+	/*
+	 * We only sanity-check that the fd is a socket. We don't
+	 * check that it's a datagram socket etc.
+	 */
+
+	runner_socket_fd = fd;
+}
+
+/**
+ * runner_connected:
+ *
+ * Returns whether set_runner_socket has been called with a valid
+ * socket fd. Note: Will be true forever after that point. This
+ * function is used to mainly determine whether log strings will be
+ * output to the socket or to stdout/stderr and that cannot be changed
+ * even if the socket is lost midway.
+ */
+bool runner_connected(void)
+{
+	return runner_socket_fd >= 0;
+}
+
+/**
+ * send_to_runner:
+ * @packet: packet to send
+ *
+ * Sends the given communications packet to igt_runner. Calls free()
+ * on the packet, don't reuse it.
+ */
+void send_to_runner(struct runnerpacket *packet)
+{
+	if (runner_connected())
+		write(runner_socket_fd, packet, packet->size);
+	free(packet);
+}
+
+/* If enough data left, copy the data to dst, advance p, reduce size */
+static void read_integer(void* dst, size_t bytes, const char **p, uint32_t *size)
+{
+	if (*size < bytes) {
+		*size = 0;
+		return;
+	}
+
+	memcpy(dst, *p, bytes);
+	*p += bytes;
+	*size -= bytes;
+}
+
+/* If nul-termination can be found, set dststr to point to the cstring, advance p, reduce size */
+static void read_cstring(const char **dststr, const char **p, uint32_t *size)
+{
+	const char *end;
+
+	end = memchr(*p, '\0', *size);
+	if (end == NULL) {
+		*size = 0;
+		return;
+	}
+
+	*dststr = *p;
+	*size -= end - *p + 1;
+	*p = end + 1;
+}
+
+/**
+ * read_runnerpacket:
+ * @packet: runner communications packet to read
+ *
+ * Checks that the internal data of the communications packet is valid
+ * and the contents can safely be inspected without further checking
+ * for out-of-bounds etc. Constructs a runnerpacket_read_helper which
+ * will, for c-style strings, point to various sub-values directly in
+ * the #data field within @packet. Those are valid only as long as
+ * @packet is valid.
+ *
+ * Returns: An appropriately constructed runnerpacket_read_helper. On
+ * data validation errors, the #type of the returned value will be
+ * #PACKETTYPE_INVALID.
+ */
+runnerpacket_read_helper read_runnerpacket(const struct runnerpacket *packet)
+{
+	runnerpacket_read_helper ret = {};
+	uint32_t sizeleft;
+	const char *p;
+
+	if (packet->size < sizeof(*packet)) {
+		ret.type = PACKETTYPE_INVALID;
+		return ret;
+	}
+
+	ret.type = packet->type;
+	sizeleft = packet->size - sizeof(*packet);
+	p = packet->data;
+
+	switch (packet->type) {
+	case PACKETTYPE_LOG:
+		read_integer(&ret.log.stream, sizeof(ret.log.stream), &p, &sizeleft);
+		read_cstring(&ret.log.text, &p, &sizeleft);
+
+		if (ret.log.text == NULL)
+			ret.type = PACKETTYPE_INVALID;
+
+		break;
+	case PACKETTYPE_EXEC:
+		read_cstring(&ret.exec.cmdline, &p, &sizeleft);
+
+		if (ret.exec.cmdline == NULL)
+			ret.type = PACKETTYPE_INVALID;
+
+		break;
+	case PACKETTYPE_EXIT:
+		read_integer(&ret.exit.exitcode, sizeof(ret.exit.exitcode), &p, &sizeleft);
+		read_cstring(&ret.exit.timeused, &p, &sizeleft);
+
+		break;
+	case PACKETTYPE_SUBTEST_START:
+		read_cstring(&ret.subteststart.name, &p, &sizeleft);
+
+		if (ret.subteststart.name == NULL)
+			ret.type = PACKETTYPE_INVALID;
+
+		break;
+	case PACKETTYPE_SUBTEST_RESULT:
+		read_cstring(&ret.subtestresult.name, &p, &sizeleft);
+		read_cstring(&ret.subtestresult.result, &p, &sizeleft);
+		read_cstring(&ret.subtestresult.timeused, &p, &sizeleft);
+		read_cstring(&ret.subtestresult.reason, &p, &sizeleft);
+
+		if (ret.subtestresult.name == NULL ||
+		    ret.subtestresult.result == NULL)
+			ret.type = PACKETTYPE_INVALID;
+
+		break;
+	case PACKETTYPE_DYNAMIC_SUBTEST_START:
+		read_cstring(&ret.dynamicsubteststart.name, &p, &sizeleft);
+
+		if (ret.dynamicsubteststart.name == NULL)
+			ret.type = PACKETTYPE_INVALID;
+
+		break;
+	case PACKETTYPE_DYNAMIC_SUBTEST_RESULT:
+		read_cstring(&ret.dynamicsubtestresult.name, &p, &sizeleft);
+		read_cstring(&ret.dynamicsubtestresult.result, &p, &sizeleft);
+		read_cstring(&ret.dynamicsubtestresult.timeused, &p, &sizeleft);
+		read_cstring(&ret.dynamicsubtestresult.reason, &p, &sizeleft);
+
+		if (ret.dynamicsubtestresult.name == NULL ||
+		    ret.dynamicsubtestresult.result == NULL)
+			ret.type = PACKETTYPE_INVALID;
+
+		break;
+	case PACKETTYPE_VERSIONSTRING:
+		read_cstring(&ret.versionstring.text, &p, &sizeleft);
+
+		if (ret.versionstring.text == NULL)
+			ret.type = PACKETTYPE_INVALID;
+
+		break;
+	case PACKETTYPE_RESULT_OVERRIDE:
+		read_cstring(&ret.resultoverride.result, &p, &sizeleft);
+
+		if (ret.resultoverride.result == NULL)
+			ret.type = PACKETTYPE_INVALID;
+
+		break;
+	default:
+		ret.type = PACKETTYPE_INVALID;
+		break;
+	}
+
+	return ret;
+}
+
+struct runnerpacket *runnerpacket_log(uint8_t stream, const char *text)
+{
+	struct runnerpacket *packet;
+	uint32_t size;
+	char *p;
+
+	size = sizeof(struct runnerpacket) + sizeof(stream) + strlen(text) + 1;
+	packet = malloc(size);
+
+	packet->size = size;
+	packet->type = PACKETTYPE_LOG;
+	packet->senderpid = getpid();
+	packet->sendertid = gettid();
+
+	p = packet->data;
+
+	memcpy(p, &stream, sizeof(stream));
+	p += sizeof(stream);
+
+	strcpy(p, text);
+	p += strlen(text) + 1;
+
+	return packet;
+}
+
+struct runnerpacket *runnerpacket_exec(char **argv)
+{
+	struct runnerpacket *packet;
+	uint32_t size;
+	char *p;
+	int i;
+
+	size = sizeof(struct runnerpacket);
+
+	for (i = 0; argv[i] != NULL; i++)
+		size += strlen(argv[i]) + 1; // followed by a space of \0 so +1 either way for each
+
+	packet = malloc(size);
+
+	packet->size = size;
+	packet->type = PACKETTYPE_EXEC;
+	packet->senderpid = getpid();
+	packet->sendertid = gettid();
+
+	p = packet->data;
+
+	for (i = 0; argv[i] != NULL; i++) {
+		if (i != 0)
+			*p++ = ' ';
+
+		strcpy(p, argv[i]);
+		p += strlen(argv[i]);
+	}
+	p[0] = '\0';
+
+	return packet;
+}
+
+struct runnerpacket *runnerpacket_exit(int32_t exitcode, const char *timeused)
+{
+	struct runnerpacket *packet;
+	uint32_t size;
+	char *p;
+
+	size = sizeof(struct runnerpacket) + sizeof(exitcode) + strlen(timeused) + 1;
+	packet = malloc(size);
+
+	packet->size = size;
+	packet->type = PACKETTYPE_EXIT;
+	packet->senderpid = getpid();
+	packet->sendertid = gettid();
+
+	p = packet->data;
+
+	memcpy(p, &exitcode, sizeof(exitcode));
+	p += sizeof(exitcode);
+
+	strcpy(p, timeused);
+	p += strlen(timeused) + 1;
+
+	return packet;
+}
+
+struct runnerpacket *runnerpacket_subtest_start(const char *name)
+{
+	struct runnerpacket *packet;
+	uint32_t size;
+	char *p;
+
+	size = sizeof(struct runnerpacket) + strlen(name) + 1;
+	packet = malloc(size);
+
+	packet->size = size;
+	packet->type = PACKETTYPE_SUBTEST_START;
+	packet->senderpid = getpid();
+	packet->sendertid = gettid();
+
+	p = packet->data;
+
+	strcpy(p, name);
+	p += strlen(name) + 1;
+
+	return packet;
+}
+
+struct runnerpacket *runnerpacket_subtest_result(const char *name, const char *result,
+						 const char *timeused, const char *reason)
+{
+	struct runnerpacket *packet;
+	uint32_t size;
+	char *p;
+
+	if (reason == NULL)
+		reason = "";
+
+	size = sizeof(struct runnerpacket) + strlen(name) + strlen(result) + strlen(timeused) + strlen(reason) + 4;
+	packet = malloc(size);
+
+	packet->size = size;
+	packet->type = PACKETTYPE_SUBTEST_RESULT;
+	packet->senderpid = getpid();
+	packet->sendertid = gettid();
+
+	p = packet->data;
+
+	strcpy(p, name);
+	p += strlen(name) + 1;
+
+	strcpy(p, result);
+	p += strlen(result) + 1;
+
+	strcpy(p, timeused);
+	p += strlen(timeused) + 1;
+
+	strcpy(p, reason);
+	p += strlen(reason) + 1;
+
+	return packet;
+}
+
+struct runnerpacket *runnerpacket_dynamic_subtest_start(const char *name)
+{
+	struct runnerpacket *packet;
+	uint32_t size;
+	char *p;
+
+	size = sizeof(struct runnerpacket) + strlen(name) + 1;
+	packet = malloc(size);
+
+	packet->size = size;
+	packet->type = PACKETTYPE_DYNAMIC_SUBTEST_START;
+	packet->senderpid = getpid();
+	packet->sendertid = gettid();
+
+	p = packet->data;
+
+	strcpy(p, name);
+	p += strlen(name) + 1;
+
+	return packet;
+}
+
+struct runnerpacket *runnerpacket_dynamic_subtest_result(const char *name, const char *result,
+							 const char *timeused, const char *reason)
+{
+	struct runnerpacket *packet;
+	uint32_t size;
+	char *p;
+
+	if (reason == NULL)
+		reason = "";
+
+	size = sizeof(struct runnerpacket) + strlen(name) + strlen(result) + strlen(timeused) + strlen(reason) + 4;
+	packet = malloc(size);
+
+	packet->size = size;
+	packet->type = PACKETTYPE_DYNAMIC_SUBTEST_RESULT;
+	packet->senderpid = getpid();
+	packet->sendertid = gettid();
+
+	p = packet->data;
+
+	strcpy(p, name);
+	p += strlen(name) + 1;
+
+	strcpy(p, result);
+	p += strlen(result) + 1;
+
+	strcpy(p, timeused);
+	p += strlen(timeused) + 1;
+
+	strcpy(p, reason);
+	p += strlen(reason) + 1;
+
+	return packet;
+}
+
+struct runnerpacket *runnerpacket_versionstring(const char *text)
+{
+	struct runnerpacket *packet;
+	uint32_t size;
+	char *p;
+
+	size = sizeof(struct runnerpacket) + strlen(text) + 1;
+	packet = malloc(size);
+
+	packet->size = size;
+	packet->type = PACKETTYPE_VERSIONSTRING;
+	packet->senderpid = getpid();
+	packet->sendertid = gettid();
+
+	p = packet->data;
+
+	strcpy(p, text);
+	p += strlen(text) + 1;
+
+	return packet;
+}
+
+struct runnerpacket *runnerpacket_resultoverride(const char *result)
+{
+	struct runnerpacket *packet;
+	uint32_t size;
+	char *p;
+
+	size = sizeof(struct runnerpacket) + strlen(result) + 1;
+	packet = malloc(size);
+
+	packet->size = size;
+	packet->type = PACKETTYPE_RESULT_OVERRIDE;
+	packet->senderpid = getpid();
+	packet->sendertid = gettid();
+
+	p = packet->data;
+
+	strcpy(p, result);
+	p += strlen(result) + 1;
+
+	return packet;
+}
+
+uint32_t socket_dump_canary(void)
+{
+	return 'I' << 24 | 'G' << 16 | 'T' << 8 | '1';
+}
+
+void log_to_runner_sig_safe(const char *str, size_t len)
+{
+	size_t prlen = len;
+
+	struct runnerpacket_log_sig_safe p = {
+					      .size = sizeof(struct runnerpacket) + sizeof(uint8_t),
+					      .type = PACKETTYPE_LOG,
+					      .senderpid = getpid(),
+					      .sendertid = 0, /* gettid() not signal safe */
+					      .stream = STDERR_FILENO,
+	};
+
+	if (len > sizeof(p.data) - 1)
+		prlen = sizeof(p.data) - 1;
+	memcpy(p.data, str, prlen);
+	p.size += prlen + 1;
+
+	write(runner_socket_fd, &p, p.size);
+
+	len -= prlen;
+	if (len)
+		log_to_runner_sig_safe(str + prlen, len);
+}
+
+/**
+ * comms_read_dump:
+ * @fd: Open fd to a comms dump file
+ * @visitor: Collection of packet handlers
+ *
+ * Reads a comms dump file, calling specified handler functions for
+ * individual packets.
+ *
+ * Returns: #COMMSPARSE_ERROR for failures reading or parsing the
+ * dump, #COMMSPARSE_EMPTY for empty dumps (no comms used),
+ * #COMMSPARSE_SUCCESS for successful read.
+ */
+int comms_read_dump(int fd, struct comms_visitor *visitor)
+{
+	struct stat statbuf;
+	char *buf, *bufend, *p;
+	int ret = COMMSPARSE_EMPTY;
+	bool cont = true;
+
+	if (fd < 0)
+		return COMMSPARSE_EMPTY;
+
+	if (fstat(fd, &statbuf))
+		return COMMSPARSE_ERROR;
+
+	if (statbuf.st_size == 0)
+		return COMMSPARSE_ERROR;
+
+	buf = mmap(NULL, statbuf.st_size, PROT_READ, MAP_SHARED, fd, 0);
+	if (buf == MAP_FAILED)
+		return COMMSPARSE_ERROR;
+
+	bufend = buf + statbuf.st_size;
+	p = buf;
+
+	while (p != NULL && p != bufend && cont) {
+		const struct runnerpacket *packet;
+		runnerpacket_read_helper helper;
+
+		if (bufend - p >= sizeof(uint32_t)) {
+			uint32_t canary;
+
+			memcpy(&canary, p, sizeof(canary));
+			if (canary != socket_dump_canary()) {
+				fprintf(stderr,
+					"Invalid canary while parsing comms: %"PRIu32", expected %"PRIu32"\n",
+					canary, socket_dump_canary());
+				munmap(buf, statbuf.st_size);
+				return COMMSPARSE_ERROR;
+			}
+		}
+		p += sizeof(uint32_t);
+
+		if (bufend -p < sizeof(struct runnerpacket)) {
+			fprintf(stderr,
+				"Error parsing comms: Expected runnerpacket after canary, truncated file?\n");
+			munmap(buf, statbuf.st_size);
+			return COMMSPARSE_ERROR;
+		}
+
+		packet = (struct runnerpacket *)p;
+		if (bufend -p < packet->size) {
+			fprintf(stderr,
+				"Error parsing comms: Unexpected end of file, truncated file?\n");
+			munmap(buf, statbuf.st_size);
+			return COMMSPARSE_ERROR;
+		}
+		p += packet->size;
+
+		/*
+		 * Runner sends EXEC itself before executing the test.
+		 * If we get other types, it indicates the test really
+		 * uses socket comms.
+		 */
+		if (packet->type != PACKETTYPE_EXEC)
+			ret = COMMSPARSE_SUCCESS;
+
+		switch (packet->type) {
+		case PACKETTYPE_INVALID:
+			printf("Warning: Unknown packet type %"PRIu32", skipping\n", packet->type);
+			break;
+		case PACKETTYPE_LOG:
+			if (visitor->log) {
+				helper = read_runnerpacket(packet);
+				cont = visitor->log(packet, helper, visitor->userdata);
+			}
+			break;
+		case PACKETTYPE_EXEC:
+			if (visitor->exec) {
+				helper = read_runnerpacket(packet);
+				cont = visitor->exec(packet, helper, visitor->userdata);
+			}
+			break;
+		case PACKETTYPE_EXIT:
+			if (visitor->exit) {
+				helper = read_runnerpacket(packet);
+				cont = visitor->exit(packet, helper, visitor->userdata);
+			}
+			break;
+		case PACKETTYPE_SUBTEST_START:
+			if (visitor->subtest_start) {
+				helper = read_runnerpacket(packet);
+				cont = visitor->subtest_start(packet, helper, visitor->userdata);
+			}
+			break;
+		case PACKETTYPE_SUBTEST_RESULT:
+			if (visitor->subtest_result) {
+				helper = read_runnerpacket(packet);
+				cont = visitor->subtest_result(packet, helper, visitor->userdata);
+			}
+			break;
+		case PACKETTYPE_DYNAMIC_SUBTEST_START:
+			if (visitor->dynamic_subtest_start) {
+				helper = read_runnerpacket(packet);
+				cont = visitor->dynamic_subtest_start(packet, helper, visitor->userdata);
+			}
+			break;
+		case PACKETTYPE_DYNAMIC_SUBTEST_RESULT:
+			if (visitor->dynamic_subtest_result) {
+				helper = read_runnerpacket(packet);
+				cont = visitor->dynamic_subtest_result(packet, helper, visitor->userdata);
+			}
+			break;
+		case PACKETTYPE_VERSIONSTRING:
+			if (visitor->versionstring) {
+				helper = read_runnerpacket(packet);
+				cont = visitor->versionstring(packet, helper, visitor->userdata);
+			}
+			break;
+		case PACKETTYPE_RESULT_OVERRIDE:
+			if (visitor->result_override) {
+				helper = read_runnerpacket(packet);
+				cont = visitor->result_override(packet, helper, visitor->userdata);
+			}
+			break;
+		default:
+			printf("Warning: Unknown packet type %"PRIu32"\n", helper.type);
+			break;
+		}
+	}
+
+	munmap(buf, statbuf.st_size);
+	return cont ? ret : COMMSPARSE_ERROR;
+}
diff --git a/lib/runnercomms.h b/lib/runnercomms.h
new file mode 100644
index 00000000..0bc30818
--- /dev/null
+++ b/lib/runnercomms.h
@@ -0,0 +1,259 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2022 Intel Corporation
+ */
+
+#ifndef IGT_RUNNERCOMMS_H
+#define IGT_RUNNERCOMMS_H
+
+#include <stdbool.h>
+#include <stddef.h>
+#include <stdint.h>
+
+/*
+ * A flat struct that can and will be directly dumped to
+ * disk. Constructed with runnerpacket_<type>() helper functions.
+ */
+struct runnerpacket {
+	uint32_t size; /* Full size of the packet in octets */
+	uint32_t type; /* runnerpacket_type, but fixed width */
+	int32_t senderpid;
+	int32_t sendertid;
+
+	char data[];
+} __attribute__((packed));
+
+_Static_assert(sizeof(struct runnerpacket) == 4 * 4, "runnerpacket structure must not change");
+_Static_assert(offsetof(struct runnerpacket, data) == 4 * 4, "runnerpacket structure must not change");
+
+/*
+ * A helper for reading and parsing runnerpacket structs. Fields will
+ * point directly into the data field of an existing runnerpacket
+ * object. Constructed with read_runnerpacket().
+ *
+ * Some fields can be left as 0 / NULL / some other applicable invalid
+ * value in the case of having older dumps read with binaries that
+ * have extended the data formats.
+ */
+typedef union runnerpacket_read_helper {
+	/*
+	 * All other fields must begin with "uint32_t type" so it's a
+	 * common initial sequence, safe to read no matter what union
+	 * field is active.
+	 */
+	uint32_t type;
+
+	struct {
+		uint32_t type;
+
+		uint8_t stream;
+		const char *text;
+	} log;
+
+	struct {
+		uint32_t type;
+
+		const char *cmdline;
+	} exec;
+
+	struct {
+		uint32_t type;
+
+		int32_t exitcode;
+		const char *timeused;
+	} exit;
+
+	struct {
+		uint32_t type;
+
+		const char *name;
+	} subteststart;
+
+	struct {
+		uint32_t type;
+
+		const char *name;
+		const char *result;
+		const char *timeused;
+		const char *reason;
+	} subtestresult;
+
+	struct {
+		uint32_t type;
+
+		const char *name;
+	} dynamicsubteststart;
+
+	struct {
+		uint32_t type;
+
+		const char *name;
+		const char *result;
+		const char *timeused;
+		const char *reason;
+	} dynamicsubtestresult;
+
+	struct {
+		uint32_t type;
+
+		const char *text;
+	} versionstring;
+
+	struct {
+		uint32_t type;
+
+		const char *result;
+	} resultoverride;
+} runnerpacket_read_helper;
+
+void set_runner_socket(int fd);
+bool runner_connected(void);
+void send_to_runner(struct runnerpacket *packet);
+
+runnerpacket_read_helper read_runnerpacket(const struct runnerpacket *packet);
+
+/*
+ * All packet types must document the format of the data[] array. The
+ * notation used is
+ *
+ * Explanation of the packet
+ * type: explanation of values
+ * type2: explanation of values
+ * (etc)
+ *
+ * The type "cstring" can be used to denote that the content is a
+ * nul-terminated string.
+ */
+enum runnerpacket_type {
+      PACKETTYPE_INVALID,
+      /* No data. This type is only used on parse failures and such. */
+
+      PACKETTYPE_LOG,
+      /*
+       * Normal log message.
+       * uint8_t: 1 = stdout, 2 = stderr
+       * cstring: Log text
+       */
+
+      PACKETTYPE_EXEC,
+      /*
+       * Command line executed. Sent by runner before calling exec().
+       * cstring: command line as one string, argv[0] included, space separated
+       */
+
+      PACKETTYPE_EXIT,
+      /*
+       * Process exit. Written by runner.
+       * int32_t: exitcode
+       * cstring: Time taken by the process from exec to exit, as a floating point value in seconds, as text
+       */
+
+      PACKETTYPE_SUBTEST_START,
+      /*
+       * Subtest begins.
+       * cstring: Name of the subtest
+       */
+
+      PACKETTYPE_SUBTEST_RESULT,
+      /*
+       * Subtest ends. Can appear without a corresponding SUBTEST_START packet.
+       * cstring: Name of the subtest
+       * cstring: Result of the subtest
+       * cstring: Time taken by the subtest, as a floating point value in seconds, as text
+       * cstring: If len > 0, the reason for the subtest result (fail/skip)
+       */
+
+      PACKETTYPE_DYNAMIC_SUBTEST_START,
+      /*
+       * Dynamic subtest begins.
+       * cstring: Name of the dynamic subtest
+       */
+
+      PACKETTYPE_DYNAMIC_SUBTEST_RESULT,
+      /*
+       * Dynamic subtest ends.
+       * cstring: Name of the dynamic subtest
+       * cstring: Result of the dynamic subtest
+       * cstring: Time taken by the dynamic subtest, as a floating point value in seconds, as text
+       * cstring: If len > 0, the reason for the dynamic subtest result (fail/skip)
+       */
+
+      PACKETTYPE_VERSIONSTRING,
+      /*
+       * Version of the running test
+       * cstring: Version string
+       */
+
+      PACKETTYPE_RESULT_OVERRIDE,
+      /*
+       * Override the result of the most recently started test/subtest/dynamic subtest. Used for timeout and abort etc.
+       * cstring: The result to use, as text. All lowercase.
+       */
+
+
+      PACKETTYPE_NUM_TYPES /* must be last */
+};
+
+struct runnerpacket *runnerpacket_log(uint8_t stream, const char *text);
+struct runnerpacket *runnerpacket_exec(char **argv);
+struct runnerpacket *runnerpacket_exit(int32_t exitcode, const char *timeused);
+struct runnerpacket *runnerpacket_subtest_start(const char *name);
+struct runnerpacket *runnerpacket_subtest_result(const char *name, const char *result,
+						 const char *timeused, const char *reason);
+struct runnerpacket *runnerpacket_dynamic_subtest_start(const char *name);
+struct runnerpacket *runnerpacket_dynamic_subtest_result(const char *name, const char *result,
+							 const char *timeused, const char *reason);
+struct runnerpacket *runnerpacket_versionstring(const char *text);
+struct runnerpacket *runnerpacket_resultoverride(const char *result);
+
+uint32_t socket_dump_canary(void);
+
+struct runnerpacket_log_sig_safe {
+	uint32_t size;
+	uint32_t type;
+	int32_t senderpid;
+	int32_t sendertid;
+
+	uint8_t stream;
+	char data[128];
+} __attribute__((packed));
+
+_Static_assert(offsetof(struct runnerpacket_log_sig_safe, stream) == 4 * 4, "signal-safe log runnerpacket must be compatible");
+_Static_assert(offsetof(struct runnerpacket_log_sig_safe, data) == 4 * 4 + 1, "signal-safe log runnerpacket must be compatible");
+
+void log_to_runner_sig_safe(const char *str, size_t len);
+
+/*
+ * Comms dump reader
+ *
+ * A visitor for reading comms dump files. Calls handlers if
+ * corresponding handler is set. Reading stops if a handler returns
+ * false.
+ *
+ * The passed arguments are the packet itself, the already-constructed
+ * read helper, and the userdata pointer from the visitor.
+ */
+typedef bool (*handler_t)(const struct runnerpacket *, runnerpacket_read_helper, void *userdata);
+
+struct comms_visitor {
+	handler_t log;
+	handler_t exec;
+	handler_t exit;
+	handler_t subtest_start;
+	handler_t subtest_result;
+	handler_t dynamic_subtest_start;
+	handler_t dynamic_subtest_result;
+	handler_t versionstring;
+	handler_t result_override;
+
+	void* userdata;
+};
+
+enum {
+	COMMSPARSE_ERROR,
+	COMMSPARSE_EMPTY,
+	COMMSPARSE_SUCCESS
+};
+int comms_read_dump(int fd, struct comms_visitor *visitor);
+
+#endif
-- 
2.30.2

[igt-dev] [PATCH i-g-t v2 2/7] lib/igt_core: Send logs to runner with comms

[Petri Latvala]

If the runner has opened a socket to talk to it, use it for sending
logs and other messages.

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arek@hiler.eu>
Acked-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>
---
 lib/igt_core.c | 177 ++++++++++++++++++++++++++++++++++++-------------
 1 file changed, 132 insertions(+), 45 deletions(-)

diff --git a/lib/igt_core.c b/lib/igt_core.c
index aad7b6d8..3941c528 100644
--- a/lib/igt_core.c
+++ b/lib/igt_core.c
@@ -76,6 +76,7 @@
 #include "igt_list.h"
 #include "igt_device_scan.h"
 #include "igt_thread.h"
+#include "runnercomms.h"
 
 #define UNW_LOCAL_ONLY
 #include <libunwind.h>
@@ -456,6 +457,84 @@ static void _igt_log_buffer_reset(void)
 	pthread_mutex_unlock(&log_buffer_mutex);
 }
 
+__attribute__((format(printf, 2, 3)))
+static void _log_line_fprintf(FILE* stream, const char *format, ...)
+{
+	va_list ap;
+	char *str;
+
+	va_start(ap, format);
+
+	if (runner_connected()) {
+		vasprintf(&str, format, ap);
+		send_to_runner(runnerpacket_log(fileno(stream), str));
+		free(str);
+	} else {
+		vfprintf(stream, format, ap);
+	}
+}
+
+enum _subtest_type {
+      _SUBTEST_TYPE_NORMAL,
+      _SUBTEST_TYPE_DYNAMIC,
+};
+
+static void _subtest_result_message(enum _subtest_type subtest_type,
+				    const char *name,
+				    const char *result,
+				    double timeelapsed)
+{
+	char timestr[32];
+
+	snprintf(timestr, sizeof(timestr), "%.3f", timeelapsed);
+
+	if (runner_connected()) {
+		if (subtest_type == _SUBTEST_TYPE_NORMAL)
+			send_to_runner(runnerpacket_subtest_result(name, result, timestr, NULL));
+		else
+			send_to_runner(runnerpacket_dynamic_subtest_result(name, result, timestr, NULL));
+
+		return;
+	}
+
+	printf("%s%s %s: %s (%ss)%s\n",
+	       (!__igt_plain_output) ? "\x1b[1m" : "",
+	       subtest_type == _SUBTEST_TYPE_NORMAL ? "Subtest" : "Dynamic subtest",
+	       name,
+	       result,
+	       timestr,
+	       (!__igt_plain_output) ? "\x1b[0m" : "");
+	fflush(stdout);
+	if (stderr_needs_sentinel)
+		fprintf(stderr, "%s %s: %s (%ss)\n",
+			subtest_type == _SUBTEST_TYPE_NORMAL ? "Subtest" : "Dynamic subtest",
+			name,
+			result,
+			timestr);
+}
+
+static void _subtest_starting_message(enum _subtest_type subtest_type,
+				      const char *name)
+{
+	if (runner_connected()) {
+		if (subtest_type == _SUBTEST_TYPE_NORMAL)
+			send_to_runner(runnerpacket_subtest_start(name));
+		else
+			send_to_runner(runnerpacket_dynamic_subtest_start(name));
+
+		return;
+	}
+
+	igt_info("Starting %s: %s\n",
+		 subtest_type == _SUBTEST_TYPE_NORMAL ? "subtest" : "dynamic subtest",
+		 name);
+	fflush(stdout);
+	if (stderr_needs_sentinel)
+		fprintf(stderr, "Starting %s: %s\n",
+			subtest_type == _SUBTEST_TYPE_NORMAL ? "subtest" : "dynamic subtest",
+			name);
+}
+
 static void _igt_log_buffer_dump(void)
 {
 	uint8_t i;
@@ -478,31 +557,31 @@ static void _igt_log_buffer_dump(void)
 	}
 
 	if (in_dynamic_subtest)
-		fprintf(stderr, "Dynamic subtest %s failed.\n", in_dynamic_subtest);
+		_log_line_fprintf(stderr, "Dynamic subtest %s failed.\n", in_dynamic_subtest);
 	else if (in_subtest)
-		fprintf(stderr, "Subtest %s failed.\n", in_subtest);
+		_log_line_fprintf(stderr, "Subtest %s failed.\n", in_subtest);
 	else
-		fprintf(stderr, "Test %s failed.\n", command_str);
+		_log_line_fprintf(stderr, "Test %s failed.\n", command_str);
 
 	if (log_buffer.start == log_buffer.end) {
-		fprintf(stderr, "No log.\n");
+		_log_line_fprintf(stderr, "No log.\n");
 		return;
 	}
 
 	pthread_mutex_lock(&log_buffer_mutex);
-	fprintf(stderr, "**** DEBUG ****\n");
+	_log_line_fprintf(stderr, "**** DEBUG ****\n");
 
 	i = log_buffer.start;
 	do {
 		char *last_line = log_buffer.entries[i];
-		fprintf(stderr, "%s", last_line);
+		_log_line_fprintf(stderr, "%s", last_line);
 		i++;
 	} while (i != log_buffer.start && i != log_buffer.end);
 
 	/* reset the buffer */
 	log_buffer.start = log_buffer.end = 0;
 
-	fprintf(stderr, "****  END  ****\n");
+	_log_line_fprintf(stderr, "****  END  ****\n");
 	pthread_mutex_unlock(&log_buffer_mutex);
 }
 
@@ -763,9 +842,19 @@ static void print_version(void)
 
 	uname(&uts);
 
-	igt_info("IGT-Version: %s-%s (%s) (%s: %s %s)\n", PACKAGE_VERSION,
-		 IGT_GIT_SHA1, TARGET_CPU_PLATFORM,
-		 uts.sysname, uts.release, uts.machine);
+	if (runner_connected()) {
+		char versionstr[256];
+
+		snprintf(versionstr, sizeof(versionstr),
+			 "IGT-Version: %s-%s (%s) (%s: %s %s)\n", PACKAGE_VERSION,
+			 IGT_GIT_SHA1, TARGET_CPU_PLATFORM,
+			 uts.sysname, uts.release, uts.machine);
+		send_to_runner(runnerpacket_versionstring(versionstr));
+	} else {
+		igt_info("IGT-Version: %s-%s (%s) (%s: %s %s)\n", PACKAGE_VERSION,
+			 IGT_GIT_SHA1, TARGET_CPU_PLATFORM,
+			 uts.sysname, uts.release, uts.machine);
+	}
 }
 
 static void print_usage(const char *help_str, bool output_on_stderr)
@@ -937,6 +1026,11 @@ static void common_init_env(void)
 	if (env) {
 		igt_rc_device = strdup(env);
 	}
+
+	env = getenv("IGT_RUNNER_SOCKET_FD");
+	if (env) {
+		set_runner_socket(atoi(env));
+	}
 }
 
 static int common_init(int *argc, char **argv,
@@ -1329,24 +1423,15 @@ bool __igt_run_subtest(const char *subtest_name, const char *file, const int lin
 
 
 	if (skip_subtests_henceforth) {
-		printf("%sSubtest %s: %s%s\n",
-		       (!__igt_plain_output) ? "\x1b[1m" : "", subtest_name,
-		       skip_subtests_henceforth == SKIP ?
-		       "SKIP" : "FAIL", (!__igt_plain_output) ? "\x1b[0m" : "");
-		fflush(stdout);
-		if (stderr_needs_sentinel)
-			fprintf(stderr, "Subtest %s: %s\n", subtest_name,
-				skip_subtests_henceforth == SKIP ?
-				"SKIP" : "FAIL");
+		_subtest_result_message(_SUBTEST_TYPE_NORMAL, subtest_name,
+					skip_subtests_henceforth == SKIP ? "SKIP" : "FAIL",
+					0.0);
 		return false;
 	}
 
 	igt_kmsg(KMSG_INFO "%s: starting subtest %s\n",
 		 command_str, subtest_name);
-	igt_info("Starting subtest: %s\n", subtest_name);
-	fflush(stdout);
-	if (stderr_needs_sentinel)
-		fprintf(stderr, "Starting subtest: %s\n", subtest_name);
+	_subtest_starting_message(_SUBTEST_TYPE_NORMAL, subtest_name);
 
 	_igt_log_buffer_reset();
 	igt_thread_clear_fail_state();
@@ -1374,10 +1459,7 @@ bool __igt_run_dynamic_subtest(const char *dynamic_subtest_name)
 
 	igt_kmsg(KMSG_INFO "%s: starting dynamic subtest %s\n",
 		 command_str, dynamic_subtest_name);
-	igt_info("Starting dynamic subtest: %s\n", dynamic_subtest_name);
-	fflush(stdout);
-	if (stderr_needs_sentinel)
-		fprintf(stderr, "Starting dynamic subtest: %s\n", dynamic_subtest_name);
+	_subtest_starting_message(_SUBTEST_TYPE_DYNAMIC, dynamic_subtest_name);
 
 	_igt_log_buffer_reset();
 	igt_thread_clear_fail_state();
@@ -1456,23 +1538,16 @@ bool __igt_enter_dynamic_container(void)
 __noreturn static void exit_subtest(const char *result)
 {
 	struct timespec now;
-	const char *subtest_text = in_dynamic_subtest ? "Dynamic subtest" : "Subtest";
 	const char **subtest_name = in_dynamic_subtest ? &in_dynamic_subtest : &in_subtest;
 	struct timespec *thentime = in_dynamic_subtest ? &dynamic_subtest_time : &subtest_time;
 	jmp_buf *jmptarget = in_dynamic_subtest ? &igt_dynamic_jmpbuf : &igt_subtest_jmpbuf;
 
 	igt_gettime(&now);
 
-	igt_info("%s%s %s: %s (%.3fs)%s\n",
-		 (!__igt_plain_output) ? "\x1b[1m" : "",
-		 subtest_text, *subtest_name, result,
-		 igt_time_elapsed(thentime, &now),
-		 (!__igt_plain_output) ? "\x1b[0m" : "");
-	fflush(stdout);
-	if (stderr_needs_sentinel)
-		fprintf(stderr, "%s %s: %s (%.3fs)\n",
-			subtest_text, *subtest_name,
-			result, igt_time_elapsed(thentime, &now));
+	_subtest_result_message(in_dynamic_subtest ? _SUBTEST_TYPE_DYNAMIC : _SUBTEST_TYPE_NORMAL,
+				*subtest_name,
+				result,
+				igt_time_elapsed(thentime, &now));
 
 	igt_terminate_spins();
 
@@ -1539,7 +1614,15 @@ void igt_skip(const char *f, ...)
 
 	if (!igt_only_list_subtests()) {
 		va_start(args, f);
-		vprintf(f, args);
+		if (runner_connected()) {
+			char *str;
+
+			vasprintf(&str, f, args);
+			send_to_runner(runnerpacket_log(STDOUT_FILENO, str));
+			free(str);
+		} else {
+			vprintf(f, args);
+		}
 		va_end(args);
 	}
 
@@ -1827,7 +1910,10 @@ static bool running_under_gdb(void)
 
 static void __write_stderr(const char *str, size_t len)
 {
-	igt_ignore_warn(write(STDERR_FILENO, str, len));
+	if (runner_connected())
+		log_to_runner_sig_safe(str, len);
+	else
+		igt_ignore_warn(write(STDERR_FILENO, str, len));
 }
 
 static void write_stderr(const char *str)
@@ -1840,7 +1926,10 @@ static const char hex[] = "0123456789abcdef";
 static void
 xputch(int c)
 {
-	igt_ignore_warn(write(STDERR_FILENO, (const void *) &c, 1));
+	if (runner_connected())
+		log_to_runner_sig_safe((const void *) &c, 1);
+	else
+		igt_ignore_warn(write(STDERR_FILENO, (const void *) &c, 1));
 }
 
 static int
@@ -2849,11 +2938,9 @@ void igt_vlog(const char *domain, enum igt_log_level level, const char *format,
 	/* prepend all except information messages with process, domain and log
 	 * level information */
 	if (level != IGT_LOG_INFO) {
-		fwrite(formatted_line, sizeof(char), strlen(formatted_line),
-		       file);
+		_log_line_fprintf(file, "%s", formatted_line);
 	} else {
-		fwrite(thread_id, sizeof(char), strlen(thread_id), file);
-		fwrite(line, sizeof(char), strlen(line), file);
+		_log_line_fprintf(file, "%s%s", thread_id, line);
 	}
 
 	pthread_mutex_unlock(&print_mutex);
-- 
2.30.2

[igt-dev] [PATCH i-g-t v2 3/7] runner: Use socket communications

[Petri Latvala]

Set up a socket and pass it to the tests. Dump received communications
to disk.

When generating results.json, use socket comms if it exists.

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arek@hiler.eu>
Acked-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>
---
 runner/executor.c  | 354 ++++++++++++++++++++++---
 runner/executor.h  |   1 +
 runner/resultgen.c | 631 ++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 947 insertions(+), 39 deletions(-)

diff --git a/runner/executor.c b/runner/executor.c
index 8a32fedd..669e8524 100644
--- a/runner/executor.c
+++ b/runner/executor.c
@@ -14,9 +14,11 @@
 #include <stdlib.h>
 #include <string.h>
 #include <sys/ioctl.h>
+#include <sys/mman.h>
 #include <sys/select.h>
 #include <sys/poll.h>
 #include <sys/signalfd.h>
+#include <sys/socket.h>
 #include <sys/stat.h>
 #include <sys/time.h>
 #include <sys/types.h>
@@ -31,6 +33,7 @@
 #include "igt_taints.h"
 #include "executor.h"
 #include "output_strings.h"
+#include "runnercomms.h"
 
 #define KMSG_HEADER "[IGT] "
 #define KMSG_WARN 4
@@ -367,7 +370,7 @@ static char *need_to_abort(const struct settings* settings)
 	return NULL;
 }
 
-static void prune_subtest(struct job_list_entry *entry, char *subtest)
+static void prune_subtest(struct job_list_entry *entry, const char *subtest)
 {
 	char *excl;
 
@@ -444,11 +447,77 @@ static bool prune_from_journal(struct job_list_entry *entry, int fd)
 	return pruned > 0;
 }
 
+struct prune_comms_data
+{
+	struct job_list_entry *entry;
+	int pruned;
+	bool got_exit;
+};
+
+static bool prune_handle_subtest_start(const struct runnerpacket *packet,
+				       runnerpacket_read_helper helper,
+				       void *userdata)
+{
+	struct prune_comms_data *data = userdata;
+
+	prune_subtest(data->entry, helper.subteststart.name);
+	data->pruned++;
+
+	return true;
+}
+
+static bool prune_handle_exit(const struct runnerpacket *packet,
+			      runnerpacket_read_helper helper,
+			      void *userdata)
+{
+	struct prune_comms_data *data = userdata;
+
+	data->got_exit = true;
+
+	return true;
+}
+
+static bool prune_from_comms(struct job_list_entry *entry, int fd)
+{
+	struct prune_comms_data data = {
+		.entry = entry,
+		.pruned = 0,
+		.got_exit = false,
+	};
+	struct comms_visitor visitor = {
+		.subtest_start = prune_handle_subtest_start,
+		.exit = prune_handle_exit,
+
+		.userdata = &data,
+	};
+	size_t old_count = entry->subtest_count;
+
+	if (comms_read_dump(fd, &visitor) == COMMSPARSE_ERROR)
+		return false;
+
+	/*
+	 * If we know the subtests we originally wanted to run, check
+	 * if we got an equal amount already.
+	 */
+	if (old_count > 0 && data.pruned >= old_count)
+		entry->binary[0] = '\0';
+
+	/*
+	 * If we don't know how many subtests there should be but we
+	 * got an exit, also consider the test fully finished.
+	 */
+	if (data.got_exit)
+		entry->binary[0] = '\0';
+
+	return data.pruned > 0;
+}
+
 static const char *filenames[_F_LAST] = {
 	[_F_JOURNAL] = "journal.txt",
 	[_F_OUT] = "out.txt",
 	[_F_ERR] = "err.txt",
 	[_F_DMESG] = "dmesg.txt",
+	[_F_SOCKET] = "comms",
 };
 
 static int open_at_end(int dirfd, const char *name)
@@ -480,6 +549,9 @@ bool open_output_files(int dirfd, int *fds, bool write)
 
 	for (i = 0; i < _F_LAST; i++) {
 		if ((fds[i] = openfunc(dirfd, filenames[i])) < 0) {
+			/* Ignore failure to open socket comms for reading */
+			if (i == _F_SOCKET && !write) continue;
+
 			while (--i >= 0)
 				close(fds[i]);
 			return false;
@@ -760,6 +832,16 @@ static int next_kill_signal(int killed)
 	}
 }
 
+static void write_packet_with_canary(int fd, struct runnerpacket *packet, bool sync)
+{
+	uint32_t canary = socket_dump_canary();
+
+	write(fd, &canary, sizeof(canary));
+	write(fd, packet, packet->size);
+	if (sync)
+		fdatasync(fd);
+}
+
 /*
  * Returns:
  *  =0 - Success
@@ -767,7 +849,8 @@ static int next_kill_signal(int killed)
  *  >0 - Timeout happened, need to recreate from journal
  */
 static int monitor_output(pid_t child,
-			  int outfd, int errfd, int kmsgfd, int sigfd,
+			  int outfd, int errfd, int socketfd,
+			  int kmsgfd, int sigfd,
 			  int *outputs,
 			  double *time_spent,
 			  struct settings *settings,
@@ -789,12 +872,15 @@ static int monitor_output(pid_t child,
 	unsigned long taints = 0;
 	bool aborting = false;
 	size_t disk_usage = 0;
+	bool socket_comms_used = false; /* whether the test actually uses comms */
 
 	igt_gettime(&time_beg);
 	time_last_activity = time_last_subtest = time_killed = time_beg;
 
 	if (errfd > nfds)
 		nfds = errfd;
+	if (socketfd > nfds)
+		nfds = socketfd;
 	if (kmsgfd > nfds)
 		nfds = kmsgfd;
 	if (sigfd > nfds)
@@ -833,6 +919,8 @@ static int monitor_output(pid_t child,
 			FD_SET(outfd, &set);
 		if (errfd >= 0)
 			FD_SET(errfd, &set);
+		if (socketfd >= 0)
+			FD_SET(socketfd, &set);
 		if (kmsgfd >= 0)
 			FD_SET(kmsgfd, &set);
 		if (sigfd >= 0)
@@ -965,6 +1053,99 @@ static int monitor_output(pid_t child,
 			}
 		}
 
+		if (socketfd >= 0 && FD_ISSET(socketfd, &set)) {
+			struct runnerpacket *packet;
+
+			time_last_activity = time_now;
+
+			/* Fully drain everything */
+			while (true) {
+				s = recv(socketfd, buf, sizeof(buf), MSG_DONTWAIT);
+
+				if (s < 0) {
+					if (errno == EAGAIN)
+						break;
+
+					errf("Error reading from communication socket: %m\n");
+
+					close(socketfd);
+					socketfd = -1;
+					goto socket_end;
+				}
+
+				packet = (struct runnerpacket *)buf;
+				if (s < sizeof(*packet) || s != packet->size) {
+					errf("Socket communication error: Received %zd bytes, expected %zd\n",
+					     s, s >= sizeof(packet->size) ? packet->size : sizeof(*packet));
+					close(socketfd);
+					socketfd = -1;
+					goto socket_end;
+				}
+
+				write_packet_with_canary(outputs[_F_SOCKET], packet, settings->sync);
+				disk_usage += packet->size;
+
+				/*
+				 * runner sends EXEC itself before executing
+				 * the test, other types indicate the test
+				 * really uses socket comms
+				 */
+				if (packet->type != PACKETTYPE_EXEC)
+					socket_comms_used = true;
+
+				if (packet->type == PACKETTYPE_SUBTEST_START ||
+				    packet->type == PACKETTYPE_DYNAMIC_SUBTEST_START)
+					time_last_subtest = time_now;
+
+				if (settings->log_level >= LOG_LEVEL_VERBOSE) {
+					runnerpacket_read_helper helper = {};
+					const char *time;
+
+					if (packet->type == PACKETTYPE_SUBTEST_START ||
+					    packet->type == PACKETTYPE_SUBTEST_RESULT ||
+					    packet->type == PACKETTYPE_DYNAMIC_SUBTEST_START ||
+					    packet->type == PACKETTYPE_DYNAMIC_SUBTEST_RESULT)
+						helper = read_runnerpacket(packet);
+
+					switch (helper.type) {
+					case PACKETTYPE_SUBTEST_START:
+						if (helper.subteststart.name)
+							outf("Starting subtest: %s\n", helper.subteststart.name);
+						break;
+					case PACKETTYPE_SUBTEST_RESULT:
+						if (helper.subtestresult.name && helper.subtestresult.result) {
+							time = "<unknown>";
+							if (helper.subtestresult.timeused)
+								time = helper.subtestresult.timeused;
+							outf("Subtest %s: %s (%ss)\n",
+							     helper.subtestresult.name,
+							     helper.subtestresult.result,
+							     time);
+						}
+						break;
+					case PACKETTYPE_DYNAMIC_SUBTEST_START:
+						if (helper.dynamicsubteststart.name)
+							outf("Starting dynamic subtest: %s\n", helper.dynamicsubteststart.name);
+						break;
+					case PACKETTYPE_DYNAMIC_SUBTEST_RESULT:
+						if (helper.dynamicsubtestresult.name && helper.dynamicsubtestresult.result) {
+							time = "<unknown>";
+							if (helper.dynamicsubtestresult.timeused)
+								time = helper.dynamicsubtestresult.timeused;
+							outf("Dynamic subtest %s: %s (%ss)\n",
+							     helper.dynamicsubtestresult.name,
+							     helper.dynamicsubtestresult.result,
+							     time);
+						}
+						break;
+					default:
+						break;
+					}
+				}
+			}
+		}
+	socket_end:
+
 		if (kmsgfd >= 0 && FD_ISSET(kmsgfd, &set)) {
 			long dmesgwritten;
 
@@ -1034,11 +1215,23 @@ static int monitor_output(pid_t child,
 					if (settings->log_level >= LOG_LEVEL_NORMAL)
 						outf("Exiting gracefully, currently running test will have a 'notrun' result\n");
 
-					dprintf(outputs[_F_JOURNAL], "%s%d (%.3fs)\n",
-						EXECUTOR_EXIT,
-						-SIGHUP, 0.0);
-					if (settings->sync)
-						fdatasync(outputs[_F_JOURNAL]);
+					if (socket_comms_used) {
+						struct runnerpacket *message, *override;
+
+						message = runnerpacket_log(STDOUT_FILENO, "runner: Exiting gracefully, overriding this test's result to be notrun\n");
+						write_packet_with_canary(outputs[_F_SOCKET], message, false); /* possible sync after the override packet */
+						free(message);
+
+						override = runnerpacket_resultoverride("notrun");
+						write_packet_with_canary(outputs[_F_SOCKET], override, settings->sync);
+						free(override);
+					} else {
+						dprintf(outputs[_F_JOURNAL], "%s%d (%.3fs)\n",
+							EXECUTOR_EXIT,
+							-SIGHUP, 0.0);
+						if (settings->sync)
+							fdatasync(outputs[_F_JOURNAL]);
+					}
 				}
 
 				aborting = true;
@@ -1055,9 +1248,10 @@ static int monitor_output(pid_t child,
 				time = 0.0;
 
 			if (!aborting) {
-				const char *exitline;
+				bool timeoutresult = false;
 
-				exitline = killed ? EXECUTOR_TIMEOUT : EXECUTOR_EXIT;
+				if (killed)
+					timeoutresult = true;
 
 				/* If we're stopping because we killed
 				 * the test for tainting, let's not
@@ -1071,7 +1265,7 @@ static int monitor_output(pid_t child,
 				 * journaling a timeout here.
 				 */
 				if (killed && is_tainted(taints)) {
-					exitline = EXECUTOR_EXIT;
+					timeoutresult = false;
 
 					/*
 					 * Also inject a message to
@@ -1084,11 +1278,22 @@ static int monitor_output(pid_t child,
 					 * have newlines on both ends
 					 * of this injection though.
 					 */
-					dprintf(outputs[_F_OUT],
-						"\nrunner: This test was killed due to a kernel taint (0x%lx).\n",
-						taints);
-					if (settings->sync)
-						fdatasync(outputs[_F_OUT]);
+					if (socket_comms_used) {
+						struct runnerpacket *message;
+						char killmsg[256];
+
+						snprintf(killmsg, sizeof(killmsg),
+							 "runner: This test was killed due to a kernel taint (0x%lx).\n", taints);
+						message = runnerpacket_log(STDOUT_FILENO, killmsg);
+						write_packet_with_canary(outputs[_F_SOCKET], message, settings->sync);
+						free(message);
+					} else {
+						dprintf(outputs[_F_OUT],
+							"\nrunner: This test was killed due to a kernel taint (0x%lx).\n",
+							taints);
+						if (settings->sync)
+							fdatasync(outputs[_F_OUT]);
+					}
 				}
 
 				/*
@@ -1096,21 +1301,58 @@ static int monitor_output(pid_t child,
 				 * exceeded the disk usage limit.
 				 */
 				if (killed && disk_usage_limit_exceeded(settings, disk_usage)) {
-					exitline = EXECUTOR_EXIT;
-					dprintf(outputs[_F_OUT],
-						"\nrunner: This test was killed due to exceeding disk usage limit. "
-						"(Used %zd bytes, limit %zd)\n",
-						disk_usage,
-						settings->disk_usage_limit);
-					if (settings->sync)
-						fdatasync(outputs[_F_OUT]);
+					timeoutresult = false;
+
+					if (socket_comms_used) {
+						struct runnerpacket *message;
+						char killmsg[256];
+
+						snprintf(killmsg, sizeof(killmsg),
+							 "runner: This test was killed due to exceeding disk usage limit. "
+							 "(Used %zd bytes, limit %zd)\n",
+							 disk_usage,
+							 settings->disk_usage_limit);
+						message = runnerpacket_log(STDOUT_FILENO, killmsg);
+						write_packet_with_canary(outputs[_F_SOCKET], message, settings->sync);
+						free(message);
+					} else {
+						dprintf(outputs[_F_OUT],
+							"\nrunner: This test was killed due to exceeding disk usage limit. "
+							"(Used %zd bytes, limit %zd)\n",
+							disk_usage,
+							settings->disk_usage_limit);
+						if (settings->sync)
+							fdatasync(outputs[_F_OUT]);
+					}
 				}
 
-				dprintf(outputs[_F_JOURNAL], "%s%d (%.3fs)\n",
-					exitline,
-					status, time);
-				if (settings->sync) {
-					fdatasync(outputs[_F_JOURNAL]);
+				if (socket_comms_used) {
+					struct runnerpacket *exitpacket;
+					char timestr[32];
+
+					snprintf(timestr, sizeof(timestr), "%.3f", time);
+
+					if (timeoutresult) {
+						struct runnerpacket *override;
+
+						override = runnerpacket_resultoverride("timeout");
+						write_packet_with_canary(outputs[_F_SOCKET], override, false); /* sync after exitpacket */
+						free(override);
+					}
+
+					exitpacket = runnerpacket_exit(status, timestr);
+					write_packet_with_canary(outputs[_F_SOCKET], exitpacket, settings->sync);
+					free(exitpacket);
+				} else {
+					const char *exitline;
+
+					exitline = timeoutresult ? EXECUTOR_TIMEOUT : EXECUTOR_EXIT;
+					dprintf(outputs[_F_JOURNAL], "%s%d (%.3fs)\n",
+						exitline,
+						status, time);
+					if (settings->sync) {
+						fdatasync(outputs[_F_JOURNAL]);
+					}
 				}
 
 				if (status == IGT_EXIT_ABORT) {
@@ -1155,6 +1397,7 @@ static int monitor_output(pid_t child,
 				free(outbuf);
 				close(outfd);
 				close(errfd);
+				close(socketfd);
 				close(kmsgfd);
 				return -1;
 			}
@@ -1178,6 +1421,7 @@ static int monitor_output(pid_t child,
 	free(outbuf);
 	close(outfd);
 	close(errfd);
+	close(socketfd);
 	close(kmsgfd);
 
 	if (aborting)
@@ -1187,7 +1431,7 @@ static int monitor_output(pid_t child,
 }
 
 static void __attribute__((noreturn))
-execute_test_process(int outfd, int errfd,
+execute_test_process(int outfd, int errfd, int socketfd,
 		     struct settings *settings,
 		     struct job_list_entry *entry)
 {
@@ -1239,6 +1483,13 @@ execute_test_process(int outfd, int errfd,
 		}
 	}
 
+	if (socketfd >= 0) {
+		struct runnerpacket *packet;
+
+		packet = runnerpacket_exec(argv);
+		write(socketfd, packet, packet->size);
+	}
+
 	execv(argv[0], argv);
 	fprintf(stderr, "Cannot execute %s\n", argv[0]);
 	exit(IGT_EXIT_INVALID);
@@ -1320,7 +1571,8 @@ static int execute_next_entry(struct execute_state *state,
 	int kmsgfd;
 	int outpipe[2] = { -1, -1 };
 	int errpipe[2] = { -1, -1 };
-	int outfd, errfd;
+	int socket[2] = { -1, -1 };
+	int outfd, errfd, socketfd;
 	char name[32];
 	pid_t child;
 	int result;
@@ -1350,6 +1602,12 @@ static int execute_next_entry(struct execute_state *state,
 		goto out_pipe;
 	}
 
+	if (socketpair(AF_UNIX, SOCK_DGRAM, 0, socket)) {
+		errf("Error creating sockets: %m\n");
+		result = -1;
+		goto out_pipe;
+	}
+
 	if ((kmsgfd = open("/dev/kmsg", O_RDONLY | O_CLOEXEC | O_NONBLOCK)) < 0) {
 		errf("Warning: Cannot open /dev/kmsg\n");
 	} else {
@@ -1390,27 +1648,39 @@ static int execute_next_entry(struct execute_state *state,
 		result = -1;
 		goto out_kmsgfd;
 	} else if (child == 0) {
+		char envstring[16];
+
 		outfd = outpipe[1];
 		errfd = errpipe[1];
+		socketfd = socket[1];
 		close(outpipe[0]);
 		close(errpipe[0]);
+		close(socket[0]);
 
 		sigprocmask(SIG_UNBLOCK, sigmask, NULL);
 
+		if (socketfd >= 0 && !getenv("IGT_RUNNER_DISABLE_SOCKET_COMMUNICATION")) {
+			snprintf(envstring, sizeof(envstring), "%d", socketfd);
+			setenv("IGT_RUNNER_SOCKET_FD", envstring, 1);
+		}
 		setenv("IGT_SENTINEL_ON_STDERR", "1", 1);
 
-		execute_test_process(outfd, errfd, settings, entry);
+		execute_test_process(outfd, errfd, socketfd, settings, entry);
 		/* unreachable */
 	}
 
 	outfd = outpipe[0];
 	errfd = errpipe[0];
+	socketfd = socket[0];
 	close(outpipe[1]);
 	close(errpipe[1]);
-	outpipe[1] = errpipe[1] = -1;
+	close(socket[1]);
+	outpipe[1] = errpipe[1] = socket[1] = -1;
 
-	result = monitor_output(child, outfd, errfd, kmsgfd, sigfd,
-				outputs, time_spent, settings, abortreason);
+	result = monitor_output(child, outfd, errfd, socketfd,
+				kmsgfd, sigfd,
+				outputs, time_spent, settings,
+				abortreason);
 
 out_kmsgfd:
 	close(kmsgfd);
@@ -1582,6 +1852,22 @@ bool initialize_execute_state_from_resume(int dirfd,
 
 	entry = &list->entries[i];
 	state->next = i;
+
+	if ((fd = openat(resdirfd, filenames[_F_SOCKET], O_RDONLY)) >= 0) {
+		if (!prune_from_comms(entry, fd)) {
+			/*
+			 * No subtests, or incomplete before the first
+			 * subtest. Not suitable to re-run.
+			 */
+			state->next = i + 1;
+		} else if (entry->binary[0] == '\0') {
+			/* Full completed */
+			state->next = i + 1;
+		}
+
+		close (fd);
+	}
+
 	if ((fd = openat(resdirfd, filenames[_F_JOURNAL], O_RDONLY)) >= 0) {
 		if (!prune_from_journal(entry, fd)) {
 			/*
diff --git a/runner/executor.h b/runner/executor.h
index 6c83e649..31f4ac16 100644
--- a/runner/executor.h
+++ b/runner/executor.h
@@ -22,6 +22,7 @@ enum {
 	_F_OUT,
 	_F_ERR,
 	_F_DMESG,
+	_F_SOCKET,
 	_F_LAST,
 };
 
diff --git a/runner/resultgen.c b/runner/resultgen.c
index 79725ca2..3d753828 100644
--- a/runner/resultgen.c
+++ b/runner/resultgen.c
@@ -1,6 +1,7 @@
 #include <assert.h>
 #include <ctype.h>
 #include <fcntl.h>
+#include <inttypes.h>
 #include <stdio.h>
 #include <string.h>
 #include <sys/mman.h>
@@ -12,6 +13,7 @@
 
 #include "igt_aux.h"
 #include "igt_core.h"
+#include "runnercomms.h"
 #include "resultgen.h"
 #include "settings.h"
 #include "executor.h"
@@ -147,7 +149,7 @@ static const char *next_line(const char *line, const char *bufend)
 		return NULL;
 }
 
-static void append_line(char **buf, size_t *buflen, char *line)
+static void append_line(char **buf, size_t *buflen, const char *line)
 {
 	size_t linelen = strlen(line);
 
@@ -1228,6 +1230,606 @@ static void fill_from_journal(int fd,
 	fclose(f);
 }
 
+typedef enum comms_state {
+	STATE_INITIAL = 0,
+	STATE_AFTER_EXEC,
+	STATE_SUBTEST_STARTED,
+	STATE_DYNAMIC_SUBTEST_STARTED,
+	STATE_BETWEEN_DYNAMIC_SUBTESTS,
+	STATE_BETWEEN_SUBTESTS,
+	STATE_EXITED,
+} comms_state_t;
+
+struct comms_context
+{
+	comms_state_t state;
+
+	struct json_object *binaryruntimeobj;
+	struct json_object *current_test;
+	struct json_object *current_dynamic_subtest;
+	char *current_subtest_name;
+	char *current_dynamic_subtest_name;
+
+	char *outbuf, *errbuf;
+	size_t outbuflen, errbuflen;
+	size_t outidx, nextoutidx;
+	size_t erridx, nexterridx;
+	size_t dynoutidx, nextdynoutidx;
+	size_t dynerridx, nextdynerridx;
+
+	char *igt_version;
+
+	char *subtestresult;
+	char *dynamicsubtestresult;
+
+	char *cmdline;
+	int exitcode;
+
+	struct subtest_list *subtests;
+	struct subtest *subtest;
+	struct results *results;
+	struct job_list_entry *entry;
+	const char *binary;
+};
+
+static void comms_free_context(struct comms_context *context)
+{
+	free(context->current_subtest_name);
+	free(context->current_dynamic_subtest_name);
+	free(context->outbuf);
+	free(context->errbuf);
+	free(context->igt_version);
+	free(context->subtestresult);
+	free(context->dynamicsubtestresult);
+	free(context->cmdline);
+}
+
+static void comms_inject_subtest_start_log(struct comms_context *context,
+					   const char *prefix,
+					   const char *subtestname)
+{
+	char msg[512];
+
+	snprintf(msg, sizeof(msg), "%s%s\n", prefix, subtestname);
+	append_line(&context->outbuf, &context->outbuflen, msg);
+	append_line(&context->errbuf, &context->errbuflen, msg);
+}
+
+static void comms_inject_subtest_end_log(struct comms_context *context,
+					 const char *prefix,
+					 const char *subtestname,
+					 const char *subtestresult,
+					 const char *timeused)
+{
+	char msg[512];
+
+	snprintf(msg, sizeof(msg), "%s%s: %s (%ss)\n", prefix, subtestname, subtestresult, timeused);
+	append_line(&context->outbuf, &context->outbuflen, msg);
+	append_line(&context->errbuf, &context->errbuflen, msg);
+}
+
+static void comms_finish_subtest(struct comms_context *context)
+{
+	json_object_object_add(context->current_test, "out",
+			       new_escaped_json_string(context->outbuf + context->outidx, context->outbuflen - context->outidx));
+	json_object_object_add(context->current_test, "err",
+			       new_escaped_json_string(context->errbuf + context->outidx, context->errbuflen - context->erridx));
+
+	if (context->igt_version)
+		add_igt_version(context->current_test, context->igt_version, strlen(context->igt_version));
+
+	if (context->subtestresult == NULL)
+		context->subtestresult = strdup("incomplete");
+	set_result(context->current_test, context->subtestresult);
+
+	free(context->subtestresult);
+	context->subtestresult = NULL;
+	context->current_test = NULL;
+
+	context->outidx = context->nextoutidx;
+	context->erridx = context->nexterridx;
+}
+
+static void comms_finish_dynamic_subtest(struct comms_context *context)
+{
+	json_object_object_add(context->current_dynamic_subtest, "out",
+			       new_escaped_json_string(context->outbuf + context->dynoutidx, context->outbuflen - context->dynoutidx));
+	json_object_object_add(context->current_dynamic_subtest, "err",
+			       new_escaped_json_string(context->errbuf + context->dynerridx, context->errbuflen - context->dynerridx));
+
+	if (context->igt_version)
+		add_igt_version(context->current_dynamic_subtest, context->igt_version, strlen(context->igt_version));
+
+	if (context->dynamicsubtestresult == NULL)
+		context->dynamicsubtestresult = strdup("incomplete");
+	set_result(context->current_dynamic_subtest, context->dynamicsubtestresult);
+
+	free(context->dynamicsubtestresult);
+	context->dynamicsubtestresult = NULL;
+	context->current_dynamic_subtest = NULL;
+
+	context->dynoutidx = context->nextdynoutidx;
+	context->dynerridx = context->nextdynerridx;
+}
+
+static void comms_add_new_subtest(struct comms_context *context,
+				  const char *subtestname)
+{
+	char piglit_name[256];
+
+	add_subtest(context->subtests, strdup(subtestname));
+	context->subtest = &context->subtests->subs[context->subtests->size - 1];
+	generate_piglit_name(context->binary, subtestname, piglit_name, sizeof(piglit_name));
+	context->current_test = get_or_create_json_object(context->results->tests, piglit_name);
+	free(context->current_subtest_name);
+	context->current_subtest_name = strdup(subtestname);
+}
+
+static void comms_add_new_dynamic_subtest(struct comms_context *context,
+					  const char *dynamic_name)
+{
+	char piglit_name[256];
+	char dynamic_piglit_name[256];
+
+	add_dynamic_subtest(context->subtest, strdup(dynamic_name));
+	generate_piglit_name(context->binary, context->current_subtest_name, piglit_name, sizeof(piglit_name));
+	generate_piglit_name_for_dynamic(piglit_name, dynamic_name, dynamic_piglit_name, sizeof(dynamic_piglit_name));
+	context->current_dynamic_subtest = get_or_create_json_object(context->results->tests, dynamic_piglit_name);
+	free(context->current_dynamic_subtest_name);
+	context->current_dynamic_subtest_name = strdup(dynamic_name);
+}
+
+static bool comms_handle_log(const struct runnerpacket *packet,
+			     runnerpacket_read_helper helper,
+			     void *userdata)
+{
+	struct comms_context *context = userdata;
+	char **textbuf;
+	size_t *textlen;
+
+	if (helper.log.stream == STDOUT_FILENO) {
+		textbuf = &context->outbuf;
+		textlen = &context->outbuflen;
+	} else {
+		textbuf = &context->errbuf;
+		textlen = &context->errbuflen;
+	}
+	append_line(textbuf, textlen, helper.log.text);
+
+	return true;
+}
+
+static bool comms_handle_exec(const struct runnerpacket *packet,
+			      runnerpacket_read_helper helper,
+			      void *userdata)
+{
+	struct comms_context *context = userdata;
+
+	switch (context->state) {
+	case STATE_INITIAL:
+		break;
+
+	case STATE_AFTER_EXEC:
+		/*
+		 * Resume after an exec that didn't involve any
+		 * subtests. Resumes can only happen for tests with
+		 * subtests, so while we might have logs already
+		 * collected, we have nowhere to put them. The joblist
+		 * doesn't help, because the ordering is up to the
+		 * test.
+		 */
+		printf("Warning: Need to discard %zd bytes of logs, no subtest data\n", context->outbuflen + context->errbuflen);
+		context->outbuflen = context->errbuflen = 0;
+		context->outidx = context->erridx = 0;
+		context->nextoutidx = context->nexterridx = 0;
+		break;
+
+	case STATE_SUBTEST_STARTED:
+	case STATE_DYNAMIC_SUBTEST_STARTED:
+	case STATE_BETWEEN_DYNAMIC_SUBTESTS:
+	case STATE_BETWEEN_SUBTESTS:
+	case STATE_EXITED:
+		/* A resume exec, so we're already collecting data. */
+		assert(context->current_test != NULL);
+		comms_finish_subtest(context);
+		break;
+	default:
+		assert(false); /* unreachable */
+	}
+
+	free(context->cmdline);
+	context->cmdline = strdup(helper.exec.cmdline);
+
+	context->state = STATE_AFTER_EXEC;
+
+	return true;
+}
+
+static bool comms_handle_exit(const struct runnerpacket *packet,
+			      runnerpacket_read_helper helper,
+			      void *userdata)
+{
+	struct comms_context *context = userdata;
+	char piglit_name[256];
+
+	if (context->state == STATE_AFTER_EXEC) {
+		/*
+		 * Exit after exec, so we didn't get any
+		 * subtests. Check if there's supposed to be any,
+		 * otherwise stuff logs into the binary's result.
+		 */
+
+		char *subtestname = NULL;
+
+		if (context->entry->subtest_count > 0) {
+			subtestname = context->entry->subtests[0];
+			add_subtest(context->subtests, strdup(subtestname));
+		}
+		generate_piglit_name(context->binary, subtestname, piglit_name, sizeof(piglit_name));
+		context->current_test = get_or_create_json_object(context->results->tests, piglit_name);
+
+		/* Get result from exitcode unless we have an override already */
+		if (context->subtestresult == NULL)
+			context->subtestresult = strdup(result_from_exitcode(helper.exit.exitcode));
+	} else if (helper.exit.exitcode == IGT_EXIT_ABORT || helper.exit.exitcode == GRACEFUL_EXITCODE) {
+		/*
+		 * If we did get subtests, we need to assign the
+		 * special exitcode results to the last subtest,
+		 * normal and dynamic
+		 */
+		const char *result = helper.exit.exitcode == IGT_EXIT_ABORT ? "abort" : "notrun";
+
+		free(context->subtestresult);
+		context->subtestresult = strdup(result);
+		free(context->dynamicsubtestresult);
+		context->dynamicsubtestresult = strdup(result);
+	}
+
+	context->exitcode = helper.exit.exitcode;
+	add_runtime(context->binaryruntimeobj, strtod(helper.exit.timeused, NULL));
+
+	context->state = STATE_EXITED;
+
+	return true;
+}
+
+static bool comms_handle_subtest_start(const struct runnerpacket *packet,
+				       runnerpacket_read_helper helper,
+				       void *userdata)
+{
+	struct comms_context *context = userdata;
+	char errmsg[512];
+
+	switch (context->state) {
+	case STATE_INITIAL:
+	case STATE_EXITED:
+		/* Subtest starts when we're not even running? (Before exec or after exit) */
+		fprintf(stderr, "Error: Unexpected subtest start (binary wasn't running)\n");
+		return false;
+	case STATE_SUBTEST_STARTED:
+	case STATE_DYNAMIC_SUBTEST_STARTED:
+	case STATE_BETWEEN_DYNAMIC_SUBTESTS:
+		/*
+		 * Subtest starts when the previous one was still
+		 * running. Text-based parsing would figure that a
+		 * resume happened, but we know the real deal with
+		 * socket comms.
+		 */
+		snprintf(errmsg, sizeof(errmsg),
+			 "\nrunner: Subtest %s already running when subtest %s starts. This is a test bug.\n",
+			 context->current_subtest_name,
+			 helper.subteststart.name);
+		append_line(&context->errbuf, &context->errbuflen, errmsg);
+
+		if (context->state == STATE_DYNAMIC_SUBTEST_STARTED ||
+		    context->state == STATE_BETWEEN_DYNAMIC_SUBTESTS)
+			comms_finish_dynamic_subtest(context);
+
+		/* fallthrough */
+	case STATE_BETWEEN_SUBTESTS:
+		/* Already collecting for a subtest, finish it up */
+		if (context->current_dynamic_subtest)
+			comms_finish_dynamic_subtest(context);
+
+		comms_finish_subtest(context);
+
+		/* fallthrough */
+	case STATE_AFTER_EXEC:
+		comms_add_new_subtest(context, helper.subteststart.name);
+
+		/* Subtest starting message is not in logs with socket comms, inject it manually */
+		comms_inject_subtest_start_log(context, STARTING_SUBTEST, helper.subteststart.name);
+
+		break;
+	default:
+		assert(false); /* unreachable */
+	}
+
+	context->state = STATE_SUBTEST_STARTED;
+
+	return true;
+}
+
+static bool comms_handle_subtest_result(const struct runnerpacket *packet,
+					runnerpacket_read_helper helper,
+					void *userdata)
+{
+	struct comms_context *context = userdata;
+	char errmsg[512];
+
+	switch (context->state) {
+	case STATE_INITIAL:
+	case STATE_EXITED:
+		/* Subtest result when we're not even running? (Before exec or after exit) */
+		fprintf(stderr, "Error: Unexpected subtest result (binary wasn't running)\n");
+		return false;
+	case STATE_DYNAMIC_SUBTEST_STARTED:
+		/*
+		 * Subtest result when dynamic subtest is still
+		 * running. Text-based parsing would consider that an
+		 * incomplete, we're able to inject a warning.
+		 */
+		snprintf(errmsg, sizeof(errmsg),
+			 "\nrunner: Dynamic subtest %s still running when subtest %s ended. This is a test bug.\n",
+			 context->current_dynamic_subtest_name,
+			 helper.subtestresult.name);
+		append_line(&context->errbuf, &context->errbuflen, errmsg);
+		comms_finish_dynamic_subtest(context);
+		break;
+	case STATE_BETWEEN_SUBTESTS:
+		/* Subtest result without starting it, and we're already collecting logs for a previous test */
+		comms_finish_subtest(context);
+		comms_add_new_subtest(context, helper.subtestresult.name);
+		break;
+	case STATE_AFTER_EXEC:
+		/* Subtest result without starting it, so comes from a fixture. We're not yet collecting logs for anything. */
+		comms_add_new_subtest(context, helper.subtestresult.name);
+		break;
+	case STATE_SUBTEST_STARTED:
+	case STATE_BETWEEN_DYNAMIC_SUBTESTS:
+		/* Normal flow */
+		break;
+	default:
+		assert(false); /* unreachable */
+	}
+
+	comms_inject_subtest_end_log(context,
+				     SUBTEST_RESULT,
+				     helper.subtestresult.name,
+				     helper.subtestresult.result,
+				     helper.subtestresult.timeused);
+
+	/* Next subtest, if any, will begin its logs right after that result line */
+	context->nextoutidx = context->outbuflen;
+	context->nexterridx = context->errbuflen;
+
+	/*
+	 * Only store the actual result from the packet if we don't
+	 * already have one. If we do, it's from an override.
+	 */
+	if (context->subtestresult == NULL) {
+		const char *mappedresult;
+
+		parse_result_string(helper.subtestresult.result,
+				    strlen(helper.subtestresult.result),
+				    &mappedresult, NULL);
+		context->subtestresult = strdup(mappedresult);
+	}
+
+	context->state = STATE_BETWEEN_SUBTESTS;
+
+	return true;
+}
+
+static bool comms_handle_dynamic_subtest_start(const struct runnerpacket *packet,
+					       runnerpacket_read_helper helper,
+					       void *userdata)
+{
+	struct comms_context *context = userdata;
+	char errmsg[512];
+
+	switch (context->state) {
+	case STATE_INITIAL:
+	case STATE_EXITED:
+		/* Dynamic subtest starts when we're not even running? (Before exec or after exit) */
+		fprintf(stderr, "Error: Unexpected dynamic subtest start (binary wasn't running)\n");
+		return false;
+	case STATE_AFTER_EXEC:
+		/* Binary was running but a subtest wasn't. We don't know where to inject an error message. */
+		fprintf(stderr, "Error: Unexpected dynamic subtest start (subtest wasn't running)\n");
+		return false;
+	case STATE_BETWEEN_SUBTESTS:
+		/*
+		 * Dynamic subtest starts when a subtest is not
+		 * running. We can't know which subtest this dynamic
+		 * subtest was supposed to be in. But we can inject a
+		 * warn into the previous subtest.
+		 */
+		snprintf(errmsg, sizeof(errmsg),
+			 "\nrunner: Dynamic subtest %s started when not inside a subtest. This is a test bug.\n",
+			 helper.dynamicsubteststart.name);
+		append_line(&context->errbuf, &context->errbuflen, errmsg);
+
+		/* Leave the state as is and hope for the best */
+		return true;
+	case STATE_DYNAMIC_SUBTEST_STARTED:
+		snprintf(errmsg, sizeof(errmsg),
+			 "\nrunner: Dynamic subtest %s already running when dynamic subtest %s starts. This is a test bug.\n",
+			 context->current_dynamic_subtest_name,
+			 helper.dynamicsubteststart.name);
+		append_line(&context->errbuf, &context->errbuflen, errmsg);
+
+		/* fallthrough */
+	case STATE_BETWEEN_DYNAMIC_SUBTESTS:
+		comms_finish_dynamic_subtest(context);
+		/* fallthrough */
+	case STATE_SUBTEST_STARTED:
+		comms_add_new_dynamic_subtest(context, helper.dynamicsubteststart.name);
+
+		/* Dynamic subtest starting message is not in logs with socket comms, inject it manually */
+		comms_inject_subtest_start_log(context, STARTING_DYNAMIC_SUBTEST, helper.dynamicsubteststart.name);
+
+		break;
+	default:
+		assert(false); /* unreachable */
+	}
+
+	context->state = STATE_DYNAMIC_SUBTEST_STARTED;
+
+	return true;
+}
+
+static bool comms_handle_dynamic_subtest_result(const struct runnerpacket *packet,
+						runnerpacket_read_helper helper,
+						void *userdata)
+{
+	struct comms_context *context = userdata;
+	char errmsg[512];
+
+	switch (context->state) {
+	case STATE_INITIAL:
+	case STATE_EXITED:
+		/* Dynamic subtest result when we're not even running? (Before exec or after exit) */
+		fprintf(stderr, "Error: Unexpected dynamic subtest result (binary wasn't running)\n");
+		return false;
+	case STATE_AFTER_EXEC:
+		/* Binary was running but a subtest wasn't. We don't know where to inject an error message. */
+		fprintf(stderr, "Error: Unexpected dynamic subtest result (subtest wasn't running)\n");
+		return false;
+	case STATE_BETWEEN_SUBTESTS:
+		/*
+		 * Dynamic subtest result when a subtest is not
+		 * running. We can't know which subtest this dynamic
+		 * subtest was supposed to be in. But we can inject a
+		 * warn into the previous subtest.
+		 */
+		snprintf(errmsg, sizeof(errmsg),
+			 "\nrunner: Dynamic subtest %s result when not inside a subtest. This is a test bug.\n",
+			 helper.dynamicsubtestresult.name);
+		append_line(&context->errbuf, &context->errbuflen, errmsg);
+
+		/* Leave the state as is and hope for the best */
+		return true;
+	case STATE_BETWEEN_DYNAMIC_SUBTESTS:
+		/*
+		 * Result without starting. There's no
+		 * skip_subtests_henceforth equivalent for dynamic
+		 * subtests so this shouldn't happen, but we can
+		 * handle it nevertheless.
+		 */
+		comms_finish_dynamic_subtest(context);
+		/* fallthrough */
+	case STATE_SUBTEST_STARTED:
+		/* Result without starting, but we aren't collecting for a dynamic subtest yet */
+		comms_add_new_dynamic_subtest(context, helper.dynamicsubtestresult.name);
+		break;
+	case STATE_DYNAMIC_SUBTEST_STARTED:
+		/* Normal flow */
+		break;
+	default:
+		assert(false); /* unreachable */
+	}
+
+	comms_inject_subtest_end_log(context,
+				     DYNAMIC_SUBTEST_RESULT,
+				     helper.dynamicsubtestresult.name,
+				     helper.dynamicsubtestresult.result,
+				     helper.dynamicsubtestresult.timeused);
+
+	/* Next dynamic subtest, if any, will begin its logs right after that result line */
+	context->nextdynoutidx = context->outbuflen;
+	context->nextdynerridx = context->errbuflen;
+
+	/*
+	 * Only store the actual result from the packet if we don't
+	 * already have one. If we do, it's from an override.
+	 */
+	if (context->dynamicsubtestresult == NULL) {
+		const char *mappedresult;
+
+		parse_result_string(helper.dynamicsubtestresult.result,
+				    strlen(helper.dynamicsubtestresult.result),
+				    &mappedresult, NULL);
+		context->dynamicsubtestresult = strdup(mappedresult);
+	}
+
+	context->state = STATE_BETWEEN_DYNAMIC_SUBTESTS;
+
+	return true;
+}
+
+static bool comms_handle_versionstring(const struct runnerpacket *packet,
+				       runnerpacket_read_helper helper,
+				       void *userdata)
+{
+	struct comms_context *context = userdata;
+
+	free(context->igt_version);
+	context->igt_version = strdup(helper.versionstring.text);
+
+	return true;
+}
+
+static bool comms_handle_result_override(const struct runnerpacket *packet,
+					 runnerpacket_read_helper helper,
+					 void *userdata)
+{
+	struct comms_context *context = userdata;
+
+	if (context->current_dynamic_subtest) {
+		free(context->dynamicsubtestresult);
+		context->dynamicsubtestresult = strdup(helper.resultoverride.result);
+	}
+
+	free(context->subtestresult);
+	context->subtestresult = strdup(helper.resultoverride.result);
+
+	return true;
+}
+
+static int fill_from_comms(int fd,
+			   struct job_list_entry *entry,
+			   struct subtest_list *subtests,
+			   struct results *results)
+{
+	struct comms_context context = {};
+	struct comms_visitor visitor = {
+		.log = comms_handle_log,
+		.exec = comms_handle_exec,
+		.exit = comms_handle_exit,
+		.subtest_start = comms_handle_subtest_start,
+		.subtest_result = comms_handle_subtest_result,
+		.dynamic_subtest_start = comms_handle_dynamic_subtest_start,
+		.dynamic_subtest_result = comms_handle_dynamic_subtest_result,
+		.versionstring = comms_handle_versionstring,
+		.result_override = comms_handle_result_override,
+
+		.userdata = &context,
+	};
+	char piglit_name[256];
+	int ret = COMMSPARSE_EMPTY;
+
+	if (fd < 0)
+		return COMMSPARSE_EMPTY;
+
+	context.entry = entry;
+	context.binary = entry->binary;
+	generate_piglit_name(entry->binary, NULL, piglit_name, sizeof(piglit_name));
+	context.binaryruntimeobj = get_or_create_json_object(results->runtimes, piglit_name);
+	context.results = results;
+	context.subtests = subtests;
+
+	ret = comms_read_dump(fd, &visitor);
+
+	if (context.current_dynamic_subtest != NULL)
+		comms_finish_dynamic_subtest(&context);
+	if (context.current_test != NULL)
+		comms_finish_subtest(&context);
+	comms_free_context(&context);
+
+	return ret;
+}
+
 static bool result_is_requested(struct job_list_entry *entry,
 				const char *subtestname,
 				const char *dynamic_name)
@@ -1505,6 +2107,7 @@ static bool parse_test_directory(int dirfd,
 	int fds[_F_LAST];
 	struct subtest_list subtests = {};
 	bool status = true;
+	int commsparsed;
 
 	if (!open_output_files(dirfd, fds, false)) {
 		fprintf(stderr, "Error opening output files\n");
@@ -1517,10 +2120,28 @@ static bool parse_test_directory(int dirfd,
 	 */
 	fill_from_journal(fds[_F_JOURNAL], entry, &subtests, results);
 
-	if (!fill_from_output(fds[_F_OUT], entry->binary, "out", &subtests, results->tests) ||
-	    !fill_from_output(fds[_F_ERR], entry->binary, "err", &subtests, results->tests) ||
-	    !fill_from_dmesg(fds[_F_DMESG], settings, entry->binary, &subtests, results->tests)) {
-		fprintf(stderr, "Error parsing output files\n");
+	/*
+	 * Get test output from socket comms if it exists, otherwise
+	 * parse stdout/stderr
+	 */
+	commsparsed = fill_from_comms(fds[_F_SOCKET], entry, &subtests, results);
+	if (commsparsed == COMMSPARSE_ERROR) {
+		fprintf(stderr, "Error parsing output files (comms)\n");
+		status = false;
+		goto parse_output_end;
+	}
+
+	if (commsparsed == COMMSPARSE_EMPTY) {
+		if (!fill_from_output(fds[_F_OUT], entry->binary, "out", &subtests, results->tests) ||
+		    !fill_from_output(fds[_F_ERR], entry->binary, "err", &subtests, results->tests)) {
+			fprintf(stderr, "Error parsing output files (out.txt, err.txt)\n");
+			status = false;
+			goto parse_output_end;
+		}
+	}
+
+	if (!fill_from_dmesg(fds[_F_DMESG], settings, entry->binary, &subtests, results->tests)) {
+		fprintf(stderr, "Error parsing output files (dmesg.txt)\n");
 		status = false;
 		goto parse_output_end;
 	}
-- 
2.30.2

[igt-dev] [PATCH i-g-t v2 4/7] runner/runner_tests: Add tests for socket communications

[Petri Latvala]

Test that initializing resumes from various states of execution gets
the correct resume-state up.

v2: Flesh out commit message (Kamil)

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arek@hiler.eu>
Acked-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>
---
 runner/runner_tests.c | 186 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 186 insertions(+)

diff --git a/runner/runner_tests.c b/runner/runner_tests.c
index db0ce2ac..a7e968f8 100644
--- a/runner/runner_tests.c
+++ b/runner/runner_tests.c
@@ -7,6 +7,7 @@
 #include <json.h>
 
 #include "igt.h"
+#include "runnercomms.h"
 
 #include "settings.h"
 #include "job_list.h"
@@ -239,6 +240,16 @@ static void assert_execution_results_exist(int dirfd)
 	assert_execution_created(dirfd, "dmesg.txt");
 }
 
+static void write_packet_with_canary(int fd, struct runnerpacket *packet)
+{
+	uint32_t canary = socket_dump_canary();
+
+	write(fd, &canary, sizeof(canary));
+	write(fd, packet, packet->size);
+
+	free(packet);
+}
+
 igt_main
 {
 	struct settings *settings = malloc(sizeof(*settings));
@@ -1226,6 +1237,62 @@ igt_main
 		}
 	}
 
+	igt_subtest_group {
+		char dirname[] = "tmpdirXXXXXX";
+		struct job_list *list = malloc(sizeof(*list));
+		volatile int dirfd = -1, subdirfd = -1, fd = -1;
+
+		igt_fixture {
+			init_job_list(list);
+			igt_require(mkdtemp(dirname) != NULL);
+		}
+
+		igt_subtest("execute-initialize-subtest-started-comms") {
+			struct execute_state state;
+			const char *argv[] = { "runner",
+					       "--allow-non-root",
+					       "--multiple-mode",
+					       "-t", "successtest",
+					       testdatadir,
+					       dirname,
+			};
+			const char excludestring[] = "!first-subtest";
+
+			igt_assert(parse_options(ARRAY_SIZE(argv), (char**)argv, settings));
+			igt_assert(create_job_list(list, settings));
+			igt_assert(list->size == 1);
+			igt_assert(list->entries[0].subtest_count == 0);
+
+			igt_assert(serialize_settings(settings));
+			igt_assert(serialize_job_list(list, settings));
+
+			igt_assert((dirfd = open(dirname, O_DIRECTORY | O_RDONLY)) >= 0);
+			igt_assert(mkdirat(dirfd, "0", 0770) == 0);
+			igt_assert((subdirfd = openat(dirfd, "0", O_DIRECTORY | O_RDONLY)) >= 0);
+			igt_assert((fd = openat(subdirfd, "comms", O_CREAT | O_WRONLY | O_EXCL, 0660)) >= 0);
+			write_packet_with_canary(fd, runnerpacket_subtest_start("first-subtest"));
+
+			free_job_list(list);
+			clear_settings(settings);
+			igt_assert(initialize_execute_state_from_resume(dirfd, &state, settings, list));
+
+			igt_assert_eq(state.next, 0);
+			igt_assert_eq(list->size, 1);
+			igt_assert_eq(list->entries[0].subtest_count, 2);
+			igt_assert_eqstr(list->entries[0].subtests[0], "*");
+			igt_assert_eqstr(list->entries[0].subtests[1], excludestring);
+		}
+
+		igt_fixture {
+			close(fd);
+			close(subdirfd);
+			close(dirfd);
+			clear_directory(dirname);
+			free_job_list(list);
+			free(list);
+		}
+	}
+
 	igt_subtest_group {
 		char dirname[] = "tmpdirXXXXXX";
 		struct job_list *list = malloc(sizeof(*list));
@@ -1282,6 +1349,62 @@ igt_main
 		}
 	}
 
+	igt_subtest_group {
+		char dirname[] = "tmpdirXXXXXX";
+		struct job_list *list = malloc(sizeof(*list));
+		volatile int dirfd = -1, subdirfd = -1, fd = -1;
+
+		igt_fixture {
+			init_job_list(list);
+			igt_require(mkdtemp(dirname) != NULL);
+		}
+
+		igt_subtest("execute-initialize-all-subtests-started-comms") {
+			struct execute_state state;
+			const char *argv[] = { "runner",
+					       "--allow-non-root",
+					       "--multiple-mode",
+					       "-t", "successtest@first-subtest",
+					       "-t", "successtest@second-subtest",
+					       testdatadir,
+					       dirname,
+			};
+
+			igt_assert(parse_options(ARRAY_SIZE(argv), (char**)argv, settings));
+			igt_assert(create_job_list(list, settings));
+			igt_assert(list->size == 1);
+			igt_assert(list->entries[0].subtest_count == 2);
+
+			igt_assert(serialize_settings(settings));
+			igt_assert(serialize_job_list(list, settings));
+
+			igt_assert((dirfd = open(dirname, O_DIRECTORY | O_RDONLY)) >= 0);
+			igt_assert(mkdirat(dirfd, "0", 0770) == 0);
+			igt_assert((subdirfd = openat(dirfd, "0", O_DIRECTORY | O_RDONLY)) >= 0);
+			igt_assert((fd = openat(subdirfd, "comms", O_CREAT | O_WRONLY | O_EXCL, 0660)) >= 0);
+			write_packet_with_canary(fd, runnerpacket_subtest_start("first-subtest"));
+			write_packet_with_canary(fd, runnerpacket_subtest_start("second-subtest"));
+
+			free_job_list(list);
+			clear_settings(settings);
+			igt_assert(initialize_execute_state_from_resume(dirfd, &state, settings, list));
+
+			/* All subtests are in journal, the entry should be considered completed */
+			igt_assert_eq(state.next, 1);
+			igt_assert_eq(list->size, 1);
+			igt_assert_eq(list->entries[0].subtest_count, 4);
+		}
+
+		igt_fixture {
+			close(fd);
+			close(subdirfd);
+			close(dirfd);
+			clear_directory(dirname);
+			free_job_list(list);
+			free(list);
+		}
+	}
+
 	igt_subtest_group {
 		char dirname[] = "tmpdirXXXXXX";
 		struct job_list *list = malloc(sizeof(*list));
@@ -1341,6 +1464,66 @@ igt_main
 		}
 	}
 
+	igt_subtest_group {
+		char dirname[] = "tmpdirXXXXXX";
+		struct job_list *list = malloc(sizeof(*list));
+		volatile int dirfd = -1, subdirfd = -1, fd = -1;
+
+		igt_fixture {
+			init_job_list(list);
+			igt_require(mkdtemp(dirname) != NULL);
+		}
+
+		igt_subtest("execute-initialize-subtests-complete-comms") {
+			struct execute_state state;
+			const char *argv[] = { "runner",
+					       "--allow-non-root",
+					       "--multiple-mode",
+					       testdatadir,
+					       dirname,
+			};
+
+			igt_assert(parse_options(ARRAY_SIZE(argv), (char**)argv, settings));
+			igt_assert(create_job_list(list, settings));
+			igt_assert(list->size == NUM_TESTDATA_BINARIES);
+
+			if (!strcmp(list->entries[0].binary, "no-subtests")) {
+				struct job_list_entry tmp = list->entries[0];
+				list->entries[0] = list->entries[1];
+				list->entries[1] = tmp;
+			}
+
+			igt_assert(list->entries[0].subtest_count == 0);
+
+			igt_assert(serialize_settings(settings));
+			igt_assert(serialize_job_list(list, settings));
+
+			igt_assert_lte(0, dirfd = open(dirname, O_DIRECTORY | O_RDONLY));
+			igt_assert_eq(mkdirat(dirfd, "0", 0770), 0);
+			igt_assert((subdirfd = openat(dirfd, "0", O_DIRECTORY | O_RDONLY)) >= 0);
+			igt_assert((fd = openat(subdirfd, "comms", O_CREAT | O_WRONLY | O_EXCL, 0660)) >= 0);
+			write_packet_with_canary(fd, runnerpacket_subtest_start("first-subtest"));
+			write_packet_with_canary(fd, runnerpacket_subtest_start("second-subtest"));
+			write_packet_with_canary(fd, runnerpacket_exit(0, "0.000s"));
+
+			free_job_list(list);
+			clear_settings(settings);
+			igt_assert(initialize_execute_state_from_resume(dirfd, &state, settings, list));
+
+			igt_assert_eq(state.next, 1);
+			igt_assert_eq(list->size, NUM_TESTDATA_BINARIES);
+		}
+
+		igt_fixture {
+			close(fd);
+			close(subdirfd);
+			close(dirfd);
+			clear_directory(dirname);
+			free_job_list(list);
+			free(list);
+		}
+	}
+
 	igt_subtest_group {
 		struct job_list *list = malloc(sizeof(*list));
 		volatile int dirfd = -1, subdirfd = -1, fd = -1;
@@ -1490,6 +1673,8 @@ igt_main
 			char dirname[] = "tmpdirXXXXXX";
 
 			igt_fixture {
+				/* This test checks that the stdout parsing for result without time data works, so use that */
+				setenv("IGT_RUNNER_DISABLE_SOCKET_COMMUNICATION", "1", 1);
 				igt_require(mkdtemp(dirname) != NULL);
 				rmdir(dirname);
 			}
@@ -1554,6 +1739,7 @@ igt_main
 				close(dirfd);
 				clear_directory(dirname);
 				free_job_list(list);
+				unsetenv("IGT_RUNNER_DISABLE_SOCKET_COMMUNICATION");
 			}
 		}
 
-- 
2.30.2

[igt-dev] [PATCH i-g-t v2 5/7] lib/tests: Add unit tests for socket communication packet creation

[Petri Latvala]

Test that packet fields get dumped and read correctly to the same
fields, and test that invalid packets get rejected.

v2: Flesh out commit message (Kamil)

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arek@hiler.eu>
Acked-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>
---
 lib/tests/igt_runnercomms_packets.c | 263 ++++++++++++++++++++++++++++
 lib/tests/meson.build               |   1 +
 2 files changed, 264 insertions(+)
 create mode 100644 lib/tests/igt_runnercomms_packets.c

diff --git a/lib/tests/igt_runnercomms_packets.c b/lib/tests/igt_runnercomms_packets.c
new file mode 100644
index 00000000..167ad8e6
--- /dev/null
+++ b/lib/tests/igt_runnercomms_packets.c
@@ -0,0 +1,263 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2022 Intel Corporation
+ */
+
+#include "runnercomms.h"
+
+#include "igt_core.h"
+
+static void igt_assert_eqstr(const char *one, const char *two)
+{
+	if (one == NULL && two == NULL)
+		return;
+
+	igt_assert_f(one != NULL && two != NULL, "Strings differ (one is NULL): %s vs %s\n", one, two);
+
+	igt_assert_f(!strcmp(one, two), "Strings differ: '%s' vs '%s'\n", one, two);
+}
+
+
+static const uint8_t num8 = 5;
+static const int32_t num32 = -67;
+static const char *text1 = "Text one";
+static const char *text2 = "Text two";
+static const char *text3 = "Text three";
+static const char *text4 = "Text four";
+
+static struct runnerpacket *create_log(void)
+{
+	return runnerpacket_log(num8, text1);
+}
+
+static void validate_log(struct runnerpacket *packet)
+{
+	runnerpacket_read_helper helper;
+
+	helper = read_runnerpacket(packet);
+
+	igt_assert_eq(packet->type, PACKETTYPE_LOG);
+	igt_assert_eq(helper.type, PACKETTYPE_LOG);
+
+	igt_assert_eq(helper.log.stream, num8);
+	igt_assert_eqstr(helper.log.text, text1);
+}
+
+static struct runnerpacket *create_exec(void)
+{
+	char *argv[] = { strdup(text1), strdup(text2), strdup(text3), strdup(text4), NULL };
+	struct runnerpacket *packet;
+
+	packet = runnerpacket_exec(argv);
+
+	free(argv[0]);
+	free(argv[1]);
+	free(argv[2]);
+	free(argv[3]);
+
+	return packet;
+}
+
+static void validate_exec(struct runnerpacket *packet)
+{
+	runnerpacket_read_helper helper;
+	char cmpstr[256];
+
+	helper = read_runnerpacket(packet);
+
+	igt_assert_eq(packet->type, PACKETTYPE_EXEC);
+	igt_assert_eq(helper.type, PACKETTYPE_EXEC);
+
+	snprintf(cmpstr, sizeof(cmpstr), "%s %s %s %s", text1, text2, text3, text4);
+	igt_assert_eqstr(helper.exec.cmdline, cmpstr);
+}
+
+static struct runnerpacket *create_exit(void)
+{
+	return runnerpacket_exit(num32, text1);
+}
+
+static void validate_exit(struct runnerpacket *packet)
+{
+	runnerpacket_read_helper helper;
+
+	helper = read_runnerpacket(packet);
+
+	igt_assert_eq(packet->type, PACKETTYPE_EXIT);
+	igt_assert_eq(helper.type, PACKETTYPE_EXIT);
+
+	igt_assert_eq(helper.exit.exitcode, num32);
+	igt_assert_eqstr(helper.exit.timeused, text1);
+}
+
+static struct runnerpacket *create_subtest_start(void)
+{
+	return runnerpacket_subtest_start(text1);
+}
+
+static void validate_subtest_start(struct runnerpacket *packet)
+{
+	runnerpacket_read_helper helper;
+
+	helper = read_runnerpacket(packet);
+
+	igt_assert_eq(packet->type, PACKETTYPE_SUBTEST_START);
+	igt_assert_eq(helper.type, PACKETTYPE_SUBTEST_START);
+
+	igt_assert_eqstr(helper.subteststart.name, text1);
+}
+
+static struct runnerpacket *create_subtest_result(void)
+{
+	return runnerpacket_subtest_result(text1, text2, text3, text4);
+}
+
+static void validate_subtest_result(struct runnerpacket *packet)
+{
+	runnerpacket_read_helper helper;
+
+	helper = read_runnerpacket(packet);
+
+	igt_assert_eq(packet->type, PACKETTYPE_SUBTEST_RESULT);
+	igt_assert_eq(helper.type, PACKETTYPE_SUBTEST_RESULT);
+
+	igt_assert_eqstr(helper.subtestresult.name, text1);
+	igt_assert_eqstr(helper.subtestresult.result, text2);
+	igt_assert_eqstr(helper.subtestresult.timeused, text3);
+	igt_assert_eqstr(helper.subtestresult.reason, text4);
+}
+
+static struct runnerpacket *create_dynamic_subtest_start(void)
+{
+	return runnerpacket_dynamic_subtest_start(text1);
+}
+
+static void validate_dynamic_subtest_start(struct runnerpacket *packet)
+{
+	runnerpacket_read_helper helper;
+
+	helper = read_runnerpacket(packet);
+
+	igt_assert_eq(packet->type, PACKETTYPE_DYNAMIC_SUBTEST_START);
+	igt_assert_eq(helper.type, PACKETTYPE_DYNAMIC_SUBTEST_START);
+
+	igt_assert_eqstr(helper.dynamicsubteststart.name, text1);
+}
+
+static struct runnerpacket *create_dynamic_subtest_result(void)
+{
+	return runnerpacket_dynamic_subtest_result(text1, text2, text3, text4);
+}
+
+static void validate_dynamic_subtest_result(struct runnerpacket *packet)
+{
+	runnerpacket_read_helper helper;
+
+	helper = read_runnerpacket(packet);
+
+	igt_assert_eq(packet->type, PACKETTYPE_DYNAMIC_SUBTEST_RESULT);
+	igt_assert_eq(helper.type, PACKETTYPE_DYNAMIC_SUBTEST_RESULT);
+
+	igt_assert_eqstr(helper.dynamicsubtestresult.name, text1);
+	igt_assert_eqstr(helper.dynamicsubtestresult.result, text2);
+	igt_assert_eqstr(helper.dynamicsubtestresult.timeused, text3);
+	igt_assert_eqstr(helper.dynamicsubtestresult.reason, text4);
+}
+
+static struct runnerpacket *create_versionstring(void)
+{
+	return runnerpacket_versionstring(text1);
+}
+
+static void validate_versionstring(struct runnerpacket *packet)
+{
+	runnerpacket_read_helper helper;
+
+	helper = read_runnerpacket(packet);
+
+	igt_assert_eq(packet->type, PACKETTYPE_VERSIONSTRING);
+	igt_assert_eq(helper.type, PACKETTYPE_VERSIONSTRING);
+
+	igt_assert_eqstr(helper.versionstring.text, text1);
+}
+
+static struct runnerpacket *create_result_override(void)
+{
+	return runnerpacket_resultoverride(text1);
+}
+
+static void validate_result_override(struct runnerpacket *packet)
+{
+	runnerpacket_read_helper helper;
+
+	helper = read_runnerpacket(packet);
+
+	igt_assert_eq(packet->type, PACKETTYPE_RESULT_OVERRIDE);
+	igt_assert_eq(helper.type, PACKETTYPE_RESULT_OVERRIDE);
+
+	igt_assert_eqstr(helper.resultoverride.result, text1);
+}
+
+struct {
+	struct runnerpacket * (*create)(void);
+	void (*validate)(struct runnerpacket *packet);
+} basic_creation[] = {
+		      { create_log, validate_log },
+		      { create_exec, validate_exec },
+		      { create_exit, validate_exit },
+		      { create_subtest_start, validate_subtest_start },
+		      { create_subtest_result, validate_subtest_result },
+		      { create_dynamic_subtest_start, validate_dynamic_subtest_start },
+		      { create_dynamic_subtest_result, validate_dynamic_subtest_result },
+		      { create_versionstring, validate_versionstring },
+		      { create_result_override, validate_result_override },
+		      { NULL, NULL }
+};
+
+igt_main
+{
+	igt_subtest("create-and-parse-normal") {
+		for (typeof (*basic_creation) *t = basic_creation; t->create; t++) {
+			struct runnerpacket *packet;
+
+			packet = t->create();
+			igt_assert(packet != NULL);
+			igt_assert(packet->type != PACKETTYPE_INVALID);
+			t->validate(packet);
+		}
+	}
+
+	igt_subtest("packet-too-short") {
+		struct runnerpacket *packet;
+		runnerpacket_read_helper helper;
+
+		packet = runnerpacket_log(1, "Hello");
+		igt_assert(packet != NULL);
+		igt_assert_eq(packet->type, PACKETTYPE_LOG);
+
+		packet->size = 4; /* not even sizeof(*packet) */
+		helper = read_runnerpacket(packet);
+		igt_assert_eq(helper.type, PACKETTYPE_INVALID);
+
+		free(packet);
+	}
+
+	igt_subtest("nul-termination-missing") {
+		/* Parsing should reject the packet when nul-termination is missing */
+		struct runnerpacket *packet;
+		runnerpacket_read_helper helper;
+
+		uint8_t num = 1;
+		const char *text = "This is text";
+		packet = runnerpacket_log(num, text);
+		igt_assert(packet != NULL);
+		igt_assert_eq(packet->type, PACKETTYPE_LOG);
+
+		/* make the packet too short to include the nul-termination in the string */
+		packet->size -= 2;
+		helper = read_runnerpacket(packet);
+		igt_assert_eq(helper.type, PACKETTYPE_INVALID);
+
+		free(packet);
+	}
+}
diff --git a/lib/tests/meson.build b/lib/tests/meson.build
index d5666c24..7a52a787 100644
--- a/lib/tests/meson.build
+++ b/lib/tests/meson.build
@@ -14,6 +14,7 @@ lib_tests = [
 	'igt_invalid_subtest_name',
 	'igt_nesting',
 	'igt_no_exit',
+	'igt_runnercomms_packets',
 	'igt_segfault',
 	'igt_simulation',
 	'igt_stats',
-- 
2.30.2

[igt-dev] [PATCH i-g-t v2 6/7] runner: Add a socket comms decoder

[Petri Latvala]

When debugging, being able to read what is sent via socket comms is
very useful.

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Cc: Arkadiusz Hiler <arek@hiler.eu>
Acked-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>
---
 runner/decoder.c   | 131 +++++++++++++++++++++++++++++++++++++++++++++
 runner/meson.build |   8 +++
 2 files changed, 139 insertions(+)
 create mode 100644 runner/decoder.c

diff --git a/runner/decoder.c b/runner/decoder.c
new file mode 100644
index 00000000..850b6ad5
--- /dev/null
+++ b/runner/decoder.c
@@ -0,0 +1,131 @@
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+
+#include "runnercomms.h"
+
+static bool handle_log(const struct runnerpacket *packet, runnerpacket_read_helper helper, void *userdata)
+{
+	printf("(pid=%d tid=%d) LOG\tstream=%d,text=%s",
+	       packet->senderpid, packet->sendertid,
+	       helper.log.stream, helper.log.text);
+	if (strlen(helper.log.text) == 0 || helper.log.text[strlen(helper.log.text) - 1] != '\n')
+		printf("\n");
+
+	return true;
+}
+
+static bool handle_exec(const struct runnerpacket *packet, runnerpacket_read_helper helper, void *userdata)
+{
+	printf("(pid=%d tid=%d) EXEC\tcmdline=%s\n",
+	       packet->senderpid, packet->sendertid,
+	       helper.exec.cmdline);
+
+	return true;
+}
+
+static bool handle_exit(const struct runnerpacket *packet, runnerpacket_read_helper helper, void *userdata)
+{
+	printf("(pid=%d tid=%d) EXIT\texitcode=%d,timeused=%s\n",
+	       packet->senderpid, packet->sendertid,
+	       helper.exit.exitcode, helper.exit.timeused);
+
+	return true;
+}
+
+static bool handle_subtest_start(const struct runnerpacket *packet, runnerpacket_read_helper helper, void *userdata)
+{
+	printf("(pid=%d tid=%d) SUBTEST_START\tname=%s\n",
+	       packet->senderpid, packet->sendertid,
+	       helper.subteststart.name);
+
+	return true;
+}
+
+static bool handle_subtest_result(const struct runnerpacket *packet, runnerpacket_read_helper helper, void *userdata)
+{
+	printf("(pid=%d tid=%d) SUBTEST_RESULT\tname=%s,result=%s,timeused=%s,reason=%s\n",
+	       packet->senderpid, packet->sendertid,
+	       helper.subtestresult.name,
+	       helper.subtestresult.result,
+	       helper.subtestresult.timeused,
+	       helper.subtestresult.reason ?: "<null>");
+
+	return true;
+}
+
+static bool handle_dynamic_subtest_start(const struct runnerpacket *packet, runnerpacket_read_helper helper, void *userdata)
+{
+	printf("(pid=%d tid=%d) DYNAMIC_SUBTEST_START\tname=%s\n",
+	       packet->senderpid, packet->sendertid,
+	       helper.dynamicsubteststart.name);
+
+	return true;
+}
+
+static bool handle_dynamic_subtest_result(const struct runnerpacket *packet, runnerpacket_read_helper helper, void *userdata)
+{
+	printf("(pid=%d tid=%d) DYNAMIC_SUBTEST_RESULT\tname=%s,result=%s,timeused=%s,reason=%s\n",
+	       packet->senderpid, packet->sendertid,
+	       helper.dynamicsubtestresult.name,
+	       helper.dynamicsubtestresult.result,
+	       helper.dynamicsubtestresult.timeused,
+	       helper.dynamicsubtestresult.reason ?: "<null>");
+
+	return true;
+}
+
+static bool handle_versionstring(const struct runnerpacket *packet, runnerpacket_read_helper helper, void *userdata)
+{
+	printf("(pid=%d tid=%d) VERSIONSTRING\ttext=%s",
+	       packet->senderpid, packet->sendertid,
+	       helper.versionstring.text);
+	if (strlen(helper.versionstring.text) == 0 || helper.versionstring.text[strlen(helper.versionstring.text) - 1] != '\n')
+		printf("\n");
+
+	return true;
+}
+
+static bool handle_result_override(const struct runnerpacket *packet, runnerpacket_read_helper helper, void *userdata)
+{
+	printf("pid=%d tid=%d) RESULT_OVERRIDE\tresult=%s\n",
+	       packet->senderpid, packet->sendertid,
+	       helper.resultoverride.result);
+
+	return true;
+}
+
+struct comms_visitor logger = {
+	.log = handle_log,
+	.exec = handle_exec,
+	.exit = handle_exit,
+	.subtest_start = handle_subtest_start,
+	.subtest_result = handle_subtest_result,
+	.dynamic_subtest_start = handle_dynamic_subtest_start,
+	.dynamic_subtest_result = handle_dynamic_subtest_result,
+	.versionstring = handle_versionstring,
+	.result_override = handle_result_override,
+};
+
+int main(int argc, char **argv)
+{
+	int fd;
+
+	if (argc < 2) {
+		printf("Usage: %s igt-comms-data-file\n", argv[0]);
+		return 2;
+	}
+
+	fd = open(argv[1], O_RDONLY);
+	if (fd < 0) {
+		fprintf(stderr, "Failure opening %s: %m\n", argv[1]);
+		return 1;
+	}
+
+	comms_read_dump(fd, &logger);
+
+	return 0;
+}
diff --git a/runner/meson.build b/runner/meson.build
index c3927af5..dadfc75f 100644
--- a/runner/meson.build
+++ b/runner/meson.build
@@ -10,6 +10,7 @@ runnerlib_sources = [ 'settings.c',
 runner_sources = [ 'runner.c' ]
 resume_sources = [ 'resume.c' ]
 results_sources = [ 'results.c' ]
+decoder_sources = [ 'decoder.c' ]
 runner_test_sources = [ 'runner_tests.c' ]
 runner_json_test_sources = [ 'runner_json_tests.c' ]
 
@@ -56,6 +57,13 @@ if jsonc.found()
 			     install_rpath : bindir_rpathdir,
 			     dependencies : igt_deps)
 
+	decoder = executable('igt_comms_decoder', decoder_sources,
+			     link_with : runnerlib,
+			     install : true,
+			     install_dir : bindir,
+			     install_rpath : bindir_rpathdir,
+			     dependencies : igt_deps)
+
 	runner_test = executable('runner_test', runner_test_sources,
 				 c_args : '-DTESTDATA_DIRECTORY="@0@"'.format(testdata_dir),
 				 link_with : runnerlib,
-- 
2.30.2

[igt-dev] [PATCH i-g-t v2 7/7] runner: Disable socket communications for now

[Petri Latvala]

Currently, having i915 CI systems test the socket communications when
testing is impossible. If a test hangs completely, the incomplete set
of results are processed into results.json with an older IGT version
from a previous post-merge build, which is unable to parse socket
communications. Disable them for now.

This commit will get reverted after suitable testing has been possible
to execute.

v2: Rename the enable var, explain its presence further in a comment
    (Kamil)

Signed-off-by: Petri Latvala <petri.latvala@intel.com>
Acked-by: Kamil Konieczny <kamil.konieczny@linux.intel.com>
---
 runner/executor.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/runner/executor.c b/runner/executor.c
index 669e8524..a79a34e0 100644
--- a/runner/executor.c
+++ b/runner/executor.c
@@ -1659,7 +1659,14 @@ static int execute_next_entry(struct execute_state *state,
 
 		sigprocmask(SIG_UNBLOCK, sigmask, NULL);
 
-		if (socketfd >= 0 && !getenv("IGT_RUNNER_DISABLE_SOCKET_COMMUNICATION")) {
+		if (socketfd >= 0 && !getenv("IGT_RUNNER_DISABLE_SOCKET_COMMUNICATION") &&
+		    getenv("IGT_RUNNER_ENABLE_SOCKET_COMMUNICATION_FIXME")) {
+			/*
+			 * The variable for enabling socket comms is
+			 * temporary. Do not use unless you really
+			 * know what you're doing. Requiring it will
+			 * go away later.
+			 */
 			snprintf(envstring, sizeof(envstring), "%d", socketfd);
 			setenv("IGT_RUNNER_SOCKET_FD", envstring, 1);
 		}
-- 
2.30.2

[igt-dev] ✓ Fi.CI.BAT: success for series starting with [i-g-t,v2,1/7] lib/runnercomms: Structured communication from tests to igt_runner

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 5573 bytes --]

== Series Details ==

Series: series starting with [i-g-t,v2,1/7] lib/runnercomms: Structured communication from tests to igt_runner
URL   : https://patchwork.freedesktop.org/series/110312/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_12323 -> IGTPW_8014
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/index.html

Participating hosts (41 -> 38)
------------------------------

  Missing    (3): fi-hsw-4770 fi-bdw-gvtdvm fi-icl-u2 

Known issues
------------

  Here are the changes found in IGTPW_8014 that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@gem_lmem_swapping@basic:
    - fi-apl-guc:         NOTRUN -> [SKIP][1] ([fdo#109271] / [i915#4613]) +3 similar issues
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/fi-apl-guc/igt@gem_lmem_swapping@basic.html

  * igt@kms_chamelium@common-hpd-after-suspend:
    - fi-rkl-11600:       NOTRUN -> [SKIP][2] ([fdo#111827])
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/fi-rkl-11600/igt@kms_chamelium@common-hpd-after-suspend.html
    - fi-rkl-guc:         NOTRUN -> [SKIP][3] ([fdo#111827])
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/fi-rkl-guc/igt@kms_chamelium@common-hpd-after-suspend.html

  * igt@kms_chamelium@hdmi-crc-fast:
    - fi-apl-guc:         NOTRUN -> [SKIP][4] ([fdo#109271] / [fdo#111827]) +8 similar issues
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/fi-apl-guc/igt@kms_chamelium@hdmi-crc-fast.html

  * igt@kms_psr@sprite_plane_onoff:
    - fi-apl-guc:         NOTRUN -> [SKIP][5] ([fdo#109271]) +11 similar issues
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/fi-apl-guc/igt@kms_psr@sprite_plane_onoff.html

  
#### Possible fixes ####

  * igt@gem_render_tiled_blits@basic:
    - fi-apl-guc:         [INCOMPLETE][6] ([i915#7056]) -> [PASS][7]
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/fi-apl-guc/igt@gem_render_tiled_blits@basic.html
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/fi-apl-guc/igt@gem_render_tiled_blits@basic.html

  * igt@i915_selftest@live@hangcheck:
    - {bat-dg2-9}:        [INCOMPLETE][8] ([i915#7349]) -> [PASS][9]
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/bat-dg2-9/igt@i915_selftest@live@hangcheck.html
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/bat-dg2-9/igt@i915_selftest@live@hangcheck.html

  * igt@i915_selftest@live@requests:
    - {bat-rpls-1}:       [INCOMPLETE][10] ([i915#6257]) -> [PASS][11]
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/bat-rpls-1/igt@i915_selftest@live@requests.html
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/bat-rpls-1/igt@i915_selftest@live@requests.html

  * igt@i915_selftest@live@workarounds:
    - {bat-adlm-1}:       [INCOMPLETE][12] -> [PASS][13]
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/bat-adlm-1/igt@i915_selftest@live@workarounds.html
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/bat-adlm-1/igt@i915_selftest@live@workarounds.html

  * igt@i915_suspend@basic-s3-without-i915:
    - fi-rkl-11600:       [INCOMPLETE][14] ([i915#4817]) -> [PASS][15]
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/fi-rkl-11600/igt@i915_suspend@basic-s3-without-i915.html
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/fi-rkl-11600/igt@i915_suspend@basic-s3-without-i915.html

  * igt@kms_pipe_crc_basic@suspend-read-crc@pipe-a-edp-1:
    - fi-cml-u2:          [INCOMPLETE][16] -> [PASS][17]
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/fi-cml-u2/igt@kms_pipe_crc_basic@suspend-read-crc@pipe-a-edp-1.html
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/fi-cml-u2/igt@kms_pipe_crc_basic@suspend-read-crc@pipe-a-edp-1.html

  
  {name}: This element is suppressed. This means it is ignored when computing
          the status of the difference (SUCCESS, WARNING, or FAILURE).

  [fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
  [fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
  [i915#1845]: https://gitlab.freedesktop.org/drm/intel/issues/1845
  [i915#2867]: https://gitlab.freedesktop.org/drm/intel/issues/2867
  [i915#4258]: https://gitlab.freedesktop.org/drm/intel/issues/4258
  [i915#4312]: https://gitlab.freedesktop.org/drm/intel/issues/4312
  [i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613
  [i915#4817]: https://gitlab.freedesktop.org/drm/intel/issues/4817
  [i915#4983]: https://gitlab.freedesktop.org/drm/intel/issues/4983
  [i915#5278]: https://gitlab.freedesktop.org/drm/intel/issues/5278
  [i915#6257]: https://gitlab.freedesktop.org/drm/intel/issues/6257
  [i915#7029]: https://gitlab.freedesktop.org/drm/intel/issues/7029
  [i915#7056]: https://gitlab.freedesktop.org/drm/intel/issues/7056
  [i915#7349]: https://gitlab.freedesktop.org/drm/intel/issues/7349


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_7031 -> IGTPW_8014

  CI-20190529: 20190529
  CI_DRM_12323: 529744257b96535c9a0b7da5c4ede40d1f69a016 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_8014: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/index.html
  IGT_7031: 54abd9445f732b55556d4e612d24f50803515904 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git


Testlist changes
----------------

-igt@debugfs_test@basic-hwmon

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/index.html

[-- Attachment #2: Type: text/html, Size: 6233 bytes --]

[igt-dev] ✓ Fi.CI.IGT: success for series starting with [i-g-t,v2,1/7] lib/runnercomms: Structured communication from tests to igt_runner

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 30185 bytes --]

== Series Details ==

Series: series starting with [i-g-t,v2,1/7] lib/runnercomms: Structured communication from tests to igt_runner
URL   : https://patchwork.freedesktop.org/series/110312/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_12323_full -> IGTPW_8014_full
====================================================

Summary
-------

  **SUCCESS**

  No regressions found.

  External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/index.html

Participating hosts (9 -> 6)
------------------------------

  Missing    (3): pig-skl-6260u pig-kbl-iris pig-glk-j5005 

Known issues
------------

  Here are the changes found in IGTPW_8014_full that come from known issues:

### IGT changes ###

#### Issues hit ####

  * igt@gem_ctx_persistence@legacy-engines-mixed:
    - shard-snb:          NOTRUN -> [SKIP][1] ([fdo#109271] / [i915#1099])
   [1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-snb2/igt@gem_ctx_persistence@legacy-engines-mixed.html

  * igt@gem_exec_balancer@parallel-bb-first:
    - shard-iclb:         [PASS][2] -> [SKIP][3] ([i915#4525])
   [2]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-iclb2/igt@gem_exec_balancer@parallel-bb-first.html
   [3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb5/igt@gem_exec_balancer@parallel-bb-first.html

  * igt@gem_exec_balancer@parallel-keep-submit-fence:
    - shard-iclb:         NOTRUN -> [SKIP][4] ([i915#4525])
   [4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb7/igt@gem_exec_balancer@parallel-keep-submit-fence.html

  * igt@gem_exec_fair@basic-flow@rcs0:
    - shard-tglb:         [PASS][5] -> [FAIL][6] ([i915#2842])
   [5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-tglb6/igt@gem_exec_fair@basic-flow@rcs0.html
   [6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb3/igt@gem_exec_fair@basic-flow@rcs0.html

  * igt@gem_exec_params@no-blt:
    - shard-iclb:         NOTRUN -> [SKIP][7] ([fdo#109283])
   [7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb6/igt@gem_exec_params@no-blt.html
    - shard-tglb:         NOTRUN -> [SKIP][8] ([fdo#109283])
   [8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb1/igt@gem_exec_params@no-blt.html

  * igt@gem_lmem_swapping@massive:
    - shard-iclb:         NOTRUN -> [SKIP][9] ([i915#4613]) +1 similar issue
   [9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb8/igt@gem_lmem_swapping@massive.html
    - shard-apl:          NOTRUN -> [SKIP][10] ([fdo#109271] / [i915#4613]) +1 similar issue
   [10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-apl8/igt@gem_lmem_swapping@massive.html
    - shard-tglb:         NOTRUN -> [SKIP][11] ([i915#4613]) +1 similar issue
   [11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb5/igt@gem_lmem_swapping@massive.html

  * igt@gem_lmem_swapping@random-engines:
    - shard-glk:          NOTRUN -> [SKIP][12] ([fdo#109271] / [i915#4613]) +1 similar issue
   [12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-glk1/igt@gem_lmem_swapping@random-engines.html

  * igt@gem_pxp@reject-modify-context-protection-on:
    - shard-tglb:         NOTRUN -> [SKIP][13] ([i915#4270])
   [13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb1/igt@gem_pxp@reject-modify-context-protection-on.html
    - shard-iclb:         NOTRUN -> [SKIP][14] ([i915#4270])
   [14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb6/igt@gem_pxp@reject-modify-context-protection-on.html

  * igt@gem_render_copy@y-tiled-ccs-to-y-tiled-mc-ccs:
    - shard-glk:          NOTRUN -> [SKIP][15] ([fdo#109271]) +32 similar issues
   [15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-glk8/igt@gem_render_copy@y-tiled-ccs-to-y-tiled-mc-ccs.html
    - shard-iclb:         NOTRUN -> [SKIP][16] ([i915#768])
   [16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb7/igt@gem_render_copy@y-tiled-ccs-to-y-tiled-mc-ccs.html

  * igt@gen9_exec_parse@basic-rejected:
    - shard-tglb:         NOTRUN -> [SKIP][17] ([i915#2527] / [i915#2856]) +1 similar issue
   [17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb1/igt@gen9_exec_parse@basic-rejected.html

  * igt@gen9_exec_parse@unaligned-jump:
    - shard-iclb:         NOTRUN -> [SKIP][18] ([i915#2856]) +1 similar issue
   [18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb5/igt@gen9_exec_parse@unaligned-jump.html

  * igt@i915_pm_rps@engine-order:
    - shard-apl:          [PASS][19] -> [FAIL][20] ([i915#6537])
   [19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-apl1/igt@i915_pm_rps@engine-order.html
   [20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-apl2/igt@i915_pm_rps@engine-order.html

  * igt@kms_async_flips@alternate-sync-async-flip@pipe-a-edp-1:
    - shard-tglb:         [PASS][21] -> [FAIL][22] ([i915#2521])
   [21]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-tglb2/igt@kms_async_flips@alternate-sync-async-flip@pipe-a-edp-1.html
   [22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb1/igt@kms_async_flips@alternate-sync-async-flip@pipe-a-edp-1.html

  * igt@kms_big_fb@yf-tiled-64bpp-rotate-90:
    - shard-tglb:         NOTRUN -> [SKIP][23] ([fdo#111615])
   [23]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb1/igt@kms_big_fb@yf-tiled-64bpp-rotate-90.html
    - shard-iclb:         NOTRUN -> [SKIP][24] ([fdo#110723])
   [24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb6/igt@kms_big_fb@yf-tiled-64bpp-rotate-90.html

  * igt@kms_big_joiner@basic:
    - shard-iclb:         NOTRUN -> [SKIP][25] ([i915#2705])
   [25]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb8/igt@kms_big_joiner@basic.html
    - shard-tglb:         NOTRUN -> [SKIP][26] ([i915#2705])
   [26]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb2/igt@kms_big_joiner@basic.html

  * igt@kms_ccs@pipe-a-bad-rotation-90-4_tiled_dg2_rc_ccs_cc:
    - shard-tglb:         NOTRUN -> [SKIP][27] ([i915#6095]) +1 similar issue
   [27]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb6/igt@kms_ccs@pipe-a-bad-rotation-90-4_tiled_dg2_rc_ccs_cc.html

  * igt@kms_ccs@pipe-a-random-ccs-data-y_tiled_gen12_mc_ccs:
    - shard-iclb:         NOTRUN -> [SKIP][28] ([fdo#109278] / [i915#3886]) +3 similar issues
   [28]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb2/igt@kms_ccs@pipe-a-random-ccs-data-y_tiled_gen12_mc_ccs.html

  * igt@kms_ccs@pipe-c-bad-pixel-format-y_tiled_gen12_mc_ccs:
    - shard-apl:          NOTRUN -> [SKIP][29] ([fdo#109271] / [i915#3886]) +4 similar issues
   [29]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-apl3/igt@kms_ccs@pipe-c-bad-pixel-format-y_tiled_gen12_mc_ccs.html

  * igt@kms_ccs@pipe-c-crc-primary-basic-y_tiled_gen12_rc_ccs:
    - shard-iclb:         NOTRUN -> [SKIP][30] ([fdo#109278]) +7 similar issues
   [30]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb6/igt@kms_ccs@pipe-c-crc-primary-basic-y_tiled_gen12_rc_ccs.html

  * igt@kms_ccs@pipe-c-crc-sprite-planes-basic-y_tiled_gen12_mc_ccs:
    - shard-tglb:         NOTRUN -> [SKIP][31] ([i915#3689] / [i915#3886]) +2 similar issues
   [31]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb6/igt@kms_ccs@pipe-c-crc-sprite-planes-basic-y_tiled_gen12_mc_ccs.html
    - shard-glk:          NOTRUN -> [SKIP][32] ([fdo#109271] / [i915#3886]) +3 similar issues
   [32]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-glk1/igt@kms_ccs@pipe-c-crc-sprite-planes-basic-y_tiled_gen12_mc_ccs.html

  * igt@kms_ccs@pipe-d-missing-ccs-buffer-y_tiled_gen12_mc_ccs:
    - shard-tglb:         NOTRUN -> [SKIP][33] ([i915#3689]) +1 similar issue
   [33]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb7/igt@kms_ccs@pipe-d-missing-ccs-buffer-y_tiled_gen12_mc_ccs.html

  * igt@kms_chamelium@dp-crc-single:
    - shard-snb:          NOTRUN -> [SKIP][34] ([fdo#109271] / [fdo#111827]) +2 similar issues
   [34]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-snb4/igt@kms_chamelium@dp-crc-single.html
    - shard-apl:          NOTRUN -> [SKIP][35] ([fdo#109271] / [fdo#111827]) +2 similar issues
   [35]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-apl2/igt@kms_chamelium@dp-crc-single.html
    - shard-tglb:         NOTRUN -> [SKIP][36] ([fdo#109284] / [fdo#111827]) +2 similar issues
   [36]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb7/igt@kms_chamelium@dp-crc-single.html
    - shard-glk:          NOTRUN -> [SKIP][37] ([fdo#109271] / [fdo#111827]) +2 similar issues
   [37]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-glk6/igt@kms_chamelium@dp-crc-single.html

  * igt@kms_color_chamelium@degamma:
    - shard-iclb:         NOTRUN -> [SKIP][38] ([fdo#109284] / [fdo#111827]) +2 similar issues
   [38]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb1/igt@kms_color_chamelium@degamma.html

  * igt@kms_cursor_crc@cursor-rapid-movement-32x10:
    - shard-tglb:         NOTRUN -> [SKIP][39] ([i915#3555])
   [39]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb8/igt@kms_cursor_crc@cursor-rapid-movement-32x10.html

  * igt@kms_cursor_legacy@flip-vs-cursor@atomic-transitions-varying-size:
    - shard-glk:          [PASS][40] -> [FAIL][41] ([i915#2346]) +1 similar issue
   [40]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-glk5/igt@kms_cursor_legacy@flip-vs-cursor@atomic-transitions-varying-size.html
   [41]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-glk3/igt@kms_cursor_legacy@flip-vs-cursor@atomic-transitions-varying-size.html

  * igt@kms_flip@2x-blocking-wf_vblank:
    - shard-iclb:         NOTRUN -> [SKIP][42] ([fdo#109274]) +1 similar issue
   [42]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb8/igt@kms_flip@2x-blocking-wf_vblank.html
    - shard-tglb:         NOTRUN -> [SKIP][43] ([fdo#109274] / [fdo#111825] / [i915#3637]) +1 similar issue
   [43]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb5/igt@kms_flip@2x-blocking-wf_vblank.html

  * igt@kms_flip_scaled_crc@flip-32bpp-4tile-to-64bpp-4tile-upscaling@pipe-a-valid-mode:
    - shard-tglb:         NOTRUN -> [SKIP][44] ([i915#2587] / [i915#2672])
   [44]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb8/igt@kms_flip_scaled_crc@flip-32bpp-4tile-to-64bpp-4tile-upscaling@pipe-a-valid-mode.html

  * igt@kms_flip_scaled_crc@flip-32bpp-yftile-to-64bpp-yftile-downscaling@pipe-a-default-mode:
    - shard-iclb:         NOTRUN -> [SKIP][45] ([i915#2672]) +6 similar issues
   [45]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb2/igt@kms_flip_scaled_crc@flip-32bpp-yftile-to-64bpp-yftile-downscaling@pipe-a-default-mode.html

  * igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-32bpp-ytileccs-downscaling@pipe-a-default-mode:
    - shard-iclb:         NOTRUN -> [SKIP][46] ([i915#3555]) +3 similar issues
   [46]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb2/igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-32bpp-ytileccs-downscaling@pipe-a-default-mode.html

  * igt@kms_flip_scaled_crc@flip-64bpp-4tile-to-32bpp-4tiledg2rcccs-downscaling@pipe-a-valid-mode:
    - shard-iclb:         NOTRUN -> [SKIP][47] ([i915#2587] / [i915#2672]) +2 similar issues
   [47]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb1/igt@kms_flip_scaled_crc@flip-64bpp-4tile-to-32bpp-4tiledg2rcccs-downscaling@pipe-a-valid-mode.html

  * igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling@pipe-a-default-mode:
    - shard-iclb:         NOTRUN -> [SKIP][48] ([i915#2672] / [i915#3555])
   [48]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb3/igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-32bpp-ytilercccs-downscaling@pipe-a-default-mode.html

  * igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-indfb-pgflip-blt:
    - shard-iclb:         NOTRUN -> [SKIP][49] ([fdo#109280]) +5 similar issues
   [49]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb5/igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-indfb-pgflip-blt.html

  * igt@kms_frontbuffer_tracking@fbcpsr-2p-scndscrn-cur-indfb-draw-blt:
    - shard-tglb:         NOTRUN -> [SKIP][50] ([fdo#109280] / [fdo#111825]) +5 similar issues
   [50]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb5/igt@kms_frontbuffer_tracking@fbcpsr-2p-scndscrn-cur-indfb-draw-blt.html

  * igt@kms_frontbuffer_tracking@fbcpsr-rgb101010-draw-render:
    - shard-tglb:         NOTRUN -> [SKIP][51] ([i915#6497]) +2 similar issues
   [51]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb6/igt@kms_frontbuffer_tracking@fbcpsr-rgb101010-draw-render.html

  * igt@kms_plane_lowres@tiling-x@pipe-b-edp-1:
    - shard-iclb:         NOTRUN -> [SKIP][52] ([i915#3536]) +2 similar issues
   [52]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb2/igt@kms_plane_lowres@tiling-x@pipe-b-edp-1.html

  * igt@kms_plane_lowres@tiling-x@pipe-c-edp-1:
    - shard-tglb:         NOTRUN -> [SKIP][53] ([i915#3536]) +3 similar issues
   [53]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb3/igt@kms_plane_lowres@tiling-x@pipe-c-edp-1.html

  * igt@kms_plane_scaling@plane-downscale-with-modifiers-factor-0-25@pipe-b-edp-1:
    - shard-iclb:         NOTRUN -> [SKIP][54] ([i915#5176]) +2 similar issues
   [54]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb7/igt@kms_plane_scaling@plane-downscale-with-modifiers-factor-0-25@pipe-b-edp-1.html

  * igt@kms_plane_scaling@plane-downscale-with-modifiers-factor-0-25@pipe-c-dp-1:
    - shard-apl:          NOTRUN -> [SKIP][55] ([fdo#109271]) +45 similar issues
   [55]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-apl8/igt@kms_plane_scaling@plane-downscale-with-modifiers-factor-0-25@pipe-c-dp-1.html

  * igt@kms_plane_scaling@plane-downscale-with-modifiers-factor-0-25@pipe-c-edp-1:
    - shard-tglb:         NOTRUN -> [SKIP][56] ([i915#5176]) +3 similar issues
   [56]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb7/igt@kms_plane_scaling@plane-downscale-with-modifiers-factor-0-25@pipe-c-edp-1.html

  * igt@kms_plane_scaling@plane-scaler-with-clipping-clamping-pixel-formats@pipe-b-edp-1:
    - shard-iclb:         [PASS][57] -> [SKIP][58] ([i915#5176]) +1 similar issue
   [57]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-iclb6/igt@kms_plane_scaling@plane-scaler-with-clipping-clamping-pixel-formats@pipe-b-edp-1.html
   [58]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb3/igt@kms_plane_scaling@plane-scaler-with-clipping-clamping-pixel-formats@pipe-b-edp-1.html

  * igt@kms_psr2_sf@cursor-plane-move-continuous-exceed-sf:
    - shard-tglb:         NOTRUN -> [SKIP][59] ([i915#2920]) +1 similar issue
   [59]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb1/igt@kms_psr2_sf@cursor-plane-move-continuous-exceed-sf.html
    - shard-glk:          NOTRUN -> [SKIP][60] ([fdo#109271] / [i915#658]) +1 similar issue
   [60]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-glk5/igt@kms_psr2_sf@cursor-plane-move-continuous-exceed-sf.html
    - shard-iclb:         NOTRUN -> [SKIP][61] ([i915#658])
   [61]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb6/igt@kms_psr2_sf@cursor-plane-move-continuous-exceed-sf.html

  * igt@kms_psr2_sf@plane-move-sf-dmg-area:
    - shard-apl:          NOTRUN -> [SKIP][62] ([fdo#109271] / [i915#658]) +1 similar issue
   [62]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-apl8/igt@kms_psr2_sf@plane-move-sf-dmg-area.html
    - shard-iclb:         NOTRUN -> [SKIP][63] ([fdo#111068] / [i915#658])
   [63]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb1/igt@kms_psr2_sf@plane-move-sf-dmg-area.html

  * igt@kms_psr2_su@page_flip-nv12@pipe-b-edp-1:
    - shard-iclb:         NOTRUN -> [FAIL][64] ([i915#5939]) +2 similar issues
   [64]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb2/igt@kms_psr2_su@page_flip-nv12@pipe-b-edp-1.html

  * igt@kms_psr@psr2_cursor_plane_onoff:
    - shard-iclb:         [PASS][65] -> [SKIP][66] ([fdo#109441]) +2 similar issues
   [65]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-iclb2/igt@kms_psr@psr2_cursor_plane_onoff.html
   [66]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb5/igt@kms_psr@psr2_cursor_plane_onoff.html

  * igt@kms_psr_stress_test@flip-primary-invalidate-overlay:
    - shard-iclb:         [PASS][67] -> [SKIP][68] ([i915#5519])
   [67]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-iclb7/igt@kms_psr_stress_test@flip-primary-invalidate-overlay.html
   [68]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb5/igt@kms_psr_stress_test@flip-primary-invalidate-overlay.html

  * igt@kms_vblank@pipe-b-ts-continuation-suspend:
    - shard-apl:          [PASS][69] -> [DMESG-WARN][70] ([i915#180])
   [69]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-apl1/igt@kms_vblank@pipe-b-ts-continuation-suspend.html
   [70]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-apl6/igt@kms_vblank@pipe-b-ts-continuation-suspend.html

  * igt@perf@per-context-mode-unprivileged:
    - shard-tglb:         NOTRUN -> [SKIP][71] ([fdo#109289])
   [71]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb1/igt@perf@per-context-mode-unprivileged.html
    - shard-iclb:         NOTRUN -> [SKIP][72] ([fdo#109289])
   [72]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb6/igt@perf@per-context-mode-unprivileged.html

  * igt@sysfs_clients@sema-50:
    - shard-snb:          NOTRUN -> [SKIP][73] ([fdo#109271]) +68 similar issues
   [73]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-snb4/igt@sysfs_clients@sema-50.html
    - shard-apl:          NOTRUN -> [SKIP][74] ([fdo#109271] / [i915#2994])
   [74]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-apl6/igt@sysfs_clients@sema-50.html
    - shard-tglb:         NOTRUN -> [SKIP][75] ([i915#2994])
   [75]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb1/igt@sysfs_clients@sema-50.html
    - shard-glk:          NOTRUN -> [SKIP][76] ([fdo#109271] / [i915#2994])
   [76]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-glk5/igt@sysfs_clients@sema-50.html
    - shard-iclb:         NOTRUN -> [SKIP][77] ([i915#2994])
   [77]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb6/igt@sysfs_clients@sema-50.html

  
#### Possible fixes ####

  * igt@gem_exec_balancer@parallel:
    - shard-iclb:         [SKIP][78] ([i915#4525]) -> [PASS][79]
   [78]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-iclb6/igt@gem_exec_balancer@parallel.html
   [79]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb2/igt@gem_exec_balancer@parallel.html

  * igt@gem_exec_fair@basic-none-share@rcs0:
    - shard-tglb:         [FAIL][80] ([i915#2842]) -> [PASS][81]
   [80]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-tglb5/igt@gem_exec_fair@basic-none-share@rcs0.html
   [81]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb3/igt@gem_exec_fair@basic-none-share@rcs0.html

  * igt@gem_exec_fair@basic-throttle@rcs0:
    - shard-glk:          [FAIL][82] ([i915#2842]) -> [PASS][83] +4 similar issues
   [82]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-glk2/igt@gem_exec_fair@basic-throttle@rcs0.html
   [83]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-glk8/igt@gem_exec_fair@basic-throttle@rcs0.html

  * igt@gem_huc_copy@huc-copy:
    - shard-tglb:         [SKIP][84] ([i915#2190]) -> [PASS][85]
   [84]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-tglb7/igt@gem_huc_copy@huc-copy.html
   [85]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb1/igt@gem_huc_copy@huc-copy.html

  * igt@gem_softpin@evict-single-offset:
    - shard-tglb:         [FAIL][86] ([i915#4171]) -> [PASS][87]
   [86]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-tglb1/igt@gem_softpin@evict-single-offset.html
   [87]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb1/igt@gem_softpin@evict-single-offset.html

  * igt@i915_pm_dc@dc6-dpms:
    - shard-iclb:         [FAIL][88] ([i915#3989] / [i915#454]) -> [PASS][89]
   [88]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-iclb3/igt@i915_pm_dc@dc6-dpms.html
   [89]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb6/igt@i915_pm_dc@dc6-dpms.html

  * igt@i915_selftest@live@hangcheck:
    - shard-tglb:         [DMESG-WARN][90] ([i915#5591]) -> [PASS][91]
   [90]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-tglb5/igt@i915_selftest@live@hangcheck.html
   [91]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb2/igt@i915_selftest@live@hangcheck.html

  * igt@kms_big_fb@linear-16bpp-rotate-0:
    - shard-iclb:         [FAIL][92] ([i915#5138]) -> [PASS][93]
   [92]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-iclb8/igt@kms_big_fb@linear-16bpp-rotate-0.html
   [93]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb5/igt@kms_big_fb@linear-16bpp-rotate-0.html

  * igt@kms_flip@flip-vs-expired-vblank-interruptible@a-hdmi-a2:
    - shard-glk:          [FAIL][94] ([i915#79]) -> [PASS][95]
   [94]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-glk6/igt@kms_flip@flip-vs-expired-vblank-interruptible@a-hdmi-a2.html
   [95]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-glk9/igt@kms_flip@flip-vs-expired-vblank-interruptible@a-hdmi-a2.html

  * igt@kms_flip@flip-vs-suspend-interruptible@a-dp1:
    - shard-apl:          [DMESG-WARN][96] ([i915#180]) -> [PASS][97]
   [96]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-apl8/igt@kms_flip@flip-vs-suspend-interruptible@a-dp1.html
   [97]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-apl3/igt@kms_flip@flip-vs-suspend-interruptible@a-dp1.html

  * igt@kms_frontbuffer_tracking@fbc-1p-primscrn-spr-indfb-move:
    - shard-snb:          [SKIP][98] ([fdo#109271]) -> [PASS][99] +1 similar issue
   [98]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-snb2/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-spr-indfb-move.html
   [99]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-snb6/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-spr-indfb-move.html

  * igt@kms_psr@psr2_cursor_mmap_gtt:
    - shard-iclb:         [SKIP][100] ([fdo#109441]) -> [PASS][101] +1 similar issue
   [100]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-iclb8/igt@kms_psr@psr2_cursor_mmap_gtt.html
   [101]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb2/igt@kms_psr@psr2_cursor_mmap_gtt.html

  * igt@kms_psr_stress_test@flip-primary-invalidate-overlay:
    - shard-tglb:         [SKIP][102] ([i915#5519]) -> [PASS][103]
   [102]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-tglb2/igt@kms_psr_stress_test@flip-primary-invalidate-overlay.html
   [103]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb6/igt@kms_psr_stress_test@flip-primary-invalidate-overlay.html

  
#### Warnings ####

  * igt@gem_pread@exhaustion:
    - shard-apl:          [INCOMPLETE][104] ([i915#7248]) -> [WARN][105] ([i915#2658]) +1 similar issue
   [104]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-apl1/igt@gem_pread@exhaustion.html
   [105]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-apl6/igt@gem_pread@exhaustion.html

  * igt@i915_pm_rc6_residency@rc6-idle@bcs0:
    - shard-iclb:         [WARN][106] ([i915#2684]) -> [FAIL][107] ([i915#2684])
   [106]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-iclb1/igt@i915_pm_rc6_residency@rc6-idle@bcs0.html
   [107]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb8/igt@i915_pm_rc6_residency@rc6-idle@bcs0.html

  * igt@kms_content_protection@mei_interface:
    - shard-tglb:         [SKIP][108] ([fdo#109300]) -> [SKIP][109] ([i915#7118])
   [108]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-tglb8/igt@kms_content_protection@mei_interface.html
   [109]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-tglb6/igt@kms_content_protection@mei_interface.html
    - shard-iclb:         [SKIP][110] ([fdo#109300]) -> [SKIP][111] ([i915#7118])
   [110]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-iclb8/igt@kms_content_protection@mei_interface.html
   [111]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb5/igt@kms_content_protection@mei_interface.html

  * igt@kms_psr2_sf@overlay-plane-update-continuous-sf:
    - shard-iclb:         [SKIP][112] ([i915#2920]) -> [SKIP][113] ([fdo#111068] / [i915#658])
   [112]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-iclb2/igt@kms_psr2_sf@overlay-plane-update-continuous-sf.html
   [113]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-iclb8/igt@kms_psr2_sf@overlay-plane-update-continuous-sf.html

  * igt@runner@aborted:
    - shard-apl:          ([FAIL][114], [FAIL][115], [FAIL][116]) ([i915#180] / [i915#3002] / [i915#4312]) -> ([FAIL][117], [FAIL][118], [FAIL][119]) ([i915#3002] / [i915#4312])
   [114]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-apl7/igt@runner@aborted.html
   [115]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-apl8/igt@runner@aborted.html
   [116]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12323/shard-apl7/igt@runner@aborted.html
   [117]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-apl3/igt@runner@aborted.html
   [118]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-apl6/igt@runner@aborted.html
   [119]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/shard-apl1/igt@runner@aborted.html

  
  [fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
  [fdo#109274]: https://bugs.freedesktop.org/show_bug.cgi?id=109274
  [fdo#109278]: https://bugs.freedesktop.org/show_bug.cgi?id=109278
  [fdo#109280]: https://bugs.freedesktop.org/show_bug.cgi?id=109280
  [fdo#109283]: https://bugs.freedesktop.org/show_bug.cgi?id=109283
  [fdo#109284]: https://bugs.freedesktop.org/show_bug.cgi?id=109284
  [fdo#109289]: https://bugs.freedesktop.org/show_bug.cgi?id=109289
  [fdo#109300]: https://bugs.freedesktop.org/show_bug.cgi?id=109300
  [fdo#109441]: https://bugs.freedesktop.org/show_bug.cgi?id=109441
  [fdo#110723]: https://bugs.freedesktop.org/show_bug.cgi?id=110723
  [fdo#111068]: https://bugs.freedesktop.org/show_bug.cgi?id=111068
  [fdo#111615]: https://bugs.freedesktop.org/show_bug.cgi?id=111615
  [fdo#111825]: https://bugs.freedesktop.org/show_bug.cgi?id=111825
  [fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
  [i915#1099]: https://gitlab.freedesktop.org/drm/intel/issues/1099
  [i915#180]: https://gitlab.freedesktop.org/drm/intel/issues/180
  [i915#2190]: https://gitlab.freedesktop.org/drm/intel/issues/2190
  [i915#2346]: https://gitlab.freedesktop.org/drm/intel/issues/2346
  [i915#2521]: https://gitlab.freedesktop.org/drm/intel/issues/2521
  [i915#2527]: https://gitlab.freedesktop.org/drm/intel/issues/2527
  [i915#2587]: https://gitlab.freedesktop.org/drm/intel/issues/2587
  [i915#2658]: https://gitlab.freedesktop.org/drm/intel/issues/2658
  [i915#2672]: https://gitlab.freedesktop.org/drm/intel/issues/2672
  [i915#2684]: https://gitlab.freedesktop.org/drm/intel/issues/2684
  [i915#2705]: https://gitlab.freedesktop.org/drm/intel/issues/2705
  [i915#2842]: https://gitlab.freedesktop.org/drm/intel/issues/2842
  [i915#2856]: https://gitlab.freedesktop.org/drm/intel/issues/2856
  [i915#2920]: https://gitlab.freedesktop.org/drm/intel/issues/2920
  [i915#2994]: https://gitlab.freedesktop.org/drm/intel/issues/2994
  [i915#3002]: https://gitlab.freedesktop.org/drm/intel/issues/3002
  [i915#3536]: https://gitlab.freedesktop.org/drm/intel/issues/3536
  [i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
  [i915#3637]: https://gitlab.freedesktop.org/drm/intel/issues/3637
  [i915#3689]: https://gitlab.freedesktop.org/drm/intel/issues/3689
  [i915#3886]: https://gitlab.freedesktop.org/drm/intel/issues/3886
  [i915#3989]: https://gitlab.freedesktop.org/drm/intel/issues/3989
  [i915#4171]: https://gitlab.freedesktop.org/drm/intel/issues/4171
  [i915#4270]: https://gitlab.freedesktop.org/drm/intel/issues/4270
  [i915#4312]: https://gitlab.freedesktop.org/drm/intel/issues/4312
  [i915#4525]: https://gitlab.freedesktop.org/drm/intel/issues/4525
  [i915#454]: https://gitlab.freedesktop.org/drm/intel/issues/454
  [i915#4613]: https://gitlab.freedesktop.org/drm/intel/issues/4613
  [i915#5138]: https://gitlab.freedesktop.org/drm/intel/issues/5138
  [i915#5176]: https://gitlab.freedesktop.org/drm/intel/issues/5176
  [i915#5519]: https://gitlab.freedesktop.org/drm/intel/issues/5519
  [i915#5591]: https://gitlab.freedesktop.org/drm/intel/issues/5591
  [i915#5939]: https://gitlab.freedesktop.org/drm/intel/issues/5939
  [i915#6095]: https://gitlab.freedesktop.org/drm/intel/issues/6095
  [i915#6497]: https://gitlab.freedesktop.org/drm/intel/issues/6497
  [i915#6537]: https://gitlab.freedesktop.org/drm/intel/issues/6537
  [i915#658]: https://gitlab.freedesktop.org/drm/intel/issues/658
  [i915#7118]: https://gitlab.freedesktop.org/drm/intel/issues/7118
  [i915#7248]: https://gitlab.freedesktop.org/drm/intel/issues/7248
  [i915#768]: https://gitlab.freedesktop.org/drm/intel/issues/768
  [i915#79]: https://gitlab.freedesktop.org/drm/intel/issues/79


Build changes
-------------

  * CI: CI-20190529 -> None
  * IGT: IGT_7031 -> IGTPW_8014
  * Piglit: piglit_4509 -> None

  CI-20190529: 20190529
  CI_DRM_12323: 529744257b96535c9a0b7da5c4ede40d1f69a016 @ git://anongit.freedesktop.org/gfx-ci/linux
  IGTPW_8014: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/index.html
  IGT_7031: 54abd9445f732b55556d4e612d24f50803515904 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
  piglit_4509: fdc5a4ca11124ab8413c7988896eec4c97336694 @ git://anongit.freedesktop.org/piglit

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8014/index.html

[-- Attachment #2: Type: text/html, Size: 37748 bytes --]