[PATCH v2 0/9] trace-cmd: Allow agent to use networking

[PATCH v2 0/9] trace-cmd: Allow agent to use networking

[Steven Rostedt]

From: "Steven Rostedt (Google)" <rostedt@goodmis.org>


There's no reason for trace-cmd agent not to be able to do the communication
over TCP instead of vsockets. Have trace-cmd record pass in a host name or
IP address (IPv4 or IPv6) to the -A option and if the cid or guest name
fails, then try host name or IP address.

This will fall back to the P2P time sync protocol and recordings from
different hosts can be synchronized.

Also add more debug prints when the --debug option is used. This proved
useful to debugging this code.

Depends on: https://patchwork.kernel.org/project/linux-trace-devel/list/?series=632870
  https://lore.kernel.org/r/20220417183354.1042657-1-rostedt@goodmis.org

Changes since v1: https://lore.kernel.org/r/20220417011958.1013591-1-rostedt@goodmis.org
 - Rebased on the above dependency.
 - Added patch to allow compiling of agent without vsocket support.

Steven Rostedt (Google) (9):
  trace-cmd record: Move port_type into instance
  trace-cmd library: Add network roles for time sync
  trace-cmd record: Allow for ip connections to agents
  trace-cmd agent: Allow for ip connections from the agent
  trace-cmd library: Create tracecmd_debug() for debug printing
  trace-cmd: Add debug prints for network connections
  trace-cmd: Add print helpers to show connections
  trace-cmd: Override tracecmd_debug() to show thread id
  trace-cmd agent: Have agent work without vsockets available

 lib/trace-cmd/Makefile                        |   2 +-
 .../include/private/trace-cmd-private.h       |   5 +
 lib/trace-cmd/include/trace-tsync-local.h     |   6 +-
 lib/trace-cmd/trace-msg.c                     |  12 +-
 lib/trace-cmd/trace-timesync-ptp.c            |   4 +-
 lib/trace-cmd/trace-timesync.c                |   2 +
 lib/trace-cmd/trace-util.c                    |  12 +
 tracecmd/Makefile                             |   5 +-
 tracecmd/include/trace-local.h                |  16 ++
 tracecmd/trace-agent.c                        | 106 ++++++--
 tracecmd/trace-cmd.c                          |  20 +-
 tracecmd/trace-listen.c                       |  84 ++++--
 tracecmd/trace-record.c                       | 256 ++++++++++++------
 tracecmd/trace-usage.c                        |   2 -
 tracecmd/trace-vsock.c                        |  20 ++
 15 files changed, 406 insertions(+), 146 deletions(-)

-- 
2.35.1

[PATCH v2 1/9] trace-cmd record: Move port_type into instance

[Steven Rostedt]

From: "Steven Rostedt (Google)" <rostedt@goodmis.org>

Instead of one global method of communication, have the "port_type" be
part of the instance. This will allow for different instances to have
different connection types.

Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
 tracecmd/include/trace-local.h |  1 +
 tracecmd/trace-record.c        | 41 ++++++++++++++++++----------------
 2 files changed, 23 insertions(+), 19 deletions(-)

diff --git a/tracecmd/include/trace-local.h b/tracecmd/include/trace-local.h
index aa1cb8d939bd..2008449bab27 100644
--- a/tracecmd/include/trace-local.h
+++ b/tracecmd/include/trace-local.h
@@ -285,6 +285,7 @@ struct buffer_instance {
 	int			*fds;
 	bool			use_fifos;
 
+	enum port_type		port_type;
 	int			tsync_loop_interval;
 	struct tracecmd_time_sync *tsync;
 };
diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c
index 674ec2c3ba65..14ba4ac5dc9e 100644
--- a/tracecmd/trace-record.c
+++ b/tracecmd/trace-record.c
@@ -90,8 +90,6 @@ static bool fork_process;
 /* Max size to let a per cpu file get */
 static int max_kb;
 
-static enum port_type port_type;
-
 static int do_ptrace;
 
 static int filter_task;
@@ -3119,7 +3117,7 @@ static void finish(int sig)
 	finished = 1;
 }
 
-static int connect_port(const char *host, unsigned int port)
+static int connect_port(const char *host, unsigned int port, enum port_type type)
 {
 	struct addrinfo hints;
 	struct addrinfo *results, *rp;
@@ -3128,17 +3126,17 @@ static int connect_port(const char *host, unsigned int port)
 
 	snprintf(buf, BUFSIZ, "%u", port);
 
-	if (port_type == USE_VSOCK)
+	if (type == USE_VSOCK)
 		return trace_vsock_open(atoi(host), port);
 
 	memset(&hints, 0, sizeof(hints));
 	hints.ai_family = AF_UNSPEC;
-	hints.ai_socktype = port_type == USE_TCP ? SOCK_STREAM : SOCK_DGRAM;
+	hints.ai_socktype = type == USE_TCP ? SOCK_STREAM : SOCK_DGRAM;
 
 	s = getaddrinfo(host, buf, &hints, &results);
 	if (s != 0)
 		die("connecting to %s server %s:%s",
-		    port_type == USE_TCP ? "TCP" : "UDP", host, buf);
+		    type == USE_TCP ? "TCP" : "UDP", host, buf);
 
 	for (rp = results; rp != NULL; rp = rp->ai_next) {
 		sfd = socket(rp->ai_family, rp->ai_socktype,
@@ -3152,7 +3150,7 @@ static int connect_port(const char *host, unsigned int port)
 
 	if (rp == NULL)
 		die("Can not connect to %s server %s:%s",
-		    port_type == USE_TCP ? "TCP" : "UDP", host, buf);
+		    type == USE_TCP ? "TCP" : "UDP", host, buf);
 
 	freeaddrinfo(results);
 
@@ -3359,7 +3357,8 @@ static int create_recorder(struct buffer_instance *instance, int cpu,
 			else
 				fd = do_accept(instance->fds[cpu]);
 		} else {
-			fd = connect_port(host, instance->client_ports[cpu]);
+			fd = connect_port(host, instance->client_ports[cpu],
+					  instance->port_type);
 		}
 		if (fd < 0)
 			die("Failed connecting to client");
@@ -3414,8 +3413,9 @@ static void check_first_msg_from_server(struct tracecmd_msg_handle *msg_handle)
 }
 
 static void communicate_with_listener_v1(struct tracecmd_msg_handle *msg_handle,
-					 unsigned int **client_ports)
+					 struct buffer_instance *instance)
 {
+	unsigned int *client_ports;
 	char buf[BUFSIZ];
 	ssize_t n;
 	int cpu, i;
@@ -3442,11 +3442,11 @@ static void communicate_with_listener_v1(struct tracecmd_msg_handle *msg_handle,
 	/* TODO, test for ipv4 */
 	if (page_size >= UDP_MAX_PACKET) {
 		warning("page size too big for UDP using TCP in live read");
-		port_type = USE_TCP;
+		instance->port_type = USE_TCP;
 		msg_handle->flags |= TRACECMD_MSG_FL_USE_TCP;
 	}
 
-	if (port_type == USE_TCP) {
+	if (instance->port_type == USE_TCP) {
 		/* Send one option */
 		write(msg_handle->fd, "1", 2);
 		/* Size 4 */
@@ -3457,8 +3457,8 @@ static void communicate_with_listener_v1(struct tracecmd_msg_handle *msg_handle,
 		/* No options */
 		write(msg_handle->fd, "0", 2);
 
-	*client_ports = malloc(local_cpu_count * sizeof(*client_ports));
-	if (!*client_ports)
+	client_ports = malloc(local_cpu_count * sizeof(*client_ports));
+	if (!client_ports)
 		die("Failed to allocate client ports for %d cpus", local_cpu_count);
 
 	/*
@@ -3476,8 +3476,10 @@ static void communicate_with_listener_v1(struct tracecmd_msg_handle *msg_handle,
 		if (i == BUFSIZ)
 			die("read bad port number");
 		buf[i] = 0;
-		(*client_ports)[cpu] = atoi(buf);
+		client_ports[cpu] = atoi(buf);
 	}
+
+	instance->client_ports = client_ports;
 }
 
 static void communicate_with_listener_v3(struct tracecmd_msg_handle *msg_handle,
@@ -3609,10 +3611,11 @@ static int connect_ip(char *thost)
 static struct tracecmd_msg_handle *setup_network(struct buffer_instance *instance)
 {
 	struct tracecmd_msg_handle *msg_handle = NULL;
+	enum port_type type = instance->port_type;
 	int sfd;
 
 again:
-	switch (port_type) {
+	switch (type) {
 	case USE_VSOCK:
 		sfd = connect_vsock(host);
 		break;
@@ -3634,7 +3637,7 @@ again:
 		msg_handle->version = V3_PROTOCOL;
 	}
 
-	switch (port_type) {
+	switch (type) {
 	case USE_TCP:
 		msg_handle->flags |= TRACECMD_MSG_FL_USE_TCP;
 		break;
@@ -3657,7 +3660,7 @@ again:
 	}
 
 	if (msg_handle->version == V1_PROTOCOL)
-		communicate_with_listener_v1(msg_handle, &instance->client_ports);
+		communicate_with_listener_v1(msg_handle, instance);
 
 	return msg_handle;
 }
@@ -6514,7 +6517,7 @@ static void parse_record_options(int argc,
 			if (ctx->output)
 				die("-V incompatible with -o");
 			host = optarg;
-			port_type = USE_VSOCK;
+			ctx->instance->port_type = USE_VSOCK;
 			break;
 		case 'm':
 			if (max_kb)
@@ -6532,7 +6535,7 @@ static void parse_record_options(int argc,
 			if (IS_EXTRACT(ctx))
 				ctx->topt = 1; /* Extract top instance also */
 			else
-				port_type = USE_TCP;
+				ctx->instance->port_type = USE_TCP;
 			break;
 		case 'b':
 			check_instance_die(ctx->instance, "-b");
-- 
2.35.1

[PATCH v2 2/9] trace-cmd library: Add network roles for time sync

[Steven Rostedt]

From: "Steven Rostedt (Google)" <rostedt@goodmis.org>

Add TRACECMD_TIME_SYNC_ROLE_CLIENT and SERVER to distringuish from
guest/host to client/server.

Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
 lib/trace-cmd/include/private/trace-cmd-private.h | 2 ++
 lib/trace-cmd/trace-timesync-ptp.c                | 4 +++-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h
index f68d17bb8e1d..6e7b346cda07 100644
--- a/lib/trace-cmd/include/private/trace-cmd-private.h
+++ b/lib/trace-cmd/include/private/trace-cmd-private.h
@@ -480,6 +480,8 @@ enum{
 enum tracecmd_time_sync_role {
 	TRACECMD_TIME_SYNC_ROLE_HOST	= (1 << 0),
 	TRACECMD_TIME_SYNC_ROLE_GUEST	= (1 << 1),
+	TRACECMD_TIME_SYNC_ROLE_CLIENT	= (1 << 2),
+	TRACECMD_TIME_SYNC_ROLE_SERVER	= (1 << 3),
 };
 
 /* Timestamp synchronization flags */
diff --git a/lib/trace-cmd/trace-timesync-ptp.c b/lib/trace-cmd/trace-timesync-ptp.c
index 653d176e2e79..20e6e6f1ab59 100644
--- a/lib/trace-cmd/trace-timesync-ptp.c
+++ b/lib/trace-cmd/trace-timesync-ptp.c
@@ -702,7 +702,9 @@ int ptp_clock_sync_register(void)
 {
 	return tracecmd_tsync_proto_register(PTP_NAME, PTP_ACCURACY,
 					     TRACECMD_TIME_SYNC_ROLE_GUEST |
-					     TRACECMD_TIME_SYNC_ROLE_HOST,
+					     TRACECMD_TIME_SYNC_ROLE_HOST |
+					     TRACECMD_TIME_SYNC_ROLE_CLIENT |
+					     TRACECMD_TIME_SYNC_ROLE_SERVER,
 					     0, TRACECMD_TSYNC_FLAG_INTERPOLATE,
 					     ptp_clock_sync_init,
 					     ptp_clock_sync_free,
-- 
2.35.1

[PATCH v2 3/9] trace-cmd record: Allow for ip connections to agents

[Steven Rostedt]

From: "Steven Rostedt (Google)" <rostedt@goodmis.org>

Have the -A option to trace-cmd record accept IP addresses (v4 or v6)
or host names that can connect to agents that are not running on guests
(or maybe they are!) but are running on other machines connecting to the
network.

Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
 tracecmd/include/trace-local.h |   3 +
 tracecmd/trace-record.c        | 214 +++++++++++++++++++++++----------
 2 files changed, 151 insertions(+), 66 deletions(-)

diff --git a/tracecmd/include/trace-local.h b/tracecmd/include/trace-local.h
index 2008449bab27..afbcab422c2f 100644
--- a/tracecmd/include/trace-local.h
+++ b/tracecmd/include/trace-local.h
@@ -189,6 +189,7 @@ enum buffer_instance_flags {
 	BUFFER_FL_AGENT		= 1 << 3,
 	BUFFER_FL_HAS_CLOCK	= 1 << 4,
 	BUFFER_FL_TSC2NSEC	= 1 << 5,
+	BUFFER_FL_NETWORK	= 1 << 6,
 };
 
 struct func_list {
@@ -280,6 +281,7 @@ struct buffer_instance {
 	int			argc;
 	char			**argv;
 
+	struct addrinfo		*result;
 	unsigned int		cid;
 	unsigned int		port;
 	int			*fds;
@@ -302,6 +304,7 @@ extern struct buffer_instance *first_instance;
 
 #define is_agent(instance)	((instance)->flags & BUFFER_FL_AGENT)
 #define is_guest(instance)	((instance)->flags & BUFFER_FL_GUEST)
+#define is_network(instance)	((instance)->flags & BUFFER_FL_NETWORK)
 
 struct buffer_instance *allocate_instance(const char *name);
 void add_instance(struct buffer_instance *instance, int cpu_count);
diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c
index 14ba4ac5dc9e..dfcf3d494053 100644
--- a/tracecmd/trace-record.c
+++ b/tracecmd/trace-record.c
@@ -83,6 +83,8 @@ static bool no_fifos;
 
 static char *host;
 
+static const char *gai_err;
+
 static bool quiet;
 
 static bool fork_process;
@@ -3117,26 +3119,33 @@ static void finish(int sig)
 	finished = 1;
 }
 
-static int connect_port(const char *host, unsigned int port, enum port_type type)
+static struct addrinfo *do_getaddrinfo(const char *host, unsigned int port,
+				       enum port_type type)
 {
+	struct addrinfo *results;
 	struct addrinfo hints;
-	struct addrinfo *results, *rp;
-	int s, sfd;
 	char buf[BUFSIZ];
+	int s;
 
 	snprintf(buf, BUFSIZ, "%u", port);
 
-	if (type == USE_VSOCK)
-		return trace_vsock_open(atoi(host), port);
-
 	memset(&hints, 0, sizeof(hints));
 	hints.ai_family = AF_UNSPEC;
 	hints.ai_socktype = type == USE_TCP ? SOCK_STREAM : SOCK_DGRAM;
 
 	s = getaddrinfo(host, buf, &hints, &results);
-	if (s != 0)
-		die("connecting to %s server %s:%s",
-		    type == USE_TCP ? "TCP" : "UDP", host, buf);
+	if (s != 0) {
+		gai_err = gai_strerror(s);
+		return NULL;
+	}
+
+	return results;
+}
+
+static int connect_addr(struct addrinfo *results)
+{
+	struct addrinfo *rp;
+	int sfd = -1;
 
 	for (rp = results; rp != NULL; rp = rp->ai_next) {
 		sfd = socket(rp->ai_family, rp->ai_socktype,
@@ -3149,11 +3158,33 @@ static int connect_port(const char *host, unsigned int port, enum port_type type
 	}
 
 	if (rp == NULL)
-		die("Can not connect to %s server %s:%s",
-		    type == USE_TCP ? "TCP" : "UDP", host, buf);
+		return -1;
+
+	return sfd;
+}
+
+static int connect_port(const char *host, unsigned int port, enum port_type type)
+{
+	struct addrinfo *results;
+	int sfd;
+
+	if (type == USE_VSOCK)
+		return trace_vsock_open(atoi(host), port);
+
+	results = do_getaddrinfo(host, port, type);
+
+	if (!results)
+		die("connecting to %s server %s:%u",
+		    type == USE_TCP ? "TCP" : "UDP", host, port);
+
+	sfd = connect_addr(results);
 
 	freeaddrinfo(results);
 
+	if (sfd < 0)
+		die("Can not connect to %s server %s:%u",
+		    type == USE_TCP ? "TCP" : "UDP", host, port);
+
 	return sfd;
 }
 
@@ -3207,14 +3238,22 @@ static void find_tasks(struct trace_guest *guest)
 	closedir(dir);
 }
 
-static char *parse_guest_name(char *gname, int *cid, int *port)
+static char *parse_guest_name(char *gname, int *cid, int *port,
+			      struct addrinfo **res)
 {
-	struct trace_guest *guest;
+	struct trace_guest *guest = NULL;
+	struct addrinfo *result;
+	char *ip = NULL;
 	char *p;
 
+	*res = NULL;
+
 	*port = -1;
-	p = strrchr(gname, ':');
-	if (p) {
+	for (p = gname + strlen(gname); p > gname; p--) {
+		if (*p == ':')
+			break;
+	}
+	if (p > gname) {
 		*p = '\0';
 		*port = atoi(p + 1);
 	}
@@ -3224,13 +3263,19 @@ static char *parse_guest_name(char *gname, int *cid, int *port)
 	if (p) {
 		*p = '\0';
 		*cid = atoi(p + 1);
-	} else if (is_digits(gname))
+	} else if (is_digits(gname)) {
 		*cid = atoi(gname);
+	} else {
+		/* Check if this is an IP address */
+		if (strstr(gname, ":") || strstr(gname, "."))
+			ip = gname;
+	}
 
-	if (*cid < 0)
+	if (!ip && *cid < 0)
 		read_qemu_guests();
 
-	guest = trace_get_guest(*cid, gname);
+	if (!ip)
+		guest = trace_get_guest(*cid, gname);
 	if (guest) {
 		*cid = guest->cid;
 		/* Mapping not found, search for them */
@@ -3239,6 +3284,13 @@ static char *parse_guest_name(char *gname, int *cid, int *port)
 		return guest->name;
 	}
 
+	/* Test to see if this is an internet address */
+	result = do_getaddrinfo(gname, *port, USE_TCP);
+	if (!result)
+		return NULL;
+
+	*res = result;
+
 	return gname;
 }
 
@@ -3280,6 +3332,7 @@ create_recorder_instance(struct buffer_instance *instance, const char *file, int
 			 int *brass)
 {
 	struct tracecmd_recorder *record;
+	struct addrinfo *result;
 	char *path;
 
 	if (is_guest(instance)) {
@@ -3288,7 +3341,17 @@ create_recorder_instance(struct buffer_instance *instance, const char *file, int
 
 		if (instance->use_fifos)
 			fd = instance->fds[cpu];
-		else
+		else if (is_network(instance)) {
+			result = do_getaddrinfo(instance->name,
+						instance->client_ports[cpu],
+						instance->port_type);
+			if (!result)
+				die("Failed to connect to %s port %d\n",
+				    instance->name,
+				    instance->client_ports[cpu]);
+			fd = connect_addr(result);
+			freeaddrinfo(result);
+		} else
 			fd = trace_vsock_open(instance->cid, instance->client_ports[cpu]);
 		if (fd < 0)
 			die("Failed to connect to agent");
@@ -3560,9 +3623,8 @@ static int connect_vsock(char *vhost)
 
 static int connect_ip(char *thost)
 {
-	struct addrinfo hints;
-	struct addrinfo *result, *rp;
-	int sfd, s;
+	struct addrinfo *result;
+	int sfd;
 	char *server;
 	char *port;
 	char *p;
@@ -3581,30 +3643,17 @@ static int connect_ip(char *thost)
 		port = strtok_r(NULL, ":", &p);
 	}
 
-	memset(&hints, 0, sizeof(hints));
-	hints.ai_family = AF_UNSPEC;
-	hints.ai_socktype = SOCK_STREAM;
+	result = do_getaddrinfo(server, atoi(port), USE_TCP);
+	if (!result)
+		die("getaddrinfo: %s", gai_err);
 
-	s = getaddrinfo(server, port, &hints, &result);
-	if (s != 0)
-		die("getaddrinfo: %s", gai_strerror(s));
+	sfd = connect_addr(result);
 
-	for (rp = result; rp != NULL; rp = rp->ai_next) {
-		sfd = socket(rp->ai_family, rp->ai_socktype,
-			     rp->ai_protocol);
-		if (sfd == -1)
-			continue;
-
-		if (connect(sfd, rp->ai_addr, rp->ai_addrlen) != -1)
-			break;
-		close(sfd);
-	}
+	freeaddrinfo(result);
 
-	if (!rp)
+	if (sfd < 0)
 		die("Can not connect to %s:%s", server, port);
 
-	freeaddrinfo(result);
-
 	return sfd;
 }
 
@@ -3960,20 +4009,26 @@ static int host_tsync(struct common_record_context *ctx,
 	if (!proto)
 		return -1;
 
-	guest = trace_get_guest(instance->cid, NULL);
-	if (guest == NULL)
-		return -1;
+	if (is_network(instance)) {
+		fd = connect_port(instance->name, tsync_port,
+				  instance->port_type);
+	} else {
+		guest = trace_get_guest(instance->cid, NULL);
+		if (guest == NULL)
+			return -1;
 
-	guest_pid = guest->pid;
-	start_mapping_vcpus(guest);
+		guest_pid = guest->pid;
+		start_mapping_vcpus(guest);
+		fd = trace_vsock_open(instance->cid, tsync_port);
+	}
 
-	fd = trace_vsock_open(instance->cid, tsync_port);
 	instance->tsync = tracecmd_tsync_with_guest(top_instance.trace_id,
 						    instance->tsync_loop_interval,
 						    fd, guest_pid,
 						    instance->cpu_count,
 						    proto, ctx->clock);
-	stop_mapping_vcpus(instance, guest);
+	if (!is_network(instance))
+		stop_mapping_vcpus(instance, guest);
 
 	if (!instance->tsync)
 		return -1;
@@ -3987,6 +4042,7 @@ static void connect_to_agent(struct common_record_context *ctx,
 	struct tracecmd_tsync_protos *protos = NULL;
 	int sd, ret, nr_fifos, nr_cpus, page_size;
 	struct tracecmd_msg_handle *msg_handle;
+	enum tracecmd_time_sync_role role;
 	char *tsync_protos_reply = NULL;
 	unsigned int tsync_port = 0;
 	unsigned int *ports;
@@ -3998,10 +4054,19 @@ static void connect_to_agent(struct common_record_context *ctx,
 		use_fifos = nr_fifos > 0;
 	}
 
-	sd = trace_vsock_open(instance->cid, instance->port);
-	if (sd < 0)
-		die("Failed to connect to vsocket @%u:%u",
-		    instance->cid, instance->port);
+	if (ctx->instance->result) {
+		role = TRACECMD_TIME_SYNC_ROLE_CLIENT;
+		sd = connect_addr(ctx->instance->result);
+		if (sd < 0)
+			die("Failed to connect to host %s:%u",
+			    instance->name, instance->port);
+	} else {
+		role = TRACECMD_TIME_SYNC_ROLE_HOST;
+		sd = trace_vsock_open(instance->cid, instance->port);
+		if (sd < 0)
+			die("Failed to connect to vsocket @%u:%u",
+			    instance->cid, instance->port);
+	}
 
 	msg_handle = tracecmd_msg_handle_alloc(sd, 0);
 	if (!msg_handle)
@@ -4011,8 +4076,7 @@ static void connect_to_agent(struct common_record_context *ctx,
 		instance->clock = tracefs_get_clock(NULL);
 
 	if (instance->tsync_loop_interval >= 0)
-		tracecmd_tsync_proto_getall(&protos, instance->clock,
-					    TRACECMD_TIME_SYNC_ROLE_HOST);
+		tracecmd_tsync_proto_getall(&protos, instance->clock, role);
 
 	ret = tracecmd_msg_send_trace_req(msg_handle, instance->argc,
 					  instance->argv, use_fifos,
@@ -4219,16 +4283,23 @@ static void append_buffer(struct tracecmd_output *handle,
 static void
 add_guest_info(struct tracecmd_output *handle, struct buffer_instance *instance)
 {
-	struct trace_guest *guest = trace_get_guest(instance->cid, NULL);
+	struct trace_guest *guest;
+	const char *name;
 	char *buf, *p;
 	int size;
 	int pid;
 	int i;
 
-	if (!guest)
-		return;
+	if (is_network(instance)) {
+		name = instance->name;
+	} else {
+		guest = trace_get_guest(instance->cid, NULL);
+		if (!guest)
+			return;
+		name = guest->name;
+	}
 
-	size = strlen(guest->name) + 1;
+	size = strlen(name) + 1;
 	size += sizeof(long long);	/* trace_id */
 	size += sizeof(int);		/* cpu count */
 	size += instance->cpu_count * 2 * sizeof(int);	/* cpu,pid pair */
@@ -4237,8 +4308,8 @@ add_guest_info(struct tracecmd_output *handle, struct buffer_instance *instance)
 	if (!buf)
 		return;
 	p = buf;
-	strcpy(p, guest->name);
-	p += strlen(guest->name) + 1;
+	strcpy(p, name);
+	p += strlen(name) + 1;
 
 	memcpy(p, &instance->trace_id, sizeof(long long));
 	p += sizeof(long long);
@@ -4246,10 +4317,11 @@ add_guest_info(struct tracecmd_output *handle, struct buffer_instance *instance)
 	memcpy(p, &instance->cpu_count, sizeof(int));
 	p += sizeof(int);
 	for (i = 0; i < instance->cpu_count; i++) {
-		if (i < guest->cpu_max)
-			pid = guest->cpu_pid[i];
-		else
-			pid = -1;
+		pid = -1;
+		if (!is_network(instance)) {
+			if (i < guest->cpu_max)
+				pid = guest->cpu_pid[i];
+		}
 		memcpy(p, &i, sizeof(int));
 		p += sizeof(int);
 		memcpy(p, &pid, sizeof(int));
@@ -6165,6 +6237,7 @@ static void parse_record_options(int argc,
 	const char *option;
 	struct event_list *event = NULL;
 	struct event_list *last_event = NULL;
+	struct addrinfo *result;
 	char *pids;
 	char *pid;
 	char *sav;
@@ -6304,8 +6377,8 @@ static void parse_record_options(int argc,
 			if (!IS_RECORD(ctx))
 				die("-A is only allowed for record operations");
 
-			name = parse_guest_name(optarg, &cid, &port);
-			if (cid == -1)
+			name = parse_guest_name(optarg, &cid, &port, &result);
+			if (cid == -1 && !result)
 				die("guest %s not found", optarg);
 			if (port == -1)
 				port = TRACE_AGENT_DEFAULT_PORT;
@@ -6321,7 +6394,16 @@ static void parse_record_options(int argc,
 				die("Failed to allocate guest name");
 
 			ctx->instance = allocate_instance(name);
+			if (!ctx->instance)
+				die("Failed to allocate instance");
+
+			if (result) {
+				ctx->instance->flags |= BUFFER_FL_NETWORK;
+				ctx->instance->port_type = USE_TCP;
+			}
+
 			ctx->instance->flags |= BUFFER_FL_GUEST;
+			ctx->instance->result = result;
 			ctx->instance->cid = cid;
 			ctx->instance->port = port;
 			ctx->instance->name = name;
-- 
2.35.1

[PATCH v2 4/9] trace-cmd agent: Allow for ip connections from the agent

[Steven Rostedt]

From: "Steven Rostedt (Google)" <rostedt@goodmis.org>

Add a -N option to trace-cmd agent to listen on a network socket that
can be used by trace-cmd record -A to connect with.

Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
 tracecmd/include/trace-local.h |  6 +++
 tracecmd/trace-agent.c         | 89 ++++++++++++++++++++++++++--------
 tracecmd/trace-listen.c        | 58 ++++++++++++++--------
 3 files changed, 113 insertions(+), 40 deletions(-)

diff --git a/tracecmd/include/trace-local.h b/tracecmd/include/trace-local.h
index afbcab422c2f..fdb0d887557f 100644
--- a/tracecmd/include/trace-local.h
+++ b/tracecmd/include/trace-local.h
@@ -306,6 +306,12 @@ extern struct buffer_instance *first_instance;
 #define is_guest(instance)	((instance)->flags & BUFFER_FL_GUEST)
 #define is_network(instance)	((instance)->flags & BUFFER_FL_NETWORK)
 
+#define START_PORT_SEARCH 1500
+#define MAX_PORT_SEARCH 6000
+
+int trace_net_make(int port, enum port_type type);
+int trace_net_search(int start_port, int *sfd, enum port_type type);
+
 struct buffer_instance *allocate_instance(const char *name);
 void add_instance(struct buffer_instance *instance, int cpu_count);
 void update_first_instance(struct buffer_instance *instance, int topt);
diff --git a/tracecmd/trace-agent.c b/tracecmd/trace-agent.c
index 719d7126ca67..cd201ad948ad 100644
--- a/tracecmd/trace-agent.c
+++ b/tracecmd/trace-agent.c
@@ -41,6 +41,32 @@ static void make_vsocks(int nr, int *fds, unsigned int *ports)
 	}
 }
 
+static void make_net(int nr, int *fds, unsigned int *ports)
+{
+	int port;
+	int i, fd;
+	int start_port = START_PORT_SEARCH;
+
+	for (i = 0; i < nr; i++) {
+		port = trace_net_search(start_port, &fd, USE_TCP);
+		if (port < 0)
+			die("Failed to open socket");
+		if (listen(fd, 5) < 0)
+			die("Failed to listen on port %d\n", port);
+		fds[i] = fd;
+		ports[i] = port;
+		start_port = port + 1;
+	}
+}
+
+static void make_sockets(int nr, int *fds, unsigned int *ports, bool network)
+{
+	if (network)
+		return make_net(nr, fds, ports);
+	else
+		return make_vsocks(nr, fds, ports);
+}
+
 static int open_agent_fifos(int nr_cpus, int *fds)
 {
 	char path[PATH_MAX];
@@ -80,7 +106,7 @@ static char *get_clock(int argc, char **argv)
 	return NULL;
 }
 
-static void agent_handle(int sd, int nr_cpus, int page_size)
+static void agent_handle(int sd, int nr_cpus, int page_size, bool network)
 {
 	struct tracecmd_tsync_protos *tsync_protos = NULL;
 	struct tracecmd_time_sync *tsync = NULL;
@@ -117,17 +143,31 @@ static void agent_handle(int sd, int nr_cpus, int page_size)
 		use_fifos = false;
 
 	if (!use_fifos)
-		make_vsocks(nr_cpus, fds, ports);
+		make_sockets(nr_cpus, fds, ports, network);
 	if (tsync_protos && tsync_protos->names) {
-		if (get_vsocket_params(msg_handle->fd, &local_id,
-				       &remote_id)) {
-			warning("Failed to get local and remote ids");
-			/* Just make something up */
-			remote_id = -1;
-			local_id = -2;
+		if (network) {
+			/* For now just use something */
+			remote_id = 2;
+			local_id = 1;
+			tsync_port = trace_net_search(START_PORT_SEARCH, &fd, USE_TCP);
+			if (listen(fd, 5) < 0)
+				die("Failed to listen on %d\n", tsync_port);
+		} else {
+			if (get_vsocket_params(msg_handle->fd, &local_id,
+					       &remote_id)) {
+				warning("Failed to get local and remote ids");
+				/* Just make something up */
+				remote_id = -1;
+				local_id = -2;
+			}
+			fd = trace_vsock_make_any();
+			if (fd >= 0 &&
+			    trace_vsock_get_port(fd, &tsync_port) < 0) {
+				close(fd);
+				fd = -1;
+			}
 		}
-		fd = trace_vsock_make_any();
-		if (fd >= 0 && trace_vsock_get_port(fd, &tsync_port) >= 0) {
+		if (fd >= 0) {
 			tsync = tracecmd_tsync_with_host(fd, tsync_protos,
 							 get_clock(argc, argv),
 							 remote_id, local_id);
@@ -193,7 +233,7 @@ static pid_t do_fork()
 	return fork();
 }
 
-static void agent_serve(unsigned int port, bool do_daemon)
+static void agent_serve(unsigned int port, bool do_daemon, bool network)
 {
 	int sd, cd, nr_cpus;
 	unsigned int cid;
@@ -204,14 +244,21 @@ static void agent_serve(unsigned int port, bool do_daemon)
 	nr_cpus = tracecmd_count_cpus();
 	page_size = getpagesize();
 
-	sd = trace_vsock_make(port);
+	if (network) {
+		sd = trace_net_make(port, USE_TCP);
+		if (listen(sd, 5) < 0)
+			die("Failed to listen on %d\n", port);
+	} else
+		sd = trace_vsock_make(port);
 	if (sd < 0)
-		die("Failed to open vsocket");
+		die("Failed to open socket");
 	tracecmd_tsync_init();
 
-	cid = trace_vsock_local_cid();
-	if (cid >= 0)
-		printf("listening on @%u:%u\n", cid, port);
+	if (!network) {
+		cid = trace_vsock_local_cid();
+		if (cid >= 0)
+			printf("listening on @%u:%u\n", cid, port);
+	}
 
 	if (do_daemon && daemon(1, 0))
 		die("daemon");
@@ -231,7 +278,7 @@ static void agent_serve(unsigned int port, bool do_daemon)
 		if (pid == 0) {
 			close(sd);
 			signal(SIGCHLD, SIG_DFL);
-			agent_handle(cd, nr_cpus, page_size);
+			agent_handle(cd, nr_cpus, page_size, network);
 		}
 		if (pid > 0)
 			handler_pid = pid;
@@ -250,6 +297,7 @@ void trace_agent(int argc, char **argv)
 {
 	bool do_daemon = false;
 	unsigned int port = TRACE_AGENT_DEFAULT_PORT;
+	bool network = false;
 
 	if (argc < 2)
 		usage(argv);
@@ -267,7 +315,7 @@ void trace_agent(int argc, char **argv)
 			{NULL, 0, NULL, 0}
 		};
 
-		c = getopt_long(argc-1, argv+1, "+hp:D",
+		c = getopt_long(argc-1, argv+1, "+hp:DN",
 				long_options, &option_index);
 		if (c == -1)
 			break;
@@ -275,6 +323,9 @@ void trace_agent(int argc, char **argv)
 		case 'h':
 			usage(argv);
 			break;
+		case 'N':
+			network = true;
+			break;
 		case 'p':
 			port = atoi(optarg);
 			break;
@@ -296,5 +347,5 @@ void trace_agent(int argc, char **argv)
 	if (optind < argc-1)
 		usage(argv);
 
-	agent_serve(port, do_daemon);
+	agent_serve(port, do_daemon, network);
 }
diff --git a/tracecmd/trace-listen.c b/tracecmd/trace-listen.c
index 2338173fc8a6..8476fa51dadd 100644
--- a/tracecmd/trace-listen.c
+++ b/tracecmd/trace-listen.c
@@ -234,52 +234,68 @@ static int setup_vsock_port(int start_port, int *sfd)
 	return start_port;
 }
 
-#define START_PORT_SEARCH 1500
-#define MAX_PORT_SEARCH 6000
-
-static int bind_a_port(int start_port, int *sfd, enum port_type type)
+int trace_net_make(int port, enum port_type type)
 {
 	struct addrinfo hints;
 	struct addrinfo *result, *rp;
 	char buf[BUFSIZ];
+	int sd;
 	int s;
-	int num_port = start_port;
 
-	if (type == USE_VSOCK)
-		return setup_vsock_port(start_port, sfd);
- again:
-	snprintf(buf, BUFSIZ, "%d", num_port);
+	snprintf(buf, BUFSIZ, "%d", port);
 
 	memset(&hints, 0, sizeof(hints));
 	hints.ai_family = AF_UNSPEC;
-	hints.ai_socktype = type == USE_TCP ? SOCK_STREAM : SOCK_DGRAM;
 	hints.ai_flags = AI_PASSIVE;
 
+	switch (type) {
+	case USE_TCP:
+		hints.ai_socktype = SOCK_STREAM;
+		break;
+	case USE_UDP:
+		hints.ai_socktype = SOCK_DGRAM;
+		break;
+	default:
+		return -1;
+	}
+
 	s = getaddrinfo(NULL, buf, &hints, &result);
 	if (s != 0)
 		pdie("getaddrinfo: error opening socket");
 
 	for (rp = result; rp != NULL; rp = rp->ai_next) {
-		*sfd = socket(rp->ai_family, rp->ai_socktype,
-			      rp->ai_protocol);
-		if (*sfd < 0)
+		sd = socket(rp->ai_family, rp->ai_socktype,
+			    rp->ai_protocol);
+		if (sd < 0)
 			continue;
 
-		if (bind(*sfd, rp->ai_addr, rp->ai_addrlen) == 0)
+		if (bind(sd, rp->ai_addr, rp->ai_addrlen) == 0)
 			break;
 
-		close(*sfd);
+		close(sd);
 	}
+	freeaddrinfo(result);
+
+	if (rp == NULL)
+		return -1;
+
+	return sd;
+}
+
+int trace_net_search(int start_port, int *sfd, enum port_type type)
+{
+	int num_port = start_port;
 
-	if (rp == NULL) {
-		freeaddrinfo(result);
+	if (type == USE_VSOCK)
+		return setup_vsock_port(start_port, sfd);
+ again:
+	*sfd = trace_net_make(num_port, type);
+	if (*sfd < 0) {
 		if (++num_port > MAX_PORT_SEARCH)
 			pdie("No available ports to bind");
 		goto again;
 	}
 
-	freeaddrinfo(result);
-
 	return num_port;
 }
 
@@ -309,10 +325,10 @@ static int open_port(const char *node, const char *port, int *pid,
 	int num_port;
 
 	/*
-	 * bind_a_port() currently does not return an error, but if that
+	 * trace_net_search() currently does not return an error, but if that
 	 * changes in the future, we have a check for it now.
 	 */
-	num_port = bind_a_port(start_port, &sfd, type);
+	num_port = trace_net_search(start_port, &sfd, type);
 	if (num_port < 0)
 		return num_port;
 
-- 
2.35.1

[PATCH v2 5/9] trace-cmd library: Create tracecmd_debug() for debug printing

[Steven Rostedt]

From: "Steven Rostedt (Google)" <rostedt@goodmis.org>

Move the dprint() of trace-msg.c into trace-utils.c and export it such
that other applications could use it as well. It is now renamed to
tracecmd_debug().

Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
 lib/trace-cmd/include/private/trace-cmd-private.h |  3 +++
 lib/trace-cmd/trace-msg.c                         | 12 +-----------
 lib/trace-cmd/trace-util.c                        | 12 ++++++++++++
 3 files changed, 16 insertions(+), 11 deletions(-)

diff --git a/lib/trace-cmd/include/private/trace-cmd-private.h b/lib/trace-cmd/include/private/trace-cmd-private.h
index 6e7b346cda07..766e0a762c2b 100644
--- a/lib/trace-cmd/include/private/trace-cmd-private.h
+++ b/lib/trace-cmd/include/private/trace-cmd-private.h
@@ -43,6 +43,9 @@ enum {
 	RINGBUF_TYPE_TIME_STAMP		= 31,
 };
 
+/* Can be overridden */
+void tracecmd_debug(const char *fmt, ...);
+
 void tracecmd_record_ref(struct tep_record *record);
 
 void tracecmd_set_debug(bool set_debug);
diff --git a/lib/trace-cmd/trace-msg.c b/lib/trace-cmd/trace-msg.c
index 6cf74f9b1c99..39465ade8ab3 100644
--- a/lib/trace-cmd/trace-msg.c
+++ b/lib/trace-cmd/trace-msg.c
@@ -31,17 +31,7 @@
 typedef __u32 u32;
 typedef __be32 be32;
 
-static inline void dprint(const char *fmt, ...)
-{
-	va_list ap;
-
-	if (!tracecmd_get_debug())
-		return;
-
-	va_start(ap, fmt);
-	vprintf(fmt, ap);
-	va_end(ap);
-}
+#define dprint(fmt, ...)	tracecmd_debug(fmt, ##__VA_ARGS__)
 
 /* Two (4k) pages is the max transfer for now */
 #define MSG_MAX_LEN			8192
diff --git a/lib/trace-cmd/trace-util.c b/lib/trace-cmd/trace-util.c
index b5a0a1a601d4..9564c81a5c99 100644
--- a/lib/trace-cmd/trace-util.c
+++ b/lib/trace-cmd/trace-util.c
@@ -415,6 +415,18 @@ void __weak tracecmd_critical(const char *fmt, ...)
 	}
 }
 
+void __weak tracecmd_debug(const char *fmt, ...)
+{
+	va_list ap;
+
+	if (!tracecmd_get_debug())
+		return;
+
+	va_start(ap, fmt);
+	vprintf(fmt, ap);
+	va_end(ap);
+}
+
 #define LOG_BUF_SIZE 1024
 static void __plog(const char *prefix, const char *fmt, va_list ap, FILE *fp)
 {
-- 
2.35.1

[PATCH v2 6/9] trace-cmd: Add debug prints for network connections

[Steven Rostedt]

From: "Steven Rostedt (Google)" <rostedt@goodmis.org>

To help debug the network agent and listener, add some debug prints that
are enabled when --debug option is used.

Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
 lib/trace-cmd/trace-timesync.c | 2 ++
 tracecmd/trace-agent.c         | 3 +++
 tracecmd/trace-listen.c        | 4 ++++
 tracecmd/trace-record.c        | 9 +++++++++
 4 files changed, 18 insertions(+)

diff --git a/lib/trace-cmd/trace-timesync.c b/lib/trace-cmd/trace-timesync.c
index 14cf20c870cb..cc44af38e8ad 100644
--- a/lib/trace-cmd/trace-timesync.c
+++ b/lib/trace-cmd/trace-timesync.c
@@ -925,7 +925,9 @@ static void *tsync_agent_thread(void *data)
 	int sd;
 
 	while (true) {
+		tracecmd_debug("Listening on fd:%d\n", tsync->msg_handle->fd);
 		sd = accept(tsync->msg_handle->fd, NULL, NULL);
+		tracecmd_debug("Accepted fd:%d\n", sd);
 		if (sd < 0) {
 			if (errno == EINTR)
 				continue;
diff --git a/tracecmd/trace-agent.c b/tracecmd/trace-agent.c
index cd201ad948ad..819ac016612f 100644
--- a/tracecmd/trace-agent.c
+++ b/tracecmd/trace-agent.c
@@ -22,6 +22,8 @@
 #include "trace-local.h"
 #include "trace-msg.h"
 
+#define dprint(fmt, ...)	tracecmd_debug(fmt, ##__VA_ARGS__)
+
 static void make_vsocks(int nr, int *fds, unsigned int *ports)
 {
 	unsigned int port;
@@ -55,6 +57,7 @@ static void make_net(int nr, int *fds, unsigned int *ports)
 			die("Failed to listen on port %d\n", port);
 		fds[i] = fd;
 		ports[i] = port;
+		dprint("CPU[%d]: fd:%d port:%d\n", i, fd, port);
 		start_port = port + 1;
 	}
 }
diff --git a/tracecmd/trace-listen.c b/tracecmd/trace-listen.c
index 8476fa51dadd..3b446acea972 100644
--- a/tracecmd/trace-listen.c
+++ b/tracecmd/trace-listen.c
@@ -26,6 +26,8 @@
 #include "trace-local.h"
 #include "trace-msg.h"
 
+#define dprint(fmt, ...)	tracecmd_debug(fmt, ##__VA_ARGS__)
+
 #define MAX_OPTION_SIZE 4096
 
 #define _VAR_DIR_Q(dir)		#dir
@@ -279,6 +281,8 @@ int trace_net_make(int port, enum port_type type)
 	if (rp == NULL)
 		return -1;
 
+	dprint("Create listen port: %d fd:%d\n", port, sd);
+
 	return sd;
 }
 
diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c
index dfcf3d494053..9c930920c89e 100644
--- a/tracecmd/trace-record.c
+++ b/tracecmd/trace-record.c
@@ -56,6 +56,8 @@
 #define FUNC_STACK_TRACE "func_stack_trace"
 #define TSC_CLOCK	"x86-tsc"
 
+#define dprint(fmt, ...)	tracecmd_debug(fmt, ##__VA_ARGS__)
+
 enum trace_type {
 	TRACE_TYPE_RECORD	= 1,
 	TRACE_TYPE_START	= (1 << 1),
@@ -3139,6 +3141,9 @@ static struct addrinfo *do_getaddrinfo(const char *host, unsigned int port,
 		return NULL;
 	}
 
+	dprint("Attached port %s: %d to results: %p\n",
+	       type == USE_TCP ? "TCP" : "UDP", port, results);
+
 	return results;
 }
 
@@ -3160,6 +3165,8 @@ static int connect_addr(struct addrinfo *results)
 	if (rp == NULL)
 		return -1;
 
+	dprint("connect results: %p with fd: %d\n", results, sfd);
+
 	return sfd;
 }
 
@@ -3193,7 +3200,9 @@ static int do_accept(int sd)
 	int cd;
 
 	for (;;) {
+		dprint("Wait on accept: %d\n", sd);
 		cd = accept(sd, NULL, NULL);
+		dprint("accepted: %d\n", cd);
 		if (cd < 0) {
 			if (errno == EINTR)
 				continue;
-- 
2.35.1

[PATCH v2 7/9] trace-cmd: Add print helpers to show connections

[Steven Rostedt]

From: "Steven Rostedt (Google)" <rostedt@goodmis.org>

Add helper functions trace_net_print_connection() and
trace_vsock_print_connection() to print the host/port or cid/port that
has connected.

Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
 tracecmd/include/trace-local.h |  6 ++++++
 tracecmd/trace-agent.c         | 14 ++++++++++++++
 tracecmd/trace-listen.c        | 22 ++++++++++++++++++++++
 tracecmd/trace-vsock.c         | 20 ++++++++++++++++++++
 4 files changed, 62 insertions(+)

diff --git a/tracecmd/include/trace-local.h b/tracecmd/include/trace-local.h
index fdb0d887557f..b3230529a45c 100644
--- a/tracecmd/include/trace-local.h
+++ b/tracecmd/include/trace-local.h
@@ -311,6 +311,7 @@ extern struct buffer_instance *first_instance;
 
 int trace_net_make(int port, enum port_type type);
 int trace_net_search(int start_port, int *sfd, enum port_type type);
+int trace_net_print_connection(int fd);
 
 struct buffer_instance *allocate_instance(const char *name);
 void add_instance(struct buffer_instance *instance, int cpu_count);
@@ -366,6 +367,7 @@ int get_vsocket_params(int fd, unsigned int *lcid, unsigned int *rcid);
 int trace_vsock_get_port(int sd, unsigned int *port);
 bool trace_vsock_can_splice_read(void);
 int trace_vsock_local_cid(void);
+int trace_vsock_print_connection(int fd);
 #else
 static inline int trace_vsock_open(unsigned int cid, unsigned int port)
 {
@@ -403,6 +405,10 @@ static inline int trace_vsock_local_cid(void)
 {
 	return -ENOTSUP;
 }
+static inline int trace_vsock_print_connection(int fd)
+{
+	return -1;
+}
 #endif /* VSOCK */
 
 /* No longer in event-utils.h */
diff --git a/tracecmd/trace-agent.c b/tracecmd/trace-agent.c
index 819ac016612f..59cecae770a6 100644
--- a/tracecmd/trace-agent.c
+++ b/tracecmd/trace-agent.c
@@ -109,6 +109,18 @@ static char *get_clock(int argc, char **argv)
 	return NULL;
 }
 
+static void trace_print_connection(int fd, bool network)
+{
+	int ret;
+
+	if (network)
+		ret = trace_net_print_connection(fd);
+	else
+		ret = trace_vsock_print_connection(fd);
+	if (ret < 0)
+		tracecmd_debug("Could not print connection fd:%d\n", fd);
+}
+
 static void agent_handle(int sd, int nr_cpus, int page_size, bool network)
 {
 	struct tracecmd_tsync_protos *tsync_protos = NULL;
@@ -273,6 +285,8 @@ static void agent_serve(unsigned int port, bool do_daemon, bool network)
 				continue;
 			die("accept");
 		}
+		if (tracecmd_get_debug())
+			trace_print_connection(cd, network);
 
 		if (handler_pid)
 			goto busy;
diff --git a/tracecmd/trace-listen.c b/tracecmd/trace-listen.c
index 3b446acea972..b7be761d032e 100644
--- a/tracecmd/trace-listen.c
+++ b/tracecmd/trace-listen.c
@@ -756,6 +756,28 @@ static int do_fork(int cfd)
 	return 0;
 }
 
+int trace_net_print_connection(int fd)
+{
+	char host[NI_MAXHOST], service[NI_MAXSERV];
+	struct sockaddr_storage net_addr;
+	socklen_t addr_len;
+
+	addr_len = sizeof(net_addr);
+	if (getpeername(fd, (struct sockaddr *)&net_addr, &addr_len))
+		return -1;
+
+	if (getnameinfo((struct sockaddr *)&net_addr, addr_len,
+			host, NI_MAXHOST,
+			service, NI_MAXSERV, NI_NUMERICSERV))
+		return -1;
+
+	if (tracecmd_get_debug())
+		tracecmd_debug("Connected to %s:%s fd:%d\n", host, service, fd);
+	else
+		tracecmd_plog("Connected to %s:%s\n", host, service);
+	return 0;
+}
+
 static int do_connection(int cfd, struct sockaddr *addr,
 			  socklen_t addr_len)
 {
diff --git a/tracecmd/trace-vsock.c b/tracecmd/trace-vsock.c
index d18ecb45004e..39294e7a2a3c 100644
--- a/tracecmd/trace-vsock.c
+++ b/tracecmd/trace-vsock.c
@@ -94,6 +94,26 @@ int get_vsocket_params(int fd, unsigned int *lcid, unsigned int *rcid)
 	return 0;
 }
 
+int trace_vsock_print_connection(int fd)
+{
+	struct sockaddr_vm vm_addr;
+	socklen_t addr_len;
+	int cid, port;
+
+	addr_len = sizeof(vm_addr);
+	if (getpeername(fd, (struct sockaddr *)&vm_addr, &addr_len))
+		return -1;
+	if (vm_addr.svm_family != AF_VSOCK)
+		return -1;
+	cid = vm_addr.svm_cid;
+	port = vm_addr.svm_port;
+	if (tracecmd_get_debug())
+		tracecmd_debug("Connected to @%u:%u fd:%d\n", cid, port, fd);
+	else
+		tracecmd_plog("Connected to @%u:%u\n", cid, port);
+	return 0;
+}
+
 static int try_splice_read_vsock(void)
 {
 	int ret, sd, brass[2];
-- 
2.35.1

[PATCH v2 8/9] trace-cmd: Override tracecmd_debug() to show thread id

[Steven Rostedt]

From: "Steven Rostedt (Google)" <rostedt@goodmis.org>

To differentiate threads, have tracecmd_debug() show the thread id of
the debug prints.

Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
 tracecmd/trace-cmd.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/tracecmd/trace-cmd.c b/tracecmd/trace-cmd.c
index a83a8d0bb9e4..3f3e69a7f1e2 100644
--- a/tracecmd/trace-cmd.c
+++ b/tracecmd/trace-cmd.c
@@ -10,12 +10,17 @@
 #include <dirent.h>
 #include <errno.h>
 #include <stdlib.h>
+#include <sys/syscall.h>
 
 #include "trace-local.h"
 
 int silence_warnings;
 int show_status;
 
+#ifndef gettid
+#define gettid() syscall(__NR_gettid)
+#endif
+
 void warning(const char *fmt, ...)
 {
 	va_list ap;
@@ -45,6 +50,19 @@ void *malloc_or_die(unsigned int size)
 	return data;
 }
 
+void tracecmd_debug(const char *fmt, ...)
+{
+	va_list ap;
+
+	if (!tracecmd_get_debug())
+		return;
+
+	va_start(ap, fmt);
+	printf("[%d] ", (int)gettid());
+	vprintf(fmt, ap);
+	va_end(ap);
+}
+
 static struct trace_log_severity {
 	int		id;
 	const char	*name;
-- 
2.35.1

[PATCH v2 9/9] trace-cmd agent: Have agent work without vsockets available

[Steven Rostedt]

From: "Steven Rostedt (Google)" <rostedt@goodmis.org>

Compile agent code without vsockets available to allow it to work with
networking. This includes allowing the ptp time synchronization protocol
to be used.

Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
---
 lib/trace-cmd/Makefile                    | 2 +-
 lib/trace-cmd/include/trace-tsync-local.h | 6 +-----
 tracecmd/Makefile                         | 5 ++---
 tracecmd/trace-cmd.c                      | 2 --
 tracecmd/trace-usage.c                    | 2 --
 5 files changed, 4 insertions(+), 13 deletions(-)

diff --git a/lib/trace-cmd/Makefile b/lib/trace-cmd/Makefile
index da0ad4deeb4f..9374b163b5f3 100644
--- a/lib/trace-cmd/Makefile
+++ b/lib/trace-cmd/Makefile
@@ -21,8 +21,8 @@ ifeq ($(PERF_DEFINED), 1)
 OBJS += trace-perf.o
 endif
 OBJS += trace-timesync.o
-ifeq ($(VSOCK_DEFINED), 1)
 OBJS += trace-timesync-ptp.o
+ifeq ($(VSOCK_DEFINED), 1)
 OBJS += trace-timesync-kvm.o
 endif
 OBJS += trace-compress.o
diff --git a/lib/trace-cmd/include/trace-tsync-local.h b/lib/trace-cmd/include/trace-tsync-local.h
index 885c9f51d891..5bbc1db622c4 100644
--- a/lib/trace-cmd/include/trace-tsync-local.h
+++ b/lib/trace-cmd/include/trace-tsync-local.h
@@ -64,15 +64,11 @@ int tracecmd_tsync_proto_register(const char *proto_name, int accuracy, int role
 					      long long *, long long *, long long*,
 					      long long *, unsigned int));
 int tracecmd_tsync_proto_unregister(char *proto_name);
+int ptp_clock_sync_register(void);
 
 #ifdef VSOCK
-int ptp_clock_sync_register(void);
 int kvm_clock_sync_register(void);
 #else
-static inline int ptp_clock_sync_register(void)
-{
-	return 0;
-}
 static inline int kvm_clock_sync_register(void)
 {
 	return 0;
diff --git a/tracecmd/Makefile b/tracecmd/Makefile
index 13f7776e8e45..0114948fe385 100644
--- a/tracecmd/Makefile
+++ b/tracecmd/Makefile
@@ -37,11 +37,10 @@ TRACE_CMD_OBJS += trace-dump.o
 TRACE_CMD_OBJS += trace-clear.o
 TRACE_CMD_OBJS += trace-vm.o
 TRACE_CMD_OBJS += trace-convert.o
-TRACE_CMD_OBJS += trace-vsock.o
-
-ifeq ($(VSOCK_DEFINED), 1)
 TRACE_CMD_OBJS += trace-agent.o
 TRACE_CMD_OBJS += trace-setup-guest.o
+ifeq ($(VSOCK_DEFINED), 1)
+TRACE_CMD_OBJS += trace-vsock.o
 endif
 
 ALL_OBJS := $(TRACE_CMD_OBJS:%.o=$(bdir)/%.o)
diff --git a/tracecmd/trace-cmd.c b/tracecmd/trace-cmd.c
index 3f3e69a7f1e2..69800d26c5ee 100644
--- a/tracecmd/trace-cmd.c
+++ b/tracecmd/trace-cmd.c
@@ -127,10 +127,8 @@ struct command commands[] = {
 	{"hist", trace_hist},
 	{"mem", trace_mem},
 	{"listen", trace_listen},
-#ifdef VSOCK
 	{"agent", trace_agent},
 	{"setup-guest", trace_setup_guest},
-#endif
 	{"split", trace_split},
 	{"restore", trace_restore},
 	{"stack", trace_stack},
diff --git a/tracecmd/trace-usage.c b/tracecmd/trace-usage.c
index ec6376557394..88eac10a4e75 100644
--- a/tracecmd/trace-usage.c
+++ b/tracecmd/trace-usage.c
@@ -325,7 +325,6 @@ static struct usage_help usage_help[] = {
 		"          -l logfile to write messages to.\n"
 		"          --verbose 'level' Set the desired log level\n"
 	},
-#ifdef VSOCK
 	{
 		"agent",
 		"listen on a vsocket for trace clients",
@@ -344,7 +343,6 @@ static struct usage_help usage_help[] = {
 		"          -g FIFOs group owner\n"
 		"          -a Attach FIFOs to guest VM config\n"
 	},
-#endif
 	{
 		"list",
 		"list the available events, plugins or options",
-- 
2.35.1

Re: [PATCH v2 1/9] trace-cmd record: Move port_type into instance

[Tzvetomir Stoyanov]

On Mon, Apr 18, 2022 at 10:11 AM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> From: "Steven Rostedt (Google)" <rostedt@goodmis.org>
>
> Instead of one global method of communication, have the "port_type" be
> part of the instance. This will allow for different instances to have
> different connection types.
>
> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
> ---
>  tracecmd/include/trace-local.h |  1 +
>  tracecmd/trace-record.c        | 41 ++++++++++++++++++----------------
>  2 files changed, 23 insertions(+), 19 deletions(-)
>
> diff --git a/tracecmd/include/trace-local.h b/tracecmd/include/trace-local.h
> index aa1cb8d939bd..2008449bab27 100644
> --- a/tracecmd/include/trace-local.h
> +++ b/tracecmd/include/trace-local.h
> @@ -285,6 +285,7 @@ struct buffer_instance {
>         int                     *fds;
>         bool                    use_fifos;
>
> +       enum port_type          port_type;

I would suggest setting the default value of port_type explicitly in
allocate_instance(). Even though USE_UDP is 0, it is more clear. The
first time when I went through these changes, I was wondering why UDP
is not set at all. Then found that it was a static boolean in the
previous implementation, defaulting to false.


>         int                     tsync_loop_interval;
>         struct tracecmd_time_sync *tsync;
>  };
> diff --git a/tracecmd/trace-record.c b/tracecmd/trace-record.c
> index 674ec2c3ba65..14ba4ac5dc9e 100644
> --- a/tracecmd/trace-record.c
> +++ b/tracecmd/trace-record.c
> @@ -90,8 +90,6 @@ static bool fork_process;
>  /* Max size to let a per cpu file get */
>  static int max_kb;
>
> -static enum port_type port_type;
> -
>  static int do_ptrace;
>
>  static int filter_task;
> @@ -3119,7 +3117,7 @@ static void finish(int sig)
>         finished = 1;
>  }
>
> -static int connect_port(const char *host, unsigned int port)
> +static int connect_port(const char *host, unsigned int port, enum port_type type)
>  {
>         struct addrinfo hints;
>         struct addrinfo *results, *rp;
> @@ -3128,17 +3126,17 @@ static int connect_port(const char *host, unsigned int port)
>
>         snprintf(buf, BUFSIZ, "%u", port);
>
> -       if (port_type == USE_VSOCK)
> +       if (type == USE_VSOCK)
>                 return trace_vsock_open(atoi(host), port);
>
>         memset(&hints, 0, sizeof(hints));
>         hints.ai_family = AF_UNSPEC;
> -       hints.ai_socktype = port_type == USE_TCP ? SOCK_STREAM : SOCK_DGRAM;
> +       hints.ai_socktype = type == USE_TCP ? SOCK_STREAM : SOCK_DGRAM;
>
>         s = getaddrinfo(host, buf, &hints, &results);
>         if (s != 0)
>                 die("connecting to %s server %s:%s",
> -                   port_type == USE_TCP ? "TCP" : "UDP", host, buf);
> +                   type == USE_TCP ? "TCP" : "UDP", host, buf);
>
>         for (rp = results; rp != NULL; rp = rp->ai_next) {
>                 sfd = socket(rp->ai_family, rp->ai_socktype,
> @@ -3152,7 +3150,7 @@ static int connect_port(const char *host, unsigned int port)
>
>         if (rp == NULL)
>                 die("Can not connect to %s server %s:%s",
> -                   port_type == USE_TCP ? "TCP" : "UDP", host, buf);
> +                   type == USE_TCP ? "TCP" : "UDP", host, buf);
>
>         freeaddrinfo(results);
>
> @@ -3359,7 +3357,8 @@ static int create_recorder(struct buffer_instance *instance, int cpu,
>                         else
>                                 fd = do_accept(instance->fds[cpu]);
>                 } else {
> -                       fd = connect_port(host, instance->client_ports[cpu]);
> +                       fd = connect_port(host, instance->client_ports[cpu],
> +                                         instance->port_type);
>                 }
>                 if (fd < 0)
>                         die("Failed connecting to client");
> @@ -3414,8 +3413,9 @@ static void check_first_msg_from_server(struct tracecmd_msg_handle *msg_handle)
>  }
>
>  static void communicate_with_listener_v1(struct tracecmd_msg_handle *msg_handle,
> -                                        unsigned int **client_ports)
> +                                        struct buffer_instance *instance)
>  {
> +       unsigned int *client_ports;
>         char buf[BUFSIZ];
>         ssize_t n;
>         int cpu, i;
> @@ -3442,11 +3442,11 @@ static void communicate_with_listener_v1(struct tracecmd_msg_handle *msg_handle,
>         /* TODO, test for ipv4 */
>         if (page_size >= UDP_MAX_PACKET) {
>                 warning("page size too big for UDP using TCP in live read");
> -               port_type = USE_TCP;
> +               instance->port_type = USE_TCP;
>                 msg_handle->flags |= TRACECMD_MSG_FL_USE_TCP;
>         }
>
> -       if (port_type == USE_TCP) {
> +       if (instance->port_type == USE_TCP) {
>                 /* Send one option */
>                 write(msg_handle->fd, "1", 2);
>                 /* Size 4 */
> @@ -3457,8 +3457,8 @@ static void communicate_with_listener_v1(struct tracecmd_msg_handle *msg_handle,
>                 /* No options */
>                 write(msg_handle->fd, "0", 2);
>
> -       *client_ports = malloc(local_cpu_count * sizeof(*client_ports));
> -       if (!*client_ports)
> +       client_ports = malloc(local_cpu_count * sizeof(*client_ports));
> +       if (!client_ports)
>                 die("Failed to allocate client ports for %d cpus", local_cpu_count);
>
>         /*
> @@ -3476,8 +3476,10 @@ static void communicate_with_listener_v1(struct tracecmd_msg_handle *msg_handle,
>                 if (i == BUFSIZ)
>                         die("read bad port number");
>                 buf[i] = 0;
> -               (*client_ports)[cpu] = atoi(buf);
> +               client_ports[cpu] = atoi(buf);
>         }
> +
> +       instance->client_ports = client_ports;
>  }
>
>  static void communicate_with_listener_v3(struct tracecmd_msg_handle *msg_handle,
> @@ -3609,10 +3611,11 @@ static int connect_ip(char *thost)
>  static struct tracecmd_msg_handle *setup_network(struct buffer_instance *instance)
>  {
>         struct tracecmd_msg_handle *msg_handle = NULL;
> +       enum port_type type = instance->port_type;
>         int sfd;
>
>  again:
> -       switch (port_type) {
> +       switch (type) {
>         case USE_VSOCK:
>                 sfd = connect_vsock(host);
>                 break;
> @@ -3634,7 +3637,7 @@ again:
>                 msg_handle->version = V3_PROTOCOL;
>         }
>
> -       switch (port_type) {
> +       switch (type) {
>         case USE_TCP:
>                 msg_handle->flags |= TRACECMD_MSG_FL_USE_TCP;
>                 break;
> @@ -3657,7 +3660,7 @@ again:
>         }
>
>         if (msg_handle->version == V1_PROTOCOL)
> -               communicate_with_listener_v1(msg_handle, &instance->client_ports);
> +               communicate_with_listener_v1(msg_handle, instance);
>
>         return msg_handle;
>  }
> @@ -6514,7 +6517,7 @@ static void parse_record_options(int argc,
>                         if (ctx->output)
>                                 die("-V incompatible with -o");
>                         host = optarg;
> -                       port_type = USE_VSOCK;
> +                       ctx->instance->port_type = USE_VSOCK;
>                         break;
>                 case 'm':
>                         if (max_kb)
> @@ -6532,7 +6535,7 @@ static void parse_record_options(int argc,
>                         if (IS_EXTRACT(ctx))
>                                 ctx->topt = 1; /* Extract top instance also */
>                         else
> -                               port_type = USE_TCP;
> +                               ctx->instance->port_type = USE_TCP;
>                         break;
>                 case 'b':
>                         check_instance_die(ctx->instance, "-b");
> --
> 2.35.1
>


--
Tzvetomir (Ceco) Stoyanov
VMware Open Source Technology Center

Re: [PATCH v2 4/9] trace-cmd agent: Allow for ip connections from the agent

[Tzvetomir Stoyanov]

On Mon, Apr 18, 2022 at 11:28 AM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> From: "Steven Rostedt (Google)" <rostedt@goodmis.org>
>
> Add a -N option to trace-cmd agent to listen on a network socket that
> can be used by trace-cmd record -A to connect with.
>

I have concerns about exposing the agent over the network, without any
client authentication. It runs with root privileges, the kernel
tracing data will be exposed to anyone over the network. At least,
this should be written clearly in the documentation and even should be
printed by "trace-cmd agent -N ...". I saw your patch that adds a name
or IP address of the client as parameter, it is some security, but the
tracing data itself still flows unencrypted over the network and could
be visible to anyone.


> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
> ---
>  tracecmd/include/trace-local.h |  6 +++
>  tracecmd/trace-agent.c         | 89 ++++++++++++++++++++++++++--------
>  tracecmd/trace-listen.c        | 58 ++++++++++++++--------
>  3 files changed, 113 insertions(+), 40 deletions(-)
>
> diff --git a/tracecmd/include/trace-local.h b/tracecmd/include/trace-local.h
> index afbcab422c2f..fdb0d887557f 100644
> --- a/tracecmd/include/trace-local.h
> +++ b/tracecmd/include/trace-local.h
> @@ -306,6 +306,12 @@ extern struct buffer_instance *first_instance;
>  #define is_guest(instance)     ((instance)->flags & BUFFER_FL_GUEST)
>  #define is_network(instance)   ((instance)->flags & BUFFER_FL_NETWORK)
>
> +#define START_PORT_SEARCH 1500
> +#define MAX_PORT_SEARCH 6000
> +
> +int trace_net_make(int port, enum port_type type);
> +int trace_net_search(int start_port, int *sfd, enum port_type type);
> +
>  struct buffer_instance *allocate_instance(const char *name);
>  void add_instance(struct buffer_instance *instance, int cpu_count);
>  void update_first_instance(struct buffer_instance *instance, int topt);
> diff --git a/tracecmd/trace-agent.c b/tracecmd/trace-agent.c
> index 719d7126ca67..cd201ad948ad 100644
> --- a/tracecmd/trace-agent.c
> +++ b/tracecmd/trace-agent.c
> @@ -41,6 +41,32 @@ static void make_vsocks(int nr, int *fds, unsigned int *ports)
>         }
>  }
>
> +static void make_net(int nr, int *fds, unsigned int *ports)
> +{
> +       int port;
> +       int i, fd;
> +       int start_port = START_PORT_SEARCH;
> +
> +       for (i = 0; i < nr; i++) {
> +               port = trace_net_search(start_port, &fd, USE_TCP);
> +               if (port < 0)
> +                       die("Failed to open socket");
> +               if (listen(fd, 5) < 0)
> +                       die("Failed to listen on port %d\n", port);
> +               fds[i] = fd;
> +               ports[i] = port;
> +               start_port = port + 1;
> +       }
> +}
> +
> +static void make_sockets(int nr, int *fds, unsigned int *ports, bool network)
> +{
> +       if (network)
> +               return make_net(nr, fds, ports);
> +       else
> +               return make_vsocks(nr, fds, ports);
> +}
> +
>  static int open_agent_fifos(int nr_cpus, int *fds)
>  {
>         char path[PATH_MAX];
> @@ -80,7 +106,7 @@ static char *get_clock(int argc, char **argv)
>         return NULL;
>  }
>
> -static void agent_handle(int sd, int nr_cpus, int page_size)
> +static void agent_handle(int sd, int nr_cpus, int page_size, bool network)
>  {
>         struct tracecmd_tsync_protos *tsync_protos = NULL;
>         struct tracecmd_time_sync *tsync = NULL;
> @@ -117,17 +143,31 @@ static void agent_handle(int sd, int nr_cpus, int page_size)
>                 use_fifos = false;
>
>         if (!use_fifos)
> -               make_vsocks(nr_cpus, fds, ports);
> +               make_sockets(nr_cpus, fds, ports, network);
>         if (tsync_protos && tsync_protos->names) {
> -               if (get_vsocket_params(msg_handle->fd, &local_id,
> -                                      &remote_id)) {
> -                       warning("Failed to get local and remote ids");
> -                       /* Just make something up */
> -                       remote_id = -1;
> -                       local_id = -2;
> +               if (network) {
> +                       /* For now just use something */
> +                       remote_id = 2;
> +                       local_id = 1;
> +                       tsync_port = trace_net_search(START_PORT_SEARCH, &fd, USE_TCP);
> +                       if (listen(fd, 5) < 0)
> +                               die("Failed to listen on %d\n", tsync_port);
> +               } else {
> +                       if (get_vsocket_params(msg_handle->fd, &local_id,
> +                                              &remote_id)) {
> +                               warning("Failed to get local and remote ids");
> +                               /* Just make something up */
> +                               remote_id = -1;
> +                               local_id = -2;
> +                       }
> +                       fd = trace_vsock_make_any();
> +                       if (fd >= 0 &&
> +                           trace_vsock_get_port(fd, &tsync_port) < 0) {
> +                               close(fd);
> +                               fd = -1;
> +                       }
>                 }
> -               fd = trace_vsock_make_any();
> -               if (fd >= 0 && trace_vsock_get_port(fd, &tsync_port) >= 0) {
> +               if (fd >= 0) {
>                         tsync = tracecmd_tsync_with_host(fd, tsync_protos,
>                                                          get_clock(argc, argv),
>                                                          remote_id, local_id);
> @@ -193,7 +233,7 @@ static pid_t do_fork()
>         return fork();
>  }
>
> -static void agent_serve(unsigned int port, bool do_daemon)
> +static void agent_serve(unsigned int port, bool do_daemon, bool network)
>  {
>         int sd, cd, nr_cpus;
>         unsigned int cid;
> @@ -204,14 +244,21 @@ static void agent_serve(unsigned int port, bool do_daemon)
>         nr_cpus = tracecmd_count_cpus();
>         page_size = getpagesize();
>
> -       sd = trace_vsock_make(port);
> +       if (network) {
> +               sd = trace_net_make(port, USE_TCP);
> +               if (listen(sd, 5) < 0)
> +                       die("Failed to listen on %d\n", port);
> +       } else
> +               sd = trace_vsock_make(port);
>         if (sd < 0)
> -               die("Failed to open vsocket");
> +               die("Failed to open socket");
>         tracecmd_tsync_init();
>
> -       cid = trace_vsock_local_cid();
> -       if (cid >= 0)
> -               printf("listening on @%u:%u\n", cid, port);
> +       if (!network) {
> +               cid = trace_vsock_local_cid();
> +               if (cid >= 0)
> +                       printf("listening on @%u:%u\n", cid, port);
> +       }
>
>         if (do_daemon && daemon(1, 0))
>                 die("daemon");
> @@ -231,7 +278,7 @@ static void agent_serve(unsigned int port, bool do_daemon)
>                 if (pid == 0) {
>                         close(sd);
>                         signal(SIGCHLD, SIG_DFL);
> -                       agent_handle(cd, nr_cpus, page_size);
> +                       agent_handle(cd, nr_cpus, page_size, network);
>                 }
>                 if (pid > 0)
>                         handler_pid = pid;
> @@ -250,6 +297,7 @@ void trace_agent(int argc, char **argv)
>  {
>         bool do_daemon = false;
>         unsigned int port = TRACE_AGENT_DEFAULT_PORT;
> +       bool network = false;
>
>         if (argc < 2)
>                 usage(argv);
> @@ -267,7 +315,7 @@ void trace_agent(int argc, char **argv)
>                         {NULL, 0, NULL, 0}
>                 };
>
> -               c = getopt_long(argc-1, argv+1, "+hp:D",
> +               c = getopt_long(argc-1, argv+1, "+hp:DN",
>                                 long_options, &option_index);
>                 if (c == -1)
>                         break;
> @@ -275,6 +323,9 @@ void trace_agent(int argc, char **argv)
>                 case 'h':
>                         usage(argv);
>                         break;
> +               case 'N':
> +                       network = true;
> +                       break;
>                 case 'p':
>                         port = atoi(optarg);
>                         break;
> @@ -296,5 +347,5 @@ void trace_agent(int argc, char **argv)
>         if (optind < argc-1)
>                 usage(argv);
>
> -       agent_serve(port, do_daemon);
> +       agent_serve(port, do_daemon, network);
>  }
> diff --git a/tracecmd/trace-listen.c b/tracecmd/trace-listen.c
> index 2338173fc8a6..8476fa51dadd 100644
> --- a/tracecmd/trace-listen.c
> +++ b/tracecmd/trace-listen.c
> @@ -234,52 +234,68 @@ static int setup_vsock_port(int start_port, int *sfd)
>         return start_port;
>  }
>
> -#define START_PORT_SEARCH 1500
> -#define MAX_PORT_SEARCH 6000
> -
> -static int bind_a_port(int start_port, int *sfd, enum port_type type)
> +int trace_net_make(int port, enum port_type type)
>  {
>         struct addrinfo hints;
>         struct addrinfo *result, *rp;
>         char buf[BUFSIZ];
> +       int sd;
>         int s;
> -       int num_port = start_port;
>
> -       if (type == USE_VSOCK)
> -               return setup_vsock_port(start_port, sfd);
> - again:
> -       snprintf(buf, BUFSIZ, "%d", num_port);
> +       snprintf(buf, BUFSIZ, "%d", port);
>
>         memset(&hints, 0, sizeof(hints));
>         hints.ai_family = AF_UNSPEC;
> -       hints.ai_socktype = type == USE_TCP ? SOCK_STREAM : SOCK_DGRAM;
>         hints.ai_flags = AI_PASSIVE;
>
> +       switch (type) {
> +       case USE_TCP:
> +               hints.ai_socktype = SOCK_STREAM;
> +               break;
> +       case USE_UDP:
> +               hints.ai_socktype = SOCK_DGRAM;
> +               break;
> +       default:
> +               return -1;
> +       }
> +
>         s = getaddrinfo(NULL, buf, &hints, &result);
>         if (s != 0)
>                 pdie("getaddrinfo: error opening socket");
>
>         for (rp = result; rp != NULL; rp = rp->ai_next) {
> -               *sfd = socket(rp->ai_family, rp->ai_socktype,
> -                             rp->ai_protocol);
> -               if (*sfd < 0)
> +               sd = socket(rp->ai_family, rp->ai_socktype,
> +                           rp->ai_protocol);
> +               if (sd < 0)
>                         continue;
>
> -               if (bind(*sfd, rp->ai_addr, rp->ai_addrlen) == 0)
> +               if (bind(sd, rp->ai_addr, rp->ai_addrlen) == 0)
>                         break;
>
> -               close(*sfd);
> +               close(sd);
>         }
> +       freeaddrinfo(result);
> +
> +       if (rp == NULL)
> +               return -1;
> +
> +       return sd;
> +}
> +
> +int trace_net_search(int start_port, int *sfd, enum port_type type)
> +{
> +       int num_port = start_port;
>
> -       if (rp == NULL) {
> -               freeaddrinfo(result);
> +       if (type == USE_VSOCK)
> +               return setup_vsock_port(start_port, sfd);
> + again:
> +       *sfd = trace_net_make(num_port, type);
> +       if (*sfd < 0) {
>                 if (++num_port > MAX_PORT_SEARCH)
>                         pdie("No available ports to bind");
>                 goto again;
>         }
>
> -       freeaddrinfo(result);
> -
>         return num_port;
>  }
>
> @@ -309,10 +325,10 @@ static int open_port(const char *node, const char *port, int *pid,
>         int num_port;
>
>         /*
> -        * bind_a_port() currently does not return an error, but if that
> +        * trace_net_search() currently does not return an error, but if that
>          * changes in the future, we have a check for it now.
>          */
> -       num_port = bind_a_port(start_port, &sfd, type);
> +       num_port = trace_net_search(start_port, &sfd, type);
>         if (num_port < 0)
>                 return num_port;
>
> --
> 2.35.1
>


--
Tzvetomir (Ceco) Stoyanov
VMware Open Source Technology Center

Re: [PATCH v2 1/9] trace-cmd record: Move port_type into instance

[Steven Rostedt]

On Mon, 18 Apr 2022 13:22:21 +0300
Tzvetomir Stoyanov <tz.stoyanov@gmail.com> wrote:

> > +++ b/tracecmd/include/trace-local.h
> > @@ -285,6 +285,7 @@ struct buffer_instance {
> >         int                     *fds;
> >         bool                    use_fifos;
> >
> > +       enum port_type          port_type;  
> 
> I would suggest setting the default value of port_type explicitly in
> allocate_instance(). Even though USE_UDP is 0, it is more clear. The
> first time when I went through these changes, I was wondering why UDP
> is not set at all. Then found that it was a static boolean in the
> previous implementation, defaulting to false.

I could add a comment. I purposely had USE_UDP to zero for this purpose.
This is a common trick to do in the kernel as well.

But it should have a comment stating that here. I'll update it in v3.

Thanks for the review.

-- Steve

Re: [PATCH v2 4/9] trace-cmd agent: Allow for ip connections from the agent

[Steven Rostedt]

On Mon, 18 Apr 2022 13:22:51 +0300
Tzvetomir Stoyanov <tz.stoyanov@gmail.com> wrote:

> On Mon, Apr 18, 2022 at 11:28 AM Steven Rostedt <rostedt@goodmis.org> wrote:
> >
> > From: "Steven Rostedt (Google)" <rostedt@goodmis.org>
> >
> > Add a -N option to trace-cmd agent to listen on a network socket that
> > can be used by trace-cmd record -A to connect with.
> >  
> 
> I have concerns about exposing the agent over the network, without any
> client authentication. It runs with root privileges, the kernel
> tracing data will be exposed to anyone over the network. At least,
> this should be written clearly in the documentation and even should be
> printed by "trace-cmd agent -N ...". I saw your patch that adds a name
> or IP address of the client as parameter, it is some security, but the
> tracing data itself still flows unencrypted over the network and could
> be visible to anyone.

True. But it's similar to telnet. The use case for this option is for local
networks where you expect to "trust" the network.

I'm fine with adding more warnings and stating that this is "NOT SECURE" in
big letters to let people know the dangers of it. I'm hoping to have more
authentication in the future.

I could even have it print a warning "UNSECURE CONNECTION" or something to
enforce that this is not something you want to use in any scenario.

-- Steve