[Xenomai-core] [RFC][PATCH 1/2] real-time print library

[Xenomai-core] [RFC][PATCH 1/2] real-time print library

[Jan Kiszka]

[-- Attachment #1.1: Type: text/plain, Size: 1574 bytes --]

This is a stand-alone real-time library for printf services. It is
embedded into the Xenomai user space part but actually doesn't depend
on any Xenomai service, just using plain Linux POSIX.

The librtprint API looks much like the printf(3) man page:

rt_vfprintf
rt_vprintf
rt_fprintf
rt_printf

Moreover it provides a few setup/cleanup services as you can grab from
the code and the included example.

The basic idea of librtprint is to keep the print side as cheap as
possible: no locks, no syscalls at all. Each thread that uses rt_printf
& friends has its own local ring buffer. A central (per process) non-RT
output thread takes care of forwarding the content of all thread ring
buffers to the output streams. The message order is preserved loosely by
a global sequence counter (arch-dependent CPU-wide or system-wide
atomic_inc_return might be used in the future). The output thread uses
periodic polling (default: 10 Hz) to avoid the need for special
signalling mechanisms.

Further features:
 o Explicit or on-demand thread buffer creation
 o Override of default buffer size and polling period via environment
   variables ("RT_PRINT_BUFFER" and "RT_PRINT_PERIOD")
 o Support for buffer names (useful for the following patch e.g.)
 o Target output stream can be specified for each print invocation
 o Breaks all builds except for x86 (missing ubarrier.h headers) :o)

The code is stuffed into src/rtprint for now as patch 2/2 will need this
lib to be built before the skin libraries. Better suggestions are
welcome though.

Jan

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: librtprint.patch --]
[-- Type: text/x-patch; name="librtprint.patch", Size: 21882 bytes --]

---
 configure.in                 |    1 
 examples/native/Makefile     |    8 +
 examples/native/rtprint.c    |   48 +++++++
 include/Makefile.am          |    5 
 include/asm-i386/Makefile.am |    2 
 include/asm-i386/ubarrier.h  |   26 ++++
 include/rtprint.h            |   59 +++++++++
 src/Makefile.am              |    2 
 src/rtprint/Makefile.am      |   12 +
 src/rtprint/api.c            |  273 +++++++++++++++++++++++++++++++++++++++++++
 src/rtprint/init.c           |   75 +++++++++++
 src/rtprint/internal.h       |   69 ++++++++++
 src/rtprint/output.c         |   92 ++++++++++++++
 13 files changed, 669 insertions(+), 3 deletions(-)

Index: xenomai/configure.in
===================================================================
--- xenomai.orig/configure.in
+++ xenomai/configure.in
@@ -611,6 +611,7 @@ AC_CONFIG_FILES([ \
 	scripts/xeno-load \
 	scripts/xeno-test \
 	src/Makefile \
+	src/rtprint/Makefile \
 	src/skins/Makefile \
        	src/skins/posix/Makefile \
        	src/skins/native/Makefile \
Index: xenomai/examples/native/Makefile
===================================================================
--- xenomai.orig/examples/native/Makefile
+++ xenomai/examples/native/Makefile
@@ -1,7 +1,7 @@
 ###### CONFIGURATION ######
 
 ### List of applications to be build
-APPLICATIONS = trivial-periodic sigxcpu
+APPLICATIONS = trivial-periodic sigxcpu rtprint
 
 ### Note: to override the search path for the xeno-config script, use "make XENO=..."
 
@@ -44,6 +44,12 @@ endif
 
 
 
+###### SPECIAL TARGET RULES ######
+rtprint: rtprint.c
+	$(CC) $(CFLAGS) $? $(LDFLAGS) -lrtprint -o $@
+
+
+
 ###### KERNEL MODULE BUILD (no change required normally) ######
 ifneq ($(MODULES),)
 
Index: xenomai/examples/native/rtprint.c
===================================================================
--- /dev/null
+++ xenomai/examples/native/rtprint.c
@@ -0,0 +1,48 @@
+#include <stdio.h>
+#include <sys/mman.h>
+#include <native/task.h>
+#include <rtprint.h>
+
+void task2_func(void *arg)
+{
+	int i = 0;
+
+	rt_printf("This triggers auto-init of rt_print for the "
+		  "calling thread.\n"
+		  "A last switch to secondary mode can occure here, "
+		  "but future invocations of rt_printf are safe.\n");
+
+	rt_task_set_mode(0, T_WARNSW, NULL);
+
+	while (1) {
+		rt_task_sleep(3333333LL);
+		rt_fprintf(stderr, "%s: #%d Yet another RT printer - "
+			   "but to stderr.\n", rt_print_buffer_name(), ++i);
+	}
+}
+
+int main(int argc, char **argv)
+{
+	RT_TASK task1, task2;
+	int i = 0;
+
+	mlockall(MCL_CURRENT|MCL_FUTURE);
+
+	/* Perform auto-init of rt_print buffers if the task doesn't do so */
+	rt_print_auto_init(1);
+
+	/* Initialise the rt_print buffer for this task explicitly */
+	rt_print_init(4096, "Task 1");
+
+	rt_task_shadow(&task1, "Task 1", 10, 0);
+	rt_task_spawn(&task2, "Task 2", 0, 11, 0, task2_func, NULL);
+
+	/* To demonstrate that rt_printf is safe */
+	rt_task_set_mode(0, T_WARNSW, NULL);
+
+	while (1) {
+		rt_task_sleep(5000000LL);
+		rt_printf("%s: #%d Hello RT world!\n",
+			  rt_print_buffer_name(), ++i);
+	}
+}
Index: xenomai/include/Makefile.am
===================================================================
--- xenomai.orig/include/Makefile.am
+++ xenomai/include/Makefile.am
@@ -1,3 +1,8 @@
+includedir = $(prefix)/include
+
+include_HEADERS = \
+	rtprint.h
+
 nodist_include_HEADERS=$(CONFIG_HEADER)
 
 SUBDIRS = \
Index: xenomai/include/asm-i386/Makefile.am
===================================================================
--- xenomai.orig/include/asm-i386/Makefile.am
+++ xenomai/include/asm-i386/Makefile.am
@@ -1,6 +1,6 @@
 includedir = $(prefix)/include/asm-i386
 
 include_HEADERS = atomic.h calibration.h features.h \
-		hal.h smi.h syscall.h system.h wrappers.h fptest.h
+		hal.h smi.h syscall.h system.h wrappers.h fptest.h ubarrier.h
 
 SUBDIRS = bits
Index: xenomai/include/asm-i386/ubarrier.h
===================================================================
--- /dev/null
+++ xenomai/include/asm-i386/ubarrier.h
@@ -0,0 +1,26 @@
+/*
+ * Copyright (C) 2007 Jan Kiszka <jan.kiszka@domain.hid>.
+ *
+ * Xenomai is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published
+ * by the Free Software Foundation; either version 2 of the License,
+ * or (at your option) any later version.
+ *
+ * Xenomai is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with Xenomai; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
+ * 02111-1307, USA.
+ */
+
+#ifndef _XENO_ASM_I386_UBARRIER_H
+#define _XENO_ASM_I386_UBARRIER_H
+
+#define rmb()	__asm__ __volatile__ ("lock; addl $0,0(%%esp)": : :"memory")
+#define wmb()	__asm__ __volatile__ ("": : :"memory")
+
+#endif /* _XENO_ASM_I386_UBARRIER_H */
Index: xenomai/include/rtprint.h
===================================================================
--- /dev/null
+++ xenomai/include/rtprint.h
@@ -0,0 +1,59 @@
+/*
+ * Copyright (C) 2007 Jan Kiszka <jan.kiszka@domain.hid>.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA.
+ */
+
+#ifndef _RTPRINT_H
+#define _RTPRINT_H
+
+#ifdef __KERNEL__
+
+#define rt_printf(format, ...)	printk(format __VA_ARGS__)
+
+static inline int rt_print_init(size_t buffer_size, const char *buffer_name)
+{
+	return 0;
+}
+
+#define rt_print_cleanup()	do { } while (0)
+
+static inline void rt_print_auto_init(int enable)
+{
+}
+
+static inline const char *rt_print_buffer_name(void)
+{
+	return "<unknown>";
+}
+
+#else /* !__KERNEL__ */
+
+#include <stdio.h>
+#include <stdarg.h>
+
+int rt_vfprintf(FILE *stream, const char *format, va_list args);
+int rt_vprintf(const char *format, va_list args);
+int rt_fprintf(FILE *stream, const char *format, ...);
+int rt_printf(const char *format, ...);
+
+int rt_print_init(size_t buffer_size, const char *name);
+void rt_print_cleanup(void);
+void rt_print_auto_init(int enable);
+const char *rt_print_buffer_name(void);
+
+#endif /* !KERNEL */
+
+#endif /* !_RTPRINT_H */
Index: xenomai/src/Makefile.am
===================================================================
--- xenomai.orig/src/Makefile.am
+++ xenomai/src/Makefile.am
@@ -1,2 +1,2 @@
 
-SUBDIRS = include skins testsuite utils
+SUBDIRS = include rtprint skins testsuite utils
Index: xenomai/src/rtprint/Makefile.am
===================================================================
--- /dev/null
+++ xenomai/src/rtprint/Makefile.am
@@ -0,0 +1,12 @@
+lib_LTLIBRARIES = librtprint.la
+
+librtprint_la_LDFLAGS = -version-info 0:0:0 -lpthread
+
+librtprint_la_SOURCES = \
+	init.c \
+	output.c \
+	api.c
+
+librtprint_la_CPPFLAGS = \
+	@XENO_USER_CFLAGS@ \
+	-I$(top_srcdir)/include
Index: xenomai/src/rtprint/api.c
===================================================================
--- /dev/null
+++ xenomai/src/rtprint/api.c
@@ -0,0 +1,273 @@
+/*
+ * Copyright (C) 2007 Jan Kiszka <jan.kiszka@domain.hid>.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA.
+ */
+
+#include <errno.h>
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <rtprint.h>
+#include <asm/xenomai/ubarrier.h>
+
+#include "internal.h"
+
+int rt_vfprintf(FILE *stream, const char *format, va_list args)
+{
+	struct print_buffer *buffer = pthread_getspecific(__buffer_key);
+	off_t write_pos, read_pos;
+	struct entry_head *head;
+	int len;
+	int res;
+
+	if (!buffer) {
+		res = 0;
+		if (__auto_init)
+			res = rt_print_init(0, NULL);
+		else
+			res = EIO;
+
+		if (res) {
+			errno = res;
+			return -1;
+		}
+		buffer = pthread_getspecific(__buffer_key);
+	}
+
+	/* Take a snapshot of the ring buffer state */
+	write_pos = buffer->write_pos;
+	read_pos = buffer->read_pos;
+	rmb();
+
+	/* Is our write limit the end of the ring buffer? */
+	if (write_pos >= read_pos) {
+		/* Keep a savety margin to the end for at least an empty entry */
+		len = buffer->size - write_pos - sizeof(struct entry_head);
+
+		/* Special case: We were stuck at the end of the ring buffer
+		   with space left there only for one empty entry. Now
+		   read_pos was moved forward and we can wrap around. */
+		if (len == 0 && read_pos > sizeof(struct entry_head)) {
+			/* Write out empty entry */
+			head = buffer->ring + write_pos;
+			head->seq_no = __seq_no;
+			head->text[0] = 0;
+
+			/* Forward to the ring buffer start */
+			write_pos = 0;
+			len = read_pos - 1;
+		}
+	} else {
+		/* Our limit is the read_pos ahead of our write_pos. One byte
+		   margin is required to detect a full ring. */
+		len = read_pos - write_pos - 1;
+	}
+
+	/* Account for head length */
+	len -= sizeof(struct entry_head);
+	if (len < 0)
+		len = 0;
+
+	head = buffer->ring + write_pos;
+
+	res = vsnprintf(head->text, len, format, args);
+
+	if (res < len) {
+		/* Text was written completely, res contains its length */
+		len = res;
+	} else {
+		/* Text was truncated, remove closing \0 that entry_head
+		   already includes */
+		len--;
+	}
+
+	/* If we were able to write some text, finalise the entry */
+	if (len > 0) {
+		head->seq_no = ++__seq_no;
+		head->dest = stream;
+
+		/* Move forward by text and head length */
+		write_pos += len + sizeof(struct entry_head);
+	}
+
+	/* Wrap around early if there is more space on the other side */
+	if (write_pos >= buffer->size - RT_PRINT_LINE_BREAK &&
+	    read_pos <= write_pos && read_pos > buffer->size - write_pos) {
+		/* An empty entry marks the wrap-around */
+		head = buffer->ring + write_pos;
+		head->seq_no = __seq_no;
+		head->text[0] = 0;
+
+		write_pos = 0;
+	}
+
+	/* All entry data must be written before we can update write_pos */
+	wmb();
+
+	buffer->write_pos = write_pos;
+
+	return res;
+}
+
+int rt_vprintf(const char *format, va_list args)
+{
+	return rt_vfprintf(stdout, format, args);
+}
+
+int rt_fprintf(FILE *stream, const char *format, ...)
+{
+	va_list args;
+	int n;
+
+	va_start(args, format);
+	n = rt_vfprintf(stream, format, args);
+	va_end(args);
+
+	return n;
+}
+
+int rt_printf(const char *format, ...)
+{
+	va_list args;
+	int n;
+
+	va_start(args, format);
+	n = rt_vfprintf(stdout, format, args);
+	va_end(args);
+
+	return n;
+}
+
+static void set_buffer_name(struct print_buffer *buffer, const char *name)
+{
+	int n;
+
+	n = sprintf(buffer->name, "%08lx", (unsigned long)pthread_self());
+	if (name) {
+		buffer->name[n++] = ' ';
+		strncpy(buffer->name+n, name, sizeof(buffer->name)-n-1);
+		buffer->name[sizeof(buffer->name)-1] = 0;
+	}
+}
+
+int rt_print_init(size_t buffer_size, const char *buffer_name)
+{
+	struct print_buffer *buffer = pthread_getspecific(__buffer_key);
+	size_t size = buffer_size;
+
+	if (!size)
+		size = __default_buffer_size;
+	else if (size < RT_PRINT_LINE_BREAK)
+		return EINVAL;
+
+	if (buffer) {
+		/* Only set name if buffer size is unchanged or default */
+		if (size == buffer->size || !buffer_size) {
+			set_buffer_name(buffer, buffer_name);
+			return 0;
+		}
+		__cleanup_buffer(buffer);
+	}
+
+	buffer = malloc(sizeof(*buffer));
+	if (!buffer)
+		return ENOMEM;
+
+	buffer->ring = malloc(size);
+	if (!buffer->ring) {
+		free(buffer);
+		return ENOMEM;
+	}
+	memset(buffer->ring, 0, size);
+
+	buffer->read_pos  = 0;
+	buffer->write_pos = 0;
+
+	buffer->size = size;
+
+	set_buffer_name(buffer, buffer_name);
+
+	buffer->prev = NULL;
+
+	pthread_mutex_lock(&__buffer_lock);
+
+	buffer->next = __first_buffer;
+	if (__first_buffer)
+		__first_buffer->prev = buffer;
+	__first_buffer = buffer;
+
+	pthread_mutex_unlock(&__buffer_lock);
+
+	pthread_setspecific(__buffer_key, buffer);
+
+	return 0;
+}
+
+void rt_print_auto_init(int enable)
+{
+	__auto_init = enable;
+}
+
+void __cleanup_buffer(struct print_buffer *buffer)
+{
+	struct print_buffer *prev, *next;
+
+	pthread_setspecific(__buffer_key, NULL);
+
+	pthread_mutex_lock(&__buffer_lock);
+
+	__print_buffers();
+
+	prev = buffer->prev;
+	next = buffer->next;
+
+	if (prev)
+		prev->next = next;
+	else
+		__first_buffer = next;
+	if (next)
+		next->prev = prev;
+
+	pthread_mutex_unlock(&__buffer_lock);
+
+	free(buffer->ring);
+	free(buffer);
+}
+
+void rt_print_cleanup(void)
+{
+	__cleanup_buffer(pthread_getspecific(__buffer_key));
+}
+
+const char *rt_print_buffer_name(void)
+{
+	struct print_buffer *buffer = pthread_getspecific(__buffer_key);
+
+	if (!buffer) {
+		int res = -1;
+
+		if (__auto_init)
+			res = rt_print_init(0, NULL);
+
+		if (res)
+			return NULL;
+
+		buffer = pthread_getspecific(__buffer_key);
+	}
+
+	return buffer->name;
+}
Index: xenomai/src/rtprint/init.c
===================================================================
--- /dev/null
+++ xenomai/src/rtprint/init.c
@@ -0,0 +1,75 @@
+/*
+ * Copyright (C) 2007 Jan Kiszka <jan.kiszka@domain.hid>.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA.
+ */
+
+#include <errno.h>
+#include <limits.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include "internal.h"
+
+struct print_buffer *__first_buffer;
+uint32_t __seq_no;
+size_t __default_buffer_size;
+struct timespec __print_period;
+int __auto_init;
+pthread_mutex_t __buffer_lock;
+pthread_key_t __buffer_key;
+
+static __attribute__ ((constructor)) void __init_rtprint(void)
+{
+	pthread_t thread;
+	pthread_attr_t thattr;
+	const char *value_str;
+	unsigned long long period;
+
+	__first_buffer = NULL;
+	__seq_no = 0;
+	__auto_init = 0;
+
+	__default_buffer_size = RT_PRINT_DEFAULT_BUFFER;
+	value_str = getenv(RT_PRINT_BUFFER_ENV);
+	if (value_str) {
+		errno = 0;
+		__default_buffer_size = strtol(value_str, NULL, 10);
+		if (errno || __default_buffer_size < RT_PRINT_LINE_BREAK) {
+			fprintf(stderr, "Invalid %s\n", RT_PRINT_BUFFER_ENV);
+			exit(1);
+		}
+	}
+
+	period = RT_PRINT_DEFAULT_PERIOD;
+	value_str = getenv(RT_PRINT_PERIOD_ENV);
+	if (value_str) {
+		errno = 0;
+		period = strtoll(value_str, NULL, 10);
+		if (errno) {
+			fprintf(stderr, "Invalid %s\n", RT_PRINT_PERIOD_ENV);
+			exit(1);
+		}
+	}
+	__print_period.tv_sec  = period / 1000;
+	__print_period.tv_nsec = (period % 1000) * 1000000;
+
+	pthread_mutex_init(&__buffer_lock, NULL);
+	pthread_key_create(&__buffer_key, (void (*)(void*))__cleanup_buffer);
+
+	pthread_attr_init(&thattr);
+	pthread_attr_setstacksize(&thattr, PTHREAD_STACK_MIN);
+	pthread_create(&thread, &thattr, __printer_thread, NULL);
+}
Index: xenomai/src/rtprint/internal.h
===================================================================
--- /dev/null
+++ xenomai/src/rtprint/internal.h
@@ -0,0 +1,69 @@
+/*
+ * Copyright (C) 2007 Jan Kiszka <jan.kiszka@domain.hid>.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA.
+ */
+
+#ifndef _RTPRINT_INTERNAL_H
+#define _RTPRINT_INTERNAL_H
+
+#include <pthread.h>
+#include <inttypes.h>
+#include <unistd.h>
+
+#define RT_PRINT_BUFFER_ENV		"RT_PRINT_BUFFER"
+#define RT_PRINT_DEFAULT_BUFFER		16*1024
+
+#define RT_PRINT_PERIOD_ENV		"RT_PRINT_PERIOD"
+#define RT_PRINT_DEFAULT_PERIOD		100 /* ms */
+
+#define RT_PRINT_LINE_BREAK		256
+
+struct entry_head {
+	FILE *dest;
+	uint32_t seq_no;
+	char text[1];
+} __attribute__((packed));
+
+struct print_buffer {
+	off_t write_pos;
+
+	struct print_buffer *next, *prev;
+
+	void *ring;
+	size_t size;
+
+	char name[32];
+
+	/*
+	 * Keep read_pos separated from write_pos to optimise write
+	 * caching on SMP.
+	 */
+	off_t read_pos;
+};
+
+extern struct print_buffer *__first_buffer;
+extern uint32_t __seq_no;
+extern int __auto_init;
+extern size_t __default_buffer_size;
+extern struct timespec __print_period;
+extern pthread_mutex_t __buffer_lock;
+extern pthread_key_t __buffer_key;
+
+void __cleanup_buffer(struct print_buffer *buffer);
+void __print_buffers(void);
+void *__printer_thread(void *arg);
+
+#endif /* !_RTPRINT_INTERNAL_H */
Index: xenomai/src/rtprint/output.c
===================================================================
--- /dev/null
+++ xenomai/src/rtprint/output.c
@@ -0,0 +1,92 @@
+/*
+ * Copyright (C) 2007 Jan Kiszka <jan.kiszka@domain.hid>.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA.
+ */
+
+#include <stdio.h>
+#include <string.h>
+#include <time.h>
+#include <asm/xenomai/ubarrier.h>
+
+#include "internal.h"
+
+static inline uint32_t get_next_seq_no(struct print_buffer *buffer)
+{
+	struct entry_head *head = buffer->ring + buffer->read_pos;
+	return head->seq_no;
+}
+
+static struct print_buffer *get_next_buffer(void)
+{
+	struct print_buffer *pos = __first_buffer;
+	struct print_buffer *buffer = NULL;
+	uint32_t next_seq_no;
+
+	while (pos) {
+		if (pos->read_pos != pos->write_pos &&
+		    (!buffer || get_next_seq_no(pos) < next_seq_no)) {
+			buffer = pos;
+			next_seq_no = get_next_seq_no(pos);
+		}
+		pos = pos->next;
+	}
+
+	return buffer;
+}
+
+void __print_buffers(void)
+{
+	struct print_buffer *buffer;
+	struct entry_head *head;
+	off_t read_pos;
+	int len;
+
+	while (1) {
+		buffer = get_next_buffer();
+		if (!buffer)
+			break;
+
+		read_pos = buffer->read_pos;
+		head = buffer->ring + read_pos;
+		len = strlen(head->text);
+
+		if (len) {
+			/* Print out non-empty entry and proceed */
+			fprintf(head->dest, "%s", head->text);
+			read_pos += sizeof(*head) + len;
+		} else {
+			/* Emptry entries mark the wrap-around */
+			read_pos = 0;
+		}
+
+		rmb();
+		buffer->read_pos = read_pos;
+		wmb();
+	}
+}
+
+void *__printer_thread(void *arg)
+{
+	while (1) {
+		nanosleep(&__print_period, NULL);
+
+		pthread_mutex_lock(&__buffer_lock);
+
+		__print_buffers();
+
+		pthread_mutex_unlock(&__buffer_lock);
+	}
+}


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 249 bytes --]

Re: [Xenomai-core] [RFC][PATCH 1/2] real-time print library

[Philippe Gerum]

On Sat, 2007-02-17 at 10:02 +0100, Jan Kiszka wrote:
> This is a stand-alone real-time library for printf services. It is
> embedded into the Xenomai user space part but actually doesn't depend
> on any Xenomai service, just using plain Linux POSIX.
> 
> The librtprint API looks much like the printf(3) man page:

[...]

> Further features:
>  o Explicit or on-demand thread buffer creation
>  o Override of default buffer size and polling period via environment
>    variables ("RT_PRINT_BUFFER" and "RT_PRINT_PERIOD")
>  o Support for buffer names (useful for the following patch e.g.)
>  o Target output stream can be specified for each print invocation
>  o Breaks all builds except for x86 (missing ubarrier.h headers) :o)
> 

ubarrier.h is redundant. include/asm-*/atomic.h is already there for
such purpose.

> The code is stuffed into src/rtprint for now as patch 2/2 will need this
> lib to be built before the skin libraries. Better suggestions are
> welcome though.

I basically agree that we need to provide more
co-scheduler-based-rt-safe services, so a non-intrusive print out
support should be welcome. A few things more:

- looking beyond print out, we will probably want to iron more ANSI
services in the future. In order to prevent API proliferation, let's
consider the hardened print out support as the beginning of some
libansi_rt service collection.

- the same way libpthread_rt shadows 1003.1c services to iron them over
a co-scheduler, we should do the same for the ironed ANSI services. In
that sense, there is no need for rt_printf, rt_vsprintf and so on, we
just need to shadow printf, vsprintf and friends the same way we did for
pthread_create and such for substituting the 1003.1c support. Again, API
proliferation, especially of non-orthogonal stuff, needs to be fought
now. The bonus is that we would not have to ask people to rely on crufty
preprocessor tricks in order to compile their code in either real-time
layers X3 is going to provide (i.e. I-pipe-based co-scheduler and native
preemption). This would be consistent with the all-time Xenomai's
mantra, i.e. "integrate as seamlessly as you can, don't get uselessly
peculiar".

-- 
Philippe.

Re: [Xenomai-core] [RFC][PATCH 1/2] real-time print library

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 3666 bytes --]

Philippe Gerum wrote:
> On Sat, 2007-02-17 at 10:02 +0100, Jan Kiszka wrote:
>> This is a stand-alone real-time library for printf services. It is
>> embedded into the Xenomai user space part but actually doesn't depend
>> on any Xenomai service, just using plain Linux POSIX.
>>
>> The librtprint API looks much like the printf(3) man page:
> 
> [...]
> 
>> Further features:
>>  o Explicit or on-demand thread buffer creation
>>  o Override of default buffer size and polling period via environment
>>    variables ("RT_PRINT_BUFFER" and "RT_PRINT_PERIOD")
>>  o Support for buffer names (useful for the following patch e.g.)
>>  o Target output stream can be specified for each print invocation
>>  o Breaks all builds except for x86 (missing ubarrier.h headers) :o)
>>
> 
> ubarrier.h is redundant. include/asm-*/atomic.h is already there for
> such purpose.

Guess you mean Xenomai's atomic.h, not the kernel ones. I didn't
realised that it is also used from user space already. OK, will #define
something like xnarch_rmb/wmb (or xnarch_read_memory_barrier?) in that
file instead.

> 
>> The code is stuffed into src/rtprint for now as patch 2/2 will need this
>> lib to be built before the skin libraries. Better suggestions are
>> welcome though.
> 
> I basically agree that we need to provide more
> co-scheduler-based-rt-safe services, so a non-intrusive print out
> support should be welcome. A few things more:

Actually, rt-safe printing is independent of co- vs. native scheduling.
Even with a fully preemptible kernel, the worst-case complexity of
normal printf may not be acceptable. I think rt_printf is generally
useful in real-time programs.

> 
> - looking beyond print out, we will probably want to iron more ANSI
> services in the future. In order to prevent API proliferation, let's
> consider the hardened print out support as the beginning of some
> libansi_rt service collection.

I was already considering to include the TLSF memory allocator for
real-time malloc/free into some Xenomai library, but there were still
technical issues with its lacking 64-bit support e.g., preventing some
quick-hack.

What further services do you have on your radar? My impression is that
there will not be a lot beyond printing and memory allocation.

> 
> - the same way libpthread_rt shadows 1003.1c services to iron them over
> a co-scheduler, we should do the same for the ironed ANSI services. In
> that sense, there is no need for rt_printf, rt_vsprintf and so on, we
> just need to shadow printf, vsprintf and friends the same way we did for
> pthread_create and such for substituting the 1003.1c support. Again, API
> proliferation, especially of non-orthogonal stuff, needs to be fought
> now. The bonus is that we would not have to ask people to rely on crufty
> preprocessor tricks in order to compile their code in either real-time
> layers X3 is going to provide (i.e. I-pipe-based co-scheduler and native
> preemption). This would be consistent with the all-time Xenomai's
> mantra, i.e. "integrate as seamlessly as you can, don't get uselessly
> peculiar".
> 

Think of RTDM: we have transparent overloading with the POSIX skin, but
all others use the explicit selection via rt_dev prefix. I forgot to
mention this, but my plan was to apply the same pattern for librtprint
services.

I don't think we should overload every printf outside the POSIX skin's
scope automatically. Well, at least for the native skin, which is in my
opinion quite a lot about _explicit_ real-time programming, I would
really prefer to keep it like it is even under Xenomai 3.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

Re: [Xenomai-core] [RFC][PATCH 1/2] real-time print library

[Philippe Gerum]

On Sat, 2007-02-17 at 18:43 +0100, Jan Kiszka wrote: 
> Philippe Gerum wrote:
> > On Sat, 2007-02-17 at 10:02 +0100, Jan Kiszka wrote:
> >> This is a stand-alone real-time library for printf services. It is
> >> embedded into the Xenomai user space part but actually doesn't depend
> >> on any Xenomai service, just using plain Linux POSIX.
> >>
> >> The librtprint API looks much like the printf(3) man page:
> > 
> > [...]
> > 
> >> Further features:
> >>  o Explicit or on-demand thread buffer creation
> >>  o Override of default buffer size and polling period via environment
> >>    variables ("RT_PRINT_BUFFER" and "RT_PRINT_PERIOD")
> >>  o Support for buffer names (useful for the following patch e.g.)
> >>  o Target output stream can be specified for each print invocation
> >>  o Breaks all builds except for x86 (missing ubarrier.h headers) :o)
> >>
> > 
> > ubarrier.h is redundant. include/asm-*/atomic.h is already there for
> > such purpose.
> 
> Guess you mean Xenomai's atomic.h, not the kernel ones.

Yes, since this needs to be user-space readable.

>  I didn't
> realised that it is also used from user space already. OK, will #define
> something like xnarch_rmb/wmb (or xnarch_read_memory_barrier?) in that
> file instead.
> 
> > 
> >> The code is stuffed into src/rtprint for now as patch 2/2 will need this
> >> lib to be built before the skin libraries. Better suggestions are
> >> welcome though.
> > 
> > I basically agree that we need to provide more
> > co-scheduler-based-rt-safe services, so a non-intrusive print out
> > support should be welcome. A few things more:
> 
> Actually, rt-safe printing is independent of co- vs. native scheduling.
> Even with a fully preemptible kernel, the worst-case complexity of
> normal printf may not be acceptable. I think rt_printf is generally
> useful in real-time programs.

You seem to be talking specifically about a (non-intrusive) trace
logging feature, rather than plain printf() then, because the latter
does not make much sense to be used in a real-time scenario unless the
caller does accept to pay the price of running a complex operation
anyway.

If, on the other hand, you want to provide more efficient replacements
of existing standard routines in terms of performance, then those should
bear the standard names. Otherwise, people would start mixing printf()
and rt_printf() in their application [experience already showed us that
it's inevitable], and for instance, you would start receiving complaints
about inaccurate output order, leading to incorrect interpretation of
traces, because both implementations would obviously rely on distinct
internal buffers.

> 
> > 
> > - looking beyond print out, we will probably want to iron more ANSI
> > services in the future. In order to prevent API proliferation, let's
> > consider the hardened print out support as the beginning of some
> > libansi_rt service collection.
> 
> I was already considering to include the TLSF memory allocator for
> real-time malloc/free into some Xenomai library, but there were still
> technical issues with its lacking 64-bit support e.g., preventing some
> quick-hack.
> 
> What further services do you have on your radar? My impression is that
> there will not be a lot beyond printing and memory allocation.
> 

And that would already be enough to explain why we would not want to ask
people to stuff their Makefile with LDFLAGS += -lrt_printf -lrt_malloc,
IMO. Because after a few years, my feeling is that the LDFLAGS would
grow out of any reasonable proportion, and our maintenance burden the
same way.

I would also consider things like stdio in general, not just printf, by
mean of server thread(s) the way you did it. Being able to send/receive
a file stream without resorting -at least explicitely- to a proxy thread
would simplify a bunch of existing applications for instance.

Aside of purely ANSI services, there are some POSIX/SVID calls which are
not relevant to libpthread_rt, but for which a hardened counterpart
would make sense over a co-kernel, like SunRPC (over RTNet's UDP, that
is), syslog routines, random, or libcrypt members for instance.

My point being that we cannot anticipate the exact list of features we
would want to harden in the future, but we do know already that
applications ported/created over Xenomai are always more complex, so we
will likely need more of those features. Componentization must have a
limit, and my guts feeling is that going for a specific API for each and
every feature would lead to maintenance chaos and user confusion.
Additionally, sharing internal routines common to those features would
be possible with a centralized architecture (unless we go for
yet-another-library exporting those routines).

> > 
> > - the same way libpthread_rt shadows 1003.1c services to iron them over
> > a co-scheduler, we should do the same for the ironed ANSI services. In
> > that sense, there is no need for rt_printf, rt_vsprintf and so on, we
> > just need to shadow printf, vsprintf and friends the same way we did for
> > pthread_create and such for substituting the 1003.1c support. Again, API
> > proliferation, especially of non-orthogonal stuff, needs to be fought
> > now. The bonus is that we would not have to ask people to rely on crufty
> > preprocessor tricks in order to compile their code in either real-time
> > layers X3 is going to provide (i.e. I-pipe-based co-scheduler and native
> > preemption). This would be consistent with the all-time Xenomai's
> > mantra, i.e. "integrate as seamlessly as you can, don't get uselessly
> > peculiar".
> > 
> 
> Think of RTDM: we have transparent overloading with the POSIX skin, but
> all others use the explicit selection via rt_dev prefix. I forgot to
> mention this, but my plan was to apply the same pattern for librtprint
> services.
> 
> I don't think we should overload every printf outside the POSIX skin's
> scope automatically. Well, at least for the native skin, which is in my
> opinion quite a lot about _explicit_ real-time programming, I would
> really prefer to keep it like it is even under Xenomai 3.
> 

IMO, the only way to make "explicit real-time programming" a rule is to
disable all access to real-time unsafe calls, i.e. basically to the
glibc, and this is obviously something we don't want to, because this
would contradict the fundamental goal of Xenomai toward integration
within the regular Linux programming model.

Because of this, that's a fact of life that users are most often going
to consider that a commonly used glibc routine they need is real-time
safe enough to be used until proven wrong (*), instead of suspecting
such routine to be unsafe a priori, and looking for a safe replacement
in some specific API we might provide (this is why we all had to warn a
number of people countless times on the list about their use of printf()
in deemed time-critical sections from various pieces of code they showed
us).

This is also the reason why I still think we should "upgrade" the
commonly used standard routines to be real-time aware by gradually
providing hardened replacements for them, not trying to convince people
about the fact that using standard routines might not be the right
solution to common problems in the real-time case.

(*) actually, the real problem is that some users might think that we
are smart enough to make deterministic and efficient code out of any
regular Linux service they might think of, albeit the founder of the
Xenomai project might be a total bonehead. Oh, well, anyway...

-- 
Philippe.

[Xenomai-core] Re: [RFC][PATCH 1/2] real-time print library

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 176 bytes --]

Philippe Gerum wrote:
> ubarrier.h is redundant. include/asm-*/atomic.h is already there for
> such purpose.

Here comes version 2 of the patch, getting rid of ubarrier.h

Jan

[-- Attachment #2: librtprint-v2.patch --]
[-- Type: text/x-patch, Size: 20156 bytes --]

---
 configure.in              |    1 
 examples/native/Makefile  |    8 +
 examples/native/rtprint.c |   48 ++++++++
 include/Makefile.am       |    5 
 include/asm-i386/atomic.h |    4 
 include/rtprint.h         |   59 +++++++++
 src/Makefile.am           |    2 
 src/rtprint/Makefile.am   |   12 ++
 src/rtprint/api.c         |  273 ++++++++++++++++++++++++++++++++++++++++++++++
 src/rtprint/init.c        |   75 ++++++++++++
 src/rtprint/internal.h    |   69 +++++++++++
 src/rtprint/output.c      |   96 ++++++++++++++++
 12 files changed, 650 insertions(+), 2 deletions(-)

Index: xenomai/configure.in
===================================================================
--- xenomai.orig/configure.in
+++ xenomai/configure.in
@@ -611,6 +611,7 @@ AC_CONFIG_FILES([ \
 	scripts/xeno-load \
 	scripts/xeno-test \
 	src/Makefile \
+	src/rtprint/Makefile \
 	src/skins/Makefile \
        	src/skins/posix/Makefile \
        	src/skins/native/Makefile \
Index: xenomai/examples/native/Makefile
===================================================================
--- xenomai.orig/examples/native/Makefile
+++ xenomai/examples/native/Makefile
@@ -1,7 +1,7 @@
 ###### CONFIGURATION ######
 
 ### List of applications to be build
-APPLICATIONS = trivial-periodic sigxcpu
+APPLICATIONS = trivial-periodic sigxcpu rtprint
 
 ### Note: to override the search path for the xeno-config script, use "make XENO=..."
 
@@ -44,6 +44,12 @@ endif
 
 
 
+###### SPECIAL TARGET RULES ######
+rtprint: rtprint.c
+	$(CC) $(CFLAGS) $? $(LDFLAGS) -lrtprint -o $@
+
+
+
 ###### KERNEL MODULE BUILD (no change required normally) ######
 ifneq ($(MODULES),)
 
Index: xenomai/examples/native/rtprint.c
===================================================================
--- /dev/null
+++ xenomai/examples/native/rtprint.c
@@ -0,0 +1,48 @@
+#include <stdio.h>
+#include <sys/mman.h>
+#include <native/task.h>
+#include <rtprint.h>
+
+void task2_func(void *arg)
+{
+	int i = 0;
+
+	rt_printf("This triggers auto-init of rt_print for the "
+		  "calling thread.\n"
+		  "A last switch to secondary mode can occure here, "
+		  "but future invocations of rt_printf are safe.\n");
+
+	rt_task_set_mode(0, T_WARNSW, NULL);
+
+	while (1) {
+		rt_task_sleep(3333333LL);
+		rt_fprintf(stderr, "%s: #%d Yet another RT printer - "
+			   "but to stderr.\n", rt_print_buffer_name(), ++i);
+	}
+}
+
+int main(int argc, char **argv)
+{
+	RT_TASK task1, task2;
+	int i = 0;
+
+	mlockall(MCL_CURRENT|MCL_FUTURE);
+
+	/* Perform auto-init of rt_print buffers if the task doesn't do so */
+	rt_print_auto_init(1);
+
+	/* Initialise the rt_print buffer for this task explicitly */
+	rt_print_init(4096, "Task 1");
+
+	rt_task_shadow(&task1, "Task 1", 10, 0);
+	rt_task_spawn(&task2, "Task 2", 0, 11, 0, task2_func, NULL);
+
+	/* To demonstrate that rt_printf is safe */
+	rt_task_set_mode(0, T_WARNSW, NULL);
+
+	while (1) {
+		rt_task_sleep(5000000LL);
+		rt_printf("%s: #%d Hello RT world!\n",
+			  rt_print_buffer_name(), ++i);
+	}
+}
Index: xenomai/include/Makefile.am
===================================================================
--- xenomai.orig/include/Makefile.am
+++ xenomai/include/Makefile.am
@@ -1,3 +1,8 @@
+includedir = $(prefix)/include
+
+include_HEADERS = \
+	rtprint.h
+
 nodist_include_HEADERS=$(CONFIG_HEADER)
 
 SUBDIRS = \
Index: xenomai/include/rtprint.h
===================================================================
--- /dev/null
+++ xenomai/include/rtprint.h
@@ -0,0 +1,59 @@
+/*
+ * Copyright (C) 2007 Jan Kiszka <jan.kiszka@domain.hid>.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA.
+ */
+
+#ifndef _RTPRINT_H
+#define _RTPRINT_H
+
+#ifdef __KERNEL__
+
+#define rt_printf(format, ...)	printk(format __VA_ARGS__)
+
+static inline int rt_print_init(size_t buffer_size, const char *buffer_name)
+{
+	return 0;
+}
+
+#define rt_print_cleanup()	do { } while (0)
+
+static inline void rt_print_auto_init(int enable)
+{
+}
+
+static inline const char *rt_print_buffer_name(void)
+{
+	return "<unknown>";
+}
+
+#else /* !__KERNEL__ */
+
+#include <stdio.h>
+#include <stdarg.h>
+
+int rt_vfprintf(FILE *stream, const char *format, va_list args);
+int rt_vprintf(const char *format, va_list args);
+int rt_fprintf(FILE *stream, const char *format, ...);
+int rt_printf(const char *format, ...);
+
+int rt_print_init(size_t buffer_size, const char *name);
+void rt_print_cleanup(void);
+void rt_print_auto_init(int enable);
+const char *rt_print_buffer_name(void);
+
+#endif /* !KERNEL */
+
+#endif /* !_RTPRINT_H */
Index: xenomai/src/Makefile.am
===================================================================
--- xenomai.orig/src/Makefile.am
+++ xenomai/src/Makefile.am
@@ -1,2 +1,2 @@
 
-SUBDIRS = include skins testsuite utils
+SUBDIRS = include rtprint skins testsuite utils
Index: xenomai/src/rtprint/Makefile.am
===================================================================
--- /dev/null
+++ xenomai/src/rtprint/Makefile.am
@@ -0,0 +1,12 @@
+lib_LTLIBRARIES = librtprint.la
+
+librtprint_la_LDFLAGS = -version-info 0:0:0 -lpthread
+
+librtprint_la_SOURCES = \
+	init.c \
+	output.c \
+	api.c
+
+librtprint_la_CPPFLAGS = \
+	@XENO_USER_CFLAGS@ \
+	-I$(top_srcdir)/include
Index: xenomai/src/rtprint/api.c
===================================================================
--- /dev/null
+++ xenomai/src/rtprint/api.c
@@ -0,0 +1,273 @@
+/*
+ * Copyright (C) 2007 Jan Kiszka <jan.kiszka@domain.hid>.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA.
+ */
+
+#include <errno.h>
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <rtprint.h>
+#include <asm/xenomai/atomic.h>
+
+#include "internal.h"
+
+int rt_vfprintf(FILE *stream, const char *format, va_list args)
+{
+	struct print_buffer *buffer = pthread_getspecific(__buffer_key);
+	off_t write_pos, read_pos;
+	struct entry_head *head;
+	int len;
+	int res;
+
+	if (!buffer) {
+		res = 0;
+		if (__auto_init)
+			res = rt_print_init(0, NULL);
+		else
+			res = EIO;
+
+		if (res) {
+			errno = res;
+			return -1;
+		}
+		buffer = pthread_getspecific(__buffer_key);
+	}
+
+	/* Take a snapshot of the ring buffer state */
+	write_pos = buffer->write_pos;
+	read_pos = buffer->read_pos;
+	xnarch_read_memory_barrier();
+
+	/* Is our write limit the end of the ring buffer? */
+	if (write_pos >= read_pos) {
+		/* Keep a savety margin to the end for at least an empty entry */
+		len = buffer->size - write_pos - sizeof(struct entry_head);
+
+		/* Special case: We were stuck at the end of the ring buffer
+		   with space left there only for one empty entry. Now
+		   read_pos was moved forward and we can wrap around. */
+		if (len == 0 && read_pos > sizeof(struct entry_head)) {
+			/* Write out empty entry */
+			head = buffer->ring + write_pos;
+			head->seq_no = __seq_no;
+			head->text[0] = 0;
+
+			/* Forward to the ring buffer start */
+			write_pos = 0;
+			len = read_pos - 1;
+		}
+	} else {
+		/* Our limit is the read_pos ahead of our write_pos. One byte
+		   margin is required to detect a full ring. */
+		len = read_pos - write_pos - 1;
+	}
+
+	/* Account for head length */
+	len -= sizeof(struct entry_head);
+	if (len < 0)
+		len = 0;
+
+	head = buffer->ring + write_pos;
+
+	res = vsnprintf(head->text, len, format, args);
+
+	if (res < len) {
+		/* Text was written completely, res contains its length */
+		len = res;
+	} else {
+		/* Text was truncated, remove closing \0 that entry_head
+		   already includes */
+		len--;
+	}
+
+	/* If we were able to write some text, finalise the entry */
+	if (len > 0) {
+		head->seq_no = ++__seq_no;
+		head->dest = stream;
+
+		/* Move forward by text and head length */
+		write_pos += len + sizeof(struct entry_head);
+	}
+
+	/* Wrap around early if there is more space on the other side */
+	if (write_pos >= buffer->size - RT_PRINT_LINE_BREAK &&
+	    read_pos <= write_pos && read_pos > buffer->size - write_pos) {
+		/* An empty entry marks the wrap-around */
+		head = buffer->ring + write_pos;
+		head->seq_no = __seq_no;
+		head->text[0] = 0;
+
+		write_pos = 0;
+	}
+
+	/* All entry data must be written before we can update write_pos */
+	xnarch_write_memory_barrier();
+
+	buffer->write_pos = write_pos;
+
+	return res;
+}
+
+int rt_vprintf(const char *format, va_list args)
+{
+	return rt_vfprintf(stdout, format, args);
+}
+
+int rt_fprintf(FILE *stream, const char *format, ...)
+{
+	va_list args;
+	int n;
+
+	va_start(args, format);
+	n = rt_vfprintf(stream, format, args);
+	va_end(args);
+
+	return n;
+}
+
+int rt_printf(const char *format, ...)
+{
+	va_list args;
+	int n;
+
+	va_start(args, format);
+	n = rt_vfprintf(stdout, format, args);
+	va_end(args);
+
+	return n;
+}
+
+static void set_buffer_name(struct print_buffer *buffer, const char *name)
+{
+	int n;
+
+	n = sprintf(buffer->name, "%08lx", (unsigned long)pthread_self());
+	if (name) {
+		buffer->name[n++] = ' ';
+		strncpy(buffer->name+n, name, sizeof(buffer->name)-n-1);
+		buffer->name[sizeof(buffer->name)-1] = 0;
+	}
+}
+
+int rt_print_init(size_t buffer_size, const char *buffer_name)
+{
+	struct print_buffer *buffer = pthread_getspecific(__buffer_key);
+	size_t size = buffer_size;
+
+	if (!size)
+		size = __default_buffer_size;
+	else if (size < RT_PRINT_LINE_BREAK)
+		return EINVAL;
+
+	if (buffer) {
+		/* Only set name if buffer size is unchanged or default */
+		if (size == buffer->size || !buffer_size) {
+			set_buffer_name(buffer, buffer_name);
+			return 0;
+		}
+		__cleanup_buffer(buffer);
+	}
+
+	buffer = malloc(sizeof(*buffer));
+	if (!buffer)
+		return ENOMEM;
+
+	buffer->ring = malloc(size);
+	if (!buffer->ring) {
+		free(buffer);
+		return ENOMEM;
+	}
+	memset(buffer->ring, 0, size);
+
+	buffer->read_pos  = 0;
+	buffer->write_pos = 0;
+
+	buffer->size = size;
+
+	set_buffer_name(buffer, buffer_name);
+
+	buffer->prev = NULL;
+
+	pthread_mutex_lock(&__buffer_lock);
+
+	buffer->next = __first_buffer;
+	if (__first_buffer)
+		__first_buffer->prev = buffer;
+	__first_buffer = buffer;
+
+	pthread_mutex_unlock(&__buffer_lock);
+
+	pthread_setspecific(__buffer_key, buffer);
+
+	return 0;
+}
+
+void rt_print_auto_init(int enable)
+{
+	__auto_init = enable;
+}
+
+void __cleanup_buffer(struct print_buffer *buffer)
+{
+	struct print_buffer *prev, *next;
+
+	pthread_setspecific(__buffer_key, NULL);
+
+	pthread_mutex_lock(&__buffer_lock);
+
+	__print_buffers();
+
+	prev = buffer->prev;
+	next = buffer->next;
+
+	if (prev)
+		prev->next = next;
+	else
+		__first_buffer = next;
+	if (next)
+		next->prev = prev;
+
+	pthread_mutex_unlock(&__buffer_lock);
+
+	free(buffer->ring);
+	free(buffer);
+}
+
+void rt_print_cleanup(void)
+{
+	__cleanup_buffer(pthread_getspecific(__buffer_key));
+}
+
+const char *rt_print_buffer_name(void)
+{
+	struct print_buffer *buffer = pthread_getspecific(__buffer_key);
+
+	if (!buffer) {
+		int res = -1;
+
+		if (__auto_init)
+			res = rt_print_init(0, NULL);
+
+		if (res)
+			return NULL;
+
+		buffer = pthread_getspecific(__buffer_key);
+	}
+
+	return buffer->name;
+}
Index: xenomai/src/rtprint/init.c
===================================================================
--- /dev/null
+++ xenomai/src/rtprint/init.c
@@ -0,0 +1,75 @@
+/*
+ * Copyright (C) 2007 Jan Kiszka <jan.kiszka@domain.hid>.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA.
+ */
+
+#include <errno.h>
+#include <limits.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include "internal.h"
+
+struct print_buffer *__first_buffer;
+uint32_t __seq_no;
+size_t __default_buffer_size;
+struct timespec __print_period;
+int __auto_init;
+pthread_mutex_t __buffer_lock;
+pthread_key_t __buffer_key;
+
+static __attribute__ ((constructor)) void __init_rtprint(void)
+{
+	pthread_t thread;
+	pthread_attr_t thattr;
+	const char *value_str;
+	unsigned long long period;
+
+	__first_buffer = NULL;
+	__seq_no = 0;
+	__auto_init = 0;
+
+	__default_buffer_size = RT_PRINT_DEFAULT_BUFFER;
+	value_str = getenv(RT_PRINT_BUFFER_ENV);
+	if (value_str) {
+		errno = 0;
+		__default_buffer_size = strtol(value_str, NULL, 10);
+		if (errno || __default_buffer_size < RT_PRINT_LINE_BREAK) {
+			fprintf(stderr, "Invalid %s\n", RT_PRINT_BUFFER_ENV);
+			exit(1);
+		}
+	}
+
+	period = RT_PRINT_DEFAULT_PERIOD;
+	value_str = getenv(RT_PRINT_PERIOD_ENV);
+	if (value_str) {
+		errno = 0;
+		period = strtoll(value_str, NULL, 10);
+		if (errno) {
+			fprintf(stderr, "Invalid %s\n", RT_PRINT_PERIOD_ENV);
+			exit(1);
+		}
+	}
+	__print_period.tv_sec  = period / 1000;
+	__print_period.tv_nsec = (period % 1000) * 1000000;
+
+	pthread_mutex_init(&__buffer_lock, NULL);
+	pthread_key_create(&__buffer_key, (void (*)(void*))__cleanup_buffer);
+
+	pthread_attr_init(&thattr);
+	pthread_attr_setstacksize(&thattr, PTHREAD_STACK_MIN);
+	pthread_create(&thread, &thattr, __printer_thread, NULL);
+}
Index: xenomai/src/rtprint/internal.h
===================================================================
--- /dev/null
+++ xenomai/src/rtprint/internal.h
@@ -0,0 +1,69 @@
+/*
+ * Copyright (C) 2007 Jan Kiszka <jan.kiszka@domain.hid>.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA.
+ */
+
+#ifndef _RTPRINT_INTERNAL_H
+#define _RTPRINT_INTERNAL_H
+
+#include <pthread.h>
+#include <inttypes.h>
+#include <unistd.h>
+
+#define RT_PRINT_BUFFER_ENV		"RT_PRINT_BUFFER"
+#define RT_PRINT_DEFAULT_BUFFER		16*1024
+
+#define RT_PRINT_PERIOD_ENV		"RT_PRINT_PERIOD"
+#define RT_PRINT_DEFAULT_PERIOD		100 /* ms */
+
+#define RT_PRINT_LINE_BREAK		256
+
+struct entry_head {
+	FILE *dest;
+	uint32_t seq_no;
+	char text[1];
+} __attribute__((packed));
+
+struct print_buffer {
+	off_t write_pos;
+
+	struct print_buffer *next, *prev;
+
+	void *ring;
+	size_t size;
+
+	char name[32];
+
+	/*
+	 * Keep read_pos separated from write_pos to optimise write
+	 * caching on SMP.
+	 */
+	off_t read_pos;
+};
+
+extern struct print_buffer *__first_buffer;
+extern uint32_t __seq_no;
+extern int __auto_init;
+extern size_t __default_buffer_size;
+extern struct timespec __print_period;
+extern pthread_mutex_t __buffer_lock;
+extern pthread_key_t __buffer_key;
+
+void __cleanup_buffer(struct print_buffer *buffer);
+void __print_buffers(void);
+void *__printer_thread(void *arg);
+
+#endif /* !_RTPRINT_INTERNAL_H */
Index: xenomai/src/rtprint/output.c
===================================================================
--- /dev/null
+++ xenomai/src/rtprint/output.c
@@ -0,0 +1,96 @@
+/*
+ * Copyright (C) 2007 Jan Kiszka <jan.kiszka@domain.hid>.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA.
+ */
+
+#include <stdio.h>
+#include <string.h>
+#include <time.h>
+#include <asm/xenomai/atomic.h>
+
+#include "internal.h"
+
+static inline uint32_t get_next_seq_no(struct print_buffer *buffer)
+{
+	struct entry_head *head = buffer->ring + buffer->read_pos;
+	return head->seq_no;
+}
+
+static struct print_buffer *get_next_buffer(void)
+{
+	struct print_buffer *pos = __first_buffer;
+	struct print_buffer *buffer = NULL;
+	uint32_t next_seq_no;
+
+	while (pos) {
+		if (pos->read_pos != pos->write_pos &&
+		    (!buffer || get_next_seq_no(pos) < next_seq_no)) {
+			buffer = pos;
+			next_seq_no = get_next_seq_no(pos);
+		}
+		pos = pos->next;
+	}
+
+	return buffer;
+}
+
+void __print_buffers(void)
+{
+	struct print_buffer *buffer;
+	struct entry_head *head;
+	off_t read_pos;
+	int len;
+
+	while (1) {
+		buffer = get_next_buffer();
+		if (!buffer)
+			break;
+
+		read_pos = buffer->read_pos;
+		head = buffer->ring + read_pos;
+		len = strlen(head->text);
+
+		if (len) {
+			/* Print out non-empty entry and proceed */
+			fprintf(head->dest, "%s", head->text);
+			read_pos += sizeof(*head) + len;
+		} else {
+			/* Emptry entries mark the wrap-around */
+			read_pos = 0;
+		}
+
+		/* Make sure we have read the entry competely before
+		   forwarding read_pos */
+		xnarch_read_memory_barrier();
+		buffer->read_pos = read_pos;
+
+		/* Enforce the read_pos update before proceeding */
+		xnarch_write_memory_barrier();
+	}
+}
+
+void *__printer_thread(void *arg)
+{
+	while (1) {
+		nanosleep(&__print_period, NULL);
+
+		pthread_mutex_lock(&__buffer_lock);
+
+		__print_buffers();
+
+		pthread_mutex_unlock(&__buffer_lock);
+	}
+}
Index: xenomai/include/asm-i386/atomic.h
===================================================================
--- xenomai.orig/include/asm-i386/atomic.h
+++ xenomai/include/asm-i386/atomic.h
@@ -63,6 +63,10 @@ static inline unsigned long xnarch_atomi
 
 #define xnarch_memory_barrier()  __asm__ __volatile__("": : :"memory")
 
+#define xnarch_read_memory_barrier() \
+	__asm__ __volatile__ ("lock; addl $0,0(%%esp)": : :"memory")
+#define xnarch_write_memory_barrier()	xnarch_memory_barrier()
+
 #endif /* __KERNEL__ */
 
 typedef unsigned long atomic_flags_t;

Re: [Xenomai-core] [RFC][PATCH 1/2] real-time print library

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 9611 bytes --]

Philippe Gerum wrote:
> On Sat, 2007-02-17 at 18:43 +0100, Jan Kiszka wrote: 
>> Philippe Gerum wrote:
>>> On Sat, 2007-02-17 at 10:02 +0100, Jan Kiszka wrote:
>>>> The code is stuffed into src/rtprint for now as patch 2/2 will need this
>>>> lib to be built before the skin libraries. Better suggestions are
>>>> welcome though.
>>> I basically agree that we need to provide more
>>> co-scheduler-based-rt-safe services, so a non-intrusive print out
>>> support should be welcome. A few things more:
>> Actually, rt-safe printing is independent of co- vs. native scheduling.
>> Even with a fully preemptible kernel, the worst-case complexity of
>> normal printf may not be acceptable. I think rt_printf is generally
>> useful in real-time programs.
> 
> You seem to be talking specifically about a (non-intrusive) trace
> logging feature, rather than plain printf() then, because the latter
> does not make much sense to be used in a real-time scenario unless the
> caller does accept to pay the price of running a complex operation
> anyway.

No, I'm also talking about plain printf scenarios from hard-RT threads.
There is a difference between just doing a fairly simple snprintf to
format the output and going into the kernel to write it to some console,
file, network link, or whatever.

> 
> If, on the other hand, you want to provide more efficient replacements
> of existing standard routines in terms of performance, then those should
> bear the standard names. Otherwise, people would start mixing printf()
> and rt_printf() in their application [experience already showed us that
> it's inevitable], and for instance, you would start receiving complaints
> about inaccurate output order, leading to incorrect interpretation of
> traces, because both implementations would obviously rely on distinct
> internal buffers.

I agree that there is an ordering issue when writing to stdout or stderr
/wrt to standard glibc routines. But the question is where to draw the
line. We surely do not want to redirect each and every file write to a
server thread _automatically_ (would be quite a performance PITA...)
just because some fprintf may otherwise happen to come out in an
unexpected order. We cannot avoid that the user has to be - to some
degree - really _aware_ of what (s)he's doing.

> 
>>> - looking beyond print out, we will probably want to iron more ANSI
>>> services in the future. In order to prevent API proliferation, let's
>>> consider the hardened print out support as the beginning of some
>>> libansi_rt service collection.
>> I was already considering to include the TLSF memory allocator for
>> real-time malloc/free into some Xenomai library, but there were still
>> technical issues with its lacking 64-bit support e.g., preventing some
>> quick-hack.
>>
>> What further services do you have on your radar? My impression is that
>> there will not be a lot beyond printing and memory allocation.
>>
> 
> And that would already be enough to explain why we would not want to ask
> people to stuff their Makefile with LDFLAGS += -lrt_printf -lrt_malloc,
> IMO. Because after a few years, my feeling is that the LDFLAGS would
> grow out of any reasonable proportion, and our maintenance burden the
> same way.

So your point is single vs. multiple rt-helper libs? I don't see a
problem bundling fairly small services (a few kB) into a single lib,
common to all skins. But once it takes a bit more to provide a certain
service, we have to keep memory-restricted embedded applications in
mind. Configuring features at build time is one way. Providing
reasonably clustered libraries is another. Hard to draw a line now
without having all components already at hand or at least in mind.

> 
> I would also consider things like stdio in general, not just printf, by
> mean of server thread(s) the way you did it. Being able to send/receive
> a file stream without resorting -at least explicitely- to a proxy thread
> would simplify a bunch of existing applications for instance.

Yeah, the way rt_printf writes out data might be extended to a common
pattern also for other stdio output services. If async stdio-like
reading is also worth providing in a generic way is something I would
like to see backed up by concrete application scenarios. Maybe.

> 
> Aside of purely ANSI services, there are some POSIX/SVID calls which are
> not relevant to libpthread_rt, but for which a hardened counterpart
> would make sense over a co-kernel, like SunRPC (over RTNet's UDP, that
> is), syslog routines, random, or libcrypt members for instance.
> 
> My point being that we cannot anticipate the exact list of features we
> would want to harden in the future, but we do know already that
> applications ported/created over Xenomai are always more complex, so we
> will likely need more of those features. Componentization must have a
> limit, and my guts feeling is that going for a specific API for each and
> every feature would lead to maintenance chaos and user confusion.
> Additionally, sharing internal routines common to those features would
> be possible with a centralized architecture (unless we go for
> yet-another-library exporting those routines).

Well, would you be more happy if we call the printing library
libstdio_rt without yet implementing all the related services, but
pointing out the potential path of this lib in the future?

> 
>>> - the same way libpthread_rt shadows 1003.1c services to iron them over
>>> a co-scheduler, we should do the same for the ironed ANSI services. In
>>> that sense, there is no need for rt_printf, rt_vsprintf and so on, we
>>> just need to shadow printf, vsprintf and friends the same way we did for
>>> pthread_create and such for substituting the 1003.1c support. Again, API
>>> proliferation, especially of non-orthogonal stuff, needs to be fought
>>> now. The bonus is that we would not have to ask people to rely on crufty
>>> preprocessor tricks in order to compile their code in either real-time
>>> layers X3 is going to provide (i.e. I-pipe-based co-scheduler and native
>>> preemption). This would be consistent with the all-time Xenomai's
>>> mantra, i.e. "integrate as seamlessly as you can, don't get uselessly
>>> peculiar".
>>>
>> Think of RTDM: we have transparent overloading with the POSIX skin, but
>> all others use the explicit selection via rt_dev prefix. I forgot to
>> mention this, but my plan was to apply the same pattern for librtprint
>> services.
>>
>> I don't think we should overload every printf outside the POSIX skin's
>> scope automatically. Well, at least for the native skin, which is in my
>> opinion quite a lot about _explicit_ real-time programming, I would
>> really prefer to keep it like it is even under Xenomai 3.
>>
> 
> IMO, the only way to make "explicit real-time programming" a rule is to
> disable all access to real-time unsafe calls, i.e. basically to the
> glibc, and this is obviously something we don't want to, because this
> would contradict the fundamental goal of Xenomai toward integration
> within the regular Linux programming model.

No, disabling such access is surely a no-go. But I really think we
should strengthen the mode-switch detection, maybe be considering to
invert its default (always or just for debugging). Mode-switch detection
by the user space tracer could be a first step in this direction
(without immediately breaking the API).

> 
> Because of this, that's a fact of life that users are most often going
> to consider that a commonly used glibc routine they need is real-time
> safe enough to be used until proven wrong (*), instead of suspecting
> such routine to be unsafe a priori, and looking for a safe replacement
> in some specific API we might provide (this is why we all had to warn a
> number of people countless times on the list about their use of printf()
> in deemed time-critical sections from various pieces of code they showed
> us).
> 
> This is also the reason why I still think we should "upgrade" the
> commonly used standard routines to be real-time aware by gradually
> providing hardened replacements for them, not trying to convince people
> about the fact that using standard routines might not be the right
> solution to common problems in the real-time case.
> 
> (*) actually, the real problem is that some users might think that we
> are smart enough to make deterministic and efficient code out of any
> regular Linux service they might think of, albeit the founder of the
> Xenomai project might be a total bonehead. Oh, well, anyway...
> 

As I said, overloading, just trying to shadow the differences doesn't
solve the problem automagically. You often need special configuration
APIs for the shadowed one, thus you need understanding of its
mechanisms. And you need access to the original functions because of
drawbacks of the hardened variants (if they had none, we could just
submit glibc or whatever patches...).

Overloading is only as long comfortable as you _mostly_ make use of the
replacement services in your application. Once it is the other way
around, this automatism becomes quite a PITA. We luckily already have
two programming models with POSIX vs. native skin matching both use
cases, and I consider this a strength, not a weakness of Xenomai.

Granted, alternatives can cause confusion, and we may have to improve
beginner guides on the homepage or wherever to underline and explain
these differences more explicitly.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 249 bytes --]

Re: [Xenomai-core] [RFC][PATCH 1/2] real-time print library

[Philippe Gerum]

On Sun, 2007-02-18 at 20:38 +0100, Jan Kiszka wrote: 
> Philippe Gerum wrote:
> > On Sat, 2007-02-17 at 18:43 +0100, Jan Kiszka wrote: 
> >> Philippe Gerum wrote:
> >>> On Sat, 2007-02-17 at 10:02 +0100, Jan Kiszka wrote:
> >>>> The code is stuffed into src/rtprint for now as patch 2/2 will need this
> >>>> lib to be built before the skin libraries. Better suggestions are
> >>>> welcome though.
> >>> I basically agree that we need to provide more
> >>> co-scheduler-based-rt-safe services, so a non-intrusive print out
> >>> support should be welcome. A few things more:
> >> Actually, rt-safe printing is independent of co- vs. native scheduling.
> >> Even with a fully preemptible kernel, the worst-case complexity of
> >> normal printf may not be acceptable. I think rt_printf is generally
> >> useful in real-time programs.
> > 
> > You seem to be talking specifically about a (non-intrusive) trace
> > logging feature, rather than plain printf() then, because the latter
> > does not make much sense to be used in a real-time scenario unless the
> > caller does accept to pay the price of running a complex operation
> > anyway.
> 
> No, I'm also talking about plain printf scenarios from hard-RT threads.
> There is a difference between just doing a fairly simple snprintf to
> format the output and going into the kernel to write it to some console,
> file, network link, or whatever.
> 

Right, but your patch seems to focus on printf() so far, hence my
argument. [Btw, glibc's snprintf() called from rt_vsnprintf() is likely
going to compete for the file lock in the MT_SAFE_IO case, which might
not be what you want, if you don't want to take the mode switch penalty
upon contention].

> > 
> > If, on the other hand, you want to provide more efficient replacements
> > of existing standard routines in terms of performance, then those should
> > bear the standard names. Otherwise, people would start mixing printf()
> > and rt_printf() in their application [experience already showed us that
> > it's inevitable], and for instance, you would start receiving complaints
> > about inaccurate output order, leading to incorrect interpretation of
> > traces, because both implementations would obviously rely on distinct
> > internal buffers.
> 
> I agree that there is an ordering issue when writing to stdout or stderr
> /wrt to standard glibc routines.

I was primarily talking about calling printf() then rt_printf() (or the
other way around), which would not give any guarantee about the output
sequence.

>  But the question is where to draw the
> line. We surely do not want to redirect each and every file write to a
> server thread _automatically_ (would be quite a performance PITA...)

You would still have access to the __real_xxx counterparts anyway.
> just because some fprintf may otherwise happen to come out in an
> unexpected order. We cannot avoid that the user has to be - to some
> degree - really _aware_ of what (s)he's doing.
> 

My point is different, and it goes beyond the printf() case. IOW, should
a developer:

a) have to verify whether rt_printf(), rt_vsprintf(), rt_vsnprintf()
etc. exist in some API we would provide, and decide to use them in
replacement of the original glibc calls, and understand the consequences
about predictability if they don't.

or

b) have to verify whether printf(), vsprintf(), vsnprintf() etc. are
documented as members of the calls Xenomai "hardens" when she happen to
need them, and understand the consequences about predictability if they
don't.

In both cases, such developer _must_ know what he's doing anyway, in
order to ask himself either one of those questions in the first place,
so this is clearly not about telling people "life is good because call
shadowing exists, just breath and live cluelessly". It's whether you
want them to use a specific API, or to use a hardened subset of the
original one.

I tend to prefer b) because:
- I tend to find much easier to tell people "you may use a subset of the
API you already know", than "you must use another API to access a
functional subset of the API you already know". The fact that original
symbols would just have to be prefixed by rt_ does not change this fact:
it's another API.
- when running Xenomai apps over native preemption, finding rt_* and
non-rt versions of the same symbol would look preposterous, and the
point about X3 is to allow people to move freely from one real-time
layer to another (i.e. co-kernel to native preemption and the other way
around), without any impact on their code.

But, I definitely agree with you that b) is error-prone if nothing is
made to strongly warn about potential misuses, and strenghtening the
mode switch alert policy may be a mean to do that.

> > 
> >>> - looking beyond print out, we will probably want to iron more ANSI
> >>> services in the future. In order to prevent API proliferation, let's
> >>> consider the hardened print out support as the beginning of some
> >>> libansi_rt service collection.
> >> I was already considering to include the TLSF memory allocator for
> >> real-time malloc/free into some Xenomai library, but there were still
> >> technical issues with its lacking 64-bit support e.g., preventing some
> >> quick-hack.
> >>
> >> What further services do you have on your radar? My impression is that
> >> there will not be a lot beyond printing and memory allocation.
> >>
> > 
> > And that would already be enough to explain why we would not want to ask
> > people to stuff their Makefile with LDFLAGS += -lrt_printf -lrt_malloc,
> > IMO. Because after a few years, my feeling is that the LDFLAGS would
> > grow out of any reasonable proportion, and our maintenance burden the
> > same way.
> 
> So your point is single vs. multiple rt-helper libs? I don't see a
> problem bundling fairly small services (a few kB) into a single lib,
> common to all skins. But once it takes a bit more to provide a certain
> service, we have to keep memory-restricted embedded applications in
> mind. Configuring features at build time is one way. Providing
> reasonably clustered libraries is another. Hard to draw a line now
> without having all components already at hand or at least in mind.
> 

My point is about avoiding this approach:
http://www.slac.stanford.edu/exp/glast/flight/sw/vxdocs/vxworks/ref/libIndex.htm#A

> > 
> > I would also consider things like stdio in general, not just printf, by
> > mean of server thread(s) the way you did it. Being able to send/receive
> > a file stream without resorting -at least explicitely- to a proxy thread
> > would simplify a bunch of existing applications for instance.
> 
> Yeah, the way rt_printf writes out data might be extended to a common
> pattern also for other stdio output services. If async stdio-like
> reading is also worth providing in a generic way is something I would
> like to see backed up by concrete application scenarios. Maybe.
> 
> > 
> > Aside of purely ANSI services, there are some POSIX/SVID calls which are
> > not relevant to libpthread_rt, but for which a hardened counterpart
> > would make sense over a co-kernel, like SunRPC (over RTNet's UDP, that
> > is), syslog routines, random, or libcrypt members for instance.
> > 
> > My point being that we cannot anticipate the exact list of features we
> > would want to harden in the future, but we do know already that
> > applications ported/created over Xenomai are always more complex, so we
> > will likely need more of those features. Componentization must have a
> > limit, and my guts feeling is that going for a specific API for each and
> > every feature would lead to maintenance chaos and user confusion.
> > Additionally, sharing internal routines common to those features would
> > be possible with a centralized architecture (unless we go for
> > yet-another-library exporting those routines).
> 
> Well, would you be more happy if we call the printing library
> libstdio_rt without yet implementing all the related services, but
> pointing out the potential path of this lib in the future?
> 

There would remain the malloc support issue, which would obviously not
fit in the stdio lib.

> > 
> >>> - the same way libpthread_rt shadows 1003.1c services to iron them over
> >>> a co-scheduler, we should do the same for the ironed ANSI services. In
> >>> that sense, there is no need for rt_printf, rt_vsprintf and so on, we
> >>> just need to shadow printf, vsprintf and friends the same way we did for
> >>> pthread_create and such for substituting the 1003.1c support. Again, API
> >>> proliferation, especially of non-orthogonal stuff, needs to be fought
> >>> now. The bonus is that we would not have to ask people to rely on crufty
> >>> preprocessor tricks in order to compile their code in either real-time
> >>> layers X3 is going to provide (i.e. I-pipe-based co-scheduler and native
> >>> preemption). This would be consistent with the all-time Xenomai's
> >>> mantra, i.e. "integrate as seamlessly as you can, don't get uselessly
> >>> peculiar".
> >>>
> >> Think of RTDM: we have transparent overloading with the POSIX skin, but
> >> all others use the explicit selection via rt_dev prefix. I forgot to
> >> mention this, but my plan was to apply the same pattern for librtprint
> >> services.
> >>
> >> I don't think we should overload every printf outside the POSIX skin's
> >> scope automatically. Well, at least for the native skin, which is in my
> >> opinion quite a lot about _explicit_ real-time programming, I would
> >> really prefer to keep it like it is even under Xenomai 3.
> >>
> > 
> > IMO, the only way to make "explicit real-time programming" a rule is to
> > disable all access to real-time unsafe calls, i.e. basically to the
> > glibc, and this is obviously something we don't want to, because this
> > would contradict the fundamental goal of Xenomai toward integration
> > within the regular Linux programming model.
> 
> No, disabling such access is surely a no-go. But I really think we
> should strengthen the mode-switch detection, maybe be considering to
> invert its default (always or just for debugging). Mode-switch detection
> by the user space tracer could be a first step in this direction
> (without immediately breaking the API).
> 

Agreed, other means to strengthen the alerts about loss of
predictability should be found.

> > 
> > Because of this, that's a fact of life that users are most often going
> > to consider that a commonly used glibc routine they need is real-time
> > safe enough to be used until proven wrong (*), instead of suspecting
> > such routine to be unsafe a priori, and looking for a safe replacement
> > in some specific API we might provide (this is why we all had to warn a
> > number of people countless times on the list about their use of printf()
> > in deemed time-critical sections from various pieces of code they showed
> > us).
> > 
> > This is also the reason why I still think we should "upgrade" the
> > commonly used standard routines to be real-time aware by gradually
> > providing hardened replacements for them, not trying to convince people
> > about the fact that using standard routines might not be the right
> > solution to common problems in the real-time case.
> > 
> > (*) actually, the real problem is that some users might think that we
> > are smart enough to make deterministic and efficient code out of any
> > regular Linux service they might think of, albeit the founder of the
> > Xenomai project might be a total bonehead. Oh, well, anyway...
> > 
> 
> As I said, overloading, just trying to shadow the differences doesn't
> solve the problem automagically. You often need special configuration
> APIs for the shadowed one, thus you need understanding of its
> mechanisms. And you need access to the original functions because of
> drawbacks of the hardened variants (if they had none, we could just
> submit glibc or whatever patches...).
> Overloading is only as long comfortable as you _mostly_ make use of the
> replacement services in your application. Once it is the other way
> around, this automatism becomes quite a PITA. We luckily already have
> two programming models with POSIX vs. native skin matching both use
> cases, and I consider this a strength, not a weakness of Xenomai.

I did not say it was a weakness (even if I always thought that the
rt_dev interface to RTDM was redundant with the POSIXish one), but that
we should keep the finger on the bloat control when it comes to
co-kernel-specific-APIs, because as much as there is life beyond x86,
there is also life beyond co-kernels.

The natural tendency of Xenomai has always been to blur the frontier
between the regular Linux environment and the co-kernel environment, so
that people don't get schizophrenic when implementing a real-time
application over Xenomai, with powerful features on both sides, but no
ability to combine those strengthes efficiently. I'm convinced this has
some value in the picture too.

> 
> Granted, alternatives can cause confusion, and we may have to improve
> beginner guides on the homepage or wherever to underline and explain
> these differences more explicitly.
> 

Ack.

> Jan
> 
-- 
Philippe.

Re: [Xenomai-core] [RFC][PATCH 1/2] real-time print library

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 14851 bytes --]

Philippe Gerum wrote:
> On Sun, 2007-02-18 at 20:38 +0100, Jan Kiszka wrote: 
>> Philippe Gerum wrote:
>>> On Sat, 2007-02-17 at 18:43 +0100, Jan Kiszka wrote: 
>>>> Philippe Gerum wrote:
>>>>> On Sat, 2007-02-17 at 10:02 +0100, Jan Kiszka wrote:
>>>>>> The code is stuffed into src/rtprint for now as patch 2/2 will need this
>>>>>> lib to be built before the skin libraries. Better suggestions are
>>>>>> welcome though.
>>>>> I basically agree that we need to provide more
>>>>> co-scheduler-based-rt-safe services, so a non-intrusive print out
>>>>> support should be welcome. A few things more:
>>>> Actually, rt-safe printing is independent of co- vs. native scheduling.
>>>> Even with a fully preemptible kernel, the worst-case complexity of
>>>> normal printf may not be acceptable. I think rt_printf is generally
>>>> useful in real-time programs.
>>> You seem to be talking specifically about a (non-intrusive) trace
>>> logging feature, rather than plain printf() then, because the latter
>>> does not make much sense to be used in a real-time scenario unless the
>>> caller does accept to pay the price of running a complex operation
>>> anyway.
>> No, I'm also talking about plain printf scenarios from hard-RT threads.
>> There is a difference between just doing a fairly simple snprintf to
>> format the output and going into the kernel to write it to some console,
>> file, network link, or whatever.
>>
> 
> Right, but your patch seems to focus on printf() so far, hence my
> argument. [Btw, glibc's snprintf() called from rt_vsnprintf() is likely
> going to compete for the file lock in the MT_SAFE_IO case, which might
> not be what you want, if you don't want to take the mode switch penalty
> upon contention].

Going through glibc's stdio/libio stuff is hell, but based on my current
analysis (and also when considering what *snprintf does) it is highly
unlikely that there is anything to compete for. All required buffers are
on the stack or in user hand, and that MT_SAFE_IO lock you referred to
is set to NULL.

> 
>>> If, on the other hand, you want to provide more efficient replacements
>>> of existing standard routines in terms of performance, then those should
>>> bear the standard names. Otherwise, people would start mixing printf()
>>> and rt_printf() in their application [experience already showed us that
>>> it's inevitable], and for instance, you would start receiving complaints
>>> about inaccurate output order, leading to incorrect interpretation of
>>> traces, because both implementations would obviously rely on distinct
>>> internal buffers.
>> I agree that there is an ordering issue when writing to stdout or stderr
>> /wrt to standard glibc routines.
> 
> I was primarily talking about calling printf() then rt_printf() (or the
> other way around), which would not give any guarantee about the output
> sequence.
> 
>>  But the question is where to draw the
>> line. We surely do not want to redirect each and every file write to a
>> server thread _automatically_ (would be quite a performance PITA...)
> 
> You would still have access to the __real_xxx counterparts anyway.
>> just because some fprintf may otherwise happen to come out in an
>> unexpected order. We cannot avoid that the user has to be - to some
>> degree - really _aware_ of what (s)he's doing.
>>
> 
> My point is different, and it goes beyond the printf() case. IOW, should
> a developer:
> 
> a) have to verify whether rt_printf(), rt_vsprintf(), rt_vsnprintf()
> etc. exist in some API we would provide, and decide to use them in
> replacement of the original glibc calls, and understand the consequences
> about predictability if they don't.
> 
> or
> 
> b) have to verify whether printf(), vsprintf(), vsnprintf() etc. are
> documented as members of the calls Xenomai "hardens" when she happen to
> need them, and understand the consequences about predictability if they
> don't.
> 
> In both cases, such developer _must_ know what he's doing anyway, in
> order to ask himself either one of those questions in the first place,
> so this is clearly not about telling people "life is good because call
> shadowing exists, just breath and live cluelessly". It's whether you
> want them to use a specific API, or to use a hardened subset of the
> original one.
> 
> I tend to prefer b) because:
> - I tend to find much easier to tell people "you may use a subset of the
> API you already know", than "you must use another API to access a
> functional subset of the API you already know". The fact that original
> symbols would just have to be prefixed by rt_ does not change this fact:
> it's another API.
> - when running Xenomai apps over native preemption, finding rt_* and

As I said: rt_printf vs. printf is a good example where native
preemption does not change the picture noticeably, where you want
special services even if the kernel knows about priority inheritance in
order to boost such complex printing jobs over others.

> non-rt versions of the same symbol would look preposterous, and the
> point about X3 is to allow people to move freely from one real-time
> layer to another (i.e. co-kernel to native preemption and the other way
> around), without any impact on their code.
> 
> But, I definitely agree with you that b) is error-prone if nothing is
> made to strongly warn about potential misuses, and strenghtening the
> mode switch alert policy may be a mean to do that.
> 
>>>>> - looking beyond print out, we will probably want to iron more ANSI
>>>>> services in the future. In order to prevent API proliferation, let's
>>>>> consider the hardened print out support as the beginning of some
>>>>> libansi_rt service collection.
>>>> I was already considering to include the TLSF memory allocator for
>>>> real-time malloc/free into some Xenomai library, but there were still
>>>> technical issues with its lacking 64-bit support e.g., preventing some
>>>> quick-hack.
>>>>
>>>> What further services do you have on your radar? My impression is that
>>>> there will not be a lot beyond printing and memory allocation.
>>>>
>>> And that would already be enough to explain why we would not want to ask
>>> people to stuff their Makefile with LDFLAGS += -lrt_printf -lrt_malloc,
>>> IMO. Because after a few years, my feeling is that the LDFLAGS would
>>> grow out of any reasonable proportion, and our maintenance burden the
>>> same way.
>> So your point is single vs. multiple rt-helper libs? I don't see a
>> problem bundling fairly small services (a few kB) into a single lib,
>> common to all skins. But once it takes a bit more to provide a certain
>> service, we have to keep memory-restricted embedded applications in
>> mind. Configuring features at build time is one way. Providing
>> reasonably clustered libraries is another. Hard to draw a line now
>> without having all components already at hand or at least in mind.
>>
> 
> My point is about avoiding this approach:
> http://www.slac.stanford.edu/exp/glast/flight/sw/vxdocs/vxworks/ref/libIndex.htm#A

Ack, that is certainly a bad example for componentisation. The goal must
be to find a reasonable path between both extremes.

> 
>>> I would also consider things like stdio in general, not just printf, by
>>> mean of server thread(s) the way you did it. Being able to send/receive
>>> a file stream without resorting -at least explicitely- to a proxy thread
>>> would simplify a bunch of existing applications for instance.
>> Yeah, the way rt_printf writes out data might be extended to a common
>> pattern also for other stdio output services. If async stdio-like
>> reading is also worth providing in a generic way is something I would
>> like to see backed up by concrete application scenarios. Maybe.
>>
>>> Aside of purely ANSI services, there are some POSIX/SVID calls which are
>>> not relevant to libpthread_rt, but for which a hardened counterpart
>>> would make sense over a co-kernel, like SunRPC (over RTNet's UDP, that
>>> is), syslog routines, random, or libcrypt members for instance.
>>>
>>> My point being that we cannot anticipate the exact list of features we
>>> would want to harden in the future, but we do know already that
>>> applications ported/created over Xenomai are always more complex, so we
>>> will likely need more of those features. Componentization must have a
>>> limit, and my guts feeling is that going for a specific API for each and
>>> every feature would lead to maintenance chaos and user confusion.
>>> Additionally, sharing internal routines common to those features would
>>> be possible with a centralized architecture (unless we go for
>>> yet-another-library exporting those routines).
>> Well, would you be more happy if we call the printing library
>> libstdio_rt without yet implementing all the related services, but
>> pointing out the potential path of this lib in the future?
>>
> 
> There would remain the malloc support issue, which would obviously not
> fit in the stdio lib.

True.

> 
>>>>> - the same way libpthread_rt shadows 1003.1c services to iron them over
>>>>> a co-scheduler, we should do the same for the ironed ANSI services. In
>>>>> that sense, there is no need for rt_printf, rt_vsprintf and so on, we
>>>>> just need to shadow printf, vsprintf and friends the same way we did for
>>>>> pthread_create and such for substituting the 1003.1c support. Again, API
>>>>> proliferation, especially of non-orthogonal stuff, needs to be fought
>>>>> now. The bonus is that we would not have to ask people to rely on crufty
>>>>> preprocessor tricks in order to compile their code in either real-time
>>>>> layers X3 is going to provide (i.e. I-pipe-based co-scheduler and native
>>>>> preemption). This would be consistent with the all-time Xenomai's
>>>>> mantra, i.e. "integrate as seamlessly as you can, don't get uselessly
>>>>> peculiar".
>>>>>
>>>> Think of RTDM: we have transparent overloading with the POSIX skin, but
>>>> all others use the explicit selection via rt_dev prefix. I forgot to
>>>> mention this, but my plan was to apply the same pattern for librtprint
>>>> services.
>>>>
>>>> I don't think we should overload every printf outside the POSIX skin's
>>>> scope automatically. Well, at least for the native skin, which is in my
>>>> opinion quite a lot about _explicit_ real-time programming, I would
>>>> really prefer to keep it like it is even under Xenomai 3.
>>>>
>>> IMO, the only way to make "explicit real-time programming" a rule is to
>>> disable all access to real-time unsafe calls, i.e. basically to the
>>> glibc, and this is obviously something we don't want to, because this
>>> would contradict the fundamental goal of Xenomai toward integration
>>> within the regular Linux programming model.
>> No, disabling such access is surely a no-go. But I really think we
>> should strengthen the mode-switch detection, maybe be considering to
>> invert its default (always or just for debugging). Mode-switch detection
>> by the user space tracer could be a first step in this direction
>> (without immediately breaking the API).
>>
> 
> Agreed, other means to strengthen the alerts about loss of
> predictability should be found.
> 
>>> Because of this, that's a fact of life that users are most often going
>>> to consider that a commonly used glibc routine they need is real-time
>>> safe enough to be used until proven wrong (*), instead of suspecting
>>> such routine to be unsafe a priori, and looking for a safe replacement
>>> in some specific API we might provide (this is why we all had to warn a
>>> number of people countless times on the list about their use of printf()
>>> in deemed time-critical sections from various pieces of code they showed
>>> us).
>>>
>>> This is also the reason why I still think we should "upgrade" the
>>> commonly used standard routines to be real-time aware by gradually
>>> providing hardened replacements for them, not trying to convince people
>>> about the fact that using standard routines might not be the right
>>> solution to common problems in the real-time case.
>>>
>>> (*) actually, the real problem is that some users might think that we
>>> are smart enough to make deterministic and efficient code out of any
>>> regular Linux service they might think of, albeit the founder of the
>>> Xenomai project might be a total bonehead. Oh, well, anyway...
>>>
>> As I said, overloading, just trying to shadow the differences doesn't
>> solve the problem automagically. You often need special configuration
>> APIs for the shadowed one, thus you need understanding of its
>> mechanisms. And you need access to the original functions because of
>> drawbacks of the hardened variants (if they had none, we could just
>> submit glibc or whatever patches...).
>> Overloading is only as long comfortable as you _mostly_ make use of the
>> replacement services in your application. Once it is the other way
>> around, this automatism becomes quite a PITA. We luckily already have
>> two programming models with POSIX vs. native skin matching both use
>> cases, and I consider this a strength, not a weakness of Xenomai.
> 
> I did not say it was a weakness (even if I always thought that the
> rt_dev interface to RTDM was redundant with the POSIXish one), but that
> we should keep the finger on the bloat control when it comes to
> co-kernel-specific-APIs, because as much as there is life beyond x86,
> there is also life beyond co-kernels.

Again: we are not discussing co-kernel-specific APIs here. Rather, this
tiny little librtprint could be seen as a potential piece of a
"co-glibc" approach for optimising user-space real-time issues, actually
independent of the underlying kernel.

> 
> The natural tendency of Xenomai has always been to blur the frontier
> between the regular Linux environment and the co-kernel environment, so
> that people don't get schizophrenic when implementing a real-time
> application over Xenomai, with powerful features on both sides, but no
> ability to combine those strengthes efficiently. I'm convinced this has
> some value in the picture too.
> 

OK.

So, what is your suggestion for the printing services? How can we merge
them while not yet knowing precisely what services we would like to add
in the future, and how much code they will carry into whatever Xenomai
libraries?

Beyond this, how to find The Right usage model (API, required
compiler/linker switches)? Your suggestion is to wrap every *printf if
one decides to use them (BTW, how to express this decision?). Mine is to
only wrap for the POSIX skin integration, maybe also for legacy RTOS skins.

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]