[Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Richard Cochran]

This patch adds a class driver for raw Ethernet drivers under
Xenomai. The goal is to support industrial protocols such as EtherCAT
and IEC 61850, where the "stack" is a user space program needing
direct access at the packet level. The class driver offers interfaces
for registration, buffer management, and packet sending/receiving.

Although this patch is a kind of first draft, still I have it working
on the Freescale P2020 with a real world application, with very good
results. I can post a patch series for the gianfar driver in the ipipe
tree, if anyone is interested.

The user space interface is a character device and not a socket, simply
because my applications will probably never need fancy socket
options. The class driver could surely be made to offer a socket
instead, but I think the character is sufficient.

The class driver is clearly in the wrong directory within the source
tree, but I put it there just to get started. It really does not fit
to any of the other drivers, so it probably would need its own place
under ksrc/drivers.

Thanks in advance for your comments,

Richard


Richard Cochran (1):
  Add a class driver for raw Ethernet packets.

 include/rtdm/rtpacket.h       |   99 ++++++++++++
 ksrc/drivers/testing/Kconfig  |    8 +
 ksrc/drivers/testing/Makefile |    3 +
 ksrc/drivers/testing/packet.c |  352 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 462 insertions(+), 0 deletions(-)
 create mode 100644 include/rtdm/rtpacket.h
 create mode 100644 ksrc/drivers/testing/packet.c

-- 
1.7.2.5

[Xenomai-core] [RFC 1/1] Add a class driver for raw Ethernet packets.

[Richard Cochran]

This patch adds a class driver for sending and receiving raw Ethernet
packets over a character device interface.

Signed-off-by: Richard Cochran <richard.cochran@domain.hid>
---
 include/rtdm/rtpacket.h       |   99 ++++++++++++
 ksrc/drivers/testing/Kconfig  |    8 +
 ksrc/drivers/testing/Makefile |    3 +
 ksrc/drivers/testing/packet.c |  352 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 462 insertions(+), 0 deletions(-)
 create mode 100644 include/rtdm/rtpacket.h
 create mode 100644 ksrc/drivers/testing/packet.c

diff --git a/include/rtdm/rtpacket.h b/include/rtdm/rtpacket.h
new file mode 100644
index 0000000..87930de
--- /dev/null
+++ b/include/rtdm/rtpacket.h
@@ -0,0 +1,99 @@
+/*
+ * Xenomai Packet Interface - class driver for raw Ethernet packets
+ *
+ * Copyright (C) 2011 OMICRON electronics GmbH
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+#ifndef _RTPACKET_H
+#define _RTPACKET_H
+
+#include <rtdm/rtdm.h>
+
+#define RTPACKET_MAX_FILTER	16
+
+struct rtpacket_filter {
+	__u16 ethertype[RTPACKET_MAX_FILTER];
+};
+
+#define RTPACKET_FILTER _IOW(RTDM_CLASS_TESTING, 0x80, struct rtpacket_filter)
+
+#ifdef __KERNEL__
+
+#include <linux/list.h>
+#include <rtdm/rtdm_driver.h>
+
+#define RTPACKET_TX_POOL_SIZE	128
+#define RTPACKET_BUFFER_SIZE	1536
+
+struct rtp_addr {
+	void *va;
+	dma_addr_t pa;
+};
+
+struct rtpacket {
+	struct list_head list;
+	struct rtp_addr base;
+	struct rtp_addr data;
+	int length;
+};
+
+struct rtpacket_pool {
+	struct list_head list;
+	rtdm_lock_t lock;
+	struct rtpacket *pkt;
+	unsigned int npkt;
+	unsigned int pktsize;
+	struct device *dev;
+};
+
+struct rtpacket_driver_ops {
+	struct module *owner;
+	char name[16];
+	int (*filter)(struct rtpacket_driver_ops *p, struct rtpacket_filter *f);
+	int (*transmit)(struct rtpacket_driver_ops *ops, struct rtpacket *rtp);
+	void (*recycle)(struct rtpacket_driver_ops *ops, struct rtpacket *rtp);
+};
+
+struct rtpacket_interface;
+
+/* packet pool methods */
+
+void rtpacket_pool_drain(struct rtpacket_pool *p);
+
+int rtpacket_pool_init(struct rtpacket_pool *p, struct device *dev,
+		       unsigned int reserved,
+		       unsigned int npkt, unsigned int bufsize);
+
+struct rtpacket *rtpacket_pool_pop(struct rtpacket_pool *p);
+
+void rtpacket_pool_push(struct rtpacket_pool *p, struct rtpacket *rtp);
+
+/* registration methods */
+
+void rtpacket_deregister(struct rtpacket_interface *iface);
+
+struct rtpacket_interface *rtpacket_register(struct rtpacket_driver_ops *ops,
+					     struct device *dev,
+					     unsigned int tx_reserved);
+
+/* receive and transmit path methods */
+
+int rtpacket_receive(struct rtpacket_interface *iface, struct rtpacket *rtp);
+
+void rtpacket_recycle(struct rtpacket_interface *iface, struct rtpacket *rtp);
+
+#endif /* __KERNEL__ */
+#endif
diff --git a/ksrc/drivers/testing/Kconfig b/ksrc/drivers/testing/Kconfig
index 28504dc..6f67e7c 100644
--- a/ksrc/drivers/testing/Kconfig
+++ b/ksrc/drivers/testing/Kconfig
@@ -9,6 +9,14 @@ config XENO_DRIVERS_TESTING_LEGACY_NAMES
 	Only enable this if you plan to use old userspace tools with the
 	drivers.
 
+config XENO_DRIVERS_PACKET
+	depends on XENO_SKIN_RTDM
+	tristate "Raw Ethernet packet driver"
+	default y
+	help
+	Provides a class driver for sending and receiving raw Ethernet
+	packets over a simple character device interface.
+
 config XENO_DRIVERS_TIMERBENCH
 	depends on XENO_SKIN_RTDM
 	tristate "Timer benchmark driver"
diff --git a/ksrc/drivers/testing/Makefile b/ksrc/drivers/testing/Makefile
index 17a6cf1..78679c8 100644
--- a/ksrc/drivers/testing/Makefile
+++ b/ksrc/drivers/testing/Makefile
@@ -4,6 +4,7 @@ ifeq ($(PATCHLEVEL),6)
 
 EXTRA_CFLAGS += -D__IN_XENOMAI__ -Iinclude/xenomai
 
+obj-$(CONFIG_XENO_DRIVERS_PACKET)     += xeno_packet.o
 obj-$(CONFIG_XENO_DRIVERS_TIMERBENCH) += xeno_timerbench.o
 obj-$(CONFIG_XENO_DRIVERS_IRQBENCH)   += xeno_irqbench.o
 obj-$(CONFIG_XENO_DRIVERS_SWITCHTEST) += xeno_switchtest.o
@@ -11,6 +12,8 @@ obj-$(CONFIG_XENO_DRIVERS_KLATENCY)   += xeno_klat.o
 obj-$(CONFIG_XENO_DRIVERS_SIGTEST)    += xeno_sigtest.o
 obj-$(CONFIG_XENO_DRIVERS_RTDMTEST)   += xeno_rtdmtest.o
 
+xeno_packet-y := packet.o
+
 xeno_timerbench-y := timerbench.o
 
 xeno_irqbench-y := irqbench.o
diff --git a/ksrc/drivers/testing/packet.c b/ksrc/drivers/testing/packet.c
new file mode 100644
index 0000000..b933888
--- /dev/null
+++ b/ksrc/drivers/testing/packet.c
@@ -0,0 +1,352 @@
+/*
+ * Xenomai Packet Interface - class driver for raw Ethernet packets
+ *
+ * Copyright (C) 2011 OMICRON electronics GmbH
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, write to the Free Software
+ *  Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+#include <linux/dma-mapping.h>
+#include <linux/module.h>
+#include <rtdm/rtpacket.h>
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("richard.cochran@domain.hid");
+
+struct rtpacket_interface {
+	struct device *dev;
+	struct rtdm_device rtdev;
+	struct rtpacket_driver_ops *ops;
+	struct rtpacket_pool tx_pool;
+	struct rtpacket_pool rx_fifo;
+	rtdm_event_t rx_event;
+	nanosecs_rel_t rx_tmo;
+};
+
+/* Receive fifo */
+
+static struct rtpacket *fifo_dequeue(struct rtpacket_pool *p)
+{
+	return rtpacket_pool_pop(p);
+}
+
+static int fifo_empty(struct rtpacket_pool *p)
+{
+	int empty;
+	rtdm_lockctx_t ctx;
+	rtdm_lock_get_irqsave(&p->lock, ctx);
+	empty = list_empty(&p->list);
+	rtdm_lock_put_irqrestore(&p->lock, ctx);
+	return empty;
+}
+
+static void fifo_enqueue(struct rtpacket_pool *p, struct rtpacket *rtp)
+{
+	rtdm_lockctx_t ctx;
+	rtdm_lock_get_irqsave(&p->lock, ctx);
+	list_add_tail(&rtp->list, &p->list);
+	rtdm_lock_put_irqrestore(&p->lock, ctx);
+}
+
+/* Device operations */
+
+static int rtpacket_open(struct rtdm_dev_context *ctx,
+			 rtdm_user_info_t *info, int oflags)
+{
+	return 0;
+}
+
+static int rtpacket_close(struct rtdm_dev_context *ctx, rtdm_user_info_t *info)
+{
+	return 0;
+}
+
+static int rtpacket_ioctl_rt(struct rtdm_dev_context *ctx,
+			     rtdm_user_info_t *info,
+			     unsigned int request, void __user *arg)
+{
+	struct rtpacket_interface *pif = ctx->device->device_data;
+	struct rtpacket_filter __user *user = arg;
+	struct rtpacket_filter kf;
+	int err = 0;
+
+	switch (request) {
+	case RTPACKET_FILTER:
+		err = rtdm_safe_copy_from_user(info, &kf, user, sizeof(kf));
+		if (err)
+			return err;
+
+		err = pif->ops->filter(pif->ops, &kf);
+		break;
+	default:
+		err = -ENOTTY;
+	}
+
+	return err;
+}
+
+static ssize_t rtpacket_read_rt(struct rtdm_dev_context *ctx,
+				rtdm_user_info_t *info, void *buf, size_t nbyte)
+{
+	struct rtpacket_interface *pif = ctx->device->device_data;
+	struct rtpacket *rtp;
+	size_t len;
+	int err;
+
+	if (nbyte > RTPACKET_BUFFER_SIZE)
+		return -EINVAL;
+
+	if (fifo_empty(&pif->rx_fifo)) {
+		err = rtdm_event_timedwait(&pif->rx_event, pif->rx_tmo, NULL);
+		if (err == -ETIMEDOUT)
+			return 0;
+		else if (err)
+			return err;
+	}
+	rtp = fifo_dequeue(&pif->rx_fifo);
+	if (!rtp)
+		return 0;
+
+	len = rtp->length;
+	if (len > nbyte)
+		len = nbyte;
+
+	err = rtdm_safe_copy_to_user(info, buf, rtp->data.va, len);
+
+	pif->ops->recycle(pif->ops, rtp);
+
+	return err ? err : len;
+}
+
+static ssize_t rtpacket_write_rt(struct rtdm_dev_context *ctx,
+				 rtdm_user_info_t *info,
+				 const void *buf, size_t nbyte)
+{
+	struct rtpacket_interface *pif = ctx->device->device_data;
+	struct rtpacket *rtp;
+	int err;
+
+	if (nbyte > RTPACKET_BUFFER_SIZE)
+		return -EINVAL;
+
+	rtp = rtpacket_pool_pop(&pif->tx_pool);
+	if (!rtp)
+		return -EBUSY;
+
+	err = rtdm_safe_copy_from_user(info, rtp->data.va, buf, nbyte);
+	if (err)
+		goto bail;
+
+	rtp->length = nbyte;
+
+	err = pif->ops->transmit(pif->ops, rtp);
+	if (err)
+		goto bail;
+
+	return nbyte;
+bail:
+	rtpacket_pool_push(&pif->tx_pool, rtp);
+	return err;
+}
+
+/* Public methods */
+
+void rtpacket_pool_drain(struct rtpacket_pool *p)
+{
+	struct rtpacket *rtp;
+	int count;
+
+	if (!p->npkt)
+		return;
+
+	for (count = 0; ; count++) {
+		rtp = rtpacket_pool_pop(p);
+		if (!rtp)
+			break;
+	}
+
+	if (count != p->npkt)
+		pr_err("packet pool mismatch, %d of %d", count, p->npkt);
+
+	dma_free_coherent(p->dev, p->npkt * p->pktsize,
+			  p->pkt[0].base.va, p->pkt[0].base.pa);
+
+	kfree(p->pkt);
+}
+EXPORT_SYMBOL_GPL(rtpacket_pool_drain);
+
+int rtpacket_pool_init(struct rtpacket_pool *p, struct device *dev,
+		       unsigned int reserved,
+		       unsigned int npkt, unsigned int bufsize)
+{
+	void *va;
+	dma_addr_t pa;
+	int i;
+
+	INIT_LIST_HEAD(&p->list);
+	rtdm_lock_init(&p->lock);
+	p->pkt = NULL;
+	p->npkt = npkt;
+	p->pktsize = bufsize;
+	p->dev = dev;
+
+	if (!npkt)
+		return 0;
+
+	p->pkt = kmalloc(npkt * sizeof(*p->pkt), GFP_KERNEL);
+	if (!p->pkt)
+		goto no_pkt;
+
+	va = dma_alloc_coherent(dev, npkt * bufsize, &pa, GFP_KERNEL);
+	if (!va)
+		goto no_dma;
+
+	for (i = 0; i < npkt; i++) {
+		p->pkt[i].base.va = va;
+		p->pkt[i].base.pa = pa;
+		p->pkt[i].data.va = va + reserved;
+		p->pkt[i].data.pa = pa + reserved;
+		rtpacket_pool_push(p, &p->pkt[i]);
+		va += bufsize;
+		pa += bufsize;
+	}
+	return 0;
+no_dma:
+	kfree(p->pkt);
+no_pkt:
+	return -ENOMEM;
+}
+EXPORT_SYMBOL_GPL(rtpacket_pool_init);
+
+struct rtpacket *rtpacket_pool_pop(struct rtpacket_pool *p)
+{
+	struct rtpacket *rtp = NULL;
+	rtdm_lockctx_t ctx;
+	rtdm_lock_get_irqsave(&p->lock, ctx);
+	if (!list_empty(&p->list)) {
+		rtp = list_first_entry(&p->list, struct rtpacket, list);
+		list_del_init(&rtp->list);
+	}
+	rtdm_lock_put_irqrestore(&p->lock, ctx);
+	return rtp;
+}
+EXPORT_SYMBOL_GPL(rtpacket_pool_pop);
+
+void rtpacket_pool_push(struct rtpacket_pool *p, struct rtpacket *rtp)
+{
+	rtdm_lockctx_t ctx;
+	rtdm_lock_get_irqsave(&p->lock, ctx);
+	list_add(&rtp->list, &p->list);
+	rtdm_lock_put_irqrestore(&p->lock, ctx);
+}
+EXPORT_SYMBOL_GPL(rtpacket_pool_push);
+
+void rtpacket_deregister(struct rtpacket_interface *pif)
+{
+	rtdm_dev_unregister(&pif->rtdev, 10);
+	rtpacket_pool_drain(&pif->rx_fifo);
+	rtpacket_pool_drain(&pif->tx_pool);
+	rtdm_event_destroy(&pif->rx_event);
+	kfree(pif);
+}
+EXPORT_SYMBOL_GPL(rtpacket_deregister);
+
+struct rtpacket_interface *rtpacket_register(struct rtpacket_driver_ops *ops,
+					     struct device *dev,
+					     unsigned int tx_reserved)
+{
+	struct rtpacket_interface *pif;
+
+	pif = kzalloc(sizeof(struct rtpacket_interface), GFP_KERNEL);
+	if (pif == NULL)
+		goto no_memory;
+
+	pif->ops = ops;
+	snprintf(pif->rtdev.device_name, RTDM_MAX_DEVNAME_LEN,
+		 "rt-%s", ops->name);
+	pif->rtdev.struct_version	= RTDM_DEVICE_STRUCT_VER;
+	pif->rtdev.device_flags		= RTDM_NAMED_DEVICE;
+	pif->rtdev.context_size		= 0;
+	pif->rtdev.open_nrt		= rtpacket_open;
+	pif->rtdev.ops.close_nrt	= rtpacket_close;
+	pif->rtdev.ops.ioctl_rt		= rtpacket_ioctl_rt,
+	pif->rtdev.ops.read_rt		= rtpacket_read_rt;
+	pif->rtdev.ops.write_rt		= rtpacket_write_rt;
+	pif->rtdev.device_class		= RTDM_CLASS_EXPERIMENTAL;
+	pif->rtdev.device_sub_class	= 0;
+	pif->rtdev.profile_version	= 1;
+	pif->rtdev.driver_name		= pif->rtdev.device_name;
+	pif->rtdev.driver_version	= RTDM_DRIVER_VER(1,0,0);
+	pif->rtdev.peripheral_name	= pif->rtdev.device_name;
+	pif->rtdev.provider_name	= pif->rtdev.device_name;
+	pif->rtdev.proc_name		= pif->rtdev.device_name;
+	pif->rtdev.device_id		= 0;
+	pif->rtdev.device_data		= pif;
+
+	if (rtpacket_pool_init(&pif->tx_pool, dev, tx_reserved,
+			       RTPACKET_TX_POOL_SIZE,
+			       RTPACKET_BUFFER_SIZE))
+		goto no_txpool;
+
+	if (rtpacket_pool_init(&pif->rx_fifo, dev, 0, 0, 0))
+		goto no_rxfifo;
+
+	rtdm_event_init(&pif->rx_event, 0);
+	pif->rx_tmo = 1000000000;
+
+	if (rtdm_dev_register(&pif->rtdev)) {
+		pr_err("register rtdm %s failed\n", pif->rtdev.device_name);
+		goto no_register;
+	}
+
+	return pif;
+
+no_register:
+	rtpacket_pool_drain(&pif->rx_fifo);
+no_rxfifo:
+	rtpacket_pool_drain(&pif->tx_pool);
+no_txpool:
+	kfree(pif);
+no_memory:
+	return NULL;
+}
+EXPORT_SYMBOL_GPL(rtpacket_register);
+
+int rtpacket_receive(struct rtpacket_interface *pif, struct rtpacket *rtp)
+{
+	fifo_enqueue(&pif->rx_fifo, rtp);
+	rtdm_event_signal(&pif->rx_event);
+	return 0;
+}
+EXPORT_SYMBOL_GPL(rtpacket_receive);
+
+void rtpacket_recycle(struct rtpacket_interface *pif, struct rtpacket *rtp)
+{
+	rtpacket_pool_push(&pif->tx_pool, rtp);
+}
+EXPORT_SYMBOL_GPL(rtpacket_recycle);
+
+/* Module code */
+
+static int __init rtpacket_init(void)
+{
+	return 0;
+}
+
+static void rtpacket_exit(void)
+{
+}
+
+subsys_initcall(rtpacket_init);
+module_exit(rtpacket_exit);
-- 
1.7.2.5

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Richard Cochran]

PS Applies to Xenomai 2.5.6.

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Peter Soetens]

On Friday 23 September 2011 13:02:19 Richard Cochran wrote:
> This patch adds a class driver for raw Ethernet drivers under
> Xenomai. The goal is to support industrial protocols such as EtherCAT
> and IEC 61850, where the "stack" is a user space program needing
> direct access at the packet level. The class driver offers interfaces
> for registration, buffer management, and packet sending/receiving.
> 
> Although this patch is a kind of first draft, still I have it working
> on the Freescale P2020 with a real world application, with very good
> results. I can post a patch series for the gianfar driver in the ipipe
> tree, if anyone is interested.
> 
> The user space interface is a character device and not a socket, simply
> because my applications will probably never need fancy socket
> options. The class driver could surely be made to offer a socket
> instead, but I think the character is sufficient.
> 
> The class driver is clearly in the wrong directory within the source
> tree, but I put it there just to get started. It really does not fit
> to any of the other drivers, so it probably would need its own place
> under ksrc/drivers.
> 
> Thanks in advance for your comments,

How does this relate to rtnet, i.e. why didn't you write an rtnet driver ? 

Peter

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Jan Kiszka]

On 2011-09-23 13:02, Richard Cochran wrote:
> This patch adds a class driver for raw Ethernet drivers under
> Xenomai. The goal is to support industrial protocols such as EtherCAT
> and IEC 61850, where the "stack" is a user space program needing
> direct access at the packet level. The class driver offers interfaces
> for registration, buffer management, and packet sending/receiving.
> 
> Although this patch is a kind of first draft, still I have it working
> on the Freescale P2020 with a real world application, with very good
> results. I can post a patch series for the gianfar driver in the ipipe
> tree, if anyone is interested.
> 
> The user space interface is a character device and not a socket, simply
> because my applications will probably never need fancy socket
> options. The class driver could surely be made to offer a socket
> instead, but I think the character is sufficient.

Many interesting interfaces already exists for standard PF_PACKET (e.g.
time stamping), plus you gain portability this way. So let's not create
something special here even if it's sufficient for the first use case.

> 
> The class driver is clearly in the wrong directory within the source
> tree, but I put it there just to get started. It really does not fit
> to any of the other drivers, so it probably would need its own place
> under ksrc/drivers.

New class, new directory.

However, the key question is how this approach relates to RTnet. Right
now its likely comparing apples to onions, but that may change as things
evolve in the right direction.

Can you explain a bit about your NIC driver architecture and the
maintenance strategy? It looks like you are taking an approach of
patching existing drivers in-tree. Anything that resolves the
maintenance pain we have with the RTnet model is already worth
considering. Did you take this approach intentionally? What pros and
cons do you see?

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Richard Cochran]

On Fri, Sep 23, 2011 at 03:50:42PM +0200, Jan Kiszka wrote:
> On 2011-09-23 13:02, Richard Cochran wrote:
> > This patch adds a class driver for raw Ethernet drivers under
> > Xenomai. The goal is to support industrial protocols such as EtherCAT
> > and IEC 61850, where the "stack" is a user space program needing
> > direct access at the packet level. The class driver offers interfaces
> > for registration, buffer management, and packet sending/receiving.
> > 
> > Although this patch is a kind of first draft, still I have it working
> > on the Freescale P2020 with a real world application, with very good
> > results. I can post a patch series for the gianfar driver in the ipipe
> > tree, if anyone is interested.
> > 
> > The user space interface is a character device and not a socket, simply
> > because my applications will probably never need fancy socket
> > options. The class driver could surely be made to offer a socket
> > instead, but I think the character is sufficient.
> 
> Many interesting interfaces already exists for standard PF_PACKET (e.g.
> time stamping), plus you gain portability this way. So let's not create
> something special here even if it's sufficient for the first use case.

Okay, so if the raw packet driver idea finds acceptance (outside of
rtnet), then I am willing to recast the thing as a socket.
 
> > 
> > The class driver is clearly in the wrong directory within the source
> > tree, but I put it there just to get started. It really does not fit
> > to any of the other drivers, so it probably would need its own place
> > under ksrc/drivers.
> 
> New class, new directory.
> 
> However, the key question is how this approach relates to RTnet. Right
> now its likely comparing apples to onions, but that may change as things
> evolve in the right direction.
> 
> Can you explain a bit about your NIC driver architecture and the
> maintenance strategy? It looks like you are taking an approach of
> patching existing drivers in-tree.

Yes, that is right.

> Anything that resolves the
> maintenance pain we have with the RTnet model is already worth
> considering. Did you take this approach intentionally? What pros and
> cons do you see?

So, here is my story. Many months ago I needed a Xenomai program to
send and receive raw packets. I looked at rtnet and decided that it
was way too complicated. So, I added a rtdm character device to the
Linux Ethernet MAC driver, a rather simple hack:

 drivers/net/arm/Makefile     |    4 +
 drivers/net/arm/ixp4xx_eth.c |  316
 +++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 319 insertions(+), 1 deletions(-)

That worked great. I did not bother to implement more than read/write,
but I used a normal socket to set the multicast Rx list.

Now, I needed to port that application to a new platform. Again, I
looked at rtnet (and the recently posted gianfar code), and again I
came to the conclusion that I was better off with a direct driver
hack.

Most of those 316 additional lines, above, were rtdm driver boiler
plate, and I this time I wanted to abstract the code that will surely
be the same, like device registration, user space send/receive, and
buffer management. (Because the eTSEC (gianfar) is much more complex,
and because I wanted hardware packet classification, the driver patch
is much larger than before.)

I admire the rtnet project. Based on my limited understanding of it,
if I really needed deterministic Ethernet, then I would write a brand
new driver for the eTSEC (perhaps using gianfar.c as a reference), but
I would *not* try and adapt the Linux driver.

Conversely, if I were writing a brand new driver, I would surely offer
it as an rtnet driver, even thought I only need raw packets.

However, for the sole requirement of raw Ethernet, I think my simple
driver API is much easier to work into existing drivers. I also think
it is way easier to maintain by rebasing on later changes.

IMHO, the pros of my approach are:

- Simple to implement new drivers.

  Compare my rtpacket.h with the rtnet driver headers to see what I
  mean. Or, read rtnet/Documentation/README.drvporting and ask
  yourself, is it easy to port a driver to rtnet?

- Easier to maintain drivers.

  Making regular drivers into real time drivers will always be a
  chore. But, with its simple interface, the packet class driver hack
  is way less painful. (Look at my gianfar example. There really are
  not many changes to the Linux driver.)

- Better integration with Xenomai. 

  This class driver is simple enough to ship with Xenomai. It would be
  nice to offer raw Ethernet to users directly, not make them jump
  through addition hoops. For example, the last time I tried to
  compile the latest rtnet from git (few months ago) I got:

     checking for Xenomai version... configure: error:
     *** Unsupported Xenomai version 2.5.6 in home/cochran/stage/usr/xenomai

I really don't know how many people would need such an interface (and
not need rtnet), but I would imagine that a need exists.

Thanks,

Richard

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Jan Kiszka]

On 2011-09-26 13:41, Richard Cochran wrote:
> On Fri, Sep 23, 2011 at 03:50:42PM +0200, Jan Kiszka wrote:
>> On 2011-09-23 13:02, Richard Cochran wrote:
>>> This patch adds a class driver for raw Ethernet drivers under
>>> Xenomai. The goal is to support industrial protocols such as EtherCAT
>>> and IEC 61850, where the "stack" is a user space program needing
>>> direct access at the packet level. The class driver offers interfaces
>>> for registration, buffer management, and packet sending/receiving.
>>>
>>> Although this patch is a kind of first draft, still I have it working
>>> on the Freescale P2020 with a real world application, with very good
>>> results. I can post a patch series for the gianfar driver in the ipipe
>>> tree, if anyone is interested.
>>>
>>> The user space interface is a character device and not a socket, simply
>>> because my applications will probably never need fancy socket
>>> options. The class driver could surely be made to offer a socket
>>> instead, but I think the character is sufficient.
>>
>> Many interesting interfaces already exists for standard PF_PACKET (e.g.
>> time stamping), plus you gain portability this way. So let's not create
>> something special here even if it's sufficient for the first use case.
> 
> Okay, so if the raw packet driver idea finds acceptance (outside of
> rtnet), then I am willing to recast the thing as a socket.
>  
>>>
>>> The class driver is clearly in the wrong directory within the source
>>> tree, but I put it there just to get started. It really does not fit
>>> to any of the other drivers, so it probably would need its own place
>>> under ksrc/drivers.
>>
>> New class, new directory.
>>
>> However, the key question is how this approach relates to RTnet. Right
>> now its likely comparing apples to onions, but that may change as things
>> evolve in the right direction.
>>
>> Can you explain a bit about your NIC driver architecture and the
>> maintenance strategy? It looks like you are taking an approach of
>> patching existing drivers in-tree.
> 
> Yes, that is right.
> 
>> Anything that resolves the
>> maintenance pain we have with the RTnet model is already worth
>> considering. Did you take this approach intentionally? What pros and
>> cons do you see?
> 
> So, here is my story. Many months ago I needed a Xenomai program to
> send and receive raw packets. I looked at rtnet and decided that it
> was way too complicated. So, I added a rtdm character device to the
> Linux Ethernet MAC driver, a rather simple hack:
> 
>  drivers/net/arm/Makefile     |    4 +
>  drivers/net/arm/ixp4xx_eth.c |  316
>  +++++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 319 insertions(+), 1 deletions(-)
> 
> That worked great. I did not bother to implement more than read/write,
> but I used a normal socket to set the multicast Rx list.
> 
> Now, I needed to port that application to a new platform. Again, I
> looked at rtnet (and the recently posted gianfar code), and again I
> came to the conclusion that I was better off with a direct driver
> hack.
> 
> Most of those 316 additional lines, above, were rtdm driver boiler
> plate, and I this time I wanted to abstract the code that will surely
> be the same, like device registration, user space send/receive, and
> buffer management. (Because the eTSEC (gianfar) is much more complex,
> and because I wanted hardware packet classification, the driver patch
> is much larger than before.)
> 
> I admire the rtnet project. Based on my limited understanding of it,
> if I really needed deterministic Ethernet, then I would write a brand
> new driver for the eTSEC (perhaps using gianfar.c as a reference), but
> I would *not* try and adapt the Linux driver.
> 
> Conversely, if I were writing a brand new driver, I would surely offer
> it as an rtnet driver, even thought I only need raw packets.
> 
> However, for the sole requirement of raw Ethernet, I think my simple
> driver API is much easier to work into existing drivers. I also think
> it is way easier to maintain by rebasing on later changes.
> 
> IMHO, the pros of my approach are:
> 
> - Simple to implement new drivers.
> 
>   Compare my rtpacket.h with the rtnet driver headers to see what I
>   mean. Or, read rtnet/Documentation/README.drvporting and ask
>   yourself, is it easy to port a driver to rtnet?

I would be careful with deriving generic properties from a potentially
lucky first example. Already tried to apply your pattern on a standard
PCI NIC, e.g. the common Intel 8257x series?

> 
> - Easier to maintain drivers.
> 
>   Making regular drivers into real time drivers will always be a
>   chore. But, with its simple interface, the packet class driver hack
>   is way less painful. (Look at my gianfar example. There really are
>   not many changes to the Linux driver.)

What is your "simple interface"? That's what I'm interested in. What is
the pattern to apply on an arbitrary driver to add RT support? How does
interface claiming work (so the RT is not conflicting with Linux)? How
does the configuration work? How do you deal with things like watchdogs,
error handling, IRQ coalescing avoidance, etc.? Hmm, looks like your
concept is completely lacking IRQ support. That's of course a major
limitation, specifically with RT Ethernet protocols where the network
provides a clock source.

> 
> - Better integration with Xenomai. 
> 
>   This class driver is simple enough to ship with Xenomai. It would be
>   nice to offer raw Ethernet to users directly, not make them jump
>   through addition hoops. For example, the last time I tried to
>   compile the latest rtnet from git (few months ago) I got:
> 
>      checking for Xenomai version... configure: error:
>      *** Unsupported Xenomai version 2.5.6 in home/cochran/stage/usr/xenomai
> 
> I really don't know how many people would need such an interface (and
> not need rtnet), but I would imagine that a need exists.

I'm surely not voting against Xenomai integration of some RT Ethernet
(micro-)stack. I'm even not voting against rewriting the RTnet mess from
scratch.

What I'm skeptic about are statements like "this is so much simpler
because it will never require X or Y". Reminds me of the people
reimplementing QEMU as kvm-tool. The architecture of an RT Ethernet
stack should at least be based on the experience of RTnet, not widely
ignore it.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Richard Cochran]

On Tue, Sep 27, 2011 at 10:26:51AM +0200, Jan Kiszka wrote:
> On 2011-09-26 13:41, Richard Cochran wrote:
> > - Simple to implement new drivers.
> > 
> >   Compare my rtpacket.h with the rtnet driver headers to see what I
> >   mean. Or, read rtnet/Documentation/README.drvporting and ask
> >   yourself, is it easy to port a driver to rtnet?
> 
> I would be careful with deriving generic properties from a potentially
> lucky first example. Already tried to apply your pattern on a standard
> PCI NIC, e.g. the common Intel 8257x series?

Of course, you are right about this. It would make sense to try a few
different flavors of MAC driver. But I do think my class API makes
sense from a design standpoint.

> > - Easier to maintain drivers.
> > 
> >   Making regular drivers into real time drivers will always be a
> >   chore. But, with its simple interface, the packet class driver hack
> >   is way less painful. (Look at my gianfar example. There really are
> >   not many changes to the Linux driver.)
> 
> What is your "simple interface"? That's what I'm interested in.

* class driver provides:
  - rtpacket_receive  Called by MAC driver when a packet arrives.
  - rtpacket_recycle  Called by MAC driver to yield transmitted buffers
                      back to class driver.
* MAC driver provides:
  - ops.transmit      Called by class driver to send a packet.
  - ops.recycle       Called by class driver to yield delivered buffers
                      back to MAC driver.
  - ops.filter        User space must provide a list of allowed Ether
                      types, and the MAC driver must filter out types
                      not in the list.
* That's all, folks.

> What is the pattern to apply on an arbitrary driver to add RT
> support?

There is no getting around the fact that you must get to know the MAC
driver (and perhaps HW). How to splice the interface into the driver
must be seen on a case by case basis.

> How does interface claiming work (so the RT is not conflicting with
> Linux)?

The idea is to always offer both, so every ethX creates a rt-ethX.
The RT traffic should always take priority over non-RT, to the extent
possible in the hardware.

> How does the configuration work?

The RT port is up whenever the normal network interface is. I do *not*
want to duplicate anything from the Linux network stack at all.

The only "nice-to-have" I think missing is MAC address filtering or
enabling mutlicast MAC reception or promiscuous mode. Since there are
global (at the hardware level), you can just set this using a normal
socket. But perhaps a user API that just calls the normal Linux kernel
code would work.

> How do you deal with things like watchdogs, error handling, IRQ
> coalescing avoidance, etc.?

Again, every MAC driver needs to be tastefully and wisely adapted. I
don't necessarily need to avoid coalescing. The goal (for me) is *not*
to provide deterministic Ethernet performance. Instead the RT packets
should just be delivered ASAP.

The userland RT send/recv methods must not block or switch to
secondary mode, however.

> Hmm, looks like your concept is completely lacking IRQ
> support. That's of course a major limitation, specifically with RT
> Ethernet protocols where the network provides a clock source.

I don't know of any Ethernet protocols where the network provides a
clock, except for NTP and PTP, and for these RT performance is not
required. (EtherCAT does not count because the clock source is in the
first slave.)

Again, I know that rtnet is making Ethernet deterministic, but I just
don't need this.

> I'm surely not voting against Xenomai integration of some RT Ethernet
> (micro-)stack. I'm even not voting against rewriting the RTnet mess from
> scratch.

So, I think raw Ethernet access is not even a micro-stack. It is no
stack at all, just packets. The stack logic, if any, goes into the
user space program.

> What I'm skeptic about are statements like "this is so much simpler
> because it will never require X or Y". Reminds me of the people
> reimplementing QEMU as kvm-tool. The architecture of an RT Ethernet
> stack should at least be based on the experience of RTnet, not widely
> ignore it.

no stack, no stack, no stack!

It sounds like to me that the requirements on rtnet drivers
practically force you to write a special rtnet driver. I don't need
rtnet, just raw Ethernet from a Xenomai program in primary mode. This
is relatively easy to do by splicing into the existing MAC drivers.

Thanks,

Richard

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Jan Kiszka]

On 2011-09-27 14:01, Richard Cochran wrote:
> Again, every MAC driver needs to be tastefully and wisely adapted. I
> don't necessarily need to avoid coalescing. The goal (for me) is *not*
> to provide deterministic Ethernet performance. Instead the RT packets
> should just be delivered ASAP.

This is obviously the point I completely missed. And it makes the whole
thing fairly uninteresting IMHO. If you want to do Ethercat, PowerLink
or Profinet (RT), you do need a certain level of determinism along the
*whole* packet path. And for the latter two, you definitely need RT IRQ
support, Ethercat can be OK to poll in fast setups.

>From that POV, your approach is likely OK. But I doubt its of generic
use, specifically for industrial RT Ethernet.

Jan

PS: You do have a stack, even if you don't like it: driver, packet
layer, application. :)

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Richard Cochran]

On Tue, Sep 27, 2011 at 02:20:43PM +0200, Jan Kiszka wrote:
> On 2011-09-27 14:01, Richard Cochran wrote:
> > Again, every MAC driver needs to be tastefully and wisely adapted. I
> > don't necessarily need to avoid coalescing. The goal (for me) is *not*
> > to provide deterministic Ethernet performance. Instead the RT packets
> > should just be delivered ASAP.
> 
> This is obviously the point I completely missed. And it makes the whole
> thing fairly uninteresting IMHO. If you want to do Ethercat, PowerLink
> or Profinet (RT), you do need a certain level of determinism along the
> *whole* packet path. And for the latter two, you definitely need RT IRQ
> support, Ethercat can be OK to poll in fast setups.
> 
> From that POV, your approach is likely OK. But I doubt its of generic
> use, specifically for industrial RT Ethernet.

So, how does rtnet support EtherCAT?

Does it support PowerLink and Profinet?

Thanks,
Richard

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Jan Kiszka]

On 2011-09-27 17:10, Richard Cochran wrote:
> On Tue, Sep 27, 2011 at 02:20:43PM +0200, Jan Kiszka wrote:
>> On 2011-09-27 14:01, Richard Cochran wrote:
>>> Again, every MAC driver needs to be tastefully and wisely adapted. I
>>> don't necessarily need to avoid coalescing. The goal (for me) is *not*
>>> to provide deterministic Ethernet performance. Instead the RT packets
>>> should just be delivered ASAP.
>>
>> This is obviously the point I completely missed. And it makes the whole
>> thing fairly uninteresting IMHO. If you want to do Ethercat, PowerLink
>> or Profinet (RT), you do need a certain level of determinism along the
>> *whole* packet path. And for the latter two, you definitely need RT IRQ
>> support, Ethercat can be OK to poll in fast setups.
>>
>> From that POV, your approach is likely OK. But I doubt its of generic
>> use, specifically for industrial RT Ethernet.
> 
> So, how does rtnet support EtherCAT?

There was once the EtherCAT Master Library. IIRC, it was discontinued
and removed from the web for non-technical reasons.

> 
> Does it support PowerLink and Profinet?

Not that I know, but that's not the point. You said your approach could
provide the technical foundation for such a class of use cases while I
doubt it would work as is.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Richard Cochran]

On Tue, Sep 27, 2011 at 05:16:09PM +0200, Jan Kiszka wrote:
> On 2011-09-27 17:10, Richard Cochran wrote:
> > On Tue, Sep 27, 2011 at 02:20:43PM +0200, Jan Kiszka wrote:
> >> On 2011-09-27 14:01, Richard Cochran wrote:
> >>> Again, every MAC driver needs to be tastefully and wisely adapted. I
> >>> don't necessarily need to avoid coalescing. The goal (for me) is *not*
> >>> to provide deterministic Ethernet performance. Instead the RT packets
> >>> should just be delivered ASAP.
> >>
> >> This is obviously the point I completely missed. And it makes the whole
> >> thing fairly uninteresting IMHO. If you want to do Ethercat, PowerLink
> >> or Profinet (RT), you do need a certain level of determinism along the
> >> *whole* packet path. And for the latter two, you definitely need RT IRQ
> >> support, Ethercat can be OK to poll in fast setups.
> >>
> >> From that POV, your approach is likely OK. But I doubt its of generic
> >> use, specifically for industrial RT Ethernet.
> > 
> > So, how does rtnet support EtherCAT?
> 
> There was once the EtherCAT Master Library. IIRC, it was discontinued
> and removed from the web for non-technical reasons.
> 
> > 
> > Does it support PowerLink and Profinet?
> 
> Not that I know, but that's not the point. You said your approach could
> provide the technical foundation for such a class of use cases while I
> doubt it would work as is.

But that is the point. Correct me please if I am wrong, but isn't
rtnet a competitor or alternative to EtherCAT, PowerLink, and
Profinet?

Unless rtnet implements (or helps to implement) these, it is kind of
silly to say, "your way won't work, you should use rtnet instead."

I don't know PowerLink or Profinet, but I do know EtherCAT and IEC
61850, and those two can surely be implemented on the interface that I
am talking about.

Also, the interface that I propose does not preclude the use of RTDM
ISR in the driver, so I wonder what exactly bothers you about it?

Thanks,
Richard

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Jan Kiszka]

On 2011-09-27 18:05, Richard Cochran wrote:
> On Tue, Sep 27, 2011 at 05:16:09PM +0200, Jan Kiszka wrote:
>> On 2011-09-27 17:10, Richard Cochran wrote:
>>> On Tue, Sep 27, 2011 at 02:20:43PM +0200, Jan Kiszka wrote:
>>>> On 2011-09-27 14:01, Richard Cochran wrote:
>>>>> Again, every MAC driver needs to be tastefully and wisely adapted. I
>>>>> don't necessarily need to avoid coalescing. The goal (for me) is *not*
>>>>> to provide deterministic Ethernet performance. Instead the RT packets
>>>>> should just be delivered ASAP.
>>>>
>>>> This is obviously the point I completely missed. And it makes the whole
>>>> thing fairly uninteresting IMHO. If you want to do Ethercat, PowerLink
>>>> or Profinet (RT), you do need a certain level of determinism along the
>>>> *whole* packet path. And for the latter two, you definitely need RT IRQ
>>>> support, Ethercat can be OK to poll in fast setups.
>>>>
>>>> From that POV, your approach is likely OK. But I doubt its of generic
>>>> use, specifically for industrial RT Ethernet.
>>>
>>> So, how does rtnet support EtherCAT?
>>
>> There was once the EtherCAT Master Library. IIRC, it was discontinued
>> and removed from the web for non-technical reasons.
>>
>>>
>>> Does it support PowerLink and Profinet?
>>
>> Not that I know, but that's not the point. You said your approach could
>> provide the technical foundation for such a class of use cases while I
>> doubt it would work as is.
> 
> But that is the point. Correct me please if I am wrong, but isn't
> rtnet a competitor or alternative to EtherCAT, PowerLink, and
> Profinet?

That's a common misunderstanding: RTnet is a networking stack with many
_optional_ components (like RTmac, RTcfg etc.). I would bet that it's
more frequently used today in minimal setups, i.e. just the core, some
driver, and either PF_PACKET or UDP/IP.

> 
> Unless rtnet implements (or helps to implement) these, it is kind of
> silly to say, "your way won't work, you should use rtnet instead."
> 
> I don't know PowerLink or Profinet, but I do know EtherCAT and IEC
> 61850, and those two can surely be implemented on the interface that I
> am talking about.

It works, but it won't give you a deterministic control loop as you
still have Linux in the game.

> 
> Also, the interface that I propose does not preclude the use of RTDM
> ISR in the driver, so I wonder what exactly bothers you about it?

That you claim it's simpler based on lacking features - like RT IRQ
support and the associated proper driver integration of that. Maybe it
turns out to be simpler, but that is by far proven yet.

I was simply hoping to collect some new ideas how to address the driver
maintenance issue in a better way but without dropping key features
needed for RT networking. Something like "let's add generic RT channels
to Linux upstream drivers and then only patch them fit RTDM". Not sure
if that works, but it would come with a vision how to keep things more
maintainable.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Jan Kiszka]

On 2011-09-27 18:26, Jan Kiszka wrote:
> I was simply hoping to collect some new ideas how to address the driver
> maintenance issue in a better way but without dropping key features
> needed for RT networking. Something like "let's add generic RT channels
> to Linux upstream drivers and then only patch them fit RTDM". Not sure
> if that works, but it would come with a vision how to keep things more
> maintainable.

+this could be useful for other scenarios - on PREEMPT-RT.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Richard Cochran]

On Tue, Sep 27, 2011 at 06:26:44PM +0200, Jan Kiszka wrote:
> On 2011-09-27 18:05, Richard Cochran wrote:
> 
> That's a common misunderstanding: RTnet is a networking stack with many
> _optional_ components (like RTmac, RTcfg etc.). I would bet that it's
> more frequently used today in minimal setups, i.e. just the core, some
> driver, and either PF_PACKET or UDP/IP.

I understood about the modular design, but I really want to know if
rtnet will help me if I want to use of the industrial Ethernet
protocols. AFAICT, rtnet really doesn't offer these.

So I'll ask the direct question once again. Does rtnet help me with
industrial Ethernet (apart from the rtnet protocols), or not?

> > Unless rtnet implements (or helps to implement) these, it is kind of
> > silly to say, "your way won't work, you should use rtnet instead."
> > 
> > I don't know PowerLink or Profinet, but I do know EtherCAT and IEC
> > 61850, and those two can surely be implemented on the interface that I
> > am talking about.
> 
> It works, but it won't give you a deterministic control loop as you
> still have Linux in the game.

It really depends on how the driver is written. While my gianfar
example does make use of normal Linux driver interrupts, it would not
necessarily have to do so.

> I was simply hoping to collect some new ideas how to address the driver
> maintenance issue in a better way but without dropping key features
> needed for RT networking. Something like "let's add generic RT channels
> to Linux upstream drivers and then only patch them fit RTDM". Not sure
> if that works, but it would come with a vision how to keep things more
> maintainable.

Well, can you turn the issue around and convince me that writing a
rtnet driver is the best way to acheive raw Ethernet packet access?

You talk about the rtnet driver model, but is it described anywhere?

(BTW rtnet/Documentation/README.drvporting is horrible. It is just a
random list of 40+ odd points without any sense. That document gave me
the impression that developing an rtnet driver is a kind of extended
"hack until it starts working.")

Thanks,
Richard

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Richard Cochran]

On Tue, Sep 27, 2011 at 06:30:00PM +0200, Jan Kiszka wrote:
> On 2011-09-27 18:26, Jan Kiszka wrote:
> > I was simply hoping to collect some new ideas how to address the driver
> > maintenance issue in a better way but without dropping key features
> > needed for RT networking. Something like "let's add generic RT channels
> > to Linux upstream drivers and then only patch them fit RTDM". Not sure
> > if that works, but it would come with a vision how to keep things more
> > maintainable.
> 
> +this could be useful for other scenarios - on PREEMPT-RT.

(But PREEMPT-RT will make the whole kernel deterministic, right? ;)

Adding low-latency channels (eg. working against coalescing) will be a
very hard sell upstream.

Richard

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Jan Kiszka]

On 2011-09-27 19:00, Richard Cochran wrote:
> On Tue, Sep 27, 2011 at 06:26:44PM +0200, Jan Kiszka wrote:
>> On 2011-09-27 18:05, Richard Cochran wrote:
>>
>> That's a common misunderstanding: RTnet is a networking stack with many
>> _optional_ components (like RTmac, RTcfg etc.). I would bet that it's
>> more frequently used today in minimal setups, i.e. just the core, some
>> driver, and either PF_PACKET or UDP/IP.
> 
> I understood about the modular design, but I really want to know if
> rtnet will help me if I want to use of the industrial Ethernet
> protocols. AFAICT, rtnet really doesn't offer these.
> 
> So I'll ask the direct question once again. Does rtnet help me with
> industrial Ethernet (apart from the rtnet protocols), or not?

It manages buffers for you, provides NIC drivers and interfaces to
either build the higher protocol layers in the kernel or in user space.
That's the mission, but I would not exclude that there is room for
improvements (lacking safe copy-to/from user, unneeded RX thread for
single-user scenarios and possibly more). Still, the Ethercat master
library folks chose it back then as a platform, maybe you want to ask them.

> 
>>> Unless rtnet implements (or helps to implement) these, it is kind of
>>> silly to say, "your way won't work, you should use rtnet instead."
>>>
>>> I don't know PowerLink or Profinet, but I do know EtherCAT and IEC
>>> 61850, and those two can surely be implemented on the interface that I
>>> am talking about.
>>
>> It works, but it won't give you a deterministic control loop as you
>> still have Linux in the game.
> 
> It really depends on how the driver is written. While my gianfar
> example does make use of normal Linux driver interrupts, it would not
> necessarily have to do so.
> 
>> I was simply hoping to collect some new ideas how to address the driver
>> maintenance issue in a better way but without dropping key features
>> needed for RT networking. Something like "let's add generic RT channels
>> to Linux upstream drivers and then only patch them fit RTDM". Not sure
>> if that works, but it would come with a vision how to keep things more
>> maintainable.
> 
> Well, can you turn the issue around and convince me that writing a
> rtnet driver is the best way to acheive raw Ethernet packet access?

It would at least avoid having to reinvent user interfaces and buffer
management layers. They may appear simple now, but that's how everything
once started.

> 
> You talk about the rtnet driver model, but is it described anywhere?
> 
> (BTW rtnet/Documentation/README.drvporting is horrible. It is just a
> random list of 40+ odd points without any sense. That document gave me
> the impression that developing an rtnet driver is a kind of extended
> "hack until it starts working.")

I know best. It once worked well, allowed you to more or less
mechanically convert a driver within a few hours, but that's quite a few
years ago. Driver complexity almost exploded since then.

But my goal is not necessarily convincing you to use current RTnet as
is, but to make you think ahead, reusing the experience of this project
if you start something new. That need not replace RTnet immediately, but
it should not block a transition architecturally. The first step to this
remains studying the legacy, even if not using it in the end.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Jan Kiszka]

On 2011-09-27 19:04, Richard Cochran wrote:
> On Tue, Sep 27, 2011 at 06:30:00PM +0200, Jan Kiszka wrote:
>> On 2011-09-27 18:26, Jan Kiszka wrote:
>>> I was simply hoping to collect some new ideas how to address the driver
>>> maintenance issue in a better way but without dropping key features
>>> needed for RT networking. Something like "let's add generic RT channels
>>> to Linux upstream drivers and then only patch them fit RTDM". Not sure
>>> if that works, but it would come with a vision how to keep things more
>>> maintainable.
>>
>> +this could be useful for other scenarios - on PREEMPT-RT.
> 
> (But PREEMPT-RT will make the whole kernel deterministic, right? ;)

Yes. Except where not.

> 
> Adding low-latency channels (eg. working against coalescing) will be a
> very hard sell upstream.

That also depends on the invasiveness. Key requirements like separate
packet pools slowly sneak in (for swap over net). Also, separate IRQs
for separate channels on modern NICs may make the split-out smoother -
you may no longer need to disable features that used to affect the same
handler or shared some locks.

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Gilles Chanteperdrix]

On 09/27/2011 07:00 PM, Richard Cochran wrote:
> On Tue, Sep 27, 2011 at 06:26:44PM +0200, Jan Kiszka wrote:
>> On 2011-09-27 18:05, Richard Cochran wrote:
>>
>> That's a common misunderstanding: RTnet is a networking stack with many
>> _optional_ components (like RTmac, RTcfg etc.). I would bet that it's
>> more frequently used today in minimal setups, i.e. just the core, some
>> driver, and either PF_PACKET or UDP/IP.
> 
> I understood about the modular design, but I really want to know if
> rtnet will help me if I want to use of the industrial Ethernet
> protocols. AFAICT, rtnet really doesn't offer these.
> 
> So I'll ask the direct question once again. Does rtnet help me with
> industrial Ethernet (apart from the rtnet protocols), or not?
> 
>>> Unless rtnet implements (or helps to implement) these, it is kind of
>>> silly to say, "your way won't work, you should use rtnet instead."
>>>
>>> I don't know PowerLink or Profinet, but I do know EtherCAT and IEC
>>> 61850, and those two can surely be implemented on the interface that I
>>> am talking about.
>>
>> It works, but it won't give you a deterministic control loop as you
>> still have Linux in the game.
> 
> It really depends on how the driver is written. While my gianfar
> example does make use of normal Linux driver interrupts, it would not
> necessarily have to do so.
> 
>> I was simply hoping to collect some new ideas how to address the driver
>> maintenance issue in a better way but without dropping key features
>> needed for RT networking. Something like "let's add generic RT channels
>> to Linux upstream drivers and then only patch them fit RTDM". Not sure
>> if that works, but it would come with a vision how to keep things more
>> maintainable.
> 
> Well, can you turn the issue around and convince me that writing a
> rtnet driver is the best way to acheive raw Ethernet packet access?

>From the point of view of someone a bit external to the rtnet project,
rtnet is a TCP/IP stack, which contains the stack, useless for your
purposes, but defines an interface between drivers and the stack. By
following this interface to write a driver on one-side, and support for
raw packets on the other side you get:
- raw packets support for all drivers in rtnet repository
- TCP/IP support for the NIC you wrote a driver for.

As for the in-kernel driver patch versus out-of-tree driver, in-kernel
driver will have to be adapted for each release of the linux kernel,
when porting the I-pipe patch, and that is quickly going to become a
nightmare.

> You talk about the rtnet driver model, but is it described anywhere?
> 
> (BTW rtnet/Documentation/README.drvporting is horrible. It is just a
> random list of 40+ odd points without any sense. That document gave me
> the impression that developing an rtnet driver is a kind of extended
> "hack until it starts working.")

README.drvporting is frightening at first, but in fact, the job to port
a network driver to rtnet is not so hard.

> 
> Thanks,
> Richard
> 
> _______________________________________________
> Xenomai-core mailing list
> Xenomai-core@domain.hid
> https://mail.gna.org/listinfo/xenomai-core
> 


-- 
                                                                Gilles.

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Richard Cochran]

On Tue, Sep 27, 2011 at 07:25:07PM +0200, Jan Kiszka wrote:
> 
> It manages buffers for you, provides NIC drivers and interfaces to
> either build the higher protocol layers in the kernel or in user space.
> That's the mission, but I would not exclude that there is room for
> improvements (lacking safe copy-to/from user, unneeded RX thread for
> single-user scenarios and possibly more). Still, the Ethercat master
> library folks chose it back then as a platform, maybe you want to ask them.

Getting a little off topic, I wonder, who are these "Ethercat folks" of
whom you speak?

I do know of a few open source implementations, but none are based
Xenomai:

* IgH etherlab
  This is a complete stack in the kernel. Although they claim it works
  with Xenomai, in fact it does not, since it uses regular kernel spin
  locks, etc. However, it could be adapted to work with Xenomai.
* SOEM
  This is a user space (really simple) stack based on normal raw
  sockets. It could also be adapted to use Xenomai, by adding some
  sort of raw RT socket.
* OSADL
  This was withdrawn because of license concerns. I never saw the
  code, but I do beleive it was a user space solution using standard
  sockets.
* Another?
  There once was some C++ program from some institute in the
  Netherlands (or Belgium? can't remember), but it was also withdrawn.

So, did you mean any of these, or yet another implementation?

Thanks,
Richard

Re: [Xenomai-core] [RFC 0/1] Class driver for raw Ethernet packets

[Jan Kiszka]

On 2011-09-28 10:16, Richard Cochran wrote:
> On Tue, Sep 27, 2011 at 07:25:07PM +0200, Jan Kiszka wrote:
>>
>> It manages buffers for you, provides NIC drivers and interfaces to
>> either build the higher protocol layers in the kernel or in user space.
>> That's the mission, but I would not exclude that there is room for
>> improvements (lacking safe copy-to/from user, unneeded RX thread for
>> single-user scenarios and possibly more). Still, the Ethercat master
>> library folks chose it back then as a platform, maybe you want to ask them.
> 
> Getting a little off topic, I wonder, who are these "Ethercat folks" of
> whom you speak?
> 
> I do know of a few open source implementations, but none are based
> Xenomai:
> 
> * IgH etherlab
>   This is a complete stack in the kernel. Although they claim it works
>   with Xenomai, in fact it does not, since it uses regular kernel spin
>   locks, etc. However, it could be adapted to work with Xenomai.
> * SOEM
>   This is a user space (really simple) stack based on normal raw
>   sockets. It could also be adapted to use Xenomai, by adding some
>   sort of raw RT socket.
> * OSADL
>   This was withdrawn because of license concerns. I never saw the
>   code, but I do beleive it was a user space solution using standard
>   sockets.
> * Another?
>   There once was some C++ program from some institute in the
>   Netherlands (or Belgium? can't remember), but it was also withdrawn.
> 
> So, did you mean any of these, or yet another implementation?

I meat the first hit for "ethercat master library" in a search engine. See

http://ethercatmaster.berlios.de/
http://www.fmtc.be/downloads/15_FMTC%20open%20sources.pdf

or ask Peter or Klaas (peter@domain.hid, klaas.gadeyne@domain.hid -
not sure if Klaas is still with fmtc).

Jan

-- 
Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux