[PATCH 0/2] fixes for FreeBSD compilation

[PATCH 0/2] fixes for FreeBSD compilation

[Bruce Richardson]

Bruce Richardson (2):
  examples/performance-thread: fix FreeBSD compilation
  net/ark: fix FreeBSD compilation

 drivers/net/ark/Makefile                                |  2 ++
 examples/performance-thread/l3fwd-thread/main.c         |  4 ++++
 examples/performance-thread/pthread_shim/main.c         |  4 ++++
 examples/performance-thread/pthread_shim/pthread_shim.c | 15 +++++++++++++--
 examples/performance-thread/pthread_shim/pthread_shim.h |  9 +++++++++
 5 files changed, 32 insertions(+), 2 deletions(-)

-- 
2.11.0

[PATCH 1/2] examples/performance-thread: fix FreeBSD compilation

[Bruce Richardson]

This set of sample apps did not compile on FreeBSD due to use of a number
of Linux/glibc-specific APIs, or APIs which existed in different headers
on FreeBSD. Specifically, the following APIs has problems:
  * sched_getcpu() is a glibc extension
  * pthread_yield() returns int on Linux, but void on FreeBSD
  * APIs for managing cpu affinity are in pthread_np.h on FreeBSD, rather
    than in pthread.h
  * the type for managing cpu sets is cpuset_t on FreeBSD rather than
    cpu_set_t as on Linux.

Fixes: 433ba6228f9a ("examples/performance-thread: add pthread_shim app")
Fixes: d48415e1fee3 ("examples/performance-thread: add l3fwd-thread app")

CC: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 examples/performance-thread/l3fwd-thread/main.c         |  4 ++++
 examples/performance-thread/pthread_shim/main.c         |  4 ++++
 examples/performance-thread/pthread_shim/pthread_shim.c | 15 +++++++++++++--
 examples/performance-thread/pthread_shim/pthread_shim.h |  9 +++++++++
 4 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/examples/performance-thread/l3fwd-thread/main.c b/examples/performance-thread/l3fwd-thread/main.c
index f6154ac2a..2d98473eb 100644
--- a/examples/performance-thread/l3fwd-thread/main.c
+++ b/examples/performance-thread/l3fwd-thread/main.c
@@ -90,6 +90,10 @@
 #define APP_LOOKUP_METHOD             APP_LOOKUP_LPM
 #endif
 
+#ifndef __GLIBC__ /* sched_getcpu() is glibc specific */
+#define sched_getcpu() rte_lcore_id()
+#endif
+
 static int
 check_ptype(int portid)
 {
diff --git a/examples/performance-thread/pthread_shim/main.c b/examples/performance-thread/pthread_shim/main.c
index f03572181..f7074006e 100644
--- a/examples/performance-thread/pthread_shim/main.c
+++ b/examples/performance-thread/pthread_shim/main.c
@@ -59,6 +59,10 @@
 #define DEBUG_APP 0
 #define HELLOW_WORLD_MAX_LTHREADS 10
 
+#ifndef __GLIBC__ /* sched_getcpu() is glibc-specific */
+#define sched_getcpu() rte_lcore_id()
+#endif
+
 __thread int print_count;
 __thread pthread_mutex_t print_lock;
 
diff --git a/examples/performance-thread/pthread_shim/pthread_shim.c b/examples/performance-thread/pthread_shim/pthread_shim.c
index 0d6100c90..1b2e1de03 100644
--- a/examples/performance-thread/pthread_shim/pthread_shim.c
+++ b/examples/performance-thread/pthread_shim/pthread_shim.c
@@ -576,15 +576,26 @@ int pthread_rwlock_wrlock(pthread_rwlock_t *a)
 	return _sys_pthread_funcs.f_pthread_rwlock_wrlock(a);
 }
 
-int pthread_yield(void)
+#ifdef RTE_EXEC_ENV_LINUXAPP
+int
+pthread_yield(void)
 {
 	if (override) {
 		lthread_yield();
 		return 0;
 	}
 	return _sys_pthread_funcs.f_pthread_yield();
-
 }
+#else
+void
+pthread_yield(void)
+{
+	if (override)
+		lthread_yield();
+	else
+		_sys_pthread_funcs.f_pthread_yield();
+}
+#endif
 
 pthread_t pthread_self(void)
 {
diff --git a/examples/performance-thread/pthread_shim/pthread_shim.h b/examples/performance-thread/pthread_shim/pthread_shim.h
index 78bbb5ac0..527640bd6 100644
--- a/examples/performance-thread/pthread_shim/pthread_shim.h
+++ b/examples/performance-thread/pthread_shim/pthread_shim.h
@@ -36,6 +36,15 @@
 #include <pthread.h>
 
 /*
+ * on BSD, the non-std calls are in pthread_np.h,
+ * and cpuset_t has only one "_" rather than two.
+ */
+#ifdef RTE_EXEC_ENV_BSDAPP
+#include <pthread_np.h>
+#define cpu_set_t cpuset_t
+#endif
+
+/*
  * This pthread shim is an example that demonstrates how legacy code
  * that makes use of POSIX pthread services can make use of lthreads
  * with reduced porting effort.
-- 
2.11.0

[PATCH 2/2] net/ark: fix FreeBSD compilation

[Bruce Richardson]

On FreeBSD it's not necessary to use -ldl to link apps which use
dlopen. This error only showed up with a shared library gcc build,
not standard build using static libs.

Fixes: 1131cbf0fb2b ("net/ark: stub PMD for Atomic Rules Arkville")

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 drivers/net/ark/Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/net/ark/Makefile b/drivers/net/ark/Makefile
index b3d462ff5..ca64b1957 100644
--- a/drivers/net/ark/Makefile
+++ b/drivers/net/ark/Makefile
@@ -59,6 +59,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_udm.c
 
 # this lib depends upon:
 LDLIBS += -lpthread
+ifdef CONFIG_RTE_EXEC_ENV_LINUXAPP
 LDLIBS += -ldl
+endif
 
 include $(RTE_SDK)/mk/rte.lib.mk
-- 
2.11.0

Re: [PATCH 2/2] net/ark: fix FreeBSD compilation

[john miller]

> On Apr 20, 2017, at 12:32 PM, Bruce Richardson <bruce.richardson@intel.com> wrote:
> 
> On FreeBSD it's not necessary to use -ldl to link apps which use
> dlopen. This error only showed up with a shared library gcc build,
> not standard build using static libs.
> 
> Fixes: 1131cbf0fb2b ("net/ark: stub PMD for Atomic Rules Arkville")
> 
> Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>


Thanks Bruce.

Acked-by: John Miller <john.miller@atomicrules.com <mailto:john.miller@atomicrules.com>>

Re: [dpdk-stable] [PATCH 1/2] examples/performance-thread: fix FreeBSD compilation

[Thomas Monjalon]

20/04/2017 18:32, Bruce Richardson:
> This set of sample apps did not compile on FreeBSD due to use of a number
> of Linux/glibc-specific APIs, or APIs which existed in different headers
> on FreeBSD. Specifically, the following APIs has problems:
>   * sched_getcpu() is a glibc extension
>   * pthread_yield() returns int on Linux, but void on FreeBSD
>   * APIs for managing cpu affinity are in pthread_np.h on FreeBSD, rather
>     than in pthread.h
>   * the type for managing cpu sets is cpuset_t on FreeBSD rather than
>     cpu_set_t as on Linux.

It can be fixed by using rte_cpuset_t defined in
lib/librte_eal/common/include/rte_lcore.h

[PATCH v2 0/4] Fixes for BSD compilation

[Bruce Richardson]

Fixes a number of issues with compiling on BSD, and a general fix to
clean up the compilation of the performance-thread app.

V2 changes: 
* adjust fix for perf thread compilation to use rte_lcore.h
  instead of extra #ifdefs.
* added in two extra patches, enabling examples_clean to run
  on BSD, and removing the "common" build dir in perf thread example.


Bruce Richardson (4):
  net/ark: fix FreeBSD compilation
  examples: fix examples_clean build target on FreeBSD
  examples/performance-thread: fix FreeBSD compilation
  examples/performance-thread: use a single build output dir

 drivers/net/ark/Makefile                           |  2 ++
 examples/exception_path/Makefile                   |  1 +
 examples/netmap_compat/bridge/Makefile             |  1 +
 examples/performance-thread/common/common.mk       | 20 ++++++++++---------
 examples/performance-thread/l3fwd-thread/main.c    |  4 ++++
 examples/performance-thread/pthread_shim/main.c    |  8 ++++++--
 .../performance-thread/pthread_shim/pthread_shim.c | 23 ++++++++++++++++------
 .../performance-thread/pthread_shim/pthread_shim.h |  3 ++-
 examples/qos_sched/Makefile                        |  1 +
 9 files changed, 45 insertions(+), 18 deletions(-)

-- 
2.11.0

[PATCH v2 1/4] net/ark: fix FreeBSD compilation

[Bruce Richardson]

On FreeBSD it's not necessary to use -ldl to link apps which use
dlopen. This error only showed up with a shared library gcc build,
not standard build using static libs.

Fixes: 1131cbf0fb2b ("net/ark: stub PMD for Atomic Rules Arkville")

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: John Miller <john.miller@atomicrules.com>
---
 drivers/net/ark/Makefile | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/net/ark/Makefile b/drivers/net/ark/Makefile
index b3d462ff5..ca64b1957 100644
--- a/drivers/net/ark/Makefile
+++ b/drivers/net/ark/Makefile
@@ -59,6 +59,8 @@ SRCS-$(CONFIG_RTE_LIBRTE_ARK_PMD) += ark_udm.c
 
 # this lib depends upon:
 LDLIBS += -lpthread
+ifdef CONFIG_RTE_EXEC_ENV_LINUXAPP
 LDLIBS += -ldl
+endif
 
 include $(RTE_SDK)/mk/rte.lib.mk
-- 
2.11.0

[PATCH v2 2/4] examples: fix examples_clean build target on FreeBSD

[Bruce Richardson]

The "examples_clean" top-level build target calls "make clean" for each
individual example. However, for a number of those which were linux-only
examples, the "if" condition only had a dummy "all" target i.e. no "clean"
target, causing an error with examples_clean.

Fixes: 3417cd687e5e ("examples: ignore linux apps on bsd")

CC: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 examples/exception_path/Makefile       | 1 +
 examples/netmap_compat/bridge/Makefile | 1 +
 examples/qos_sched/Makefile            | 1 +
 3 files changed, 3 insertions(+)

diff --git a/examples/exception_path/Makefile b/examples/exception_path/Makefile
index 959914a2a..4b6e07175 100644
--- a/examples/exception_path/Makefile
+++ b/examples/exception_path/Makefile
@@ -42,6 +42,7 @@ ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
 $(info This application can only operate in a linuxapp environment, \
 please change the definition of the RTE_TARGET environment variable)
 all:
+clean:
 else
 
 # binary name
diff --git a/examples/netmap_compat/bridge/Makefile b/examples/netmap_compat/bridge/Makefile
index 50d96e819..1d4ddfffc 100644
--- a/examples/netmap_compat/bridge/Makefile
+++ b/examples/netmap_compat/bridge/Makefile
@@ -42,6 +42,7 @@ ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
 $(info This application can only operate in a linuxapp environment, \
 please change the definition of the RTE_TARGET environment variable)
 all:
+clean:
 else
 
 # binary name
diff --git a/examples/qos_sched/Makefile b/examples/qos_sched/Makefile
index f59645f52..e41ac500f 100644
--- a/examples/qos_sched/Makefile
+++ b/examples/qos_sched/Makefile
@@ -42,6 +42,7 @@ ifneq ($(CONFIG_RTE_EXEC_ENV),"linuxapp")
 $(info This application can only operate in a linuxapp environment, \
 please change the definition of the RTE_TARGET environment variable)
 all:
+clean:
 else
 
 # binary name
-- 
2.11.0

[PATCH v2 3/4] examples/performance-thread: fix FreeBSD compilation

[Bruce Richardson]

This set of sample apps did not compile on FreeBSD due to use of a number
of Linux/glibc-specific APIs, or APIs which existed in different headers
on FreeBSD. Specifically, the following APIs has problems:
  * sched_getcpu() is a glibc extension, so use rte_lcore_id() on BSD
  * pthread_yield() returns int on Linux, but void on FreeBSD, so
    we have to create two slightly different copies of the function.
  * the type for managing cpu sets is cpuset_t on FreeBSD rather than
    cpu_set_t as on Linux, so use rte_cpuset_t from rte_lcore.h.
  * APIs for managing cpu affinity are in pthread_np.h on FreeBSD, rather
    than in pthread.h, also fixed by including rte_lcore.h

Fixes: 433ba6228f9a ("examples/performance-thread: add pthread_shim app")
Fixes: d48415e1fee3 ("examples/performance-thread: add l3fwd-thread app")

CC: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
V2: use rte_lcore.h to fix cpu_set_t type and pthread_np.h inclusion,
rather than ifdefs in the app itself.
---
 examples/performance-thread/l3fwd-thread/main.c    |  4 ++++
 examples/performance-thread/pthread_shim/main.c    |  8 ++++++--
 .../performance-thread/pthread_shim/pthread_shim.c | 23 ++++++++++++++++------
 .../performance-thread/pthread_shim/pthread_shim.h |  3 ++-
 4 files changed, 29 insertions(+), 9 deletions(-)

diff --git a/examples/performance-thread/l3fwd-thread/main.c b/examples/performance-thread/l3fwd-thread/main.c
index f6154ac2a..2d98473eb 100644
--- a/examples/performance-thread/l3fwd-thread/main.c
+++ b/examples/performance-thread/l3fwd-thread/main.c
@@ -90,6 +90,10 @@
 #define APP_LOOKUP_METHOD             APP_LOOKUP_LPM
 #endif
 
+#ifndef __GLIBC__ /* sched_getcpu() is glibc specific */
+#define sched_getcpu() rte_lcore_id()
+#endif
+
 static int
 check_ptype(int portid)
 {
diff --git a/examples/performance-thread/pthread_shim/main.c b/examples/performance-thread/pthread_shim/main.c
index f03572181..850b009d3 100644
--- a/examples/performance-thread/pthread_shim/main.c
+++ b/examples/performance-thread/pthread_shim/main.c
@@ -59,6 +59,10 @@
 #define DEBUG_APP 0
 #define HELLOW_WORLD_MAX_LTHREADS 10
 
+#ifndef __GLIBC__ /* sched_getcpu() is glibc-specific */
+#define sched_getcpu() rte_lcore_id()
+#endif
+
 __thread int print_count;
 __thread pthread_mutex_t print_lock;
 
@@ -175,12 +179,12 @@ static void initial_lthread(void *args __attribute__((unused)))
 		 * use an attribute to pass the desired lcore
 		 */
 		pthread_attr_t attr;
-		cpu_set_t cpuset;
+		rte_cpuset_t cpuset;
 
 		CPU_ZERO(&cpuset);
 		CPU_SET(lcore, &cpuset);
 		pthread_attr_init(&attr);
-		pthread_attr_setaffinity_np(&attr, sizeof(cpu_set_t), &cpuset);
+		pthread_attr_setaffinity_np(&attr, sizeof(rte_cpuset_t), &cpuset);
 
 		/* create the thread */
 		pthread_create(&tid[i], &attr, helloworld_pthread, (void *) i);
diff --git a/examples/performance-thread/pthread_shim/pthread_shim.c b/examples/performance-thread/pthread_shim/pthread_shim.c
index 0d6100c90..71673119a 100644
--- a/examples/performance-thread/pthread_shim/pthread_shim.c
+++ b/examples/performance-thread/pthread_shim/pthread_shim.c
@@ -159,7 +159,7 @@ int (*f_pthread_setschedparam)
 int (*f_pthread_yield)
 	(void);
 int (*f_pthread_setaffinity_np)
-	(pthread_t thread, size_t cpusetsize, const cpu_set_t *cpuset);
+	(pthread_t thread, size_t cpusetsize, const rte_cpuset_t *cpuset);
 int (*f_nanosleep)
 	(const struct timespec *req, struct timespec *rem);
 } _sys_pthread_funcs = {
@@ -390,11 +390,11 @@ pthread_create(pthread_t *__restrict tid,
 
 		if (attr != NULL) {
 			/* determine CPU being requested */
-			cpu_set_t cpuset;
+			rte_cpuset_t cpuset;
 
 			CPU_ZERO(&cpuset);
 			pthread_attr_getaffinity_np(attr,
-						sizeof(cpu_set_t),
+						sizeof(rte_cpuset_t),
 						&cpuset);
 
 			if (CPU_COUNT(&cpuset) != 1)
@@ -576,15 +576,26 @@ int pthread_rwlock_wrlock(pthread_rwlock_t *a)
 	return _sys_pthread_funcs.f_pthread_rwlock_wrlock(a);
 }
 
-int pthread_yield(void)
+#ifdef RTE_EXEC_ENV_LINUXAPP
+int
+pthread_yield(void)
 {
 	if (override) {
 		lthread_yield();
 		return 0;
 	}
 	return _sys_pthread_funcs.f_pthread_yield();
-
 }
+#else
+void
+pthread_yield(void)
+{
+	if (override)
+		lthread_yield();
+	else
+		_sys_pthread_funcs.f_pthread_yield();
+}
+#endif
 
 pthread_t pthread_self(void)
 {
@@ -686,7 +697,7 @@ int nanosleep(const struct timespec *req, struct timespec *rem)
 
 int
 pthread_setaffinity_np(pthread_t thread, size_t cpusetsize,
-		       const cpu_set_t *cpuset)
+		       const rte_cpuset_t *cpuset)
 {
 	if (override) {
 		/* we only allow affinity with a single CPU */
diff --git a/examples/performance-thread/pthread_shim/pthread_shim.h b/examples/performance-thread/pthread_shim/pthread_shim.h
index 78bbb5ac0..10f878940 100644
--- a/examples/performance-thread/pthread_shim/pthread_shim.h
+++ b/examples/performance-thread/pthread_shim/pthread_shim.h
@@ -33,7 +33,8 @@
 
 #ifndef _PTHREAD_SHIM_H_
 #define _PTHREAD_SHIM_H_
-#include <pthread.h>
+
+#include <rte_lcore.h>
 
 /*
  * This pthread shim is an example that demonstrates how legacy code
-- 
2.11.0

[PATCH v2 4/4] examples/performance-thread: use a single build output dir

[Bruce Richardson]

When building any of the perf-thread examples, the output .o files were
placed in two separate directories for each app: the regular build dir and
a "common" build directory. This was due to the way the files to be built
were specified, using a relative path. Switching to use VPATH to find the
files causes Make to put all .o's into the one build directory.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
---
 examples/performance-thread/common/common.mk | 20 +++++++++++---------
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/examples/performance-thread/common/common.mk b/examples/performance-thread/common/common.mk
index d3de5fc60..f6cab7718 100644
--- a/examples/performance-thread/common/common.mk
+++ b/examples/performance-thread/common/common.mk
@@ -30,13 +30,15 @@
 #   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 #   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
-# list the C files belonhing to the lthread subsystem, these are common to all lthread apps
-SRCS-y +=	../common/lthread.c \
-			../common/lthread_sched.c \
-			../common/lthread_cond.c \
-			../common/lthread_tls.c \
-			../common/lthread_mutex.c \
-			../common/lthread_diag.c \
-			../common/arch/x86/ctx.c
+# list the C files belonging to the lthread subsystem, these are common to all
+# lthread apps. Any makefile including this should set VPATH to include this
+# directory path
+#
+
+MKFILE_PATH=$(abspath $(dir $(lastword $(MAKEFILE_LIST))))
+
+VPATH := $(MKFILE_PATH) $(MKFILE_PATH)/arch/x86
+
+SRCS-y += lthread.c lthread_sched.c lthread_cond.c lthread_tls.c lthread_mutex.c lthread_diag.c ctx.c
 
-INCLUDES += -I$(RTE_SDK)/examples/performance-thread/common/ -I$(RTE_SDK)/examples/performance-thread/common/arch/x86/
+INCLUDES += -I$(MKFILE_PATH) -I$(MKFILE_PATH)/arch/x86/
-- 
2.11.0

Re: [PATCH v2 0/4] Fixes for BSD compilation

[Thomas Monjalon]

21/04/2017 15:50, Bruce Richardson:
> Fixes a number of issues with compiling on BSD, and a general fix to
> clean up the compilation of the performance-thread app.
> 
> V2 changes:
> * adjust fix for perf thread compilation to use rte_lcore.h
>   instead of extra #ifdefs.
> * added in two extra patches, enabling examples_clean to run
>   on BSD, and removing the "common" build dir in perf thread example.
> 
> 
> Bruce Richardson (4):
>   net/ark: fix FreeBSD compilation
>   examples: fix examples_clean build target on FreeBSD
>   examples/performance-thread: fix FreeBSD compilation
>   examples/performance-thread: use a single build output dir

Applied, thanks