[PATCH 1/2] app/testpmd: make locking memory configurable

[PATCH 1/2] app/testpmd: make locking memory configurable

[Anatoly Burakov]

Add two new command-line parameters for either enabling or
disabling locking all memory at app startup.

Cc: wenzhuo.lu@intel.com
Cc: jingjing.wu@intel.com

Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 app/test-pmd/parameters.c             |  8 ++++++++
 app/test-pmd/testpmd.c                | 32 ++++++++++++++++++--------------
 app/test-pmd/testpmd.h                |  4 ++--
 doc/guides/testpmd_app_ug/run_app.rst |  8 ++++++++
 4 files changed, 36 insertions(+), 16 deletions(-)

diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c
index aea8af8..6489bf4 100644
--- a/app/test-pmd/parameters.c
+++ b/app/test-pmd/parameters.c
@@ -188,6 +188,8 @@ usage(char* progname)
 	printf("  --tx-offloads=0xXXXXXXXX: hexadecimal bitmask of TX queue offloads\n");
 	printf("  --hot-plug: enable hot plug for device.\n");
 	printf("  --vxlan-gpe-port=N: UPD port of tunnel VXLAN-GPE\n");
+	printf("  --mlockall: lock all memory\n");
+	printf("  --no-mlockall: do not lock all memory\n");
 }
 
 #ifdef RTE_LIBRTE_CMDLINE
@@ -629,6 +631,8 @@ launch_args_parse(int argc, char** argv)
 		{ "tx-offloads",		1, 0, 0 },
 		{ "hot-plug",			0, 0, 0 },
 		{ "vxlan-gpe-port",		1, 0, 0 },
+		{ "mlockall",			0, 0, 0 },
+		{ "no-mlockall",		0, 0, 0 },
 		{ 0, 0, 0, 0 },
 	};
 
@@ -1145,6 +1149,10 @@ launch_args_parse(int argc, char** argv)
 				}
 			if (!strcmp(lgopts[opt_idx].name, "hot-plug"))
 				hot_plug = 1;
+			if (!strcmp(lgopts[opt_idx].name, "mlockall"))
+				do_mlockall = 1;
+			if (!strcmp(lgopts[opt_idx].name, "no-mlockall"))
+				do_mlockall = 0;
 			break;
 		case 'h':
 			usage(argv[0]);
diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index db23f23..77490be 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -299,6 +299,10 @@ uint32_t event_print_mask = (UINT32_C(1) << RTE_ETH_EVENT_UNKNOWN) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_IPSEC) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_MACSEC) |
 			    (UINT32_C(1) << RTE_ETH_EVENT_INTR_RMV);
+/*
+ * Decide if all memory are locked for performance.
+ */
+int do_mlockall = 0;
 
 /*
  * NIC bypass mode configuration options.
@@ -2603,7 +2607,20 @@ main(int argc, char** argv)
 		rte_panic("Cannot register log type");
 	rte_log_set_level(testpmd_logtype, RTE_LOG_DEBUG);
 
-	if (mlockall(MCL_CURRENT | MCL_FUTURE)) {
+	/* Bitrate/latency stats disabled by default */
+#ifdef RTE_LIBRTE_BITRATE
+	bitrate_enabled = 0;
+#endif
+#ifdef RTE_LIBRTE_LATENCY_STATS
+	latencystats_enabled = 0;
+#endif
+
+	argc -= diag;
+	argv += diag;
+	if (argc > 1)
+		launch_args_parse(argc, argv);
+
+	if (do_mlockall && mlockall(MCL_CURRENT | MCL_FUTURE)) {
 		TESTPMD_LOG(NOTICE, "mlockall() failed with error \"%s\"\n",
 			strerror(errno));
 	}
@@ -2625,19 +2642,6 @@ main(int argc, char** argv)
 		rte_panic("Empty set of forwarding logical cores - check the "
 			  "core mask supplied in the command parameters\n");
 
-	/* Bitrate/latency stats disabled by default */
-#ifdef RTE_LIBRTE_BITRATE
-	bitrate_enabled = 0;
-#endif
-#ifdef RTE_LIBRTE_LATENCY_STATS
-	latencystats_enabled = 0;
-#endif
-
-	argc -= diag;
-	argv += diag;
-	if (argc > 1)
-		launch_args_parse(argc, argv);
-
 	if (tx_first && interactive)
 		rte_exit(EXIT_FAILURE, "--tx-first cannot be used on "
 				"interactive mode.\n");
diff --git a/app/test-pmd/testpmd.h b/app/test-pmd/testpmd.h
index 1af87b8..f7bc58e 100644
--- a/app/test-pmd/testpmd.h
+++ b/app/test-pmd/testpmd.h
@@ -334,9 +334,9 @@ extern volatile int test_done; /* stop packet forwarding when set to 1. */
 extern uint8_t lsc_interrupt; /**< disabled by "--no-lsc-interrupt" parameter */
 extern uint8_t rmv_interrupt; /**< disabled by "--no-rmv-interrupt" parameter */
 extern uint32_t event_print_mask;
-extern uint8_t hot_plug; /**< enable by "--hot-plug" parameter */
-
 /**< set by "--print-event xxxx" and "--mask-event xxxx parameters */
+extern uint8_t hot_plug; /**< enable by "--hot-plug" parameter */
+extern int do_mlockall; /**< set by "--mlockall" or "--no-mlockall" parameter */
 
 #ifdef RTE_LIBRTE_IXGBE_BYPASS
 extern uint32_t bypass_timeout; /**< Store the NIC bypass watchdog timeout */
diff --git a/doc/guides/testpmd_app_ug/run_app.rst b/doc/guides/testpmd_app_ug/run_app.rst
index 351d826..f301c2b 100644
--- a/doc/guides/testpmd_app_ug/run_app.rst
+++ b/doc/guides/testpmd_app_ug/run_app.rst
@@ -490,3 +490,11 @@ The commandline options are:
 
     Set the UDP port number of tunnel VXLAN-GPE to N.
     The default value is 4790.
+
+*   ``--mlockall``
+
+    Enable locking all memory.
+
+*   ``--no-mlockall``
+
+    Disable locking all memory.
-- 
2.7.4

[PATCH 2/2] app/tesetpmd: disable memory locking for bsdapp

[Anatoly Burakov]

If mlockall() is called while allocated VA space is bigger than
amount of available RAM, FreeBSD kernel will deadlock and need
a hard reboot. We do allocate big amounts of memory because of
how new memory subsystem works, so calling mlockall() will cause
a deadlock. So, disable mlockall() by default on FreeBSD.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 app/test-pmd/testpmd.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index 77490be..38b8b7a 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -2615,6 +2615,13 @@ main(int argc, char** argv)
 	latencystats_enabled = 0;
 #endif
 
+	/* on FreeBSD, mlockall() is disabled by default */
+#ifdef RTE_EXEC_ENV_BSDAPP
+	do_mlockall = 0;
+#else
+	do_mlockall = 1;
+#endif
+
 	argc -= diag;
 	argv += diag;
 	if (argc > 1)
-- 
2.7.4

Re: [PATCH 1/2] app/testpmd: make locking memory configurable

[Remy Horton]

Looks good to me, and passed quick smoke test.

On 03/05/2018 13:38, Anatoly Burakov wrote:
[..]
> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>

Acked-by: Remy Horton <remy.horton@intel.com>

Re: [PATCH 2/2] app/tesetpmd: disable memory locking for bsdapp

[Ananyev, Konstantin]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Anatoly Burakov
> Sent: Thursday, May 3, 2018 1:38 PM
> To: dev@dpdk.org
> Cc: Lu, Wenzhuo <wenzhuo.lu@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Tan, Jianfeng <jianfeng.tan@intel.com>
> Subject: [dpdk-dev] [PATCH 2/2] app/tesetpmd: disable memory locking for bsdapp
> 
> If mlockall() is called while allocated VA space is bigger than
> amount of available RAM, FreeBSD kernel will deadlock and need
> a hard reboot. We do allocate big amounts of memory because of
> how new memory subsystem works, so calling mlockall() will cause
> a deadlock. So, disable mlockall() by default on FreeBSD.
> 
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
>  app/test-pmd/testpmd.c | 7 +++++++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
> index 77490be..38b8b7a 100644
> --- a/app/test-pmd/testpmd.c
> +++ b/app/test-pmd/testpmd.c
> @@ -2615,6 +2615,13 @@ main(int argc, char** argv)
>  	latencystats_enabled = 0;
>  #endif
> 
> +	/* on FreeBSD, mlockall() is disabled by default */
> +#ifdef RTE_EXEC_ENV_BSDAPP
> +	do_mlockall = 0;
> +#else
> +	do_mlockall = 1;
> +#endif
> +
>  	argc -= diag;
>  	argv += diag;
>  	if (argc > 1)
> --

Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

> 2.7.4

Re: [PATCH 1/2] app/testpmd: make locking memory configurable

[Ananyev, Konstantin]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Anatoly Burakov
> Sent: Thursday, May 3, 2018 1:38 PM
> To: dev@dpdk.org
> Cc: Lu, Wenzhuo <wenzhuo.lu@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Mcnamara, John <john.mcnamara@intel.com>;
> Kovacevic, Marko <marko.kovacevic@intel.com>; Tan, Jianfeng <jianfeng.tan@intel.com>
> Subject: [dpdk-dev] [PATCH 1/2] app/testpmd: make locking memory configurable
> 
> Add two new command-line parameters for either enabling or
> disabling locking all memory at app startup.
> 
> Cc: wenzhuo.lu@intel.com
> Cc: jingjing.wu@intel.com
> 
> Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---

Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

Re: [PATCH 2/2] app/tesetpmd: disable memory locking for bsdapp

[Thomas Monjalon]

> > If mlockall() is called while allocated VA space is bigger than
> > amount of available RAM, FreeBSD kernel will deadlock and need
> > a hard reboot. We do allocate big amounts of memory because of
> > how new memory subsystem works, so calling mlockall() will cause
> > a deadlock. So, disable mlockall() by default on FreeBSD.
> > 
> > Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> 
> Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>

Series applied, thanks