[PATCH 0/2] limit xen vnic max queues number to online cpu number

[PATCH 0/2] limit xen vnic max queues number to online cpu number

[Joe Jin]

Currently xen vnic allowed to create lots of queues by set module param
max_queues(both netback and netfront), when queues number larger than
cpu number, it does not help for performance but need more cpu time.

This patchset limit netback and netfront max queues number to online
cpus number.

Joe Jin (2):
  xen-netback: limit xen vif max queues number to online cpus
  xen-front: limit vnic max_queues number to online cpus

 drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++------
 drivers/net/xen-netfront.c        |   27 +++++++++++++++++++++------
 2 files changed, 43 insertions(+), 12 deletions(-)

[PATCH 1/2] xen-netback: limit xen vif max queues number to online cpus

[Joe Jin]

Should not allocate xen vif queues number more than online cpus.

Signed-off-by: Joe Jin <joe.jin@oracle.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++------
 1 files changed, 22 insertions(+), 6 deletions(-)

diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
index ec98d43..387fbd1 100644
--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -68,7 +68,9 @@ unsigned int rx_stall_timeout_msecs = 60000;
 module_param(rx_stall_timeout_msecs, uint, 0444);
 
 unsigned int xenvif_max_queues;
-module_param_named(max_queues, xenvif_max_queues, uint, 0644);
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp);
+module_param_call(max_queues, xennet_set_max_queues, param_get_int,
+		  &xenvif_max_queues, 0600);
 MODULE_PARM_DESC(max_queues,
 		 "Maximum number of queues per virtual interface");
 
@@ -107,6 +109,20 @@ static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue
 					     u16      size,
 					     u16      flags);
 
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp)
+{
+	unsigned int cpus = num_online_cpus();
+	unsigned int max_queues = simple_strtoul(val, NULL, 10);
+
+	if (max_queues == 0 || max_queues > cpus) {
+		pr_info("max_queues %d is out of range [0 - %d]!\n", 
+			max_queues, cpus);
+		return -EINVAL;
+	}
+
+	return param_set_int(val, kp);
+}
+
 static inline unsigned long idx_to_pfn(struct xenvif_queue *queue,
 				       u16 idx)
 {
@@ -2110,15 +2126,15 @@ int xenvif_dealloc_kthread(void *data)
 static int __init netback_init(void)
 {
 	int rc = 0;
+	unsigned int cpus = num_online_cpus();
 
 	if (!xen_domain())
 		return -ENODEV;
 
-	/* Allow as many queues as there are CPUs if user has not
-	 * specified a value.
-	 */
-	if (xenvif_max_queues == 0)
-		xenvif_max_queues = num_online_cpus();
+	/* Allow at most as many queues as CPUs. */
+	if (xenvif_max_queues == 0 || xenvif_max_queues > cpus)
+		xenvif_max_queues = cpus;
+	pr_info("vif max_queues: %d\n", xenvif_max_queues);
 
 	if (fatal_skb_slots < XEN_NETBK_LEGACY_SLOTS_MAX) {
 		pr_info("fatal_skb_slots too small (%d), bump it to XEN_NETBK_LEGACY_SLOTS_MAX (%d)\n",
-- 
1.7.1

[PATCH 1/2] xen-netback: limit xen vif max queues number to online cpus

[Joe Jin]

Should not allocate xen vif queues number more than online cpus.

Signed-off-by: Joe Jin <joe.jin@oracle.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++------
 1 files changed, 22 insertions(+), 6 deletions(-)

diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
index ec98d43..387fbd1 100644
--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -68,7 +68,9 @@ unsigned int rx_stall_timeout_msecs = 60000;
 module_param(rx_stall_timeout_msecs, uint, 0444);
 
 unsigned int xenvif_max_queues;
-module_param_named(max_queues, xenvif_max_queues, uint, 0644);
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp);
+module_param_call(max_queues, xennet_set_max_queues, param_get_int,
+		  &xenvif_max_queues, 0600);
 MODULE_PARM_DESC(max_queues,
 		 "Maximum number of queues per virtual interface");
 
@@ -107,6 +109,20 @@ static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue
 					     u16      size,
 					     u16      flags);
 
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp)
+{
+	unsigned int cpus = num_online_cpus();
+	unsigned int max_queues = simple_strtoul(val, NULL, 10);
+
+	if (max_queues == 0 || max_queues > cpus) {
+		pr_info("max_queues %d is out of range [0 - %d]!\n", 
+			max_queues, cpus);
+		return -EINVAL;
+	}
+
+	return param_set_int(val, kp);
+}
+
 static inline unsigned long idx_to_pfn(struct xenvif_queue *queue,
 				       u16 idx)
 {
@@ -2110,15 +2126,15 @@ int xenvif_dealloc_kthread(void *data)
 static int __init netback_init(void)
 {
 	int rc = 0;
+	unsigned int cpus = num_online_cpus();
 
 	if (!xen_domain())
 		return -ENODEV;
 
-	/* Allow as many queues as there are CPUs if user has not
-	 * specified a value.
-	 */
-	if (xenvif_max_queues == 0)
-		xenvif_max_queues = num_online_cpus();
+	/* Allow at most as many queues as CPUs. */
+	if (xenvif_max_queues == 0 || xenvif_max_queues > cpus)
+		xenvif_max_queues = cpus;
+	pr_info("vif max_queues: %d\n", xenvif_max_queues);
 
 	if (fatal_skb_slots < XEN_NETBK_LEGACY_SLOTS_MAX) {
 		pr_info("fatal_skb_slots too small (%d), bump it to XEN_NETBK_LEGACY_SLOTS_MAX (%d)\n",
-- 
1.7.1

[PATCH 2/2] xen-front: limit vnic max_queues number to online cpus

[Joe Jin]

Should not allocate vnic queues number more than online cpus.

Signed-off-by: Joe Jin <joe.jin@oracle.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 drivers/net/xen-netfront.c |   27 +++++++++++++++++++++------
 1 files changed, 21 insertions(+), 6 deletions(-)

diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
index f821a97..1c8a7cf 100644
--- a/drivers/net/xen-netfront.c
+++ b/drivers/net/xen-netfront.c
@@ -58,7 +58,9 @@
 
 /* Module parameters */
 static unsigned int xennet_max_queues;
-module_param_named(max_queues, xennet_max_queues, uint, 0644);
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp);
+module_param_call(max_queues, xennet_set_max_queues, param_get_int,
+		  &xennet_max_queues, 0600);
 MODULE_PARM_DESC(max_queues,
 		 "Maximum number of queues per virtual interface");
 
@@ -164,6 +166,19 @@ struct netfront_rx_info {
 	struct xen_netif_extra_info extras[XEN_NETIF_EXTRA_TYPE_MAX - 1];
 };
 
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp)
+{
+	unsigned int cpus = num_online_cpus();
+	unsigned int max_queues = simple_strtoul(val, NULL, 10);
+
+	if (max_queues == 0 || max_queues > cpus) {
+		pr_err("max_queues %d is out of range [0 - %d]!\n", 
+		       max_queues, cpus);
+		return -EINVAL;
+	}
+	return param_set_int(val, kp);
+}
+
 static void skb_entry_set_link(union skb_entry *list, unsigned short id)
 {
 	list->link = id;
@@ -2126,6 +2141,8 @@ static struct xenbus_driver netfront_driver = {
 
 static int __init netif_init(void)
 {
+	unsigned int cpus = num_online_cpus();
+
 	if (!xen_domain())
 		return -ENODEV;
 
@@ -2134,11 +2151,9 @@ static int __init netif_init(void)
 
 	pr_info("Initialising Xen virtual ethernet driver\n");
 
-	/* Allow as many queues as there are CPUs if user has not
-	 * specified a value.
-	 */
-	if (xennet_max_queues == 0)
-		xennet_max_queues = num_online_cpus();
+	/* Allow at most as many queues as there are CPUs. */
+	if (xennet_max_queues == 0 || xennet_max_queues > cpus)
+		xennet_max_queues = cpus;
 
 	return xenbus_register_frontend(&netfront_driver);
 }
-- 
1.7.1

[PATCH 2/2] xen-front: limit vnic max_queues number to online cpus

[Joe Jin]

Should not allocate vnic queues number more than online cpus.

Signed-off-by: Joe Jin <joe.jin@oracle.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 drivers/net/xen-netfront.c |   27 +++++++++++++++++++++------
 1 files changed, 21 insertions(+), 6 deletions(-)

diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
index f821a97..1c8a7cf 100644
--- a/drivers/net/xen-netfront.c
+++ b/drivers/net/xen-netfront.c
@@ -58,7 +58,9 @@
 
 /* Module parameters */
 static unsigned int xennet_max_queues;
-module_param_named(max_queues, xennet_max_queues, uint, 0644);
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp);
+module_param_call(max_queues, xennet_set_max_queues, param_get_int,
+		  &xennet_max_queues, 0600);
 MODULE_PARM_DESC(max_queues,
 		 "Maximum number of queues per virtual interface");
 
@@ -164,6 +166,19 @@ struct netfront_rx_info {
 	struct xen_netif_extra_info extras[XEN_NETIF_EXTRA_TYPE_MAX - 1];
 };
 
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp)
+{
+	unsigned int cpus = num_online_cpus();
+	unsigned int max_queues = simple_strtoul(val, NULL, 10);
+
+	if (max_queues == 0 || max_queues > cpus) {
+		pr_err("max_queues %d is out of range [0 - %d]!\n", 
+		       max_queues, cpus);
+		return -EINVAL;
+	}
+	return param_set_int(val, kp);
+}
+
 static void skb_entry_set_link(union skb_entry *list, unsigned short id)
 {
 	list->link = id;
@@ -2126,6 +2141,8 @@ static struct xenbus_driver netfront_driver = {
 
 static int __init netif_init(void)
 {
+	unsigned int cpus = num_online_cpus();
+
 	if (!xen_domain())
 		return -ENODEV;
 
@@ -2134,11 +2151,9 @@ static int __init netif_init(void)
 
 	pr_info("Initialising Xen virtual ethernet driver\n");
 
-	/* Allow as many queues as there are CPUs if user has not
-	 * specified a value.
-	 */
-	if (xennet_max_queues == 0)
-		xennet_max_queues = num_online_cpus();
+	/* Allow at most as many queues as there are CPUs. */
+	if (xennet_max_queues == 0 || xennet_max_queues > cpus)
+		xennet_max_queues = cpus;
 
 	return xenbus_register_frontend(&netfront_driver);
 }
-- 
1.7.1

Re: [Xen-devel] [PATCH 1/2] xen-netback: limit xen vif max queues number to online cpus

[Jan Beulich]

>>> On 23.10.15 at 09:58, <joe.jin@oracle.com> wrote:
> --- a/drivers/net/xen-netback/netback.c
> +++ b/drivers/net/xen-netback/netback.c
> @@ -68,7 +68,9 @@ unsigned int rx_stall_timeout_msecs = 60000;
>  module_param(rx_stall_timeout_msecs, uint, 0444);
>  
>  unsigned int xenvif_max_queues;
> -module_param_named(max_queues, xenvif_max_queues, uint, 0644);
> +static int xennet_set_max_queues(const char *val, struct kernel_param *kp);
> +module_param_call(max_queues, xennet_set_max_queues, param_get_int,

param_get_uint

> +		  &xenvif_max_queues, 0600);

Why the change from mode 0644 to 0600?

> @@ -107,6 +109,20 @@ static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue
>  					     u16      size,
>  					     u16      flags);
>  
> +static int xennet_set_max_queues(const char *val, struct kernel_param *kp)
> +{
> +	unsigned int cpus = num_online_cpus();
> +	unsigned int max_queues = simple_strtoul(val, NULL, 10);
> +
> +	if (max_queues == 0 || max_queues > cpus) {
> +		pr_info("max_queues %d is out of range [0 - %d]!\n", 

%u in both cases.

> +			max_queues, cpus);
> +		return -EINVAL;

Considering the message: -ERANGE?

> +	}
> +
> +	return param_set_int(val, kp);

param_set_uint()

> @@ -2110,15 +2126,15 @@ int xenvif_dealloc_kthread(void *data)
>  static int __init netback_init(void)
>  {
>  	int rc = 0;
> +	unsigned int cpus = num_online_cpus();
>  
>  	if (!xen_domain())
>  		return -ENODEV;
>  
> -	/* Allow as many queues as there are CPUs if user has not
> -	 * specified a value.
> -	 */
> -	if (xenvif_max_queues == 0)
> -		xenvif_max_queues = num_online_cpus();
> +	/* Allow at most as many queues as CPUs. */
> +	if (xenvif_max_queues == 0 || xenvif_max_queues > cpus)
> +		xenvif_max_queues = cpus;
> +	pr_info("vif max_queues: %d\n", xenvif_max_queues);

%u again.

Jan

Re: [PATCH 1/2] xen-netback: limit xen vif max queues number to online cpus

[Jan Beulich]

>>> On 23.10.15 at 09:58, <joe.jin@oracle.com> wrote:
> --- a/drivers/net/xen-netback/netback.c
> +++ b/drivers/net/xen-netback/netback.c
> @@ -68,7 +68,9 @@ unsigned int rx_stall_timeout_msecs = 60000;
>  module_param(rx_stall_timeout_msecs, uint, 0444);
>  
>  unsigned int xenvif_max_queues;
> -module_param_named(max_queues, xenvif_max_queues, uint, 0644);
> +static int xennet_set_max_queues(const char *val, struct kernel_param *kp);
> +module_param_call(max_queues, xennet_set_max_queues, param_get_int,

param_get_uint

> +		  &xenvif_max_queues, 0600);

Why the change from mode 0644 to 0600?

> @@ -107,6 +109,20 @@ static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue
>  					     u16      size,
>  					     u16      flags);
>  
> +static int xennet_set_max_queues(const char *val, struct kernel_param *kp)
> +{
> +	unsigned int cpus = num_online_cpus();
> +	unsigned int max_queues = simple_strtoul(val, NULL, 10);
> +
> +	if (max_queues == 0 || max_queues > cpus) {
> +		pr_info("max_queues %d is out of range [0 - %d]!\n", 

%u in both cases.

> +			max_queues, cpus);
> +		return -EINVAL;

Considering the message: -ERANGE?

> +	}
> +
> +	return param_set_int(val, kp);

param_set_uint()

> @@ -2110,15 +2126,15 @@ int xenvif_dealloc_kthread(void *data)
>  static int __init netback_init(void)
>  {
>  	int rc = 0;
> +	unsigned int cpus = num_online_cpus();
>  
>  	if (!xen_domain())
>  		return -ENODEV;
>  
> -	/* Allow as many queues as there are CPUs if user has not
> -	 * specified a value.
> -	 */
> -	if (xenvif_max_queues == 0)
> -		xenvif_max_queues = num_online_cpus();
> +	/* Allow at most as many queues as CPUs. */
> +	if (xenvif_max_queues == 0 || xenvif_max_queues > cpus)
> +		xenvif_max_queues = cpus;
> +	pr_info("vif max_queues: %d\n", xenvif_max_queues);

%u again.

Jan

Re: [Xen-devel] [PATCH 1/2] xen-netback: limit xen vif max queues number to online cpus

[Joe Jin]

Hi Jan,

Thanks for you review, will create new patches with your comments.

Regards,
Joe
On 10/23/2015 04:20 PM, Jan Beulich wrote:
>>>> On 23.10.15 at 09:58, <joe.jin@oracle.com> wrote:
>> --- a/drivers/net/xen-netback/netback.c
>> +++ b/drivers/net/xen-netback/netback.c
>> @@ -68,7 +68,9 @@ unsigned int rx_stall_timeout_msecs = 60000;
>>  module_param(rx_stall_timeout_msecs, uint, 0444);
>>  
>>  unsigned int xenvif_max_queues;
>> -module_param_named(max_queues, xenvif_max_queues, uint, 0644);
>> +static int xennet_set_max_queues(const char *val, struct kernel_param *kp);
>> +module_param_call(max_queues, xennet_set_max_queues, param_get_int,
> 
> param_get_uint
> 
>> +		  &xenvif_max_queues, 0600);
> 
> Why the change from mode 0644 to 0600?
> 
>> @@ -107,6 +109,20 @@ static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue
>>  					     u16      size,
>>  					     u16      flags);
>>  
>> +static int xennet_set_max_queues(const char *val, struct kernel_param *kp)
>> +{
>> +	unsigned int cpus = num_online_cpus();
>> +	unsigned int max_queues = simple_strtoul(val, NULL, 10);
>> +
>> +	if (max_queues == 0 || max_queues > cpus) {
>> +		pr_info("max_queues %d is out of range [0 - %d]!\n", 
> 
> %u in both cases.
> 
>> +			max_queues, cpus);
>> +		return -EINVAL;
> 
> Considering the message: -ERANGE?
> 
>> +	}
>> +
>> +	return param_set_int(val, kp);
> 
> param_set_uint()
> 
>> @@ -2110,15 +2126,15 @@ int xenvif_dealloc_kthread(void *data)
>>  static int __init netback_init(void)
>>  {
>>  	int rc = 0;
>> +	unsigned int cpus = num_online_cpus();
>>  
>>  	if (!xen_domain())
>>  		return -ENODEV;
>>  
>> -	/* Allow as many queues as there are CPUs if user has not
>> -	 * specified a value.
>> -	 */
>> -	if (xenvif_max_queues == 0)
>> -		xenvif_max_queues = num_online_cpus();
>> +	/* Allow at most as many queues as CPUs. */
>> +	if (xenvif_max_queues == 0 || xenvif_max_queues > cpus)
>> +		xenvif_max_queues = cpus;
>> +	pr_info("vif max_queues: %d\n", xenvif_max_queues);
> 
> %u again.
> 
> Jan
> 


-- 
Oracle <http://www.oracle.com>
Joe Jin | Software Development Director | +8610.6106.5624
ORACLE | Linux and Virtualization
No. 24 Zhongguancun Software Park, Haidian District | 100193 Beijing 

Re: [PATCH 1/2] xen-netback: limit xen vif max queues number to online cpus

[Joe Jin]

Hi Jan,

Thanks for you review, will create new patches with your comments.

Regards,
Joe
On 10/23/2015 04:20 PM, Jan Beulich wrote:
>>>> On 23.10.15 at 09:58, <joe.jin@oracle.com> wrote:
>> --- a/drivers/net/xen-netback/netback.c
>> +++ b/drivers/net/xen-netback/netback.c
>> @@ -68,7 +68,9 @@ unsigned int rx_stall_timeout_msecs = 60000;
>>  module_param(rx_stall_timeout_msecs, uint, 0444);
>>  
>>  unsigned int xenvif_max_queues;
>> -module_param_named(max_queues, xenvif_max_queues, uint, 0644);
>> +static int xennet_set_max_queues(const char *val, struct kernel_param *kp);
>> +module_param_call(max_queues, xennet_set_max_queues, param_get_int,
> 
> param_get_uint
> 
>> +		  &xenvif_max_queues, 0600);
> 
> Why the change from mode 0644 to 0600?
> 
>> @@ -107,6 +109,20 @@ static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue
>>  					     u16      size,
>>  					     u16      flags);
>>  
>> +static int xennet_set_max_queues(const char *val, struct kernel_param *kp)
>> +{
>> +	unsigned int cpus = num_online_cpus();
>> +	unsigned int max_queues = simple_strtoul(val, NULL, 10);
>> +
>> +	if (max_queues == 0 || max_queues > cpus) {
>> +		pr_info("max_queues %d is out of range [0 - %d]!\n", 
> 
> %u in both cases.
> 
>> +			max_queues, cpus);
>> +		return -EINVAL;
> 
> Considering the message: -ERANGE?
> 
>> +	}
>> +
>> +	return param_set_int(val, kp);
> 
> param_set_uint()
> 
>> @@ -2110,15 +2126,15 @@ int xenvif_dealloc_kthread(void *data)
>>  static int __init netback_init(void)
>>  {
>>  	int rc = 0;
>> +	unsigned int cpus = num_online_cpus();
>>  
>>  	if (!xen_domain())
>>  		return -ENODEV;
>>  
>> -	/* Allow as many queues as there are CPUs if user has not
>> -	 * specified a value.
>> -	 */
>> -	if (xenvif_max_queues == 0)
>> -		xenvif_max_queues = num_online_cpus();
>> +	/* Allow at most as many queues as CPUs. */
>> +	if (xenvif_max_queues == 0 || xenvif_max_queues > cpus)
>> +		xenvif_max_queues = cpus;
>> +	pr_info("vif max_queues: %d\n", xenvif_max_queues);
> 
> %u again.
> 
> Jan
> 


-- 
Oracle <http://www.oracle.com>
Joe Jin | Software Development Director | +8610.6106.5624
ORACLE | Linux and Virtualization
No. 24 Zhongguancun Software Park, Haidian District | 100193 Beijing 

RE: [PATCH 0/2] limit xen vnic max queues number to online cpu number

[Paul Durrant]

> -----Original Message-----
> From: netdev-owner@vger.kernel.org [mailto:netdev-
> owner@vger.kernel.org] On Behalf Of Joe Jin
> Sent: 23 October 2015 08:54
> To: Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David S.
> Miller
> Cc: netdev@vger.kernel.org; xen-devel@lists.xenproject.org
> Subject: [PATCH 0/2] limit xen vnic max queues number to online cpu
> number
> 
> Currently xen vnic allowed to create lots of queues by set module param
> max_queues(both netback and netfront), when queues number larger than
> cpu number, it does not help for performance but need more cpu time.
> 

But it's an override, so why would you want to limit it? The parameter should not be set in the common case.

  Paul

> This patchset limit netback and netfront max queues number to online
> cpus number.
> 
> Joe Jin (2):
>   xen-netback: limit xen vif max queues number to online cpus
>   xen-front: limit vnic max_queues number to online cpus
> 
>  drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++------
>  drivers/net/xen-netfront.c        |   27 +++++++++++++++++++++------
>  2 files changed, 43 insertions(+), 12 deletions(-)
> 
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 0/2] limit xen vnic max queues number to online cpu number

[Paul Durrant]

> -----Original Message-----
> From: netdev-owner@vger.kernel.org [mailto:netdev-
> owner@vger.kernel.org] On Behalf Of Joe Jin
> Sent: 23 October 2015 08:54
> To: Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David S.
> Miller
> Cc: netdev@vger.kernel.org; xen-devel@lists.xenproject.org
> Subject: [PATCH 0/2] limit xen vnic max queues number to online cpu
> number
> 
> Currently xen vnic allowed to create lots of queues by set module param
> max_queues(both netback and netfront), when queues number larger than
> cpu number, it does not help for performance but need more cpu time.
> 

But it's an override, so why would you want to limit it? The parameter should not be set in the common case.

  Paul

> This patchset limit netback and netfront max queues number to online
> cpus number.
> 
> Joe Jin (2):
>   xen-netback: limit xen vif max queues number to online cpus
>   xen-front: limit vnic max_queues number to online cpus
> 
>  drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++------
>  drivers/net/xen-netfront.c        |   27 +++++++++++++++++++++------
>  2 files changed, 43 insertions(+), 12 deletions(-)
> 
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 0/2] limit xen vnic max queues number to online cpu number

[Joe Jin]

On 10/23/2015 04:47 PM, Paul Durrant wrote:
>> -----Original Message-----
>> From: netdev-owner@vger.kernel.org [mailto:netdev-
>> owner@vger.kernel.org] On Behalf Of Joe Jin
>> Sent: 23 October 2015 08:54
>> To: Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David S.
>> Miller
>> Cc: netdev@vger.kernel.org; xen-devel@lists.xenproject.org
>> Subject: [PATCH 0/2] limit xen vnic max queues number to online cpu
>> number
>>
>> Currently xen vnic allowed to create lots of queues by set module param
>> max_queues(both netback and netfront), when queues number larger than
>> cpu number, it does not help for performance but need more cpu time.
>>
> 
> But it's an override, so why would you want to limit it? The parameter should not be set in the common case.

Always we can not stop people use it because we provided it :)

If queues number is larger than cpu number, with heavy network load,
cpus have to take more time for interrupt, this lead others less
chance to be scheduled.
Imaging dom0 have 64 cpus, and assigned 4 vcpus to the guest, if
set max_queues to 64 on guest, it will consumed more cpu times
and bandwidth on backend, I think this is not we expected?

Thanks,
Joe
> 
>   Paul
> 
>> This patchset limit netback and netfront max queues number to online
>> cpus number.
>>
>> Joe Jin (2):
>>   xen-netback: limit xen vif max queues number to online cpus
>>   xen-front: limit vnic max_queues number to online cpus
>>
>>  drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++------
>>  drivers/net/xen-netfront.c        |   27 +++++++++++++++++++++------
>>  2 files changed, 43 insertions(+), 12 deletions(-)
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 0/2] limit xen vnic max queues number to online cpu number

[Joe Jin]

On 10/23/2015 04:47 PM, Paul Durrant wrote:
>> -----Original Message-----
>> From: netdev-owner@vger.kernel.org [mailto:netdev-
>> owner@vger.kernel.org] On Behalf Of Joe Jin
>> Sent: 23 October 2015 08:54
>> To: Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David S.
>> Miller
>> Cc: netdev@vger.kernel.org; xen-devel@lists.xenproject.org
>> Subject: [PATCH 0/2] limit xen vnic max queues number to online cpu
>> number
>>
>> Currently xen vnic allowed to create lots of queues by set module param
>> max_queues(both netback and netfront), when queues number larger than
>> cpu number, it does not help for performance but need more cpu time.
>>
> 
> But it's an override, so why would you want to limit it? The parameter should not be set in the common case.

Always we can not stop people use it because we provided it :)

If queues number is larger than cpu number, with heavy network load,
cpus have to take more time for interrupt, this lead others less
chance to be scheduled.
Imaging dom0 have 64 cpus, and assigned 4 vcpus to the guest, if
set max_queues to 64 on guest, it will consumed more cpu times
and bandwidth on backend, I think this is not we expected?

Thanks,
Joe
> 
>   Paul
> 
>> This patchset limit netback and netfront max queues number to online
>> cpus number.
>>
>> Joe Jin (2):
>>   xen-netback: limit xen vif max queues number to online cpus
>>   xen-front: limit vnic max_queues number to online cpus
>>
>>  drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++------
>>  drivers/net/xen-netfront.c        |   27 +++++++++++++++++++++------
>>  2 files changed, 43 insertions(+), 12 deletions(-)
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe netdev" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Xen-devel] [PATCH 0/2] limit xen vnic max queues number to online cpu number

[Jan Beulich]

>>> On 23.10.15 at 11:05, <joe.jin@oracle.com> wrote:
> On 10/23/2015 04:47 PM, Paul Durrant wrote:
>>> -----Original Message-----
>>> From: netdev-owner@vger.kernel.org [mailto:netdev-
>>> owner@vger.kernel.org] On Behalf Of Joe Jin
>>> Sent: 23 October 2015 08:54
>>> To: Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David S.
>>> Miller
>>> Cc: netdev@vger.kernel.org; xen-devel@lists.xenproject.org 
>>> Subject: [PATCH 0/2] limit xen vnic max queues number to online cpu
>>> number
>>>
>>> Currently xen vnic allowed to create lots of queues by set module param
>>> max_queues(both netback and netfront), when queues number larger than
>>> cpu number, it does not help for performance but need more cpu time.
>>>
>> 
>> But it's an override, so why would you want to limit it? The parameter 
> should not be set in the common case.
> 
> Always we can not stop people use it because we provided it :)

Well, it's always a question of whether preventing the admin to
shoot himself in the foot makes sense: When it leads to an unusable
system, it probably does. When it leads to a sub-optimal working
system, perhaps we should allow them their freedom?

Jan

Re: [PATCH 0/2] limit xen vnic max queues number to online cpu number

[Jan Beulich]

>>> On 23.10.15 at 11:05, <joe.jin@oracle.com> wrote:
> On 10/23/2015 04:47 PM, Paul Durrant wrote:
>>> -----Original Message-----
>>> From: netdev-owner@vger.kernel.org [mailto:netdev-
>>> owner@vger.kernel.org] On Behalf Of Joe Jin
>>> Sent: 23 October 2015 08:54
>>> To: Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David S.
>>> Miller
>>> Cc: netdev@vger.kernel.org; xen-devel@lists.xenproject.org 
>>> Subject: [PATCH 0/2] limit xen vnic max queues number to online cpu
>>> number
>>>
>>> Currently xen vnic allowed to create lots of queues by set module param
>>> max_queues(both netback and netfront), when queues number larger than
>>> cpu number, it does not help for performance but need more cpu time.
>>>
>> 
>> But it's an override, so why would you want to limit it? The parameter 
> should not be set in the common case.
> 
> Always we can not stop people use it because we provided it :)

Well, it's always a question of whether preventing the admin to
shoot himself in the foot makes sense: When it leads to an unusable
system, it probably does. When it leads to a sub-optimal working
system, perhaps we should allow them their freedom?

Jan

RE: [PATCH 0/2] limit xen vnic max queues number to online cpu number

[Paul Durrant]

> -----Original Message-----
> From: Joe Jin [mailto:joe.jin@oracle.com]
> Sent: 23 October 2015 10:05
> To: Paul Durrant; Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek
> Wilk; David S. Miller
> Cc: netdev@vger.kernel.org; xen-devel@lists.xenproject.org
> Subject: Re: [PATCH 0/2] limit xen vnic max queues number to online cpu
> number
> 
> On 10/23/2015 04:47 PM, Paul Durrant wrote:
> >> -----Original Message-----
> >> From: netdev-owner@vger.kernel.org [mailto:netdev-
> >> owner@vger.kernel.org] On Behalf Of Joe Jin
> >> Sent: 23 October 2015 08:54
> >> To: Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David
> S.
> >> Miller
> >> Cc: netdev@vger.kernel.org; xen-devel@lists.xenproject.org
> >> Subject: [PATCH 0/2] limit xen vnic max queues number to online cpu
> >> number
> >>
> >> Currently xen vnic allowed to create lots of queues by set module param
> >> max_queues(both netback and netfront), when queues number larger
> than
> >> cpu number, it does not help for performance but need more cpu time.
> >>
> >
> > But it's an override, so why would you want to limit it? The parameter
> should not be set in the common case.
> 
> Always we can not stop people use it because we provided it :)
> 

Indeed, and I believe it was provided largely for test purposes... to force the limit to whatever the admin wants.

> If queues number is larger than cpu number, with heavy network load,
> cpus have to take more time for interrupt, this lead others less
> chance to be scheduled.
> Imaging dom0 have 64 cpus, and assigned 4 vcpus to the guest, if
> set max_queues to 64 on guest, it will consumed more cpu times
> and bandwidth on backend, I think this is not we expected?
> 

I think that would be entirely expected.

  Paul

> Thanks,
> Joe
> >
> >   Paul
> >
> >> This patchset limit netback and netfront max queues number to online
> >> cpus number.
> >>
> >> Joe Jin (2):
> >>   xen-netback: limit xen vif max queues number to online cpus
> >>   xen-front: limit vnic max_queues number to online cpus
> >>
> >>  drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++---
> ---
> >>  drivers/net/xen-netfront.c        |   27 +++++++++++++++++++++------
> >>  2 files changed, 43 insertions(+), 12 deletions(-)
> >>
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe netdev" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 0/2] limit xen vnic max queues number to online cpu number

[Paul Durrant]

> -----Original Message-----
> From: Joe Jin [mailto:joe.jin@oracle.com]
> Sent: 23 October 2015 10:05
> To: Paul Durrant; Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek
> Wilk; David S. Miller
> Cc: netdev@vger.kernel.org; xen-devel@lists.xenproject.org
> Subject: Re: [PATCH 0/2] limit xen vnic max queues number to online cpu
> number
> 
> On 10/23/2015 04:47 PM, Paul Durrant wrote:
> >> -----Original Message-----
> >> From: netdev-owner@vger.kernel.org [mailto:netdev-
> >> owner@vger.kernel.org] On Behalf Of Joe Jin
> >> Sent: 23 October 2015 08:54
> >> To: Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David
> S.
> >> Miller
> >> Cc: netdev@vger.kernel.org; xen-devel@lists.xenproject.org
> >> Subject: [PATCH 0/2] limit xen vnic max queues number to online cpu
> >> number
> >>
> >> Currently xen vnic allowed to create lots of queues by set module param
> >> max_queues(both netback and netfront), when queues number larger
> than
> >> cpu number, it does not help for performance but need more cpu time.
> >>
> >
> > But it's an override, so why would you want to limit it? The parameter
> should not be set in the common case.
> 
> Always we can not stop people use it because we provided it :)
> 

Indeed, and I believe it was provided largely for test purposes... to force the limit to whatever the admin wants.

> If queues number is larger than cpu number, with heavy network load,
> cpus have to take more time for interrupt, this lead others less
> chance to be scheduled.
> Imaging dom0 have 64 cpus, and assigned 4 vcpus to the guest, if
> set max_queues to 64 on guest, it will consumed more cpu times
> and bandwidth on backend, I think this is not we expected?
> 

I think that would be entirely expected.

  Paul

> Thanks,
> Joe
> >
> >   Paul
> >
> >> This patchset limit netback and netfront max queues number to online
> >> cpus number.
> >>
> >> Joe Jin (2):
> >>   xen-netback: limit xen vif max queues number to online cpus
> >>   xen-front: limit vnic max_queues number to online cpus
> >>
> >>  drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++---
> ---
> >>  drivers/net/xen-netfront.c        |   27 +++++++++++++++++++++------
> >>  2 files changed, 43 insertions(+), 12 deletions(-)
> >>
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe netdev" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/2] limit xen vnic max queues number to online cpu number

[Joe Jin]

Currently xen vnic allowed to create lots of queues by set module param
max_queues(both netback and netfront), when queues number larger than
cpu number, it does not help for performance but need more cpu time.

This patchset limit netback and netfront max queues number to online
cpus number.

Joe Jin (2):
  xen-netback: limit xen vif max queues number to online cpus
  xen-front: limit vnic max_queues number to online cpus

 drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++------
 drivers/net/xen-netfront.c        |   27 +++++++++++++++++++++------
 2 files changed, 43 insertions(+), 12 deletions(-)