[PATCH 0/2] xen/net: limit number of tx/rx queues

[PATCH 0/2] xen/net: limit number of tx/rx queues

[Juergen Gross]

The Xen network frontend/backend supports multiple tx/rx queues for one
virtual interface. The number of queues supported by the backend is
set to the number of cpus of the backend driver domain (usually dom0)
and the number of queues requested by the frontend is limited by the
number of vcpus of the related guest.

On large systems this can lead to ridiculous large number of queues
exhausting the required number of grant pages rather quick.

To avoid this limit the default maximum on both sides to 8. Both
frontend and backend maximum can be individually tuned via module
parameters.

Juergen Gross (2):
  xen/netfront: set default upper limit of tx/rx queues to 8
  xen/netback: set default upper limit of tx/rx queues to 8

 drivers/net/xen-netback/netback.c | 6 ++++--
 drivers/net/xen-netfront.c        | 6 ++++--
 2 files changed, 8 insertions(+), 4 deletions(-)

-- 
2.10.2

[PATCH 1/2] xen/netfront: set default upper limit of tx/rx queues to 8

[Juergen Gross]

The default for the number of tx/rx queues of one interface is the
number of vcpus of the system today. As each queue pair reserves 512
grant pages this default consumes a ridiculous number of grants for
large guests.

Limit the queue number to 8 as default. This value can be modified
via a module parameter if required.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 drivers/net/xen-netfront.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
index a479cd9..490c865 100644
--- a/drivers/net/xen-netfront.c
+++ b/drivers/net/xen-netfront.c
@@ -57,6 +57,7 @@
 #include <xen/interface/grant_table.h>
 
 /* Module parameters */
+#define MAX_QUEUES_DEFAULT 8
 static unsigned int xennet_max_queues;
 module_param_named(max_queues, xennet_max_queues, uint, 0644);
 MODULE_PARM_DESC(max_queues,
@@ -2164,11 +2165,12 @@ static int __init netif_init(void)
 
 	pr_info("Initialising Xen virtual ethernet driver\n");
 
-	/* Allow as many queues as there are CPUs if user has not
+	/* Allow as many queues as there are CPUs inut max. 8 if user has not
 	 * specified a value.
 	 */
 	if (xennet_max_queues == 0)
-		xennet_max_queues = num_online_cpus();
+		xennet_max_queues = min_t(unsigned int, MAX_QUEUES_DEFAULT,
+					  num_online_cpus());
 
 	return xenbus_register_frontend(&netfront_driver);
 }
-- 
2.10.2

[PATCH 2/2] xen/netback: set default upper limit of tx/rx queues to 8

[Juergen Gross]

The default for the maximum number of tx/rx queues of one interface is
the number of cpus of the system today. As each queue pair reserves 512
grant pages this default consumes a ridiculous number of grants for
large guests.

Limit the queue number to 8 as default. This value can be modified
via a module parameter if required.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 drivers/net/xen-netback/netback.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/net/xen-netback/netback.c b/drivers/net/xen-netback/netback.c
index 47b4810..f9bcf4a 100644
--- a/drivers/net/xen-netback/netback.c
+++ b/drivers/net/xen-netback/netback.c
@@ -67,6 +67,7 @@ module_param(rx_drain_timeout_msecs, uint, 0444);
 unsigned int rx_stall_timeout_msecs = 60000;
 module_param(rx_stall_timeout_msecs, uint, 0444);
 
+#define MAX_QUEUES_DEFAULT 8
 unsigned int xenvif_max_queues;
 module_param_named(max_queues, xenvif_max_queues, uint, 0644);
 MODULE_PARM_DESC(max_queues,
@@ -1622,11 +1623,12 @@ static int __init netback_init(void)
 	if (!xen_domain())
 		return -ENODEV;
 
-	/* Allow as many queues as there are CPUs if user has not
+	/* Allow as many queues as there are CPUs but max. 8 if user has not
 	 * specified a value.
 	 */
 	if (xenvif_max_queues == 0)
-		xenvif_max_queues = num_online_cpus();
+		xenvif_max_queues = min_t(unsigned int, MAX_QUEUES_DEFAULT,
+					  num_online_cpus());
 
 	if (fatal_skb_slots < XEN_NETBK_LEGACY_SLOTS_MAX) {
 		pr_info("fatal_skb_slots too small (%d), bump it to XEN_NETBK_LEGACY_SLOTS_MAX (%d)\n",
-- 
2.10.2

Re: [PATCH 0/2] xen/net: limit number of tx/rx queues

[Wei Liu]

On Tue, Jan 10, 2017 at 02:32:50PM +0100, Juergen Gross wrote:
> The Xen network frontend/backend supports multiple tx/rx queues for one
> virtual interface. The number of queues supported by the backend is
> set to the number of cpus of the backend driver domain (usually dom0)
> and the number of queues requested by the frontend is limited by the
> number of vcpus of the related guest.
> 
> On large systems this can lead to ridiculous large number of queues
> exhausting the required number of grant pages rather quick.
> 
> To avoid this limit the default maximum on both sides to 8. Both
> frontend and backend maximum can be individually tuned via module
> parameters.
> 
> Juergen Gross (2):
>   xen/netfront: set default upper limit of tx/rx queues to 8
>   xen/netback: set default upper limit of tx/rx queues to 8
> 

Acked-by: Wei Liu <wei.liu2@citrix.com>

Re: [PATCH 1/2] xen/netfront: set default upper limit of tx/rx queues to 8

[Boris Ostrovsky]

On 01/10/2017 08:32 AM, Juergen Gross wrote:
> The default for the number of tx/rx queues of one interface is the
> number of vcpus of the system today. As each queue pair reserves 512
> grant pages this default consumes a ridiculous number of grants for
> large guests.
>
> Limit the queue number to 8 as default. This value can be modified
> via a module parameter if required.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>
> ---
>  drivers/net/xen-netfront.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
> index a479cd9..490c865 100644
> --- a/drivers/net/xen-netfront.c
> +++ b/drivers/net/xen-netfront.c
> @@ -57,6 +57,7 @@
>  #include <xen/interface/grant_table.h>
>  
>  /* Module parameters */
> +#define MAX_QUEUES_DEFAULT 8
>  static unsigned int xennet_max_queues;
>  module_param_named(max_queues, xennet_max_queues, uint, 0644);
>  MODULE_PARM_DESC(max_queues,
> @@ -2164,11 +2165,12 @@ static int __init netif_init(void)
>  
>  	pr_info("Initialising Xen virtual ethernet driver\n");
>  
> -	/* Allow as many queues as there are CPUs if user has not
> +	/* Allow as many queues as there are CPUs inut max. 8 if user has not

Based on comment change in the second patch: s/inut/but/ ? Also, comment
style in both patches.

Other than that, for both:

Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>


>  	 * specified a value.
>  	 */
>  	if (xennet_max_queues == 0)
> -		xennet_max_queues = num_online_cpus();
> +		xennet_max_queues = min_t(unsigned int, MAX_QUEUES_DEFAULT,
> +					  num_online_cpus());
>  
>  	return xenbus_register_frontend(&netfront_driver);
>  }

Re: [PATCH 1/2] xen/netfront: set default upper limit of tx/rx queues to 8

[Juergen Gross]

On 10/01/17 15:53, Boris Ostrovsky wrote:
> On 01/10/2017 08:32 AM, Juergen Gross wrote:
>> The default for the number of tx/rx queues of one interface is the
>> number of vcpus of the system today. As each queue pair reserves 512
>> grant pages this default consumes a ridiculous number of grants for
>> large guests.
>>
>> Limit the queue number to 8 as default. This value can be modified
>> via a module parameter if required.
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>>  drivers/net/xen-netfront.c | 6 ++++--
>>  1 file changed, 4 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
>> index a479cd9..490c865 100644
>> --- a/drivers/net/xen-netfront.c
>> +++ b/drivers/net/xen-netfront.c
>> @@ -57,6 +57,7 @@
>>  #include <xen/interface/grant_table.h>
>>  
>>  /* Module parameters */
>> +#define MAX_QUEUES_DEFAULT 8
>>  static unsigned int xennet_max_queues;
>>  module_param_named(max_queues, xennet_max_queues, uint, 0644);
>>  MODULE_PARM_DESC(max_queues,
>> @@ -2164,11 +2165,12 @@ static int __init netif_init(void)
>>  
>>  	pr_info("Initialising Xen virtual ethernet driver\n");
>>  
>> -	/* Allow as many queues as there are CPUs if user has not
>> +	/* Allow as many queues as there are CPUs inut max. 8 if user has not
> 
> Based on comment change in the second patch: s/inut/but/ ? Also, comment
> style in both patches.

Typo: yes
Style: no. checkpatch tells me that the net directory doesn't follow
common kernel style.

> 
> Other than that, for both:
> 
> Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>

Thanks,

Juergen

Re: [PATCH 0/2] xen/net: limit number of tx/rx queues

[Boris Ostrovsky]

On 01/10/2017 08:32 AM, Juergen Gross wrote:
> The Xen network frontend/backend supports multiple tx/rx queues for one
> virtual interface. The number of queues supported by the backend is
> set to the number of cpus of the backend driver domain (usually dom0)
> and the number of queues requested by the frontend is limited by the
> number of vcpus of the related guest.
>
> On large systems this can lead to ridiculous large number of queues
> exhausting the required number of grant pages rather quick.
>
> To avoid this limit the default maximum on both sides to 8. Both
> frontend and backend maximum can be individually tuned via module
> parameters.
>
> Juergen Gross (2):
>   xen/netfront: set default upper limit of tx/rx queues to 8
>   xen/netback: set default upper limit of tx/rx queues to 8
>
>  drivers/net/xen-netback/netback.c | 6 ++++--
>  drivers/net/xen-netfront.c        | 6 ++++--
>  2 files changed, 8 insertions(+), 4 deletions(-)
>

Applied to for-linus-4.11

-boris