[Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs

[Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs

[Alexey Kardashevskiy]

From: Greg Kurz <gkurz@linux.vnet.ibm.com>

Some systems can already provide more than 255 hardware threads.

Bumping the QEMU limit to 1024 seems reasonable:
- it has no visible overhead in top;
- the limit itself has no effect on hot paths.

Cc: Greg Kurz <gkurz@linux.vnet.ibm.com>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---

With ulimit -u/-n bumped (nproc and nofile), I was able to boot a guest
with 1024 CPUs, both with threads=1 and threads=8.

It takes time though - 3:15 to get to the guest shell but it is probably
expected on 160-threads machine.

---
 hw/ppc/spapr.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index e465d7ac98..46b81a625d 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2712,7 +2712,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     mc->init = ppc_spapr_init;
     mc->reset = ppc_spapr_reset;
     mc->block_default_type = IF_SCSI;
-    mc->max_cpus = 255;
+    mc->max_cpus = 1024;
     mc->no_parallel = 1;
     mc->default_boot_order = "";
     mc->default_ram_size = 512 * M_BYTE;
-- 
2.11.0

Re: [Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1511 bytes --]

On Fri, Feb 24, 2017 at 03:55:31PM +1100, Alexey Kardashevskiy wrote:
> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> 
> Some systems can already provide more than 255 hardware threads.
> 
> Bumping the QEMU limit to 1024 seems reasonable:
> - it has no visible overhead in top;
> - the limit itself has no effect on hot paths.
> 
> Cc: Greg Kurz <gkurz@linux.vnet.ibm.com>
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> ---
> 
> With ulimit -u/-n bumped (nproc and nofile), I was able to boot a guest
> with 1024 CPUs, both with threads=1 and threads=8.
> 
> It takes time though - 3:15 to get to the guest shell but it is probably
> expected on 160-threads machine.

Applied, thanks.

> 
> ---
>  hw/ppc/spapr.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index e465d7ac98..46b81a625d 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -2712,7 +2712,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
>      mc->init = ppc_spapr_init;
>      mc->reset = ppc_spapr_reset;
>      mc->block_default_type = IF_SCSI;
> -    mc->max_cpus = 255;
> +    mc->max_cpus = 1024;
>      mc->no_parallel = 1;
>      mc->default_boot_order = "";
>      mc->default_ram_size = 512 * M_BYTE;

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [PATCH qemu] sysemu: support up to 1024 vCPUs

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 1584 bytes --]

On Fri, 24 Feb 2017 15:55:31 +1100
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> 
> Some systems can already provide more than 255 hardware threads.
> 
> Bumping the QEMU limit to 1024 seems reasonable:
> - it has no visible overhead in top;
> - the limit itself has no effect on hot paths.
> 
> Cc: Greg Kurz <gkurz@linux.vnet.ibm.com>
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> ---
> 
> With ulimit -u/-n bumped (nproc and nofile), I was able to boot a guest
> with 1024 CPUs, both with threads=1 and threads=8.
> 
> It takes time though - 3:15 to get to the guest shell but it is probably
> expected on 160-threads machine.
> 

I remember something similiar at the time... also I had to give more
RAM to the guest to be able to run 1024 CPUs (sth like 6 gigs versus
512 megs for 1 CPU). With the same amount of guest RAM, each extra CPU
would cause the memory used by QEMU to grow about 8 megs.

> ---
>  hw/ppc/spapr.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> index e465d7ac98..46b81a625d 100644
> --- a/hw/ppc/spapr.c
> +++ b/hw/ppc/spapr.c
> @@ -2712,7 +2712,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
>      mc->init = ppc_spapr_init;
>      mc->reset = ppc_spapr_reset;
>      mc->block_default_type = IF_SCSI;
> -    mc->max_cpus = 255;
> +    mc->max_cpus = 1024;
>      mc->no_parallel = 1;
>      mc->default_boot_order = "";
>      mc->default_ram_size = 512 * M_BYTE;



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [PATCH qemu] sysemu: support up to 1024 vCPUs

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2184 bytes --]

On Fri, Feb 24, 2017 at 10:13:50AM +0100, Greg Kurz wrote:
> On Fri, 24 Feb 2017 15:55:31 +1100
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
> > From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> > 
> > Some systems can already provide more than 255 hardware threads.
> > 
> > Bumping the QEMU limit to 1024 seems reasonable:
> > - it has no visible overhead in top;
> > - the limit itself has no effect on hot paths.
> > 
> > Cc: Greg Kurz <gkurz@linux.vnet.ibm.com>
> > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> > ---
> > 
> > With ulimit -u/-n bumped (nproc and nofile), I was able to boot a guest
> > with 1024 CPUs, both with threads=1 and threads=8.
> > 
> > It takes time though - 3:15 to get to the guest shell but it is probably
> > expected on 160-threads machine.

Yes, I'd expect so, that's a lot of overcommit.  Plus, switching from
one vcpu to another on the same host thread will, IIRC, require two
full partition switches, which are pretty slow on Power.

> I remember something similiar at the time... also I had to give more
> RAM to the guest to be able to run 1024 CPUs (sth like 6 gigs versus
> 512 megs for 1 CPU). With the same amount of guest RAM, each extra CPU
> would cause the memory used by QEMU to grow about 8 megs.

Hm... that seems like rather a lot.  Any idea why?

> 
> > ---
> >  hw/ppc/spapr.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > index e465d7ac98..46b81a625d 100644
> > --- a/hw/ppc/spapr.c
> > +++ b/hw/ppc/spapr.c
> > @@ -2712,7 +2712,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
> >      mc->init = ppc_spapr_init;
> >      mc->reset = ppc_spapr_reset;
> >      mc->block_default_type = IF_SCSI;
> > -    mc->max_cpus = 255;
> > +    mc->max_cpus = 1024;
> >      mc->no_parallel = 1;
> >      mc->default_boot_order = "";
> >      mc->default_ram_size = 512 * M_BYTE;
> 
> 



-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [Qemu-devel] [Qemu-ppc] [PATCH qemu] sysemu: support up to 1024 vCPUs

[Greg Kurz]

[-- Attachment #1: Type: text/plain, Size: 2283 bytes --]

On Mon, 27 Feb 2017 12:09:53 +1100
David Gibson <david@gibson.dropbear.id.au> wrote:

> On Fri, Feb 24, 2017 at 10:13:50AM +0100, Greg Kurz wrote:
> > On Fri, 24 Feb 2017 15:55:31 +1100
> > Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> >   
> > > From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> > > 
> > > Some systems can already provide more than 255 hardware threads.
> > > 
> > > Bumping the QEMU limit to 1024 seems reasonable:
> > > - it has no visible overhead in top;
> > > - the limit itself has no effect on hot paths.
> > > 
> > > Cc: Greg Kurz <gkurz@linux.vnet.ibm.com>
> > > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> > > ---
> > > 
> > > With ulimit -u/-n bumped (nproc and nofile), I was able to boot a guest
> > > with 1024 CPUs, both with threads=1 and threads=8.
> > > 
> > > It takes time though - 3:15 to get to the guest shell but it is probably
> > > expected on 160-threads machine.  
> 
> Yes, I'd expect so, that's a lot of overcommit.  Plus, switching from
> one vcpu to another on the same host thread will, IIRC, require two
> full partition switches, which are pretty slow on Power.
> 
> > I remember something similiar at the time... also I had to give more
> > RAM to the guest to be able to run 1024 CPUs (sth like 6 gigs versus
> > 512 megs for 1 CPU). With the same amount of guest RAM, each extra CPU
> > would cause the memory used by QEMU to grow about 8 megs.  
> 
> Hm... that seems like rather a lot.  Any idea why?
> 

No but I'll try again with the current code and I'll have a closer look.

> >   
> > > ---
> > >  hw/ppc/spapr.c | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > 
> > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
> > > index e465d7ac98..46b81a625d 100644
> > > --- a/hw/ppc/spapr.c
> > > +++ b/hw/ppc/spapr.c
> > > @@ -2712,7 +2712,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
> > >      mc->init = ppc_spapr_init;
> > >      mc->reset = ppc_spapr_reset;
> > >      mc->block_default_type = IF_SCSI;
> > > -    mc->max_cpus = 255;
> > > +    mc->max_cpus = 1024;
> > >      mc->no_parallel = 1;
> > >      mc->default_boot_order = "";
> > >      mc->default_ram_size = 512 * M_BYTE;  
> > 
> >   
> 
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs

[Alexey Kardashevskiy]

From: Greg Kurz <gkurz@linux.vnet.ibm.com>

Some systems can already provide more than 255 hardware threads.

Bumping the QEMU limit to 1024 seems reasonable:
- it has no visible overhead in top;
- the limit itself has no effect on hot paths.

Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
---
 include/sysemu/sysemu.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index ef2c50b..2ec0bd8 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -173,7 +173,7 @@ extern int mem_prealloc;
  *
  * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
  */
-#define MAX_CPUMASK_BITS 255
+#define MAX_CPUMASK_BITS 1024
 
 #define MAX_OPTION_ROMS 16
 typedef struct QEMUOptionRom {
-- 
2.5.0.rc3

Re: [Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs

[Alexey Kardashevskiy]

Ping, anyone?


On 04/10/16 11:33, Alexey Kardashevskiy wrote:
> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> 
> Some systems can already provide more than 255 hardware threads.
> 
> Bumping the QEMU limit to 1024 seems reasonable:
> - it has no visible overhead in top;
> - the limit itself has no effect on hot paths.
> 
> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> ---
>  include/sysemu/sysemu.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> index ef2c50b..2ec0bd8 100644
> --- a/include/sysemu/sysemu.h
> +++ b/include/sysemu/sysemu.h
> @@ -173,7 +173,7 @@ extern int mem_prealloc;
>   *
>   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
>   */
> -#define MAX_CPUMASK_BITS 255
> +#define MAX_CPUMASK_BITS 1024
>  
>  #define MAX_OPTION_ROMS 16
>  typedef struct QEMUOptionRom {
> 


-- 
Alexey

Re: [Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs

[Alexey Kardashevskiy]

Ping, anyone?

I rather expected floods of mails on such a controversial topic :)


On 11/10/16 09:19, Alexey Kardashevskiy wrote:
> Ping, anyone?
> 
> 
> On 04/10/16 11:33, Alexey Kardashevskiy wrote:
>> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
>>
>> Some systems can already provide more than 255 hardware threads.
>>
>> Bumping the QEMU limit to 1024 seems reasonable:
>> - it has no visible overhead in top;
>> - the limit itself has no effect on hot paths.
>>
>> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>> ---
>>  include/sysemu/sysemu.h | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
>> index ef2c50b..2ec0bd8 100644
>> --- a/include/sysemu/sysemu.h
>> +++ b/include/sysemu/sysemu.h
>> @@ -173,7 +173,7 @@ extern int mem_prealloc;
>>   *
>>   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
>>   */
>> -#define MAX_CPUMASK_BITS 255
>> +#define MAX_CPUMASK_BITS 1024
>>  
>>  #define MAX_OPTION_ROMS 16
>>  typedef struct QEMUOptionRom {
>>
> 
> 


-- 
Alexey

Re: [Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1948 bytes --]

On Tue, Oct 18, 2016 at 12:07:20PM +1100, Alexey Kardashevskiy wrote:
> Ping, anyone?
> 
> I rather expected floods of mails on such a controversial topic :)

I think it's the reverse of the bike-shed problem.  Hardly anyone
feels qualified to comment on it.

It looks ok to me, but I just don't know if there are other subtle
dependencies on the size of the cpumask buried in the code.

Oh well,  let's say:

Reviewed-by: David Gibson <david@gibson.dropbear.id.au>

and find any problems as they arise.

Not sure who we need to convince to take this into their tree though.

> 
> 
> On 11/10/16 09:19, Alexey Kardashevskiy wrote:
> > Ping, anyone?
> > 
> > 
> > On 04/10/16 11:33, Alexey Kardashevskiy wrote:
> >> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> >>
> >> Some systems can already provide more than 255 hardware threads.
> >>
> >> Bumping the QEMU limit to 1024 seems reasonable:
> >> - it has no visible overhead in top;
> >> - the limit itself has no effect on hot paths.
> >>
> >> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> >> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> >> ---
> >>  include/sysemu/sysemu.h | 2 +-
> >>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> >> index ef2c50b..2ec0bd8 100644
> >> --- a/include/sysemu/sysemu.h
> >> +++ b/include/sysemu/sysemu.h
> >> @@ -173,7 +173,7 @@ extern int mem_prealloc;
> >>   *
> >>   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
> >>   */
> >> -#define MAX_CPUMASK_BITS 255
> >> +#define MAX_CPUMASK_BITS 1024
> >>  
> >>  #define MAX_OPTION_ROMS 16
> >>  typedef struct QEMUOptionRom {
> >>
> > 
> > 
> 
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs

[Igor Mammedov]

On Tue, 11 Oct 2016 09:19:10 +1100
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> Ping, anyone?
I have a similar patch
http://patchwork.ozlabs.org/patch/681709/
which bumps limit to 288 and does a little bit more
so it wouldn't affect current users.

After that's merged, I plan to get rid of this limit and
make that part of numa parsing code dynamic so that it
wouldn't impose such limit/any limits on target code.

> 
> 
> On 04/10/16 11:33, Alexey Kardashevskiy wrote:
> > From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> > 
> > Some systems can already provide more than 255 hardware threads.
> > 
> > Bumping the QEMU limit to 1024 seems reasonable:
> > - it has no visible overhead in top;
> > - the limit itself has no effect on hot paths.
> > 
> > Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> > ---
> >  include/sysemu/sysemu.h | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> > index ef2c50b..2ec0bd8 100644
> > --- a/include/sysemu/sysemu.h
> > +++ b/include/sysemu/sysemu.h
> > @@ -173,7 +173,7 @@ extern int mem_prealloc;
> >   *
> >   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
> >   */
> > -#define MAX_CPUMASK_BITS 255
> > +#define MAX_CPUMASK_BITS 1024
> >  
> >  #define MAX_OPTION_ROMS 16
> >  typedef struct QEMUOptionRom {
> >   
> 
> 

Re: [Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs

[Greg Kurz]

On Tue, 18 Oct 2016 13:00:07 +0200
Igor Mammedov <imammedo@redhat.com> wrote:

> On Tue, 11 Oct 2016 09:19:10 +1100
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
> > Ping, anyone?  
> I have a similar patch
> http://patchwork.ozlabs.org/patch/681709/
> which bumps limit to 288 and does a little bit more
> so it wouldn't affect current users.
> 

Heh the bumping to 1024 was kinda arbitrary but 288 would be an
improvement anyway. FYI, only Alpine class servers would benefit
from that since they have 320 threads, even if they're not officially
supported to run a KVM based hypervisor (biggest supported model is
Tuleta with 192 threads).

BTW, the afore mentioned patch keeps the current 255 limit for
pseries machine types. I guess David's answer means that the
hw/ppc/spapr.c hunk can be safely dropped.

> After that's merged, I plan to get rid of this limit and
> make that part of numa parsing code dynamic so that it
> wouldn't impose such limit/any limits on target code.
> 
> > 
> > 
> > On 04/10/16 11:33, Alexey Kardashevskiy wrote:  
> > > From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> > > 
> > > Some systems can already provide more than 255 hardware threads.
> > > 
> > > Bumping the QEMU limit to 1024 seems reasonable:
> > > - it has no visible overhead in top;
> > > - the limit itself has no effect on hot paths.
> > > 
> > > Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> > > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> > > ---
> > >  include/sysemu/sysemu.h | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > 
> > > diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> > > index ef2c50b..2ec0bd8 100644
> > > --- a/include/sysemu/sysemu.h
> > > +++ b/include/sysemu/sysemu.h
> > > @@ -173,7 +173,7 @@ extern int mem_prealloc;
> > >   *
> > >   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
> > >   */
> > > -#define MAX_CPUMASK_BITS 255
> > > +#define MAX_CPUMASK_BITS 1024
> > >  
> > >  #define MAX_OPTION_ROMS 16
> > >  typedef struct QEMUOptionRom {
> > >     
> > 
> >   
> 

Re: [Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs

[Alexey Kardashevskiy]

On 18/10/16 22:00, Igor Mammedov wrote:
> On Tue, 11 Oct 2016 09:19:10 +1100
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
>> Ping, anyone?
> I have a similar patch
> http://patchwork.ozlabs.org/patch/681709/
> which bumps limit to 288 and does a little bit more
> so it wouldn't affect current users.

Why 288 (not oldlimit<<n)? :)

What does happen to the Greg's patch now?


> 
> After that's merged, I plan to get rid of this limit and
> make that part of numa parsing code dynamic so that it
> wouldn't impose such limit/any limits on target code.
> 
>>
>>
>> On 04/10/16 11:33, Alexey Kardashevskiy wrote:
>>> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
>>>
>>> Some systems can already provide more than 255 hardware threads.
>>>
>>> Bumping the QEMU limit to 1024 seems reasonable:
>>> - it has no visible overhead in top;
>>> - the limit itself has no effect on hot paths.
>>>
>>> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>> ---
>>>  include/sysemu/sysemu.h | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
>>> index ef2c50b..2ec0bd8 100644
>>> --- a/include/sysemu/sysemu.h
>>> +++ b/include/sysemu/sysemu.h
>>> @@ -173,7 +173,7 @@ extern int mem_prealloc;
>>>   *
>>>   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
>>>   */
>>> -#define MAX_CPUMASK_BITS 255
>>> +#define MAX_CPUMASK_BITS 1024
>>>  
>>>  #define MAX_OPTION_ROMS 16
>>>  typedef struct QEMUOptionRom {
>>>   
>>
>>
> 


-- 
Alexey

Re: [Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs

[Igor Mammedov]

On Wed, 19 Oct 2016 10:39:12 +1100
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 18/10/16 22:00, Igor Mammedov wrote:
> > On Tue, 11 Oct 2016 09:19:10 +1100
> > Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> >   
> >> Ping, anyone?  
> > I have a similar patch
> > http://patchwork.ozlabs.org/patch/681709/
> > which bumps limit to 288 and does a little bit more
> > so it wouldn't affect current users.  
> 
> Why 288 (not oldlimit<<n)? :)
That's how many vCPU we plan to support for target-i386.


> What does happen to the Greg's patch now?
it could be amended to apply top of above patch,

where it would change not only MAX_CPUMASK_BITS but also
maxcpus in ppc target since above patch unties maxcpus
from MAX_CPUMASK_BITS for all targets.

So followup increases of MAX_CPUMASK_BITS won't affect
other targets.

> 
> 
> > 
> > After that's merged, I plan to get rid of this limit and
> > make that part of numa parsing code dynamic so that it
> > wouldn't impose such limit/any limits on target code.
> >   
> >>
> >>
> >> On 04/10/16 11:33, Alexey Kardashevskiy wrote:  
> >>> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> >>>
> >>> Some systems can already provide more than 255 hardware threads.
> >>>
> >>> Bumping the QEMU limit to 1024 seems reasonable:
> >>> - it has no visible overhead in top;
> >>> - the limit itself has no effect on hot paths.
> >>>
> >>> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> >>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> >>> ---
> >>>  include/sysemu/sysemu.h | 2 +-
> >>>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>>
> >>> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> >>> index ef2c50b..2ec0bd8 100644
> >>> --- a/include/sysemu/sysemu.h
> >>> +++ b/include/sysemu/sysemu.h
> >>> @@ -173,7 +173,7 @@ extern int mem_prealloc;
> >>>   *
> >>>   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
> >>>   */
> >>> -#define MAX_CPUMASK_BITS 255
> >>> +#define MAX_CPUMASK_BITS 1024
> >>>  
> >>>  #define MAX_OPTION_ROMS 16
> >>>  typedef struct QEMUOptionRom {
> >>>     
> >>
> >>  
> >   
> 
> 

Re: [Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs

[Alexey Kardashevskiy]

On 19/10/16 23:23, Igor Mammedov wrote:
> On Wed, 19 Oct 2016 10:39:12 +1100
> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> 
>> On 18/10/16 22:00, Igor Mammedov wrote:
>>> On Tue, 11 Oct 2016 09:19:10 +1100
>>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
>>>   
>>>> Ping, anyone?  
>>> I have a similar patch
>>> http://patchwork.ozlabs.org/patch/681709/
>>> which bumps limit to 288 and does a little bit more
>>> so it wouldn't affect current users.  
>>
>> Why 288 (not oldlimit<<n)? :)
> That's how many vCPU we plan to support for target-i386.


So that patch from patchworks should actually be 3 patches -
- untie MAX_CPUMASK_BITS from maxcpus;
- increase MAX_CPUMASK_BITS to 1024 (till we get this thing totally dynamic);
- increase maxcpus for x86 to 288.

and on top of this I (or Greg) would repost pseries change to maxcpus=1024

Correct? Has your patch gone anywhere yet?



> 
> 
>> What does happen to the Greg's patch now?
> it could be amended to apply top of above patch,
> 
> where it would change not only MAX_CPUMASK_BITS but also
> maxcpus in ppc target since above patch unties maxcpus
> from MAX_CPUMASK_BITS for all targets.
> 
> So followup increases of MAX_CPUMASK_BITS won't affect
> other targets.
> 
>>
>>
>>>
>>> After that's merged, I plan to get rid of this limit and
>>> make that part of numa parsing code dynamic so that it
>>> wouldn't impose such limit/any limits on target code.
>>>   
>>>>
>>>>
>>>> On 04/10/16 11:33, Alexey Kardashevskiy wrote:  
>>>>> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
>>>>>
>>>>> Some systems can already provide more than 255 hardware threads.
>>>>>
>>>>> Bumping the QEMU limit to 1024 seems reasonable:
>>>>> - it has no visible overhead in top;
>>>>> - the limit itself has no effect on hot paths.
>>>>>
>>>>> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
>>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
>>>>> ---
>>>>>  include/sysemu/sysemu.h | 2 +-
>>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
>>>>> index ef2c50b..2ec0bd8 100644
>>>>> --- a/include/sysemu/sysemu.h
>>>>> +++ b/include/sysemu/sysemu.h
>>>>> @@ -173,7 +173,7 @@ extern int mem_prealloc;
>>>>>   *
>>>>>   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
>>>>>   */
>>>>> -#define MAX_CPUMASK_BITS 255
>>>>> +#define MAX_CPUMASK_BITS 1024
>>>>>  
>>>>>  #define MAX_OPTION_ROMS 16
>>>>>  typedef struct QEMUOptionRom {
>>>>>     
>>>>
>>>>  
>>>   
>>
>>
> 


-- 
Alexey

Re: [Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs

[Igor Mammedov]

On Mon, 24 Oct 2016 12:53:10 +1100
Alexey Kardashevskiy <aik@ozlabs.ru> wrote:

> On 19/10/16 23:23, Igor Mammedov wrote:
> > On Wed, 19 Oct 2016 10:39:12 +1100
> > Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> >   
> >> On 18/10/16 22:00, Igor Mammedov wrote:  
> >>> On Tue, 11 Oct 2016 09:19:10 +1100
> >>> Alexey Kardashevskiy <aik@ozlabs.ru> wrote:
> >>>     
> >>>> Ping, anyone?    
> >>> I have a similar patch
> >>> http://patchwork.ozlabs.org/patch/681709/
> >>> which bumps limit to 288 and does a little bit more
> >>> so it wouldn't affect current users.    
> >>
> >> Why 288 (not oldlimit<<n)? :)  
> > That's how many vCPU we plan to support for target-i386.  
> 
> 
> So that patch from patchworks should actually be 3 patches -
> - untie MAX_CPUMASK_BITS from maxcpus;
> - increase MAX_CPUMASK_BITS to 1024 (till we get this thing totally dynamic);
> - increase maxcpus for x86 to 288.
> 
> and on top of this I (or Greg) would repost pseries change to maxcpus=1024
> 
> Correct? Has your patch gone anywhere yet?
It's been reviewed and I expect it merged soon.

on top of it 
MAX_CPUMASK_BITS could be bumped up along with pseries.maxcpus

 
> >   
> >> What does happen to the Greg's patch now?  
> > it could be amended to apply top of above patch,
> > 
> > where it would change not only MAX_CPUMASK_BITS but also
> > maxcpus in ppc target since above patch unties maxcpus
> > from MAX_CPUMASK_BITS for all targets.
> > 
> > So followup increases of MAX_CPUMASK_BITS won't affect
> > other targets.
> >   
> >>
> >>  
> >>>
> >>> After that's merged, I plan to get rid of this limit and
> >>> make that part of numa parsing code dynamic so that it
> >>> wouldn't impose such limit/any limits on target code.
> >>>     
> >>>>
> >>>>
> >>>> On 04/10/16 11:33, Alexey Kardashevskiy wrote:    
> >>>>> From: Greg Kurz <gkurz@linux.vnet.ibm.com>
> >>>>>
> >>>>> Some systems can already provide more than 255 hardware threads.
> >>>>>
> >>>>> Bumping the QEMU limit to 1024 seems reasonable:
> >>>>> - it has no visible overhead in top;
> >>>>> - the limit itself has no effect on hot paths.
> >>>>>
> >>>>> Signed-off-by: Greg Kurz <gkurz@linux.vnet.ibm.com>
> >>>>> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
> >>>>> ---
> >>>>>  include/sysemu/sysemu.h | 2 +-
> >>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>>>>
> >>>>> diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
> >>>>> index ef2c50b..2ec0bd8 100644
> >>>>> --- a/include/sysemu/sysemu.h
> >>>>> +++ b/include/sysemu/sysemu.h
> >>>>> @@ -173,7 +173,7 @@ extern int mem_prealloc;
> >>>>>   *
> >>>>>   * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
> >>>>>   */
> >>>>> -#define MAX_CPUMASK_BITS 255
> >>>>> +#define MAX_CPUMASK_BITS 1024
> >>>>>  
> >>>>>  #define MAX_OPTION_ROMS 16
> >>>>>  typedef struct QEMUOptionRom {
> >>>>>       
> >>>>
> >>>>    
> >>>     
> >>
> >>  
> >   
> 
>