* [Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs @ 2017-02-24 4:55 Alexey Kardashevskiy 2017-02-24 6:16 ` David Gibson 2017-02-24 9:13 ` [Qemu-devel] [Qemu-ppc] " Greg Kurz 0 siblings, 2 replies; 5+ messages in thread From: Alexey Kardashevskiy @ 2017-02-24 4:55 UTC (permalink / raw) To: qemu-devel Cc: Alexey Kardashevskiy, qemu-ppc, David Gibson, Greg Kurz, Greg Kurz From: Greg Kurz <gkurz@linux.vnet.ibm.com> Some systems can already provide more than 255 hardware threads. Bumping the QEMU limit to 1024 seems reasonable: - it has no visible overhead in top; - the limit itself has no effect on hot paths. Cc: Greg Kurz <gkurz@linux.vnet.ibm.com> Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> --- With ulimit -u/-n bumped (nproc and nofile), I was able to boot a guest with 1024 CPUs, both with threads=1 and threads=8. It takes time though - 3:15 to get to the guest shell but it is probably expected on 160-threads machine. --- hw/ppc/spapr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index e465d7ac98..46b81a625d 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -2712,7 +2712,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) mc->init = ppc_spapr_init; mc->reset = ppc_spapr_reset; mc->block_default_type = IF_SCSI; - mc->max_cpus = 255; + mc->max_cpus = 1024; mc->no_parallel = 1; mc->default_boot_order = ""; mc->default_ram_size = 512 * M_BYTE; -- 2.11.0 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs 2017-02-24 4:55 [Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs Alexey Kardashevskiy @ 2017-02-24 6:16 ` David Gibson 2017-02-24 9:13 ` [Qemu-devel] [Qemu-ppc] " Greg Kurz 1 sibling, 0 replies; 5+ messages in thread From: David Gibson @ 2017-02-24 6:16 UTC (permalink / raw) To: Alexey Kardashevskiy; +Cc: qemu-devel, qemu-ppc, Greg Kurz, Greg Kurz [-- Attachment #1: Type: text/plain, Size: 1511 bytes --] On Fri, Feb 24, 2017 at 03:55:31PM +1100, Alexey Kardashevskiy wrote: > From: Greg Kurz <gkurz@linux.vnet.ibm.com> > > Some systems can already provide more than 255 hardware threads. > > Bumping the QEMU limit to 1024 seems reasonable: > - it has no visible overhead in top; > - the limit itself has no effect on hot paths. > > Cc: Greg Kurz <gkurz@linux.vnet.ibm.com> > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> > --- > > With ulimit -u/-n bumped (nproc and nofile), I was able to boot a guest > with 1024 CPUs, both with threads=1 and threads=8. > > It takes time though - 3:15 to get to the guest shell but it is probably > expected on 160-threads machine. Applied, thanks. > > --- > hw/ppc/spapr.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c > index e465d7ac98..46b81a625d 100644 > --- a/hw/ppc/spapr.c > +++ b/hw/ppc/spapr.c > @@ -2712,7 +2712,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) > mc->init = ppc_spapr_init; > mc->reset = ppc_spapr_reset; > mc->block_default_type = IF_SCSI; > - mc->max_cpus = 255; > + mc->max_cpus = 1024; > mc->no_parallel = 1; > mc->default_boot_order = ""; > mc->default_ram_size = 512 * M_BYTE; -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Qemu-devel] [Qemu-ppc] [PATCH qemu] sysemu: support up to 1024 vCPUs 2017-02-24 4:55 [Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs Alexey Kardashevskiy 2017-02-24 6:16 ` David Gibson @ 2017-02-24 9:13 ` Greg Kurz 2017-02-27 1:09 ` David Gibson 1 sibling, 1 reply; 5+ messages in thread From: Greg Kurz @ 2017-02-24 9:13 UTC (permalink / raw) To: Alexey Kardashevskiy; +Cc: qemu-devel, qemu-ppc, Greg Kurz, David Gibson [-- Attachment #1: Type: text/plain, Size: 1584 bytes --] On Fri, 24 Feb 2017 15:55:31 +1100 Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > From: Greg Kurz <gkurz@linux.vnet.ibm.com> > > Some systems can already provide more than 255 hardware threads. > > Bumping the QEMU limit to 1024 seems reasonable: > - it has no visible overhead in top; > - the limit itself has no effect on hot paths. > > Cc: Greg Kurz <gkurz@linux.vnet.ibm.com> > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> > --- > > With ulimit -u/-n bumped (nproc and nofile), I was able to boot a guest > with 1024 CPUs, both with threads=1 and threads=8. > > It takes time though - 3:15 to get to the guest shell but it is probably > expected on 160-threads machine. > I remember something similiar at the time... also I had to give more RAM to the guest to be able to run 1024 CPUs (sth like 6 gigs versus 512 megs for 1 CPU). With the same amount of guest RAM, each extra CPU would cause the memory used by QEMU to grow about 8 megs. > --- > hw/ppc/spapr.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c > index e465d7ac98..46b81a625d 100644 > --- a/hw/ppc/spapr.c > +++ b/hw/ppc/spapr.c > @@ -2712,7 +2712,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) > mc->init = ppc_spapr_init; > mc->reset = ppc_spapr_reset; > mc->block_default_type = IF_SCSI; > - mc->max_cpus = 255; > + mc->max_cpus = 1024; > mc->no_parallel = 1; > mc->default_boot_order = ""; > mc->default_ram_size = 512 * M_BYTE; [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 181 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Qemu-devel] [Qemu-ppc] [PATCH qemu] sysemu: support up to 1024 vCPUs 2017-02-24 9:13 ` [Qemu-devel] [Qemu-ppc] " Greg Kurz @ 2017-02-27 1:09 ` David Gibson 2017-02-27 22:13 ` Greg Kurz 0 siblings, 1 reply; 5+ messages in thread From: David Gibson @ 2017-02-27 1:09 UTC (permalink / raw) To: Greg Kurz; +Cc: Alexey Kardashevskiy, qemu-devel, qemu-ppc, Greg Kurz [-- Attachment #1: Type: text/plain, Size: 2184 bytes --] On Fri, Feb 24, 2017 at 10:13:50AM +0100, Greg Kurz wrote: > On Fri, 24 Feb 2017 15:55:31 +1100 > Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > > > From: Greg Kurz <gkurz@linux.vnet.ibm.com> > > > > Some systems can already provide more than 255 hardware threads. > > > > Bumping the QEMU limit to 1024 seems reasonable: > > - it has no visible overhead in top; > > - the limit itself has no effect on hot paths. > > > > Cc: Greg Kurz <gkurz@linux.vnet.ibm.com> > > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> > > --- > > > > With ulimit -u/-n bumped (nproc and nofile), I was able to boot a guest > > with 1024 CPUs, both with threads=1 and threads=8. > > > > It takes time though - 3:15 to get to the guest shell but it is probably > > expected on 160-threads machine. Yes, I'd expect so, that's a lot of overcommit. Plus, switching from one vcpu to another on the same host thread will, IIRC, require two full partition switches, which are pretty slow on Power. > I remember something similiar at the time... also I had to give more > RAM to the guest to be able to run 1024 CPUs (sth like 6 gigs versus > 512 megs for 1 CPU). With the same amount of guest RAM, each extra CPU > would cause the memory used by QEMU to grow about 8 megs. Hm... that seems like rather a lot. Any idea why? > > > --- > > hw/ppc/spapr.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c > > index e465d7ac98..46b81a625d 100644 > > --- a/hw/ppc/spapr.c > > +++ b/hw/ppc/spapr.c > > @@ -2712,7 +2712,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) > > mc->init = ppc_spapr_init; > > mc->reset = ppc_spapr_reset; > > mc->block_default_type = IF_SCSI; > > - mc->max_cpus = 255; > > + mc->max_cpus = 1024; > > mc->no_parallel = 1; > > mc->default_boot_order = ""; > > mc->default_ram_size = 512 * M_BYTE; > > -- David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Qemu-devel] [Qemu-ppc] [PATCH qemu] sysemu: support up to 1024 vCPUs 2017-02-27 1:09 ` David Gibson @ 2017-02-27 22:13 ` Greg Kurz 0 siblings, 0 replies; 5+ messages in thread From: Greg Kurz @ 2017-02-27 22:13 UTC (permalink / raw) To: David Gibson; +Cc: qemu-ppc, qemu-devel [-- Attachment #1: Type: text/plain, Size: 2283 bytes --] On Mon, 27 Feb 2017 12:09:53 +1100 David Gibson <david@gibson.dropbear.id.au> wrote: > On Fri, Feb 24, 2017 at 10:13:50AM +0100, Greg Kurz wrote: > > On Fri, 24 Feb 2017 15:55:31 +1100 > > Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > > > > > From: Greg Kurz <gkurz@linux.vnet.ibm.com> > > > > > > Some systems can already provide more than 255 hardware threads. > > > > > > Bumping the QEMU limit to 1024 seems reasonable: > > > - it has no visible overhead in top; > > > - the limit itself has no effect on hot paths. > > > > > > Cc: Greg Kurz <gkurz@linux.vnet.ibm.com> > > > Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru> > > > --- > > > > > > With ulimit -u/-n bumped (nproc and nofile), I was able to boot a guest > > > with 1024 CPUs, both with threads=1 and threads=8. > > > > > > It takes time though - 3:15 to get to the guest shell but it is probably > > > expected on 160-threads machine. > > Yes, I'd expect so, that's a lot of overcommit. Plus, switching from > one vcpu to another on the same host thread will, IIRC, require two > full partition switches, which are pretty slow on Power. > > > I remember something similiar at the time... also I had to give more > > RAM to the guest to be able to run 1024 CPUs (sth like 6 gigs versus > > 512 megs for 1 CPU). With the same amount of guest RAM, each extra CPU > > would cause the memory used by QEMU to grow about 8 megs. > > Hm... that seems like rather a lot. Any idea why? > No but I'll try again with the current code and I'll have a closer look. > > > > > --- > > > hw/ppc/spapr.c | 2 +- > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c > > > index e465d7ac98..46b81a625d 100644 > > > --- a/hw/ppc/spapr.c > > > +++ b/hw/ppc/spapr.c > > > @@ -2712,7 +2712,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data) > > > mc->init = ppc_spapr_init; > > > mc->reset = ppc_spapr_reset; > > > mc->block_default_type = IF_SCSI; > > > - mc->max_cpus = 255; > > > + mc->max_cpus = 1024; > > > mc->no_parallel = 1; > > > mc->default_boot_order = ""; > > > mc->default_ram_size = 512 * M_BYTE; > > > > > > > [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 181 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2017-02-27 22:14 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-02-24 4:55 [Qemu-devel] [PATCH qemu] sysemu: support up to 1024 vCPUs Alexey Kardashevskiy 2017-02-24 6:16 ` David Gibson 2017-02-24 9:13 ` [Qemu-devel] [Qemu-ppc] " Greg Kurz 2017-02-27 1:09 ` David Gibson 2017-02-27 22:13 ` Greg Kurz
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.