* [Qemu-devel] [PATCH v3 1/1] s390-ccw-virtio: allow for systems larger that 7.999TB
@ 2017-12-11 12:21 Christian Borntraeger
2017-12-11 13:55 ` [Qemu-devel] [qemu-s390x] " David Hildenbrand
2017-12-14 13:21 ` [Qemu-devel] " Cornelia Huck
0 siblings, 2 replies; 7+ messages in thread
From: Christian Borntraeger @ 2017-12-11 12:21 UTC (permalink / raw)
To: Cornelia Huck
Cc: qemu-devel, qemu-s390x, Halil Pasic, Alexander Graf, Thomas Huth,
David Hildenbrand, Richard Henderson, Christian Borntraeger
KVM does not allow memory regions > KVM_MEM_MAX_NR_PAGES, basically
limiting the memory per slot to 8TB-4k. As memory slots on s390/kvm must
be a multiple of 1MB we need start a new memory region if we cross
8TB-1M.
With that (and optimistic overcommitment in the kernel) I was able to
start a 24TB guest on a 1TB system.
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
hw/s390x/s390-virtio-ccw.c | 28 +++++++++++++++++++++++++---
1 file changed, 25 insertions(+), 3 deletions(-)
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 8425534..073f6ed 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -154,14 +154,36 @@ static void virtio_ccw_register_hcalls(void)
virtio_ccw_hcall_early_printk);
}
+/*
+ * KVM does only support memory slots up to KVM_MEM_MAX_NR_PAGES pages
+ * as the dirty bitmap must be managed by bitops that take an int as
+ * position indicator. If we have a guest beyond that we will split off
+ * new subregions. The split must happen on a segment boundary (1MB).
+ */
+#define KVM_MEM_MAX_NR_PAGES ((1UL << 31) - 1)
+#define SEG_MSK (~0xfffffULL)
+#define KVM_SLOT_MAX_BYTES ((KVM_MEM_MAX_NR_PAGES * TARGET_PAGE_SIZE) & SEG_MSK)
static void s390_memory_init(ram_addr_t mem_size)
{
MemoryRegion *sysmem = get_system_memory();
- MemoryRegion *ram = g_new(MemoryRegion, 1);
+ ram_addr_t chunk, offset = 0;
+ unsigned int number = 0;
+ gchar *name;
/* allocate RAM for core */
- memory_region_allocate_system_memory(ram, NULL, "s390.ram", mem_size);
- memory_region_add_subregion(sysmem, 0, ram);
+ name = g_strdup_printf("s390.ram");
+ while (mem_size) {
+ MemoryRegion *ram = g_new(MemoryRegion, 1);
+ /* KVM does not allow memslots >= 8 TB */
+ chunk = MIN(mem_size, KVM_SLOT_MAX_BYTES);
+ memory_region_allocate_system_memory(ram, NULL, name, chunk);
+ memory_region_add_subregion(sysmem, offset, ram);
+ mem_size -= chunk;
+ offset += chunk;
+ g_free(name);
+ name = g_strdup_printf("s390.ram.%u", ++number);
+ }
+ g_free(name);
/* Initialize storage key device */
s390_skeys_init();
--
2.9.4
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [Qemu-devel] [qemu-s390x] [PATCH v3 1/1] s390-ccw-virtio: allow for systems larger that 7.999TB
2017-12-11 12:21 [Qemu-devel] [PATCH v3 1/1] s390-ccw-virtio: allow for systems larger that 7.999TB Christian Borntraeger
@ 2017-12-11 13:55 ` David Hildenbrand
2017-12-11 14:04 ` Christian Borntraeger
2017-12-14 13:21 ` [Qemu-devel] " Cornelia Huck
1 sibling, 1 reply; 7+ messages in thread
From: David Hildenbrand @ 2017-12-11 13:55 UTC (permalink / raw)
To: Christian Borntraeger, Cornelia Huck
Cc: Alexander Graf, Thomas Huth, Halil Pasic, qemu-devel, qemu-s390x,
Richard Henderson
On 11.12.2017 13:21, Christian Borntraeger wrote:
> KVM does not allow memory regions > KVM_MEM_MAX_NR_PAGES, basically
> limiting the memory per slot to 8TB-4k. As memory slots on s390/kvm must
> be a multiple of 1MB we need start a new memory region if we cross
> 8TB-1M.
>
> With that (and optimistic overcommitment in the kernel) I was able to
> start a 24TB guest on a 1TB system.
>
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> ---
> hw/s390x/s390-virtio-ccw.c | 28 +++++++++++++++++++++++++---
> 1 file changed, 25 insertions(+), 3 deletions(-)
>
> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
> index 8425534..073f6ed 100644
> --- a/hw/s390x/s390-virtio-ccw.c
> +++ b/hw/s390x/s390-virtio-ccw.c
> @@ -154,14 +154,36 @@ static void virtio_ccw_register_hcalls(void)
> virtio_ccw_hcall_early_printk);
> }
>
> +/*
> + * KVM does only support memory slots up to KVM_MEM_MAX_NR_PAGES pages
> + * as the dirty bitmap must be managed by bitops that take an int as
> + * position indicator. If we have a guest beyond that we will split off
> + * new subregions. The split must happen on a segment boundary (1MB).
> + */
> +#define KVM_MEM_MAX_NR_PAGES ((1UL << 31) - 1)
> +#define SEG_MSK (~0xfffffULL)
> +#define KVM_SLOT_MAX_BYTES ((KVM_MEM_MAX_NR_PAGES * TARGET_PAGE_SIZE) & SEG_MSK)
Just wondering if we could get into trouble when calculating
KVM_MEM_MAX_NR_PAGES * TARGET_PAGE_SIZE
on a host with sizeof(long) == 4
could it wrap? (e.g. crazy mingw stuff)
> static void s390_memory_init(ram_addr_t mem_size)
> {
> MemoryRegion *sysmem = get_system_memory();
> - MemoryRegion *ram = g_new(MemoryRegion, 1);
> + ram_addr_t chunk, offset = 0;
> + unsigned int number = 0;
> + gchar *name;
>
> /* allocate RAM for core */
> - memory_region_allocate_system_memory(ram, NULL, "s390.ram", mem_size);
> - memory_region_add_subregion(sysmem, 0, ram);
> + name = g_strdup_printf("s390.ram");
> + while (mem_size) {
> + MemoryRegion *ram = g_new(MemoryRegion, 1);
I'd add an empty line here.
Reviewed-by: David Hildenbrand <david@redhat.com>
> + /* KVM does not allow memslots >= 8 TB */
> + chunk = MIN(mem_size, KVM_SLOT_MAX_BYTES);
> + memory_region_allocate_system_memory(ram, NULL, name, chunk);
> + memory_region_add_subregion(sysmem, offset, ram);
> + mem_size -= chunk;
> + offset += chunk;
> + g_free(name);
> + name = g_strdup_printf("s390.ram.%u", ++number);
> + }
> + g_free(name);
>
> /* Initialize storage key device */
> s390_skeys_init();
>
--
Thanks,
David / dhildenb
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Qemu-devel] [qemu-s390x] [PATCH v3 1/1] s390-ccw-virtio: allow for systems larger that 7.999TB
2017-12-11 13:55 ` [Qemu-devel] [qemu-s390x] " David Hildenbrand
@ 2017-12-11 14:04 ` Christian Borntraeger
2017-12-11 15:11 ` Cornelia Huck
0 siblings, 1 reply; 7+ messages in thread
From: Christian Borntraeger @ 2017-12-11 14:04 UTC (permalink / raw)
To: David Hildenbrand, Cornelia Huck
Cc: Alexander Graf, Thomas Huth, Halil Pasic, qemu-devel, qemu-s390x,
Richard Henderson
On 12/11/2017 02:55 PM, David Hildenbrand wrote:
> On 11.12.2017 13:21, Christian Borntraeger wrote:
>> KVM does not allow memory regions > KVM_MEM_MAX_NR_PAGES, basically
>> limiting the memory per slot to 8TB-4k. As memory slots on s390/kvm must
>> be a multiple of 1MB we need start a new memory region if we cross
>> 8TB-1M.
>>
>> With that (and optimistic overcommitment in the kernel) I was able to
>> start a 24TB guest on a 1TB system.
>>
>> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
>> ---
>> hw/s390x/s390-virtio-ccw.c | 28 +++++++++++++++++++++++++---
>> 1 file changed, 25 insertions(+), 3 deletions(-)
>>
>> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
>> index 8425534..073f6ed 100644
>> --- a/hw/s390x/s390-virtio-ccw.c
>> +++ b/hw/s390x/s390-virtio-ccw.c
>> @@ -154,14 +154,36 @@ static void virtio_ccw_register_hcalls(void)
>> virtio_ccw_hcall_early_printk);
>> }
>>
>> +/*
>> + * KVM does only support memory slots up to KVM_MEM_MAX_NR_PAGES pages
>> + * as the dirty bitmap must be managed by bitops that take an int as
>> + * position indicator. If we have a guest beyond that we will split off
>> + * new subregions. The split must happen on a segment boundary (1MB).
>> + */
>> +#define KVM_MEM_MAX_NR_PAGES ((1UL << 31) - 1)
>> +#define SEG_MSK (~0xfffffULL)
>> +#define KVM_SLOT_MAX_BYTES ((KVM_MEM_MAX_NR_PAGES * TARGET_PAGE_SIZE) & SEG_MSK)
>
> Just wondering if we could get into trouble when calculating
>
> KVM_MEM_MAX_NR_PAGES * TARGET_PAGE_SIZE
maybe just using 1ULL instead of 1UL?
>
> on a host with sizeof(long) == 4
>
> could it wrap? (e.g. crazy mingw stuff)
>
>> static void s390_memory_init(ram_addr_t mem_size)
>> {
>> MemoryRegion *sysmem = get_system_memory();
>> - MemoryRegion *ram = g_new(MemoryRegion, 1);
>> + ram_addr_t chunk, offset = 0;
>> + unsigned int number = 0;
>> + gchar *name;
>>
>> /* allocate RAM for core */
>> - memory_region_allocate_system_memory(ram, NULL, "s390.ram", mem_size);
>> - memory_region_add_subregion(sysmem, 0, ram);
>> + name = g_strdup_printf("s390.ram");
>> + while (mem_size) {
>> + MemoryRegion *ram = g_new(MemoryRegion, 1);
>
> I'd add an empty line here.
>
> Reviewed-by: David Hildenbrand <david@redhat.com>
>
>> + /* KVM does not allow memslots >= 8 TB */
>> + chunk = MIN(mem_size, KVM_SLOT_MAX_BYTES);
>> + memory_region_allocate_system_memory(ram, NULL, name, chunk);
>> + memory_region_add_subregion(sysmem, offset, ram);
>> + mem_size -= chunk;
>> + offset += chunk;
>> + g_free(name);
>> + name = g_strdup_printf("s390.ram.%u", ++number);
>> + }
>> + g_free(name);
>>
>> /* Initialize storage key device */
>> s390_skeys_init();
>>
>
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Qemu-devel] [qemu-s390x] [PATCH v3 1/1] s390-ccw-virtio: allow for systems larger that 7.999TB
2017-12-11 14:04 ` Christian Borntraeger
@ 2017-12-11 15:11 ` Cornelia Huck
0 siblings, 0 replies; 7+ messages in thread
From: Cornelia Huck @ 2017-12-11 15:11 UTC (permalink / raw)
To: Christian Borntraeger
Cc: David Hildenbrand, Alexander Graf, Thomas Huth, Halil Pasic,
qemu-devel, qemu-s390x, Richard Henderson
On Mon, 11 Dec 2017 15:04:13 +0100
Christian Borntraeger <borntraeger@de.ibm.com> wrote:
> On 12/11/2017 02:55 PM, David Hildenbrand wrote:
> > On 11.12.2017 13:21, Christian Borntraeger wrote:
> >> KVM does not allow memory regions > KVM_MEM_MAX_NR_PAGES, basically
> >> limiting the memory per slot to 8TB-4k. As memory slots on s390/kvm must
> >> be a multiple of 1MB we need start a new memory region if we cross
> >> 8TB-1M.
> >>
> >> With that (and optimistic overcommitment in the kernel) I was able to
> >> start a 24TB guest on a 1TB system.
> >>
> >> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> >> ---
> >> hw/s390x/s390-virtio-ccw.c | 28 +++++++++++++++++++++++++---
> >> 1 file changed, 25 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
> >> index 8425534..073f6ed 100644
> >> --- a/hw/s390x/s390-virtio-ccw.c
> >> +++ b/hw/s390x/s390-virtio-ccw.c
> >> @@ -154,14 +154,36 @@ static void virtio_ccw_register_hcalls(void)
> >> virtio_ccw_hcall_early_printk);
> >> }
> >>
> >> +/*
> >> + * KVM does only support memory slots up to KVM_MEM_MAX_NR_PAGES pages
> >> + * as the dirty bitmap must be managed by bitops that take an int as
> >> + * position indicator. If we have a guest beyond that we will split off
> >> + * new subregions. The split must happen on a segment boundary (1MB).
> >> + */
> >> +#define KVM_MEM_MAX_NR_PAGES ((1UL << 31) - 1)
> >> +#define SEG_MSK (~0xfffffULL)
> >> +#define KVM_SLOT_MAX_BYTES ((KVM_MEM_MAX_NR_PAGES * TARGET_PAGE_SIZE) & SEG_MSK)
> >
> > Just wondering if we could get into trouble when calculating
> >
> > KVM_MEM_MAX_NR_PAGES * TARGET_PAGE_SIZE
>
> maybe just using 1ULL instead of 1UL?
I vote for 1ULL, just to be on the safe side.
>
>
> >
> > on a host with sizeof(long) == 4
> >
> > could it wrap? (e.g. crazy mingw stuff)
FWIW, both the 1UL and 1ULL variants compile fine for me both when
building natively and cross-building with mingw.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Qemu-devel] [PATCH v3 1/1] s390-ccw-virtio: allow for systems larger that 7.999TB
2017-12-11 12:21 [Qemu-devel] [PATCH v3 1/1] s390-ccw-virtio: allow for systems larger that 7.999TB Christian Borntraeger
2017-12-11 13:55 ` [Qemu-devel] [qemu-s390x] " David Hildenbrand
@ 2017-12-14 13:21 ` Cornelia Huck
2017-12-14 13:26 ` David Hildenbrand
1 sibling, 1 reply; 7+ messages in thread
From: Cornelia Huck @ 2017-12-14 13:21 UTC (permalink / raw)
To: Christian Borntraeger
Cc: qemu-devel, qemu-s390x, Halil Pasic, Alexander Graf, Thomas Huth,
David Hildenbrand, Richard Henderson
On Mon, 11 Dec 2017 13:21:46 +0100
Christian Borntraeger <borntraeger@de.ibm.com> wrote:
> KVM does not allow memory regions > KVM_MEM_MAX_NR_PAGES, basically
> limiting the memory per slot to 8TB-4k. As memory slots on s390/kvm must
> be a multiple of 1MB we need start a new memory region if we cross
> 8TB-1M.
>
> With that (and optimistic overcommitment in the kernel) I was able to
> start a 24TB guest on a 1TB system.
>
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> ---
> hw/s390x/s390-virtio-ccw.c | 28 +++++++++++++++++++++++++---
> 1 file changed, 25 insertions(+), 3 deletions(-)
Thanks, applied (with 1UL -> 1ULL).
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Qemu-devel] [PATCH v3 1/1] s390-ccw-virtio: allow for systems larger that 7.999TB
2017-12-14 13:21 ` [Qemu-devel] " Cornelia Huck
@ 2017-12-14 13:26 ` David Hildenbrand
2017-12-14 13:28 ` Cornelia Huck
0 siblings, 1 reply; 7+ messages in thread
From: David Hildenbrand @ 2017-12-14 13:26 UTC (permalink / raw)
To: Cornelia Huck, Christian Borntraeger
Cc: qemu-devel, qemu-s390x, Halil Pasic, Alexander Graf, Thomas Huth,
Richard Henderson
On 14.12.2017 14:21, Cornelia Huck wrote:
> On Mon, 11 Dec 2017 13:21:46 +0100
> Christian Borntraeger <borntraeger@de.ibm.com> wrote:
>
>> KVM does not allow memory regions > KVM_MEM_MAX_NR_PAGES, basically
>> limiting the memory per slot to 8TB-4k. As memory slots on s390/kvm must
>> be a multiple of 1MB we need start a new memory region if we cross
>> 8TB-1M.
>>
>> With that (and optimistic overcommitment in the kernel) I was able to
>> start a 24TB guest on a 1TB system.
>>
>> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
>> ---
>> hw/s390x/s390-virtio-ccw.c | 28 +++++++++++++++++++++++++---
>> 1 file changed, 25 insertions(+), 3 deletions(-)
>
> Thanks, applied (with 1UL -> 1ULL).
>
Can you add my
Reviewed-by: David Hildenbrand <david@redhat.com>
?
--
Thanks,
David / dhildenb
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Qemu-devel] [PATCH v3 1/1] s390-ccw-virtio: allow for systems larger that 7.999TB
2017-12-14 13:26 ` David Hildenbrand
@ 2017-12-14 13:28 ` Cornelia Huck
0 siblings, 0 replies; 7+ messages in thread
From: Cornelia Huck @ 2017-12-14 13:28 UTC (permalink / raw)
To: David Hildenbrand
Cc: Christian Borntraeger, qemu-devel, qemu-s390x, Halil Pasic,
Alexander Graf, Thomas Huth, Richard Henderson
On Thu, 14 Dec 2017 14:26:17 +0100
David Hildenbrand <david@redhat.com> wrote:
> On 14.12.2017 14:21, Cornelia Huck wrote:
> > On Mon, 11 Dec 2017 13:21:46 +0100
> > Christian Borntraeger <borntraeger@de.ibm.com> wrote:
> >
> >> KVM does not allow memory regions > KVM_MEM_MAX_NR_PAGES, basically
> >> limiting the memory per slot to 8TB-4k. As memory slots on s390/kvm must
> >> be a multiple of 1MB we need start a new memory region if we cross
> >> 8TB-1M.
> >>
> >> With that (and optimistic overcommitment in the kernel) I was able to
> >> start a 24TB guest on a 1TB system.
> >>
> >> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> >> ---
> >> hw/s390x/s390-virtio-ccw.c | 28 +++++++++++++++++++++++++---
> >> 1 file changed, 25 insertions(+), 3 deletions(-)
> >
> > Thanks, applied (with 1UL -> 1ULL).
> >
>
> Can you add my
>
> Reviewed-by: David Hildenbrand <david@redhat.com>
>
> ?
>
Should be already in there :)
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2017-12-14 13:28 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-12-11 12:21 [Qemu-devel] [PATCH v3 1/1] s390-ccw-virtio: allow for systems larger that 7.999TB Christian Borntraeger
2017-12-11 13:55 ` [Qemu-devel] [qemu-s390x] " David Hildenbrand
2017-12-11 14:04 ` Christian Borntraeger
2017-12-11 15:11 ` Cornelia Huck
2017-12-14 13:21 ` [Qemu-devel] " Cornelia Huck
2017-12-14 13:26 ` David Hildenbrand
2017-12-14 13:28 ` Cornelia Huck
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.