Re: [PATCH RFC 2/5] s390x: implement diag260

[David Hildenbrand <david@redhat.com>]

On 15.07.20 12:43, Heiko Carstens wrote:
> On Wed, Jul 15, 2020 at 11:42:37AM +0200, David Hildenbrand wrote:
>> So, in summary, we want to indicate to the guest a memory region that
>> will be used to place memory devices ("device memory region"). The
>> region might have holes and the memory within this region might have
>> different semantics than ordinary system memory. Memory that belongs to
>> memory devices should only be detected+used if the guest OS has support
>> for them (e.g., virtio-mem, virtio-pmem, ...). An unmodified guest
>> (e.g., no virtio-mem driver) should not accidentally make use of such
>> memory.
>>
>> We need a way to
>> a) Tell the guest about boot memory (currently ram_size)
>> b) Tell the guest about the maximum possible ram address, including
>> device memory. (We could also indicate the special "device memory
>> region" explicitly)
>>
>> AFAIK, we have three options:
>>
>> 1. Indicate maxram_size via SCLP, indicate ram_size via diag260(0x10)
>>
>> This is what this series (RFCv1 does).
>>
>> Advantages:
>> - No need for a new diag. No need for memory sensing kernel changes.
>> Disadvantages
>> - Older guests without support for diag260 (<v4.2, kvm-unit-tests) will
>>   assume all memory is accessible. Bad.
> 
> Why would old guests assume that?
> 
> At least in v4.1 the kernel will calculate the max address by using
> increment size * increment number and then test if *each* increment is
> available with tprot.

Yes, we do the same in kvm-unit-tests. But it's not sufficient for
memory devices.

Just because a tprot succeed (for memory belonging to a memory device)
does not mean the kernel should silently start to use that memory.

Note: memory devices are not just DIMMs that can be mapped to storage
increments. The memory might have completely different semantics, that's
why they are glued to a managing virtio device.

For example: a tprot might succeed on a memory region provided by
virtio-mem, this does, however, not mean that the memory can (and
should) be used by the guest.

> 
>> - The semantics of the value returned in ry via diag260(0xc) is somewhat
>>   unclear. Should we return the end address of the highest memory
>>   device? OTOH, an unmodified guest OS (without support for memory
>>   devices) should not have to care at all about any such memory.
> 
> I'm confused. The kernel currently only uses diag260(0x10). How is
> diag260(0xc) relevant here?

We have to implement diag260(0x10) if we implement diag260(0xc), no? Or
can we simply throw a specification exception?

> 
>> 3. Indicate maxram_size and ram_size via SCLP (using the SCLP standby
>>    memory)
>>
>> I did not look into the details, because -ENODOCUMENTATION. At least we
>> would run into some alignment issues (again, having to align
>> ram_size/maxram_size to storage increments - which would no longer be
>> 1MB). We would run into issues later, trying to also support standby memory.
> 
> That doesn't make sense to me: either support memory hotplug via
> sclp/standby memory, or with your new method. But trying to support
> both.. what's the use case?

Not sure if there is any, it just feels cleaner to me to separate the
architectured (sclp memory/reserved/standby) bits that specify a
semantic when used via rnmax+tprot from QEMU specific memory ranges that
have special semantics.

virtio-mem is only one type of a virtio-based memory device. In the
future we might want to have virtio-pmem, but there might be more ...

-- 
Thanks,

David / dhildenb