Re: [PATCH RFC 2/5] s390x: implement diag260

From: David Hildenbrand <david@redhat.com>
To: Heiko Carstens <hca@linux.ibm.com>
Cc: Thomas Huth <thuth@redhat.com>,
	Janosch Frank <frankja@linux.ibm.com>,
	"Michael S . Tsirkin" <mst@redhat.com>,
	Cornelia Huck <cohuck@redhat.com>,
	qemu-devel@nongnu.org, Halil Pasic <pasic@linux.ibm.com>,
	Christian Borntraeger <borntraeger@de.ibm.com>,
	qemu-s390x@nongnu.org, Claudio Imbrenda <imbrenda@linux.ibm.com>,
	Richard Henderson <rth@twiddle.net>
Subject: Re: [PATCH RFC 2/5] s390x: implement diag260
Date: Wed, 15 Jul 2020 19:51:27 +0200	[thread overview]
Message-ID: <f30637d2-cfc0-33f3-42ce-df1eaceded8e@redhat.com> (raw)
In-Reply-To: <d612da2a-c8fb-ffa6-0913-6cc0ec9b0f7f@redhat.com>

On 15.07.20 19:38, David Hildenbrand wrote:
> On 15.07.20 18:14, Heiko Carstens wrote:
>> On Wed, Jul 15, 2020 at 01:42:02PM +0200, David Hildenbrand wrote:
>>>> So, are you saying that even at IPL time there might already be memory
>>>> devices attached to the system? And the kernel should _not_ treat them
>>>> as normal memory?
>>>
>>> Sorry if that was unclear. Yes, we can have such devices (including
>>> memory areas) on a cold boot/reboot/kexec. In addition, they might pop
>>> up at runtime (e.g., hotplugging a virtio-mem device). The device is in
>>> charge of exposing that area and deciding what to do with it.
>>>
>>> The kernel should never treat them as normal memory (IOW, system RAM).
>>> Not during a cold boot, not during a reboot. The device driver is
>>> responsible for deciding how to use that memory (e.g., add it as system
>>> RAM), and which parts of that memory are actually valid to be used (even
>>> if a tprot might succeed it might not be valid to use just yet - I guess
>>> somewhat similar to doing a tport on a dcss area - AFAIK, you also don't
>>> want to use it like normal memory).
>>>
>>> E.g., on x86-64, memory exposed via virtio-mem or virtio-pmem is never
>>> exposed via the e820 map. The only trace that there might be *something*
>>> now/in the future is indicated via ACPI SRAT tables. This takes
>>> currently care of indicating the maximum possible PFN.
>>
>> Ok, but all of this needa to be documented somewhere. This raises a
>> couple of questions to me:
> 
> I assume this mostly targets virtio-mem, because the semantics of
> virtio-mem provided memory are extra-weird (in contrast to rather static
> virtio-pmem, which is essentially just an emulated NVDIMM - a disk
> mapped into physical memory).
> 
> Regarding documentation (some linked in the cover letter), so far I have
> (generic/x86-64)
> 
> 1. https://virtio-mem.gitlab.io/
> 2. virtio spec proposal [1]
> 3. QEMU 910b25766b33 ("virtio-mem: Paravirtualized memory hot(un)plug")
> 4. Linux 5f1f79bbc9 ("virtio-mem: Paravirtualized memory hotplug")
> 5. Linux cover letter [2]
> 6. KVM forum talk [3] [4]
> 
> As your questions go quite into technical detail, and I don't feel like
> rewriting the doc here :) , I suggest looking at [2], 1, and 5.

Sorry, I suggest looking at [3] (not [2]) first. Includes pictures and a
comparison to memory ballooning (and DIMM-based memory hotplug).

> [3]
> https://events19.linuxfoundation.org/wp-content/uploads/2017/12/virtio-mem-Paravirtualized-Memory-David-Hildenbrand-Red-Hat-1.pdf
> [4] https://www.youtube.com/watch?v=H65FDUDPu9s
> 

-- 
Thanks,

David / dhildenb