From: Steven Price <steven.price@arm.com>
To: Marc Zyngier <maz@kernel.org>
Cc: kvm@vger.kernel.org, "Radim Krčmář" <rkrcmar@redhat.com>,
"Catalin Marinas" <catalin.marinas@arm.com>,
"Suzuki K Pouloze" <suzuki.poulose@arm.com>,
linux-doc@vger.kernel.org, "Russell King" <linux@armlinux.org.uk>,
linux-kernel@vger.kernel.org, "James Morse" <james.morse@arm.com>,
linux-arm-kernel@lists.infradead.org,
"Paolo Bonzini" <pbonzini@redhat.com>,
"Will Deacon" <will@kernel.org>,
kvmarm@lists.cs.columbia.edu,
"Julien Thierry" <julien.thierry.kdev@gmail.com>
Subject: Re: [PATCH 0/9] arm64: Stolen time support
Date: Mon, 5 Aug 2019 14:06:05 +0100 [thread overview]
Message-ID: <6789f477-8ab5-cc54-1ad2-8627917b07c9@arm.com> (raw)
In-Reply-To: <20190803190522.5fec8f7d@why>
On 03/08/2019 19:05, Marc Zyngier wrote:
> On Fri, 2 Aug 2019 15:50:08 +0100
> Steven Price <steven.price@arm.com> wrote:
>
> Hi Steven,
>
>> This series add support for paravirtualized time for arm64 guests and
>> KVM hosts following the specification in Arm's document DEN 0057A:
>>
>> https://developer.arm.com/docs/den0057/a
>>
>> It implements support for stolen time, allowing the guest to
>> identify time when it is forcibly not executing.
>>
>> It doesn't implement support for Live Physical Time (LPT) as there are
>> some concerns about the overheads and approach in the above
>> specification, and I expect an updated version of the specification to
>> be released soon with just the stolen time parts.
>
> Thanks for posting this.
>
> My current concern with this series is around the fact that we allocate
> memory from the kernel on behalf of the guest. It is the first example
> of such thing in the ARM port, and I can't really say I'm fond of it.
>
> x86 seems to get away with it by having the memory allocated from
> userspace, why I tend to like more. Yes, put_user is more
> expensive than a straight store, but this isn't done too often either.
>
> What is the rational for your current approach?
As I see it there are 3 approaches that can be taken here:
1. Hypervisor allocates memory and adds it to the virtual machine. This
means that everything to do with the 'device' is encapsulated behind the
KVM_CREATE_DEVICE / KVM_[GS]ET_DEVICE_ATTR ioctls. But since we want the
stolen time structure to be fast it cannot be a trapping region and has
to be backed by real memory - in this case allocated by the host kernel.
2. Host user space allocates memory. Similar to above, but this time
user space needs to manage the memory region as well as the usual
KVM_CREATE_DEVICE dance. I've no objection to this, but it means
kvmtool/QEMU needs to be much more aware of what is going on (e.g. how
to size the memory region).
3. Guest kernel "donates" the memory to the hypervisor for the
structure. As far as I'm aware this is what x86 does. The problems I see
this approach are:
a) kexec becomes much more tricky - there needs to be a disabling
mechanism for the guest to stop the hypervisor scribbling on memory
before starting the new kernel.
b) If there is more than one entity that is interested in the
information (e.g. firmware and kernel) then this requires some form of
arbitration in the guest because the hypervisor doesn't want to have to
track an arbitrary number of regions to update.
c) Performance can suffer if the host kernel doesn't have a suitably
aligned/sized area to use. As you say - put_user() is more expensive.
The structure is updated on every return to the VM.
Of course x86 does prove the third approach can work, but I'm not sure
which is actually better. Avoid the kexec cancellation requirements was
the main driver of the current approach. Although many of the
conversations about this were also tied up with Live Physical Time which
adds its own complications.
Steve
WARNING: multiple messages have this Message-ID (diff)
From: Steven Price <steven.price@arm.com>
To: Marc Zyngier <maz@kernel.org>
Cc: kvm@vger.kernel.org, Catalin Marinas <catalin.marinas@arm.com>,
linux-doc@vger.kernel.org, Russell King <linux@armlinux.org.uk>,
linux-kernel@vger.kernel.org, Paolo Bonzini <pbonzini@redhat.com>,
Will Deacon <will@kernel.org>,
kvmarm@lists.cs.columbia.edu,
linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH 0/9] arm64: Stolen time support
Date: Mon, 5 Aug 2019 14:06:05 +0100 [thread overview]
Message-ID: <6789f477-8ab5-cc54-1ad2-8627917b07c9@arm.com> (raw)
In-Reply-To: <20190803190522.5fec8f7d@why>
On 03/08/2019 19:05, Marc Zyngier wrote:
> On Fri, 2 Aug 2019 15:50:08 +0100
> Steven Price <steven.price@arm.com> wrote:
>
> Hi Steven,
>
>> This series add support for paravirtualized time for arm64 guests and
>> KVM hosts following the specification in Arm's document DEN 0057A:
>>
>> https://developer.arm.com/docs/den0057/a
>>
>> It implements support for stolen time, allowing the guest to
>> identify time when it is forcibly not executing.
>>
>> It doesn't implement support for Live Physical Time (LPT) as there are
>> some concerns about the overheads and approach in the above
>> specification, and I expect an updated version of the specification to
>> be released soon with just the stolen time parts.
>
> Thanks for posting this.
>
> My current concern with this series is around the fact that we allocate
> memory from the kernel on behalf of the guest. It is the first example
> of such thing in the ARM port, and I can't really say I'm fond of it.
>
> x86 seems to get away with it by having the memory allocated from
> userspace, why I tend to like more. Yes, put_user is more
> expensive than a straight store, but this isn't done too often either.
>
> What is the rational for your current approach?
As I see it there are 3 approaches that can be taken here:
1. Hypervisor allocates memory and adds it to the virtual machine. This
means that everything to do with the 'device' is encapsulated behind the
KVM_CREATE_DEVICE / KVM_[GS]ET_DEVICE_ATTR ioctls. But since we want the
stolen time structure to be fast it cannot be a trapping region and has
to be backed by real memory - in this case allocated by the host kernel.
2. Host user space allocates memory. Similar to above, but this time
user space needs to manage the memory region as well as the usual
KVM_CREATE_DEVICE dance. I've no objection to this, but it means
kvmtool/QEMU needs to be much more aware of what is going on (e.g. how
to size the memory region).
3. Guest kernel "donates" the memory to the hypervisor for the
structure. As far as I'm aware this is what x86 does. The problems I see
this approach are:
a) kexec becomes much more tricky - there needs to be a disabling
mechanism for the guest to stop the hypervisor scribbling on memory
before starting the new kernel.
b) If there is more than one entity that is interested in the
information (e.g. firmware and kernel) then this requires some form of
arbitration in the guest because the hypervisor doesn't want to have to
track an arbitrary number of regions to update.
c) Performance can suffer if the host kernel doesn't have a suitably
aligned/sized area to use. As you say - put_user() is more expensive.
The structure is updated on every return to the VM.
Of course x86 does prove the third approach can work, but I'm not sure
which is actually better. Avoid the kexec cancellation requirements was
the main driver of the current approach. Although many of the
conversations about this were also tied up with Live Physical Time which
adds its own complications.
Steve
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
WARNING: multiple messages have this Message-ID (diff)
From: Steven Price <steven.price@arm.com>
To: Marc Zyngier <maz@kernel.org>
Cc: "Radim Krčmář" <rkrcmar@redhat.com>,
kvm@vger.kernel.org, "Suzuki K Pouloze" <suzuki.poulose@arm.com>,
"Catalin Marinas" <catalin.marinas@arm.com>,
linux-doc@vger.kernel.org, "Russell King" <linux@armlinux.org.uk>,
linux-kernel@vger.kernel.org, "James Morse" <james.morse@arm.com>,
"Julien Thierry" <julien.thierry.kdev@gmail.com>,
"Paolo Bonzini" <pbonzini@redhat.com>,
"Will Deacon" <will@kernel.org>,
kvmarm@lists.cs.columbia.edu,
linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH 0/9] arm64: Stolen time support
Date: Mon, 5 Aug 2019 14:06:05 +0100 [thread overview]
Message-ID: <6789f477-8ab5-cc54-1ad2-8627917b07c9@arm.com> (raw)
In-Reply-To: <20190803190522.5fec8f7d@why>
On 03/08/2019 19:05, Marc Zyngier wrote:
> On Fri, 2 Aug 2019 15:50:08 +0100
> Steven Price <steven.price@arm.com> wrote:
>
> Hi Steven,
>
>> This series add support for paravirtualized time for arm64 guests and
>> KVM hosts following the specification in Arm's document DEN 0057A:
>>
>> https://developer.arm.com/docs/den0057/a
>>
>> It implements support for stolen time, allowing the guest to
>> identify time when it is forcibly not executing.
>>
>> It doesn't implement support for Live Physical Time (LPT) as there are
>> some concerns about the overheads and approach in the above
>> specification, and I expect an updated version of the specification to
>> be released soon with just the stolen time parts.
>
> Thanks for posting this.
>
> My current concern with this series is around the fact that we allocate
> memory from the kernel on behalf of the guest. It is the first example
> of such thing in the ARM port, and I can't really say I'm fond of it.
>
> x86 seems to get away with it by having the memory allocated from
> userspace, why I tend to like more. Yes, put_user is more
> expensive than a straight store, but this isn't done too often either.
>
> What is the rational for your current approach?
As I see it there are 3 approaches that can be taken here:
1. Hypervisor allocates memory and adds it to the virtual machine. This
means that everything to do with the 'device' is encapsulated behind the
KVM_CREATE_DEVICE / KVM_[GS]ET_DEVICE_ATTR ioctls. But since we want the
stolen time structure to be fast it cannot be a trapping region and has
to be backed by real memory - in this case allocated by the host kernel.
2. Host user space allocates memory. Similar to above, but this time
user space needs to manage the memory region as well as the usual
KVM_CREATE_DEVICE dance. I've no objection to this, but it means
kvmtool/QEMU needs to be much more aware of what is going on (e.g. how
to size the memory region).
3. Guest kernel "donates" the memory to the hypervisor for the
structure. As far as I'm aware this is what x86 does. The problems I see
this approach are:
a) kexec becomes much more tricky - there needs to be a disabling
mechanism for the guest to stop the hypervisor scribbling on memory
before starting the new kernel.
b) If there is more than one entity that is interested in the
information (e.g. firmware and kernel) then this requires some form of
arbitration in the guest because the hypervisor doesn't want to have to
track an arbitrary number of regions to update.
c) Performance can suffer if the host kernel doesn't have a suitably
aligned/sized area to use. As you say - put_user() is more expensive.
The structure is updated on every return to the VM.
Of course x86 does prove the third approach can work, but I'm not sure
which is actually better. Avoid the kexec cancellation requirements was
the main driver of the current approach. Although many of the
conversations about this were also tied up with Live Physical Time which
adds its own complications.
Steve
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2019-08-05 13:06 UTC|newest]
Thread overview: 141+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-02 14:50 [PATCH 0/9] arm64: Stolen time support Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-02 14:50 ` [PATCH 1/9] KVM: arm64: Document PV-time interface Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-03 11:13 ` Marc Zyngier
2019-08-03 11:13 ` Marc Zyngier
2019-08-03 11:13 ` Marc Zyngier
2019-08-05 13:06 ` Steven Price
2019-08-05 13:06 ` Steven Price
2019-08-05 13:06 ` Steven Price
2019-08-05 3:23 ` Zenghui Yu
2019-08-05 3:23 ` Zenghui Yu
2019-08-05 3:23 ` Zenghui Yu
2019-08-05 13:06 ` Steven Price
2019-08-05 13:06 ` Steven Price
2019-08-05 13:06 ` Steven Price
2019-08-05 16:40 ` Christophe de Dinechin
2019-08-05 16:40 ` Christophe de Dinechin
2019-08-05 16:40 ` Christophe de Dinechin
2019-08-07 13:21 ` Steven Price
2019-08-07 13:21 ` Steven Price
2019-08-07 13:21 ` Steven Price
2019-08-07 14:28 ` Christophe de Dinechin
2019-08-07 15:26 ` Steven Price
2019-08-07 15:26 ` Steven Price
2019-08-07 15:26 ` Steven Price
2019-08-02 14:50 ` [PATCH 2/9] KVM: arm/arm64: Factor out hypercall handling from PSCI code Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-02 14:50 ` [PATCH 3/9] KVM: arm64: Implement PV_FEATURES call Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-03 11:21 ` Marc Zyngier
2019-08-03 11:21 ` Marc Zyngier
2019-08-03 11:21 ` Marc Zyngier
2019-08-05 13:14 ` Steven Price
2019-08-05 13:14 ` Steven Price
2019-08-05 13:14 ` Steven Price
2019-08-02 14:50 ` [PATCH 4/9] KVM: arm64: Support stolen time reporting via shared structure Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-03 11:55 ` Marc Zyngier
2019-08-03 11:55 ` Marc Zyngier
2019-08-03 11:55 ` Marc Zyngier
2019-08-05 14:09 ` Steven Price
2019-08-05 14:09 ` Steven Price
2019-08-05 14:09 ` Steven Price
2019-08-03 17:58 ` Marc Zyngier
2019-08-03 17:58 ` Marc Zyngier
2019-08-03 17:58 ` Marc Zyngier
2019-08-03 18:13 ` Marc Zyngier
2019-08-03 18:13 ` Marc Zyngier
2019-08-03 18:13 ` Marc Zyngier
2019-08-05 14:18 ` Steven Price
2019-08-05 14:18 ` Steven Price
2019-08-05 14:18 ` Steven Price
2019-08-02 14:50 ` [PATCH 5/9] KVM: Allow kvm_device_ops to be const Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-02 14:50 ` [PATCH 6/9] KVM: arm64: Provide a PV_TIME device to user space Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-03 12:51 ` Marc Zyngier
2019-08-03 12:51 ` Marc Zyngier
2019-08-03 12:51 ` Marc Zyngier
2019-08-03 17:34 ` Marc Zyngier
2019-08-03 17:34 ` Marc Zyngier
2019-08-03 17:34 ` Marc Zyngier
2019-08-07 13:39 ` Steven Price
2019-08-07 13:39 ` Steven Price
2019-08-07 13:39 ` Steven Price
2019-08-07 13:51 ` Marc Zyngier
2019-08-07 13:51 ` Marc Zyngier
2019-08-07 13:51 ` Marc Zyngier
2019-08-05 16:10 ` Steven Price
2019-08-05 16:10 ` Steven Price
2019-08-05 16:10 ` Steven Price
2019-08-05 16:28 ` Marc Zyngier
2019-08-05 16:28 ` Marc Zyngier
2019-08-05 16:28 ` Marc Zyngier
2019-08-02 14:50 ` [PATCH 7/9] arm/arm64: Provide a wrapper for SMCCC 1.1 calls Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-05 10:03 ` Will Deacon
2019-08-05 10:03 ` Will Deacon
2019-08-05 10:03 ` Will Deacon
2019-08-02 14:50 ` [PATCH 8/9] arm/arm64: Make use of the SMCCC 1.1 wrapper Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-02 14:50 ` [PATCH 9/9] arm64: Retrieve stolen time as paravirtualized guest Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-02 14:50 ` Steven Price
2019-08-04 9:53 ` Marc Zyngier
2019-08-04 9:53 ` Marc Zyngier
2019-08-04 9:53 ` Marc Zyngier
2019-08-08 15:29 ` Steven Price
2019-08-08 15:29 ` Steven Price
2019-08-08 15:29 ` Steven Price
2019-08-08 15:49 ` Marc Zyngier
2019-08-08 15:49 ` Marc Zyngier
2019-08-08 15:49 ` Marc Zyngier
2019-08-09 13:51 ` Zenghui Yu
2019-08-09 13:51 ` Zenghui Yu
2019-08-09 13:51 ` Zenghui Yu
2019-08-12 10:39 ` Steven Price
2019-08-12 10:39 ` Steven Price
2019-08-12 10:39 ` Steven Price
2019-08-13 6:06 ` Zenghui Yu
2019-08-13 6:06 ` Zenghui Yu
2019-08-13 6:06 ` Zenghui Yu
2019-08-03 18:05 ` [PATCH 0/9] arm64: Stolen time support Marc Zyngier
2019-08-03 18:05 ` Marc Zyngier
2019-08-03 18:05 ` Marc Zyngier
2019-08-05 13:06 ` Steven Price [this message]
2019-08-05 13:06 ` Steven Price
2019-08-05 13:06 ` Steven Price
2019-08-05 13:26 ` Marc Zyngier
2019-08-05 13:26 ` Marc Zyngier
2019-08-05 13:26 ` Marc Zyngier
2019-08-14 13:02 ` Alexander Graf
2019-08-14 13:02 ` Alexander Graf
2019-08-14 13:02 ` Alexander Graf
2019-08-14 14:19 ` Marc Zyngier
2019-08-14 14:19 ` Marc Zyngier
2019-08-14 14:52 ` [UNVERIFIED SENDER] " Alexander Graf
2019-08-14 14:52 ` Alexander Graf
2019-08-14 14:52 ` Alexander Graf
2019-08-16 10:23 ` Steven Price
2019-08-16 10:23 ` Steven Price
2019-08-16 10:23 ` Steven Price
2020-07-21 3:26 ` zhukeqian
2020-07-21 3:26 ` zhukeqian
2020-07-21 3:26 ` zhukeqian
2020-07-27 10:48 ` Steven Price
2020-07-27 10:48 ` Steven Price
2020-07-27 10:48 ` Steven Price
2020-07-29 2:57 ` zhukeqian
2020-07-29 2:57 ` zhukeqian
2020-07-29 2:57 ` zhukeqian
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6789f477-8ab5-cc54-1ad2-8627917b07c9@arm.com \
--to=steven.price@arm.com \
--cc=catalin.marinas@arm.com \
--cc=james.morse@arm.com \
--cc=julien.thierry.kdev@gmail.com \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@armlinux.org.uk \
--cc=maz@kernel.org \
--cc=pbonzini@redhat.com \
--cc=rkrcmar@redhat.com \
--cc=suzuki.poulose@arm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.