From: Michael Kelley via Virtualization <virtualization@lists.linux-foundation.org>
To: Nuno Das Neves <nunodasneves@linux.microsoft.com>,
"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>
Cc: "wei.liu@kernel.org" <wei.liu@kernel.org>,
Lillian Grassin-Drake <Lillian.GrassinDrake@microsoft.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"virtualization@lists.linux-foundation.org"
<virtualization@lists.linux-foundation.org>,
Sunil Muthuswamy <sunilmut@microsoft.com>,
"viremana@linux.microsoft.com" <viremana@linux.microsoft.com>
Subject: RE: [RFC PATCH 07/18] virt/mshv: withdraw memory hypercall
Date: Mon, 8 Feb 2021 19:44:52 +0000 [thread overview]
Message-ID: <MWHPR21MB159305DE956E80D2B45ABA85D78F9@MWHPR21MB1593.namprd21.prod.outlook.com> (raw)
In-Reply-To: <1605918637-12192-8-git-send-email-nunodasneves@linux.microsoft.com>
From: Nuno Das Neves <nunodasneves@linux.microsoft.com> Sent: Friday, November 20, 2020 4:30 PM
>
> Withdraw the memory from a finalized partition and free the pages.
> The partition is now cleaned up correctly when the fd is released.
>
> Co-developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
> Signed-off-by: Nuno Das Neves <nunodasneves@linux.microsoft.com>
> ---
> include/asm-generic/hyperv-tlfs.h | 10 ++++++
> virt/mshv/mshv_main.c | 54 ++++++++++++++++++++++++++++++-
> 2 files changed, 63 insertions(+), 1 deletion(-)
>
> diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h
> index ab6ae6c164f5..2a49503b7396 100644
> --- a/include/asm-generic/hyperv-tlfs.h
> +++ b/include/asm-generic/hyperv-tlfs.h
> @@ -148,6 +148,7 @@ struct ms_hyperv_tsc_page {
> #define HVCALL_DELETE_PARTITION 0x0043
> #define HVCALL_GET_PARTITION_ID 0x0046
> #define HVCALL_DEPOSIT_MEMORY 0x0048
> +#define HVCALL_WITHDRAW_MEMORY 0x0049
> #define HVCALL_CREATE_VP 0x004e
> #define HVCALL_GET_VP_REGISTERS 0x0050
> #define HVCALL_SET_VP_REGISTERS 0x0051
> @@ -472,6 +473,15 @@ union hv_proximity_domain_info {
> u64 as_uint64;
> };
>
> +struct hv_withdraw_memory_in {
> + u64 partition_id;
> + union hv_proximity_domain_info proximity_domain_info;
> +};
> +
> +struct hv_withdraw_memory_out {
> + u64 gpa_page_list[0];
For a variable size array, the Linux kernel community has an effort
underway to replace occurrences of [0] and [1] with just []. I think
[] can be used here.
> +};
> +
Add __packed to the above two structs.
> struct hv_lp_startup_status {
> u64 hv_status;
> u64 substatus1;
> diff --git a/virt/mshv/mshv_main.c b/virt/mshv/mshv_main.c
> index c4130a6508e5..162a1bb42a4a 100644
> --- a/virt/mshv/mshv_main.c
> +++ b/virt/mshv/mshv_main.c
> @@ -14,6 +14,7 @@
> #include <linux/slab.h>
> #include <linux/file.h>
> #include <linux/anon_inodes.h>
> +#include <linux/mm.h>
> #include <linux/mshv.h>
> #include <asm/mshyperv.h>
>
> @@ -57,8 +58,58 @@ static struct miscdevice mshv_dev = {
> .mode = 600,
> };
>
> +#define HV_WITHDRAW_BATCH_SIZE (PAGE_SIZE / sizeof(u64))
Use HV_HYP_PAGE_SIZE so that we're explicit that the dependency
is on the page size used by Hyper-V, which might be different from the
guest page size (at least on architectures like ARM64).
> #define HV_INIT_PARTITION_DEPOSIT_PAGES 208
>
> +static int
> +hv_call_withdraw_memory(u64 count, int node, u64 partition_id)
> +{
> + struct hv_withdraw_memory_in *input_page;
> + struct hv_withdraw_memory_out *output_page;
> + u16 completed;
> + u64 hypercall_status;
> + unsigned long remaining = count;
> + int status;
> + int i;
> + unsigned long flags;
> +
> + while (remaining) {
> + local_irq_save(flags);
> +
> + input_page = (struct hv_withdraw_memory_in *)(*this_cpu_ptr(
> + hyperv_pcpu_input_arg));
> + output_page = (struct hv_withdraw_memory_out *)(*this_cpu_ptr(
> + hyperv_pcpu_output_arg));
> +
> + input_page->partition_id = partition_id;
> + input_page->proximity_domain_info.as_uint64 = 0;
> + hypercall_status = hv_do_rep_hypercall(
> + HVCALL_WITHDRAW_MEMORY,
> + min(remaining, HV_WITHDRAW_BATCH_SIZE), 0, input_page,
> + output_page);
> +
> + completed = (hypercall_status & HV_HYPERCALL_REP_COMP_MASK) >>
> + HV_HYPERCALL_REP_COMP_OFFSET;
> +
> + for (i = 0; i < completed; i++)
> + __free_page(pfn_to_page(output_page->gpa_page_list[i]));
> +
> + local_irq_restore(flags);
Seems like there's some risk that we have interrupts disabled for too long.
We could be calling __free_page() up to 512 times. It might be better for this
function to allocate its own page to be used as the output page, so that interrupts
can be enabled immediately after the hypercall completes. Then the __free_page()
loop can execute with interrupts enabled. We have the per-cpu input and output
pages to avoid the overhead of allocating/freeing pages for each hypercall, but in this
case a private output page might be warranted.
> +
> + status = hypercall_status & HV_HYPERCALL_RESULT_MASK;
> + if (status != HV_STATUS_SUCCESS) {
> + if (status != HV_STATUS_NO_RESOURCES)
> + pr_err("%s: %s\n", __func__,
> + hv_status_to_string(status));
> + break;
> + }
> +
> + remaining -= completed;
> + }
> +
> + return -hv_status_to_errno(status);
> +}
> +
> static int
> hv_call_create_partition(
> u64 flags,
> @@ -230,7 +281,8 @@ destroy_partition(struct mshv_partition *partition)
>
> /* Deallocates and unmaps everything including vcpus, GPA mappings etc */
> hv_call_finalize_partition(partition->id);
> - /* TODO: Withdraw and free all pages we deposited */
> + /* Withdraw and free all pages we deposited */
> + hv_call_withdraw_memory(U64_MAX, NUMA_NO_NODE, partition->id);
>
> hv_call_delete_partition(partition->id);
>
> --
> 2.25.1
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
next prev parent reply other threads:[~2021-02-08 19:45 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <1605918637-12192-1-git-send-email-nunodasneves@linux.microsoft.com>
2021-02-08 19:40 ` [RFC PATCH 00/18] Microsoft Hypervisor root partition ioctl interface Michael Kelley via Virtualization
[not found] ` <1605918637-12192-5-git-send-email-nunodasneves@linux.microsoft.com>
2021-02-08 19:41 ` [RFC PATCH 04/18] virt/mshv: request version ioctl Michael Kelley via Virtualization
[not found] ` <87y2fxmlmb.fsf@vitty.brq.redhat.com>
[not found] ` <194e0dad-495e-ae94-3f51-d2c95da52139@linux.microsoft.com>
2021-03-05 9:18 ` Vitaly Kuznetsov
[not found] ` <fc88ba72-83ab-025e-682d-4981762ed4f6@linux.microsoft.com>
2021-04-07 7:38 ` Vitaly Kuznetsov
[not found] ` <20210407134302.ng6n4el2km7sujfp@liuwe-devbox-debian-v2>
2021-04-07 14:02 ` Vitaly Kuznetsov
[not found] ` <1605918637-12192-7-git-send-email-nunodasneves@linux.microsoft.com>
2021-02-08 19:42 ` [RFC PATCH 06/18] virt/mshv: create, initialize, finalize, delete partition hypercalls Michael Kelley via Virtualization
[not found] ` <e6cc796d-f9ee-5203-95a9-05906f95d3f8@linux.microsoft.com>
2021-03-04 23:58 ` Michael Kelley via Virtualization
[not found] ` <1605918637-12192-8-git-send-email-nunodasneves@linux.microsoft.com>
2021-02-08 19:44 ` Michael Kelley via Virtualization [this message]
[not found] ` <1605918637-12192-9-git-send-email-nunodasneves@linux.microsoft.com>
2021-02-08 19:45 ` [RFC PATCH 08/18] virt/mshv: map and unmap guest memory Michael Kelley via Virtualization
[not found] ` <d63330fa-de83-85de-c8ec-74cc90d680e3@linux.microsoft.com>
2021-03-08 19:30 ` Michael Kelley via Virtualization
[not found] ` <1605918637-12192-11-git-send-email-nunodasneves@linux.microsoft.com>
2021-02-08 19:47 ` [RFC PATCH 10/18] virt/mshv: get and set vcpu registers ioctls Michael Kelley via Virtualization
[not found] ` <1605918637-12192-12-git-send-email-nunodasneves@linux.microsoft.com>
2021-02-08 19:47 ` [RFC PATCH 11/18] virt/mshv: set up synic pages for intercept messages Michael Kelley via Virtualization
[not found] ` <9e06a119-880f-5199-903b-056675331d6f@linux.microsoft.com>
2021-03-11 20:45 ` Michael Kelley via Virtualization
[not found] ` <1605918637-12192-16-git-send-email-nunodasneves@linux.microsoft.com>
2021-02-08 19:48 ` [RFC PATCH 15/18] virt/mshv: get and set vp state ioctls Michael Kelley via Virtualization
[not found] ` <1605918637-12192-17-git-send-email-nunodasneves@linux.microsoft.com>
2021-02-08 19:49 ` [RFC PATCH 16/18] virt/mshv: mmap vp register page Michael Kelley via Virtualization
[not found] ` <1605918637-12192-2-git-send-email-nunodasneves@linux.microsoft.com>
2021-02-09 13:04 ` [RFC PATCH 01/18] x86/hyperv: convert hyperv statuses to linux error codes Vitaly Kuznetsov
[not found] ` <1605918637-12192-6-git-send-email-nunodasneves@linux.microsoft.com>
2021-02-09 13:15 ` [RFC PATCH 05/18] virt/mshv: create partition ioctl Vitaly Kuznetsov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=MWHPR21MB159305DE956E80D2B45ABA85D78F9@MWHPR21MB1593.namprd21.prod.outlook.com \
--to=virtualization@lists.linux-foundation.org \
--cc=Lillian.GrassinDrake@microsoft.com \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mikelley@microsoft.com \
--cc=nunodasneves@linux.microsoft.com \
--cc=sunilmut@microsoft.com \
--cc=viremana@linux.microsoft.com \
--cc=wei.liu@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).