From: Nuno Das Neves <nunodasneves@linux.microsoft.com>
To: Michael Kelley <mikelley@microsoft.com>,
"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>
Cc: "virtualization@lists.linux-foundation.org"
<virtualization@lists.linux-foundation.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"viremana@linux.microsoft.com" <viremana@linux.microsoft.com>,
Sunil Muthuswamy <sunilmut@microsoft.com>,
"wei.liu@kernel.org" <wei.liu@kernel.org>,
Lillian Grassin-Drake <Lillian.GrassinDrake@microsoft.com>,
KY Srinivasan <kys@microsoft.com>
Subject: Re: [RFC PATCH 07/18] virt/mshv: withdraw memory hypercall
Date: Fri, 5 Mar 2021 13:01:36 -0800 [thread overview]
Message-ID: <0044d83d-e207-50d4-ce71-00d605c6ea7c@linux.microsoft.com> (raw)
In-Reply-To: <MWHPR21MB159305DE956E80D2B45ABA85D78F9@MWHPR21MB1593.namprd21.prod.outlook.com>
On 2/8/2021 11:44 AM, Michael Kelley wrote:
> From: Nuno Das Neves <nunodasneves@linux.microsoft.com> Sent: Friday, November 20, 2020 4:30 PM
>>
>> Withdraw the memory from a finalized partition and free the pages.
>> The partition is now cleaned up correctly when the fd is released.
>>
>> Co-developed-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
>> Signed-off-by: Lillian Grassin-Drake <ligrassi@microsoft.com>
>> Signed-off-by: Nuno Das Neves <nunodasneves@linux.microsoft.com>
>> ---
>> include/asm-generic/hyperv-tlfs.h | 10 ++++++
>> virt/mshv/mshv_main.c | 54 ++++++++++++++++++++++++++++++-
>> 2 files changed, 63 insertions(+), 1 deletion(-)
>>
>> diff --git a/include/asm-generic/hyperv-tlfs.h b/include/asm-generic/hyperv-tlfs.h
>> index ab6ae6c164f5..2a49503b7396 100644
>> --- a/include/asm-generic/hyperv-tlfs.h
>> +++ b/include/asm-generic/hyperv-tlfs.h
>> @@ -148,6 +148,7 @@ struct ms_hyperv_tsc_page {
>> #define HVCALL_DELETE_PARTITION 0x0043
>> #define HVCALL_GET_PARTITION_ID 0x0046
>> #define HVCALL_DEPOSIT_MEMORY 0x0048
>> +#define HVCALL_WITHDRAW_MEMORY 0x0049
>> #define HVCALL_CREATE_VP 0x004e
>> #define HVCALL_GET_VP_REGISTERS 0x0050
>> #define HVCALL_SET_VP_REGISTERS 0x0051
>> @@ -472,6 +473,15 @@ union hv_proximity_domain_info {
>> u64 as_uint64;
>> };
>>
>> +struct hv_withdraw_memory_in {
>> + u64 partition_id;
>> + union hv_proximity_domain_info proximity_domain_info;
>> +};
>> +
>> +struct hv_withdraw_memory_out {
>> + u64 gpa_page_list[0];
>
> For a variable size array, the Linux kernel community has an effort
> underway to replace occurrences of [0] and [1] with just []. I think
> [] can be used here.
>
It seems the compiler doesn't like that, because there's no other members in the struct?
./include/asm-generic/hyperv-tlfs.h:452:6: error: flexible array member in a struct with no named members
452 | u64 gpa_page_list[];
I'll add a comment explaining that it's a hack to work around the compiler.
>> +};
>> +
>
> Add __packed to the above two structs.
>
Will do.
>> struct hv_lp_startup_status {
>> u64 hv_status;
>> u64 substatus1;
>> diff --git a/virt/mshv/mshv_main.c b/virt/mshv/mshv_main.c
>> index c4130a6508e5..162a1bb42a4a 100644
>> --- a/virt/mshv/mshv_main.c
>> +++ b/virt/mshv/mshv_main.c
>> @@ -14,6 +14,7 @@
>> #include <linux/slab.h>
>> #include <linux/file.h>
>> #include <linux/anon_inodes.h>
>> +#include <linux/mm.h>
>> #include <linux/mshv.h>
>> #include <asm/mshyperv.h>
>>
>> @@ -57,8 +58,58 @@ static struct miscdevice mshv_dev = {
>> .mode = 600,
>> };
>>
>> +#define HV_WITHDRAW_BATCH_SIZE (PAGE_SIZE / sizeof(u64))
>
> Use HV_HYP_PAGE_SIZE so that we're explicit that the dependency
> is on the page size used by Hyper-V, which might be different from the
> guest page size (at least on architectures like ARM64).
>
Yes, will do.
>> #define HV_INIT_PARTITION_DEPOSIT_PAGES 208
>>
>> +static int
>> +hv_call_withdraw_memory(u64 count, int node, u64 partition_id)
>> +{
>> + struct hv_withdraw_memory_in *input_page;
>> + struct hv_withdraw_memory_out *output_page;
>> + u16 completed;
>> + u64 hypercall_status;
>> + unsigned long remaining = count;
>> + int status;
>> + int i;
>> + unsigned long flags;
>> +
>> + while (remaining) {
>> + local_irq_save(flags);
>> +
>> + input_page = (struct hv_withdraw_memory_in *)(*this_cpu_ptr(
>> + hyperv_pcpu_input_arg));
>> + output_page = (struct hv_withdraw_memory_out *)(*this_cpu_ptr(
>> + hyperv_pcpu_output_arg));
>> +
>> + input_page->partition_id = partition_id;
>> + input_page->proximity_domain_info.as_uint64 = 0;
>> + hypercall_status = hv_do_rep_hypercall(
>> + HVCALL_WITHDRAW_MEMORY,
>> + min(remaining, HV_WITHDRAW_BATCH_SIZE), 0, input_page,
>> + output_page);
>> +
>> + completed = (hypercall_status & HV_HYPERCALL_REP_COMP_MASK) >>
>> + HV_HYPERCALL_REP_COMP_OFFSET;
>> +
>> + for (i = 0; i < completed; i++)
>> + __free_page(pfn_to_page(output_page->gpa_page_list[i]));
>> +
>> + local_irq_restore(flags);
>
> Seems like there's some risk that we have interrupts disabled for too long.
> We could be calling __free_page() up to 512 times. It might be better for this
> function to allocate its own page to be used as the output page, so that interrupts
> can be enabled immediately after the hypercall completes. Then the __free_page()
> loop can execute with interrupts enabled. We have the per-cpu input and output
> pages to avoid the overhead of allocating/freeing pages for each hypercall, but in this
> case a private output page might be warranted.
>
Good idea, I'll do that.
>> +
>> + status = hypercall_status & HV_HYPERCALL_RESULT_MASK;
>> + if (status != HV_STATUS_SUCCESS) {
>> + if (status != HV_STATUS_NO_RESOURCES)
>> + pr_err("%s: %s\n", __func__,
>> + hv_status_to_string(status));
>> + break;
>> + }
>> +
>> + remaining -= completed;
>> + }
>> +
>> + return -hv_status_to_errno(status);
>> +}
>> +
>> static int
>> hv_call_create_partition(
>> u64 flags,
>> @@ -230,7 +281,8 @@ destroy_partition(struct mshv_partition *partition)
>>
>> /* Deallocates and unmaps everything including vcpus, GPA mappings etc */
>> hv_call_finalize_partition(partition->id);
>> - /* TODO: Withdraw and free all pages we deposited */
>> + /* Withdraw and free all pages we deposited */
>> + hv_call_withdraw_memory(U64_MAX, NUMA_NO_NODE, partition->id);
>>
>> hv_call_delete_partition(partition->id);
>>
>> --
>> 2.25.1
next prev parent reply other threads:[~2021-03-05 21:02 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-21 0:30 [RFC PATCH 00/18] Microsoft Hypervisor root partition ioctl interface Nuno Das Neves
2020-11-21 0:30 ` [RFC PATCH 01/18] x86/hyperv: convert hyperv statuses to linux error codes Nuno Das Neves
2021-02-09 13:04 ` Vitaly Kuznetsov
2021-03-04 18:24 ` Nuno Das Neves
2020-11-21 0:30 ` [RFC PATCH 02/18] asm-generic/hyperv: convert hyperv statuses to strings Nuno Das Neves
2020-11-21 0:30 ` [RFC PATCH 03/18] virt/mshv: minimal mshv module (/dev/mshv/) Nuno Das Neves
2020-11-21 0:30 ` [RFC PATCH 04/18] virt/mshv: request version ioctl Nuno Das Neves
2021-02-08 19:41 ` Michael Kelley
2021-03-04 21:35 ` Nuno Das Neves
2021-02-09 13:11 ` Vitaly Kuznetsov
2021-03-04 18:43 ` Nuno Das Neves
2021-03-05 9:18 ` Vitaly Kuznetsov
2021-04-07 0:21 ` Nuno Das Neves
2021-04-07 7:38 ` Vitaly Kuznetsov
2021-04-07 13:43 ` Wei Liu
2021-04-07 14:02 ` Vitaly Kuznetsov
2021-04-07 14:19 ` Wei Liu
2020-11-21 0:30 ` [RFC PATCH 05/18] virt/mshv: create partition ioctl Nuno Das Neves
2021-02-09 13:15 ` Vitaly Kuznetsov
2021-03-04 18:44 ` Nuno Das Neves
2020-11-21 0:30 ` [RFC PATCH 06/18] virt/mshv: create, initialize, finalize, delete partition hypercalls Nuno Das Neves
2021-02-08 19:42 ` Michael Kelley
2021-03-04 23:49 ` Nuno Das Neves
2021-03-04 23:58 ` Michael Kelley
2020-11-21 0:30 ` [RFC PATCH 07/18] virt/mshv: withdraw memory hypercall Nuno Das Neves
2021-02-08 19:44 ` Michael Kelley
2021-03-05 21:01 ` Nuno Das Neves [this message]
2020-11-21 0:30 ` [RFC PATCH 08/18] virt/mshv: map and unmap guest memory Nuno Das Neves
2021-02-08 19:45 ` Michael Kelley
2021-03-08 19:14 ` Nuno Das Neves
2021-03-08 19:30 ` Michael Kelley
2020-11-21 0:30 ` [RFC PATCH 09/18] virt/mshv: create vcpu ioctl Nuno Das Neves
2020-11-21 0:30 ` [RFC PATCH 10/18] virt/mshv: get and set vcpu registers ioctls Nuno Das Neves
2021-02-08 19:47 ` Michael Kelley
2021-03-09 1:39 ` Nuno Das Neves
2020-11-21 0:30 ` [RFC PATCH 11/18] virt/mshv: set up synic pages for intercept messages Nuno Das Neves
2021-02-08 19:47 ` Michael Kelley
2021-03-11 19:37 ` Nuno Das Neves
2021-03-11 20:45 ` Michael Kelley
2020-11-21 0:30 ` [RFC PATCH 12/18] virt/mshv: run vp ioctl and isr Nuno Das Neves
2020-11-24 16:15 ` Wei Liu
2020-11-21 0:30 ` [RFC PATCH 13/18] virt/mshv: install intercept ioctl Nuno Das Neves
2020-11-21 0:30 ` [RFC PATCH 14/18] virt/mshv: assert interrupt ioctl Nuno Das Neves
2020-11-21 0:30 ` [RFC PATCH 15/18] virt/mshv: get and set vp state ioctls Nuno Das Neves
2021-02-08 19:48 ` Michael Kelley
2021-03-11 23:38 ` Nuno Das Neves
2020-11-21 0:30 ` [RFC PATCH 16/18] virt/mshv: mmap vp register page Nuno Das Neves
2021-02-08 19:49 ` Michael Kelley
2021-03-25 17:36 ` Nuno Das Neves
2020-11-21 0:30 ` [RFC PATCH 17/18] virt/mshv: get and set partition property ioctls Nuno Das Neves
2020-11-21 0:30 ` [RFC PATCH 18/18] virt/mshv: Add enlightenment bits to create partition ioctl Nuno Das Neves
2020-11-24 16:18 ` [RFC PATCH 00/18] Microsoft Hypervisor root partition ioctl interface Wei Liu
2021-02-08 19:40 ` Michael Kelley
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0044d83d-e207-50d4-ce71-00d605c6ea7c@linux.microsoft.com \
--to=nunodasneves@linux.microsoft.com \
--cc=Lillian.GrassinDrake@microsoft.com \
--cc=kys@microsoft.com \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mikelley@microsoft.com \
--cc=sunilmut@microsoft.com \
--cc=viremana@linux.microsoft.com \
--cc=virtualization@lists.linux-foundation.org \
--cc=wei.liu@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).