* [PATCH v13 0/4] Add throttling detection to sev-guest @ 2023-01-24 21:14 Dionna Glaze 2023-01-24 21:14 ` [PATCH v13 1/4] virt/coco/sev-guest: Add throttling awareness Dionna Glaze ` (3 more replies) 0 siblings, 4 replies; 13+ messages in thread From: Dionna Glaze @ 2023-01-24 21:14 UTC (permalink / raw) To: linux-kernel, x86; +Cc: Dionna Glaze The guest request synchronous API from SEV-SNP VMs to the host's security processor consumes a global resource. For this reason, AMD's docs recommend that the host implements a throttling mechanism. In order for the guest to know it's been throttled and should try its request again, we need some good-faith communication from the host that the request has been throttled. These patches work with the existing /dev/sev-guest ABI to detect a throttling code. Changes from v12: * Reordered fix patch to the beginning and kept it minimal. * Changed documentation in same patch as the respective change to the header. * Changed exitinfo2 in dev_alert to print in hex. Changes from v11: * Squashed all type changing patches into 1 that modifies both sev-guest and x86/kernel/sev.c. * Removed fw_err field from sev-guest command struct (renamed exitinfo2). Changes from v10: * Added sev_guestreq_err_t typedef early in chain to change a signature acress x86/sev and virt/coco/sev-guest in a single change. This makes all patches build. I have 3 cleanup patches to change the type and subsequently remove the typedef. * Changed exitinfo2 initial undefined value back to 0xff since Thomas indicated that a firmware error is only 16 bits. Changes from v9: * Rebased on v6.2-rc3 Changes from v8: * Added documentation changes. * Changed commit messages to use passive voice. * Simplified control flow for __sev_platform_init_locked. Changes from v7: * Replaced handle_guest_request arguments msg_ver and fw_err with a pointer to the snp_guest_request_ioctl argument struct. Changes from v6: * Rebased on the IV reuse fix patch * renamed rate_hz to rate_s and fixed its MODULE_PARM_DESC to use the correct variable name. * Changed sleep_timeout_interrutible (not defined) to schedule_timeout_interruptible. Changes from v5: * Fixed commit prefix text * Added all get_maintainers.pl folks to commits' Cc tags * Changed SET_RET_NO_FW_CALL commit's metadata to show pgonda signs off and is the author. Changes from v4: * Clarified comment on SEV_RET_NO_FW_CALL * Changed ratelimit loop to use sleep_timeout_interruptible Changes from v3: * sev-guest ratelimits itself to one request twice a second. * Fixed a type signature to use u64 instead of unsigned int * Set *exitinfo2 unconditionally after the ghcb_hv_call. Changes from v2: * Codified the non-firmware-call firmware error code as (u32)-1. * Changed sev_issue_guest_request unsigned long *fw_err argument to u64 *exitinfo2 to more accurately and type-safely describe the value that it outputs. * Changed sev_issue_guest_request to always set its exitinfo2 argument to either the non-firmware-call error code, the EXIT_INFO_2 returned from the VMM if the request failed, or 0 on success. This fixes a bug that returned uninitialized kernel stack memory to the user when there is no error. * Changed the throttle behavior to retry in the driver instead of returning -EAGAIN, due to possible message sequence number reuse on different message contents. Changes from v1: * Changed throttle error code to 2 Dionna Glaze (3): virt/coco/sev-guest: Add throttling awareness x86/sev: Change snp_guest_issue_request's fw_err virt: sev-guest: self-throttle guest request retries Peter Gonda (1): crypto: ccp - Name -1 return value as SEV_RET_NO_FW_CALL Documentation/virt/coco/sev-guest.rst | 21 ++++--- arch/x86/include/asm/sev-common.h | 3 - arch/x86/include/asm/sev.h | 4 +- arch/x86/kernel/sev.c | 13 +++-- drivers/crypto/ccp/sev-dev.c | 22 ++++--- drivers/virt/coco/sev-guest/sev-guest.c | 77 +++++++++++++++++-------- include/uapi/linux/psp-sev.h | 7 +++ include/uapi/linux/sev-guest.h | 18 +++++- 8 files changed, 114 insertions(+), 51 deletions(-) -- 2.39.1.405.gd4c25cc71f-goog ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH v13 1/4] virt/coco/sev-guest: Add throttling awareness 2023-01-24 21:14 [PATCH v13 0/4] Add throttling detection to sev-guest Dionna Glaze @ 2023-01-24 21:14 ` Dionna Glaze 2023-01-25 17:28 ` Tom Lendacky 2023-01-30 11:13 ` Borislav Petkov 2023-01-24 21:14 ` [PATCH v13 2/4] crypto: ccp - Name -1 return value as SEV_RET_NO_FW_CALL Dionna Glaze ` (2 subsequent siblings) 3 siblings, 2 replies; 13+ messages in thread From: Dionna Glaze @ 2023-01-24 21:14 UTC (permalink / raw) To: linux-kernel, x86 Cc: Dionna Glaze, Tom Lendacky, Paolo Bonzini, Joerg Roedel, Peter Gonda, Thomas Gleixner, Dave Hansen, Ingo Molnar, Borislav Petkov, H. Peter Anvin, Venu Busireddy, Michael Roth, Kirill A. Shutemov, Michael Sterritt The host is permitted and encouraged to throttle guest requests to the AMD-SP since it is a shared resource across all VMs. Without throttling-awareness, the host returning an error will immediately lock out access to the VMPCK, which makes the VM less useful as it can't attest itself. Since throttling is expected to be a common occurrence, a cooperative host can return a VMM error code that the request was throttled. The driver interprets the upper 32 bits of exitinfo2 as a VMM error code. For safety, since the encryption algorithm in GHCBv2 is AES_GCM, control must remain in the kernel to complete the request with the current sequence number. Returning without finishing the request allows the the guest to make another request but with different message contents. This is IV reuse, and breaks cryptographic protections. A guest request may not make it to the AMD-SP before the host returns to the guest, so the err local variable in handle_guest_request must be initialized the same way fw_err is. snp_issue_guest_request similarly should set fw_err whether or not the value is non-zero, in order to appropriately clear the error value when zero. Cc: Tom Lendacky <Thomas.Lendacky@amd.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Joerg Roedel <jroedel@suse.de> Cc: Peter Gonda <pgonda@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Borislav Petkov <Borislav.Petkov@amd.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Venu Busireddy <venu.busireddy@oracle.com> Cc: Michael Roth <michael.roth@amd.com> Cc: "Kirill A. Shutemov" <kirill@shutemov.name> Cc: Michael Sterritt <sterritt@google.com> Fixes: d5af44dde546 ("x86/sev: Provide support for SNP guest request NAEs") Signed-off-by: Dionna Glaze <dionnaglaze@google.com> --- arch/x86/include/asm/sev-common.h | 3 ++- arch/x86/kernel/sev.c | 3 +-- drivers/virt/coco/sev-guest/sev-guest.c | 11 ++++++++++- 3 files changed, 13 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/sev-common.h b/arch/x86/include/asm/sev-common.h index b8357d6ecd47..b63be696b776 100644 --- a/arch/x86/include/asm/sev-common.h +++ b/arch/x86/include/asm/sev-common.h @@ -128,8 +128,9 @@ struct snp_psc_desc { struct psc_entry entries[VMGEXIT_PSC_MAX_ENTRY]; } __packed; -/* Guest message request error code */ +/* Guest message request error codes */ #define SNP_GUEST_REQ_INVALID_LEN BIT_ULL(32) +#define SNP_GUEST_REQ_ERR_BUSY BIT_ULL(33) #define GHCB_MSR_TERM_REQ 0x100 #define GHCB_MSR_TERM_REASON_SET_POS 12 diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index 679026a640ef..a908ffc2dfba 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -2212,14 +2212,13 @@ int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned if (ret) goto e_put; + *fw_err = ghcb->save.sw_exit_info_2; if (ghcb->save.sw_exit_info_2) { /* Number of expected pages are returned in RBX */ if (exit_code == SVM_VMGEXIT_EXT_GUEST_REQUEST && ghcb->save.sw_exit_info_2 == SNP_GUEST_REQ_INVALID_LEN) input->data_npages = ghcb_get_rbx(ghcb); - *fw_err = ghcb->save.sw_exit_info_2; - ret = -EIO; } diff --git a/drivers/virt/coco/sev-guest/sev-guest.c b/drivers/virt/coco/sev-guest/sev-guest.c index 4ec4174e05a3..3d6551fdf06f 100644 --- a/drivers/virt/coco/sev-guest/sev-guest.c +++ b/drivers/virt/coco/sev-guest/sev-guest.c @@ -322,7 +322,7 @@ static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, in u8 type, void *req_buf, size_t req_sz, void *resp_buf, u32 resp_sz, __u64 *fw_err) { - unsigned long err; + unsigned long err = 0xff; u64 seqno; int rc; @@ -338,6 +338,7 @@ static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, in if (rc) return rc; +retry: /* * Call firmware to process the request. In this function the encrypted * message enters shared memory with the host. So after this call the @@ -346,6 +347,14 @@ static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, in */ rc = snp_issue_guest_request(exit_code, &snp_dev->input, &err); + /* + * The host may return SNP_GUEST_REQ_ERR_EBUSY if the request has been + * throttled. Retry in the driver to avoid returning and reusing the + * message sequence number on a different message. + */ + if (err == SNP_GUEST_REQ_ERR_BUSY) + goto retry; + /* * If the extended guest request fails due to having too small of a * certificate data buffer, retry the same guest request without the -- 2.39.1.405.gd4c25cc71f-goog ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH v13 1/4] virt/coco/sev-guest: Add throttling awareness 2023-01-24 21:14 ` [PATCH v13 1/4] virt/coco/sev-guest: Add throttling awareness Dionna Glaze @ 2023-01-25 17:28 ` Tom Lendacky 2023-01-25 17:48 ` Dionna Amalie Glaze 2023-01-30 11:13 ` Borislav Petkov 1 sibling, 1 reply; 13+ messages in thread From: Tom Lendacky @ 2023-01-25 17:28 UTC (permalink / raw) To: Dionna Glaze, linux-kernel, x86 Cc: Paolo Bonzini, Joerg Roedel, Peter Gonda, Thomas Gleixner, Dave Hansen, Ingo Molnar, Borislav Petkov, H. Peter Anvin, Venu Busireddy, Michael Roth, Kirill A. Shutemov, Michael Sterritt On 1/24/23 15:14, Dionna Glaze wrote: > The host is permitted and encouraged to throttle guest requests to the > AMD-SP since it is a shared resource across all VMs. Without > throttling-awareness, the host returning an error will immediately lock > out access to the VMPCK, which makes the VM less useful as it can't > attest itself. Since throttling is expected to be a common occurrence, a It's not expected to be a common occurrence, but a host should protect itself from an un-cooperative guest. > cooperative host can return a VMM error code that the request was > throttled. > > The driver interprets the upper 32 bits of exitinfo2 as a VMM error code. > For safety, since the encryption algorithm in GHCBv2 is AES_GCM, control > must remain in the kernel to complete the request with the current > sequence number. Returning without finishing the request allows the the > guest to make another request but with different message contents. This > is IV reuse, and breaks cryptographic protections. > > A guest request may not make it to the AMD-SP before the host returns to > the guest, so the err local variable in handle_guest_request must be > initialized the same way fw_err is. snp_issue_guest_request similarly > should set fw_err whether or not the value is non-zero, in order to > appropriately clear the error value when zero. > > Cc: Tom Lendacky <Thomas.Lendacky@amd.com> > Cc: Paolo Bonzini <pbonzini@redhat.com> > Cc: Joerg Roedel <jroedel@suse.de> > Cc: Peter Gonda <pgonda@google.com> > Cc: Thomas Gleixner <tglx@linutronix.de> > Cc: Dave Hansen <dave.hansen@linux.intel.com> > Cc: Ingo Molnar <mingo@redhat.com> > Cc: Borislav Petkov <Borislav.Petkov@amd.com> > Cc: "H. Peter Anvin" <hpa@zytor.com> > Cc: Venu Busireddy <venu.busireddy@oracle.com> > Cc: Michael Roth <michael.roth@amd.com> > Cc: "Kirill A. Shutemov" <kirill@shutemov.name> > Cc: Michael Sterritt <sterritt@google.com> > > Fixes: d5af44dde546 ("x86/sev: Provide support for SNP guest request > NAEs") > > Signed-off-by: Dionna Glaze <dionnaglaze@google.com> > --- > arch/x86/include/asm/sev-common.h | 3 ++- > arch/x86/kernel/sev.c | 3 +-- > drivers/virt/coco/sev-guest/sev-guest.c | 11 ++++++++++- > 3 files changed, 13 insertions(+), 4 deletions(-) > > diff --git a/arch/x86/include/asm/sev-common.h b/arch/x86/include/asm/sev-common.h > index b8357d6ecd47..b63be696b776 100644 > --- a/arch/x86/include/asm/sev-common.h > +++ b/arch/x86/include/asm/sev-common.h > @@ -128,8 +128,9 @@ struct snp_psc_desc { > struct psc_entry entries[VMGEXIT_PSC_MAX_ENTRY]; > } __packed; > > -/* Guest message request error code */ > +/* Guest message request error codes */ > #define SNP_GUEST_REQ_INVALID_LEN BIT_ULL(32) > +#define SNP_GUEST_REQ_ERR_BUSY BIT_ULL(33) It was probably a short cut to use BIT_ULL() to start with because the error codes are not intended to be single bit positions, e.g., the next value will be 3. So these should really be: #define SNP_GUEST_REQ_INVALID_LEN (1ULL << 32) #define SNP_GUEST_REQ_ERR_BUSY (2ULL << 32) Thanks, Tom > > #define GHCB_MSR_TERM_REQ 0x100 > #define GHCB_MSR_TERM_REASON_SET_POS 12 > diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c > index 679026a640ef..a908ffc2dfba 100644 > --- a/arch/x86/kernel/sev.c > +++ b/arch/x86/kernel/sev.c > @@ -2212,14 +2212,13 @@ int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned > if (ret) > goto e_put; > > + *fw_err = ghcb->save.sw_exit_info_2; > if (ghcb->save.sw_exit_info_2) { > /* Number of expected pages are returned in RBX */ > if (exit_code == SVM_VMGEXIT_EXT_GUEST_REQUEST && > ghcb->save.sw_exit_info_2 == SNP_GUEST_REQ_INVALID_LEN) > input->data_npages = ghcb_get_rbx(ghcb); > > - *fw_err = ghcb->save.sw_exit_info_2; > - > ret = -EIO; > } > > diff --git a/drivers/virt/coco/sev-guest/sev-guest.c b/drivers/virt/coco/sev-guest/sev-guest.c > index 4ec4174e05a3..3d6551fdf06f 100644 > --- a/drivers/virt/coco/sev-guest/sev-guest.c > +++ b/drivers/virt/coco/sev-guest/sev-guest.c > @@ -322,7 +322,7 @@ static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, in > u8 type, void *req_buf, size_t req_sz, void *resp_buf, > u32 resp_sz, __u64 *fw_err) > { > - unsigned long err; > + unsigned long err = 0xff; > u64 seqno; > int rc; > > @@ -338,6 +338,7 @@ static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, in > if (rc) > return rc; > > +retry: > /* > * Call firmware to process the request. In this function the encrypted > * message enters shared memory with the host. So after this call the > @@ -346,6 +347,14 @@ static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, in > */ > rc = snp_issue_guest_request(exit_code, &snp_dev->input, &err); > > + /* > + * The host may return SNP_GUEST_REQ_ERR_EBUSY if the request has been > + * throttled. Retry in the driver to avoid returning and reusing the > + * message sequence number on a different message. > + */ > + if (err == SNP_GUEST_REQ_ERR_BUSY) > + goto retry; > + > /* > * If the extended guest request fails due to having too small of a > * certificate data buffer, retry the same guest request without the ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v13 1/4] virt/coco/sev-guest: Add throttling awareness 2023-01-25 17:28 ` Tom Lendacky @ 2023-01-25 17:48 ` Dionna Amalie Glaze 2023-01-25 18:31 ` Tom Lendacky 0 siblings, 1 reply; 13+ messages in thread From: Dionna Amalie Glaze @ 2023-01-25 17:48 UTC (permalink / raw) To: Tom Lendacky Cc: linux-kernel, x86, Paolo Bonzini, Joerg Roedel, Peter Gonda, Thomas Gleixner, Dave Hansen, Ingo Molnar, Borislav Petkov, H. Peter Anvin, Venu Busireddy, Michael Roth, Kirill A. Shutemov, Michael Sterritt > > So these should really be: > > #define SNP_GUEST_REQ_INVALID_LEN (1ULL << 32) > #define SNP_GUEST_REQ_ERR_BUSY (2ULL << 32) > > Thanks, > Tom > Patch 3/4 cleans them up with just such a shift. They also move from arch/x86/include/asm/sev-common.h to include/uapi/linux/sev-guest.h. Is it okay to delay that cleanup until after the fix patch? -- -Dionna Glaze, PhD (she/her) ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v13 1/4] virt/coco/sev-guest: Add throttling awareness 2023-01-25 17:48 ` Dionna Amalie Glaze @ 2023-01-25 18:31 ` Tom Lendacky 0 siblings, 0 replies; 13+ messages in thread From: Tom Lendacky @ 2023-01-25 18:31 UTC (permalink / raw) To: Dionna Amalie Glaze Cc: linux-kernel, x86, Paolo Bonzini, Joerg Roedel, Peter Gonda, Thomas Gleixner, Dave Hansen, Ingo Molnar, Borislav Petkov, H. Peter Anvin, Venu Busireddy, Michael Roth, Kirill A. Shutemov, Michael Sterritt On 1/25/23 11:48, Dionna Amalie Glaze wrote: >> >> So these should really be: >> >> #define SNP_GUEST_REQ_INVALID_LEN (1ULL << 32) >> #define SNP_GUEST_REQ_ERR_BUSY (2ULL << 32) >> >> Thanks, >> Tom >> > > Patch 3/4 cleans them up with just such a shift. They also move from > arch/x86/include/asm/sev-common.h to include/uapi/linux/sev-guest.h. > Is it okay to delay that cleanup until after the fix patch? Yeah, if they're being deleted later and fixed up, I'm ok with that. Thanks, Tom > > > -- > -Dionna Glaze, PhD (she/her) ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v13 1/4] virt/coco/sev-guest: Add throttling awareness 2023-01-24 21:14 ` [PATCH v13 1/4] virt/coco/sev-guest: Add throttling awareness Dionna Glaze 2023-01-25 17:28 ` Tom Lendacky @ 2023-01-30 11:13 ` Borislav Petkov 2023-01-30 16:36 ` Tom Lendacky 1 sibling, 1 reply; 13+ messages in thread From: Borislav Petkov @ 2023-01-30 11:13 UTC (permalink / raw) To: Dionna Glaze Cc: linux-kernel, x86, Tom Lendacky, Paolo Bonzini, Joerg Roedel, Peter Gonda, Thomas Gleixner, Dave Hansen, Ingo Molnar, H. Peter Anvin, Venu Busireddy, Michael Roth, Kirill A. Shutemov, Michael Sterritt On Tue, Jan 24, 2023 at 09:14:52PM +0000, Dionna Glaze wrote: > The host is permitted and encouraged to throttle guest requests to the > AMD-SP since it is a shared resource across all VMs. Without > throttling-awareness, the host returning an error will immediately lock > out access to the VMPCK, which makes the VM less useful as it can't > attest itself. Since throttling is expected to be a common occurrence, a > cooperative host can return a VMM error code that the request was > throttled. So where is this concept of guest throttling documented? It sounds like this is something hypervisors do and it is all fine and great to do that but where does it say: yes, this is what we do and this is the usual behavior that's expected from guests and HVs to adhere to when accessing this shared resource? Tom, is that in the spec somewhere perhaps? Or was this decided upon on some call? In any case, I'd like for us to document it somewhere eventually if that hasn't happened yet so that all parties are clear on what is supposed to happen and what the protocol is. > +retry: > /* > * Call firmware to process the request. In this function the encrypted > * message enters shared memory with the host. So after this call the > @@ -346,6 +347,14 @@ static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, in > */ > rc = snp_issue_guest_request(exit_code, &snp_dev->input, &err); > > + /* > + * The host may return SNP_GUEST_REQ_ERR_EBUSY if the request has been > + * throttled. Retry in the driver to avoid returning and reusing the > + * message sequence number on a different message. > + */ > + if (err == SNP_GUEST_REQ_ERR_BUSY) > + goto retry; I don't like even potential endless loops. How about you turn this into a loop with a sufficiently large retry count which, when depleted, gets this request failed with a -ETIMEDOUT or what not? You could also stick a cond_resched() in that loop so that it can take a breather between the requests and doesn't hammer the hw as much. Thx. -- Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v13 1/4] virt/coco/sev-guest: Add throttling awareness 2023-01-30 11:13 ` Borislav Petkov @ 2023-01-30 16:36 ` Tom Lendacky 2023-02-08 19:24 ` Dionna Amalie Glaze 0 siblings, 1 reply; 13+ messages in thread From: Tom Lendacky @ 2023-01-30 16:36 UTC (permalink / raw) To: Borislav Petkov, Dionna Glaze Cc: linux-kernel, x86, Paolo Bonzini, Joerg Roedel, Peter Gonda, Thomas Gleixner, Dave Hansen, Ingo Molnar, H. Peter Anvin, Venu Busireddy, Michael Roth, Kirill A. Shutemov, Michael Sterritt On 1/30/23 05:13, Borislav Petkov wrote: > On Tue, Jan 24, 2023 at 09:14:52PM +0000, Dionna Glaze wrote: >> The host is permitted and encouraged to throttle guest requests to the >> AMD-SP since it is a shared resource across all VMs. Without >> throttling-awareness, the host returning an error will immediately lock >> out access to the VMPCK, which makes the VM less useful as it can't >> attest itself. Since throttling is expected to be a common occurrence, a >> cooperative host can return a VMM error code that the request was >> throttled. > > So where is this concept of guest throttling documented? > > It sounds like this is something hypervisors do and it is all fine and > great to do that but where does it say: yes, this is what we do and this > is the usual behavior that's expected from guests and HVs to adhere to > when accessing this shared resource? > > Tom, is that in the spec somewhere perhaps? Or was this decided upon on > some call? Yes, this is part of the GHCB 2.02 specification document that is in the process of being published. Thanks, Tom > > In any case, I'd like for us to document it somewhere eventually if that > hasn't happened yet so that all parties are clear on what is supposed to > happen and what the protocol is. > >> +retry: >> /* >> * Call firmware to process the request. In this function the encrypted >> * message enters shared memory with the host. So after this call the >> @@ -346,6 +347,14 @@ static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, in >> */ >> rc = snp_issue_guest_request(exit_code, &snp_dev->input, &err); >> >> + /* >> + * The host may return SNP_GUEST_REQ_ERR_EBUSY if the request has been >> + * throttled. Retry in the driver to avoid returning and reusing the >> + * message sequence number on a different message. >> + */ >> + if (err == SNP_GUEST_REQ_ERR_BUSY) >> + goto retry; > > I don't like even potential endless loops. > > How about you turn this into a loop with a sufficiently large retry > count which, when depleted, gets this request failed with a -ETIMEDOUT > or what not? > > You could also stick a cond_resched() in that loop so that it can take a > breather between the requests and doesn't hammer the hw as much. > > Thx. > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v13 1/4] virt/coco/sev-guest: Add throttling awareness 2023-01-30 16:36 ` Tom Lendacky @ 2023-02-08 19:24 ` Dionna Amalie Glaze 2023-02-08 19:29 ` Borislav Petkov 0 siblings, 1 reply; 13+ messages in thread From: Dionna Amalie Glaze @ 2023-02-08 19:24 UTC (permalink / raw) To: Tom Lendacky Cc: Borislav Petkov, linux-kernel, x86, Paolo Bonzini, Joerg Roedel, Peter Gonda, Thomas Gleixner, Dave Hansen, Ingo Molnar, H. Peter Anvin, Venu Busireddy, Michael Roth, Kirill A. Shutemov, Michael Sterritt > > > > > In any case, I'd like for us to document it somewhere eventually if that > > hasn't happened yet so that all parties are clear on what is supposed to > > happen and what the protocol is. > > Hi y'all, checking in on this thread. Are we waiting for the new GHCB spec to be published before merging this fix? -- -Dionna Glaze, PhD (she/her) ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v13 1/4] virt/coco/sev-guest: Add throttling awareness 2023-02-08 19:24 ` Dionna Amalie Glaze @ 2023-02-08 19:29 ` Borislav Petkov 2023-02-08 19:30 ` Dionna Amalie Glaze 0 siblings, 1 reply; 13+ messages in thread From: Borislav Petkov @ 2023-02-08 19:29 UTC (permalink / raw) To: Dionna Amalie Glaze Cc: Tom Lendacky, linux-kernel, x86, Paolo Bonzini, Joerg Roedel, Peter Gonda, Thomas Gleixner, Dave Hansen, Ingo Molnar, H. Peter Anvin, Venu Busireddy, Michael Roth, Kirill A. Shutemov, Michael Sterritt On Wed, Feb 08, 2023 at 11:24:46AM -0800, Dionna Amalie Glaze wrote: > Hi y'all, checking in on this thread. Are we waiting for the new GHCB > spec to be published before merging this fix? Not really - I'm waiting for you to remove the potential endless loop: https://lore.kernel.org/r/Y9emVjoTBrM2%2BY5P@zn.tnic Thx. -- Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v13 1/4] virt/coco/sev-guest: Add throttling awareness 2023-02-08 19:29 ` Borislav Petkov @ 2023-02-08 19:30 ` Dionna Amalie Glaze 0 siblings, 0 replies; 13+ messages in thread From: Dionna Amalie Glaze @ 2023-02-08 19:30 UTC (permalink / raw) To: Borislav Petkov Cc: Tom Lendacky, linux-kernel, x86, Paolo Bonzini, Joerg Roedel, Peter Gonda, Thomas Gleixner, Dave Hansen, Ingo Molnar, H. Peter Anvin, Venu Busireddy, Michael Roth, Kirill A. Shutemov, Michael Sterritt > Not really - I'm waiting for you to remove the potential endless loop: > > https://lore.kernel.org/r/Y9emVjoTBrM2%2BY5P@zn.tnic > > Thx. > Ah that slipped my attention. Thanks, I'll update that. -- -Dionna Glaze, PhD (she/her) ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH v13 2/4] crypto: ccp - Name -1 return value as SEV_RET_NO_FW_CALL 2023-01-24 21:14 [PATCH v13 0/4] Add throttling detection to sev-guest Dionna Glaze 2023-01-24 21:14 ` [PATCH v13 1/4] virt/coco/sev-guest: Add throttling awareness Dionna Glaze @ 2023-01-24 21:14 ` Dionna Glaze 2023-01-24 21:14 ` [PATCH v13 3/4] x86/sev: Change snp_guest_issue_request's fw_err Dionna Glaze 2023-01-24 21:14 ` [PATCH v13 4/4] virt: sev-guest: self-throttle guest request retries Dionna Glaze 3 siblings, 0 replies; 13+ messages in thread From: Dionna Glaze @ 2023-01-24 21:14 UTC (permalink / raw) To: linux-kernel, x86 Cc: Peter Gonda, Thomas Lendacky, Paolo Bonzini, Joerg Roedel, Ingo Molnar, Andy Lutomirsky, John Allen, Herbert Xu, David S. Miller, Borislav Petkov, Dionna Glaze From: Peter Gonda <pgonda@google.com> The PSP can return a "firmware error" code of -1 in circumstances where the PSP is not actually called. To make this protocol unambiguous, the value is named SEV_RET_NO_FW_CALL. Cc: Thomas Lendacky <Thomas.Lendacky@amd.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Joerg Roedel <jroedel@suse.de> Cc: Ingo Molnar <mingo@redhat.com> Cc: Andy Lutomirsky <luto@kernel.org> Cc: John Allen <john.allen@amd.com> Cc: Herbert Xu <herbert@gondor.apana.org.au> Cc: "David S. Miller" <davem@davemloft.net> Cc: Borislav Petkov <Borislav.Petkov@amd.com> Signed-off-by: Peter Gonda <pgonda@google.com> Signed-off-by: Dionna Glaze <dionnaglaze@google.com> --- Documentation/virt/coco/sev-guest.rst | 2 +- drivers/crypto/ccp/sev-dev.c | 22 ++++++++++++++-------- include/uapi/linux/psp-sev.h | 7 +++++++ 3 files changed, 22 insertions(+), 9 deletions(-) diff --git a/Documentation/virt/coco/sev-guest.rst b/Documentation/virt/coco/sev-guest.rst index bf593e88cfd9..e76393e389eb 100644 --- a/Documentation/virt/coco/sev-guest.rst +++ b/Documentation/virt/coco/sev-guest.rst @@ -41,7 +41,7 @@ The guest ioctl should be issued on a file descriptor of the /dev/sev-guest devi The ioctl accepts struct snp_user_guest_request. The input and output structure is specified through the req_data and resp_data field respectively. If the ioctl fails to execute due to a firmware error, then fw_err code will be set otherwise the -fw_err will be set to 0x00000000000000ff. +fw_err will be set to 0x00000000ffffffff, i.e., the lower 32-bits are -1. The firmware checks that the message sequence counter is one greater than the guests message sequence counter. If guest driver fails to increment message diff --git a/drivers/crypto/ccp/sev-dev.c b/drivers/crypto/ccp/sev-dev.c index 06fc7156c04f..ac205f78a595 100644 --- a/drivers/crypto/ccp/sev-dev.c +++ b/drivers/crypto/ccp/sev-dev.c @@ -440,12 +440,19 @@ static int __sev_init_ex_locked(int *error) return __sev_do_cmd_locked(SEV_CMD_INIT_EX, &data, error); } +static inline int __sev_do_init_locked(int *psp_ret) +{ + if (sev_init_ex_buffer) + return __sev_init_ex_locked(psp_ret); + else + return __sev_init_locked(psp_ret); +} + static int __sev_platform_init_locked(int *error) { struct psp_device *psp = psp_master; struct sev_device *sev; - int rc = 0, psp_ret = -1; - int (*init_function)(int *error); + int rc = 0, psp_ret = SEV_RET_NO_FW_CALL; if (!psp || !psp->sev_data) return -ENODEV; @@ -456,15 +463,12 @@ static int __sev_platform_init_locked(int *error) return 0; if (sev_init_ex_buffer) { - init_function = __sev_init_ex_locked; rc = sev_read_init_ex_file(); if (rc) return rc; - } else { - init_function = __sev_init_locked; } - rc = init_function(&psp_ret); + rc = __sev_do_init_locked(&psp_ret); if (rc && psp_ret == SEV_RET_SECURE_DATA_INVALID) { /* * Initialization command returned an integrity check failure @@ -473,9 +477,11 @@ static int __sev_platform_init_locked(int *error) * initialization function should succeed by replacing the state * with a reset state. */ - dev_err(sev->dev, "SEV: retrying INIT command because of SECURE_DATA_INVALID error. Retrying once to reset PSP SEV state."); - rc = init_function(&psp_ret); + dev_err(sev->dev, +"SEV: retrying INIT command because of SECURE_DATA_INVALID error. Retrying once to reset PSP SEV state."); + rc = __sev_do_init_locked(&psp_ret); } + if (error) *error = psp_ret; diff --git a/include/uapi/linux/psp-sev.h b/include/uapi/linux/psp-sev.h index 91b4c63d5cbf..e8cfb8bde0d7 100644 --- a/include/uapi/linux/psp-sev.h +++ b/include/uapi/linux/psp-sev.h @@ -36,6 +36,13 @@ enum { * SEV Firmware status code */ typedef enum { + /* + * This error code is not in the SEV spec but is added to convey that + * there was an error that prevented the SEV Firmware from being called. + * The SEV API error codes are 16 bits, so the -1 value will not overlap + * with possible values from the specification. + */ + SEV_RET_NO_FW_CALL = -1, SEV_RET_SUCCESS = 0, SEV_RET_INVALID_PLATFORM_STATE, SEV_RET_INVALID_GUEST_STATE, -- 2.39.1.405.gd4c25cc71f-goog ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v13 3/4] x86/sev: Change snp_guest_issue_request's fw_err 2023-01-24 21:14 [PATCH v13 0/4] Add throttling detection to sev-guest Dionna Glaze 2023-01-24 21:14 ` [PATCH v13 1/4] virt/coco/sev-guest: Add throttling awareness Dionna Glaze 2023-01-24 21:14 ` [PATCH v13 2/4] crypto: ccp - Name -1 return value as SEV_RET_NO_FW_CALL Dionna Glaze @ 2023-01-24 21:14 ` Dionna Glaze 2023-01-24 21:14 ` [PATCH v13 4/4] virt: sev-guest: self-throttle guest request retries Dionna Glaze 3 siblings, 0 replies; 13+ messages in thread From: Dionna Glaze @ 2023-01-24 21:14 UTC (permalink / raw) To: linux-kernel, x86 Cc: Dionna Glaze, Tom Lendacky, Paolo Bonzini, Joerg Roedel, Peter Gonda, Thomas Gleixner, Dave Hansen, Ingo Molnar, Borislav Petkov, H. Peter Anvin, Venu Busireddy, Michael Roth, Kirill A. Shutemov, Michael Sterritt The GHCB specification declares that the firmware error value for a guest request will be stored in the lower 32 bits of EXIT_INFO_2. The upper 32 bits are for the VMM's own error code. The fw_err argument is thus a misnomer, and callers will need access to all 64 bits. The type of unsigned long also causes problems, since sw_exit_info2 is u64 (unsigned long long) vs the argument's unsigned long*. This type is changed for issuing the guest request. The ioctl command struct's error field is passed directly instead of a local variable, since an incomplete guest request may not set the error code, and uninitialized stack memory would be written back to user space. The firmware might not even be called, so the call is bookended with the no firmware call error and clearing the error. Since the "fw_err" field is really exitinfo2 split into the upper bits' vmm error code and lower bits' firmware error code, sev-guest.h is updated to represent the 64 bit value as a union. Cc: Tom Lendacky <Thomas.Lendacky@amd.com> Cc: Paolo Bonzini <pbonzini@redhat.com> Cc: Joerg Roedel <jroedel@suse.de> Cc: Peter Gonda <pgonda@google.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: Ingo Molnar <mingo@redhat.com> Cc: Borislav Petkov <Borislav.Petkov@amd.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Venu Busireddy <venu.busireddy@oracle.com> Cc: Michael Roth <michael.roth@amd.com> Cc: "Kirill A. Shutemov" <kirill@shutemov.name> Cc: Michael Sterritt <sterritt@google.com> Signed-off-by: Dionna Glaze <dionnaglaze@google.com> --- Documentation/virt/coco/sev-guest.rst | 21 +++++++---- arch/x86/include/asm/sev-common.h | 4 -- arch/x86/include/asm/sev.h | 4 +- arch/x86/kernel/sev.c | 12 ++++-- drivers/virt/coco/sev-guest/sev-guest.c | 50 ++++++++++++------------- include/uapi/linux/sev-guest.h | 18 ++++++++- 6 files changed, 64 insertions(+), 45 deletions(-) diff --git a/Documentation/virt/coco/sev-guest.rst b/Documentation/virt/coco/sev-guest.rst index e76393e389eb..48e89ea2a618 100644 --- a/Documentation/virt/coco/sev-guest.rst +++ b/Documentation/virt/coco/sev-guest.rst @@ -37,11 +37,12 @@ along with a description: the return value. General error numbers (-ENOMEM, -EINVAL) are not detailed, but errors with specific meanings are. -The guest ioctl should be issued on a file descriptor of the /dev/sev-guest device. -The ioctl accepts struct snp_user_guest_request. The input and output structure is -specified through the req_data and resp_data field respectively. If the ioctl fails -to execute due to a firmware error, then fw_err code will be set otherwise the -fw_err will be set to 0x00000000ffffffff, i.e., the lower 32-bits are -1. +The guest ioctl should be issued on a file descriptor of the +/dev/sev-guest device. The ioctl accepts struct +snp_user_guest_request. The input and output structure is specified +through the req_data and resp_data field respectively. If the ioctl +fails to execute due to a firmware error, then the fw_error code will +be set, otherwise fw_error will be set to -1. The firmware checks that the message sequence counter is one greater than the guests message sequence counter. If guest driver fails to increment message @@ -57,8 +58,14 @@ counter (e.g. counter overflow), then -EIO will be returned. __u64 req_data; __u64 resp_data; - /* firmware error code on failure (see psp-sev.h) */ - __u64 fw_err; + /* bits[63:32]: VMM error code, bits[31:0] firmware error code (see psp-sev.h) */ + union { + __u64 exitinfo2; + struct { + __u32 fw_error; + __u32 vmm_error; + }; + }; }; 2.1 SNP_GET_REPORT diff --git a/arch/x86/include/asm/sev-common.h b/arch/x86/include/asm/sev-common.h index b63be696b776..0759af9b1acf 100644 --- a/arch/x86/include/asm/sev-common.h +++ b/arch/x86/include/asm/sev-common.h @@ -128,10 +128,6 @@ struct snp_psc_desc { struct psc_entry entries[VMGEXIT_PSC_MAX_ENTRY]; } __packed; -/* Guest message request error codes */ -#define SNP_GUEST_REQ_INVALID_LEN BIT_ULL(32) -#define SNP_GUEST_REQ_ERR_BUSY BIT_ULL(33) - #define GHCB_MSR_TERM_REQ 0x100 #define GHCB_MSR_TERM_REASON_SET_POS 12 #define GHCB_MSR_TERM_REASON_SET_MASK 0xf diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h index ebc271bb6d8e..05de34d10d89 100644 --- a/arch/x86/include/asm/sev.h +++ b/arch/x86/include/asm/sev.h @@ -196,7 +196,7 @@ void snp_set_memory_private(unsigned long vaddr, unsigned int npages); void snp_set_wakeup_secondary_cpu(void); bool snp_init(struct boot_params *bp); void __init __noreturn snp_abort(void); -int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned long *fw_err); +int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, u64 *exitinfo2); #else static inline void sev_es_ist_enter(struct pt_regs *regs) { } static inline void sev_es_ist_exit(void) { } @@ -217,7 +217,7 @@ static inline void snp_set_wakeup_secondary_cpu(void) { } static inline bool snp_init(struct boot_params *bp) { return false; } static inline void snp_abort(void) { } static inline int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, - unsigned long *fw_err) + u64 *exitinfo2) { return -ENOTTY; } diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c index a908ffc2dfba..82aa81fafb71 100644 --- a/arch/x86/kernel/sev.c +++ b/arch/x86/kernel/sev.c @@ -22,6 +22,8 @@ #include <linux/efi.h> #include <linux/platform_device.h> #include <linux/io.h> +#include <linux/psp-sev.h> +#include <uapi/linux/sev-guest.h> #include <asm/cpu_entry_area.h> #include <asm/stacktrace.h> @@ -2175,7 +2177,7 @@ static int __init init_sev_config(char *str) } __setup("sev=", init_sev_config); -int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned long *fw_err) +int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, u64 *exitinfo2) { struct ghcb_state state; struct es_em_ctxt ctxt; @@ -2186,9 +2188,11 @@ int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned if (!cc_platform_has(CC_ATTR_GUEST_SEV_SNP)) return -ENODEV; - if (!fw_err) + if (!exitinfo2) return -EINVAL; + *exitinfo2 = SEV_RET_NO_FW_CALL; + /* * __sev_get_ghcb() needs to run with IRQs disabled because it is using * a per-CPU GHCB. @@ -2212,11 +2216,11 @@ int snp_issue_guest_request(u64 exit_code, struct snp_req_data *input, unsigned if (ret) goto e_put; - *fw_err = ghcb->save.sw_exit_info_2; + *exitinfo2 = ghcb->save.sw_exit_info_2; if (ghcb->save.sw_exit_info_2) { /* Number of expected pages are returned in RBX */ if (exit_code == SVM_VMGEXIT_EXT_GUEST_REQUEST && - ghcb->save.sw_exit_info_2 == SNP_GUEST_REQ_INVALID_LEN) + ghcb->save.sw_exit_info_2 == SNP_GUEST_VMM_ERR(SNP_GUEST_VMM_ERR_INVALID_LEN)) input->data_npages = ghcb_get_rbx(ghcb); ret = -EIO; diff --git a/drivers/virt/coco/sev-guest/sev-guest.c b/drivers/virt/coco/sev-guest/sev-guest.c index 3d6551fdf06f..e82f080aa679 100644 --- a/drivers/virt/coco/sev-guest/sev-guest.c +++ b/drivers/virt/coco/sev-guest/sev-guest.c @@ -318,11 +318,11 @@ static int enc_payload(struct snp_guest_dev *snp_dev, u64 seqno, int version, u8 return __enc_payload(snp_dev, req, payload, sz); } -static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, int msg_ver, - u8 type, void *req_buf, size_t req_sz, void *resp_buf, - u32 resp_sz, __u64 *fw_err) +static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, + struct snp_guest_request_ioctl *arg, + u8 type, void *req_buf, size_t req_sz, + void *resp_buf, u32 resp_sz) { - unsigned long err = 0xff; u64 seqno; int rc; @@ -334,7 +334,7 @@ static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, in memset(snp_dev->response, 0, sizeof(struct snp_guest_msg)); /* Encrypt the userspace provided payload */ - rc = enc_payload(snp_dev, seqno, msg_ver, type, req_buf, req_sz); + rc = enc_payload(snp_dev, seqno, arg->msg_version, type, req_buf, req_sz); if (rc) return rc; @@ -345,14 +345,15 @@ static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, in * sequence number must be incremented or the VMPCK must be deleted to * prevent reuse of the IV. */ - rc = snp_issue_guest_request(exit_code, &snp_dev->input, &err); + rc = snp_issue_guest_request(exit_code, &snp_dev->input, + &arg->exitinfo2); /* - * The host may return SNP_GUEST_REQ_ERR_EBUSY if the request has been + * The host may return SNP_GUEST_VMM_ERR_BUSY if the request has been * throttled. Retry in the driver to avoid returning and reusing the * message sequence number on a different message. */ - if (err == SNP_GUEST_REQ_ERR_BUSY) + if (arg->vmm_error == SNP_GUEST_VMM_ERR_BUSY) goto retry; /* @@ -362,7 +363,7 @@ static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, in * and thus avoid IV reuse. */ if (exit_code == SVM_VMGEXIT_EXT_GUEST_REQUEST && - err == SNP_GUEST_REQ_INVALID_LEN) { + arg->vmm_error == SNP_GUEST_VMM_ERR_INVALID_LEN) { const unsigned int certs_npages = snp_dev->input.data_npages; exit_code = SVM_VMGEXIT_GUEST_REQUEST; @@ -375,24 +376,22 @@ static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, in * of the VMPCK and the error code being propagated back to the * user as an ioctl() return code. */ - rc = snp_issue_guest_request(exit_code, &snp_dev->input, &err); + rc = snp_issue_guest_request(exit_code, &snp_dev->input, + &arg->exitinfo2); /* * Override the error to inform callers the given extended * request buffer size was too small and give the caller the * required buffer size. */ - err = SNP_GUEST_REQ_INVALID_LEN; + arg->vmm_error = SNP_GUEST_VMM_ERR_INVALID_LEN; snp_dev->input.data_npages = certs_npages; } - if (fw_err) - *fw_err = err; - if (rc) { dev_alert(snp_dev->dev, - "Detected error from ASP request. rc: %d, fw_err: %llu\n", - rc, *fw_err); + "Detected error from ASP request. rc: %d, exitinfo2: %llx\n", + rc, arg->exitinfo2); goto disable_vmpck; } @@ -439,9 +438,9 @@ static int get_report(struct snp_guest_dev *snp_dev, struct snp_guest_request_io if (!resp) return -ENOMEM; - rc = handle_guest_request(snp_dev, SVM_VMGEXIT_GUEST_REQUEST, arg->msg_version, + rc = handle_guest_request(snp_dev, SVM_VMGEXIT_GUEST_REQUEST, arg, SNP_MSG_REPORT_REQ, &req, sizeof(req), resp->data, - resp_len, &arg->fw_err); + resp_len); if (rc) goto e_free; @@ -479,9 +478,8 @@ static int get_derived_key(struct snp_guest_dev *snp_dev, struct snp_guest_reque if (copy_from_user(&req, (void __user *)arg->req_data, sizeof(req))) return -EFAULT; - rc = handle_guest_request(snp_dev, SVM_VMGEXIT_GUEST_REQUEST, arg->msg_version, - SNP_MSG_KEY_REQ, &req, sizeof(req), buf, resp_len, - &arg->fw_err); + rc = handle_guest_request(snp_dev, SVM_VMGEXIT_GUEST_REQUEST, arg, + SNP_MSG_KEY_REQ, &req, sizeof(req), buf, resp_len); if (rc) return rc; @@ -541,12 +539,12 @@ static int get_ext_report(struct snp_guest_dev *snp_dev, struct snp_guest_reques return -ENOMEM; snp_dev->input.data_npages = npages; - ret = handle_guest_request(snp_dev, SVM_VMGEXIT_EXT_GUEST_REQUEST, arg->msg_version, + ret = handle_guest_request(snp_dev, SVM_VMGEXIT_EXT_GUEST_REQUEST, arg, SNP_MSG_REPORT_REQ, &req.data, - sizeof(req.data), resp->data, resp_len, &arg->fw_err); + sizeof(req.data), resp->data, resp_len); /* If certs length is invalid then copy the returned length */ - if (arg->fw_err == SNP_GUEST_REQ_INVALID_LEN) { + if (arg->vmm_error == SNP_GUEST_VMM_ERR_INVALID_LEN) { req.certs_len = snp_dev->input.data_npages << PAGE_SHIFT; if (copy_to_user((void __user *)arg->req_data, &req, sizeof(req))) @@ -581,7 +579,7 @@ static long snp_guest_ioctl(struct file *file, unsigned int ioctl, unsigned long if (copy_from_user(&input, argp, sizeof(input))) return -EFAULT; - input.fw_err = 0xff; + input.exitinfo2 = 0xff; /* Message version must be non-zero */ if (!input.msg_version) @@ -612,7 +610,7 @@ static long snp_guest_ioctl(struct file *file, unsigned int ioctl, unsigned long mutex_unlock(&snp_cmd_mutex); - if (input.fw_err && copy_to_user(argp, &input, sizeof(input))) + if (input.exitinfo2 && copy_to_user(argp, &input, sizeof(input))) return -EFAULT; return ret; diff --git a/include/uapi/linux/sev-guest.h b/include/uapi/linux/sev-guest.h index 256aaeff7e65..2aa39112cf8d 100644 --- a/include/uapi/linux/sev-guest.h +++ b/include/uapi/linux/sev-guest.h @@ -52,8 +52,14 @@ struct snp_guest_request_ioctl { __u64 req_data; __u64 resp_data; - /* firmware error code on failure (see psp-sev.h) */ - __u64 fw_err; + /* bits[63:32]: VMM error code, bits[31:0] firmware error code (see psp-sev.h) */ + union { + __u64 exitinfo2; + struct { + __u32 fw_error; + __u32 vmm_error; + }; + }; }; struct snp_ext_report_req { @@ -77,4 +83,12 @@ struct snp_ext_report_req { /* Get SNP extended report as defined in the GHCB specification version 2. */ #define SNP_GET_EXT_REPORT _IOWR(SNP_GUEST_REQ_IOC_TYPE, 0x2, struct snp_guest_request_ioctl) +/* Guest message request EXIT_INFO_2 constants */ +#define SNP_GUEST_FW_ERR_MASK GENMASK_ULL(31, 0) +#define SNP_GUEST_VMM_ERR_SHIFT 32 +#define SNP_GUEST_VMM_ERR(x) (((u64)x) << SNP_GUEST_VMM_ERR_SHIFT) + +#define SNP_GUEST_VMM_ERR_INVALID_LEN 1 +#define SNP_GUEST_VMM_ERR_BUSY 2 + #endif /* __UAPI_LINUX_SEV_GUEST_H_ */ -- 2.39.1.405.gd4c25cc71f-goog ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH v13 4/4] virt: sev-guest: self-throttle guest request retries 2023-01-24 21:14 [PATCH v13 0/4] Add throttling detection to sev-guest Dionna Glaze ` (2 preceding siblings ...) 2023-01-24 21:14 ` [PATCH v13 3/4] x86/sev: Change snp_guest_issue_request's fw_err Dionna Glaze @ 2023-01-24 21:14 ` Dionna Glaze 3 siblings, 0 replies; 13+ messages in thread From: Dionna Glaze @ 2023-01-24 21:14 UTC (permalink / raw) To: linux-kernel, x86 Cc: Dionna Glaze, Tom Lendacky, Peter Gonda, Borislav Petkov, Tom Lendacky, Liam Merwick, Yang Yingliang, Haowen Bai When throttled, the driver will reschedule itself and then try again after sleeping half its ratelimit time to avoid a big wait queue. The ioctl may block indefinitely, but that has always been the case when deferring these requests to the host. Cc: Tom Lendacky <Thomas.Lendacky@amd.com> Cc: Peter Gonda <pgonda@google.com> Cc: Borislav Petkov <Borislav.Petkov@amd.com> Cc: Tom Lendacky <thomas.lendacky@amd.com> Cc: Liam Merwick <liam.merwick@oracle.com> Cc: Yang Yingliang <yangyingliang@huawei.com> Cc: Haowen Bai <baihaowen@meizu.com> Signed-off-by: Dionna Glaze <dionnaglaze@google.com> --- drivers/virt/coco/sev-guest/sev-guest.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/drivers/virt/coco/sev-guest/sev-guest.c b/drivers/virt/coco/sev-guest/sev-guest.c index e82f080aa679..af0645f40e95 100644 --- a/drivers/virt/coco/sev-guest/sev-guest.c +++ b/drivers/virt/coco/sev-guest/sev-guest.c @@ -14,6 +14,7 @@ #include <linux/io.h> #include <linux/platform_device.h> #include <linux/miscdevice.h> +#include <linux/ratelimit.h> #include <linux/set_memory.h> #include <linux/fs.h> #include <crypto/aead.h> @@ -48,12 +49,22 @@ struct snp_guest_dev { struct snp_req_data input; u32 *os_area_msg_seqno; u8 *vmpck; + + struct ratelimit_state rs; }; static u32 vmpck_id; module_param(vmpck_id, uint, 0444); MODULE_PARM_DESC(vmpck_id, "The VMPCK ID to use when communicating with the PSP."); +static int rate_s = 1; +module_param(rate_s, int, 0444); +MODULE_PARM_DESC(rate_s, "The rate limit interval in seconds to limit requests to."); + +static int rate_burst = 2; +module_param(rate_burst, int, 0444); +MODULE_PARM_DESC(rate_burst, "The rate limit burst amount to limit requests to."); + /* Mutex to serialize the shared buffer access and command handling. */ static DEFINE_MUTEX(snp_cmd_mutex); @@ -339,6 +350,15 @@ static int handle_guest_request(struct snp_guest_dev *snp_dev, u64 exit_code, return rc; retry: + /* + * Rate limit commands internally since the host can also throttle, and + * the guest shouldn't create a tight request spin that could end up + * getting this VM throttled more heavily. + */ + if (!__ratelimit(&snp_dev->rs)) { + schedule_timeout_interruptible((rate_s * HZ) / 2); + goto retry; + } /* * Call firmware to process the request. In this function the encrypted * message enters shared memory with the host. So after this call the @@ -760,6 +780,8 @@ static int __init sev_guest_probe(struct platform_device *pdev) if (ret) goto e_free_cert_data; + ratelimit_state_init(&snp_dev->rs, rate_s * HZ, rate_burst); + dev_info(dev, "Initialized SEV guest driver (using vmpck_id %d)\n", vmpck_id); return 0; -- 2.39.1.405.gd4c25cc71f-goog ^ permalink raw reply related [flat|nested] 13+ messages in thread
end of thread, other threads:[~2023-02-08 19:30 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-01-24 21:14 [PATCH v13 0/4] Add throttling detection to sev-guest Dionna Glaze 2023-01-24 21:14 ` [PATCH v13 1/4] virt/coco/sev-guest: Add throttling awareness Dionna Glaze 2023-01-25 17:28 ` Tom Lendacky 2023-01-25 17:48 ` Dionna Amalie Glaze 2023-01-25 18:31 ` Tom Lendacky 2023-01-30 11:13 ` Borislav Petkov 2023-01-30 16:36 ` Tom Lendacky 2023-02-08 19:24 ` Dionna Amalie Glaze 2023-02-08 19:29 ` Borislav Petkov 2023-02-08 19:30 ` Dionna Amalie Glaze 2023-01-24 21:14 ` [PATCH v13 2/4] crypto: ccp - Name -1 return value as SEV_RET_NO_FW_CALL Dionna Glaze 2023-01-24 21:14 ` [PATCH v13 3/4] x86/sev: Change snp_guest_issue_request's fw_err Dionna Glaze 2023-01-24 21:14 ` [PATCH v13 4/4] virt: sev-guest: self-throttle guest request retries Dionna Glaze
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).