From: Kai Huang <kai.huang@intel.com>
To: Sathyanarayanan Kuppuswamy
<sathyanarayanan.kuppuswamy@linux.intel.com>,
"Yao, Jiewen" <jiewen.yao@intel.com>,
"Nakajima, Jun" <jun.nakajima@intel.com>
Cc: Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
Dave Hansen <dave.hansen@linux.intel.com>,
"x86@kernel.org" <x86@kernel.org>,
"H . Peter Anvin" <hpa@zytor.com>,
"Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>,
"Luck, Tony" <tony.luck@intel.com>,
Andi Kleen <ak@linux.intel.com>,
Wander Lairson Costa <wander@redhat.com>,
Isaku Yamahata <isaku.yamahata@gmail.com>,
"marcelo.cerri@canonical.com" <marcelo.cerri@canonical.com>,
"tim.gardner@canonical.com" <tim.gardner@canonical.com>,
"khalid.elmously@canonical.com" <khalid.elmously@canonical.com>,
"Cox, Philip" <philip.cox@canonical.com>,
LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v8 2/5] x86/tdx: Add TDX Guest event notify interrupt support
Date: Thu, 14 Jul 2022 22:42:57 +1200 [thread overview]
Message-ID: <434ff0edcd5a0f1eb671bb2850ef5444ac1359a3.camel@intel.com> (raw)
In-Reply-To: <503e7135-782c-b72b-6f55-3c4acf55921b@linux.intel.com>
On Wed, 2022-07-13 at 17:46 -0700, Sathyanarayanan Kuppuswamy wrote:
> Hi Kai/Dave,
>
> On 6/27/22 4:21 AM, Kai Huang wrote:
> > On Sat, 2022-06-25 at 15:35 +1200, Yao, Jiewen wrote:
> > > Thank you, Jun.
> > >
> > > Yes. I confirmed that we will include below change to GHCI.next spec.
> > >
> > > ================
> > > 3.5 TDG.VP.VMCALL<SetupEventNotifyInterrupt>
> > >
> > > From: "The host VMM should use SEAMCALL [TDWRVPS] leaf to inject an interrupt at the requested-interrupt vector into the TD via the posted-interrupt descriptor. "
> > >
> > > To: "The host VMM should use SEAMCALL [TDWRVPS] leaf to inject an interrupt at the requested-interrupt vector into the TD VCPU that executed TDG.VP.VMCALL <SetupEventNotifyInterrupt> via the posted-interrupt descriptor. "
> > >
> >
> > Hi Sathy,
> >
> > With this change, I don't think we should use system vector anymore. Instead,
> > we just need one non-migratable IRQ which has a fixed vector on a fixed cpu.
> >
>
> Thanks. As suggested, I have attempted to allocate IRQ vector at runtime
> using irq_domain_alloc_irqs() call. Vector is allocated from
> "x86_vector_domain" as Kai suggested.
I am not expert either. I said "idea only" in my first reply :)
>
> Since I am not well versed in this area, I would like expert comments on it.
> Mainly for IRQ allocation logic in tdx_late_init(). I have tested this version using
> QEMU and it works fine.
>
>
> diff --git a/arch/x86/coco/tdx/tdx.c b/arch/x86/coco/tdx/tdx.c
> index 928dcf7a20d9..dcc878546574 100644
> --- a/arch/x86/coco/tdx/tdx.c
> +++ b/arch/x86/coco/tdx/tdx.c
> @@ -5,12 +5,16 @@
> #define pr_fmt(fmt) "tdx: " fmt
>
> #include <linux/cpufeature.h>
> +#include <linux/interrupt.h>
> +#include <linux/irq.h>
> +#include <linux/numa.h>
> #include <asm/coco.h>
> #include <asm/tdx.h>
> #include <asm/vmx.h>
> #include <asm/insn.h>
> #include <asm/insn-eval.h>
> #include <asm/pgtable.h>
> +#include <asm/irqdomain.h>
>
> /* TDX module Call Leaf IDs */
> #define TDX_GET_INFO 1
> @@ -19,6 +23,7 @@
>
> /* TDX hypercall Leaf IDs */
> #define TDVMCALL_MAP_GPA 0x10001
> +#define TDVMCALL_SETUP_NOTIFY_INTR 0x10004
>
> /* MMIO direction */
> #define EPT_READ 0
> @@ -34,6 +39,26 @@
> #define VE_GET_PORT_NUM(e) ((e) >> 16)
> #define VE_IS_IO_STRING(e) ((e) & BIT(4))
>
> +/*
> + * Handler used to report notifications about
> + * TDX_GUEST_EVENT_NOTIFY_VECTOR IRQ. Currently it will be
> + * used only by the attestation driver. So, race condition
> + * with read/write operation is not considered.
> + */
> +static void (*tdx_event_notify_handler)(void);
> +
> +/* Helper function to register tdx_event_notify_handler */
> +void tdx_setup_ev_notify_handler(void (*handler)(void))
> +{
> + tdx_event_notify_handler = handler;
> +}
> +
> +/* Helper function to unregister tdx_event_notify_handler */
> +void tdx_remove_ev_notify_handler(void)
> +{
> + tdx_event_notify_handler = NULL;
> +}
> +
Looks it's weird that you still need it. Couldn't we pass the function to
handle GetQuote directly to request_irq()?
> /*
> * Wrapper for standard use of __tdx_hypercall with no output aside from
> * return code.
> @@ -98,6 +123,31 @@ static inline void tdx_module_call(u64 fn, u64 rcx, u64 rdx, u64 r8, u64 r9,
> panic("TDCALL %lld failed (Buggy TDX module!)\n", fn);
> }
>
> +/*
> + * tdx_hcall_set_notify_intr() - Setup Event Notify Interrupt Vector.
> + *
> + * @vector: Vector address to be used for notification.
> + *
> + * return 0 on success or failure error number.
> + */
> +static long tdx_hcall_set_notify_intr(u8 vector)
> +{
> + /* Minimum vector value allowed is 32 */
> + if (vector < 32)
> + return -EINVAL;
> +
> + /*
> + * Register callback vector address with VMM. More details
> + * about the ABI can be found in TDX Guest-Host-Communication
> + * Interface (GHCI), sec titled
> + * "TDG.VP.VMCALL<SetupEventNotifyInterrupt>".
> + */
> + if (_tdx_hypercall(TDVMCALL_SETUP_NOTIFY_INTR, vector, 0, 0, 0))
> + return -EIO;
> +
> + return 0;
> +}
> +
> static u64 get_cc_mask(void)
> {
> struct tdx_module_output out;
> @@ -775,3 +825,52 @@ void __init tdx_early_init(void)
>
> pr_info("Guest detected\n");
> }
> +
> +static irqreturn_t tdx_ev_handler(int irq, void *dev_id)
> +{
> + tdx_event_notify_handler();
> + return IRQ_HANDLED;
> +}
> +
> +static int __init tdx_late_init(void)
> +{
> + struct irq_alloc_info info;
> + struct irq_cfg *cfg;
> + int evirq, cpu;
> +
> + if (!cpu_feature_enabled(X86_FEATURE_TDX_GUEST))
> + return 0;
> +
> + if (!x86_vector_domain) {
> + pr_err("x86 vector domain is NULL\n");
> + return 0;
> + }
> +
> + init_irq_alloc_info(&info, NULL);
> +
> + evirq = irq_domain_alloc_irqs(x86_vector_domain, 1, NUMA_NO_NODE, &info);
If I read correctly, if you set info->mask to cpumask_of(cpu), and pass it to
irq_domain_alloc_irqs(), the x86_vector_domain.alloc will immediately allocate a
vector on the given cpu, so you don't need to call irq_set_affinity() and wait
to allocate the vector on _this_ cpu until request_irq().
> +
> + cpu = get_cpu();
> +
> + irq_set_handler(evirq, handle_edge_irq);
> +
> + /*
> + * Event notification vector will be delivered to the CPU
> + * in which TDVMCALL_SETUP_NOTIFY_INTR hypercall is requested.
> + * So set the IRQ affinity to the current CPU.
> + */
> + irq_set_affinity(evirq, cpumask_of(cpu));
> +
> + if (request_irq(evirq, tdx_ev_handler, 0, "tdx_evirq", NULL))
> + pr_err("Request event IRQ failed\n");
> +
> + cfg = irq_cfg(evirq);
> +
> + if (tdx_hcall_set_notify_intr(cfg->vector))
> + pr_err("Setting event notification interrupt failed\n");
> +
> + put_cpu();
So the IRQ's affinity isn't kernel managed. Also looks it doesn't have anything
like IRQF_NOBALANCING to prevent it from being migrated. If I understand
correctly, userspace can still change the affinity which could cause the vector
being reallocated on another cpu?
Perhaps you can pass IRQF_NO_BALANCING to request_irq()?
--
Thanks,
-Kai
next prev parent reply other threads:[~2022-07-14 10:43 UTC|newest]
Thread overview: 77+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-09 2:52 [PATCH v8 0/5] Add TDX Guest Attestation support Kuppuswamy Sathyanarayanan
2022-06-09 2:52 ` [PATCH v8 1/5] x86/tdx: Add TDX Guest attestation interface driver Kuppuswamy Sathyanarayanan
2022-06-24 16:51 ` Dave Hansen
2022-06-27 14:50 ` Sathyanarayanan Kuppuswamy
2022-06-27 17:24 ` Dave Hansen
2022-06-30 23:50 ` Sathyanarayanan Kuppuswamy
2022-07-05 12:07 ` Kai Huang
2022-07-05 18:45 ` Sathyanarayanan Kuppuswamy
2022-07-05 18:52 ` Dave Hansen
2022-07-05 21:21 ` Sathyanarayanan Kuppuswamy
2022-07-05 22:31 ` Kai Huang
2022-07-06 22:27 ` Sathyanarayanan Kuppuswamy
2022-07-06 22:59 ` Kai Huang
2022-07-18 22:52 ` Sathyanarayanan Kuppuswamy
2022-06-09 2:52 ` [PATCH v8 2/5] x86/tdx: Add TDX Guest event notify interrupt support Kuppuswamy Sathyanarayanan
2022-06-20 12:33 ` Kai Huang
2022-06-20 15:44 ` Sathyanarayanan Kuppuswamy
2022-06-23 9:46 ` Kai Huang
2022-06-23 10:24 ` Kai Huang
2022-06-24 22:23 ` Sathyanarayanan Kuppuswamy
2022-06-24 23:41 ` Nakajima, Jun
2022-06-25 3:35 ` Yao, Jiewen
2022-06-27 11:21 ` Kai Huang
2022-06-27 14:56 ` Sathyanarayanan Kuppuswamy
2022-07-14 0:46 ` Sathyanarayanan Kuppuswamy
2022-07-14 10:42 ` Kai Huang [this message]
2022-07-14 20:55 ` Sathyanarayanan Kuppuswamy
2022-07-14 23:58 ` Kai Huang
2022-06-09 2:52 ` [PATCH v8 3/5] x86/mm: Make tdx_enc_status_changed() vmalloc address compatible Kuppuswamy Sathyanarayanan
2022-06-09 2:52 ` [PATCH v8 4/5] x86/mm: Add noalias variants of set_memory_*crypted() functions Kuppuswamy Sathyanarayanan
2022-06-24 13:19 ` Dave Hansen
2022-06-27 15:12 ` Kirill A. Shutemov
2022-06-27 18:24 ` Dave Hansen
2022-06-28 1:15 ` Kai Huang
2022-07-05 15:29 ` Kirill A. Shutemov
2022-07-18 14:22 ` Sathyanarayanan Kuppuswamy
2022-07-19 16:13 ` Kirill A. Shutemov
2022-07-19 17:10 ` Sathyanarayanan Kuppuswamy
2022-07-19 21:55 ` Kirill A. Shutemov
2022-07-20 14:56 ` Sathyanarayanan Kuppuswamy
2022-07-20 16:17 ` Kirill A. Shutemov
2022-07-20 16:58 ` Sathyanarayanan Kuppuswamy
2022-06-09 2:52 ` [PATCH v8 5/5] x86/tdx: Add Quote generation support Kuppuswamy Sathyanarayanan
2022-06-14 12:30 ` Wander Lairson Costa
2022-06-14 12:58 ` Sathyanarayanan Kuppuswamy
2022-07-21 16:08 ` Dave Hansen
2022-07-21 16:42 ` Sathyanarayanan Kuppuswamy
2022-07-21 16:49 ` Dave Hansen
2022-07-21 16:54 ` Sathyanarayanan Kuppuswamy
2022-07-21 17:02 ` Dave Hansen
2022-07-21 17:16 ` Sathyanarayanan Kuppuswamy
2022-07-21 17:19 ` Dave Hansen
2022-07-21 18:31 ` Sathyanarayanan Kuppuswamy
2022-07-21 18:42 ` Isaku Yamahata
2022-07-21 18:52 ` Dave Hansen
2022-07-21 18:57 ` Sathyanarayanan Kuppuswamy
2022-07-21 19:23 ` Dave Hansen
2022-07-21 22:08 ` Sathyanarayanan Kuppuswamy
2022-07-21 23:16 ` Kai Huang
2022-07-21 23:32 ` Kai Huang
2022-07-22 0:27 ` Dave Hansen
2022-07-22 19:05 ` Isaku Yamahata
2022-07-22 19:13 ` Dave Hansen
2022-07-22 21:18 ` Sathyanarayanan Kuppuswamy
2022-07-22 21:24 ` Dave Hansen
2022-07-25 20:19 ` Nakajima, Jun
2022-07-25 20:23 ` Dave Hansen
2022-07-25 21:56 ` Nakajima, Jun
2022-07-25 22:06 ` Sathyanarayanan Kuppuswamy
2022-08-09 6:20 ` Guorui Yu
2022-11-21 2:04 ` Guorui Yu
2022-11-21 2:26 ` Dave Hansen
2023-01-07 0:58 ` Erdem Aktas
2022-07-25 11:05 ` Kai Huang
2022-06-24 18:24 ` [PATCH v8 0/5] Add TDX Guest Attestation support Dave Hansen
2022-06-27 14:51 ` Sathyanarayanan Kuppuswamy
2022-06-27 18:51 ` Dave Hansen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=434ff0edcd5a0f1eb671bb2850ef5444ac1359a3.camel@intel.com \
--to=kai.huang@intel.com \
--cc=ak@linux.intel.com \
--cc=bp@alien8.de \
--cc=dave.hansen@linux.intel.com \
--cc=hpa@zytor.com \
--cc=isaku.yamahata@gmail.com \
--cc=jiewen.yao@intel.com \
--cc=jun.nakajima@intel.com \
--cc=khalid.elmously@canonical.com \
--cc=kirill.shutemov@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=marcelo.cerri@canonical.com \
--cc=mingo@redhat.com \
--cc=philip.cox@canonical.com \
--cc=sathyanarayanan.kuppuswamy@linux.intel.com \
--cc=tglx@linutronix.de \
--cc=tim.gardner@canonical.com \
--cc=tony.luck@intel.com \
--cc=wander@redhat.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).