From: Frederic Barrat <fbarrat@linux.ibm.com>
To: "Alastair D'Silva" <alastair@au1.ibm.com>, linuxppc-dev@lists.ozlabs.org
Cc: linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org,
mikey@neuling.org, vaibhav@linux.vnet.ibm.com,
aneesh.kumar@linux.vnet.ibm.com, malat@debian.org,
felix@linux.vnet.ibm.com, pombredanne@nexb.com,
sukadev@linux.vnet.ibm.com, npiggin@gmail.com,
gregkh@linuxfoundation.org, arnd@arndb.de,
andrew.donnellan@au1.ibm.com, fbarrat@linux.vnet.ibm.com,
corbet@lwn.net, "Alastair D'Silva" <alastair@d-silva.org>
Subject: Re: [PATCH v5 5/7] ocxl: Expose the thread_id needed for wait on POWER9
Date: Fri, 11 May 2018 11:25:23 +0200 [thread overview]
Message-ID: <9656b063-eafd-a882-c678-a1c4d504cd54@linux.ibm.com> (raw)
In-Reply-To: <20180511061303.10728-6-alastair@au1.ibm.com>
Le 11/05/2018 à 08:13, Alastair D'Silva a écrit :
> From: Alastair D'Silva <alastair@d-silva.org>
>
> In order to successfully issue as_notify, an AFU needs to know the TID
> to notify, which in turn means that this information should be
> available in userspace so it can be communicated to the AFU.
>
> Signed-off-by: Alastair D'Silva <alastair@d-silva.org>
> ---
Ok, so we keep the limitation of having only one thread per context able
to call 'wait', even though we don't have to worry about depleting the
pool of TIDs any more. I think that's acceptable, though we don't really
have a reason to justify it any more. Any reason you want to keep it
that way?
Fred
> drivers/misc/ocxl/context.c | 5 ++-
> drivers/misc/ocxl/file.c | 53 +++++++++++++++++++++++++++++++
> drivers/misc/ocxl/link.c | 36 +++++++++++++++++++++
> drivers/misc/ocxl/ocxl_internal.h | 1 +
> include/misc/ocxl.h | 9 ++++++
> include/uapi/misc/ocxl.h | 8 +++++
> 6 files changed, 111 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/misc/ocxl/context.c b/drivers/misc/ocxl/context.c
> index 909e8807824a..95f74623113e 100644
> --- a/drivers/misc/ocxl/context.c
> +++ b/drivers/misc/ocxl/context.c
> @@ -34,6 +34,8 @@ int ocxl_context_init(struct ocxl_context *ctx, struct ocxl_afu *afu,
> mutex_init(&ctx->xsl_error_lock);
> mutex_init(&ctx->irq_lock);
> idr_init(&ctx->irq_idr);
> + ctx->tidr = 0;
> +
> /*
> * Keep a reference on the AFU to make sure it's valid for the
> * duration of the life of the context
> @@ -65,6 +67,7 @@ int ocxl_context_attach(struct ocxl_context *ctx, u64 amr)
> {
> int rc;
>
> + // Locks both status & tidr
> mutex_lock(&ctx->status_mutex);
> if (ctx->status != OPENED) {
> rc = -EIO;
> @@ -72,7 +75,7 @@ int ocxl_context_attach(struct ocxl_context *ctx, u64 amr)
> }
>
> rc = ocxl_link_add_pe(ctx->afu->fn->link, ctx->pasid,
> - current->mm->context.id, 0, amr, current->mm,
> + current->mm->context.id, ctx->tidr, amr, current->mm,
> xsl_fault_error, ctx);
> if (rc)
> goto out;
> diff --git a/drivers/misc/ocxl/file.c b/drivers/misc/ocxl/file.c
> index 038509e5d031..eb409a469f21 100644
> --- a/drivers/misc/ocxl/file.c
> +++ b/drivers/misc/ocxl/file.c
> @@ -5,6 +5,8 @@
> #include <linux/sched/signal.h>
> #include <linux/uaccess.h>
> #include <uapi/misc/ocxl.h>
> +#include <asm/reg.h>
> +#include <asm/switch_to.h>
> #include "ocxl_internal.h"
>
>
> @@ -123,11 +125,55 @@ static long afu_ioctl_get_metadata(struct ocxl_context *ctx,
> return 0;
> }
>
> +#ifdef CONFIG_PPC64
> +static long afu_ioctl_enable_p9_wait(struct ocxl_context *ctx,
> + struct ocxl_ioctl_p9_wait __user *uarg)
> +{
> + struct ocxl_ioctl_p9_wait arg;
> +
> + memset(&arg, 0, sizeof(arg));
> +
> + if (cpu_has_feature(CPU_FTR_P9_TIDR)) {
> + enum ocxl_context_status status;
> +
> + // Locks both status & tidr
> + mutex_lock(&ctx->status_mutex);
> + if (!ctx->tidr) {
> + if (set_thread_tidr(current))
> + return -ENOENT;
> +
> + ctx->tidr = current->thread.tidr;
> + }
> +
> + status = ctx->status;
> + mutex_unlock(&ctx->status_mutex);
> +
> + if (status == ATTACHED) {
> + int rc;
> + struct link *link = ctx->afu->fn->link;
> +
> + rc = ocxl_link_update_pe(link, ctx->pasid, ctx->tidr);
> + if (rc)
> + return rc;
> + }
> +
> + arg.thread_id = ctx->tidr;
> + } else
> + return -ENOENT;
> +
> + if (copy_to_user(uarg, &arg, sizeof(arg)))
> + return -EFAULT;
> +
> + return 0;
> +}
> +#endif
> +
> #define CMD_STR(x) (x == OCXL_IOCTL_ATTACH ? "ATTACH" : \
> x == OCXL_IOCTL_IRQ_ALLOC ? "IRQ_ALLOC" : \
> x == OCXL_IOCTL_IRQ_FREE ? "IRQ_FREE" : \
> x == OCXL_IOCTL_IRQ_SET_FD ? "IRQ_SET_FD" : \
> x == OCXL_IOCTL_GET_METADATA ? "GET_METADATA" : \
> + x == OCXL_IOCTL_ENABLE_P9_WAIT ? "ENABLE_P9_WAIT" : \
> "UNKNOWN")
>
> static long afu_ioctl(struct file *file, unsigned int cmd,
> @@ -186,6 +232,13 @@ static long afu_ioctl(struct file *file, unsigned int cmd,
> (struct ocxl_ioctl_metadata __user *) args);
> break;
>
> +#ifdef CONFIG_PPC64
> + case OCXL_IOCTL_ENABLE_P9_WAIT:
> + rc = afu_ioctl_enable_p9_wait(ctx,
> + (struct ocxl_ioctl_p9_wait __user *) args);
> + break;
> +#endif
> +
> default:
> rc = -EINVAL;
> }
> diff --git a/drivers/misc/ocxl/link.c b/drivers/misc/ocxl/link.c
> index 656e8610eec2..88876ae8f330 100644
> --- a/drivers/misc/ocxl/link.c
> +++ b/drivers/misc/ocxl/link.c
> @@ -544,6 +544,42 @@ int ocxl_link_add_pe(void *link_handle, int pasid, u32 pidr, u32 tidr,
> }
> EXPORT_SYMBOL_GPL(ocxl_link_add_pe);
>
> +int ocxl_link_update_pe(void *link_handle, int pasid, __u16 tid)
> +{
> + struct link *link = (struct link *) link_handle;
> + struct spa *spa = link->spa;
> + struct ocxl_process_element *pe;
> + int pe_handle, rc;
> +
> + if (pasid > SPA_PASID_MAX)
> + return -EINVAL;
> +
> + pe_handle = pasid & SPA_PE_MASK;
> + pe = spa->spa_mem + pe_handle;
> +
> + mutex_lock(&spa->spa_lock);
> +
> + pe->tid = tid;
> +
> + /*
> + * The barrier makes sure the PE is updated
> + * before we clear the NPU context cache below, so that the
> + * old PE cannot be reloaded erroneously.
> + */
> + mb();
> +
> + /*
> + * hook to platform code
> + * On powerpc, the entry needs to be cleared from the context
> + * cache of the NPU.
> + */
> + rc = pnv_ocxl_spa_remove_pe_from_cache(link->platform_data, pe_handle);
> + WARN_ON(rc);
> +
> + mutex_unlock(&spa->spa_lock);
> + return rc;
> +}
> +
> int ocxl_link_remove_pe(void *link_handle, int pasid)
> {
> struct link *link = (struct link *) link_handle;
> diff --git a/drivers/misc/ocxl/ocxl_internal.h b/drivers/misc/ocxl/ocxl_internal.h
> index 5d421824afd9..a32f2151029f 100644
> --- a/drivers/misc/ocxl/ocxl_internal.h
> +++ b/drivers/misc/ocxl/ocxl_internal.h
> @@ -77,6 +77,7 @@ struct ocxl_context {
> struct ocxl_xsl_error xsl_error;
> struct mutex irq_lock;
> struct idr irq_idr;
> + u16 tidr; // Thread ID used for P9 wait implementation
> };
>
> struct ocxl_process_element {
> diff --git a/include/misc/ocxl.h b/include/misc/ocxl.h
> index 51ccf76db293..9ff6ddc28e22 100644
> --- a/include/misc/ocxl.h
> +++ b/include/misc/ocxl.h
> @@ -188,6 +188,15 @@ extern int ocxl_link_add_pe(void *link_handle, int pasid, u32 pidr, u32 tidr,
> void (*xsl_err_cb)(void *data, u64 addr, u64 dsisr),
> void *xsl_err_data);
>
> +/**
> + * Update values within a Process Element
> + *
> + * link_handle: the link handle associated with the process element
> + * pasid: the PASID for the AFU context
> + * tid: the new thread id for the process element
> + */
> +extern int ocxl_link_update_pe(void *link_handle, int pasid, __u16 tid);
> +
> /*
> * Remove a Process Element from the Shared Process Area for a link
> */
> diff --git a/include/uapi/misc/ocxl.h b/include/uapi/misc/ocxl.h
> index 0af83d80fb3e..561e6f0dfcb7 100644
> --- a/include/uapi/misc/ocxl.h
> +++ b/include/uapi/misc/ocxl.h
> @@ -48,6 +48,13 @@ struct ocxl_ioctl_metadata {
> __u64 reserved[13]; // Total of 16*u64
> };
>
> +struct ocxl_ioctl_p9_wait {
> + __u16 thread_id; // The thread ID required to wake this thread
> + __u16 reserved1;
> + __u32 reserved2;
> + __u64 reserved3[3];
> +};
> +
> struct ocxl_ioctl_irq_fd {
> __u64 irq_offset;
> __s32 eventfd;
> @@ -62,5 +69,6 @@ struct ocxl_ioctl_irq_fd {
> #define OCXL_IOCTL_IRQ_FREE _IOW(OCXL_MAGIC, 0x12, __u64)
> #define OCXL_IOCTL_IRQ_SET_FD _IOW(OCXL_MAGIC, 0x13, struct ocxl_ioctl_irq_fd)
> #define OCXL_IOCTL_GET_METADATA _IOR(OCXL_MAGIC, 0x14, struct ocxl_ioctl_metadata)
> +#define OCXL_IOCTL_ENABLE_P9_WAIT _IOR(OCXL_MAGIC, 0x15, struct ocxl_ioctl_p9_wait)
>
> #endif /* _UAPI_MISC_OCXL_H */
>
next prev parent reply other threads:[~2018-05-11 9:25 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-05-11 6:12 [PATCH v5 0/7] ocxl: Implement Power9 as_notify/wait for OpenCAPI Alastair D'Silva
2018-05-11 6:12 ` [PATCH v5 1/7] powerpc: Add TIDR CPU feature for POWER9 Alastair D'Silva
2018-05-11 8:38 ` Frederic Barrat
2018-05-31 4:19 ` Andrew Donnellan
2018-06-04 14:10 ` [v5,1/7] " Michael Ellerman
2018-05-11 6:12 ` [PATCH v5 2/7] powerpc: Use TIDR CPU feature to control TIDR allocation Alastair D'Silva
2018-05-11 8:46 ` Frederic Barrat
2018-05-31 4:20 ` Andrew Donnellan
2018-05-11 6:12 ` [PATCH v5 3/7] powerpc: use task_pid_nr() for TID allocation Alastair D'Silva
2018-05-11 8:48 ` Frederic Barrat
2018-05-31 4:29 ` Andrew Donnellan
2018-05-11 6:13 ` [PATCH v5 4/7] ocxl: Rename pnv_ocxl_spa_remove_pe to clarify it's action Alastair D'Silva
2018-05-11 8:49 ` Frederic Barrat
2018-05-31 4:32 ` Andrew Donnellan
2018-05-11 6:13 ` [PATCH v5 5/7] ocxl: Expose the thread_id needed for wait on POWER9 Alastair D'Silva
2018-05-11 9:25 ` Frederic Barrat [this message]
2018-05-11 10:06 ` Alastair D'Silva
2018-05-11 11:03 ` Frederic Barrat
2018-05-31 4:42 ` Andrew Donnellan
2018-05-11 6:13 ` [PATCH v5 6/7] ocxl: Add an IOCTL so userspace knows what OCXL features are available Alastair D'Silva
2018-05-11 9:27 ` Frederic Barrat
2018-05-31 4:46 ` Andrew Donnellan
2018-05-11 6:13 ` [PATCH v5 7/7] ocxl: Document new OCXL IOCTLs Alastair D'Silva
2018-05-11 9:28 ` Frederic Barrat
2018-05-31 4:48 ` Andrew Donnellan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9656b063-eafd-a882-c678-a1c4d504cd54@linux.ibm.com \
--to=fbarrat@linux.ibm.com \
--cc=alastair@au1.ibm.com \
--cc=alastair@d-silva.org \
--cc=andrew.donnellan@au1.ibm.com \
--cc=aneesh.kumar@linux.vnet.ibm.com \
--cc=arnd@arndb.de \
--cc=corbet@lwn.net \
--cc=fbarrat@linux.vnet.ibm.com \
--cc=felix@linux.vnet.ibm.com \
--cc=gregkh@linuxfoundation.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=malat@debian.org \
--cc=mikey@neuling.org \
--cc=npiggin@gmail.com \
--cc=pombredanne@nexb.com \
--cc=sukadev@linux.vnet.ibm.com \
--cc=vaibhav@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).