linux-rdma.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Leon Romanovsky <leon@kernel.org>
To: Gal Pressman <galpress@amazon.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>,
	Doug Ledford <dledford@redhat.com>,
	linux-rdma@vger.kernel.org,
	Alexander Matushevsky <matua@amazon.com>,
	Firas JahJah <firasj@amazon.com>,
	Yossi Leybovich <sleybo@amazon.com>
Subject: Re: [PATCH for-next] RDMA/efa: Move provider specific attributes to ucontext allocation response
Date: Tue, 16 Jun 2020 09:30:45 +0300	[thread overview]
Message-ID: <20200616063045.GC2141420@unreal> (raw)
In-Reply-To: <20200615075920.58936-1-galpress@amazon.com>

On Mon, Jun 15, 2020 at 10:59:20AM +0300, Gal Pressman wrote:
> Provider specific attributes which are necessary for the userspace
> functionality should be part of the alloc ucontext response, not query
> device. This way a userspace provider could work without issuing a query
> device verb call. However, the fields will remain in the query device
> ABI in order to maintain backwards compatibility.

I don't really understand why "should be ..."? Device properties exposed
here are per-device and will be equal to all ucontexts, so instead of
doing one very fast system call, you are "punishing" every ucontext
call.

What is wrong with calling one query_device before allocating any
ucontext? What are you trying to achieve and what will it give?

Thanks

>
> Reviewed-by: Firas JahJah <firasj@amazon.com>
> Reviewed-by: Yossi Leybovich <sleybo@amazon.com>
> Signed-off-by: Gal Pressman <galpress@amazon.com>
> ---
> PR was sent:
> https://github.com/linux-rdma/rdma-core/pull/775
> ---
>  drivers/infiniband/hw/efa/efa_verbs.c | 10 ++++++++++
>  include/uapi/rdma/efa-abi.h           | 23 ++++++++++++++++++++++-
>  2 files changed, 32 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/infiniband/hw/efa/efa_verbs.c b/drivers/infiniband/hw/efa/efa_verbs.c
> index 08313f7c73bc..519cc959acfe 100644
> --- a/drivers/infiniband/hw/efa/efa_verbs.c
> +++ b/drivers/infiniband/hw/efa/efa_verbs.c
> @@ -1520,11 +1520,21 @@ int efa_alloc_ucontext(struct ib_ucontext *ibucontext, struct ib_udata *udata)
>
>  	ucontext->uarn = result.uarn;
>
> +	resp.comp_mask |= EFA_ALLOC_UCONTEXT_RESP_DEV_ATTR;
>  	resp.cmds_supp_udata_mask |= EFA_USER_CMDS_SUPP_UDATA_QUERY_DEVICE;
>  	resp.cmds_supp_udata_mask |= EFA_USER_CMDS_SUPP_UDATA_CREATE_AH;
>  	resp.sub_cqs_per_cq = dev->dev_attr.sub_cqs_per_cq;
>  	resp.inline_buf_size = dev->dev_attr.inline_buf_size;
>  	resp.max_llq_size = dev->dev_attr.max_llq_size;
> +	resp.max_sq_sge = dev->dev_attr.max_sq_sge;
> +	resp.max_rq_sge = dev->dev_attr.max_rq_sge;
> +	resp.max_sq_wr = dev->dev_attr.max_sq_depth;
> +	resp.max_rq_wr = dev->dev_attr.max_rq_depth;
> +	resp.max_rdma_size = dev->dev_attr.max_rdma_size;
> +	resp.max_wr_rdma_sge = dev->dev_attr.max_wr_rdma_sge;
> +
> +	if (is_rdma_read_cap(dev))
> +		resp.device_caps |= EFA_ALLOC_UCONTEXT_DEVICE_CAPS_RDMA_READ;
>
>  	if (udata && udata->outlen) {
>  		err = ib_copy_to_udata(udata, &resp,
> diff --git a/include/uapi/rdma/efa-abi.h b/include/uapi/rdma/efa-abi.h
> index 53b6e2036a9b..12df5c1659b6 100644
> --- a/include/uapi/rdma/efa-abi.h
> +++ b/include/uapi/rdma/efa-abi.h
> @@ -1,6 +1,6 @@
>  /* SPDX-License-Identifier: ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) */
>  /*
> - * Copyright 2018-2019 Amazon.com, Inc. or its affiliates. All rights reserved.
> + * Copyright 2018-2020 Amazon.com, Inc. or its affiliates. All rights reserved.
>   */
>
>  #ifndef EFA_ABI_USER_H
> @@ -20,6 +20,14 @@
>   * hex bit offset of the field.
>   */
>
> +enum {
> +	EFA_ALLOC_UCONTEXT_RESP_DEV_ATTR = 1 << 0,
> +};
> +
> +enum {
> +	EFA_ALLOC_UCONTEXT_DEVICE_CAPS_RDMA_READ = 1 << 0,
> +};
> +
>  enum efa_ibv_user_cmds_supp_udata {
>  	EFA_USER_CMDS_SUPP_UDATA_QUERY_DEVICE = 1 << 0,
>  	EFA_USER_CMDS_SUPP_UDATA_CREATE_AH    = 1 << 1,
> @@ -31,6 +39,14 @@ struct efa_ibv_alloc_ucontext_resp {
>  	__u16 sub_cqs_per_cq;
>  	__u16 inline_buf_size;
>  	__u32 max_llq_size; /* bytes */
> +	__u32 max_sq_wr;
> +	__u32 max_rq_wr;
> +	__u16 max_sq_sge;
> +	__u16 max_rq_sge;
> +	__u32 max_rdma_size;
> +	__u32 device_caps;
> +	__u16 max_wr_rdma_sge;
> +	__u8 reserved_130[2];
>  };
>
>  struct efa_ibv_alloc_pd_resp {
> @@ -96,6 +112,11 @@ enum {
>
>  struct efa_ibv_ex_query_device_resp {
>  	__u32 comp_mask;
> +	/*
> +	 * Attributes which are required for userspace provider functionality
> +	 * should be in alloc ucontext response, the following fields have been
> +	 * moved.
> +	 */
>  	__u32 max_sq_wr;
>  	__u32 max_rq_wr;
>  	__u16 max_sq_sge;
>
> base-commit: fba97dc7fc76b2c9a909fa0b3786d30a9899f5cf
> --
> 2.27.0
>

  reply	other threads:[~2020-06-16  8:09 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-06-15  7:59 Gal Pressman
2020-06-16  6:30 ` Leon Romanovsky [this message]
2020-06-16  8:53   ` Gal Pressman
2020-06-16  9:38     ` Leon Romanovsky
2020-06-16 17:44       ` Gal Pressman
2020-06-17  4:55         ` Leon Romanovsky
2020-06-17 15:36         ` Jason Gunthorpe
2020-06-17 17:49           ` Gal Pressman
2020-06-18 11:30             ` Gal Pressman
2020-06-25 10:53 ` Gal Pressman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200616063045.GC2141420@unreal \
    --to=leon@kernel.org \
    --cc=dledford@redhat.com \
    --cc=firasj@amazon.com \
    --cc=galpress@amazon.com \
    --cc=jgg@ziepe.ca \
    --cc=linux-rdma@vger.kernel.org \
    --cc=matua@amazon.com \
    --cc=sleybo@amazon.com \
    --subject='Re: [PATCH for-next] RDMA/efa: Move provider specific attributes to ucontext allocation response' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).