Re: [PATCH V4 linux-next 00/12] VDPA support for Mellanox ConnectX devices

Re: [PATCH V4 linux-next 00/12] VDPA support for Mellanox ConnectX devices

[Michael S. Tsirkin]

On Tue, Aug 04, 2020 at 07:20:36PM +0300, Eli Cohen wrote:
> Hi Michael,
> please note that this series depends on mlx5 core device driver patches
> in mlx5-next branch in
> git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git.

Thanks! OK so what's the plan for merging this?
Do patches at least build well enough that I can push them
upstream? Or do they have to go on top of the mellanox tree?


> git pull git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git mlx5-next 
> 
> They also depend Jason Wang's patches: https://lkml.org/lkml/2020/7/1/301

The ones you included, right?

> Jason, I had to resolve some conflicts so I would appreciate of you can verify
> that it is ok.
> 
> The following series of patches provide VDPA support for Mellanox
> devices. The supported devices are ConnectX6 DX and newer.
> 
> Currently, only a network driver is implemented; future patches will
> introduce a block device driver. iperf performance on a single queue is
> around 12 Gbps.  Future patches will introduce multi queue support.
> 
> The files are organized in such a way that code that can be used by
> different VDPA implementations will be placed in a common are resides in
> drivers/vdpa/mlx5/core.
> 
> Only virtual functions are currently supported. Also, certain firmware
> capabilities must be set to enable the driver. Physical functions (PFs)
> are skipped by the driver.
> 
> To make use of the VDPA net driver, one must load mlx5_vdpa. In such
> case, VFs will be operated by the VDPA driver. Although one can see a
> regular instance of a network driver on the VF, the VDPA driver takes
> precedence over the NIC driver, steering-wize.
> 
> Currently, the device/interface infrastructure in mlx5_core is used to
> probe drivers. Future patches will introduce virtbus as a means to
> register devices and drivers and VDPA will be adapted to it.
> 
> The mlx5 mode of operation required to support VDPA is switchdev mode.
> Once can use Linux or OVS bridge to take care of layer 2 switching.
> 
> In order to provide virtio networking to a guest, an updated version of
> qemu is required. This version has been tested by the following quemu
> version:
> 
> url: https://github.com/jasowang/qemu.git
> branch: vdpa
> Commit ID: 6f4e59b807db
> 
> 
> V2->V3
> Fix makefile to use include path relative to the root of the kernel
> 
> V3-V4
> Rebase Jason's patches on linux-next branch
> Fix krobot error on mips arch
> Make use of the free callback to destroy resoruces on unload
> Use VIRTIO_F_ACCESS_PLATFORM instead of legacy VIRTIO_F_IOMMU_PLATFORM
> Add empty implementations for get_vq_notification() and get_vq_irq()
> 
> 
> Eli Cohen (6):
>   net/vdpa: Use struct for set/get vq state
>   vdpa: Modify get_vq_state() to return error code
>   vdpa/mlx5: Add hardware descriptive header file
>   vdpa/mlx5: Add support library for mlx5 VDPA implementation
>   vdpa/mlx5: Add shared memory registration code
>   vdpa/mlx5: Add VDPA driver for supported mlx5 devices
> 
> Jason Wang (5):
>   vhost-vdpa: refine ioctl pre-processing
>   vhost: generialize backend features setting/getting
>   vhost-vdpa: support get/set backend features
>   vhost-vdpa: support IOTLB batching hints
>   vdpasim: support batch updating
> 
> Max Gurtovoy (1):
>   vdpa: remove hard coded virtq num
> 
>  drivers/vdpa/Kconfig                   |   19 +
>  drivers/vdpa/Makefile                  |    1 +
>  drivers/vdpa/ifcvf/ifcvf_base.c        |    4 +-
>  drivers/vdpa/ifcvf/ifcvf_base.h        |    4 +-
>  drivers/vdpa/ifcvf/ifcvf_main.c        |   13 +-
>  drivers/vdpa/mlx5/Makefile             |    4 +
>  drivers/vdpa/mlx5/core/mlx5_vdpa.h     |   91 ++
>  drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h |  168 ++
>  drivers/vdpa/mlx5/core/mr.c            |  484 ++++++
>  drivers/vdpa/mlx5/core/resources.c     |  284 ++++
>  drivers/vdpa/mlx5/net/main.c           |   76 +
>  drivers/vdpa/mlx5/net/mlx5_vnet.c      | 1965 ++++++++++++++++++++++++
>  drivers/vdpa/mlx5/net/mlx5_vnet.h      |   24 +
>  drivers/vdpa/vdpa.c                    |    3 +
>  drivers/vdpa/vdpa_sim/vdpa_sim.c       |   53 +-
>  drivers/vhost/net.c                    |   18 +-
>  drivers/vhost/vdpa.c                   |   76 +-
>  drivers/vhost/vhost.c                  |   15 +
>  drivers/vhost/vhost.h                  |    2 +
>  include/linux/vdpa.h                   |   24 +-
>  include/uapi/linux/vhost.h             |    2 +
>  include/uapi/linux/vhost_types.h       |   11 +
>  22 files changed, 3284 insertions(+), 57 deletions(-)
>  create mode 100644 drivers/vdpa/mlx5/Makefile
>  create mode 100644 drivers/vdpa/mlx5/core/mlx5_vdpa.h
>  create mode 100644 drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h
>  create mode 100644 drivers/vdpa/mlx5/core/mr.c
>  create mode 100644 drivers/vdpa/mlx5/core/resources.c
>  create mode 100644 drivers/vdpa/mlx5/net/main.c
>  create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.c
>  create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.h
> 
> -- 
> 2.26.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH V4 linux-next 00/12] VDPA support for Mellanox ConnectX devices

[Jason Wang]

On 2020/8/5 下午1:01, Eli Cohen wrote:
> On Tue, Aug 04, 2020 at 05:29:09PM -0400, Michael S. Tsirkin wrote:
>> On Tue, Aug 04, 2020 at 07:20:36PM +0300, Eli Cohen wrote:
>>> Hi Michael,
>>> please note that this series depends on mlx5 core device driver patches
>>> in mlx5-next branch in
>>> git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git.
>> Thanks! OK so what's the plan for merging this?
>> Do patches at least build well enough that I can push them
>> upstream? Or do they have to go on top of the mellanox tree?
>>
> The patches are built on your linux-next branch which I updated
> yesterday.
>
> I am based on this commit:
> 776b7b25f10b (origin/linux-next) vhost: add an RPMsg API
>
> On top of that I merged
> git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git
>
> and after that I have Jason's patches (five patches), than one patch
> from Max and then my patches (seven).
>
> It builds fine on x84_64.
> I fixed some conflicts on Jason's patches.
>
> I also tested it to verify it's working.
>
> BTW, for some reason I did not get all the patches into my mailbox and I
> suspect they were not all sent. Did you get all the series 0-13?


I can see patch 0 to patch 12 but not patch 13 (I guess 12 is all).

Thanks


>
> Please let me know, and if needed I'll resend.
>
>>> git pull git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git mlx5-next
>>>
>>> They also depend Jason Wang's patches: https://lkml.org/lkml/2020/7/1/301
>> The ones you included, right?
>>
> Right.
>   
>>> Jason, I had to resolve some conflicts so I would appreciate of you can verify
>>> that it is ok.
>>>
>>> The following series of patches provide VDPA support for Mellanox
>>> devices. The supported devices are ConnectX6 DX and newer.
>>>
>>> Currently, only a network driver is implemented; future patches will
>>> introduce a block device driver. iperf performance on a single queue is
>>> around 12 Gbps.  Future patches will introduce multi queue support.
>>>
>>> The files are organized in such a way that code that can be used by
>>> different VDPA implementations will be placed in a common are resides in
>>> drivers/vdpa/mlx5/core.
>>>
>>> Only virtual functions are currently supported. Also, certain firmware
>>> capabilities must be set to enable the driver. Physical functions (PFs)
>>> are skipped by the driver.
>>>
>>> To make use of the VDPA net driver, one must load mlx5_vdpa. In such
>>> case, VFs will be operated by the VDPA driver. Although one can see a
>>> regular instance of a network driver on the VF, the VDPA driver takes
>>> precedence over the NIC driver, steering-wize.
>>>
>>> Currently, the device/interface infrastructure in mlx5_core is used to
>>> probe drivers. Future patches will introduce virtbus as a means to
>>> register devices and drivers and VDPA will be adapted to it.
>>>
>>> The mlx5 mode of operation required to support VDPA is switchdev mode.
>>> Once can use Linux or OVS bridge to take care of layer 2 switching.
>>>
>>> In order to provide virtio networking to a guest, an updated version of
>>> qemu is required. This version has been tested by the following quemu
>>> version:
>>>
>>> url: https://github.com/jasowang/qemu.git
>>> branch: vdpa
>>> Commit ID: 6f4e59b807db
>>>
>>>
>>> V2->V3
>>> Fix makefile to use include path relative to the root of the kernel
>>>
>>> V3-V4
>>> Rebase Jason's patches on linux-next branch
>>> Fix krobot error on mips arch
>>> Make use of the free callback to destroy resoruces on unload
>>> Use VIRTIO_F_ACCESS_PLATFORM instead of legacy VIRTIO_F_IOMMU_PLATFORM
>>> Add empty implementations for get_vq_notification() and get_vq_irq()
>>>
>>>
>>> Eli Cohen (6):
>>>    net/vdpa: Use struct for set/get vq state
>>>    vdpa: Modify get_vq_state() to return error code
>>>    vdpa/mlx5: Add hardware descriptive header file
>>>    vdpa/mlx5: Add support library for mlx5 VDPA implementation
>>>    vdpa/mlx5: Add shared memory registration code
>>>    vdpa/mlx5: Add VDPA driver for supported mlx5 devices
>>>
>>> Jason Wang (5):
>>>    vhost-vdpa: refine ioctl pre-processing
>>>    vhost: generialize backend features setting/getting
>>>    vhost-vdpa: support get/set backend features
>>>    vhost-vdpa: support IOTLB batching hints
>>>    vdpasim: support batch updating
>>>
>>> Max Gurtovoy (1):
>>>    vdpa: remove hard coded virtq num
>>>
>>>   drivers/vdpa/Kconfig                   |   19 +
>>>   drivers/vdpa/Makefile                  |    1 +
>>>   drivers/vdpa/ifcvf/ifcvf_base.c        |    4 +-
>>>   drivers/vdpa/ifcvf/ifcvf_base.h        |    4 +-
>>>   drivers/vdpa/ifcvf/ifcvf_main.c        |   13 +-
>>>   drivers/vdpa/mlx5/Makefile             |    4 +
>>>   drivers/vdpa/mlx5/core/mlx5_vdpa.h     |   91 ++
>>>   drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h |  168 ++
>>>   drivers/vdpa/mlx5/core/mr.c            |  484 ++++++
>>>   drivers/vdpa/mlx5/core/resources.c     |  284 ++++
>>>   drivers/vdpa/mlx5/net/main.c           |   76 +
>>>   drivers/vdpa/mlx5/net/mlx5_vnet.c      | 1965 ++++++++++++++++++++++++
>>>   drivers/vdpa/mlx5/net/mlx5_vnet.h      |   24 +
>>>   drivers/vdpa/vdpa.c                    |    3 +
>>>   drivers/vdpa/vdpa_sim/vdpa_sim.c       |   53 +-
>>>   drivers/vhost/net.c                    |   18 +-
>>>   drivers/vhost/vdpa.c                   |   76 +-
>>>   drivers/vhost/vhost.c                  |   15 +
>>>   drivers/vhost/vhost.h                  |    2 +
>>>   include/linux/vdpa.h                   |   24 +-
>>>   include/uapi/linux/vhost.h             |    2 +
>>>   include/uapi/linux/vhost_types.h       |   11 +
>>>   22 files changed, 3284 insertions(+), 57 deletions(-)
>>>   create mode 100644 drivers/vdpa/mlx5/Makefile
>>>   create mode 100644 drivers/vdpa/mlx5/core/mlx5_vdpa.h
>>>   create mode 100644 drivers/vdpa/mlx5/core/mlx5_vdpa_ifc.h
>>>   create mode 100644 drivers/vdpa/mlx5/core/mr.c
>>>   create mode 100644 drivers/vdpa/mlx5/core/resources.c
>>>   create mode 100644 drivers/vdpa/mlx5/net/main.c
>>>   create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.c
>>>   create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.h
>>>
>>> -- 
>>> 2.26.0

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH V4 linux-next 12/12] vdpa/mlx5: Add VDPA driver for supported mlx5 devices

[Jason Wang]

On 2020/8/5 上午12:20, Eli Cohen wrote:
> Add a front end VDPA driver that registers in the VDPA bus and provides
> networking to a guest. The VDPA driver creates the necessary resources
> on the VF it is driving such that data path will be offloaded.
>
> Notifications are being communicated through the driver.
>
> Currently, only VFs are supported. In subsequent patches we will have
> devlink support to control which VF is used for VDPA and which function
> is used for regular networking.
>
> Reviewed-by: Parav Pandit<parav@mellanox.com>
> Signed-off-by: Eli Cohen<eli@mellanox.com>
> ---


Acked-by: Jason Wang <jasowang@redhat.com>


_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH V4 linux-next 08/12] vdpa: Modify get_vq_state() to return error code

[Jason Wang]

On 2020/8/5 上午12:20, Eli Cohen wrote:
> Modify get_vq_state() so it returns an error code. In case of hardware
> acceleration, the available index may be retrieved from the device, an
> operation that can possibly fail.
>
> Reviewed-by: Parav Pandit <parav@mellanox.com>
> Signed-off-by: Eli Cohen <eli@mellanox.com>


Acked-by: Jason Wang <jasowang@redhat.com>


> ---
>   drivers/vdpa/ifcvf/ifcvf_main.c  | 5 +++--
>   drivers/vdpa/vdpa_sim/vdpa_sim.c | 5 +++--
>   drivers/vhost/vdpa.c             | 5 ++++-
>   include/linux/vdpa.h             | 4 ++--
>   4 files changed, 12 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/vdpa/ifcvf/ifcvf_main.c b/drivers/vdpa/ifcvf/ifcvf_main.c
> index dc311e972b9e..076d7ac5e723 100644
> --- a/drivers/vdpa/ifcvf/ifcvf_main.c
> +++ b/drivers/vdpa/ifcvf/ifcvf_main.c
> @@ -237,12 +237,13 @@ static u16 ifcvf_vdpa_get_vq_num_max(struct vdpa_device *vdpa_dev)
>   	return IFCVF_QUEUE_MAX;
>   }
>   
> -static void ifcvf_vdpa_get_vq_state(struct vdpa_device *vdpa_dev, u16 qid,
> -				    struct vdpa_vq_state *state)
> +static int ifcvf_vdpa_get_vq_state(struct vdpa_device *vdpa_dev, u16 qid,
> +				   struct vdpa_vq_state *state)
>   {
>   	struct ifcvf_hw *vf = vdpa_to_vf(vdpa_dev);
>   
>   	state->avail_index = ifcvf_get_vq_state(vf, qid);
> +	return 0;
>   }
>   
>   static int ifcvf_vdpa_set_vq_state(struct vdpa_device *vdpa_dev, u16 qid,
> diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
> index f1c351d5959b..c68741363643 100644
> --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
> +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
> @@ -464,14 +464,15 @@ static int vdpasim_set_vq_state(struct vdpa_device *vdpa, u16 idx,
>   	return 0;
>   }
>   
> -static void vdpasim_get_vq_state(struct vdpa_device *vdpa, u16 idx,
> -				 struct vdpa_vq_state *state)
> +static int vdpasim_get_vq_state(struct vdpa_device *vdpa, u16 idx,
> +				struct vdpa_vq_state *state)
>   {
>   	struct vdpasim *vdpasim = vdpa_to_sim(vdpa);
>   	struct vdpasim_virtqueue *vq = &vdpasim->vqs[idx];
>   	struct vringh *vrh = &vq->vring;
>   
>   	state->avail_index = vrh->last_avail_idx;
> +	return 0;
>   }
>   
>   static u32 vdpasim_get_vq_align(struct vdpa_device *vdpa)
> diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
> index c2e1e2d55084..a0b7c91948e1 100644
> --- a/drivers/vhost/vdpa.c
> +++ b/drivers/vhost/vdpa.c
> @@ -392,7 +392,10 @@ static long vhost_vdpa_vring_ioctl(struct vhost_vdpa *v, unsigned int cmd,
>   		ops->set_vq_ready(vdpa, idx, s.num);
>   		return 0;
>   	case VHOST_GET_VRING_BASE:
> -		ops->get_vq_state(v->vdpa, idx, &vq_state);
> +		r = ops->get_vq_state(v->vdpa, idx, &vq_state);
> +		if (r)
> +			return r;
> +
>   		vq->last_avail_idx = vq_state.avail_index;
>   		break;
>   	case VHOST_GET_BACKEND_FEATURES:
> diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h
> index 8f412620071d..fd6e560d70f9 100644
> --- a/include/linux/vdpa.h
> +++ b/include/linux/vdpa.h
> @@ -192,8 +192,8 @@ struct vdpa_config_ops {
>   	bool (*get_vq_ready)(struct vdpa_device *vdev, u16 idx);
>   	int (*set_vq_state)(struct vdpa_device *vdev, u16 idx,
>   			    const struct vdpa_vq_state *state);
> -	void (*get_vq_state)(struct vdpa_device *vdev, u16 idx,
> -			     struct vdpa_vq_state *state);
> +	int (*get_vq_state)(struct vdpa_device *vdev, u16 idx,
> +			    struct vdpa_vq_state *state);
>   	struct vdpa_notification_area
>   	(*get_vq_notification)(struct vdpa_device *vdev, u16 idx);
>   	int (*get_vq_irq)(struct vdpa_device *vdv, u16 idx);

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH V4 linux-next 00/12] VDPA support for Mellanox ConnectX devices

[Jason Wang]

On 2020/8/5 上午12:20, Eli Cohen wrote:
> Hi Michael,
> please note that this series depends on mlx5 core device driver patches
> in mlx5-next branch in
> git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git.
>
> git pull git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git mlx5-next
>
> They also depend Jason Wang's patches:https://lkml.org/lkml/2020/7/1/301
>
> Jason, I had to resolve some conflicts so I would appreciate of you can verify
> that it is ok.


Looks good to me.

Thanks

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH V4 linux-next 00/12] VDPA support for Mellanox ConnectX devices

[Michael S. Tsirkin]

On Tue, Aug 04, 2020 at 07:20:36PM +0300, Eli Cohen wrote:
> Hi Michael,
> please note that this series depends on mlx5 core device driver patches
> in mlx5-next branch in
> git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git.
> 
> git pull git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git mlx5-next 
> 
> They also depend Jason Wang's patches: https://lkml.org/lkml/2020/7/1/301

So if I apply this to linux-next branch of my tree, I get:


  CALL    scripts/checksyscalls.sh
  CALL    scripts/atomic/check-atomics.sh
  DESCEND  objtool
  CHK     include/generated/compile.h
  CC [M]  drivers/vdpa/mlx5/net/main.o
In file included from ./include/linux/swab.h:5,
                 from ./include/uapi/linux/byteorder/little_endian.h:13,
                 from ./include/linux/byteorder/little_endian.h:5,
                 from ./arch/x86/include/uapi/asm/byteorder.h:5,
                 from ./include/asm-generic/bitops/le.h:6,
                 from ./arch/x86/include/asm/bitops.h:395,
                 from ./include/linux/bitops.h:29,
                 from ./include/linux/kernel.h:12,
                 from ./include/linux/list.h:9,
                 from ./include/linux/module.h:12,
                 from drivers/vdpa/mlx5/net/main.c:4:
drivers/vdpa/mlx5/net/main.c: In function ‘required_caps_supported’:
././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’
  129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b)
      |                                   ^~~~~~~~~~~~~~~~~~
./include/uapi/linux/swab.h:115:54: note: in definition of macro ‘__swab32’
  115 | #define __swab32(x) (__u32)__builtin_bswap32((__u32)(x))
      |                                                      ^
./include/linux/byteorder/generic.h:95:21: note: in expansion of macro ‘__be32_to_cpu’
   95 | #define be32_to_cpu __be32_to_cpu
      |                     ^~~~~~~~~~~~~
./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’
   17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER)
      |                                ^~~~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
   51 | #define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
      |                                   ^~~~~~~~
./include/linux/mlx5/device.h:53:34: note: in expansion of macro ‘__mlx5_bit_off’
   53 | #define __mlx5_dw_off(typ, fld) (__mlx5_bit_off(typ, fld) / 32)
      |                                  ^~~~~~~~~~~~~~
./include/linux/mlx5/device.h:96:1: note: in expansion of macro ‘__mlx5_dw_off’
   96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
      | ^~~~~~~~~~~~~
./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
 1355 |  MLX5_GET(device_virtio_emulation_cap, \
      |  ^~~~~~~~
drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
   24 |  event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode);
      |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ./include/linux/mlx5/driver.h:52,
                 from drivers/vdpa/mlx5/net/main.c:5:
./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’
   50 | #define __mlx5_bit_sz(typ, fld) sizeof(__mlx5_nullp(typ)->fld)
      |                                                         ^~
./include/linux/mlx5/device.h:56:43: note: in expansion of macro ‘__mlx5_bit_sz’
   56 | #define __mlx5_dw_bit_off(typ, fld) (32 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0x1f))
      |                                           ^~~~~~~~~~~~~
./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’
   96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
      |                              ^~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
 1355 |  MLX5_GET(device_virtio_emulation_cap, \
      |  ^~~~~~~~
drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
   24 |  event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode);
      |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from <command-line>:
././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’
  129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b)
      |                                   ^~~~~~~~~~~~~~~~~~
./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’
   17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER)
      |                                ^~~~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
   51 | #define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
      |                                   ^~~~~~~~
./include/linux/mlx5/device.h:56:70: note: in expansion of macro ‘__mlx5_bit_off’
   56 | #define __mlx5_dw_bit_off(typ, fld) (32 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0x1f))
      |                                                                      ^~~~~~~~~~~~~~
./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’
   96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
      |                              ^~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
 1355 |  MLX5_GET(device_virtio_emulation_cap, \
      |  ^~~~~~~~
drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
   24 |  event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode);
      |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ./include/linux/mlx5/driver.h:52,
                 from drivers/vdpa/mlx5/net/main.c:5:
./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’
   50 | #define __mlx5_bit_sz(typ, fld) sizeof(__mlx5_nullp(typ)->fld)
      |                                                         ^~
./include/linux/mlx5/device.h:57:47: note: in expansion of macro ‘__mlx5_bit_sz’
   57 | #define __mlx5_mask(typ, fld) ((u32)((1ull << __mlx5_bit_sz(typ, fld)) - 1))
      |                                               ^~~~~~~~~~~~~
./include/linux/mlx5/device.h:97:1: note: in expansion of macro ‘__mlx5_mask’
   97 | __mlx5_mask(typ, fld))
      | ^~~~~~~~~~~
./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
 1355 |  MLX5_GET(device_virtio_emulation_cap, \
      |  ^~~~~~~~
drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
   24 |  event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode);
      |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ./include/linux/swab.h:5,
                 from ./include/uapi/linux/byteorder/little_endian.h:13,
                 from ./include/linux/byteorder/little_endian.h:5,
                 from ./arch/x86/include/uapi/asm/byteorder.h:5,
                 from ./include/asm-generic/bitops/le.h:6,
                 from ./arch/x86/include/asm/bitops.h:395,
                 from ./include/linux/bitops.h:29,
                 from ./include/linux/kernel.h:12,
                 from ./include/linux/list.h:9,
                 from ./include/linux/module.h:12,
                 from drivers/vdpa/mlx5/net/main.c:4:
././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’
  129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b)
      |                                   ^~~~~~~~~~~~~~~~~~
./include/uapi/linux/swab.h:115:54: note: in definition of macro ‘__swab32’
  115 | #define __swab32(x) (__u32)__builtin_bswap32((__u32)(x))
      |                                                      ^
./include/linux/byteorder/generic.h:95:21: note: in expansion of macro ‘__be32_to_cpu’
   95 | #define be32_to_cpu __be32_to_cpu
      |                     ^~~~~~~~~~~~~
./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’
   17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER)
      |                                ^~~~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
   51 | #define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
      |                                   ^~~~~~~~
./include/linux/mlx5/device.h:53:34: note: in expansion of macro ‘__mlx5_bit_off’
   53 | #define __mlx5_dw_off(typ, fld) (__mlx5_bit_off(typ, fld) / 32)
      |                                  ^~~~~~~~~~~~~~
./include/linux/mlx5/device.h:96:1: note: in expansion of macro ‘__mlx5_dw_off’
   96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
      | ^~~~~~~~~~~~~
./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
 1355 |  MLX5_GET(device_virtio_emulation_cap, \
      |  ^~~~~~~~
drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
   28 |  if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type))
      |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ./include/linux/mlx5/driver.h:52,
                 from drivers/vdpa/mlx5/net/main.c:5:
./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’
   50 | #define __mlx5_bit_sz(typ, fld) sizeof(__mlx5_nullp(typ)->fld)
      |                                                         ^~
./include/linux/mlx5/device.h:56:43: note: in expansion of macro ‘__mlx5_bit_sz’
   56 | #define __mlx5_dw_bit_off(typ, fld) (32 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0x1f))
      |                                           ^~~~~~~~~~~~~
./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’
   96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
      |                              ^~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
 1355 |  MLX5_GET(device_virtio_emulation_cap, \
      |  ^~~~~~~~
drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
   28 |  if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type))
      |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from <command-line>:
././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’
  129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b)
      |                                   ^~~~~~~~~~~~~~~~~~
./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’
   17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER)
      |                                ^~~~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
   51 | #define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
      |                                   ^~~~~~~~
./include/linux/mlx5/device.h:56:70: note: in expansion of macro ‘__mlx5_bit_off’
   56 | #define __mlx5_dw_bit_off(typ, fld) (32 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0x1f))
      |                                                                      ^~~~~~~~~~~~~~
./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’
   96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
      |                              ^~~~~~~~~~~~~~~~~
./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
 1355 |  MLX5_GET(device_virtio_emulation_cap, \
      |  ^~~~~~~~
drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
   28 |  if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type))
      |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ./include/linux/mlx5/driver.h:52,
                 from drivers/vdpa/mlx5/net/main.c:5:
./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’
   50 | #define __mlx5_bit_sz(typ, fld) sizeof(__mlx5_nullp(typ)->fld)
      |                                                         ^~
./include/linux/mlx5/device.h:57:47: note: in expansion of macro ‘__mlx5_bit_sz’
   57 | #define __mlx5_mask(typ, fld) ((u32)((1ull << __mlx5_bit_sz(typ, fld)) - 1))
      |                                               ^~~~~~~~~~~~~
./include/linux/mlx5/device.h:97:1: note: in expansion of macro ‘__mlx5_mask’
   97 | __mlx5_mask(typ, fld))
      | ^~~~~~~~~~~
./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
 1355 |  MLX5_GET(device_virtio_emulation_cap, \
      |  ^~~~~~~~
drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
   28 |  if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type))
      |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/vdpa/mlx5/net/main.c: At top level:
drivers/vdpa/mlx5/net/main.c:62:14: error: ‘MLX5_INTERFACE_PROTOCOL_VDPA’ undeclared here (not in a function); did you mean ‘MLX5_INTERFACE_PROTOCOL_ETH’?
   62 |  .protocol = MLX5_INTERFACE_PROTOCOL_VDPA,
      |              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |              MLX5_INTERFACE_PROTOCOL_ETH
make[3]: *** [scripts/Makefile.build:281: drivers/vdpa/mlx5/net/main.o] Error 1
make[2]: *** [scripts/Makefile.build:497: drivers/vdpa/mlx5] Error 2
make[1]: *** [scripts/Makefile.build:497: drivers/vdpa] Error 2
make: *** [Makefile:1756: drivers] Error 2


I am guessing this is because of the missing dependency, right?
So what's the plan for merging this?


-- 
MST

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH V4 linux-next 00/12] VDPA support for Mellanox ConnectX devices

[Michael S. Tsirkin]

On Wed, Aug 05, 2020 at 03:40:54PM +0300, Eli Cohen wrote:
> On Wed, Aug 05, 2020 at 08:00:55AM -0400, Michael S. Tsirkin wrote:
> > On Tue, Aug 04, 2020 at 07:20:36PM +0300, Eli Cohen wrote:
> > > Hi Michael,
> > > please note that this series depends on mlx5 core device driver patches
> > > in mlx5-next branch in
> > > git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git.
> > > 
> > > git pull git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git mlx5-next 
> > > 
> > > They also depend Jason Wang's patches: https://lkml.org/lkml/2020/7/1/301
> > 
> > So if I apply this to linux-next branch of my tree, I get:
> > 
> 
> Did you merge this?:
> git pull git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git mlx5-next


I can only merge this tree if no one else will. Linus does not like
getting same patches through two trees.

Is this the case? Is mlx5-next going to be merged through
my tree in this cycle?


> > 
> >   CALL    scripts/checksyscalls.sh
> >   CALL    scripts/atomic/check-atomics.sh
> >   DESCEND  objtool
> >   CHK     include/generated/compile.h
> >   CC [M]  drivers/vdpa/mlx5/net/main.o
> > In file included from ./include/linux/swab.h:5,
> >                  from ./include/uapi/linux/byteorder/little_endian.h:13,
> >                  from ./include/linux/byteorder/little_endian.h:5,
> >                  from ./arch/x86/include/uapi/asm/byteorder.h:5,
> >                  from ./include/asm-generic/bitops/le.h:6,
> >                  from ./arch/x86/include/asm/bitops.h:395,
> >                  from ./include/linux/bitops.h:29,
> >                  from ./include/linux/kernel.h:12,
> >                  from ./include/linux/list.h:9,
> >                  from ./include/linux/module.h:12,
> >                  from drivers/vdpa/mlx5/net/main.c:4:
> > drivers/vdpa/mlx5/net/main.c: In function ‘required_caps_supported’:
> > ././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’
> >   129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b)
> >       |                                   ^~~~~~~~~~~~~~~~~~
> > ./include/uapi/linux/swab.h:115:54: note: in definition of macro ‘__swab32’
> >   115 | #define __swab32(x) (__u32)__builtin_bswap32((__u32)(x))
> >       |                                                      ^
> > ./include/linux/byteorder/generic.h:95:21: note: in expansion of macro ‘__be32_to_cpu’
> >    95 | #define be32_to_cpu __be32_to_cpu
> >       |                     ^~~~~~~~~~~~~
> > ./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’
> >    17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER)
> >       |                                ^~~~~~~~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
> >    51 | #define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
> >       |                                   ^~~~~~~~
> > ./include/linux/mlx5/device.h:53:34: note: in expansion of macro ‘__mlx5_bit_off’
> >    53 | #define __mlx5_dw_off(typ, fld) (__mlx5_bit_off(typ, fld) / 32)
> >       |                                  ^~~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:96:1: note: in expansion of macro ‘__mlx5_dw_off’
> >    96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
> >       | ^~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
> >  1355 |  MLX5_GET(device_virtio_emulation_cap, \
> >       |  ^~~~~~~~
> > drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
> >    24 |  event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode);
> >       |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > In file included from ./include/linux/mlx5/driver.h:52,
> >                  from drivers/vdpa/mlx5/net/main.c:5:
> > ./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’
> >    50 | #define __mlx5_bit_sz(typ, fld) sizeof(__mlx5_nullp(typ)->fld)
> >       |                                                         ^~
> > ./include/linux/mlx5/device.h:56:43: note: in expansion of macro ‘__mlx5_bit_sz’
> >    56 | #define __mlx5_dw_bit_off(typ, fld) (32 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0x1f))
> >       |                                           ^~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’
> >    96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
> >       |                              ^~~~~~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
> >  1355 |  MLX5_GET(device_virtio_emulation_cap, \
> >       |  ^~~~~~~~
> > drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
> >    24 |  event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode);
> >       |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > In file included from <command-line>:
> > ././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’
> >   129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b)
> >       |                                   ^~~~~~~~~~~~~~~~~~
> > ./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’
> >    17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER)
> >       |                                ^~~~~~~~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
> >    51 | #define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
> >       |                                   ^~~~~~~~
> > ./include/linux/mlx5/device.h:56:70: note: in expansion of macro ‘__mlx5_bit_off’
> >    56 | #define __mlx5_dw_bit_off(typ, fld) (32 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0x1f))
> >       |                                                                      ^~~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’
> >    96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
> >       |                              ^~~~~~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
> >  1355 |  MLX5_GET(device_virtio_emulation_cap, \
> >       |  ^~~~~~~~
> > drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
> >    24 |  event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode);
> >       |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > In file included from ./include/linux/mlx5/driver.h:52,
> >                  from drivers/vdpa/mlx5/net/main.c:5:
> > ./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘event_mode’
> >    50 | #define __mlx5_bit_sz(typ, fld) sizeof(__mlx5_nullp(typ)->fld)
> >       |                                                         ^~
> > ./include/linux/mlx5/device.h:57:47: note: in expansion of macro ‘__mlx5_bit_sz’
> >    57 | #define __mlx5_mask(typ, fld) ((u32)((1ull << __mlx5_bit_sz(typ, fld)) - 1))
> >       |                                               ^~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:97:1: note: in expansion of macro ‘__mlx5_mask’
> >    97 | __mlx5_mask(typ, fld))
> >       | ^~~~~~~~~~~
> > ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
> >  1355 |  MLX5_GET(device_virtio_emulation_cap, \
> >       |  ^~~~~~~~
> > drivers/vdpa/mlx5/net/main.c:24:15: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
> >    24 |  event_mode = MLX5_CAP_DEV_VDPA_EMULATION(mdev, event_mode);
> >       |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > In file included from ./include/linux/swab.h:5,
> >                  from ./include/uapi/linux/byteorder/little_endian.h:13,
> >                  from ./include/linux/byteorder/little_endian.h:5,
> >                  from ./arch/x86/include/uapi/asm/byteorder.h:5,
> >                  from ./include/asm-generic/bitops/le.h:6,
> >                  from ./arch/x86/include/asm/bitops.h:395,
> >                  from ./include/linux/bitops.h:29,
> >                  from ./include/linux/kernel.h:12,
> >                  from ./include/linux/list.h:9,
> >                  from ./include/linux/module.h:12,
> >                  from drivers/vdpa/mlx5/net/main.c:4:
> > ././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’
> >   129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b)
> >       |                                   ^~~~~~~~~~~~~~~~~~
> > ./include/uapi/linux/swab.h:115:54: note: in definition of macro ‘__swab32’
> >   115 | #define __swab32(x) (__u32)__builtin_bswap32((__u32)(x))
> >       |                                                      ^
> > ./include/linux/byteorder/generic.h:95:21: note: in expansion of macro ‘__be32_to_cpu’
> >    95 | #define be32_to_cpu __be32_to_cpu
> >       |                     ^~~~~~~~~~~~~
> > ./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’
> >    17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER)
> >       |                                ^~~~~~~~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
> >    51 | #define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
> >       |                                   ^~~~~~~~
> > ./include/linux/mlx5/device.h:53:34: note: in expansion of macro ‘__mlx5_bit_off’
> >    53 | #define __mlx5_dw_off(typ, fld) (__mlx5_bit_off(typ, fld) / 32)
> >       |                                  ^~~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:96:1: note: in expansion of macro ‘__mlx5_dw_off’
> >    96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
> >       | ^~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
> >  1355 |  MLX5_GET(device_virtio_emulation_cap, \
> >       |  ^~~~~~~~
> > drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
> >    28 |  if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type))
> >       |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > In file included from ./include/linux/mlx5/driver.h:52,
> >                  from drivers/vdpa/mlx5/net/main.c:5:
> > ./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’
> >    50 | #define __mlx5_bit_sz(typ, fld) sizeof(__mlx5_nullp(typ)->fld)
> >       |                                                         ^~
> > ./include/linux/mlx5/device.h:56:43: note: in expansion of macro ‘__mlx5_bit_sz’
> >    56 | #define __mlx5_dw_bit_off(typ, fld) (32 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0x1f))
> >       |                                           ^~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’
> >    96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
> >       |                              ^~~~~~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
> >  1355 |  MLX5_GET(device_virtio_emulation_cap, \
> >       |  ^~~~~~~~
> > drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
> >    28 |  if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type))
> >       |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > In file included from <command-line>:
> > ././include/linux/compiler_types.h:129:35: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’
> >   129 | #define __compiler_offsetof(a, b) __builtin_offsetof(a, b)
> >       |                                   ^~~~~~~~~~~~~~~~~~
> > ./include/linux/stddef.h:17:32: note: in expansion of macro ‘__compiler_offsetof’
> >    17 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER)
> >       |                                ^~~~~~~~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:51:35: note: in expansion of macro ‘offsetof’
> >    51 | #define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
> >       |                                   ^~~~~~~~
> > ./include/linux/mlx5/device.h:56:70: note: in expansion of macro ‘__mlx5_bit_off’
> >    56 | #define __mlx5_dw_bit_off(typ, fld) (32 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0x1f))
> >       |                                                                      ^~~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:96:30: note: in expansion of macro ‘__mlx5_dw_bit_off’
> >    96 | __mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
> >       |                              ^~~~~~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
> >  1355 |  MLX5_GET(device_virtio_emulation_cap, \
> >       |  ^~~~~~~~
> > drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
> >    28 |  if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type))
> >       |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > In file included from ./include/linux/mlx5/driver.h:52,
> >                  from drivers/vdpa/mlx5/net/main.c:5:
> > ./include/linux/mlx5/device.h:50:57: error: ‘struct mlx5_ifc_device_virtio_emulation_cap_bits’ has no member named ‘eth_frame_offload_type’
> >    50 | #define __mlx5_bit_sz(typ, fld) sizeof(__mlx5_nullp(typ)->fld)
> >       |                                                         ^~
> > ./include/linux/mlx5/device.h:57:47: note: in expansion of macro ‘__mlx5_bit_sz’
> >    57 | #define __mlx5_mask(typ, fld) ((u32)((1ull << __mlx5_bit_sz(typ, fld)) - 1))
> >       |                                               ^~~~~~~~~~~~~
> > ./include/linux/mlx5/device.h:97:1: note: in expansion of macro ‘__mlx5_mask’
> >    97 | __mlx5_mask(typ, fld))
> >       | ^~~~~~~~~~~
> > ./include/linux/mlx5/device.h:1355:2: note: in expansion of macro ‘MLX5_GET’
> >  1355 |  MLX5_GET(device_virtio_emulation_cap, \
> >       |  ^~~~~~~~
> > drivers/vdpa/mlx5/net/main.c:28:7: note: in expansion of macro ‘MLX5_CAP_DEV_VDPA_EMULATION’
> >    28 |  if (!MLX5_CAP_DEV_VDPA_EMULATION(mdev, eth_frame_offload_type))
> >       |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~
> > drivers/vdpa/mlx5/net/main.c: At top level:
> > drivers/vdpa/mlx5/net/main.c:62:14: error: ‘MLX5_INTERFACE_PROTOCOL_VDPA’ undeclared here (not in a function); did you mean ‘MLX5_INTERFACE_PROTOCOL_ETH’?
> >    62 |  .protocol = MLX5_INTERFACE_PROTOCOL_VDPA,
> >       |              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >       |              MLX5_INTERFACE_PROTOCOL_ETH
> > make[3]: *** [scripts/Makefile.build:281: drivers/vdpa/mlx5/net/main.o] Error 1
> > make[2]: *** [scripts/Makefile.build:497: drivers/vdpa/mlx5] Error 2
> > make[1]: *** [scripts/Makefile.build:497: drivers/vdpa] Error 2
> > make: *** [Makefile:1756: drivers] Error 2
> > 
> > 
> > I am guessing this is because of the missing dependency, right?
> > So what's the plan for merging this?
> > 
> > 
> > -- 
> > MST
> > 

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH V4 linux-next 00/12] VDPA support for Mellanox ConnectX devices

[Michael S. Tsirkin]

On Wed, Aug 05, 2020 at 04:01:58PM +0300, Eli Cohen wrote:
> On Wed, Aug 05, 2020 at 08:48:52AM -0400, Michael S. Tsirkin wrote:
> > > 
> > > Did you merge this?:
> > > git pull git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.git mlx5-next
> > 
> > 
> > I can only merge this tree if no one else will. Linus does not like
> > getting same patches through two trees.
> > 
> > Is this the case? Is mlx5-next going to be merged through
> > my tree in this cycle?
> > 
> 
> Saeed Mahameed from Mellanox (located in California) usuaally sends out
> net patches. So he's supposed to send that to Dave Miller.
> 
> I think Saeed should answer this. Let's wait a few more hours till he
> wakes up.

Alternatives:
- merge vdpa through Saeed's tree. I can ack that, we'll need to
  resolve any conflicts by merging the two trees and show the
  result to Linus so he can resolve the merge in the same way.
- extract just the necessary patches that are needed for vdpa and
  merge through my tree.
- if Saeed sends his pull today, it's likely it will be merged
  early next week. Then I can rebase and send a pull with your patches
  on top. A bit risky.
- do some tricks with build. E.g. disable build of your code,
  and enable in Saeed's tree when everything is merged together.
  Can be somewhat hard.

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH V4 linux-next 00/12] VDPA support for Mellanox ConnectX devices

[Saeed Mahameed]

On Wed, 2020-08-05 at 09:12 -0400, Michael S. Tsirkin wrote:
> On Wed, Aug 05, 2020 at 04:01:58PM +0300, Eli Cohen wrote:
> > On Wed, Aug 05, 2020 at 08:48:52AM -0400, Michael S. Tsirkin wrote:
> > > > Did you merge this?:
> > > > git pull
> > > > git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gi
> > > > t mlx5-next
> > > 
> > > I can only merge this tree if no one else will. Linus does not
> > > like
> > > getting same patches through two trees.
> > > 
> > > Is this the case? Is mlx5-next going to be merged through
> > > my tree in this cycle?
> > > 
> > 
> > Saeed Mahameed from Mellanox (located in California) usuaally sends
> > out
> > net patches. So he's supposed to send that to Dave Miller.
> > 
> > I think Saeed should answer this. Let's wait a few more hours till
> > he
> > wakes up.
> 
> Alternatives:
> - merge vdpa through Saeed's tree. I can ack that, we'll need to
>   resolve any conflicts by merging the two trees and show the
>   result to Linus so he can resolve the merge in the same way.
> - extract just the necessary patches that are needed for vdpa and
>   merge through my tree.
> - if Saeed sends his pull today, it's likely it will be merged
>   early next week. Then I can rebase and send a pull with your
> patches
>   on top. A bit risky.
> - do some tricks with build. E.g. disable build of your code,
>   and enable in Saeed's tree when everything is merged together.
>   Can be somewhat hard.
> 

Hi Michael,

We do this all the time with net-next and rdma,
mlx5-next is a very small branch based on a very early rc that includes
mlx5 shared stuff between rdma and net-next, and now virtio as well.

we send pull requests of mlx5-next to both rdma and net-next with the
respective features, exactly as we did here, and it works nicely, since
we reduce the number of conflicts to 0 between different subsystems
that rely on mlx5 core.

all the alternative you suggested have never been tried before :),
net-next is Closed, so i can't do further submissions.


_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH V4 linux-next 00/12] VDPA support for Mellanox ConnectX devices

[Jason Gunthorpe]

On Wed, Aug 05, 2020 at 07:01:52PM +0000, Saeed Mahameed wrote:
> On Wed, 2020-08-05 at 09:12 -0400, Michael S. Tsirkin wrote:
> > On Wed, Aug 05, 2020 at 04:01:58PM +0300, Eli Cohen wrote:
> > > On Wed, Aug 05, 2020 at 08:48:52AM -0400, Michael S. Tsirkin wrote:
> > > > > Did you merge this?:
> > > > > git pull
> > > > > git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gi
> > > > > t mlx5-next
> > > > 
> > > > I can only merge this tree if no one else will. Linus does not
> > > > like getting same patches through two trees.

This is not quite the case

Linus does not like multiple *copies* of the same patches. The same
actual git commits can be OK.

Linus also does not like unnecessarily cross linking trees, mlx5-next
is designed to be small enough and approved enough that it is not
controversial.

Linus really doesn't like it when people jams stuff together in rc7 or
the weeks of the merge window. He wants to see stuff be in linux-next
for at least a bit. So it may be too late regardless.

> We do this all the time with net-next and rdma,
> mlx5-next is a very small branch based on a very early rc that includes
> mlx5 shared stuff between rdma and net-next, and now virtio as well.

Yes, going on two years now? Been working well

Jason
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH V4 linux-next 00/12] VDPA support for Mellanox ConnectX devices

[Michael S. Tsirkin]

On Wed, Aug 05, 2020 at 04:46:46PM -0300, Jason Gunthorpe wrote:
> On Wed, Aug 05, 2020 at 07:01:52PM +0000, Saeed Mahameed wrote:
> > On Wed, 2020-08-05 at 09:12 -0400, Michael S. Tsirkin wrote:
> > > On Wed, Aug 05, 2020 at 04:01:58PM +0300, Eli Cohen wrote:
> > > > On Wed, Aug 05, 2020 at 08:48:52AM -0400, Michael S. Tsirkin wrote:
> > > > > > Did you merge this?:
> > > > > > git pull
> > > > > > git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux.gi
> > > > > > t mlx5-next
> > > > > 
> > > > > I can only merge this tree if no one else will. Linus does not
> > > > > like getting same patches through two trees.
> 
> This is not quite the case
> 
> Linus does not like multiple *copies* of the same patches. The same
> actual git commits can be OK.
> 
> Linus also does not like unnecessarily cross linking trees, mlx5-next
> is designed to be small enough and approved enough that it is not
> controversial.
> 
> Linus really doesn't like it when people jams stuff together in rc7 or
> the weeks of the merge window. He wants to see stuff be in linux-next
> for at least a bit. So it may be too late regardless.

I'll try, let's see what happens.

> > We do this all the time with net-next and rdma,
> > mlx5-next is a very small branch based on a very early rc that includes
> > mlx5 shared stuff between rdma and net-next, and now virtio as well.
> 
> Yes, going on two years now? Been working well
> 
> Jason

OK, I'll merge it then. Thanks!

-- 
MST

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization