From: Yuval Shaia <yuval.shaia@oracle.com>
To: yuval.shaia@oracle.com, marcel.apfelbaum@gmail.com,
dmitry.fleytman@gmail.com, jasowang@redhat.com,
eblake@redhat.com, armbru@redhat.com, pbonzini@redhat.com,
qemu-devel@nongnu.org, shamir.rabinovitch@oracle.com,
cohuck@redhat.com
Subject: [Qemu-devel] [PATCH v3 00/23] Add support for RDMA MAD
Date: Tue, 13 Nov 2018 09:12:49 +0200 [thread overview]
Message-ID: <20181113071336.6242-1-yuval.shaia@oracle.com> (raw)
Hi all.
This is a major enhancement to the pvrdma device to allow it to work with
state of the art applications such as MPI.
As described in patch #5, MAD packets are management packets that are used
for many purposes including but not limited to communication layer above IB
verbs API.
Patch 1 exposes new external executable (under contrib) that aims to
address a specific limitation in the RDMA usrespace MAD stack.
This patch-set mainly present MAD enhancement but during the work on it i
came across some bugs and enhancement needed to be implemented before doing
any MAD coding. This is the role of patches 2 to 4, 7 to 9 and 15 to 17.
Patches 6 and 18 are cosmetic changes while not relevant to this patchset
still introduce with it since (at least for 6) hard to decouple.
Patches 12 to 15 couple pvrdma device with vmxnet3 device as this is the
configuration enforced by pvrdma driver in guest - a vmxnet3 device in
function 0 and pvrdma device in function 1 in the same PCI slot. Patch 12
moves needed code from vmxnet3 device to a new header file that can be used
by pvrdma code while Patches 13 to 15 use of it.
Along with this patch-set there is a parallel patch posted to libvirt to
apply the change needed there as part of the process implemented in patches
10 and 11. This change is needed so that guest would be able to configure
any IP to the Ethernet function of the pvrdma device.
https://www.redhat.com/archives/libvir-list/2018-November/msg00135.html
Since we maintain external resources such as GIDs on host GID table we need
to do some cleanup before going down. This is the job of patches 19 and 20.
Patches 20 and 21 contain a fixes for bugs detected during the work on
processing cleanup code during shutdown.
v1 -> v2:
* Fix compilation issue detected when compiling for mingw
* Address comment from Eric Blake re version of QEMU in json
message
* Fix example from QMP message in json file
* Fix case where a VM tries to remove an invalid GID from GID table
* rdmacm-mux: Cleanup entries in socket-gids table when socket is
closed
* Cleanup resources (GIDs, QPs etc) when VM goes down
v2 -> v3:
* Address comment from Cornelia Huck for patch #19
* Add some R-Bs from Marcel Apfelbaum and Dmitry Fleytman
* Update docs/pvrdma.txt with the changes made by this patchset
* Address comments from Shamir Rabinovitch for UMAD multiplexer
Thanks,
Yuval
Yuval Shaia (23):
contrib/rdmacm-mux: Add implementation of RDMA User MAD multiplexer
hw/rdma: Add ability to force notification without re-arm
hw/rdma: Return qpn 1 if ibqp is NULL
hw/rdma: Abort send-op if fail to create addr handler
hw/rdma: Add support for MAD packets
hw/pvrdma: Make function reset_device return void
hw/pvrdma: Make default pkey 0xFFFF
hw/pvrdma: Set the correct opcode for recv completion
hw/pvrdma: Set the correct opcode for send completion
json: Define new QMP message for pvrdma
hw/pvrdma: Add support to allow guest to configure GID table
vmxnet3: Move some definitions to header file
hw/pvrdma: Make sure PCI function 0 is vmxnet3
hw/rdma: Initialize node_guid from vmxnet3 mac address
hw/pvrdma: Make device state depend on Ethernet function state
hw/pvrdma: Fill all CQE fields
hw/pvrdma: Fill error code in command's response
hw/rdma: Remove unneeded code that handles more that one port
vl: Introduce shutdown_notifiers
hw/pvrdma: Clean device's resource when system is shutdown
hw/rdma: Do not use bitmap_zero_extend to free bitmap
hw/rdma: Do not call rdma_backend_del_gid on an empty gid
docs: Update pvrdma device documentation
MAINTAINERS | 2 +
Makefile | 6 +-
Makefile.objs | 5 +
contrib/rdmacm-mux/Makefile.objs | 4 +
contrib/rdmacm-mux/main.c | 771 +++++++++++++++++++++++++++++++
contrib/rdmacm-mux/rdmacm-mux.h | 56 +++
docs/pvrdma.txt | 103 ++++-
hw/net/vmxnet3.c | 116 +----
hw/net/vmxnet3_defs.h | 133 ++++++
hw/rdma/rdma_backend.c | 461 +++++++++++++++---
hw/rdma/rdma_backend.h | 28 +-
hw/rdma/rdma_backend_defs.h | 13 +-
hw/rdma/rdma_rm.c | 120 ++++-
hw/rdma/rdma_rm.h | 17 +-
hw/rdma/rdma_rm_defs.h | 21 +-
hw/rdma/rdma_utils.h | 24 +
hw/rdma/vmw/pvrdma.h | 10 +-
hw/rdma/vmw/pvrdma_cmd.c | 119 +++--
hw/rdma/vmw/pvrdma_main.c | 49 +-
hw/rdma/vmw/pvrdma_qp_ops.c | 62 ++-
include/sysemu/sysemu.h | 1 +
qapi/qapi-schema.json | 1 +
qapi/rdma.json | 38 ++
vl.c | 15 +-
24 files changed, 1868 insertions(+), 307 deletions(-)
create mode 100644 contrib/rdmacm-mux/Makefile.objs
create mode 100644 contrib/rdmacm-mux/main.c
create mode 100644 contrib/rdmacm-mux/rdmacm-mux.h
create mode 100644 hw/net/vmxnet3_defs.h
create mode 100644 qapi/rdma.json
--
2.17.2
next reply other threads:[~2018-11-13 7:14 UTC|newest]
Thread overview: 70+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-13 7:12 Yuval Shaia [this message]
2018-11-13 7:12 ` [Qemu-devel] [PATCH v3 01/23] contrib/rdmacm-mux: Add implementation of RDMA User MAD multiplexer Yuval Shaia
2018-11-13 7:12 ` [Qemu-devel] [PATCH v3 02/23] hw/rdma: Add ability to force notification without re-arm Yuval Shaia
2018-11-13 7:12 ` [Qemu-devel] [PATCH v3 03/23] hw/rdma: Return qpn 1 if ibqp is NULL Yuval Shaia
2018-11-17 11:42 ` Marcel Apfelbaum
2018-11-13 7:12 ` [Qemu-devel] [PATCH v3 04/23] hw/rdma: Abort send-op if fail to create addr handler Yuval Shaia
2018-11-13 7:12 ` [Qemu-devel] [PATCH v3 05/23] hw/rdma: Add support for MAD packets Yuval Shaia
2018-11-17 12:06 ` Marcel Apfelbaum
2018-11-18 9:33 ` Yuval Shaia
2018-11-13 7:12 ` [Qemu-devel] [PATCH v3 06/23] hw/pvrdma: Make function reset_device return void Yuval Shaia
2018-11-13 7:12 ` [Qemu-devel] [PATCH v3 07/23] hw/pvrdma: Make default pkey 0xFFFF Yuval Shaia
2018-11-13 7:12 ` [Qemu-devel] [PATCH v3 08/23] hw/pvrdma: Set the correct opcode for recv completion Yuval Shaia
2018-11-13 7:12 ` [Qemu-devel] [PATCH v3 09/23] hw/pvrdma: Set the correct opcode for send completion Yuval Shaia
2018-11-17 12:07 ` Marcel Apfelbaum
2018-11-13 7:12 ` [Qemu-devel] [PATCH v3 10/23] json: Define new QMP message for pvrdma Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 11/23] hw/pvrdma: Add support to allow guest to configure GID table Yuval Shaia
2018-11-17 12:48 ` Marcel Apfelbaum
2018-11-18 8:13 ` Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 12/23] vmxnet3: Move some definitions to header file Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 13/23] hw/pvrdma: Make sure PCI function 0 is vmxnet3 Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 14/23] hw/rdma: Initialize node_guid from vmxnet3 mac address Yuval Shaia
2018-11-17 12:10 ` Marcel Apfelbaum
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 15/23] hw/pvrdma: Make device state depend on Ethernet function state Yuval Shaia
2018-11-17 12:11 ` Marcel Apfelbaum
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 16/23] hw/pvrdma: Fill all CQE fields Yuval Shaia
2018-11-17 12:19 ` Marcel Apfelbaum
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 17/23] hw/pvrdma: Fill error code in command's response Yuval Shaia
2018-11-17 12:22 ` Marcel Apfelbaum
2018-11-18 8:24 ` Yuval Shaia
2018-11-25 7:35 ` Marcel Apfelbaum
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 18/23] hw/rdma: Remove unneeded code that handles more that one port Yuval Shaia
2018-11-17 12:23 ` Marcel Apfelbaum
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 19/23] vl: Introduce shutdown_notifiers Yuval Shaia
2018-11-13 9:34 ` Cornelia Huck
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 20/23] hw/pvrdma: Clean device's resource when system is shutdown Yuval Shaia
2018-11-17 12:24 ` Marcel Apfelbaum
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 21/23] hw/rdma: Do not use bitmap_zero_extend to free bitmap Yuval Shaia
2018-11-17 12:25 ` Marcel Apfelbaum
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 22/23] hw/rdma: Do not call rdma_backend_del_gid on an empty gid Yuval Shaia
2018-11-17 12:25 ` Marcel Apfelbaum
2018-11-18 9:42 ` Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 23/23] docs: Update pvrdma device documentation Yuval Shaia
2018-11-17 12:34 ` Marcel Apfelbaum
2018-11-18 7:27 ` Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 00/23] Add support for RDMA MAD Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 01/23] contrib/rdmacm-mux: Add implementation of RDMA User MAD multiplexer Yuval Shaia
2018-11-17 17:27 ` Shamir Rabinovitch
2018-11-18 10:17 ` Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 02/23] hw/rdma: Add ability to force notification without re-arm Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 03/23] hw/rdma: Return qpn 1 if ibqp is NULL Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 04/23] hw/rdma: Abort send-op if fail to create addr handler Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 05/23] hw/rdma: Add support for MAD packets Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 06/23] hw/pvrdma: Make function reset_device return void Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 07/23] hw/pvrdma: Make default pkey 0xFFFF Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 08/23] hw/pvrdma: Set the correct opcode for recv completion Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 09/23] hw/pvrdma: Set the correct opcode for send completion Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 10/23] json: Define new QMP message for pvrdma Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 11/23] hw/pvrdma: Add support to allow guest to configure GID table Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 12/23] vmxnet3: Move some definitions to header file Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 13/23] hw/pvrdma: Make sure PCI function 0 is vmxnet3 Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 14/23] hw/rdma: Initialize node_guid from vmxnet3 mac address Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 15/23] hw/pvrdma: Make device state depend on Ethernet function state Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 16/23] hw/pvrdma: Fill all CQE fields Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 17/23] hw/pvrdma: Fill error code in command's response Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 18/23] hw/rdma: Remove unneeded code that handles more that one port Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 19/23] vl: Introduce shutdown_notifiers Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 20/23] hw/pvrdma: Clean device's resource when system is shutdown Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 21/23] hw/rdma: Do not use bitmap_zero_extend to free bitmap Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 22/23] hw/rdma: Do not call rdma_backend_del_gid on an empty gid Yuval Shaia
2018-11-13 7:13 ` [Qemu-devel] [PATCH v3 23/23] docs: Update pvrdma device documentation Yuval Shaia
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181113071336.6242-1-yuval.shaia@oracle.com \
--to=yuval.shaia@oracle.com \
--cc=armbru@redhat.com \
--cc=cohuck@redhat.com \
--cc=dmitry.fleytman@gmail.com \
--cc=eblake@redhat.com \
--cc=jasowang@redhat.com \
--cc=marcel.apfelbaum@gmail.com \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=shamir.rabinovitch@oracle.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.