Fwd: RDMACM Python test cases

Fwd: RDMACM Python test cases

[Bob Pearson]

-------- Forwarded Message --------
Subject: RDMACM Python test cases
Date: Thu, 17 Jun 2021 16:40:31 -0500
From: Bob Pearson <rpearsonhpe@gmail.com>
To: Edward Srouji <edwards@nvidia.com>, Jason Gunthorpe <jgg@nvidia.com>, Zhu Yanjun <zyjzyj2000@gmail.com>, linux-rdma@vger.linux.org

Edward,

Currently all the RDMACM python test cases in rdma-core fail on the rxe driver. They can be made to pass but I am not sure it is the correct way. It is

Before calling ib_register_device() set the ib_device>device>parent pointer to point at the PCIe bus device that is the parent of the network device that is being used to carry soft RoCE packets. Then the kernel will create /sys/class/infiniband/rxe_<N>/device/net/<NDEV> in the kobject tree.

This makes all the python RDMACM tests pass. But, it makes libibverbs complain if there are other RDMA devices used as NDEVs for rxe. For example running rxe over mlx5 (using a fake UDP dport) breaks. There seem to be a lot of assumptions being made about the /sys file system that can fail. Like there is only one device in device/net and there is only one device in device/infiniband.

The reason the python tests fail is that there is code that attempts to a) discover the network device name from /sys and b) then use % ip addr dev name ... to discover the IP address. For rxe there is another way to do this that doesn't need ...rxe_<N>/device/net... by using % rdma link which gives a list of rdma devices and matching ndevs. It seems to me that it should be fairly easy to fall back to this if the test for /sys/class/infiniband/{}/device/net fails.

Regards,

Bob Pearson