All of lore.kernel.org
 help / color / mirror / Atom feed
From: Yanjun Zhu <yanjun.zhu@linux.dev>
To: Alexander Kalentyev <comrad.karlovich@gmail.com>
Cc: linux-rdma@vger.kernel.org
Subject: Re: rdma_rxe usage problem
Date: Thu, 20 Jan 2022 21:21:40 +0800	[thread overview]
Message-ID: <40803074-640b-7e42-8a0f-e515ab7c2874@linux.dev> (raw)
In-Reply-To: <CABrV9YuiYkCKMBMsrnd38ZxwqxJeWMw+xXFRpXc1gMtdAN9bFA@mail.gmail.com>

If I remember correctly, can you use this command to make tests?

server:

ibv_rc_pingpong -d rxe0 -g 1

Client:

ibv_rc_pingpong -d rxe0 -g 1 server_ip_addr

Zhu Yanjun

在 2022/1/20 1:53, Alexander Kalentyev 写道:
> With rping everything was fiine, but I had to use a real Ip address.
>   >rping -s -C 10 -v
> server ping data: rdma-ping-0:
> ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqr
> server ping data: rdma-ping-1:
> BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrs
> server ping data: rdma-ping-2:
> CDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrst
> server ping data: rdma-ping-3:
> DEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstu
> server ping data: rdma-ping-4:
> EFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuv
> server ping data: rdma-ping-5:
> FGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvw
> server ping data: rdma-ping-6:
> GHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwx
> server ping data: rdma-ping-7:
> HIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxy
> server ping data: rdma-ping-8:
> IJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz
> server ping data: rdma-ping-9:
> JKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzA
> server DISCONNECT EVENT...
> wait for RDMA_READ_ADV state 10
>
>> rping -c -a 192.168.0.176 -C 10 -v
> ping data: rdma-ping-0: ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqr
> ping data: rdma-ping-1: BCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrs
> ping data: rdma-ping-2: CDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrst
> ping data: rdma-ping-3: DEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstu
> ping data: rdma-ping-4: EFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuv
> ping data: rdma-ping-5: FGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvw
> ping data: rdma-ping-6: GHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwx
> ping data: rdma-ping-7: HIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxy
> ping data: rdma-ping-8: IJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz
> ping data: rdma-ping-9: JKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyzA
> client DISCONNECT EVENT...
>
> Anyway the ibv_rc_pingpong shows an error:
>
>> ibv_rc_pingpong -d rxe0 -g 0
>    local address:  LID 0x0000, QPN 0x000015, PSN 0x015dd8, GID
> fe80::4a51:c5ff:fef6:e159
> Failed to modify QP to RTR
> Couldn't connect to remote QP
>
>> ibv_rc_pingpong -d rxe0 -g 0 192.168.0.176
>    local address:  LID 0x0000, QPN 0x000016, PSN 0x007fa7, GID
> fe80::4a51:c5ff:fef6:e159
> client read/write: No space left on device
> Couldn't read/write remote address
>
> ср, 19 янв. 2022 г. в 15:12, Yanjun Zhu <yanjun.zhu@linux.dev>:
>> 在 2022/1/19 19:42, Alexander Kalentyev 写道:
>>> I am trying to install and use soft RoCE for development purposes
>>> (right now on a localhost).
>>> I installed rdma-core on a MANJARO system from AUR.
>>> Then I did:
>>>
>>> sudo modprobe rdma_rxe
>>> sudo rdma link add rxe0 type rxe netdev wlp60s0
>>>
>>> then ibstat shows:
>>> CA 'rxe0'
>>>           CA type:
>>>           Number of ports: 1
>>>           Firmware version:
>>>           Hardware version:
>>>           Node GUID: 0x4a51c5fffef6e159
>>>           System image GUID: 0x4a51c5fffef6e159
>>>           Port 1:
>>>                   State: Active
>>>                   Physical state: LinkUp
>>>                   Rate: 2.5
>>>                   Base lid: 0
>>>                   LMC: 0
>>>                   SM lid: 0
>>>                   Capability mask: 0x00010000
>>>                   Port GUID: 0x4a51c5fffef6e159
>>>                   Link layer: Ethernet
>> Can rping work after you configured this test environment?
>>
>> Zhu Yanjun
>>
>>> But launching any example I always get an error by call of: ibv_modify_qp
>>> with an attempt to modify QP state to RTR (for example by launching
>>> .ibv_rc_pingpong)
>>> The type of the error is EINVAL.
>>> I believe I miss something very obvious.

      parent reply	other threads:[~2022-01-20 13:21 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-01-19 11:42 rdma_rxe usage problem Alexander Kalentyev
2022-01-19 14:12 ` Yanjun Zhu
2022-01-19 17:53   ` Alexander Kalentyev
2022-01-19 19:54     ` Pearson, Robert B
2022-01-20 11:32       ` Alexander Kalentyev
2022-01-20 12:23         ` Leon Romanovsky
2022-01-20 13:21     ` Yanjun Zhu [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=40803074-640b-7e42-8a0f-e515ab7c2874@linux.dev \
    --to=yanjun.zhu@linux.dev \
    --cc=comrad.karlovich@gmail.com \
    --cc=linux-rdma@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.