"Connect rejected: status 28" from "nvme discover"

"Connect rejected: status 28" from "nvme discover"

[Chris Worley]

Running CentOS 8.2, using native drivers but following instructions at
https://community.mellanox.com/s/article/howto-configure-nvme-over-fabrics
(but not overriding the native drivers with MOFED).

Hardware is connectX-3 HCAs.

Target setup goes okay:

[root@localhost 1]# dmesg | grep nvmet_rdma
[ 6366.612458] nvmet_rdma: enabling port 1 (10.4.0.6:4420)
[ 6376.078826] nvmet_rdma: enabling port 2 (10.4.0.6:4421)
[ 6385.662419] nvmet_rdma: enabling port 3 (10.4.0.6:4423)

But discover goes awry on client:

[root@localhost network-scripts]# nvme discover -t rdma -a 10.4.0.6 -s 4420
Failed to write to /dev/nvme-fabrics: Connection reset by peer
[root@localhost network-scripts]# dmesg | tail
[ 1688.058291] nvme nvme4: Connect rejected: status 28 (consumer
defined) nvme status 0 (unrecognized reason).
[ 1688.058410] nvme nvme4: rdma connection establishment failed (-104)

Systems ping each other.

Hnts?

Thanks,

Chris

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: "Connect rejected: status 28" from "nvme discover"

[Keith Busch]

On Tue, Aug 25, 2020 at 11:59:28AM -1000, Chris Worley wrote:
> Running CentOS 8.2, using native drivers but following instructions at
> https://community.mellanox.com/s/article/howto-configure-nvme-over-fabrics
> (but not overriding the native drivers with MOFED).
> 
> Hardware is connectX-3 HCAs.
> 
> Target setup goes okay:
> 
> [root@localhost 1]# dmesg | grep nvmet_rdma
> [ 6366.612458] nvmet_rdma: enabling port 1 (10.4.0.6:4420)
> [ 6376.078826] nvmet_rdma: enabling port 2 (10.4.0.6:4421)
> [ 6385.662419] nvmet_rdma: enabling port 3 (10.4.0.6:4423)
> 
> But discover goes awry on client:
> 
> [root@localhost network-scripts]# nvme discover -t rdma -a 10.4.0.6 -s 4420
> Failed to write to /dev/nvme-fabrics: Connection reset by peer
> [root@localhost network-scripts]# dmesg | tail
> [ 1688.058291] nvme nvme4: Connect rejected: status 28 (consumer defined) nvme status 0 (unrecognized reason).
> [ 1688.058410] nvme nvme4: rdma connection establishment failed (-104)
> 
> Systems ping each other.
> 
> Hnts?

Does the target's dmesg mention anything after the failed connection?

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: "Connect rejected: status 28" from "nvme discover"

[Chris Worley]

On Tue, Aug 25, 2020 at 3:03 PM Keith Busch <kbusch@kernel.org> wrote:
> On Tue, Aug 25, 2020 at 11:59:28AM -1000, Chris Worley wrote:
...
> > Hnts?
>
> Does the target's dmesg mention anything after the failed connection?

No, the "enabling port 1" are the tail of dmesg.

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: "Connect rejected: status 28" from "nvme discover"

[Chris Worley]

Never mind... redid it all and it works.

On Tue, Aug 25, 2020 at 5:19 PM Chris Worley <worleys@gmail.com> wrote:
>
> On Tue, Aug 25, 2020 at 3:03 PM Keith Busch <kbusch@kernel.org> wrote:
> > On Tue, Aug 25, 2020 at 11:59:28AM -1000, Chris Worley wrote:
> ...
> > > Hnts?
> >
> > Does the target's dmesg mention anything after the failed connection?
>
> No, the "enabling port 1" are the tail of dmesg.

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme