From: "Bernard Metzler" <BMT@zurich.ibm.com>
To: "Kamal Heib" <kamalheib1@gmail.com>
Cc: linux-rdma@vger.kernel.org, "Jason Gunthorpe" <jgg@ziepe.ca>,
"Doug Ledford" <dledford@redhat.com>
Subject: RE: [PATH for-next] RDMA/siw: Fix setting active_{speed, width} attributes
Date: Mon, 17 Feb 2020 14:27:54 +0000 [thread overview]
Message-ID: <OF2F71177A.B62840BF-ON00258511.004F75BA-00258511.004F75C2@notes.na.collabserv.com> (raw)
In-Reply-To: <20200217141146.GA1908@kheib-workstation>
-----"Kamal Heib" <kamalheib1@gmail.com> wrote: -----
>To: "Bernard Metzler" <BMT@zurich.ibm.com>
>From: "Kamal Heib" <kamalheib1@gmail.com>
>Date: 02/17/2020 03:12PM
>Cc: linux-rdma@vger.kernel.org, "Jason Gunthorpe" <jgg@ziepe.ca>,
>"Doug Ledford" <dledford@redhat.com>
>Subject: [EXTERNAL] Re: [PATH for-next] RDMA/siw: Fix setting
>active_{speed, width} attributes
>
>On Mon, Feb 17, 2020 at 10:13:21AM +0000, Bernard Metzler wrote:
>> -----"Kamal Heib" <kamalheib1@gmail.com> wrote: -----
>>
>> >To: "Bernard Metzler" <BMT@zurich.ibm.com>
>> >From: "Kamal Heib" <kamalheib1@gmail.com>
>> >Date: 02/16/2020 02:43PM
>> >Cc: linux-rdma@vger.kernel.org, "Jason Gunthorpe" <jgg@ziepe.ca>,
>> >"Doug Ledford" <dledford@redhat.com>
>> >Subject: [EXTERNAL] Re: [PATH for-next] RDMA/siw: Fix setting
>> >active_{speed, width} attributes
>> >
>> >On Thu, Feb 13, 2020 at 01:59:30PM +0000, Bernard Metzler wrote:
>> >> -----"Kamal Heib" <kamalheib1@gmail.com> wrote: -----
>> >>
>> >> >To: linux-rdma@vger.kernel.org
>> >> >From: "Kamal Heib" <kamalheib1@gmail.com>
>> >> >Date: 02/13/2020 02:07PM
>> >> >Cc: "Jason Gunthorpe" <jgg@ziepe.ca>, "Doug Ledford"
>> >> ><dledford@redhat.com>, "Bernard Metzler" <bmt@zurich.ibm.com>,
>> >"Kamal
>> >> >Heib" <kamalheib1@gmail.com>
>> >> >Subject: [EXTERNAL] [PATH for-next] RDMA/siw: Fix setting
>> >> >active_{speed, width} attributes
>> >> >
>> >> >Make sure to set the active_{speed, width} attributes to avoid
>> >> >reporting
>> >> >the same values regardless of the underlying device.
>> >> >
>> >> >Fixes: 303ae1cdfdf7 ("rdma/siw: application interface")
>> >> >Signed-off-by: Kamal Heib <kamalheib1@gmail.com>
>> >> >---
>> >> > drivers/infiniband/sw/siw/siw_verbs.c | 7 ++++---
>> >> > 1 file changed, 4 insertions(+), 3 deletions(-)
>> >> >
>> >> >diff --git a/drivers/infiniband/sw/siw/siw_verbs.c
>> >> >b/drivers/infiniband/sw/siw/siw_verbs.c
>> >> >index 73485d0da907..b1aaec912edb 100644
>> >> >--- a/drivers/infiniband/sw/siw/siw_verbs.c
>> >> >+++ b/drivers/infiniband/sw/siw/siw_verbs.c
>> >> >@@ -165,11 +165,12 @@ int siw_query_port(struct ib_device
>> >*base_dev,
>> >> >u8 port,
>> >> > struct ib_port_attr *attr)
>> >> > {
>> >> > struct siw_device *sdev = to_siw_dev(base_dev);
>> >> >+ int rc;
>> >> >
>> >> > memset(attr, 0, sizeof(*attr));
>> >> >
>> >> >- attr->active_speed = 2;
>> >> >- attr->active_width = 2;
>> >> >+ rc = ib_get_eth_speed(base_dev, port, &attr->active_speed,
>> >> >+ &attr->active_width);
>> >> > attr->gid_tbl_len = 1;
>> >> > attr->max_msg_sz = -1;
>> >> > attr->max_mtu = ib_mtu_int_to_enum(sdev->netdev->mtu);
>> >> >@@ -192,7 +193,7 @@ int siw_query_port(struct ib_device
>*base_dev,
>> >u8
>> >> >port,
>> >> > * attr->subnet_timeout = 0;
>> >> > * attr->init_type_repy = 0;
>> >> > */
>> >> >- return 0;
>> >> >+ return rc;
>> >> > }
>> >> >
>> >> > int siw_get_port_immutable(struct ib_device *base_dev, u8
>port,
>> >> >--
>> >> >2.21.1
>> >> >
>> >> >
>> >> Hi Kamal,
>> >
>> >Hi Bernard,
>> >
>> >> Many thanks for looking after this! So there definitely seem to
>> >> be applications which are taking care of those values. So, good
>> >> to get my obvious laziness fixed.
>> >>
>> >
>> >Sure :)
>> >
>> >> I tried your patch on a 40Gbs Ethernet link (Chelsio cxgb4
>driver).
>> >> Works in principle, but reported numbers are off. I am not
>saying
>> >> I would get right numbers when using Chelsio HW iWarp
>(iw_cxgb4),
>> >> but it's closer to reality (using ibv_devinfo <ibname> -vv)
>> >>
>> >> iw_cxgb4 driver:
>> >> ...
>> >> active_width: 4X (2)
>> >> active_speed: 25.0 Gbps (32)
>> >>
>> >> siw driver with your patch:
>> >> ...
>> >> active_width: 4X (2)
>> >> active_speed: 10.0 Gbps (8)
>> >>
>> >> Any idea how we can improve that, maybe coming even
>> >> close to reality (40Gbs)?
>> >
>> >Could you please share the output of ethtool <if_name> for the
>> >underlying
>> >net device that used for both iw_cxgb4 and siw?
>> >
>> H Kamal,
>>
>> Sure! Speed looks correct, and its also what I get
>> at maximum:
>>
>> [bmt@spoke ~]$ ethtool enp1s0f4
>> Settings for enp1s0f4:
>> Supported ports: [ FIBRE ]
>> Supported link modes: 40000baseSR4/Full
>> Supported pause frame use: Symmetric
>> Supports auto-negotiation: Yes
>> Supported FEC modes: None
>> Advertised link modes: 40000baseSR4/Full
>> Advertised pause frame use: Symmetric
>> Advertised auto-negotiation: Yes
>> Advertised FEC modes: None
>> Link partner advertised link modes: 40000baseSR4/Full
>> Link partner advertised pause frame use: Symmetric
>> Link partner advertised auto-negotiation: Yes
>> Link partner advertised FEC modes: None
>> Speed: 40000Mb/s
>> Duplex: Full
>> Port: Direct Attach Copper
>> PHYAD: 255
>> Transceiver: internal
>> Auto-negotiation: on
>> Cannot get wake-on-lan settings: Operation not permitted
>> Current message level: 0x000000ff (255)
>> drv probe link timer ifdown ifup
>rx_err tx_err
>> Link detected: yes
>>
>
>Hi Bernard,
>
>Well, this is the expected value for 40GbE, Because the reported
>value
>is 4X aggregation of FDR10. For more info please the table of speeds
>under [1].
>
>[1] -
>https://urldefense.proofpoint.com/v2/url?u=https-3A__en.wikipedia.org
>_wiki_InfiniBand&d=DwIBAg&c=jf_iaSHvJObTbx-siA1ZOg&r=2TaYXQ0T-r8ZO1PP
>1alNwU_QJcRRLfmYTAgd3QCvqSc&m=Ay2tYZ7Z-SHKRw8UZDCk76kwlZzvkXhRMrO_0jk
>YjcY&s=D4Z0CAH5UVO95WHNnUozLzrqjxRQgVe-2lc8_jwVlhw&e=
>
>Thanks,
>Kamal
>
Hi Kamal, so I have to do the math! 4 x 10Gbs = 40Gbs.
So it is all correct as reported by ibv_devinfio
(and somehow strange what the iw_cxgb4 makes out of it).
Many thanks for the info,
Bernard.
next prev parent reply other threads:[~2020-02-17 14:28 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-13 13:07 [PATH for-next] RDMA/siw: Fix setting active_{speed, width} attributes Kamal Heib
2020-02-13 13:59 ` Bernard Metzler
2020-02-16 13:42 ` Kamal Heib
2020-02-17 10:13 ` Bernard Metzler
2020-02-17 14:11 ` Kamal Heib
2020-02-17 14:27 ` Bernard Metzler [this message]
2020-02-17 14:50 ` Potnuri Bharat Teja
2020-02-17 15:18 ` Bernard Metzler
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=OF2F71177A.B62840BF-ON00258511.004F75BA-00258511.004F75C2@notes.na.collabserv.com \
--to=bmt@zurich.ibm.com \
--cc=dledford@redhat.com \
--cc=jgg@ziepe.ca \
--cc=kamalheib1@gmail.com \
--cc=linux-rdma@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).