iblinkinfo for Python

iblinkinfo for Python

[Benjamin Drung]

Hi,

we have an in-house Shell script that uses iblinkinfo to check if the
InfiniBand cabling is correct. This information can be derived from the
node names that can be seen for the HCA port. I want to improve that
check and rewrite it in Python, but I failed to find an easy and robust
way to retrieve the node names for a HCA port:

1) Call "iblinkinfo --line" and parse the output. Parsing the output
could probably be done with a complex regular expression. This solution
is too ugly IMO.

2) Extend iblinkinfo to provide a JSON output. Then let the Python
script call "iblinkinfo --json" and simply use json.load for parsing.
This solution requires some C coding and probably a good json library
should be used to avoid generating bogus JSON.

3) Use https://github.com/jgunthorpe/python-rdma but this library has
not been touched for five years and needs porting to Python 3. So that
is probably a lot of work as well.

4) Use pyverbs provided by rdma-core, but I found neither a single API
call to query similar data to iblinkinfo, nor an example for that use
case.

What should I do?

-- 
Benjamin Drung

Senior DevOps Engineer and Debian & Ubuntu Developer
Compute Platform Operations Cloud

IONOS SE | Revaler Str. 30 | 10245 Berlin | Deutschland
E-Mail: benjamin.drung@ionos.com | Web: www.ionos.de

Hauptsitz Montabaur, Amtsgericht Montabaur, HRB 24498

Vorstand: Hüseyin Dogan, Dr. Martin Endreß, Claudia Frese, Henning
Kettler, Arthur Mai, Britta Schmidt, Achim Weiß
Aufsichtsratsvorsitzender: Markus Kadelke


Member of United Internet

Re: iblinkinfo for Python

[Leon Romanovsky]

On Wed, Jan 05, 2022 at 11:32:40AM +0100, Benjamin Drung wrote:
> Hi,
> 
> we have an in-house Shell script that uses iblinkinfo to check if the
> InfiniBand cabling is correct. This information can be derived from the
> node names that can be seen for the HCA port. I want to improve that
> check and rewrite it in Python, but I failed to find an easy and robust
> way to retrieve the node names for a HCA port:
> 
> 1) Call "iblinkinfo --line" and parse the output. Parsing the output
> could probably be done with a complex regular expression. This solution
> is too ugly IMO.
> 
> 2) Extend iblinkinfo to provide a JSON output. Then let the Python
> script call "iblinkinfo --json" and simply use json.load for parsing.
> This solution requires some C coding and probably a good json library
> should be used to avoid generating bogus JSON.
> 
> 3) Use https://github.com/jgunthorpe/python-rdma but this library has
> not been touched for five years and needs porting to Python 3. So that
> is probably a lot of work as well.
> 
> 4) Use pyverbs provided by rdma-core, but I found neither a single API
> call to query similar data to iblinkinfo, nor an example for that use
> case.
> 
> What should I do?

Isn't this information available in sysfs?
[leonro@mtl-leonro-l-vm ~]$ cat /sys/class/infiniband/ibp0s9/node_desc
mtl-leonro-l-vm ibp0s9

Can you give an example?

Thanks

> 
> -- 
> Benjamin Drung
> 
> Senior DevOps Engineer and Debian & Ubuntu Developer
> Compute Platform Operations Cloud
> 
> IONOS SE | Revaler Str. 30 | 10245 Berlin | Deutschland
> E-Mail: benjamin.drung@ionos.com | Web: www.ionos.de
> 
> Hauptsitz Montabaur, Amtsgericht Montabaur, HRB 24498
> 
> Vorstand: Hüseyin Dogan, Dr. Martin Endreß, Claudia Frese, Henning
> Kettler, Arthur Mai, Britta Schmidt, Achim Weiß
> Aufsichtsratsvorsitzender: Markus Kadelke
> 
> 
> Member of United Internet
> 

Re: iblinkinfo for Python

[Benjamin Drung]

Am Mittwoch, dem 05.01.2022 um 13:52 +0200 schrieb Leon Romanovsky:
> On Wed, Jan 05, 2022 at 11:32:40AM +0100, Benjamin Drung wrote:
> > Hi,
> > 
> > we have an in-house Shell script that uses iblinkinfo to check if the
> > InfiniBand cabling is correct. This information can be derived from
> > the
> > node names that can be seen for the HCA port. I want to improve that
> > check and rewrite it in Python, but I failed to find an easy and
> > robust
> > way to retrieve the node names for a HCA port:
> > 
> > 1) Call "iblinkinfo --line" and parse the output. Parsing the output
> > could probably be done with a complex regular expression. This
> > solution
> > is too ugly IMO.
> > 
> > 2) Extend iblinkinfo to provide a JSON output. Then let the Python
> > script call "iblinkinfo --json" and simply use json.load for parsing.
> > This solution requires some C coding and probably a good json library
> > should be used to avoid generating bogus JSON.
> > 
> > 3) Use https://github.com/jgunthorpe/python-rdma but this library has
> > not been touched for five years and needs porting to Python 3. So
> > that
> > is probably a lot of work as well.
> > 
> > 4) Use pyverbs provided by rdma-core, but I found neither a single
> > API
> > call to query similar data to iblinkinfo, nor an example for that use
> > case.
> > 
> > What should I do?
> 
> Isn't this information available in sysfs?
> [leonro@mtl-leonro-l-vm ~]$ cat /sys/class/infiniband/ibp0s9/node_desc
> mtl-leonro-l-vm ibp0s9

The host names of the nodes connected to this port are required for
this check, not the host name of the host itself.

> Can you give an example?

```
$ sudo iblinkinfo -y 1 -C mlx5_0 -P 0 --line
0x04400C64D87A2543 "                  host1 mlx5_0"     13    [...]
0x04400C64D87A255c "                  host2 mlx5_0"     10    [...]
[...]
```

iblinkinfo would return the host names of all connected hosts (in this
example snippet "host1" and "host2"). In our inhouse case, we can
derive the topology from these host names.
> 

-- 
Benjamin Drung

Senior DevOps Engineer and Debian & Ubuntu Developer
Compute Platform Operations Cloud

IONOS SE | Revaler Str. 30 | 10245 Berlin | Deutschland
E-Mail: benjamin.drung@ionos.com | Web: www.ionos.de

Hauptsitz Montabaur, Amtsgericht Montabaur, HRB 24498

Vorstand: Hüseyin Dogan, Dr. Martin Endreß, Claudia Frese, Henning
Kettler, Arthur Mai, Britta Schmidt, Achim Weiß
Aufsichtsratsvorsitzender: Markus Kadelke


Member of United Internet

Re: iblinkinfo for Python

[Leon Romanovsky]

On Wed, Jan 05, 2022 at 01:05:08PM +0100, Benjamin Drung wrote:
> Am Mittwoch, dem 05.01.2022 um 13:52 +0200 schrieb Leon Romanovsky:
> > On Wed, Jan 05, 2022 at 11:32:40AM +0100, Benjamin Drung wrote:
> > > Hi,
> > > 
> > > we have an in-house Shell script that uses iblinkinfo to check if the
> > > InfiniBand cabling is correct. This information can be derived from
> > > the
> > > node names that can be seen for the HCA port. I want to improve that
> > > check and rewrite it in Python, but I failed to find an easy and
> > > robust
> > > way to retrieve the node names for a HCA port:

<...>

> > > What should I do?

<...>

The iblinkinfo receives this information through umad interface, so if I
would be you, I would write the simplified version of iblinkinfo.

Thanks

Re: iblinkinfo for Python

[Jens Domke]

Hi Benjamin,

I have written some simple parser for output of certain IB commands:
https://gitlab.com/domke/ParseInfiniband

Maybe it will help you, since it includes iblinkinfo and is in py3.

Best,
  Jens

On 1/5/22 19:32, Benjamin Drung wrote:
> Hi,
> 
> we have an in-house Shell script that uses iblinkinfo to check if the
> InfiniBand cabling is correct. This information can be derived from the
> node names that can be seen for the HCA port. I want to improve that
> check and rewrite it in Python, but I failed to find an easy and robust
> way to retrieve the node names for a HCA port:
> 
> 1) Call "iblinkinfo --line" and parse the output. Parsing the output
> could probably be done with a complex regular expression. This solution
> is too ugly IMO.
> 
> 2) Extend iblinkinfo to provide a JSON output. Then let the Python
> script call "iblinkinfo --json" and simply use json.load for parsing.
> This solution requires some C coding and probably a good json library
> should be used to avoid generating bogus JSON.
> 
> 3) Use https://github.com/jgunthorpe/python-rdma but this library has
> not been touched for five years and needs porting to Python 3. So that
> is probably a lot of work as well.
> 
> 4) Use pyverbs provided by rdma-core, but I found neither a single API
> call to query similar data to iblinkinfo, nor an example for that use
> case.
> 
> What should I do?
>