Question about the SM Assigned GUID

Question about the SM Assigned GUID

[Greg]

Greetings!

I'm working on an overlay driver for a new type of network controller
that can support traffic similar to infiniband and RDMA.  I've got an IB
driver going that reports the following when I run ibstat:

CA 'lfib0'
	CA type: LF HP HBA
	Number of ports: 1
	Firmware version: 1
	Hardware version: 1
	Node GUID: 0x14db9f33e9fe45b2
	System image GUID: 0x14db9f33e9fe45b2
	Port 1:
		State: Active
		Physical state: LinkUp
		Rate: 10
		Base lid: 2
		LMC: 0
		SM lid: 1
		Capability mask: 0x009c0002
		Port GUID: 0x9eb738f98bf43f02
		Link layer: InfiniBand

However, when I run the opensm subnet manager the log shows this:

[root@fermi ~]# systemctl status opensm -l
● opensm.service - Starts the OpenSM InfiniBand fabric Subnet Manager
   Loaded: loaded (/usr/lib/systemd/system/opensm.service; enabled;
vendor preset: disabled)
   Active: active (running) since Wed 2017-03-15 09:42:02 PDT; 35min ago
     Docs: man:opensm
  Process: 559 ExecStart=/usr/libexec/opensm-launch (code=exited,
status=0/SUCCESS)
 Main PID: 561 (opensm-launch)
   CGroup: /system.slice/opensm.service
           ├─  561 /bin/bash /usr/libexec/opensm-launch
           └─10769 /usr/sbin/opensm -p 0 -g 0x9eb738f98bf43f02
--subnet_prefix 0xfe80000000000000

Mar 15 10:16:33 fermi OpenSM[10769]: -E- Parsing error in field
sm_assigned_guid, value out of range
Mar 15 10:16:33 fermi opensm-launch[559]: -E- Parsing error in field
sm_assigned_guid, value out of range

I've looked at the opensm code and I can't tell where the
sm_assigned_guid comes from or how my driver should be using it.

I apologize for my ignorance but I'm new to IB having come from the
Ethernet world of IWARP and RDMA.  If I could get a tip or a pointer
that would be great.

Thanks for your time,

- Greg
Lightfleet Corp.


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Question about the SM Assigned GUID

[Hal Rosenstock]

Hi Greg,

On 3/15/2017 1:20 PM, Greg wrote:
> Greetings!
> 
> I'm working on an overlay driver for a new type of network controller
> that can support traffic similar to infiniband and RDMA.  I've got an IB
> driver going that reports the following when I run ibstat:
> 
> CA 'lfib0'
> 	CA type: LF HP HBA
> 	Number of ports: 1
> 	Firmware version: 1
> 	Hardware version: 1
> 	Node GUID: 0x14db9f33e9fe45b2
> 	System image GUID: 0x14db9f33e9fe45b2
> 	Port 1:
> 		State: Active
> 		Physical state: LinkUp
> 		Rate: 10
> 		Base lid: 2
> 		LMC: 0
> 		SM lid: 1
> 		Capability mask: 0x009c0002
> 		Port GUID: 0x9eb738f98bf43f02
> 		Link layer: InfiniBand
> 
> However, when I run the opensm subnet manager the log shows this:
> 
> [root@fermi ~]# systemctl status opensm -l
> ● opensm.service - Starts the OpenSM InfiniBand fabric Subnet Manager
>    Loaded: loaded (/usr/lib/systemd/system/opensm.service; enabled;
> vendor preset: disabled)
>    Active: active (running) since Wed 2017-03-15 09:42:02 PDT; 35min ago
>      Docs: man:opensm
>   Process: 559 ExecStart=/usr/libexec/opensm-launch (code=exited,
> status=0/SUCCESS)
>  Main PID: 561 (opensm-launch)
>    CGroup: /system.slice/opensm.service
>            ├─  561 /bin/bash /usr/libexec/opensm-launch
>            └─10769 /usr/sbin/opensm -p 0 -g 0x9eb738f98bf43f02
> --subnet_prefix 0xfe80000000000000
> 
> Mar 15 10:16:33 fermi OpenSM[10769]: -E- Parsing error in field
> sm_assigned_guid, value out of range
> Mar 15 10:16:33 fermi opensm-launch[559]: -E- Parsing error in field
> sm_assigned_guid, value out of range
> 
> I've looked at the opensm code and I can't tell where the
> sm_assigned_guid comes from

It comes from the opensm configuration file. There is parameter as follows:

# SM assigned GUID byte where GUID is formed from OpenFabrics OUI
# followed by 40 bits xy 00 ab cd ef where xy is the SM assigned GUID byte
# and ab cd ef is an SM autogenerated 24 bits
# SM assigned GUID byte should be configured as subnet unique
sm_assigned_guid <some value, defaults to 0>

It is byte field so can be 0-255 (0xff).

The error comes when parsing the sm_assigned_guid parameter in the config file if it is more than 0xff.

To eliminate it, find the config file being used and fix the sm_assigned_guids parameter.

> or how my driver should be using it.

SM assigned GUIDs are part of shared port virtualization where either the host or the SM can assign the alias GUIDs to be used for the VMs. Shared port virtualization is supported on ConnectX-3 HCA. It only comes into play if the host stack (SA client) issues SubAdmSet(GUIDInfoRecord). This is the original virtualization support for IB and there is a new annex for virtualization support.

The error is benign (causes no harm) and I suspect you don't need to worry about this in terms of your driver.

Hope this helps.

-- Hal
 
> I apologize for my ignorance but I'm new to IB having come from the
> Ethernet world of IWARP and RDMA.  If I could get a tip or a pointer
> that would be great.
> 
> Thanks for your time,
> 
> - Greg
> Lightfleet Corp.
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Question about the SM Assigned GUID

[Greg]

On Wed, 2017-03-15 at 13:47 -0400, Hal Rosenstock wrote:
> Hi Greg,
> 
> On 3/15/2017 1:20 PM, Greg wrote:
> > Greetings!
> > 
> > I'm working on an overlay driver for a new type of network controller
> > that can support traffic similar to infiniband and RDMA.  I've got an IB
> > driver going that reports the following when I run ibstat:
> > 
> > CA 'lfib0'
> > 	CA type: LF HP HBA
> > 	Number of ports: 1
> > 	Firmware version: 1
> > 	Hardware version: 1
> > 	Node GUID: 0x14db9f33e9fe45b2
> > 	System image GUID: 0x14db9f33e9fe45b2
> > 	Port 1:
> > 		State: Active
> > 		Physical state: LinkUp
> > 		Rate: 10
> > 		Base lid: 2
> > 		LMC: 0
> > 		SM lid: 1
> > 		Capability mask: 0x009c0002
> > 		Port GUID: 0x9eb738f98bf43f02
> > 		Link layer: InfiniBand
> > 
> > However, when I run the opensm subnet manager the log shows this:
> > 
> > [root@fermi ~]# systemctl status opensm -l
> > ● opensm.service - Starts the OpenSM InfiniBand fabric Subnet Manager
> >    Loaded: loaded (/usr/lib/systemd/system/opensm.service; enabled;
> > vendor preset: disabled)
> >    Active: active (running) since Wed 2017-03-15 09:42:02 PDT; 35min ago
> >      Docs: man:opensm
> >   Process: 559 ExecStart=/usr/libexec/opensm-launch (code=exited,
> > status=0/SUCCESS)
> >  Main PID: 561 (opensm-launch)
> >    CGroup: /system.slice/opensm.service
> >            ├─  561 /bin/bash /usr/libexec/opensm-launch
> >            └─10769 /usr/sbin/opensm -p 0 -g 0x9eb738f98bf43f02
> > --subnet_prefix 0xfe80000000000000
> > 
> > Mar 15 10:16:33 fermi OpenSM[10769]: -E- Parsing error in field
> > sm_assigned_guid, value out of range
> > Mar 15 10:16:33 fermi opensm-launch[559]: -E- Parsing error in field
> > sm_assigned_guid, value out of range
> > 
> > I've looked at the opensm code and I can't tell where the
> > sm_assigned_guid comes from
> 
> It comes from the opensm configuration file. There is parameter as follows:
> 
> # SM assigned GUID byte where GUID is formed from OpenFabrics OUI
> # followed by 40 bits xy 00 ab cd ef where xy is the SM assigned GUID byte
> # and ab cd ef is an SM autogenerated 24 bits
> # SM assigned GUID byte should be configured as subnet unique
> sm_assigned_guid <some value, defaults to 0>
> 
> It is byte field so can be 0-255 (0xff).
> 
> The error comes when parsing the sm_assigned_guid parameter in the config file if it is more than 0xff.
> 
> To eliminate it, find the config file being used and fix the sm_assigned_guids parameter.
> 
> > or how my driver should be using it.
> 
> SM assigned GUIDs are part of shared port virtualization where either the host or the SM can assign the alias GUIDs to be used for the VMs. Shared port virtualization is supported on ConnectX-3 HCA. It only comes into play if the host stack (SA client) issues SubAdmSet(GUIDInfoRecord). This is the original virtualization support for IB and there is a new annex for virtualization support.
> 
> The error is benign (causes no harm) and I suspect you don't need to worry about this in terms of your driver.
> 
> Hope this helps.

Indeed it does!

Thank you for the quick response, much appreciated.

- Greg


> 
> -- Hal
>  
> > I apologize for my ignorance but I'm new to IB having come from the
> > Ethernet world of IWARP and RDMA.  If I could get a tip or a pointer
> > that would be great.
> > 
> > Thanks for your time,
> > 
> > - Greg
> > Lightfleet Corp.
> > 
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > 


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html