Generic interface to make physical port number used by a netdevice available to user space

Generic interface to make physical port number used by a netdevice available to user space

[Narendra_K]

Hello,

It is useful to know if network interfaces from NIC partitions 'map to/ use the'
same physical port. For example,  when creating bonding in fault tolerance mode,
if two network interfaces map to/use the same physical port, it might not have the
desired result. This information is not available today in a standard format or
it is not present.  If this information can be made available in a generic  way
to user space, tools such as NetworkManager or Libteam  or Wicked can make smarter
bonding decisions (such as warn users when setting up configurations which will
not have desired effect).

The requirement is to have a generic interface using which kernel/drivers can
provide information/hints to user space about the physical port number used by
a network interface.

While looking for already existing generic facility, 'dev_id' sysfs attribute
seemed relevant. Looking into the sources seemed to indicate that majority of 
the drivers do not set it and it could be interpreted differently.

It would be great to know list's thoughts on  'dev_id' being used as the interface
to make the physical port number information used by netdevice available to user
space or do we need a new sysfs attribute for the same.

Note: I think in the scenario of SRIOV VF devices assigned to guest and being
bonded, additional information would be needed to differentiate the network
controller in the host.  But I suppose it is a different problem than this.

References to related discussions:
1. [PATCH net-next] bnx2x: Add Nic partitioning mode (57712)
http://marc.info/?l=linux-netdev&m=129098288709528&w=2
2. [PATCH net-next] bonding: Support for multi function NIC devices
http://marc.info/?l=linux-netdev&m=134240221118594&w=2

Thank you,

-- 
With regards,
Narendra K
Linux Engineering
Dell Inc.

Re: Generic interface to make physical port number used by a netdevice available to user space

[Ben Hutchings]

On Wed, 2013-05-22 at 13:24 +0530, Narendra_K@Dell.com wrote:
> Hello,
> 
> It is useful to know if network interfaces from NIC partitions 'map to/ use the'
> same physical port. For example,  when creating bonding in fault tolerance mode,
> if two network interfaces map to/use the same physical port, it might not have the
> desired result. This information is not available today in a standard format or
> it is not present.  If this information can be made available in a generic  way
> to user space, tools such as NetworkManager or Libteam  or Wicked can make smarter
> bonding decisions (such as warn users when setting up configurations which will
> not have desired effect).
> 
> The requirement is to have a generic interface using which kernel/drivers can
> provide information/hints to user space about the physical port number used by
> a network interface.
> 
> While looking for already existing generic facility, 'dev_id' sysfs attribute
> seemed relevant. Looking into the sources seemed to indicate that majority of 
> the drivers do not set it and it could be interpreted differently.

That is what it's for.  Unfortunately it is defined to be 0-based and as
you've seen the default (unknown) value is also 0, creating ambiguity.
(It also seems to be more common for user-facing documentation and
physical labels to use 1-based numbering.)

I wonder whether it would do any harm to make it signed and initialised
it to -1 in alloc_netdev_mqs() would do any harm?  That would make the
unknown case unambiguous.

> It would be great to know list's thoughts on  'dev_id' being used as the interface
> to make the physical port number information used by netdevice available to user
> space or do we need a new sysfs attribute for the same.
> 
> Note: I think in the scenario of SRIOV VF devices assigned to guest and being
> bonded, additional information would be needed to differentiate the network
> controller in the host.  But I suppose it is a different problem than this.

You're thinking about hybrid guest acceleration?  A combination of PCIe
serial number and port number should work.

Ben.

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: Generic interface to make physical port number used by a netdevice available to user space

[Narendra_K]

On Wed, May 22, 2013 at 09:34:18PM +0530, Ben Hutchings wrote:
> 
> On Wed, 2013-05-22 at 13:24 +0530, Narendra_K@Dell.com wrote:
> > While looking for already existing generic facility, 'dev_id' sysfs attribute
> > seemed relevant. Looking into the sources seemed to indicate that majority of
> > the drivers do not set it and it could be interpreted differently.
> 
> That is what it's for.  Unfortunately it is defined to be 0-based and as
> you've seen the default (unknown) value is also 0, creating ambiguity.
> (It also seems to be more common for user-facing documentation and
> physical labels to use 1-based numbering.)
> 
> I wonder whether it would do any harm to make it signed and initialised
> it to -1 in alloc_netdev_mqs() would do any harm?  That would make the
> unknown case unambiguous.
> 
> > It would be great to know list's thoughts on  'dev_id' being used as the
> interface
> > to make the physical port number information used by netdevice available to
> user
> > space or do we need a new sysfs attribute for the same.
> >
> > Note: I think in the scenario of SRIOV VF devices assigned to guest and being
> > bonded, additional information would be needed to differentiate the network
> > controller in the host.  But I suppose it is a different problem than this.
> 
> You're thinking about hybrid guest acceleration?  A combination of PCIe
> serial number and port number should work.

Hi Ben,

Thank you for the response.

I was thinking about the scenario of VF0 and VF1 coming from PF0 in the host
Network Controller 1 being direct assigned to a KVM guest via VTD and netdevices
from VF0 and VF1 being bonded in the guest. Assuming that physical port number used
by VF0 and VF1 is 1, additional information is needed to identify if port number 1
is on Network controller 1 or Network controller 2. (In the host we could use
PCI b/d/f to differentiate two Network Controllers). I think it is similar to
hybrid guest acceleration on the VF assignment aspect.

-- 
With regards,
Narendra K
Linux Engineering
Dell Inc.

Re: Generic interface to make physical port number used by a netdevice available to user space

[Ben Hutchings]

On Thu, 2013-05-23 at 06:27 -0700, Narendra_K@Dell.com wrote:
> On Wed, May 22, 2013 at 09:34:18PM +0530, Ben Hutchings wrote:
> > 
> > On Wed, 2013-05-22 at 13:24 +0530, Narendra_K@Dell.com wrote:
> > > While looking for already existing generic facility, 'dev_id' sysfs attribute
> > > seemed relevant. Looking into the sources seemed to indicate that majority of
> > > the drivers do not set it and it could be interpreted differently.
> > 
> > That is what it's for.  Unfortunately it is defined to be 0-based and as
> > you've seen the default (unknown) value is also 0, creating ambiguity.
> > (It also seems to be more common for user-facing documentation and
> > physical labels to use 1-based numbering.)
> > 
> > I wonder whether it would do any harm to make it signed and initialised
> > it to -1 in alloc_netdev_mqs() would do any harm?  That would make the
> > unknown case unambiguous.
> > 
> > > It would be great to know list's thoughts on  'dev_id' being used as the
> > interface
> > > to make the physical port number information used by netdevice available to
> > user
> > > space or do we need a new sysfs attribute for the same.
> > >
> > > Note: I think in the scenario of SRIOV VF devices assigned to guest and being
> > > bonded, additional information would be needed to differentiate the network
> > > controller in the host.  But I suppose it is a different problem than this.
> > 
> > You're thinking about hybrid guest acceleration?  A combination of PCIe
> > serial number and port number should work.
> 
> Hi Ben,
> 
> Thank you for the response.
> 
> I was thinking about the scenario of VF0 and VF1 coming from PF0 in the host
> Network Controller 1 being direct assigned to a KVM guest via VTD and netdevices
> from VF0 and VF1 being bonded in the guest. Assuming that physical port number used
> by VF0 and VF1 is 1, additional information is needed to identify if port number 1
> is on Network controller 1 or Network controller 2. (In the host we could use
> PCI b/d/f to differentiate two Network Controllers). I think it is similar to
> hybrid guest acceleration on the VF assignment aspect.

OK.  Either way, the hypervisor or management stack will have to help
the guest by providing the identifier(s) to tie the devices together.  I
suggested PCIe serial number as the controller identifier.

Ben.

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: Generic interface to make physical port number used by a netdevice available to user space

[Stephen Hemminger]

On Wed, 22 May 2013 13:24:15 +0530
<Narendra_K@Dell.com> wrote:

> Hello,
> 
> It is useful to know if network interfaces from NIC partitions 'map to/ use the'
> same physical port. For example,  when creating bonding in fault tolerance mode,
> if two network interfaces map to/use the same physical port, it might not have the
> desired result. This information is not available today in a standard format or
> it is not present.  If this information can be made available in a generic  way
> to user space, tools such as NetworkManager or Libteam  or Wicked can make smarter
> bonding decisions (such as warn users when setting up configurations which will
> not have desired effect).
> 
> The requirement is to have a generic interface using which kernel/drivers can
> provide information/hints to user space about the physical port number used by
> a network interface.
> 
> While looking for already existing generic facility, 'dev_id' sysfs attribute
> seemed relevant. Looking into the sources seemed to indicate that majority of 
> the drivers do not set it and it could be interpreted differently.
> 
> It would be great to know list's thoughts on  'dev_id' being used as the interface
> to make the physical port number information used by netdevice available to user
> space or do we need a new sysfs attribute for the same.
> 
> Note: I think in the scenario of SRIOV VF devices assigned to guest and being
> bonded, additional information would be needed to differentiate the network
> controller in the host.  But I suppose it is a different problem than this.
> 
> References to related discussions:
> 1. [PATCH net-next] bnx2x: Add Nic partitioning mode (57712)
> http://marc.info/?l=linux-netdev&m=129098288709528&w=2
> 2. [PATCH net-next] bonding: Support for multi function NIC devices
> http://marc.info/?l=linux-netdev&m=134240221118594&w=2
> 
> Thank you,
> 

One idea would be to reuse the mostly outdated if_port field.
As far as I can see only drivers using it are legacy drivers which support AUI vs. TP
back in the old bad days of BNC Ethernet.

Re: Generic interface to make physical port number used by a netdevice available to user space

[Dan Williams]

On Thu, 2013-05-23 at 17:18 +0100, Ben Hutchings wrote:
> On Thu, 2013-05-23 at 06:27 -0700, Narendra_K@Dell.com wrote:
> > On Wed, May 22, 2013 at 09:34:18PM +0530, Ben Hutchings wrote:
> > > 
> > > On Wed, 2013-05-22 at 13:24 +0530, Narendra_K@Dell.com wrote:
> > > > While looking for already existing generic facility, 'dev_id' sysfs attribute
> > > > seemed relevant. Looking into the sources seemed to indicate that majority of
> > > > the drivers do not set it and it could be interpreted differently.
> > > 
> > > That is what it's for.  Unfortunately it is defined to be 0-based and as
> > > you've seen the default (unknown) value is also 0, creating ambiguity.
> > > (It also seems to be more common for user-facing documentation and
> > > physical labels to use 1-based numbering.)
> > > 
> > > I wonder whether it would do any harm to make it signed and initialised
> > > it to -1 in alloc_netdev_mqs() would do any harm?  That would make the
> > > unknown case unambiguous.
> > > 
> > > > It would be great to know list's thoughts on  'dev_id' being used as the
> > > interface
> > > > to make the physical port number information used by netdevice available to
> > > user
> > > > space or do we need a new sysfs attribute for the same.
> > > >
> > > > Note: I think in the scenario of SRIOV VF devices assigned to guest and being
> > > > bonded, additional information would be needed to differentiate the network
> > > > controller in the host.  But I suppose it is a different problem than this.
> > > 
> > > You're thinking about hybrid guest acceleration?  A combination of PCIe
> > > serial number and port number should work.
> > 
> > Hi Ben,
> > 
> > Thank you for the response.
> > 
> > I was thinking about the scenario of VF0 and VF1 coming from PF0 in the host
> > Network Controller 1 being direct assigned to a KVM guest via VTD and netdevices
> > from VF0 and VF1 being bonded in the guest. Assuming that physical port number used
> > by VF0 and VF1 is 1, additional information is needed to identify if port number 1
> > is on Network controller 1 or Network controller 2. (In the host we could use
> > PCI b/d/f to differentiate two Network Controllers). I think it is similar to
> > hybrid guest acceleration on the VF assignment aspect.
> 
> OK.  Either way, the hypervisor or management stack will have to help
> the guest by providing the identifier(s) to tie the devices together.  I
> suggested PCIe serial number as the controller identifier.

Forgive my ignorance, but is the PCIe serial number anything like the
USB serial number?  Almost nobody sets a unique serial number for USB
devices and often it's all zeros or 0123456789abcdef, so hopefully the
PCIe one is saner.  If not, we shouldn't use it for anything important.

Dan

Re: Generic interface to make physical port number used by a netdevice available to user space

[Ben Hutchings]

On Thu, 2013-05-23 at 13:33 -0500, Dan Williams wrote:
> On Thu, 2013-05-23 at 17:18 +0100, Ben Hutchings wrote:
> > On Thu, 2013-05-23 at 06:27 -0700, Narendra_K@Dell.com wrote:
> > > On Wed, May 22, 2013 at 09:34:18PM +0530, Ben Hutchings wrote:
> > > > 
> > > > On Wed, 2013-05-22 at 13:24 +0530, Narendra_K@Dell.com wrote:
> > > > > While looking for already existing generic facility, 'dev_id' sysfs attribute
> > > > > seemed relevant. Looking into the sources seemed to indicate that majority of
> > > > > the drivers do not set it and it could be interpreted differently.
> > > > 
> > > > That is what it's for.  Unfortunately it is defined to be 0-based and as
> > > > you've seen the default (unknown) value is also 0, creating ambiguity.
> > > > (It also seems to be more common for user-facing documentation and
> > > > physical labels to use 1-based numbering.)
> > > > 
> > > > I wonder whether it would do any harm to make it signed and initialised
> > > > it to -1 in alloc_netdev_mqs() would do any harm?  That would make the
> > > > unknown case unambiguous.
> > > > 
> > > > > It would be great to know list's thoughts on  'dev_id' being used as the
> > > > interface
> > > > > to make the physical port number information used by netdevice available to
> > > > user
> > > > > space or do we need a new sysfs attribute for the same.
> > > > >
> > > > > Note: I think in the scenario of SRIOV VF devices assigned to guest and being
> > > > > bonded, additional information would be needed to differentiate the network
> > > > > controller in the host.  But I suppose it is a different problem than this.
> > > > 
> > > > You're thinking about hybrid guest acceleration?  A combination of PCIe
> > > > serial number and port number should work.
> > > 
> > > Hi Ben,
> > > 
> > > Thank you for the response.
> > > 
> > > I was thinking about the scenario of VF0 and VF1 coming from PF0 in the host
> > > Network Controller 1 being direct assigned to a KVM guest via VTD and netdevices
> > > from VF0 and VF1 being bonded in the guest. Assuming that physical port number used
> > > by VF0 and VF1 is 1, additional information is needed to identify if port number 1
> > > is on Network controller 1 or Network controller 2. (In the host we could use
> > > PCI b/d/f to differentiate two Network Controllers). I think it is similar to
> > > hybrid guest acceleration on the VF assignment aspect.
> > 
> > OK.  Either way, the hypervisor or management stack will have to help
> > the guest by providing the identifier(s) to tie the devices together.  I
> > suggested PCIe serial number as the controller identifier.
> 
> Forgive my ignorance, but is the PCIe serial number anything like the
> USB serial number?  Almost nobody sets a unique serial number for USB
> devices and often it's all zeros or 0123456789abcdef, so hopefully the
> PCIe one is saner.  If not, we shouldn't use it for anything important.

The PCIe serial number is specified as an EUI-64, which is trivially
derived from a MAC address.  So this is relatively easy to get right as
you need to assign unique MAC addresses anyway.  It's also an optional
capability, so there's no particular reason to set a dummy value.

That's not to say no-one ever gets this wrong.  SFC4000-based boards
have the bytes in reverse order.  Other potential mistakes would be
exposing different serial numbers in different functions, or changing
the serial number when the MAC address is temporarily changed.

Ben.

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: Generic interface to make physical port number used by a netdevice available to user space

[Dan Williams]

On Thu, 2013-05-23 at 19:48 +0100, Ben Hutchings wrote:
> On Thu, 2013-05-23 at 13:33 -0500, Dan Williams wrote:
> > On Thu, 2013-05-23 at 17:18 +0100, Ben Hutchings wrote:
> > > On Thu, 2013-05-23 at 06:27 -0700, Narendra_K@Dell.com wrote:
> > > > On Wed, May 22, 2013 at 09:34:18PM +0530, Ben Hutchings wrote:
> > > > > 
> > > > > On Wed, 2013-05-22 at 13:24 +0530, Narendra_K@Dell.com wrote:
> > > > > > While looking for already existing generic facility, 'dev_id' sysfs attribute
> > > > > > seemed relevant. Looking into the sources seemed to indicate that majority of
> > > > > > the drivers do not set it and it could be interpreted differently.
> > > > > 
> > > > > That is what it's for.  Unfortunately it is defined to be 0-based and as
> > > > > you've seen the default (unknown) value is also 0, creating ambiguity.
> > > > > (It also seems to be more common for user-facing documentation and
> > > > > physical labels to use 1-based numbering.)
> > > > > 
> > > > > I wonder whether it would do any harm to make it signed and initialised
> > > > > it to -1 in alloc_netdev_mqs() would do any harm?  That would make the
> > > > > unknown case unambiguous.
> > > > > 
> > > > > > It would be great to know list's thoughts on  'dev_id' being used as the
> > > > > interface
> > > > > > to make the physical port number information used by netdevice available to
> > > > > user
> > > > > > space or do we need a new sysfs attribute for the same.
> > > > > >
> > > > > > Note: I think in the scenario of SRIOV VF devices assigned to guest and being
> > > > > > bonded, additional information would be needed to differentiate the network
> > > > > > controller in the host.  But I suppose it is a different problem than this.
> > > > > 
> > > > > You're thinking about hybrid guest acceleration?  A combination of PCIe
> > > > > serial number and port number should work.
> > > > 
> > > > Hi Ben,
> > > > 
> > > > Thank you for the response.
> > > > 
> > > > I was thinking about the scenario of VF0 and VF1 coming from PF0 in the host
> > > > Network Controller 1 being direct assigned to a KVM guest via VTD and netdevices
> > > > from VF0 and VF1 being bonded in the guest. Assuming that physical port number used
> > > > by VF0 and VF1 is 1, additional information is needed to identify if port number 1
> > > > is on Network controller 1 or Network controller 2. (In the host we could use
> > > > PCI b/d/f to differentiate two Network Controllers). I think it is similar to
> > > > hybrid guest acceleration on the VF assignment aspect.
> > > 
> > > OK.  Either way, the hypervisor or management stack will have to help
> > > the guest by providing the identifier(s) to tie the devices together.  I
> > > suggested PCIe serial number as the controller identifier.
> > 
> > Forgive my ignorance, but is the PCIe serial number anything like the
> > USB serial number?  Almost nobody sets a unique serial number for USB
> > devices and often it's all zeros or 0123456789abcdef, so hopefully the
> > PCIe one is saner.  If not, we shouldn't use it for anything important.
> 
> The PCIe serial number is specified as an EUI-64, which is trivially
> derived from a MAC address.  So this is relatively easy to get right as
> you need to assign unique MAC addresses anyway.  It's also an optional
> capability, so there's no particular reason to set a dummy value.
> 
> That's not to say no-one ever gets this wrong.  SFC4000-based boards
> have the bytes in reverse order.  Other potential mistakes would be
> exposing different serial numbers in different functions, or changing
> the serial number when the MAC address is temporarily changed.

Hmm, that's not making me feel warm and fuzzy inside when thinking about
using this as an identifier of the parent interface.  If you happened to
bond two interfaces and the firmware did change the serial number in
response to a MAC address change, I think we'd be hosed, no?

Dan

Re: Generic interface to make physical port number used by a netdevice available to user space

[Ben Hutchings]

On Thu, 2013-05-23 at 14:05 -0500, Dan Williams wrote:
> On Thu, 2013-05-23 at 19:48 +0100, Ben Hutchings wrote:
> > On Thu, 2013-05-23 at 13:33 -0500, Dan Williams wrote:
> > > On Thu, 2013-05-23 at 17:18 +0100, Ben Hutchings wrote:
> > > > On Thu, 2013-05-23 at 06:27 -0700, Narendra_K@Dell.com wrote:
> > > > > On Wed, May 22, 2013 at 09:34:18PM +0530, Ben Hutchings wrote:
> > > > > > 
> > > > > > On Wed, 2013-05-22 at 13:24 +0530, Narendra_K@Dell.com wrote:
> > > > > > > While looking for already existing generic facility, 'dev_id' sysfs attribute
> > > > > > > seemed relevant. Looking into the sources seemed to indicate that majority of
> > > > > > > the drivers do not set it and it could be interpreted differently.
> > > > > > 
> > > > > > That is what it's for.  Unfortunately it is defined to be 0-based and as
> > > > > > you've seen the default (unknown) value is also 0, creating ambiguity.
> > > > > > (It also seems to be more common for user-facing documentation and
> > > > > > physical labels to use 1-based numbering.)
> > > > > > 
> > > > > > I wonder whether it would do any harm to make it signed and initialised
> > > > > > it to -1 in alloc_netdev_mqs() would do any harm?  That would make the
> > > > > > unknown case unambiguous.
> > > > > > 
> > > > > > > It would be great to know list's thoughts on  'dev_id' being used as the
> > > > > > interface
> > > > > > > to make the physical port number information used by netdevice available to
> > > > > > user
> > > > > > > space or do we need a new sysfs attribute for the same.
> > > > > > >
> > > > > > > Note: I think in the scenario of SRIOV VF devices assigned to guest and being
> > > > > > > bonded, additional information would be needed to differentiate the network
> > > > > > > controller in the host.  But I suppose it is a different problem than this.
> > > > > > 
> > > > > > You're thinking about hybrid guest acceleration?  A combination of PCIe
> > > > > > serial number and port number should work.
> > > > > 
> > > > > Hi Ben,
> > > > > 
> > > > > Thank you for the response.
> > > > > 
> > > > > I was thinking about the scenario of VF0 and VF1 coming from PF0 in the host
> > > > > Network Controller 1 being direct assigned to a KVM guest via VTD and netdevices
> > > > > from VF0 and VF1 being bonded in the guest. Assuming that physical port number used
> > > > > by VF0 and VF1 is 1, additional information is needed to identify if port number 1
> > > > > is on Network controller 1 or Network controller 2. (In the host we could use
> > > > > PCI b/d/f to differentiate two Network Controllers). I think it is similar to
> > > > > hybrid guest acceleration on the VF assignment aspect.
> > > > 
> > > > OK.  Either way, the hypervisor or management stack will have to help
> > > > the guest by providing the identifier(s) to tie the devices together.  I
> > > > suggested PCIe serial number as the controller identifier.
> > > 
> > > Forgive my ignorance, but is the PCIe serial number anything like the
> > > USB serial number?  Almost nobody sets a unique serial number for USB
> > > devices and often it's all zeros or 0123456789abcdef, so hopefully the
> > > PCIe one is saner.  If not, we shouldn't use it for anything important.
> > 
> > The PCIe serial number is specified as an EUI-64, which is trivially
> > derived from a MAC address.  So this is relatively easy to get right as
> > you need to assign unique MAC addresses anyway.  It's also an optional
> > capability, so there's no particular reason to set a dummy value.
> > 
> > That's not to say no-one ever gets this wrong.  SFC4000-based boards
> > have the bytes in reverse order.  Other potential mistakes would be
> > exposing different serial numbers in different functions, or changing
> > the serial number when the MAC address is temporarily changed.
> 
> Hmm, that's not making me feel warm and fuzzy inside when thinking about
> using this as an identifier of the parent interface.  If you happened to
> bond two interfaces and the firmware did change the serial number in
> response to a MAC address change, I think we'd be hosed, no?

If changing MAC address on a VF affects config space on the PF then the
device has *much* bigger problems!

Ben.

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

RE: Generic interface to make physical port number used by a netdevice available to user space

[Narendra_K]

> -----Original Message-----
> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> Sent: Thursday, May 23, 2013 10:04 PM
> To: K, Narendra
> Cc: netdev@vger.kernel.org
> Subject: Re: Generic interface to make physical port number used by a
> netdevice available to user space
> 
> On Wed, 22 May 2013 13:24:15 +0530
> <Narendra_K@Dell.com> wrote:
> > It would be great to know list's thoughts on  'dev_id' being used as
> > the interface to make the physical port number information used by
> > netdevice available to user space or do we need a new sysfs attribute for
> the same.
> >
> > Note: I think in the scenario of SRIOV VF devices assigned to guest
> > and being bonded, additional information would be needed to
> > differentiate the network controller in the host.  But I suppose it is a
> different problem than this.
> >
> > References to related discussions:
> > 1. [PATCH net-next] bnx2x: Add Nic partitioning mode (57712)
> > http://marc.info/?l=linux-netdev&m=129098288709528&w=2
> > 2. [PATCH net-next] bonding: Support for multi function NIC devices
> > http://marc.info/?l=linux-netdev&m=134240221118594&w=2
> >
> > Thank you,
> >
> 
> One idea would be to reuse the mostly outdated if_port field.
> As far as I can see only drivers using it are legacy drivers which support AUI vs.
> TP back in the old bad days of BNC Ethernet.

Hi Stephen, thank you.

I think if_port or dev_id would be fine. The difference seems to be  that dev_id is already exported to sysfs.

With regards,
Narendra K
Linux Engineering
Dell Inc.
 

Re: Generic interface to make physical port number used by a netdevice available to user space

[Narendra_K]

On Mon, May 27, 2013 at 06:39:00PM +0530, Narendra_K@Dell.com wrote:
> 
> > -----Original Message-----
> > From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> > Sent: Thursday, May 23, 2013 10:04 PM
> > To: K, Narendra
> > Cc: netdev@vger.kernel.org
> > One idea would be to reuse the mostly outdated if_port field.
> > As far as I can see only drivers using it are legacy drivers which support
> AUI vs.
> > TP back in the old bad days of BNC Ethernet.
> 
> Hi Stephen, thank you.
> 
> I think if_port or dev_id would be fine. The difference seems to be  that
> dev_id is already exported to sysfs.
> 

I think the following options could be explored -

1. dev_id:
Do not initialize it to -1, but use it to indicate physical port number
used by netdevice.

It is currently being used for indicating physical port number and
is also being used to differentiate OS instances as noted here.
http://marc.info/?l=linux-netdev&m=136992115300526&w=2

2. if_port:

Reuse if_port to indicate physical port number used by a netdevice as
suggested.

A quick look at the sources seems to indicate that it is also
being used to indicate the physical port number and data transfer mode.

Used to indicate physical port number by
	-drivers/net/ethernet/chelsio/cxgb/cxgb2.c
	-drivers/net/ethernet/tehuti/tehuti.c

Used to indicate data transfer mode by
	-drivers/net/ethernet/realtek/atp.c

Also, 'if_port' seems to be available to userspace via 
	-netlink (by rtnl_dump_ifinfo in net/core/rtnetlink.c)
	-SIOCGIFMAP ioctl

3. Add a new field 'physport' to 'struct net_device' and export it to sysfs.

It would be great to know your thoughts on which of the above would be
preferable.

-- 
With regards,
Narendra K
Linux Engineering
Dell Inc.

Re: Generic interface to make physical port number used by a netdevice available to user space

[John Fastabend]

On 5/31/2013 5:13 AM, Narendra_K@Dell.com wrote:
> On Mon, May 27, 2013 at 06:39:00PM +0530, Narendra_K@Dell.com wrote:
>>
>>> -----Original Message-----
>>> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
>>> Sent: Thursday, May 23, 2013 10:04 PM
>>> To: K, Narendra
>>> Cc: netdev@vger.kernel.org
>>> One idea would be to reuse the mostly outdated if_port field.
>>> As far as I can see only drivers using it are legacy drivers which support
>> AUI vs.
>>> TP back in the old bad days of BNC Ethernet.
>>
>> Hi Stephen, thank you.
>>
>> I think if_port or dev_id would be fine. The difference seems to be  that
>> dev_id is already exported to sysfs.
>>
>
> I think the following options could be explored -
>
> 1. dev_id:
> Do not initialize it to -1, but use it to indicate physical port number
> used by netdevice.
>
> It is currently being used for indicating physical port number and
> is also being used to differentiate OS instances as noted here.
> http://marc.info/?l=linux-netdev&m=136992115300526&w=2
>

My opinion would be to use dev_id as the comment in netdevice.h suggests
it can be used for this. I'm not sure the addrconf.c bits are still
valid.

If need be you could fix the check in addrconf to check for not -1.

> 2. if_port:
>
> Reuse if_port to indicate physical port number used by a netdevice as
> suggested.
>

Sure you could use this as well but I got more hits with drivers using
this to indicate the media type. I'm guessing this would break KABI for
someone somewhere, but maybe there is a way to work around it. dev_id
seems like an easier problem to me.

> A quick look at the sources seems to indicate that it is also
> being used to indicate the physical port number and data transfer mode.
>
> Used to indicate physical port number by
> 	-drivers/net/ethernet/chelsio/cxgb/cxgb2.c
> 	-drivers/net/ethernet/tehuti/tehuti.c
>
> Used to indicate data transfer mode by
> 	-drivers/net/ethernet/realtek/atp.c
>
> Also, 'if_port' seems to be available to userspace via
> 	-netlink (by rtnl_dump_ifinfo in net/core/rtnetlink.c)
> 	-SIOCGIFMAP ioctl
>
> 3. Add a new field 'physport' to 'struct net_device' and export it to sysfs.

Probably not we already have two fields that seem ill-defined no reason
to add another to the confusion. If you absolutely can't make dev_id or
if_port coherent then maybe but I really think one of the above two will
work.

>
> It would be great to know your thoughts on which of the above would be
> preferable.
>

Re: Generic interface to make physical port number used by a netdevice available to user space

[Ben Hutchings]

On Fri, 2013-06-07 at 07:59 -0700, John Fastabend wrote:
> On 5/31/2013 5:13 AM, Narendra_K@Dell.com wrote:
> > On Mon, May 27, 2013 at 06:39:00PM +0530, Narendra_K@Dell.com wrote:
> >>
> >>> -----Original Message-----
> >>> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> >>> Sent: Thursday, May 23, 2013 10:04 PM
> >>> To: K, Narendra
> >>> Cc: netdev@vger.kernel.org
> >>> One idea would be to reuse the mostly outdated if_port field.
> >>> As far as I can see only drivers using it are legacy drivers which support
> >> AUI vs.
> >>> TP back in the old bad days of BNC Ethernet.
> >>
> >> Hi Stephen, thank you.
> >>
> >> I think if_port or dev_id would be fine. The difference seems to be  that
> >> dev_id is already exported to sysfs.
> >>
> >
> > I think the following options could be explored -
> >
> > 1. dev_id:
> > Do not initialize it to -1, but use it to indicate physical port number
> > used by netdevice.
> >
> > It is currently being used for indicating physical port number and
> > is also being used to differentiate OS instances as noted here.
> > http://marc.info/?l=linux-netdev&m=136992115300526&w=2
> >
> 
> My opinion would be to use dev_id as the comment in netdevice.h suggests
> it can be used for this. I'm not sure the addrconf.c bits are still
> valid.
> 
> If need be you could fix the check in addrconf to check for not -1.

Having looked at the qeth driver now, I think the comment in netdevice.h
should be changed to state that this is for distinguishing devices that
share a link-layer address, and the drivers using it for other purposes
should stop doing so (if that doesn't break userland).

[...]
> > 3. Add a new field 'physport' to 'struct net_device' and export it to sysfs.
> 
> Probably not we already have two fields that seem ill-defined no reason
> to add another to the confusion. If you absolutely can't make dev_id or
> if_port coherent then maybe but I really think one of the above two will
> work.

I think we should tighten up documentation and implementation of the
existing fields, but there is still a need for this new one.

One thing that needs to be clearly specified is the scope of the
physport identifier - the controller, board, physical machine, ... or
universe.  In a VM, controller and board scope are pretty useless as it
typically can't tell which devices belong to the same controller or
board anyway.  A universally unique identifier is probably not too hard
to implement as there is likely to be at least one MAC address
permanently assigned to each physical port.

Ben.

> >
> > It would be great to know your thoughts on which of the above would be
> > preferable.


-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: Generic interface to make physical port number used by a netdevice available to user space

[John Fastabend]

[...]

> Having looked at the qeth driver now, I think the comment in netdevice.h
> should be changed to state that this is for distinguishing devices that
> share a link-layer address, and the drivers using it for other purposes
> should stop doing so (if that doesn't break userland).
>

Agreed, the qeth driver eluded me, its not in ./drivers/net as you
noted in the other thread.

> [...]
>>> 3. Add a new field 'physport' to 'struct net_device' and export it to sysfs.
>>
>> Probably not we already have two fields that seem ill-defined no reason
>> to add another to the confusion. If you absolutely can't make dev_id or
>> if_port coherent then maybe but I really think one of the above two will
>> work.
>
> I think we should tighten up documentation and implementation of the
> existing fields, but there is still a need for this new one.
>
> One thing that needs to be clearly specified is the scope of the
> physport identifier - the controller, board, physical machine, ... or
> universe.  In a VM, controller and board scope are pretty useless as it
> typically can't tell which devices belong to the same controller or
> board anyway.  A universally unique identifier is probably not too hard
> to implement as there is likely to be at least one MAC address
> permanently assigned to each physical port.
>
> Ben.
>

OK, sounds reasonable to me a new field should work.


-- 
John Fastabend         Intel Corporation

RE: Generic interface to make physical port number used by a netdevice available to user space

[Narendra_K]

> -----Original Message-----
> From: John Fastabend [mailto:john.fastabend@gmail.com]
> Sent: Friday, June 07, 2013 11:05 PM
> To: Ben Hutchings
> Cc: John Fastabend; K, Narendra; stephen@networkplumber.org;
> netdev@vger.kernel.org
> Subject: Re: Generic interface to make physical port number used by a
> netdevice available to user space
> 
> [...]
> 
> > Having looked at the qeth driver now, I think the comment in
> > netdevice.h should be changed to state that this is for distinguishing
> > devices that share a link-layer address, and the drivers using it for
> > other purposes should stop doing so (if that doesn't break userland).
> >
> 
> Agreed, the qeth driver eluded me, its not in ./drivers/net as you noted in
> the other thread.
> 
> > [...]
> >>> 3. Add a new field 'physport' to 'struct net_device' and export it to sysfs.
> >>
> >> Probably not we already have two fields that seem ill-defined no
> >> reason to add another to the confusion. If you absolutely can't make
> >> dev_id or if_port coherent then maybe but I really think one of the
> >> above two will work.
> >
> > I think we should tighten up documentation and implementation of the
> > existing fields, but there is still a need for this new one.
> >
> > One thing that needs to be clearly specified is the scope of the
> > physport identifier - the controller, board, physical machine, ... or
> > universe.  In a VM, controller and board scope are pretty useless as
> > it typically can't tell which devices belong to the same controller or
> > board anyway.  A universally unique identifier is probably not too
> > hard to implement as there is likely to be at least one MAC address
> > permanently assigned to each physical port.
> >
> > Ben.
> >
> 
> OK, sounds reasonable to me a new field should work.

1.  I think a universally unique physport identifier will be very useful.  If 'physport' is unique per NIC (say 0, 1, 2, 3) , then we would need PCI b/d/f to differentiate between two NICs.  But a universally unique identifier would not need any additional information. Also,  it seems like  the VM scenario  will not need any additional information if the VF (netdev) assigned to the guest exposes the unique identifier.

In the scenario of NIC partitions, universally unique identifier generation may have to consider that, each NIC partition which maps to a given physical port will have its own MAC address. (The SRIOV VF network interfaces could probably use PF's MAC address).  But  generation of a universally unique physport identifier would probably be driver specific.

2.  Also, thinking about the default value for 'physport' , could it be set to zero by default  ? A possible  interface could be

0   ==>   driver did not to set the identifier
> 0 ==> driver successfully set the identifier

With regards,
Narendra K
Linux Engineering
Dell Inc.
 

Re: Generic interface to make physical port number used by a netdevice available to user space

[Ben Hutchings]

On Mon, 2013-06-10 at 10:54 -0700, Narendra_K@Dell.com wrote:
[...]
> In the scenario of NIC partitions, universally unique identifier
> generation may have to consider that, each NIC partition which maps to
> a given physical port will have its own MAC address. (The SRIOV VF
> network interfaces could probably use PF's MAC address).  But
> generation of a universally unique physport identifier would probably
> be driver specific.

Right, I agree this has to be driver-specific.

> 2.  Also, thinking about the default value for 'physport' , could it
> be set to zero by default  ? A possible  interface could be
> 
> 0   ==>   driver did not to set the identifier
> > 0 ==> driver successfully set the identifier

I think a universally unique identifier would have to be a multibyte
value like a MAC-48, EUI-64 or 128-bit UUID - not a simple number, and
not a fixed length.  The unset value would then have a *length* of zero.

Ben.

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.