RE: [PATCH v2] eeprom/optoe: driver to read/write SFP/QSFP/CMIS EEPROMS

From: "Don Bollinger" <don@thebollingers.org>
To: "'Andrew Lunn'" <andrew@lunn.ch>
Cc: "'Jakub Kicinski'" <kuba@kernel.org>, <arndb@arndb.de>,
	<gregkh@linuxfoundation.org>, <linux-kernel@vger.kernel.org>,
	<brandon_chuang@edge-core.com>, <wally_wang@accton.com>,
	<aken_liu@edge-core.com>, <gulv@microsoft.com>,
	<jolevequ@microsoft.com>, <xinxliu@microsoft.com>,
	"'netdev'" <netdev@vger.kernel.org>,
	"'Moshe Shemesh'" <moshe@nvidia.com>
Subject: RE: [PATCH v2] eeprom/optoe: driver to read/write SFP/QSFP/CMIS EEPROMS
Date: Fri, 12 Mar 2021 11:04:06 -0800	[thread overview]
Message-ID: <018701d71772$7b0ba3f0$7122ebd0$@thebollingers.org> (raw)
In-Reply-To: <YEL3ksdKIW7cVRh5@lunn.ch>

On Fri, 5 Mar 2021 19:32 -0800 Andrew Lunn wrote:
> > I am proposing acceptance of a commonly used implementation for
> > accessing SFPs because the one used by the netdev/netlink community
> > does not fit the architecture of the white box NOS/switch community.
> 
> Please could you expand on this. I've given suggests as to how the new
> netlink KAPI could be used for this use case, without being attached to a
> netdev. And you have said nothing about why it cannot be made to work.
> You cannot argue the architecture does not fit without actually saying why
it
> does not fit.
> 
>    Andrew

Sorry it took some time to clarify this for myself.  I'm using SONiC (the
NOS
Microsoft uses to run the switches in its Azure cloud) as my example.  They
are users of optoe, and they actually initiated the request to push optoe 
upstream.  Other white box NOS vendors are similar.

SONiC manages all aspects of SFP/QSFP/CMIS interaction through user
space.  They have specified an API that is implemented by switch platform
vendors, that provides things like presence detection, LowPower mode
up/down/status, raw access to EEPROM content, interpretation of EEPROM
content (including TxPower/RxPower/bias, high/low alarm/warning thresholds,
static content like serial number and part number, and dozens of other
items).
This interface is implemented in python scripts provided by the switch
platform
vendor.  Those scripts encode the mapping of CPLD i2c muxes to i2c buses to
port numbers, specific to each switch.

At the bottom of that python stack, all EEPROM access goes through
open/seek/read/close access to the optoe managed file in 
/sys/bus/i2c/devices/{num}-0050/eeprom.

You're not going to like this, but ethtool -e and ethtool -m both return 
' Ethernet0 Cannot get EEPROM data: Operation not supported', for
every port managed by the big switch silicon.

So, my users are using Linux for all the usual Linux things (memory 
management, process management, I/O, IPC, containers), but they don't
use Linux networking to manage the ports that are managed by the big
switch silicon.  (Linux networking is still in use for the management port
that talks to the management processor running Linux.)

optoe provides the device EEPROM foundation for this architecture, but 
requires the sysfs interface (via nvmem) to provide it.  optoe can also
easily
provide the default EEPROM access for the netdev/netlink interface through
the old and new KAPI.

Don