netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Vadim Pasternak <vadimp@nvidia.com>
To: Andrew Lunn <andrew@lunn.ch>, Jiri Pirko <jiri@resnulli.us>
Cc: David Ahern <dsahern@gmail.com>, Jakub Kicinski <kuba@kernel.org>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	"davem@davemloft.net" <davem@davemloft.net>,
	"jacob.e.keller@intel.com" <jacob.e.keller@intel.com>,
	Roopa Prabhu <roopa@nvidia.com>, mlxsw <mlxsw@nvidia.com>
Subject: RE: [patch net-next RFC 00/10] introduce line card support for modular switch
Date: Fri, 29 Jan 2021 16:45:59 +0000	[thread overview]
Message-ID: <DM6PR12MB389878422F910221DB296DC2AFB99@DM6PR12MB3898.namprd12.prod.outlook.com> (raw)
In-Reply-To: <YBQujIdnFtEhWqTF@lunn.ch>



> -----Original Message-----
> From: Andrew Lunn <andrew@lunn.ch>
> Sent: Friday, January 29, 2021 5:50 PM
> To: Jiri Pirko <jiri@resnulli.us>
> Cc: David Ahern <dsahern@gmail.com>; Jakub Kicinski <kuba@kernel.org>;
> netdev@vger.kernel.org; davem@davemloft.net; jacob.e.keller@intel.com;
> Roopa Prabhu <roopa@nvidia.com>; mlxsw <mlxsw@nvidia.com>; Vadim
> Pasternak <vadimp@nvidia.com>
> Subject: Re: [patch net-next RFC 00/10] introduce line card support for
> modular switch
> 
> On Fri, Jan 29, 2021 at 08:20:15AM +0100, Jiri Pirko wrote:
> > Thu, Jan 28, 2021 at 03:17:13PM CET, andrew@lunn.ch wrote:
> > >On Thu, Jan 28, 2021 at 09:14:34AM +0100, Jiri Pirko wrote:
> > >> Wed, Jan 27, 2021 at 03:14:34PM CET, andrew@lunn.ch wrote:
> > >> >> >There are Linux standard APIs for controlling the power to
> > >> >> >devices, the regulator API. So i assume mlxreg-pm will make use
> > >> >> >of that. There are also standard APIs for thermal management,
> > >> >> >which again, mlxreg-pm should be using. The regulator API
> > >> >> >allows you to find regulators by name. So just define a
> > >> >> >sensible naming convention, and the switch driver can lookup the
> regulator, and turn it on/off as needed.
> > >> >>
> > >> >>
> > >> >> I don't think it would apply. The thing is, i2c driver has a
> > >> >> channel to the linecard eeprom, from where it can read info
> > >> >> about the linecard. The i2c driver also knows when the linecard is
> plugged in, unlike mlxsw.
> > >> >> It acts as a standalone driver. Mlxsw has no way to directly
> > >> >> find if the card was plugged in (unpowered) and which type it is.
> > >> >>
> > >> >> Not sure how to "embed" it. I don't think any existing API could help.
> > >> >> Basicall mlxsw would have to register a callback to the i2c
> > >> >> driver called every time card is inserted to do auto-provision.
> > >> >> Now consider a case when there are multiple instances of the
> > >> >> ASIC on the system. How to assemble a relationship between mlxsw
> > >> >> instance and i2c driver instance?
> > >> >
> > >> >You have that knowledge already, otherwise you cannot solve this
> > >>
> > >> No I don't have it. I'm not sure why do you say so. The mlxsw and
> > >> i2c driver act independently.
> > >
> > >Ah, so you just export some information in /sys from the i2c driver?
> > >And you expect the poor user to look at the values, and copy paste
> > >them to the correct mlxsw instance? 50/50 guess if you have two
> > >switches, and hope they don't make a typO?
> >
> > Which values are you talking about here exactly?
> 
> The i2c driver tells you what line card is actually inserted.
> Hopefully it interprets the EEPROM and gives the user a nice string. You then
> need to use this string to provision the switch, so it knows what line card has
> been inserted. Or the user can pre-prevision, make a guess as to what card will
> actually be inserted sometime in the future, tell the switch, and hope that
> actually happens.

Hi Andrew,

mlxsw I2C driver is BMC side driver. Its purpose to provide hwmon,
thermal, QSFP info for the chassis management at BMC side.
It works on top of PRM interface and it is associated with the chip I2C
slave device.
It doesn't aware of system topology, it knows nothing about system I2C
tree, what is EEPROM, where it located and so on. This is not a scope of
this driver.

Platform line card driver is aware of line card I2C topology, its
responsibility is to detect line card basic hardware type, create I2C
topology (mux), connect all the necessary I2C devices, like hotswap
devices, voltage and power regulators devices, iio/a2d devices and line
card EEPROMs, creates LED instances for LED located on a line card, exposes
line card related attributes, like CPLD and FPGA versions, reset causes,
required powered through line card hwmon interface.

> 
> > >> >I still don't actually get this use case. Why would i want to
> > >> >manually provision?
> > >>
> > >> Because user might want to see the system with all netdevices,
> > >> configure them, change the linecard if they got broken and all
> > >> config, like bridge, tc, etc will stay on the netdevices. Again,
> > >> this is the same we do for split port. This is important
> > >> requirement, user don't want to see netdevices come and go when he
> > >> is plugging/unplugging cables. Linecards are the same in this
> > >> matter. Basically is is a "splitter module", replacing the "splitter cable"
> > >
> > >So, what is the real use case here? Why might the user want to do
> > >this?
> > >
> > >Is it: The magic smoke has escaped. The user takes a spare switch,
> > >and wants to put it on her desk to configure it where she has a comfy
> > >chair and piece and quiet, unlike in the data centre, which is very
> > >noise, only has hard plastic chair, no coffee allowed. She makes her
> > >best guess at the configuration, up/downs the interfaces, reboots, to
> > >make sure it is permanent, and only then moves to the data centre to
> > >swap the dead router for the new one, and fix up whatever
> > >configuration errors there are, while sat on the hard chair?
> > >
> > >So this feature is about comfy chair vs hard chair?
> >
> > I don't really get the question, but configuring switch w/o any
> > linecard and plug the linecards in later on is definitelly a usecase.
> 
> It is a requirement, not a use case.
> 
> A use case is the big picture, what is the user doing, at the big picture level. In
> the somewhat absurd example given above, the user case is, the router chassis
> has died, but they think the line cards are O.K. They want to do as much
> configuration and testing as possible before going into the data center to
> actually replace the chassis.  By reusing the existing line cards, they reduce the
> risk of getting the cables plugged into the wrong port.
> 
> From the use case, you can derive the requirements. In order to test that ifup --
> all puts the IP addresses in the right places, it needs to have the netdevs with
> the correct names. Either they need line cards in the router, or they need to be
> able to fake line cards. There is also a requirement that the line cards are easy
> to exchange. They do not need to be fully hot-plugable, since one router is
> dead, the other can be powered off. But ideally you want simple thumb
> screws, not a Philips screwdriver or an allan key. There is also a requirement
> that this provision is persistent, since the user is likely to reboot the system in
> order to test the configurations files actually work at boot time. Either the
> switch driver needs to write the information to FLASH, or user space needs to
> tell it on every boot, a systemd service file or similar.
> 
> But since you have not been able to answer my question, i wonder if
> everything is backwards around here. Your architecture is broken, you cannot
> easily determine what line card is inserted, so you need a workaround,
> provisioning. But provision might actually be a useful feature, so lets try to sell
> the feature, and gloss over that the architecture is broken.
> 
> So, i would like to see the architecture fixed first. The switch driver somehow
> talks to the i2c driver to find out what card is in the slot, and configures itself.
> My guess is, every other switch does this, this is what the user expects as a
> base feature, it is what we want Linux to do by default.
> 
> You can later add provisioning, where if the slot is empty, you can fake a line
> card, to fulfil the use cases. And when the slot is actually filled, you can verify
> what is plugged in matches what was expected, and be very noise if not.
> 
> 	  Andrew

  parent reply	other threads:[~2021-01-29 16:48 UTC|newest]

Thread overview: 80+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-13 12:12 [patch net-next RFC 00/10] introduce line card support for modular switch Jiri Pirko
2021-01-13 12:12 ` [patch net-next RFC 01/10] devlink: add support to create line card and expose to user Jiri Pirko
2021-01-15 15:47   ` Ido Schimmel
2021-01-13 12:12 ` [patch net-next RFC 02/10] devlink: implement line card provisioning Jiri Pirko
2021-01-15 16:03   ` Ido Schimmel
2021-01-15 16:51     ` Jiri Pirko
2021-01-15 18:09       ` Ido Schimmel
2021-01-18 12:50         ` Jiri Pirko
2021-01-13 12:12 ` [patch net-next RFC 03/10] devlink: implement line card active state Jiri Pirko
2021-01-15 16:06   ` Ido Schimmel
2021-01-15 16:52     ` Jiri Pirko
2021-01-13 12:12 ` [patch net-next RFC 04/10] devlink: append split port number to the port name Jiri Pirko
2021-01-13 12:12 ` [patch net-next RFC 05/10] devlink: add port to line card relationship set Jiri Pirko
2021-01-15 16:10   ` Ido Schimmel
2021-01-15 16:53     ` Jiri Pirko
2021-01-13 12:12 ` [patch net-next RFC 06/10] netdevsim: introduce line card support Jiri Pirko
2021-01-13 12:12 ` [patch net-next RFC 07/10] netdevsim: allow port objects to be linked with line cards Jiri Pirko
2021-01-13 12:12 ` [patch net-next RFC 08/10] netdevsim: create devlink line card object and implement provisioning Jiri Pirko
2021-01-15 16:30   ` Ido Schimmel
2021-01-15 16:54     ` Jiri Pirko
2021-01-13 12:12 ` [patch net-next RFC 09/10] netdevsim: implement line card activation Jiri Pirko
2021-01-13 12:12 ` [patch net-next RFC 10/10] selftests: add netdevsim devlink lc test Jiri Pirko
2021-01-13 13:39 ` [patch iproute2/net-next RFC] devlink: add support for linecard show and provision Jiri Pirko
2021-01-14  2:07 ` [patch net-next RFC 00/10] introduce line card support for modular switch Andrew Lunn
2021-01-14  7:39   ` Jiri Pirko
2021-01-14 22:56     ` Jacob Keller
2021-01-15 14:19       ` Jiri Pirko
2021-01-19 11:56   ` Jiri Pirko
2021-01-19 14:51     ` Andrew Lunn
2021-01-20  8:36       ` Jiri Pirko
2021-01-20 13:56         ` Andrew Lunn
2021-01-20 23:41           ` Jakub Kicinski
2021-01-21  0:01             ` Andrew Lunn
2021-01-21  0:16               ` Jakub Kicinski
2021-01-21 15:34               ` Jiri Pirko
2021-01-21 15:32             ` Jiri Pirko
2021-01-21 16:38               ` David Ahern
2021-01-22  7:28                 ` Jiri Pirko
2021-01-22 14:13                   ` Andrew Lunn
2021-01-26 11:33                     ` Jiri Pirko
2021-01-26 13:56                       ` Andrew Lunn
2021-01-27  7:57                         ` Jiri Pirko
2021-01-27 14:14                           ` Andrew Lunn
2021-01-27 14:57                             ` David Ahern
2021-01-28  8:14                             ` Jiri Pirko
2021-01-28 14:17                               ` Andrew Lunn
2021-01-29  7:20                                 ` Jiri Pirko
     [not found]                                   ` <YBQujIdnFtEhWqTF@lunn.ch>
2021-01-29 16:45                                     ` Vadim Pasternak [this message]
2021-01-29 17:31                                       ` Andrew Lunn
2021-01-30 14:19                                         ` Jiri Pirko
     [not found]                                           ` <251d1e12-1d61-0922-31f8-a8313f18f194@gmail.com>
2021-02-01  8:16                                             ` Jiri Pirko
2021-02-01 13:41                                               ` Andrew Lunn
2021-02-03 14:57                                                 ` Jiri Pirko
2021-02-03 16:26                                                   ` Andrew Lunn
2021-02-01  1:43                                       ` Andrew Lunn
2021-01-22  8:05                 ` Jiri Pirko
2021-01-19 16:23     ` David Ahern
2021-01-20  8:37       ` Jiri Pirko
2021-01-14  2:27 ` Jakub Kicinski
2021-01-14  7:48   ` Jiri Pirko
2021-01-14 23:30     ` Jakub Kicinski
2021-01-15 14:39       ` Jiri Pirko
2021-01-15 19:26         ` Jakub Kicinski
2021-01-18 13:00           ` Jiri Pirko
2021-01-18 17:59             ` Jakub Kicinski
2021-01-19 11:51               ` Jiri Pirko
2021-01-18 22:55             ` David Ahern
2021-01-22  8:01               ` Jiri Pirko
2021-01-14 22:58   ` Jacob Keller
2021-01-14 23:20     ` Jakub Kicinski
2021-01-15 14:40       ` Jiri Pirko
2021-01-15 15:43 ` Ido Schimmel
2021-01-15 16:55   ` Jiri Pirko
2021-01-15 18:01     ` Ido Schimmel
2021-01-18 13:03       ` Jiri Pirko
2021-01-18 18:01 ` Edwin Peer
2021-01-18 22:57   ` David Ahern
2021-01-18 23:40     ` Edwin Peer
2021-01-19  2:39       ` David Ahern
2021-01-19  5:06         ` Edwin Peer

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=DM6PR12MB389878422F910221DB296DC2AFB99@DM6PR12MB3898.namprd12.prod.outlook.com \
    --to=vadimp@nvidia.com \
    --cc=andrew@lunn.ch \
    --cc=davem@davemloft.net \
    --cc=dsahern@gmail.com \
    --cc=jacob.e.keller@intel.com \
    --cc=jiri@resnulli.us \
    --cc=kuba@kernel.org \
    --cc=mlxsw@nvidia.com \
    --cc=netdev@vger.kernel.org \
    --cc=roopa@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).