Re: [net-next 10/10] net/mlx5e: Add support for PCI relaxed ordering

[Saeed Mahameed <saeedm@mellanox.com>]

On Thu, 2020-07-09 at 12:47 -0700, Jakub Kicinski wrote:
> On Thu, 9 Jul 2020 15:20:11 -0300 Jason Gunthorpe wrote:
> > >     2) having the driver set RO on the transactions it initiates,
> > > which
> > >        are honored iff the PCI bit is set.
> > > 
> > > It seems that in addition to the PCI core changes, there still is
> > > a need
> > > for driver controls?  Unless the driver always enables RO if it's
> > > capable?  
> > 
> > I think the PCI spec imagined that when the config space RO bit was
> > enabled the PCI device would just start using RO packets, in an
> > appropriate and device specific way.
> > 
> > So the fine grained control in #2 is something done extra by some
> > devices.
> > 
> > IMHO if the driver knows it is functionally correct with RO then it
> > should enable it fully on the device when the config space bit is
> > set.
> > 
> > I'm not sure there is a reason to allow users to finely tune RO, at
> > least I haven't heard of cases where RO is a degredation depending
> > on
> > workload.
> > 
> > If some platform doesn't work when RO is turned on then it should
> > be
> > globally black listed like is already done in some cases.
> > 
> > If the devices has bugs and uses RO wrong, or the driver has bugs
> > and
> > is only stable with !RO and Intel, then the driver shouldn't turn
> > it
> > on at all.
> > 
> > In all of these cases it is not a user tunable.
> > 
> > Development and testing reasons, like 'is my crash from a RO bug?'
> > to
> > tune should be met by the device global setpci, I think.
> 
> +1

Be careful though to load driver with RO on and then setpci RO off.. 
not sure what the side effects are, unstable driver maybe ?
And not sure what should be the procedure then ? reload driver ? FW
will get a notification from PCI ?