Linux-PCI Archive on lore.kernel.org
 help / color / Atom feed
From: Aya Levin <ayal@mellanox.com>
To: Jakub Kicinski <kuba@kernel.org>,
	Saeed Mahameed <saeedm@mellanox.com>,
	"David S. Miller" <davem@davemloft.net>
Cc: "mkubecek@suse.cz" <mkubecek@suse.cz>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"helgaas@kernel.org" <helgaas@kernel.org>,
	"davem@davemloft.net" <davem@davemloft.net>,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	Tariq Toukan <tariqt@mellanox.com>,
	"alexander.h.duyck@linux.intel.com\"" 
	<alexander.h.duyck@linux.intel.com>
Subject: Re: [net-next 10/10] net/mlx5e: Add support for PCI relaxed ordering
Date: Mon, 6 Jul 2020 16:00:59 +0300
Message-ID: <7b79eead-ceab-5d95-fd91-cabeeef82d6a@mellanox.com> (raw)
In-Reply-To: <20200624133018.5a4d238b@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>



On 6/24/2020 11:30 PM, Jakub Kicinski wrote:
> On Wed, 24 Jun 2020 20:15:14 +0000 Saeed Mahameed wrote:
>> On Wed, 2020-06-24 at 10:22 -0700, Jakub Kicinski wrote:
>>> On Wed, 24 Jun 2020 10:34:40 +0300 Aya Levin wrote:
>>>>>> I think Michal will rightly complain that this does not belong
>>>>>> in
>>>>>> private flags any more. As (/if?) ARM deployments take a
>>>>>> foothold
>>>>>> in DC this will become a common setting for most NICs.
>>>>>
>>>>> Initially we used pcie_relaxed_ordering_enabled() to
>>>>>    programmatically enable this on/off on boot but this seems to
>>>>> introduce some degradation on some Intel CPUs since the Intel
>>>>> Faulty
>>>>> CPUs list is not up to date. Aya is discussing this with Bjorn.
>>>> Adding Bjorn Helgaas
>>>
>>> I see. Simply using pcie_relaxed_ordering_enabled() and blacklisting
>>> bad CPUs seems far nicer from operational perspective. Perhaps Bjorn
>>> will chime in. Pushing the validation out to the user is not a great
>>> solution IMHO.
>>
>> Can we move on with this patch for now ? since we are going to keep the
>> user knob anyway, what is missing is setting the default value
>> automatically but this can't be done until we
>> fix pcie_relaxed_ordering_enabled()
> 
> If this patch was just adding a chicken bit that'd be fine, but opt in
> I'm not hugely comfortable with. Seems like Bjorn has provided some
> assistance already on the defaults but there doesn't appear to be much
> progress being made.

Hi Jakub, Dave

Assuming the discussions with Bjorn will conclude in a well-trusted API 
that ensures relaxed ordering in enabled, I'd still like a method to 
turn off relaxed ordering for performance debugging sake.
Bjorn highlighted the fact that the PCIe sub system can only offer a 
query method. Even if theoretically a set API will be provided, this 
will not fit a netdev debugging - I wonder if CPU vendors even support 
relaxed ordering set/unset...
On the driver's side relaxed ordering is an attribute of the mkey and 
should be available for configuration (similar to number of CPU vs. 
number of channels).
Based on the above, and binding the driver's default relaxed ordering to 
the return value from pcie_relaxed_ordering_enabled(), may I continue 
with previous direction of a private-flag to control the client side (my 
driver) ?

Aya.
> 

  parent reply index

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20200623195229.26411-1-saeedm@mellanox.com>
     [not found] ` <20200623195229.26411-11-saeedm@mellanox.com>
     [not found]   ` <20200623143118.51373eb7@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
     [not found]     ` <dda5c2b729bbaf025592aa84e2bdb84d0cda7570.camel@mellanox.com>
     [not found]       ` <082c6bfe-5146-c213-9220-65177717c342@mellanox.com>
2020-06-24 17:22         ` Jakub Kicinski
2020-06-24 20:15           ` Saeed Mahameed
     [not found]             ` <20200624133018.5a4d238b@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
2020-07-06 13:00               ` Aya Levin [this message]
2020-07-06 16:52                 ` Jakub Kicinski
2020-07-06 19:49                 ` David Miller
2040-07-08  8:22                   ` Aya Levin
2020-07-08 23:16                     ` Bjorn Helgaas
2020-07-08 23:26                       ` Jason Gunthorpe
2020-07-09 17:35                         ` Jonathan Lemon
2020-07-09 18:20                           ` Jason Gunthorpe
2020-07-09 19:47                             ` Jakub Kicinski
2020-07-10  2:18                               ` Saeed Mahameed
2020-07-10 12:21                                 ` Jason Gunthorpe
2020-07-09 20:33                             ` Jonathan Lemon
2020-07-14 10:47                       ` Aya Levin
2020-07-23 21:03                     ` Alexander Duyck
2020-06-26 20:12           ` Bjorn Helgaas
2020-06-26 20:24             ` David Miller
2020-06-29  9:32             ` Aya Levin
2020-06-29 19:33               ` Bjorn Helgaas
2020-06-29 19:57                 ` Raj, Ashok
2020-06-30  7:32                   ` Ding Tianhong
2020-07-05 11:15                     ` Aya Levin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=7b79eead-ceab-5d95-fd91-cabeeef82d6a@mellanox.com \
    --to=ayal@mellanox.com \
    --cc=alexander.h.duyck@linux.intel.com \
    --cc=davem@davemloft.net \
    --cc=helgaas@kernel.org \
    --cc=kuba@kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=mkubecek@suse.cz \
    --cc=netdev@vger.kernel.org \
    --cc=saeedm@mellanox.com \
    --cc=tariqt@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-PCI Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-pci/0 linux-pci/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-pci linux-pci/ https://lore.kernel.org/linux-pci \
		linux-pci@vger.kernel.org
	public-inbox-index linux-pci

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-pci


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git