From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751528AbdGZTF5 (ORCPT ); Wed, 26 Jul 2017 15:05:57 -0400 Received: from mail-dm3nam03on0134.outbound.protection.outlook.com ([104.47.41.134]:45408 "EHLO NAM03-DM3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751089AbdGZTFy (ORCPT ); Wed, 26 Jul 2017 15:05:54 -0400 From: Casey Leedom To: Alexander Duyck CC: Netdev , Bjorn Helgaas , "linux-arm-kernel@lists.infradead.org" , "David.Laight@aculab.com" , "ashok.raj@intel.com" , "Alex Williamson" , "l.stach@pengutronix.de" , "Suravee.Suthikulpanit@amd.com" , "catalin.marinas@arm.com" , "linux-pci@vger.kernel.org" , "will.deacon@arm.com" , Sinan Kaya , "robin.murphy@arm.com" , "linux-kernel@vger.kernel.org" , "davem@davemloft.net" , Ganesh GR , "asit.k.mallick@intel.com" , "jeffrey.t.kirsher@intel.com" , "mark.rutland@arm.com" , "gabriele.paoloni@huawei.com" , Michael Werner , "bhelgaas@google.com" , "patrick.j.cramer@intel.com" , Ding Tianhong , "linuxarm@huawei.com" , "amira@mellanox.com" , "Bob.Shaw@amd.com" Subject: Re: [PATCH v7 2/3] PCI: Enable PCIe Relaxed Ordering if supported Thread-Topic: [PATCH v7 2/3] PCI: Enable PCIe Relaxed Ordering if supported Thread-Index: AQHS++OWrGKhofvtIUCheprjRIXqU6JSQPmAgABHnoCAANEhgIAL8fcAgAPZDQCAA1q+dYAAB1V9gAAA30I= Date: Wed, 26 Jul 2017 19:05:49 +0000 Message-ID: References: <1499955692-26556-1-git-send-email-dingtianhong@huawei.com> <1499955692-26556-3-git-send-email-dingtianhong@huawei.com> <0260e398-bd8e-6615-6d5c-1f7c07b6fc09@huawei.com> <67be791f-e0cf-8284-9229-17174dc741ef@codeaurora.org> <5f9b8bfb-41a8-a17c-6fea-581aec1d5573@huawei.com> <20170724090516.2e0f0d2a@w520.home> , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=leedom@chelsio.com; x-originating-ip: [12.32.117.8] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;MWHPR12MB1438;7:/hmR/RRslgylzY7uAP4hWlRuw7fhKaRaT8eg70dZyPk4FGYsWmQ3QTISLwbuAl+pCai74Av9aAwPY9VDnOkj8kFs7VyWjFUML6U2WaDiFeO5MKAQ6JnXQDgmjAtA4nq934kxBMkt3mJ2W9BaepKaSt2j+m/WBqZgVC12z3v66O5KOgKnVTUkXqivPxneCoxYx2HuIz8e1GPg21onXeGeqUpyUR78VbCL9SCBcpK2OaHRZd02yspO5hhaoFWL8O6nrs5N1MwVT6aBd+BZ4bL3Rko1ju1xIb3AA23JeJlaQfaMWmWfgIK9V2uhDEj8iOG8ts1a1EfflZgaspdfzCHgEu/k5KsWPWPMjM9fkdYURblxhp04CrD7WXrnYGi+o/GFcaqh5r330y/woln6wqUI5R2UsEx2ZxIVjgQzj6yvB+L8VIUwukhFSc07tLxqZ4agCn7exc71kcJ2m3AgxfyA9nLYeTN5F7+Bv1pVeEfFfV2x3AUGlla23bETZodUVSeHG1gfV5rgQrAWS1aAtodLLov+o2IVT/7we/s+5JII8uj0LzHW/jxof0OgrdDqBt9E5aOUbNqzBuHEUPXrPd4xSDtv/Ne1OQvXDh8/HA5ORHMArTaK2H8znlRicYqtmshS/YKoLrsxcdtFZwXyVOgTs03KFiJ7BO1CT+uiZV+xNm3o6xvLcWxCjAEeTzwBzhaRn542tIRWoXJ+kio/wR3SHXpvY0KMgWYsiYqxCd1xLrDsQPnoZYMN8fbIY1XbHEXlL6RhaFwVwa2f3tKY+LhMmJ8IOyfeItWyqKBKVok+Cs0= x-ms-office365-filtering-correlation-id: b9d10302-3142-4499-d70d-08d4d459548e x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254075)(300000503095)(300135400095)(2017052603031)(201703131423075)(201703031133081)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);SRVR:MWHPR12MB1438; x-ms-traffictypediagnostic: MWHPR12MB1438: x-exchange-antispam-report-test: UriScan:(68840517438536); x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(100000703101)(100105400095)(93006095)(93001095)(10201501046)(3002001)(920511095)(6041248)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123564025)(20161123560025)(20161123558100)(20161123562025)(20161123555025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:MWHPR12MB1438;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:MWHPR12MB1438; x-forefront-prvs: 038002787A x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(6009001)(39400400002)(39450400003)(39410400002)(39840400002)(199003)(24454002)(51444003)(43784003)(377454003)(189002)(305945005)(6506006)(7736002)(7696004)(86362001)(106356001)(8936002)(8676002)(68736007)(81166006)(2900100001)(81156014)(8666007)(74316002)(105586002)(77096006)(93886004)(66066001)(229853002)(6916009)(7416002)(102836003)(25786009)(97736004)(110136004)(5660300001)(4326008)(54356999)(2950100002)(189998001)(6246003)(53936002)(33656002)(3660700001)(2906002)(6116002)(3846002)(3280700002)(53546010)(50986999)(76176999)(39060400002)(478600001)(99286003)(54906002)(6436002)(14454004)(9686003)(101416001)(55016002)(38730400002)(17260700006);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR12MB1438;H:MWHPR12MB1600.namprd12.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" MIME-Version: 1.0 X-OriginatorOrg: chelsio.com X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Jul 2017 19:05:49.7293 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 065db76d-a7ae-4c60-b78a-501e8fc17095 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1438 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by nfs id v6QJ61hK018569 | From: Alexander Duyck | Sent: Wednesday, July 26, 2017 11:44 AM | | On Jul 26, 2017 11:26 AM, "Casey Leedom" wrote: | | | |     I think that the patch will need to be extended to modify | |     drivers/pci.c/iov.c:sriov_enable() to explicitly turn off | |     Relaxed Ordering Enable if the Root Complex is marked |     for no RO TLPs. | | I'm not sure that would be an issue. Wouldn't most VFs inherit the PF's settings? Ah yes, you're right. This is covered in section 3.5.4 of the Single Root I/O Virtualization and Sharing Specification, Revision 1.0 (September 11, 2007), governing the PCIe Capability Device Control register. It states that the VF version of that register shall follow the setting of the corresponding PF. So we should enhance the cxgb4vf/sge.c:t4vf_sge_alloc_rxq() in the same way we did for the cxgb4 driver, but that's not critical since the Relaxed Ordering Enable supersedes the internal chip's desire to use the Relaxed Ordering Attribute. Ding, send me a note if you'd like me to work that up for you. | Also I thought most of the VF configuration space is read only. Yes, but not all of it. And when a VF is exported to a Virtual Machine, then the Hypervisor captures and interprets all accesses to the VF's PCIe Configuration Space from the VM. Thanks again for reminding me of the subtle aspect of the SR_IOV specification that I forgot. Casey