From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030616AbdIZQAg (ORCPT ); Tue, 26 Sep 2017 12:00:36 -0400 Received: from mail-sn1nam02on0085.outbound.protection.outlook.com ([104.47.36.85]:10496 "EHLO NAM02-SN1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S935993AbdIZQAe (ORCPT ); Tue, 26 Sep 2017 12:00:34 -0400 Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Vadim.Lomovtsev@cavium.com; Date: Tue, 26 Sep 2017 09:00:26 -0700 From: Vadim Lomovtsev To: Alex Williamson , bhelgaas@google.com Cc: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Wilson.Snyder@cavium.com, jcm@redhat.com, Vadim.Lomovtsev@cavium.com Subject: Re: [PATCH v5] PCI: quirks: update Cavium ThunderX ACS quirk implementation Message-ID: <20170926160026.GA26639@localhost.localdomain> References: <1505724481-28413-1-git-send-email-Vadim.Lomovtsev@caviumnetworks.com> <1506344920-24016-1-git-send-email-Vadim.Lomovtsev@caviumnetworks.com> <20170926094343.3e0d8d90@w520.home> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170926094343.3e0d8d90@w520.home> User-Agent: Mutt/1.6.1 (2016-04-27) X-Originating-IP: [50.233.148.156] X-ClientProxiedBy: SN4PR0801CA0020.namprd08.prod.outlook.com (10.161.215.158) To BN6PR07MB2993.namprd07.prod.outlook.com (10.172.106.11) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 5898635d-c29c-4918-ab1c-08d504f7b6f4 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001)(2017030254152)(2017052603199)(201703131423075)(201703031133081)(201702281549075);SRVR:BN6PR07MB2993; X-Microsoft-Exchange-Diagnostics: 1;BN6PR07MB2993;3:R2EM6Z6r6bZl78H28mnHw2Ozx4eIE7YTtdhYG0GwOla86Wv1k862g1HTrIzaFim3eK+WoPhewxUPLrM5cLBehRXmCLQdF9HT9/0crNie3fhEWFAcp48+3eHY16OFpZQQlcD7zsFsUp99xokYJpr8IiXIeXivMGdzzF15JIxxvrUtuQTvasEe49e0IHlx1hOcwG8ZJa4kOIngMmpSqju/+dlO/cnKC/n2Pp99Wk72KJRLG1r0VeZyCOI0HqPp98QA;25:eeGPsdhMWWeAqqqa/IyZyx8i0+0q5RjsowPElU1QAHuqnAFgSVSjcrB8LvD9a3D1o4g1/msEWXkZhmm/epp2ScB6lJDcexR3yUs81COrpk7jQ7UFlTq8I/665nvCwNjFGpMAl0PyHWsWb20eE5CY3zGB3H46ig4pAdcsmqrjxRzv+M7A0rMja9GkTXjD8JlSzbjzfXX9OF2MR/iaZUlugzgWaI6S80fLxh+4COScABevfI4ORvwv2s9Z2VoqURfbKxGqHVbeBWqMYKL+t8Mj/GQgu7x9GPx4e9yLtEzK1i842V8ZWza8GuA3KmfLIjtLvbqeqlGE3HRHpERdq588fA==;31:PeExkNp5SfAd8c/mLMmoJu9Po5CQyXPwpuSP7EFhmtjkcxc3kRThkQFuR7xK6La2+yGzcSN6VmhebvqaXS3wGHzDeW39EkuxN9WdH6PCrdDXrYp/k+eJXNQfwXRnytYm7XkbUTTyTgkH1idnBAjKVvywivcK0cDlql8V2C12fD/FwTOKykW2ozESNVYK7pmFJSIEj1vLgC07C2jAwGlfPcvGvF4IbI44XBiyUMi+BCI= X-MS-TrafficTypeDiagnostic: BN6PR07MB2993: X-Microsoft-Exchange-Diagnostics: 1;BN6PR07MB2993;20:7Se41K87kyGvOmupU9qoFWX4hW6Urk8cZyCkfVR4jsf5mMtv15u3zX0tsj4cQXthJY8givZL9G+i1tiFkzFe8Z+CteV+2x+cv5OGSqMASCUAdEzpGYa+Rdq6K9vxisHjMhmgNGLaQFJUQhj1NgoIdeq7VqqxJVFLBACENco6f5eRznfyka6hUq/565Z3jecY4psavJ//nkFrFtl+U23Qb1jiYQYKqwEaWEnZnrScI6mDHmm+wbXeA8JFYjqWfGwPmwHIk8EfBkyGtL5iiE4dquEYqUbtN+mhOu6Bp7AiE4bxlpiUmZoA7J/YUx2mmIiqepvwNrkI3SsKqbio98/J8svydY8CziLhtbkLeSrg9Ylu5VUihCJ1KZu0UQAzJkbaAefz93c+HuzSLNKboJKOPVLsmAAI242IVy/+pZBGS/UkwW4glaOBJ28GPAYtDrZfQj+Wo+ZXWM6C1+2Y4LQym0zFOZIrwaGW8SS22Lkv5KVnpgxAGGEhdz7lTQT01zWk1n0MyIQE2lW+NrCfQOcSosNIXcJLbjjZLosv08hVM0SSMDqcEh21jR/pWKCQXENGzyi6U6RT/fuy46DajymeZn2zDY2TeAI16YeWYcQQ+b4=;4:SfnE4uXDAgQfh1vwkw6JxrAXb8gIOAhX+vXK80RT88XRkrthz3QhGl7WODCkR5g3u/O6nZc4CPRtMegjGnxaDOmGhN97O+wLVMmWh9A8fl87mpxHFTn6MMQ23i/fGS//ZbNyVIPVvIQP6FPZbMWmrm02r/WJP5E1NIsxP8VTBKYVY+RfmDXNWeRo3aqNHMJZZ6d5AYOOd6w94YERxhjFVuNgE/CMJ6kfD49UEqEUchn4Rg4gZz0FuYNlx5AeW4FE X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(93006095)(3002001)(100000703101)(100105400095)(10201501046)(6041248)(20161123558100)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123562025)(20161123564025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:BN6PR07MB2993;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:BN6PR07MB2993; X-Forefront-PRVS: 0442E569BC X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(6069001)(6009001)(376002)(346002)(39830400002)(52284002)(199003)(189002)(24454002)(61506002)(8936002)(4326008)(107886003)(15650500001)(575784001)(6666003)(97736004)(6246003)(106356001)(7736002)(189998001)(16526017)(2950100002)(58126008)(345774005)(1076002)(83506001)(81156014)(9686003)(316002)(16586007)(23726003)(6116002)(42882006)(68736007)(3846002)(33656002)(101416001)(53936002)(55016002)(54356999)(8676002)(5660300001)(76176999)(66066001)(47776003)(50986999)(50466002)(72206003)(229853002)(105586002)(305945005)(25786009)(81166006)(6506006)(2906002)(478600001)(18370500001);DIR:OUT;SFP:1101;SCL:1;SRVR:BN6PR07MB2993;H:localhost.localdomain;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;BN6PR07MB2993;23:PNSAV3oNSuG608a0N5YX7tNj47Nb+aMiA1O994a+B?= =?us-ascii?Q?dZ6yYy6KH1AzBuNU+WoqYDTS2RWsCN1RSEX672+ngR5YEZOxXkE+76+Fl84R?= =?us-ascii?Q?dtsaexuVldMw5qeiH+QmBE/aLya8v/jzpRv3QdR5soeP0ikStgHzgIKlUVoA?= =?us-ascii?Q?wdnQHyEB2CUW5c1SizUn8Be4qTQ0ZOmoVK0TCmeeQsRktXx8NKKVKATKzim8?= =?us-ascii?Q?XllQ7aVNTNjnUX/rwFEdhvAJC+LdgXVX0XdJIZ1wXOl0CU07piFOksSpVIAP?= =?us-ascii?Q?6VQk/KEk7rgzywQTfksUcf0r7fDx/atQEHPJC9GFQmG4e2sn2QbPk160Afkk?= =?us-ascii?Q?CL/E6+S27vdf288JyMAb7jbFXZLELFmEp6rI8Fb2TxKm+xj1ltEdGUpAKvE9?= =?us-ascii?Q?1SP+vRIvdO3TZGeVfVMIcGadauwcl0gHIwUH9AXaJU065NHz6eyVxnDQcXfJ?= =?us-ascii?Q?QGOGRbJ8U7bmC9pcVWLeblPAPN2B5pbpiTrBetzEirCuNdW0zl4VNLgD3WCP?= =?us-ascii?Q?wGbOxwxgqgUFm/bG90xXVfSCsud34AfyJ7BfwV3VHCWh9Tr6rJ+LuEVL5dzn?= =?us-ascii?Q?yswiej1QKJYp0co5Y9p36WTwAKCsCWViwSqZN92CGIYk9rRzxMuVw3N4/+AQ?= =?us-ascii?Q?DOcWMJGlryqKU9sI0yi6cDCM62zHgvokdFOgwCnDqXJyI66ojJj+H6FXw7nS?= =?us-ascii?Q?lwGyyR+QvtCZyoW8ORR2vOPO9zYhYC6VjrntGq+duP6ScQIxkbOG4DHaVg77?= =?us-ascii?Q?eSDTE2hDY0U7CIlumYBUAlf0mBD89BZm9/V+37akpdKRiF4O6DyC7uh7bYMI?= =?us-ascii?Q?iKgW/5u8WnQ1BYZlBrlL4jj415705DiRvRLKTRSYOLJCCDjzRH8DZZL3ASnO?= =?us-ascii?Q?ct/oyKtZMiQBTsAHJJP2ZXuhY6m9GUC9PaaWEplL1re62w464vy7s2c18Neo?= =?us-ascii?Q?fyAHweKu+emZtLIaNpqSIBQL2pTtiHDqQymqfTGH7eXiHV20lrlwOQ8ANVKp?= =?us-ascii?Q?8lxLqp6SNaDVn6DDAHfQTEOapYKMVS8WWOwAnbT8azetoOAM3CeSJYAHc4/U?= =?us-ascii?Q?BKdVOlVeOJcFI4zraArdh3ZYS5PT4HAIFcxXMa1Mg2SVuj+tIjcF1x/sXjIn?= =?us-ascii?Q?SltiNyW0CiQ4dl/XvKZCfzxdWb2P7VZ+eYUNaD22ctnWk04JN4833lNtunb3?= =?us-ascii?Q?8OOh+sBLCFthtGH1gnAjkcq/wowLm+e0IETxWEnUq/uD7EErDExhMa1/96bN?= =?us-ascii?Q?ZGiVjm8v1aZRcAXMLXObCSvM1p4Y1Ho3H7sB/Z+TgeX+V97nGY57G7BXRLVe?= =?us-ascii?Q?xm70INF59H3DJfYd9DFJo1j4hLBQCAfu+mKV+qIaUgek2Pi1WOhXZLQxMDnN?= =?us-ascii?Q?pmESQ=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;BN6PR07MB2993;6:ZZgGP+SS2IGtq8igPFbc08kR7Adre6o40AKmWf70UurwXdnmJfBwryj8oqX9pXMS6feksVnnoK4jFIkGsApk95SGTqWRiL8BNcrTJN6rz2YscklIgw+GBaESv6nAYXs/p/iXAfgxa3YuwmOlZTkGJM3YfsnNyEJyJza5JdIKVglhB7EsJw8ExPO5i3SQxfcLhOnUOaKDx0sMFJysTwlzRm0apm21TRyIJBGTwOuPAVCXcoxx/UwN4CAa0uc677MZbFrv9HxfEGdF0SLGS2uFEj30zzGIZ8PZdWPNYWNEPZLgoAqvuEmrdChXedvWcxClCQfvPNxzNfX1Zbp5dhZiAw==;5:NH1HHwmXuA6DiJk9cKb505Nk6CkYUj7wduXO/56nDwVAFa9zuWSqpV34iMMoXUni3qEzCp8tAqVSq91qnLJI8ARKEyMd0XFpT4gonCyLzriCQaHT8nqs9d+aPwkgpTBuheHWMe4mv4T5FYCh2fJKdw==;24:sBhWE1lI5vcezV7e/YywJsHhZ4gAGNcYQwkDibEemUEWnJKW7WJTXDAJYkGr7C48YXGSzWnqLIBHD7E3KDW28pIzqPU/uScAmWTBLzHSuvs=;7:hrwxktfCkHJmLJ232adi1uCvq8YV6BKEaqyMvA7o5OIjMta1jv2rYQKAJDQSl+cKcWW6tmaDw7H3JnGjuHS6aHhxV50C4F8mywnqP/1WLIFEqSaqni2Bs74oOfABhlTHHZEAN4OwzHAY9bAi+9X71M00ALgeCjBFLP3LkDkex61G9L0mYnVZvWONx6J4XyGVqK5M8KwirClPyXbdYi5Z9Cp7fdflUNCW/fKCLqdR+Gw= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2017 16:00:30.6651 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR07MB2993 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Sep 26, 2017 at 09:43:43AM -0600, Alex Williamson wrote: > On Mon, 25 Sep 2017 06:08:40 -0700 > Vadim Lomovtsev wrote: > > > This commit makes Cavium PCI ACS quirk applicable only to Cavium > > ThunderX (CN8XXX) family PCIE Root Ports which has limited PCI capabilities > > in terms of no ACS support advertisement. However, the RTL internally > > implements similar protection as if ACS had completion/request redirection, > > upstream forwarding and validation features enabled. > > > > Current quirk implementation doesn't take into account PCIERCs which > > also needs to be quirked. So the pci device id check mask is updated > > and check of device ID moved into separate function. > > > > Signed-off-by: Vadim Lomovtsev > > --- > > drivers/pci/quirks.c | 29 +++++++++++++++++++++-------- > > 1 file changed, 21 insertions(+), 8 deletions(-) > > > Reviewed-by: Alex Williamson > > Thanks for making the updates > Thank you for your patience and for your time, Alex. Bjorn, Would you mind to pick up this patch ? WBR, Vadim > > > diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c > > index a4d3361..0fd2e15 100644 > > --- a/drivers/pci/quirks.c > > +++ b/drivers/pci/quirks.c > > @@ -4211,20 +4211,33 @@ static int pci_quirk_amd_sb_acs(struct pci_dev *dev, u16 acs_flags) > > #endif > > } > > > > -static int pci_quirk_cavium_acs(struct pci_dev *dev, u16 acs_flags) > > +/* > > + * The Cavium downstream ports doesn't advertise their ACS capability registers. > > + * However, the RTL internally implements similar protection as if > > + * ACS had completion redirection, forwarding and validation features enabled. > > + * So by this flags we're asserting that the hardware implements and > > + * enables equivalent ACS functionality for these flags. > > + */ > > +#define CAVIUM_CN8XXX_ACS_FLAGS (PCI_ACS_RR | PCI_ACS_CR | PCI_ACS_SV | PCI_ACS_UF) > > + > > +static __inline__ bool pci_quirk_cavium_acs_match(struct pci_dev *dev) > > { > > /* > > - * Cavium devices matching this quirk do not perform peer-to-peer > > - * with other functions, allowing masking out these bits as if they > > - * were unimplemented in the ACS capability. > > + * Effectively selects all downstream ports for whole ThunderX 1 family > > + * by 0xa00 mask (which represents 8 SoCs), while the lower bits of device ID > > + * are used to indicate which subdevice is used within the SoC. > > */ > > - acs_flags &= ~(PCI_ACS_SV | PCI_ACS_TB | PCI_ACS_RR | > > - PCI_ACS_CR | PCI_ACS_UF | PCI_ACS_DT); > > + return (pci_is_pcie(dev) && > > + (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) && > > + ((dev->device & 0xf800) == 0xa000)); > > +} > > > > - if (!((dev->device >= 0xa000) && (dev->device <= 0xa0ff))) > > +static int pci_quirk_cavium_acs(struct pci_dev *dev, u16 acs_flags) > > +{ > > + if (!pci_quirk_cavium_acs_match(dev)) > > return -ENOTTY; > > > > - return acs_flags ? 0 : 1; > > + return acs_flags & ~(CAVIUM_CN8XXX_ACS_FLAGS) ? 0 : 1; > > } > > > > static int pci_quirk_xgene_acs(struct pci_dev *dev, u16 acs_flags) >