From: "Liang, Kan" <kan.liang@linux.intel.com>
To: "Kroening, Gary" <gary.kroening@hpe.com>,
"mingo@redhat.com" <mingo@redhat.com>,
"hpa@zytor.com" <hpa@zytor.com>,
"tglx@linutronix.de" <tglx@linutronix.de>,
"peterz@infradead.org" <peterz@infradead.org>
Cc: "Travis, Mike" <mike.travis@hpe.com>,
"Banman, Andrew" <abanman@hpe.com>,
"Sivanich, Dimitri" <dimitri.sivanich@hpe.com>,
"Anderson, Russ" <russ.anderson@hpe.com>,
"x86@kernel.org" <x86@kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 1/1] x86/platform/x86: Fix count of CHas on multi-pci-segment arches
Date: Tue, 13 Mar 2018 08:28:41 -0400 [thread overview]
Message-ID: <560d1400-92e9-6f38-d699-011f936d0615@linux.intel.com> (raw)
In-Reply-To: <CS1PR84MB0118227E1E8F1257E37C75C3F6D20@CS1PR84MB0118.NAMPRD84.PROD.OUTLOOK.COM>
On 3/13/2018 1:06 AM, Kroening, Gary wrote:
> Thanks, Kan -- your patch looks good, and cleaner than the previous method!
>
> Tonight, I've tested it on our in-house simulator for the following configurations. The simulator has been matching hardware well for this issue -- we'll do more testing on real hardware, but I'm confident you have it right.
>
That's great. Thanks a lot for the verification. :)
Thanks,
Kan
> Terminology:
>
> "hubless" -- equivalent to "glueless" or "white box" where our BIOS sets things up with a single bus for all sockets.
>
> "scalable" -- BIOS assigns a new segment/domain for each socket
>
> Configurations tested so far:
> - single-socket hubless/scalable
> - two sockets, scalable
> - four sockets, hubless
> - eight sockets, scalable
>
> In all cases, skx_count_chabox() is returning 28 for Skylake server.
> Thanks for the quick response!
> Gary
>
>> -----Original Message-----
>> From: Liang, Kan [mailto:kan.liang@linux.intel.com]
>> Sent: Monday, March 12, 2018 8:43 PM
>> To: Kroening, Gary; mingo@redhat.com; hpa@zytor.com; tglx@linutronix.de;
>> peterz@infradead.org
>> Cc: Travis, Mike; Banman, Andrew; Sivanich, Dimitri; Anderson, Russ;
>> x86@kernel.org; linux-kernel@vger.kernel.org
>> Subject: Re: [PATCH 1/1] x86/platform/x86: Fix count of CHas on multi-pci-
>> segment arches
>>
>>
>>
>> On 3/7/2018 3:33 PM, Kroening, Gary wrote:
>>> For systems with a single PCI segment, it is sufficient to look for the
>>> bus number to change in order to determine that all of the CHa's have
>>> been counted for a single socket.
>>>
>>> However, for multi PCI segment systems, each socket is given a new
>>> segment and the bus number does NOT change. So looking only for the
>>> bus number to change ends up counting all of the CHa's on all sockets
>>> in the system. This leads to writing CPU MSRs beyond a valid range and
>>> causes an error in ivbep_uncore_msr_init_box().
>>>
>>> The fix is to check for either the bus number or segment number to
>> change.
>>>
>>
>> Hi Gary,
>>
>> There is a recommended way in uncore document to query the number of
>> CHAs on Skylake server.
>> I have a patch to implement the new way.
>>
>> Could you please take a look at the patch and see if it can fix your
>> issue?
>>
>>
>> Thanks,
>> Kan
>>
>> ------
>> From 55f54b2fa3021c691c2fd4f5cfc8f441fd104e91 Mon Sep 17 00:00:00 2001
>> From: Kan Liang <kan.liang@linux.intel.com>
>> Date: Mon, 12 Mar 2018 13:03:40 -0700
>> Subject: [PATCH] perf/x86/intel/uncore: Querying number of CHAs from
>> CAPID6 register
>>
>> The number of CHAs is miscalculated on multi PCI domain systems on
>> Skylake server
>>
>> (From Kroening, Gary:
>>
>> For systems with a single PCI segment, it is sufficient to look for the
>> bus number to change in order to determine that all of the CHa's have
>> been counted for a single socket.
>> However, for multi PCI segment systems, each socket is given a new
>> segment and the bus number does NOT change. So looking only for the
>> bus number to change ends up counting all of the CHa's on all sockets
>> in the system. This leads to writing CPU MSRs beyond a valid range and
>> causes an error in ivbep_uncore_msr_init_box().)
>>
>> To determine the number of CHAs, it should read bits 27:0 in the CAPID6
>> register located at Device 30, Function 3, Offset 0x9C. These 28 bits
>> form a bit vector of available LLC slices and the CHAs that manage those
>> slices.
>>
>> Fixes: cd34cd97b7b4 ("perf/x86/intel/uncore: Add Skylake server uncore
>> support")
>> Reported-by: Kroening, Gary <gary.kroening@hpe.com>
>> Signed-off-by: Kan Liang <kan.liang@linux.intel.com>
>> ---
>> arch/x86/events/intel/uncore_snbep.c | 24 ++++++++++--------------
>> 1 file changed, 10 insertions(+), 14 deletions(-)
>>
>> diff --git a/arch/x86/events/intel/uncore_snbep.c
>> b/arch/x86/events/intel/uncore_snbep.c
>> index d4672ed..a42715b 100644
>> --- a/arch/x86/events/intel/uncore_snbep.c
>> +++ b/arch/x86/events/intel/uncore_snbep.c
>> @@ -3575,24 +3575,20 @@ static struct intel_uncore_type
>> *skx_msr_uncores[] = {
>> NULL,
>> };
>>
>> +#define SKX_CAPID6 0x9c
>> +#define SKX_CHA_BIT_WIDTH 28
>> +
>> static int skx_count_chabox(void)
>> {
>> - struct pci_dev *chabox_dev = NULL;
>> - int bus, count = 0;
>> + struct pci_dev *dev = NULL;
>> + u32 val = 0;
>>
>> - while (1) {
>> - chabox_dev = pci_get_device(PCI_VENDOR_ID_INTEL, 0x208d,
>> chabox_dev);
>> - if (!chabox_dev)
>> - break;
>> - if (count == 0)
>> - bus = chabox_dev->bus->number;
>> - if (bus != chabox_dev->bus->number)
>> - break;
>> - count++;
>> - }
>> + dev = pci_get_device(PCI_VENDOR_ID_INTEL, 0x2083, dev);
>> + if (!dev)
>> + return 0;
>>
>> - pci_dev_put(chabox_dev);
>> - return count;
>> + pci_read_config_dword(dev, SKX_CAPID6, &val);
>> + return bitmap_weight((unsigned long *)&val, SKX_CHA_BIT_WIDTH);
>> }
>>
>> void skx_uncore_cpu_init(void)
>> --
>> 2.7.4
>
next prev parent reply other threads:[~2018-03-13 12:28 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-03-07 20:33 [PATCH 1/1] x86/platform/x86: Fix count of CHas on multi-pci-segment arches Kroening, Gary
2018-03-13 1:42 ` Liang, Kan
2018-03-13 5:06 ` Kroening, Gary
2018-03-13 12:28 ` Liang, Kan [this message]
2018-03-13 5:24 ` Kroening, Gary
2018-03-13 15:58 ` Andy Shevchenko
2018-03-13 16:00 ` Andy Shevchenko
2018-03-13 17:15 ` Liang, Kan
2018-03-13 17:22 ` Andy Shevchenko
2018-03-13 17:28 ` Liang, Kan
2018-03-13 17:31 ` Andy Shevchenko
2018-03-13 17:16 ` Liang, Kan
2018-03-13 17:42 ` Liang, Kan
2018-03-13 17:46 ` Andy Shevchenko
2018-03-13 17:49 ` Andy Shevchenko
2018-03-13 17:50 ` Andy Shevchenko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=560d1400-92e9-6f38-d699-011f936d0615@linux.intel.com \
--to=kan.liang@linux.intel.com \
--cc=abanman@hpe.com \
--cc=dimitri.sivanich@hpe.com \
--cc=gary.kroening@hpe.com \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mike.travis@hpe.com \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=russ.anderson@hpe.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).