From: Yazen Ghannam <yazen.ghannam@amd.com>
To: Borislav Petkov <bp@alien8.de>
Cc: Naveen Krishna Chatradhi <nchatrad@amd.com>,
linux-edac@vger.kernel.org, x86@kernel.org,
linux-kernel@vger.kernel.org, mchehab@kernel.org,
Muralidhara M K <muralimk@amd.com>
Subject: Re: [PATCH v3 3/3] EDAC/amd64: Enumerate memory on noncpu nodes
Date: Wed, 1 Sep 2021 18:42:26 +0000 [thread overview]
Message-ID: <YS/JkgWA8VreIx1R@yaz-ubuntu> (raw)
In-Reply-To: <YSjM8b9vvkmRew94@zn.tnic>
On Fri, Aug 27, 2021 at 01:30:57PM +0200, Borislav Petkov wrote:
> On Tue, Aug 24, 2021 at 12:24:37AM +0530, Naveen Krishna Chatradhi wrote:
...
> > @@ -1335,6 +1392,11 @@ static void determine_memory_type(struct amd64_pvt *pvt)
> > u32 dram_ctrl, dcsm;
> >
> > if (pvt->umc) {
> > + if (pvt->is_noncpu) {
> > + pvt->dram_type = MEM_HBM2;
> > + return;
> > + }
>
> I don't like this sprinkling of "if (pvt->is_noncpu)" everywhere,
> at all. Please define a separate read_mc_regs_df() or so which
> contains only the needed functionalty which you can carve out from
> read_mc_regs().
>
I like this idea.
> > +
> > if ((pvt->umc[0].dimm_cfg | pvt->umc[1].dimm_cfg) & BIT(5))
> > pvt->dram_type = MEM_LRDDR4;
> > else if ((pvt->umc[0].dimm_cfg | pvt->umc[1].dimm_cfg) & BIT(4))
> > @@ -1724,7 +1786,10 @@ static int f17_early_channel_count(struct amd64_pvt *pvt)
> >
> > /* SDP Control bit 31 (SdpInit) is clear for unused UMC channels */
> > for_each_umc(i)
> > - channels += !!(pvt->umc[i].sdp_ctrl & UMC_SDP_INIT);
> > + if (pvt->is_noncpu)
> > + channels += pvt->csels[i].b_cnt;
> > + else
> > + channels += !!(pvt->umc[i].sdp_ctrl & UMC_SDP_INIT);
> >
> > amd64_info("MCT channel count: %d\n", channels);
> >
>
> No, a separate gpu_early_channel_count() is needed here. There's a
> reason for those function pointers getting assigned depending on family.
>
Good point.
...
> > +/*
> > + * The CPUs have one channel per UMC, So a UMC number is equivalent to a
> > + * channel number. The NONCPUs have 8 channels per UMC, so the UMC number no
> > + * longer works as a channel number.
> > + * The channel number within a NONCPU UMC is given in MCA_IPID[15:12].
> > + * However, the IDs are split such that two UMC values go to one UMC, and
> > + * the channel numbers are split in two groups of four.
> > + *
> > + * Refer comment on get_noncpu_umc_base() from amd64_edac.h
> > + *
> > + * For example,
> > + * UMC0 CH[3:0] = 0x0005[3:0]000
> > + * UMC0 CH[7:4] = 0x0015[3:0]000
> > + * UMC1 CH[3:0] = 0x0025[3:0]000
> > + * UMC1 CH[7:4] = 0x0035[3:0]000
> > + */
> > +static int find_umc_channel_noncpu(struct mce *m)
> > +{
> > + u8 umc = find_umc_channel(m);
> > + u8 ch = ((m->ipid >> 12) & 0xf);
> > +
> > + return umc % 2 ? (ch + 4) : ch;
> > +}
> > +
> > static void decode_umc_error(int node_id, struct mce *m)
> > {
> > u8 ecc_type = (m->status >> 45) & 0x3;
> > @@ -2897,6 +3003,7 @@ static void decode_umc_error(int node_id, struct mce *m)
> > struct amd64_pvt *pvt;
> > struct err_info err;
> > u64 sys_addr;
> > + u8 df_inst_id;
>
> You don't need that variable and can work with err.channel just fine.
>
> > mci = edac_mc_find(node_id);
> > if (!mci)
> > @@ -2909,7 +3016,22 @@ static void decode_umc_error(int node_id, struct mce *m)
> > if (m->status & MCI_STATUS_DEFERRED)
> > ecc_type = 3;
> >
> > - err.channel = find_umc_channel(m);
> > + if (pvt->is_noncpu) {
> > + /*
> > + * The NONCPUs have one Chip Select per UMC, so the UMC number
> > + * can used as the Chip Select number. However, the UMC number
> > + * is split in the ID value so it's necessary to divide by 2.
> > + */
> > + err.csrow = find_umc_channel(m) / 2;
> > + err.channel = find_umc_channel_noncpu(m);
> > + /* On NONCPUs, instance id is calculated as below. */
> > + df_inst_id = err.csrow * 8 + err.channel;
>
> err.channel += err.csrow * 8;
>
> tadaaa!
>
err.channel still needs to be used in error_address_to_page_and_offset()
below. So changing it here messes up what's reported to EDAC.
...
> > @@ -3804,6 +3963,9 @@ static int probe_one_instance(unsigned int nid)
> > struct ecc_settings *s;
> > int ret;
> >
> > + if (!F3)
> > + return 0;
> > +
> > ret = -ENOMEM;
> > s = kzalloc(sizeof(struct ecc_settings), GFP_KERNEL);
> > if (!s)
> > @@ -3815,6 +3977,9 @@ static int probe_one_instance(unsigned int nid)
> > if (!pvt)
> > goto err_settings;
> >
> > + if (nid >= NONCPU_NODE_INDEX)
> > + pvt->is_noncpu = true;
>
> This is silly and error-prone. Proper detection should happen in
> per_family_init() and there you should read out from the hardware
> whether this is a GPU or a CPU node.
>
> Then, you should put an enum type in amd64_family_type which has
>
> { FAM_TYPE_CPU, FAM_TYPE_GPU, ... }
>
> etc and the places where you need to check whether it is CPU or a GPU,
> test those types.
>
This is a good idea. But we have a global *fam_type, so this should be moved
into struct amd64_pvt, if possible. Then each node can have its own fam_type.
..
> > @@ -389,6 +392,9 @@ struct amd64_pvt {
> > enum mem_type dram_type;
> >
> > struct amd64_umc *umc; /* UMC registers */
> > + char buf[20];
>
> A 20 char buffer in every pvt structure just so that you can sprintf
> into it when it is a GPU? Err, I don't think so.
>
> You can do the same thing as with the CPUs - the same string for every
> pvt instance.
>
Fair point. I like the idea of having unique names though. Is this possible
with the current EDAC framework? Or is it not worth it?
Thanks,
Yazen
next prev parent reply other threads:[~2021-09-01 18:42 UTC|newest]
Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-30 15:28 [PATCH 0/7] x86/edac/amd64: Add support for noncpu nodes Naveen Krishna Chatradhi
2021-06-30 15:28 ` [PATCH 1/7] x86/amd_nb: Add Aldebaran device to PCI IDs Naveen Krishna Chatradhi
2021-07-19 19:28 ` Yazen Ghannam
2021-08-10 12:45 ` Chatradhi, Naveen Krishna
2021-08-10 13:54 ` Borislav Petkov
2021-06-30 15:28 ` [PATCH 2/7] x86/amd_nb: Add support for northbridges on Aldebaran Naveen Krishna Chatradhi
2021-07-19 20:25 ` Yazen Ghannam
2021-08-10 12:45 ` Chatradhi, Naveen Krishna
2021-06-30 15:28 ` [PATCH 3/7] EDAC/mc: Add new HBM2 memory type Naveen Krishna Chatradhi
2021-07-19 20:47 ` Yazen Ghannam
2021-07-19 21:16 ` Luck, Tony
2021-07-20 12:13 ` Zhuo, Qiuxu
2021-07-20 16:30 ` [PATCH] EDAC/skx_common: Set the memory type correctly for HBM memory Luck, Tony
2021-07-20 16:33 ` [PATCH 3/7] EDAC/mc: Add new HBM2 memory type Luck, Tony
2021-06-30 15:28 ` [PATCH 4/7] EDAC/mce_amd: extract node id from InstanceHi in IPID Naveen Krishna Chatradhi
2021-07-29 16:32 ` Yazen Ghannam
2021-08-10 12:45 ` Chatradhi, Naveen Krishna
2021-06-30 15:28 ` [PATCH 5/7] EDAC/amd64: Enumerate memory on noncpu nodes Naveen Krishna Chatradhi
2021-07-29 17:55 ` Yazen Ghannam
2021-08-10 12:49 ` Chatradhi, Naveen Krishna
2021-06-30 15:28 ` [PATCH 6/7] EDAC/amd64: Add address translation support for DF3.5 Naveen Krishna Chatradhi
2021-07-29 17:59 ` Yazen Ghannam
2021-07-29 18:13 ` Chatradhi, Naveen Krishna
2021-06-30 15:28 ` [PATCH 7/7] EDAC/amd64: Add fixed UMC to CS mapping Naveen Krishna Chatradhi
2021-08-06 7:43 ` [PATCH v2 0/3] x86/edac/amd64: Add support for noncpu nodes Naveen Krishna Chatradhi
2021-08-06 7:43 ` [PATCH v2 1/3] x86/amd_nb: Add support for northbridges on Aldebaran Naveen Krishna Chatradhi
2021-08-20 15:36 ` Yazen Ghannam
2021-08-06 7:43 ` [PATCH v2 2/3] EDAC/mce_amd: Extract node id from InstanceHi in IPID Naveen Krishna Chatradhi
2021-08-20 15:43 ` Yazen Ghannam
2021-08-06 7:43 ` [PATCH v2 3/3] EDAC/amd64: Enumerate memory on noncpu nodes Naveen Krishna Chatradhi
2021-08-20 17:02 ` Yazen Ghannam
2021-08-23 16:56 ` Chatradhi, Naveen Krishna
2021-08-23 18:54 ` [PATCH v3 0/3] x86/edac/amd64: Add support for " Naveen Krishna Chatradhi
2021-08-23 18:54 ` [PATCH v3 1/3] x86/amd_nb: Add support for northbridges on Aldebaran Naveen Krishna Chatradhi
2021-08-25 10:42 ` Borislav Petkov
2021-09-01 18:17 ` Yazen Ghannam
2021-09-02 17:30 ` Borislav Petkov
2021-09-13 18:07 ` Yazen Ghannam
2021-09-16 16:36 ` Borislav Petkov
2021-10-11 14:26 ` Chatradhi, Naveen Krishna
2021-10-11 18:08 ` Borislav Petkov
2021-08-23 18:54 ` [PATCH v3 2/3] EDAC/mce_amd: Extract node id from MCA_IPID Naveen Krishna Chatradhi
2021-08-27 10:24 ` Borislav Petkov
2021-09-01 18:18 ` Yazen Ghannam
2021-08-23 18:54 ` [PATCH v3 3/3] EDAC/amd64: Enumerate memory on noncpu nodes Naveen Krishna Chatradhi
2021-08-27 11:30 ` Borislav Petkov
2021-09-01 18:42 ` Yazen Ghannam [this message]
2021-09-08 18:41 ` Borislav Petkov
2021-09-13 18:19 ` Yazen Ghannam
2021-10-14 18:50 ` [PATCH v4 0/4] x86/edac/amd64: Add support for " Naveen Krishna Chatradhi
2021-10-14 18:50 ` [PATCH v4 1/4] x86/amd_nb: Add support for northbridges on Aldebaran Naveen Krishna Chatradhi
2021-10-14 18:50 ` [PATCH v4 2/4] EDAC/mce_amd: Extract node id from MCA_IPID Naveen Krishna Chatradhi
2021-10-14 18:50 ` [PATCH 3/4] EDAC/amd64: Extend family ops functions Naveen Krishna Chatradhi
2021-10-14 18:50 ` [PATCH 4/4] EDAC/amd64: Enumerate memory on Aldebaran GPU nodes Naveen Krishna Chatradhi
2021-10-14 18:53 ` [PATCH v4 0/4] x86/edac/amd64: Add support for noncpu nodes Naveen Krishna Chatradhi
2021-10-14 18:53 ` [PATCH v4 1/4] x86/amd_nb: Add support for northbridges on Aldebaran Naveen Krishna Chatradhi
2021-10-21 15:52 ` Yazen Ghannam
2021-10-14 18:53 ` [PATCH v4 2/4] EDAC/mce_amd: Extract node id from MCA_IPID Naveen Krishna Chatradhi
2021-10-21 15:55 ` Yazen Ghannam
2021-10-14 18:53 ` [PATCH v4 3/4] EDAC/amd64: Extend family ops functions Naveen Krishna Chatradhi
2021-10-21 16:27 ` Yazen Ghannam
2021-10-14 18:54 ` [PATCH v4 4/4] EDAC/amd64: Enumerate memory on Aldebaran GPU nodes Naveen Krishna Chatradhi
2021-10-21 16:40 ` Yazen Ghannam
2021-10-14 19:53 ` [PATCH v4 0/4] x86/edac/amd64: Add support for noncpu nodes Borislav Petkov
2021-10-15 12:18 ` Chatradhi, Naveen Krishna
2021-10-15 19:25 ` Borislav Petkov
2021-08-20 17:07 ` [PATCH v2 0/3] " Yazen Ghannam
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YS/JkgWA8VreIx1R@yaz-ubuntu \
--to=yazen.ghannam@amd.com \
--cc=bp@alien8.de \
--cc=linux-edac@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mchehab@kernel.org \
--cc=muralimk@amd.com \
--cc=nchatrad@amd.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).