All of lore.kernel.org
 help / color / mirror / Atom feed
From: Yazen Ghannam <yazen.ghannam@amd.com>
To: Borislav Petkov <bp@alien8.de>
Cc: Naveen Krishna Chatradhi <nchatrad@amd.com>,
	linux-edac@vger.kernel.org, x86@kernel.org,
	linux-kernel@vger.kernel.org, mchehab@kernel.org,
	Muralidhara M K <muralimk@amd.com>
Subject: Re: [PATCH v3 3/3] EDAC/amd64: Enumerate memory on noncpu nodes
Date: Wed, 1 Sep 2021 18:42:26 +0000	[thread overview]
Message-ID: <YS/JkgWA8VreIx1R@yaz-ubuntu> (raw)
In-Reply-To: <YSjM8b9vvkmRew94@zn.tnic>

On Fri, Aug 27, 2021 at 01:30:57PM +0200, Borislav Petkov wrote:
> On Tue, Aug 24, 2021 at 12:24:37AM +0530, Naveen Krishna Chatradhi wrote:

...

> > @@ -1335,6 +1392,11 @@ static void determine_memory_type(struct amd64_pvt *pvt)
> >  	u32 dram_ctrl, dcsm;
> >  
> >  	if (pvt->umc) {
> > +		if (pvt->is_noncpu) {
> > +			pvt->dram_type = MEM_HBM2;
> > +			return;
> > +		}
> 
> I don't like this sprinkling of "if (pvt->is_noncpu)" everywhere,
> at all. Please define a separate read_mc_regs_df() or so which
> contains only the needed functionalty which you can carve out from
> read_mc_regs().
>

I like this idea.
 
> > +
> >  		if ((pvt->umc[0].dimm_cfg | pvt->umc[1].dimm_cfg) & BIT(5))
> >  			pvt->dram_type = MEM_LRDDR4;
> >  		else if ((pvt->umc[0].dimm_cfg | pvt->umc[1].dimm_cfg) & BIT(4))
> > @@ -1724,7 +1786,10 @@ static int f17_early_channel_count(struct amd64_pvt *pvt)
> >  
> >  	/* SDP Control bit 31 (SdpInit) is clear for unused UMC channels */
> >  	for_each_umc(i)
> > -		channels += !!(pvt->umc[i].sdp_ctrl & UMC_SDP_INIT);
> > +		if (pvt->is_noncpu)
> > +			channels += pvt->csels[i].b_cnt;
> > +		else
> > +			channels += !!(pvt->umc[i].sdp_ctrl & UMC_SDP_INIT);
> >  
> >  	amd64_info("MCT channel count: %d\n", channels);
> >  
> 
> No, a separate gpu_early_channel_count() is needed here. There's a
> reason for those function pointers getting assigned depending on family.
>

Good point.
 
...
> > +/*
> > + * The CPUs have one channel per UMC, So a UMC number is equivalent to a
> > + * channel number. The NONCPUs have 8 channels per UMC, so the UMC number no
> > + * longer works as a channel number.
> > + * The channel number within a NONCPU UMC is given in MCA_IPID[15:12].
> > + * However, the IDs are split such that two UMC values go to one UMC, and
> > + * the channel numbers are split in two groups of four.
> > + *
> > + * Refer comment on get_noncpu_umc_base() from amd64_edac.h
> > + *
> > + * For example,
> > + * UMC0 CH[3:0] = 0x0005[3:0]000
> > + * UMC0 CH[7:4] = 0x0015[3:0]000
> > + * UMC1 CH[3:0] = 0x0025[3:0]000
> > + * UMC1 CH[7:4] = 0x0035[3:0]000
> > + */
> > +static int find_umc_channel_noncpu(struct mce *m)
> > +{
> > +	u8 umc = find_umc_channel(m);
> > +	u8 ch = ((m->ipid >> 12) & 0xf);
> > +
> > +	return umc % 2 ? (ch + 4) : ch;
> > +}
> > +
> >  static void decode_umc_error(int node_id, struct mce *m)
> >  {
> >  	u8 ecc_type = (m->status >> 45) & 0x3;
> > @@ -2897,6 +3003,7 @@ static void decode_umc_error(int node_id, struct mce *m)
> >  	struct amd64_pvt *pvt;
> >  	struct err_info err;
> >  	u64 sys_addr;
> > +	u8 df_inst_id;
> 
> You don't need that variable and can work with err.channel just fine.
> 
> >  	mci = edac_mc_find(node_id);
> >  	if (!mci)
> > @@ -2909,7 +3016,22 @@ static void decode_umc_error(int node_id, struct mce *m)
> >  	if (m->status & MCI_STATUS_DEFERRED)
> >  		ecc_type = 3;
> >  
> > -	err.channel = find_umc_channel(m);
> > +	if (pvt->is_noncpu) {
> > +		/*
> > +		 * The NONCPUs have one Chip Select per UMC, so the UMC number
> > +		 * can used as the Chip Select number. However, the UMC number
> > +		 * is split in the ID value so it's necessary to divide by 2.
> > +		 */
> > +		err.csrow = find_umc_channel(m) / 2;
> > +		err.channel = find_umc_channel_noncpu(m);
> > +		/* On NONCPUs, instance id is calculated as below. */
> > +		df_inst_id = err.csrow * 8 + err.channel;
> 
> 		err.channel += err.csrow * 8;
> 
> tadaaa!
>

err.channel still needs to be used in error_address_to_page_and_offset()
below. So changing it here messes up what's reported to EDAC.
 
...
> > @@ -3804,6 +3963,9 @@ static int probe_one_instance(unsigned int nid)
> >  	struct ecc_settings *s;
> >  	int ret;
> >  
> > +	if (!F3)
> > +		return 0;
> > +
> >  	ret = -ENOMEM;
> >  	s = kzalloc(sizeof(struct ecc_settings), GFP_KERNEL);
> >  	if (!s)
> > @@ -3815,6 +3977,9 @@ static int probe_one_instance(unsigned int nid)
> >  	if (!pvt)
> >  		goto err_settings;
> >  
> > +	if (nid >= NONCPU_NODE_INDEX)
> > +		pvt->is_noncpu = true;
> 
> This is silly and error-prone. Proper detection should happen in
> per_family_init() and there you should read out from the hardware
> whether this is a GPU or a CPU node.
> 
> Then, you should put an enum type in amd64_family_type which has
> 
>  { FAM_TYPE_CPU, FAM_TYPE_GPU, ... }
> 
> etc and the places where you need to check whether it is CPU or a GPU,
> test those types.
>

This is a good idea. But we have a global *fam_type, so this should be moved
into struct amd64_pvt, if possible. Then each node can have its own fam_type.

..
> > @@ -389,6 +392,9 @@ struct amd64_pvt {
> >  	enum mem_type dram_type;
> >  
> >  	struct amd64_umc *umc;	/* UMC registers */
> > +	char buf[20];
> 
> A 20 char buffer in every pvt structure just so that you can sprintf
> into it when it is a GPU? Err, I don't think so.
> 
> You can do the same thing as with the CPUs - the same string for every
> pvt instance.
>

Fair point. I like the idea of having unique names though. Is this possible
with the current EDAC framework? Or is it not worth it?

Thanks,
Yazen

  reply	other threads:[~2021-09-01 18:42 UTC|newest]

Thread overview: 67+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-30 15:28 [PATCH 0/7] x86/edac/amd64: Add support for " Naveen Krishna Chatradhi
2021-06-30 15:28 ` [PATCH 1/7] x86/amd_nb: Add Aldebaran device to PCI IDs Naveen Krishna Chatradhi
2021-07-19 19:28   ` Yazen Ghannam
2021-08-10 12:45     ` Chatradhi, Naveen Krishna
2021-08-10 13:54       ` Borislav Petkov
2021-06-30 15:28 ` [PATCH 2/7] x86/amd_nb: Add support for northbridges on Aldebaran Naveen Krishna Chatradhi
2021-07-19 20:25   ` Yazen Ghannam
2021-08-10 12:45     ` Chatradhi, Naveen Krishna
2021-06-30 15:28 ` [PATCH 3/7] EDAC/mc: Add new HBM2 memory type Naveen Krishna Chatradhi
2021-07-19 20:47   ` Yazen Ghannam
2021-07-19 21:16     ` Luck, Tony
2021-07-20 12:13       ` Zhuo, Qiuxu
2021-07-20 16:30         ` [PATCH] EDAC/skx_common: Set the memory type correctly for HBM memory Luck, Tony
2021-07-20 16:33     ` [PATCH 3/7] EDAC/mc: Add new HBM2 memory type Luck, Tony
2021-06-30 15:28 ` [PATCH 4/7] EDAC/mce_amd: extract node id from InstanceHi in IPID Naveen Krishna Chatradhi
2021-07-29 16:32   ` Yazen Ghannam
2021-08-10 12:45     ` Chatradhi, Naveen Krishna
2021-06-30 15:28 ` [PATCH 5/7] EDAC/amd64: Enumerate memory on noncpu nodes Naveen Krishna Chatradhi
2021-07-29 17:55   ` Yazen Ghannam
2021-08-10 12:49     ` Chatradhi, Naveen Krishna
2021-06-30 15:28 ` [PATCH 6/7] EDAC/amd64: Add address translation support for DF3.5 Naveen Krishna Chatradhi
2021-07-29 17:59   ` Yazen Ghannam
2021-07-29 18:13     ` Chatradhi, Naveen Krishna
2021-06-30 15:28 ` [PATCH 7/7] EDAC/amd64: Add fixed UMC to CS mapping Naveen Krishna Chatradhi
2021-08-06  7:43 ` [PATCH v2 0/3] x86/edac/amd64: Add support for noncpu nodes Naveen Krishna Chatradhi
2021-08-06  7:43   ` [PATCH v2 1/3] x86/amd_nb: Add support for northbridges on Aldebaran Naveen Krishna Chatradhi
2021-08-20 15:36     ` Yazen Ghannam
2021-08-06  7:43   ` [PATCH v2 2/3] EDAC/mce_amd: Extract node id from InstanceHi in IPID Naveen Krishna Chatradhi
2021-08-20 15:43     ` Yazen Ghannam
2021-08-06  7:43   ` [PATCH v2 3/3] EDAC/amd64: Enumerate memory on noncpu nodes Naveen Krishna Chatradhi
2021-08-20 17:02     ` Yazen Ghannam
2021-08-23 16:56       ` Chatradhi, Naveen Krishna
2021-08-23 18:54     ` [PATCH v3 0/3] x86/edac/amd64: Add support for " Naveen Krishna Chatradhi
2021-08-23 18:54       ` [PATCH v3 1/3] x86/amd_nb: Add support for northbridges on Aldebaran Naveen Krishna Chatradhi
2021-08-25 10:42         ` Borislav Petkov
2021-09-01 18:17           ` Yazen Ghannam
2021-09-02 17:30             ` Borislav Petkov
2021-09-13 18:07               ` Yazen Ghannam
2021-09-16 16:36                 ` Borislav Petkov
2021-10-11 14:26           ` Chatradhi, Naveen Krishna
2021-10-11 18:08             ` Borislav Petkov
2021-08-23 18:54       ` [PATCH v3 2/3] EDAC/mce_amd: Extract node id from MCA_IPID Naveen Krishna Chatradhi
2021-08-27 10:24         ` Borislav Petkov
2021-09-01 18:18           ` Yazen Ghannam
2021-08-23 18:54       ` [PATCH v3 3/3] EDAC/amd64: Enumerate memory on noncpu nodes Naveen Krishna Chatradhi
2021-08-27 11:30         ` Borislav Petkov
2021-09-01 18:42           ` Yazen Ghannam [this message]
2021-09-08 18:41             ` Borislav Petkov
2021-09-13 18:19               ` Yazen Ghannam
2021-10-14 18:50       ` [PATCH v4 0/4] x86/edac/amd64: Add support for " Naveen Krishna Chatradhi
2021-10-14 18:50         ` [PATCH v4 1/4] x86/amd_nb: Add support for northbridges on Aldebaran Naveen Krishna Chatradhi
2021-10-14 18:50         ` [PATCH v4 2/4] EDAC/mce_amd: Extract node id from MCA_IPID Naveen Krishna Chatradhi
2021-10-14 18:50         ` [PATCH 3/4] EDAC/amd64: Extend family ops functions Naveen Krishna Chatradhi
2021-10-14 18:50         ` [PATCH 4/4] EDAC/amd64: Enumerate memory on Aldebaran GPU nodes Naveen Krishna Chatradhi
2021-10-14 18:53       ` [PATCH v4 0/4] x86/edac/amd64: Add support for noncpu nodes Naveen Krishna Chatradhi
2021-10-14 18:53         ` [PATCH v4 1/4] x86/amd_nb: Add support for northbridges on Aldebaran Naveen Krishna Chatradhi
2021-10-21 15:52           ` Yazen Ghannam
2021-10-14 18:53         ` [PATCH v4 2/4] EDAC/mce_amd: Extract node id from MCA_IPID Naveen Krishna Chatradhi
2021-10-21 15:55           ` Yazen Ghannam
2021-10-14 18:53         ` [PATCH v4 3/4] EDAC/amd64: Extend family ops functions Naveen Krishna Chatradhi
2021-10-21 16:27           ` Yazen Ghannam
2021-10-14 18:54         ` [PATCH v4 4/4] EDAC/amd64: Enumerate memory on Aldebaran GPU nodes Naveen Krishna Chatradhi
2021-10-21 16:40           ` Yazen Ghannam
2021-10-14 19:53         ` [PATCH v4 0/4] x86/edac/amd64: Add support for noncpu nodes Borislav Petkov
2021-10-15 12:18           ` Chatradhi, Naveen Krishna
2021-10-15 19:25             ` Borislav Petkov
2021-08-20 17:07   ` [PATCH v2 0/3] " Yazen Ghannam

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YS/JkgWA8VreIx1R@yaz-ubuntu \
    --to=yazen.ghannam@amd.com \
    --cc=bp@alien8.de \
    --cc=linux-edac@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mchehab@kernel.org \
    --cc=muralimk@amd.com \
    --cc=nchatrad@amd.com \
    --cc=x86@kernel.org \
    --subject='Re: [PATCH v3 3/3] EDAC/amd64: Enumerate memory on noncpu nodes' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.