From: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
To: "Luck, Tony" <tony.luck@intel.com>
Cc: Xishi Qiu <qiuxishi@huawei.com>,
Andrew Morton <akpm@linux-foundation.org>,
"nao.horiguchi@gmail.com" <nao.horiguchi@gmail.com>,
Yinghai Lu <yinghai@kernel.org>, "H. Peter Anvin" <hpa@zytor.com>,
Thomas Gleixner <tglx@linutronix.de>,
"mingo@elte.hu" <mingo@elte.hu>, Xiexiuqi <xiexiuqi@huawei.com>,
Hanjun Guo <guohanjun@huawei.com>, Linux MM <linux-mm@kvack.org>,
LKML <linux-kernel@vger.kernel.org>
Subject: Re: [RFC PATCH 00/12] mm: mirrored memory support for page buddy allocations
Date: Mon, 15 Jun 2015 00:25:00 +0000 [thread overview]
Message-ID: <20150615002500.GC4214@hori1.linux.bs1.fc.nec.co.jp> (raw)
In-Reply-To: <20150612190335.GA21994@agluck-desk.sc.intel.com>
On Fri, Jun 12, 2015 at 12:03:35PM -0700, Luck, Tony wrote:
> On Fri, Jun 12, 2015 at 08:42:33AM +0000, Naoya Horiguchi wrote:
> > 4?) I don't have the whole picture of how address ranging mirroring works,
> > but I'm curious about what happens when an uncorrected memory error happens
> > on the a mirror page. If HW/FW do some useful work invisible from kernel,
> > please document it somewhere. And my questions are:
> > - can the kernel with this patchset really continue its operation without
> > breaking consistency? More specifically, the corrupted page is replaced with
> > its mirror page, but can any other pages which have references (like struct
> > page or pfn) for the corrupted page properly switch these references to the
> > mirror page? Or no worry about that? (This is difficult for kernel pages
> > like slab, and that's why currently hwpoison doesn't handle any kernel pages.)
>
> The mirror is operated by h/w (perhaps with some platform firmware
> intervention when things start breaking badly).
>
> In normal operation there are two DIMM addresses backing each
> system physical address in the mirrored range (thus total system
> memory capacity is reduced when mirror is enabled). Memory writes
> are directed to both locations. Memory reads are interleaved to
> maintain bandwidth, so could come from either address.
I misunderstood that both of mirrored page and mirroring page are visible
to OS, which is incorrect.
> When a read returns with an ECC failure the h/w automatically:
> 1) Re-issues the read to the other DIMM address. If that also fails - then
> we do the normal machine check processing for an uncorrected error
> 2) But if the other side of the mirror is good, we can send the good
> data to the reader (cpu, or dma) and, in parallel try to fix the
> bad side by writing the good data to it.
> 3) A corrected error will be logged, it may indicate whether the
> attempt to fix succeeded or not.
> 4) If platform firmware wants, it can be notified of the correction
> and it may keep statistics on the rate of errors, correction status,
> etc. If things get very bad it may "break" the mirror and direct
> all future reads to the remaining "good" side. If does this it will
> likely tell the OS via some ACPI method.
Thanks, this fully answered my question.
> All of this is done at much less than page granularity. Cache coherence
> is maintained ... apart from some small performance glitches and the corrected
> error logs, the OS is unware of all of this.
>
> Note that in current implementations the mirror copies are both behind
> the same memory controller ... so this isn't intended to cope with high
> level failure of a memory controller ... just to deal with randomly
> distributed ECC errors.
OK, I looked at "Memory Address Range Mirroring Validation Guide" and Fig 2-2
clearly shows that.
> > - How can we test/confirm that the whole scheme works fine? Is current memory
> > error injection framework enough?
>
> Still working on that piece. To validate you need to be able to
> inject errors to just one side of the mirror, and I'm not really
> sure that the ACPI/EINJ interface is up to the task.
OK.
Thanks,
Naoya Horiguchi
next prev parent reply other threads:[~2015-06-15 0:28 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-06-04 12:54 [RFC PATCH 00/12] mm: mirrored memory support for page buddy allocations Xishi Qiu
2015-06-04 12:56 ` [RFC PATCH 01/12] mm: add a new config to manage the code Xishi Qiu
2015-06-08 11:52 ` Leon Romanovsky
2015-06-08 15:14 ` Luck, Tony
2015-06-08 16:36 ` Leon Romanovsky
2015-06-09 6:44 ` Kamezawa Hiroyuki
2015-06-09 10:10 ` Xishi Qiu
2015-06-10 3:07 ` Kamezawa Hiroyuki
2015-06-04 12:57 ` [RFC PATCH 02/12] mm: introduce mirror_info Xishi Qiu
2015-06-04 16:57 ` Luck, Tony
2015-06-05 1:53 ` Xishi Qiu
2015-06-09 6:48 ` Kamezawa Hiroyuki
2015-06-04 12:58 ` [RFC PATCH 03/12] mm: introduce MIGRATE_MIRROR to manage the mirrored, pages Xishi Qiu
2015-06-09 6:54 ` Kamezawa Hiroyuki
2015-06-04 12:59 ` [RFC PATCH 04/12] mm: add mirrored pages to buddy system Xishi Qiu
2015-06-04 13:00 ` [RFC PATCH 05/12] mm: introduce a new zone_stat_item NR_FREE_MIRROR_PAGES Xishi Qiu
2015-06-04 13:01 ` [RFC PATCH 06/12] mm: add free mirrored pages info Xishi Qiu
2015-06-04 13:02 ` [RFC PATCH 07/12] mm: introduce __GFP_MIRROR to allocate mirrored pages Xishi Qiu
2015-06-09 7:01 ` Kamezawa Hiroyuki
2015-06-04 13:02 ` [RFC PATCH 08/12] mm: use mirrorable to switch allocate mirrored memory Xishi Qiu
2015-06-04 17:01 ` Luck, Tony
2015-06-04 18:41 ` Dave Hansen
2015-06-05 3:13 ` Xishi Qiu
2015-06-09 7:06 ` Kamezawa Hiroyuki
2015-06-09 10:09 ` Xishi Qiu
2015-06-10 3:09 ` Kamezawa Hiroyuki
2015-06-12 8:05 ` Naoya Horiguchi
2015-06-04 13:03 ` [RFC PATCH 09/12] mm: enable allocate mirrored memory at boot time Xishi Qiu
2015-06-04 13:04 ` [RFC PATCH 10/12] mm: add the buddy system interface Xishi Qiu
2015-06-04 17:09 ` Luck, Tony
2015-06-05 3:14 ` Xishi Qiu
2015-06-09 7:12 ` Kamezawa Hiroyuki
2015-06-09 10:04 ` Xishi Qiu
2015-06-10 3:06 ` Kamezawa Hiroyuki
2015-06-10 20:40 ` Luck, Tony
2015-06-15 8:47 ` Kamezawa Hiroyuki
2015-06-15 17:20 ` Luck, Tony
2015-06-16 0:31 ` Kamezawa Hiroyuki
2015-06-25 9:44 ` Xishi Qiu
2015-06-25 23:54 ` Kamezawa Hiroyuki
2015-06-26 1:43 ` Xishi Qiu
2015-06-26 8:34 ` Kamezawa Hiroyuki
2015-06-26 10:38 ` Xishi Qiu
2015-06-26 18:42 ` Luck, Tony
2015-06-04 13:04 ` [RFC PATCH 11/12] mm: add the PCP interface Xishi Qiu
2015-06-04 18:44 ` Dave Hansen
2015-06-04 13:05 ` [RFC PATCH 12/12] mm: let slab/slub/slob use mirrored memory Xishi Qiu
2015-06-04 17:14 ` Luck, Tony
2015-06-12 8:42 ` [RFC PATCH 00/12] mm: mirrored memory support for page buddy allocations Naoya Horiguchi
2015-06-12 9:09 ` Xishi Qiu
2015-06-12 19:03 ` Luck, Tony
2015-06-15 0:25 ` Naoya Horiguchi [this message]
2015-06-16 7:53 ` Vlastimil Babka
2015-06-16 8:17 ` Xishi Qiu
2015-06-16 9:46 ` Vlastimil Babka
2015-06-18 1:23 ` Xishi Qiu
2015-06-18 5:58 ` Vlastimil Babka
2015-06-18 9:37 ` Xishi Qiu
2015-06-18 9:55 ` Vlastimil Babka
2015-06-18 20:33 ` Luck, Tony
2015-06-19 1:36 ` Xishi Qiu
2015-06-19 18:42 ` Luck, Tony
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150615002500.GC4214@hori1.linux.bs1.fc.nec.co.jp \
--to=n-horiguchi@ah.jp.nec.com \
--cc=akpm@linux-foundation.org \
--cc=guohanjun@huawei.com \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mingo@elte.hu \
--cc=nao.horiguchi@gmail.com \
--cc=qiuxishi@huawei.com \
--cc=tglx@linutronix.de \
--cc=tony.luck@intel.com \
--cc=xiexiuqi@huawei.com \
--cc=yinghai@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).