From: Richard Weinberger <richard@nod.at> To: linux-mtd@lists.infradead.org Cc: linux-kernel@vger.kernel.org, dedekind1@gmail.com, boris.brezillon@free-electrons.com Subject: UBI: Bitrot checking Date: Sun, 29 Mar 2015 14:13:13 +0200 [thread overview] Message-ID: <1427631197-23610-1-git-send-email-richard@nod.at> (raw) As of today UBI does not offer a way to reliable deal with data retention and especially read disturb. Read disturb means that a page X will face bitflips if you read page Y, Y != X but within the same block, very often. People who care about data retention often have cron jobs on their targets which read from time to time whole UBI volumes. Something like "dd if=/dev/ubi0_0 of=/dev/zero" every week. If UBI faces bitflips while reading it will schedule the affected PEB for scrubbing. The major downside of this approach is that you don't catch all pages. e.g. UBI EC and VID headers are not always read. Also UBI internal data structure like volume table are not checked. These pages are mostly read at attach time. So, if you run a cron job and reboot often you are on the safe side. With fastmap the issue is even worse as you don't scan every PEB at attach time. In fact, I've seen read disturb issues on a customer's target where he has installed such a cronjob. But some targets still died due to read disturb issues. It turned out that only targets with very high uptimes were affected. To overcome that issue this patch series adds a bitrot checking mechanism to UBI. It can be triggered by writing to /sys/class/ubi/ubiX/trigger_bitrot_check. Then UBI will read every PEB and schedule scrubbing or an appropriated action if it detects flipped bits. Reading is done within the wear-leveling thread such that it does not block other IO. I can also think of adding a new thread to UBI which has a very low priority. User can then replace the dd command in their cron jobs by a simple "echo 1 > /sys/class/ubi/ubiX/trigger_bitrot_check". This series is only part one of two. Part two will add new ioctl() commands to the UBI device nodes such that we can develop something like a ubi-healthd. This daemon will be able to fetch statistics about every known PEB and can trigger scrubbing. Especially for MLC NAND support more advanced techniques are needed to deal with data retention. [PATCH 1/4] UBI: Introduce ubi_schedule_fm_work() [PATCH 2/4] UBI: Introduce prepare_erase_work() [PATCH 3/4] UBI: Introduce in_pq() [PATCH 4/4] UBI: Implement bitrot checking Thanks, //richard
WARNING: multiple messages have this Message-ID (diff)
From: Richard Weinberger <richard@nod.at> To: linux-mtd@lists.infradead.org Cc: boris.brezillon@free-electrons.com, linux-kernel@vger.kernel.org, dedekind1@gmail.com Subject: UBI: Bitrot checking Date: Sun, 29 Mar 2015 14:13:13 +0200 [thread overview] Message-ID: <1427631197-23610-1-git-send-email-richard@nod.at> (raw) As of today UBI does not offer a way to reliable deal with data retention and especially read disturb. Read disturb means that a page X will face bitflips if you read page Y, Y != X but within the same block, very often. People who care about data retention often have cron jobs on their targets which read from time to time whole UBI volumes. Something like "dd if=/dev/ubi0_0 of=/dev/zero" every week. If UBI faces bitflips while reading it will schedule the affected PEB for scrubbing. The major downside of this approach is that you don't catch all pages. e.g. UBI EC and VID headers are not always read. Also UBI internal data structure like volume table are not checked. These pages are mostly read at attach time. So, if you run a cron job and reboot often you are on the safe side. With fastmap the issue is even worse as you don't scan every PEB at attach time. In fact, I've seen read disturb issues on a customer's target where he has installed such a cronjob. But some targets still died due to read disturb issues. It turned out that only targets with very high uptimes were affected. To overcome that issue this patch series adds a bitrot checking mechanism to UBI. It can be triggered by writing to /sys/class/ubi/ubiX/trigger_bitrot_check. Then UBI will read every PEB and schedule scrubbing or an appropriated action if it detects flipped bits. Reading is done within the wear-leveling thread such that it does not block other IO. I can also think of adding a new thread to UBI which has a very low priority. User can then replace the dd command in their cron jobs by a simple "echo 1 > /sys/class/ubi/ubiX/trigger_bitrot_check". This series is only part one of two. Part two will add new ioctl() commands to the UBI device nodes such that we can develop something like a ubi-healthd. This daemon will be able to fetch statistics about every known PEB and can trigger scrubbing. Especially for MLC NAND support more advanced techniques are needed to deal with data retention. [PATCH 1/4] UBI: Introduce ubi_schedule_fm_work() [PATCH 2/4] UBI: Introduce prepare_erase_work() [PATCH 3/4] UBI: Introduce in_pq() [PATCH 4/4] UBI: Implement bitrot checking Thanks, //richard
next reply other threads:[~2015-03-29 12:13 UTC|newest] Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top 2015-03-29 12:13 Richard Weinberger [this message] 2015-03-29 12:13 ` UBI: Bitrot checking Richard Weinberger 2015-03-29 12:13 ` [PATCH 1/4] UBI: Introduce ubi_schedule_fm_work() Richard Weinberger 2015-03-29 12:13 ` Richard Weinberger 2015-03-29 12:13 ` [PATCH 2/4] UBI: Introduce prepare_erase_work() Richard Weinberger 2015-03-29 12:13 ` Richard Weinberger 2015-03-29 12:13 ` [PATCH 3/4] UBI: Introduce in_pq() Richard Weinberger 2015-03-29 12:13 ` Richard Weinberger 2015-03-29 12:13 ` [PATCH 4/4] UBI: Implement bitrot checking Richard Weinberger 2015-03-29 12:13 ` Richard Weinberger 2015-04-02 17:34 ` Andrea Scian 2015-04-02 17:34 ` Andrea Scian 2015-04-02 17:54 ` Richard Weinberger 2015-04-02 17:54 ` Richard Weinberger 2015-04-02 19:19 ` Andrea Scian 2015-04-02 19:19 ` Andrea Scian 2015-04-08 10:34 ` Richard Weinberger 2015-04-08 10:34 ` Richard Weinberger 2015-04-08 21:02 ` Andrea Scian 2015-04-08 21:02 ` Andrea Scian 2015-04-08 11:48 ` David Oberhollenzer 2015-04-12 14:12 ` Boris Brezillon 2015-04-12 16:09 ` Richard Weinberger 2015-04-12 16:43 ` Boris Brezillon 2015-04-12 16:55 ` Richard Weinberger 2015-04-12 20:42 ` [PATCH 4/4] UBI: Implement bitrot checking (linux-mtd Digest, Vol 145, Issue 24) Andrea Scian 2015-04-12 20:42 ` Andrea Scian 2015-04-12 21:01 ` Richard Weinberger 2015-04-12 21:01 ` Richard Weinberger 2015-04-12 21:30 ` Boris Brezillon 2015-04-12 21:37 ` Richard Weinberger 2015-04-12 21:33 ` Andrea Scian 2015-04-12 21:33 ` Andrea Scian 2015-04-12 21:42 ` Richard Weinberger 2015-04-12 21:42 ` Richard Weinberger 2015-04-13 17:17 ` linux-mtd digest emails (was Re: [PATCH 4/4] UBI: Implement bitrot checking) Brian Norris 2015-04-13 17:17 ` Brian Norris 2015-04-12 15:14 ` [PATCH 4/4] UBI: Implement bitrot checking Boris Brezillon 2015-04-12 16:14 ` Richard Weinberger 2015-04-12 16:31 ` Boris Brezillon 2015-04-12 16:32 ` Richard Weinberger 2015-04-12 17:01 ` Boris Brezillon 2015-04-12 17:09 ` Richard Weinberger 2015-04-12 19:20 ` Boris Brezillon 2015-04-12 19:53 ` Richard Weinberger 2015-04-12 21:24 ` Boris Brezillon 2015-04-12 21:34 ` Richard Weinberger 2015-04-13 3:36 ` nick 2015-04-12 17:36 ` Richard Weinberger [not found] <mailman.40253.1428858576.22890.linux-mtd@lists.infradead.org> [not found] <mailman.38750.1427638218.22890.linux-mtd@lists.infradead.org>
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1427631197-23610-1-git-send-email-richard@nod.at \ --to=richard@nod.at \ --cc=boris.brezillon@free-electrons.com \ --cc=dedekind1@gmail.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mtd@lists.infradead.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.