* Unstable bits and JFFS2 @ 2012-03-28 11:16 Matej Kupljen 2012-04-02 17:12 ` Brian Norris 0 siblings, 1 reply; 12+ messages in thread From: Matej Kupljen @ 2012-03-28 11:16 UTC (permalink / raw) To: linux-mtd To all, during my experiments with the UBIFS crashing, I believe that I am experiencing the "unstable bit" issue. Would another FS, like JFFS2 work on such device, or I'd have the problems on such device with all file systems? Thank you and Best Regards, Matej ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Unstable bits and JFFS2 2012-03-28 11:16 Unstable bits and JFFS2 Matej Kupljen @ 2012-04-02 17:12 ` Brian Norris 2012-04-04 9:17 ` Matej Kupljen 0 siblings, 1 reply; 12+ messages in thread From: Brian Norris @ 2012-04-02 17:12 UTC (permalink / raw) To: Matej Kupljen; +Cc: linux-mtd Hi Matej, On Wed, Mar 28, 2012 at 4:16 AM, Matej Kupljen <matej.kupljen@gmail.com> wrote: > during my experiments with the UBIFS crashing, I believe that I am experiencing > the "unstable bit" issue. > > Would another FS, like JFFS2 work on such device, or I'd have the problems on > such device with all file systems? I doubt it. JFFS2 is not supported much anymore, and it is not designed for some of the problems with modern NAND flash. Have you read the information here? http://www.linux-mtd.infradead.org/doc/ubifs.html#L_unstable_bits I'm not much of an export on UBIFS or unstable bits, but I think that there are some unsolved problems. However, before ruling this an unstable bits problem, it's a good idea to rule out any other possible issues. Brian ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Unstable bits and JFFS2 2012-04-02 17:12 ` Brian Norris @ 2012-04-04 9:17 ` Matej Kupljen 2012-04-04 10:54 ` Atlant Schmidt 0 siblings, 1 reply; 12+ messages in thread From: Matej Kupljen @ 2012-04-04 9:17 UTC (permalink / raw) To: Brian Norris; +Cc: linux-mtd Brian, > On Wed, Mar 28, 2012 at 4:16 AM, Matej Kupljen <matej.kupljen@gmail.com> wrote: >> during my experiments with the UBIFS crashing, I believe that I am experiencing >> the "unstable bit" issue. >> >> Would another FS, like JFFS2 work on such device, or I'd have the problems on >> such device with all file systems? > > I doubt it. JFFS2 is not supported much anymore, and it is not > designed for some of the problems with modern NAND flash. Aha, I see. Thank you for the information. > Have you read the information here? > > http://www.linux-mtd.infradead.org/doc/ubifs.html#L_unstable_bits Yes, I have. Artem already pointed me to that FAQ. > I'm not much of an export on UBIFS or unstable bits, but I think that > there are some unsolved problems. However, before ruling this an > unstable bits problem, it's a good idea to rule out any other possible > issues. Yes, I'll check the MTD driver. At the moment I am running again integrity check on my UBIFS for almost 24 hours now, but until now no error: # ./integck -n 0 -v -e /media/card The only difference from previous run, where I saw one bit flip is that previously I run the test with power failure option set to ON. Now it is off. Artem, you said that this unstable bits only happen during power cuts, is this right? Would those appear also on simulated power cuts, the ones that can integck produce? As for the ECC correction of the empty space, nothing is mentioned in the manual. Maybe I should ask the manufacturer of the chip (NXP LPC3152)?? It uses HW error correction included in the FLASH controller, with: "The error correction code used is Reed-Solomon over GF(2^9). The primitive polynomial g(x) over GF(2) is: g(x) = x9 + x4 +1" Thank you and Best Regards, Matej ^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: Unstable bits and JFFS2 2012-04-04 9:17 ` Matej Kupljen @ 2012-04-04 10:54 ` Atlant Schmidt 2012-04-10 7:22 ` Matej Kupljen 2012-04-13 16:27 ` Artem Bityutskiy 0 siblings, 2 replies; 12+ messages in thread From: Atlant Schmidt @ 2012-04-04 10:54 UTC (permalink / raw) To: 'Matej Kupljen', Brian Norris; +Cc: linux-mtd Matej: > Artem, you said that this unstable bits only happen > during power cuts, is this right? Would those appear > also on simulated power cuts, the ones that integck > can produce? (Note: This isn't Artem replying.) Unstable bits can happen anytime, but *REAL* power-cuts while writing are certainly a great way to produce them; after all, if you've only transferred half the electrons to the floating gate when power goes away, well, that Flash cell is now a roll-of-the-quantum-dice each time you read it. Simulated power cuts (that just stop the software processing at arbitrary and random points) can't produce this effect. But any time a read-disturb or write-disturb takes place, there's some probability that a Flash cell will be left with a "near-threshold" charge on the floating gate, so unstable bits are a fact of life that must be faced by any software that drives NAND Flash memory chips. This is, of course, especially true of MLC chips and even more-true for TLC chips (with three bits per cell). Atlant -----Original Message----- From: linux-mtd-bounces@lists.infradead.org [mailto:linux-mtd-bounces@lists.infradead.org] On Behalf Of Matej Kupljen Sent: Wednesday, April 04, 2012 05:17 To: Brian Norris Cc: linux-mtd Subject: Re: Unstable bits and JFFS2 Brian, > On Wed, Mar 28, 2012 at 4:16 AM, Matej Kupljen <matej.kupljen@gmail.com> wrote: >> during my experiments with the UBIFS crashing, I believe that I am experiencing >> the "unstable bit" issue. >> >> Would another FS, like JFFS2 work on such device, or I'd have the problems on >> such device with all file systems? > > I doubt it. JFFS2 is not supported much anymore, and it is not > designed for some of the problems with modern NAND flash. Aha, I see. Thank you for the information. > Have you read the information here? > > http://www.linux-mtd.infradead.org/doc/ubifs.html#L_unstable_bits Yes, I have. Artem already pointed me to that FAQ. > I'm not much of an export on UBIFS or unstable bits, but I think that > there are some unsolved problems. However, before ruling this an > unstable bits problem, it's a good idea to rule out any other possible > issues. Yes, I'll check the MTD driver. At the moment I am running again integrity check on my UBIFS for almost 24 hours now, but until now no error: # ./integck -n 0 -v -e /media/card The only difference from previous run, where I saw one bit flip is that previously I run the test with power failure option set to ON. Now it is off. Artem, you said that this unstable bits only happen during power cuts, is this right? Would those appear also on simulated power cuts, the ones that can integck produce? As for the ECC correction of the empty space, nothing is mentioned in the manual. Maybe I should ask the manufacturer of the chip (NXP LPC3152)?? It uses HW error correction included in the FLASH controller, with: "The error correction code used is Reed-Solomon over GF(2^9). The primitive polynomial g(x) over GF(2) is: g(x) = x9 + x4 +1" Thank you and Best Regards, Matej ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/ Click https://www.mailcontrol.com/sr/kqBJlvK+JBjTndxI!oX7Ulvwl1GqFAPrim4OaE2i4BWjrlQi2a8dR8ju1tDr2izUVJQZBBFsBrWtoBNtFUgrMg== to report this email as spam. This e-mail and the information, including any attachments, it contains are intended to be a confidential communication only to the person or entity to whom it is addressed and may contain information that is privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please immediately notify the sender and destroy the original message. Thank you. Please consider the environment before printing this email. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Unstable bits and JFFS2 2012-04-04 10:54 ` Atlant Schmidt @ 2012-04-10 7:22 ` Matej Kupljen 2012-04-13 16:27 ` Artem Bityutskiy 1 sibling, 0 replies; 12+ messages in thread From: Matej Kupljen @ 2012-04-10 7:22 UTC (permalink / raw) To: Atlant Schmidt; +Cc: Brian Norris, linux-mtd To all, >> Artem, you said that this unstable bits only happen >> during power cuts, is this right? Would those appear >> also on simulated power cuts, the ones that integck >> can produce? I've been trying to find the issue with my device over last week and for now, I didn't find anything. Here is what I've done: Please NOTE, that I have used THE SAME device for testing, which showed the problems in the past. 1) I've prepared SD card and booted the device from it. Then I run the integchk without simulating power cuts. The test run form more then 96 hours (four days and nights) and nothing happened. I mean, no errors were reported. 2) I made a simple "power cut simulator", which cuts the power to the device. The device was booted from SD card, then the UBIFS partition on NAND Flash was mounted. The test file was removed, and then a new file was created with simple dd from /dev/zero to the test file on UBIFS. The test run for 72 hours and again, nothing happened. 3) Now I am running the test, with power cut simulator, bit this time the device is booted from UBIFS. It has been running for more then 12h at the moment, and for now it is still working as expected. The only real thing from previous tests is that now, I have formatted the UBIFS from the device, when it was booted from SD card and Linux was up and running. Previous tests were done (when I've noticed the problems) only when U-Boot was running and the ubinized image was flashed to Flash trough U-Boot. Could this be the problem? Maybe different timings or something in U-Boot compared to Linux? Thank you and Best Regards, Matej Kupljen ^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: Unstable bits and JFFS2 2012-04-04 10:54 ` Atlant Schmidt 2012-04-10 7:22 ` Matej Kupljen @ 2012-04-13 16:27 ` Artem Bityutskiy 2012-04-13 16:40 ` Atlant Schmidt 2012-04-13 18:36 ` Wolfram Sang 1 sibling, 2 replies; 12+ messages in thread From: Artem Bityutskiy @ 2012-04-13 16:27 UTC (permalink / raw) To: Atlant Schmidt; +Cc: 'Matej Kupljen', Brian Norris, linux-mtd [-- Attachment #1: Type: text/plain, Size: 1483 bytes --] On Wed, 2012-04-04 at 06:54 -0400, Atlant Schmidt wrote: > Matej: > > > Artem, you said that this unstable bits only happen > > during power cuts, is this right? Would those appear > > also on simulated power cuts, the ones that integck > > can produce? > > (Note: This isn't Artem replying.) > > Unstable bits can happen anytime, but *REAL* power-cuts > while writing are certainly a great way to produce them; > after all, if you've only transferred half the electrons > to the floating gate when power goes away, well, that > Flash cell is now a roll-of-the-quantum-dice each time > you read it. > > Simulated power cuts (that just stop the software > processing at arbitrary and random points) can't > produce this effect. > > But any time a read-disturb or write-disturb takes place, > there's some probability that a Flash cell will be left > with a "near-threshold" charge on the floating gate, so > unstable bits are a fact of life that must be faced by > any software that drives NAND Flash memory chips. This > is, of course, especially true of MLC chips and even > more-true for TLC chips (with three bits per cell). Yeah, thanks for correcting. Yeah, read/write-disturb may make bits to become unstable, but we assume this is a slow process which will gradually make more and more bits flip and ECC will take care of that. So I think Matej can exclude this. -- Best Regards, Artem Bityutskiy [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: Unstable bits and JFFS2 2012-04-13 16:27 ` Artem Bityutskiy @ 2012-04-13 16:40 ` Atlant Schmidt 2012-04-13 17:01 ` Artem Bityutskiy 2012-04-13 18:36 ` Wolfram Sang 1 sibling, 1 reply; 12+ messages in thread From: Atlant Schmidt @ 2012-04-13 16:40 UTC (permalink / raw) To: 'dedekind1@gmail.com' Cc: 'Matej Kupljen', Brian Norris, linux-mtd Artem: > Yeah, thanks for correcting. Yeah, read/write-disturb may make bits > to become unstable, but we assume this is a slow process which will > gradually make more and more bits flip and ECC will take care of > that. So I think Matej can exclude this. Unfortunately, ECC can only fix those PEBs that are actually read. If one of the PEBs in your filesystem is being used entirely to contain obscure data that only gets read once in a blue moon (i.e., very rarely), then multiple read-/write-disturbs can hit accumulate in that PEB and when it is finally read, it may already contains too many errors to be corrected by the ECC. We've seen this exact scenario occur with the current UBI/UBIfs and will be implementing a userland "scrubbing" system to ensure that every PEB gets read at least once in a while (e.g., weekly or whatever interval seems right). Atlant -----Original Message----- From: Artem Bityutskiy [mailto:dedekind1@gmail.com] Sent: Friday, April 13, 2012 12:28 To: Atlant Schmidt Cc: 'Matej Kupljen'; Brian Norris; linux-mtd Subject: RE: Unstable bits and JFFS2 On Wed, 2012-04-04 at 06:54 -0400, Atlant Schmidt wrote: > Matej: > > > Artem, you said that this unstable bits only happen > > during power cuts, is this right? Would those appear > > also on simulated power cuts, the ones that integck > > can produce? > > (Note: This isn't Artem replying.) > > Unstable bits can happen anytime, but *REAL* power-cuts > while writing are certainly a great way to produce them; > after all, if you've only transferred half the electrons > to the floating gate when power goes away, well, that > Flash cell is now a roll-of-the-quantum-dice each time > you read it. > > Simulated power cuts (that just stop the software > processing at arbitrary and random points) can't > produce this effect. > > But any time a read-disturb or write-disturb takes place, > there's some probability that a Flash cell will be left > with a "near-threshold" charge on the floating gate, so > unstable bits are a fact of life that must be faced by > any software that drives NAND Flash memory chips. This > is, of course, especially true of MLC chips and even > more-true for TLC chips (with three bits per cell). Yeah, thanks for correcting. Yeah, read/write-disturb may make bits to become unstable, but we assume this is a slow process which will gradually make more and more bits flip and ECC will take care of that. So I think Matej can exclude this. -- Best Regards, Artem Bityutskiy This e-mail and the information, including any attachments, it contains are intended to be a confidential communication only to the person or entity to whom it is addressed and may contain information that is privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please immediately notify the sender and destroy the original message. Thank you. Please consider the environment before printing this email. ^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: Unstable bits and JFFS2 2012-04-13 16:40 ` Atlant Schmidt @ 2012-04-13 17:01 ` Artem Bityutskiy 2012-04-14 4:21 ` Ricard Wanderlof 0 siblings, 1 reply; 12+ messages in thread From: Artem Bityutskiy @ 2012-04-13 17:01 UTC (permalink / raw) To: Atlant Schmidt; +Cc: 'Matej Kupljen', Brian Norris, linux-mtd [-- Attachment #1: Type: text/plain, Size: 1389 bytes --] On Fri, 2012-04-13 at 12:40 -0400, Atlant Schmidt wrote: > Artem: > > > Yeah, thanks for correcting. Yeah, read/write-disturb may make bits > > to become unstable, but we assume this is a slow process which will > > gradually make more and more bits flip and ECC will take care of > > that. So I think Matej can exclude this. > > Unfortunately, ECC can only fix those PEBs that > are actually read. If one of the PEBs in your > filesystem is being used entirely to contain > obscure data that only gets read once in a > blue moon (i.e., very rarely), then multiple > read-/write-disturbs can hit accumulate in > that PEB and when it is finally read, it may > already contains too many errors to be > corrected by the ECC. > > We've seen this exact scenario occur with the > current UBI/UBIfs and will be implementing a > userland "scrubbing" system to ensure that > every PEB gets read at least once in a while > (e.g., weekly or whatever interval seems right). This is right, people should have a user-space app which periodically reads all /dev/ubiX_Y. Care to send a patch against mtd-www with some explantion of this issue and extend the unstable bits issue? But in Matej's case I do not believe he is testing long enough to hit this scenario. But everything is possible of course. -- Best Regards, Artem Bityutskiy [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: Unstable bits and JFFS2 2012-04-13 17:01 ` Artem Bityutskiy @ 2012-04-14 4:21 ` Ricard Wanderlof 2012-04-22 14:25 ` Artem Bityutskiy 0 siblings, 1 reply; 12+ messages in thread From: Ricard Wanderlof @ 2012-04-14 4:21 UTC (permalink / raw) To: Artem Bityutskiy Cc: 'Matej Kupljen', Brian Norris, linux-mtd, Atlant Schmidt On Fri, 13 Apr 2012, Artem Bityutskiy wrote: > On Fri, 2012-04-13 at 12:40 -0400, Atlant Schmidt wrote: >> >> Unfortunately, ECC can only fix those PEBs that >> are actually read. If one of the PEBs in your >> filesystem is being used entirely to contain >> obscure data that only gets read once in a >> blue moon (i.e., very rarely), then multiple >> read-/write-disturbs can hit accumulate in >> that PEB and when it is finally read, it may >> already contains too many errors to be >> corrected by the ECC. >> >> We've seen this exact scenario occur with the >> current UBI/UBIfs and will be implementing a >> userland "scrubbing" system to ensure that >> every PEB gets read at least once in a while >> (e.g., weekly or whatever interval seems right). > > This is right, people should have a user-space app which periodically > reads all /dev/ubiX_Y. Care to send a patch against mtd-www with some > explantion of this issue and extend the unstable bits issue? Would it not be preferable to have this functionality within UBI itself, so that, for instance governed by some sysfs parameter, UBI reads through all PEBs at some interval, triggering scrubbing of any blocks that require too much ECC for comfort? To me it seems something that userspace should not be concerned with, given the rest of the features that UBI deals with so that userspace does not have to. /Ricard -- Ricard Wolf Wanderlöf ricardw(at)axis.com Axis Communications AB, Lund, Sweden www.axis.com Phone +46 46 272 2016 Fax +46 46 13 61 30 ^ permalink raw reply [flat|nested] 12+ messages in thread
* RE: Unstable bits and JFFS2 2012-04-14 4:21 ` Ricard Wanderlof @ 2012-04-22 14:25 ` Artem Bityutskiy 0 siblings, 0 replies; 12+ messages in thread From: Artem Bityutskiy @ 2012-04-22 14:25 UTC (permalink / raw) To: Ricard Wanderlof Cc: 'Matej Kupljen', Brian Norris, linux-mtd, Atlant Schmidt [-- Attachment #1: Type: text/plain, Size: 361 bytes --] On Sat, 2012-04-14 at 06:21 +0200, Ricard Wanderlof wrote: > To me it seems something that userspace should not be concerned with, > given the rest of the features that UBI deals with so that userspace does > not have to. Yes, may probably you are right. Now we just need the brave knight who'd send the patch. -- Best Regards, Artem Bityutskiy [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Unstable bits and JFFS2 2012-04-13 16:27 ` Artem Bityutskiy 2012-04-13 16:40 ` Atlant Schmidt @ 2012-04-13 18:36 ` Wolfram Sang 2012-04-13 19:49 ` Artem Bityutskiy 1 sibling, 1 reply; 12+ messages in thread From: Wolfram Sang @ 2012-04-13 18:36 UTC (permalink / raw) To: Artem Bityutskiy Cc: 'Matej Kupljen', Brian Norris, linux-mtd, Atlant Schmidt [-- Attachment #1: Type: text/plain, Size: 1504 bytes --] Hi, > > > Artem, you said that this unstable bits only happen > > > during power cuts, is this right? Would those appear > > > also on simulated power cuts, the ones that integck > > > can produce? A bit of hijacking this thread, yet it fits somehow to the idea I got yesterday: There is the Open Project Proposal again, where the CEWG funds topics which improve embedded linux in general [1]. I wonder if it makes sense to hand in a proposal starting to tackle the 'unstable bits' issue. I could imagine task 1 mentioned in the wiki page (1. Improve the existing power cut emulation infrastructure in UBIFS and start emulating unstable bits. Start with emulating only one type of unstable bits, e.g., type 1.) could be a good candidate for such a proposal. Since I am not deeply involved with this topic: 1) Is this task still needed? 2) Is it a good task for a 2-3 week period (I'd think so) 3) Are there people around up for the task? I'd be willing to write the proposal, if nobody else volunteers. (yet, given my constraints, I am not the right person to actually work on it). Even if the proposal doesn't get contracted, the topic itself might get some attention since it will be discussed in the CEWG. What do you think? Regards, Wolfram [1] http://elinux.org/CEWG_Open_Project_Proposal_2012 -- Pengutronix e.K. | Wolfram Sang | Industrial Linux Solutions | http://www.pengutronix.de/ | [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Unstable bits and JFFS2 2012-04-13 18:36 ` Wolfram Sang @ 2012-04-13 19:49 ` Artem Bityutskiy 0 siblings, 0 replies; 12+ messages in thread From: Artem Bityutskiy @ 2012-04-13 19:49 UTC (permalink / raw) To: Wolfram Sang Cc: 'Matej Kupljen', Brian Norris, linux-mtd, Atlant Schmidt On Fri, 2012-04-13 at 20:36 +0200, Wolfram Sang wrote: > Hi, > > > > > Artem, you said that this unstable bits only happen > > > > during power cuts, is this right? Would those appear > > > > also on simulated power cuts, the ones that integck > > > > can produce? > > A bit of hijacking this thread, yet it fits somehow to the idea I got > yesterday: > > There is the Open Project Proposal again, where the CEWG funds topics which > improve embedded linux in general [1]. I wonder if it makes sense to hand in a > proposal starting to tackle the 'unstable bits' issue. I could imagine task 1 > mentioned in the wiki page (1. Improve the existing power cut emulation > infrastructure in UBIFS and start emulating unstable bits. Start with emulating > only one type of unstable bits, e.g., type 1.) could be a good candidate for > such a proposal. Since I am not deeply involved with this topic: > > 1) Is this task still needed? > 2) Is it a good task for a 2-3 week period (I'd think so) > 3) Are there people around up for the task? > > I'd be willing to write the proposal, if nobody else volunteers. (yet, given my > constraints, I am not the right person to actually work on it). Even if the > proposal doesn't get contracted, the topic itself might get some attention > since it will be discussed in the CEWG. Yes, definitely a hot topic now, I am swamped with lots of things and never found time for writing proposals like this. Please, go ahead, would be very appreciated. I might help reviewing it and amending. ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2012-04-22 14:27 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-03-28 11:16 Unstable bits and JFFS2 Matej Kupljen 2012-04-02 17:12 ` Brian Norris 2012-04-04 9:17 ` Matej Kupljen 2012-04-04 10:54 ` Atlant Schmidt 2012-04-10 7:22 ` Matej Kupljen 2012-04-13 16:27 ` Artem Bityutskiy 2012-04-13 16:40 ` Atlant Schmidt 2012-04-13 17:01 ` Artem Bityutskiy 2012-04-14 4:21 ` Ricard Wanderlof 2012-04-22 14:25 ` Artem Bityutskiy 2012-04-13 18:36 ` Wolfram Sang 2012-04-13 19:49 ` Artem Bityutskiy
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.