* JFFS2 mount time @ 2004-01-13 12:50 Jarkko Lavinen (NMP/Helsinki) 2004-01-13 13:19 ` Joakim Tjernlund ` (2 more replies) 0 siblings, 3 replies; 82+ messages in thread From: Jarkko Lavinen (NMP/Helsinki) @ 2004-01-13 12:50 UTC (permalink / raw) To: MTD List I am working with an embedded device with 64MB nand chip. I am able to read from chip at 6MB/s and able to read the whole chip in 10 seconds using character mtd device. The device has Omap1610 running at 190MHz, kernel 2.4.21. With 80% file system usage the mount time is 8 seconds which is unacceptable for root file system on a handheld device. I assume the scan time is linear with respect to the size of the flash partition and the file system usage. Is there anything that could be done to reduce the mount time of large JFFS2 partitions? Lazy mounting with prioritized scanning? Jarkko Lavinen ^ permalink raw reply [flat|nested] 82+ messages in thread
* RE: JFFS2 mount time 2004-01-13 12:50 JFFS2 mount time Jarkko Lavinen (NMP/Helsinki) @ 2004-01-13 13:19 ` Joakim Tjernlund 2004-01-13 13:39 ` David Woodhouse 2005-01-04 15:00 ` Steven Scholz 2 siblings, 0 replies; 82+ messages in thread From: Joakim Tjernlund @ 2004-01-13 13:19 UTC (permalink / raw) To: 'Jarkko Lavinen (NMP/Helsinki)', 'MTD List' > I am working with an embedded device with 64MB nand chip. I am able to > read from chip at 6MB/s and able to read the whole chip in 10 seconds > using character mtd device. The device has Omap1610 running at 190MHz, > kernel 2.4.21. > > With 80% file system usage the mount time is 8 seconds which is > unacceptable for root file system on a handheld device. I assume the > scan time is linear with respect to the size of the flash partition > and the file system usage. > > Is there anything that could be done to reduce the mount time of large > JFFS2 partitions? Lazy mounting with prioritized scanning? Yes, it is already done in JFFS2 CVS. Mount time for me is about a second on a 64MB JFFS2 NOR flash. Jocke ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-01-13 12:50 JFFS2 mount time Jarkko Lavinen (NMP/Helsinki) 2004-01-13 13:19 ` Joakim Tjernlund @ 2004-01-13 13:39 ` David Woodhouse 2004-01-13 15:25 ` Kenneth Johansson 2005-01-04 15:00 ` Steven Scholz 2 siblings, 1 reply; 82+ messages in thread From: David Woodhouse @ 2004-01-13 13:39 UTC (permalink / raw) To: Jarkko Lavinen (NMP/Helsinki); +Cc: MTD List On Tue, 2004-01-13 at 14:50 +0200, Jarkko Lavinen (NMP/Helsinki) wrote: > Is there anything that could be done to reduce the mount time of large > JFFS2 partitions? Lazy mounting with prioritized scanning? We improved it a lot by not checking CRC32 on all nodes until later -- but we still need to read the whole medium at mount time. We _can_ fix that. There's a limited amount of knowledge we're gaining from our scan at mount time. For each inode on the medium, we need: Its nlink A list of the physical addresses of all its nodes We currently read the whole of every eraseblock to work this out. However, we could write a block 'tailer' at the _end_ of each eraseblock as we fill it, containing enough information in compressed form about the contents of the eraseblock. That would mean that we only have to read a small proportion of each properly-finished eraseblock, and should speed up the mount significantly. -- dwmw2 ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-01-13 13:39 ` David Woodhouse @ 2004-01-13 15:25 ` Kenneth Johansson 2004-01-13 15:30 ` David Woodhouse 0 siblings, 1 reply; 82+ messages in thread From: Kenneth Johansson @ 2004-01-13 15:25 UTC (permalink / raw) To: David Woodhouse; +Cc: MTD List, Jarkko Lavinen (NMP/Helsinki) On Tue, 2004-01-13 at 14:39, David Woodhouse wrote: > On Tue, 2004-01-13 at 14:50 +0200, Jarkko Lavinen (NMP/Helsinki) wrote: > > Is there anything that could be done to reduce the mount time of large > > JFFS2 partitions? Lazy mounting with prioritized scanning? > > We improved it a lot by not checking CRC32 on all nodes until later -- > but we still need to read the whole medium at mount time. > > We _can_ fix that. There's a limited amount of knowledge we're gaining > from our scan at mount time. For each inode on the medium, we need: > Its nlink > A list of the physical addresses of all its nodes > > We currently read the whole of every eraseblock to work this out. > However, we could write a block 'tailer' at the _end_ of each eraseblock > as we fill it, containing enough information in compressed form about > the contents of the eraseblock. > > That would mean that we only have to read a small proportion of each > properly-finished eraseblock, and should speed up the mount > significantly. I have done some work on the JFFS2 code in u-boot and found that the fastest way was to load the entire FLASH into a continuous disk image in DRAM and then have code that directly dereference pointers into this image without any access functions or read on demand. When I tried to be smart and skip the copy and only read in what needed to be read that was significantly slower. Never underestimate brute force :) If you have very very poor bandwidth from the flash the the reverse is true but I had to slow down my flash 10 times to break even. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-01-13 15:25 ` Kenneth Johansson @ 2004-01-13 15:30 ` David Woodhouse 2004-01-13 16:21 ` Kenneth Johansson 0 siblings, 1 reply; 82+ messages in thread From: David Woodhouse @ 2004-01-13 15:30 UTC (permalink / raw) To: Kenneth Johansson; +Cc: MTD List, Jarkko Lavinen (NMP/Helsinki) On Tue, 2004-01-13 at 16:25 +0100, Kenneth Johansson wrote: > I have done some work on the JFFS2 code in u-boot and found that the > fastest way was to load the entire FLASH into a continuous disk image in > DRAM and then have code that directly dereference pointers into this > image without any access functions or read on demand. What we do in the current JFFS2 code is similar. We directly dereference pointers into the _flash_ without access functions, rather than copying it into DRAM first. > When I tried to be smart and skip the copy and only read in what needed > to be read that was significantly slower. Bear in mind that the question was about NAND flash though -- it's not directly deferenceable, and you're not going to get the same results. -- dwmw2 ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-01-13 15:30 ` David Woodhouse @ 2004-01-13 16:21 ` Kenneth Johansson 2004-01-13 17:29 ` Jörn Engel 0 siblings, 1 reply; 82+ messages in thread From: Kenneth Johansson @ 2004-01-13 16:21 UTC (permalink / raw) To: David Woodhouse; +Cc: Jarkko Lavinen (NMP/Helsinki), MTD List On Tue, 2004-01-13 at 16:30, David Woodhouse wrote: > > When I tried to be smart and skip the copy and only read in what needed > > to be read that was significantly slower. > > Bear in mind that the question was about NAND flash though -- it's not > directly deferenceable, and you're not going to get the same results. hmm I reread the firt post why is read so slow?? When I thought about using nand I read that reading a byte took something like 50 ns. That is like 20MB per second for an 8 bit device. This is in the region where it did not pay to be smart,the accesses overhead to do read only what's needed is greater than the time wasted reading uneccessary data. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-01-13 16:21 ` Kenneth Johansson @ 2004-01-13 17:29 ` Jörn Engel 2004-01-13 17:40 ` Kenneth Johansson 0 siblings, 1 reply; 82+ messages in thread From: Jörn Engel @ 2004-01-13 17:29 UTC (permalink / raw) To: Kenneth Johansson Cc: Jarkko Lavinen (NMP/Helsinki), David Woodhouse, MTD List On Tue, 13 January 2004 17:21:15 +0100, Kenneth Johansson wrote: > > hmm I reread the firt post why is read so slow?? When I thought about > using nand I read that reading a byte took something like 50 ns. That is > like 20MB per second for an 8 bit device. This is in the region where it > did not pay to be smart,the accesses overhead to do read only what's > needed is greater than the time wasted reading uneccessary data. But on nand it is not only possible, but also likely that flash size is greater than ram size. Does your approach still work then? Jörn -- To announce that there must be no criticism of the President, or that we are to stand by the President, right or wrong, is not only unpatriotic and servile, but is morally treasonable to the American public. -- Theodore Roosevelt, Kansas City Star, 1918 ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-01-13 17:29 ` Jörn Engel @ 2004-01-13 17:40 ` Kenneth Johansson 2004-01-13 18:43 ` Jörn Engel 0 siblings, 1 reply; 82+ messages in thread From: Kenneth Johansson @ 2004-01-13 17:40 UTC (permalink / raw) To: Jörn Engel; +Cc: Jarkko Lavinen (NMP/Helsinki), David Woodhouse, MTD List On Tue, 2004-01-13 at 18:29, Jörn Engel wrote: > On Tue, 13 January 2004 17:21:15 +0100, Kenneth Johansson wrote: > > > > hmm I reread the firt post why is read so slow?? When I thought about > > using nand I read that reading a byte took something like 50 ns. That is > > like 20MB per second for an 8 bit device. This is in the region where it > > did not pay to be smart,the accesses overhead to do read only what's > > needed is greater than the time wasted reading uneccessary data. > > But on nand it is not only possible, but also likely that flash size > is greater than ram size. Does your approach still work then? > > Jörn Obviously no. But maybe it's still cheaper to do the work in big chunks I don't know you need to actually do a test to see what happens. It's not so easy to see in advance what will happen. I thought for sure that it would be faster to only read what was needed but it was not, at least not on a 200 MHz ppc if you have an P4 3GHz the picture is quite different. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-01-13 17:40 ` Kenneth Johansson @ 2004-01-13 18:43 ` Jörn Engel 0 siblings, 0 replies; 82+ messages in thread From: Jörn Engel @ 2004-01-13 18:43 UTC (permalink / raw) To: Kenneth Johansson Cc: David Woodhouse, Jarkko Lavinen (NMP/Helsinki), MTD List On Tue, 13 January 2004 18:40:09 +0100, Kenneth Johansson wrote: > > > > But on nand it is not only possible, but also likely that flash size > > is greater than ram size. Does your approach still work then? > > Obviously no. But maybe it's still cheaper to do the work in big chunks > I don't know you need to actually do a test to see what happens. It's > not so easy to see in advance what will happen. I thought for sure that > it would be faster to only read what was needed but it was not, at least > not on a 200 MHz ppc if you have an P4 3GHz the picture is quite > different. Could be, at least. As long as we don't know where the time is lost, everything else remains guesswork. Anyway, David's approach sounds quite sane to me. Jörn -- A victorious army first wins and then seeks battle. -- Sun Tzu ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-01-13 12:50 JFFS2 mount time Jarkko Lavinen (NMP/Helsinki) 2004-01-13 13:19 ` Joakim Tjernlund 2004-01-13 13:39 ` David Woodhouse @ 2005-01-04 15:00 ` Steven Scholz 2005-01-05 10:45 ` Artem B. Bityuckiy 2 siblings, 1 reply; 82+ messages in thread From: Steven Scholz @ 2005-01-04 15:00 UTC (permalink / raw) To: MTD List Hi, > Is there anything that could be done to reduce the mount time of large > JFFS2 partitions? Lazy mounting with prioritized scanning? I wonder if the current CVS head is much better (i.e. faster) in mounting a JFFS2 root fs then linux-2.6.10? Or in other words: does it make sense to use the current CVS instead of the code in linux-2.6.10? -- Steven ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2005-01-04 15:00 ` Steven Scholz @ 2005-01-05 10:45 ` Artem B. Bityuckiy 2005-01-05 11:10 ` Ferenc Havasi 0 siblings, 1 reply; 82+ messages in thread From: Artem B. Bityuckiy @ 2005-01-05 10:45 UTC (permalink / raw) To: Steven Scholz; +Cc: MTD List Hi Try JFFS3 with "summaries" enabled or Ferenc Havasi's JFFS2 "summary" patch. The mount speed should be increased with it. Note: JFFS3 development is in progress so be ready if it has bugs. Steven Scholz wrote: > Hi, > >> Is there anything that could be done to reduce the mount time of large >> JFFS2 partitions? Lazy mounting with prioritized scanning? > > > I wonder if the current CVS head is much better (i.e. faster) in > mounting a JFFS2 root fs then linux-2.6.10? > > Or in other words: does it make sense to use the current CVS instead of > the code in linux-2.6.10? > > -- > Steven > > ______________________________________________________ > Linux MTD discussion mailing list > http://lists.infradead.org/mailman/listinfo/linux-mtd/ > -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2005-01-05 10:45 ` Artem B. Bityuckiy @ 2005-01-05 11:10 ` Ferenc Havasi 2005-01-05 11:24 ` Steven Scholz 0 siblings, 1 reply; 82+ messages in thread From: Ferenc Havasi @ 2005-01-05 11:10 UTC (permalink / raw) To: Steven Scholz; +Cc: MTD List Hi Steven, > Try JFFS3 with "summaries" enabled or Ferenc Havasi's JFFS2 "summary" > patch. The mount speed should be increased with it. > > Note: JFFS3 development is in progress so be ready if it has bugs. Becauseof it, we maintain a summary patch for JFFS2. You can download the latest patch from http://www.inf.u-szeged.hu/jffs2/mount.php (I'm updating the page now, so you may wait a little.) Summary designed specially to speed up the mounttime, so it should help you. Regards, Ferenc ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2005-01-05 11:10 ` Ferenc Havasi @ 2005-01-05 11:24 ` Steven Scholz 2005-01-05 11:45 ` Artem B. Bityuckiy 0 siblings, 1 reply; 82+ messages in thread From: Steven Scholz @ 2005-01-05 11:24 UTC (permalink / raw) To: MTD List Hi Ferenc, >> Note: JFFS3 development is in progress so be ready if it has bugs. > > Becauseof it, we maintain a summary patch for JFFS2. You can download > the latest patch from http://www.inf.u-szeged.hu/jffs2/mount.php > (I'm updating the page now, so you may wait a little.) > > Summary designed specially to speed up the mounttime, so it should help > you. Thanks a million. I'll try that. I guess I can just use your Patch on a vanilla linux-2.6.10!? BTW: On the link mentioned above I read "Speed up the mount time of JFFS2 - specially when it uses NAND flash. " How about NOR flash? -- Steven ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2005-01-05 11:24 ` Steven Scholz @ 2005-01-05 11:45 ` Artem B. Bityuckiy 2005-01-06 10:50 ` Ferenc Havasi 2005-01-10 15:14 ` Steven Scholz 0 siblings, 2 replies; 82+ messages in thread From: Artem B. Bityuckiy @ 2005-01-05 11:45 UTC (permalink / raw) To: Steven Scholz; +Cc: MTD List > Thanks a million. I'll try that. I guess I can just use your Patch on a > vanilla linux-2.6.10!? > > BTW: On the link mentioned above I read "Speed up the mount time of > JFFS2 - specially when it uses NAND flash. " How about NOR flash? > My understanding of the problem is that summary may help in case of NOR, but in general it shouldn't increase the mount speed a lot (am I wrong?). If we change the patch a bit (asking not to check any CRC in the block has summary) we may have better mount speed (and iget()). And possibly in case of nor we even doesn't need to store any data in summary - only header. Header will show us that the block was successfully written (no unclean reboots) and we my trust it and do not check CRC. Moreover, looking through the patch I suspect it wasn't tested with NOR, is it? -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2005-01-05 11:45 ` Artem B. Bityuckiy @ 2005-01-06 10:50 ` Ferenc Havasi 2005-01-10 15:14 ` Steven Scholz 1 sibling, 0 replies; 82+ messages in thread From: Ferenc Havasi @ 2005-01-06 10:50 UTC (permalink / raw) To: Artem B. Bityuckiy; +Cc: MTD List, Steven Scholz Hi Steven, >> Thanks a million. I'll try that. I guess I can just use your Patch on >> a vanilla linux-2.6.10!? >> >> BTW: On the link mentioned above I read "Speed up the mount time of >> JFFS2 - specially when it uses NAND flash. " How about NOR flash? >> > My understanding of the problem is that summary may help in case of NOR, > but in > general it shouldn't increase the mount speed a lot (am I wrong?). Artem is right. > Moreover, looking through the patch I suspect it wasn't tested with NOR, > is it? It should work with NOR, too. Now we tested it again, and noticed some problems. One of them are "general", and an other is NOR specific. We will fix it soon, so you may wait a little before testing it. > If we change > the patch a bit (asking not to check any CRC in the block has summary) > we may > have better mount speed (and iget()). And possibly in case of nor we > even doesn't > need to store any data in summary - only header. Header will show us > that the block > was successfully written (no unclean reboots) and we my trust it and > do not check CRC. Yes, if everything will be stable we should start to make it more optimal. Bye, Ferenc ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2005-01-05 11:45 ` Artem B. Bityuckiy 2005-01-06 10:50 ` Ferenc Havasi @ 2005-01-10 15:14 ` Steven Scholz 2005-01-10 15:25 ` Steven Scholz 2005-01-11 8:26 ` Ferenc Havasi 1 sibling, 2 replies; 82+ messages in thread From: Steven Scholz @ 2005-01-10 15:14 UTC (permalink / raw) To: MTD List Hi, >> Thanks a million. I'll try that. I guess I can just use your Patch on >> a vanilla linux-2.6.10!? >> >> BTW: On the link mentioned above I read "Speed up the mount time of >> JFFS2 - specially when it uses NAND flash. " How about NOR flash? >> > My understanding of the problem is that summary may help in case of NOR, > but in > general it shouldn't increase the mount speed a lot (am I wrong?). As I wrote in another thread the summary patch reduced the mount time from 6 seconds to less than 1 second on my embedded ARM9 system with 8MB NAND flash. But now GC still slows down the start up of user applications as it checks all CRCs right after booting. > If we change > the patch a bit (asking not to check any CRC in the block has summary) > we may > have better mount speed (and iget()). And possibly in case of nor we > even doesn't > need to store any data in summary - only header. Header will show us > that the block > was successfully written (no unclean reboots) and we my trust it and do > not check CRC. Makes sense to me. Ferenc, what do you think about that? No need to check CRC if we have a summary, right? This could fix my GC problem mentioned above and in the thread http://lists.infradead.org/pipermail/linux-mtd/2005-January/011411.html -- Steven ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2005-01-10 15:14 ` Steven Scholz @ 2005-01-10 15:25 ` Steven Scholz 2005-01-11 8:26 ` Ferenc Havasi 1 sibling, 0 replies; 82+ messages in thread From: Steven Scholz @ 2005-01-10 15:25 UTC (permalink / raw) To: MTD List Steven Scholz wrote: > As I wrote in another thread the summary patch reduced the mount time > from 6 seconds to less than 1 second on my embedded ARM9 system with 8MB > NAND flash. Of course I meant the good old NOR flash. -- Steven ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2005-01-10 15:14 ` Steven Scholz 2005-01-10 15:25 ` Steven Scholz @ 2005-01-11 8:26 ` Ferenc Havasi 2005-01-11 8:34 ` Steven Scholz 1 sibling, 1 reply; 82+ messages in thread From: Ferenc Havasi @ 2005-01-11 8:26 UTC (permalink / raw) To: Steven Scholz; +Cc: MTD List Hi Steven, > As I wrote in another thread the summary patch reduced the mount time > from 6 seconds to less than 1 second on my embedded ARM9 system with 8MB > NAND flash. > But now GC still slows down the start up of user applications as it > checks all CRCs right after booting. >> If we change >> the patch a bit (asking not to check any CRC in the block has summary) >> we may >> have better mount speed (and iget()). And possibly in case of nor we >> even doesn't >> need to store any data in summary - only header. Header will show us >> that the block >> was successfully written (no unclean reboots) and we my trust it and >> do not check CRC. If I am right CRCs are not only against the effect of unclean reboots but also to handle flash errors. On NAND flashes the ECC handles this problem but NOR doesn't have any error detection system. > Ferenc, what do you think about that? > No need to check CRC if we have a summary, right? Give us some days to invetigate it more deeply. I think we will be able to imrove the summary patch in some way. Bye, Ferenc ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2005-01-11 8:26 ` Ferenc Havasi @ 2005-01-11 8:34 ` Steven Scholz 0 siblings, 0 replies; 82+ messages in thread From: Steven Scholz @ 2005-01-11 8:34 UTC (permalink / raw) To: Ferenc Havasi; +Cc: MTD List Hi Ferenc, > If I am right CRCs are not only against the effect of unclean reboots > but also to handle flash errors. On NAND flashes the ECC handles this > problem but NOR doesn't have any error detection system. > >> Ferenc, what do you think about that? >> No need to check CRC if we have a summary, right? > > Give us some days to invetigate it more deeply. I think we will be able > to imrove the summary patch in some way. I now get Eep. Child "libutil.so" (ino #230) of dir ino #5 doesn't exist! Eep. Child "libthread_db.so" (ino #227) of dir ino #5 doesn't exist! Eep. Child "libutil.so.1" (ino #231) of dir ino #5 doesn't exist! Eep. Child "libthread_db-1.0.so" (ino #226) of dir ino #5 doesn't exist! Eep. Child "libutil-2.3.3.so" (ino #229) of dir ino #5 doesn't exist! Eep. Child "libthread_db.so.1" (ino #228) of dir ino #5 doesn't exist! Eep. Child "adjtimex" (ino #237) of dir ino #9 doesn't exist! Eep. Child "backlight" (ino #238) of dir ino #9 doesn't exist! Eep. Child "log" (ino #299) of dir ino #13 doesn't exist! Eep. Child "demo_320_240_16.xml" (ino #234) of dir ino #232 doesn't exist! Eep. Child "demo.xml" (ino #233) of dir ino #232 doesn't exist! Eep. Child "fonttest" (ino #235) of dir ino #232 doesn't exist! VFS: Mounted root (jffs2 filesystem). Freeing init memory: 72K while booting! :-( I am not sure if this has to do with the summary patch. How can I find out? -- Steven ^ permalink raw reply [flat|nested] 82+ messages in thread
* JFFS2 mount time @ 2004-10-20 14:26 Ferenc Havasi 2004-10-20 15:26 ` [OBORONA-SPAM] " Artem B. Bityuckiy ` (3 more replies) 0 siblings, 4 replies; 82+ messages in thread From: Ferenc Havasi @ 2004-10-20 14:26 UTC (permalink / raw) To: linux-mtd, jffs-dev, dwmw2 [-- Attachment #1: Type: text/plain, Size: 1792 bytes --] Dear All, Here is the latest version of our mount time improvement. Using of it: - apply this patch on the latest version of MTD - compile sumtool (make command in mtd/util) - make your JFFS2 image as before (or you can use already created images as well) - run sumtool to insert summary information, for example: ./sumtool -i original.jffs2 -o new.jffs2 -e128KiB - recompile your kernel with "JFFS2 inode summary support" Jarkko made a measurement on a real NAND device: his JFFS2 image was 120819928 (115M), after running sumtool the new image was 123338752 (117M). Using the original mount time was 55 sec, with the new image it is only 8.5 sec. It works very similar as our previous improvement: stores special information at the end of the erase blocks, and at mount time if there is this kind of information the scaning of the erase block is unneccessary. New things compared to our previous improvement: - it was fully rewritten - we separated the user space tool from mkfs. (sumtool) - sumtool now not only inserts the summary information but also make some node-reordering. There will be two kind of erase blocks: in the "first type" there will be only jffs2_raw_inodes, and all other node (jffs2_raw_dirent) will be stored in the "second type". It generates summary at the end of all "fist type" eraseblock. (the "second type" will be scanned as before, because all information is needed in jffs_raw_dirent at mount time) Ceratinly all of these things are optional (as you can see above you have to select it from kernel config). The JFFS2 image produced by sumtool is also usable with previous kernel because the summary node is JFFS2_FEATURE_RWCOMPAT_DELETE. I think it can be usefull not only for us. David, may I commit it to the CVS? Regards, Ferenc [-- Attachment #2: jffs2-summary.patch --] [-- Type: text/x-patch, Size: 36792 bytes --] diff --unified --recursive --new-file mtd2/fs/Kconfig mtd/fs/Kconfig --- mtd2/fs/Kconfig 2004-07-16 17:20:59.000000000 +0200 +++ mtd/fs/Kconfig 2004-10-20 15:10:42.000000000 +0200 @@ -68,6 +68,19 @@ Say 'N' unless you have NAND flash and you are willing to test and develop JFFS2 support for it. +config JFFS2_FS_SUMMARY + bool "JFFS2 inode summary support (EXPERIMENTAL)" + depends on JFFS2_FS + default n + help + This feature makes it possible to use inode summary information + for faster filesystem mount - specially on NAND. + + The summary information can be inserted into a filesystem image + by the utility 'sumtool'. + + If unsure, say 'N'. + config JFFS2_COMPRESSION_OPTIONS bool "Advanced compression options for JFFS2" default n diff --unified --recursive --new-file mtd2/fs/jffs2/scan.c mtd/fs/jffs2/scan.c --- mtd2/fs/jffs2/scan.c 2004-09-12 11:56:13.000000000 +0200 +++ mtd/fs/jffs2/scan.c 2004-10-20 15:44:25.000000000 +0200 @@ -4,6 +4,8 @@ * Copyright (C) 2001-2003 Red Hat, Inc. * * Created by David Woodhouse <dwmw2@redhat.com> + * Inode summary support by Zoltan Sogor, Ferenc Havasi, Patrik Kluba + * University of Szeged, Hungary * * For licensing information, see the file 'LICENCE' in this directory. * @@ -58,6 +60,11 @@ static int jffs2_scan_dirent_node(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb, struct jffs2_raw_dirent *rd, uint32_t ofs); + +#ifdef CONFIG_JFFS2_FS_SUMMARY +static struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino); +#endif + #define BLK_STATE_ALLFF 0 #define BLK_STATE_CLEAN 1 #define BLK_STATE_PARTDIRTY 2 @@ -292,6 +299,25 @@ #ifdef CONFIG_JFFS2_FS_NAND int cleanmarkerfound = 0; #endif +#ifdef CONFIG_JFFS2_FS_SUMMARY + struct jffs2_raw_node_ref *raw; + struct jffs2_raw_node_ref *cache_ref; + struct jffs2_inode_cache *ic; + + typedef struct sum_marker { + jint32_t offset; + jint32_t magic; + } sum_marker; + + sum_marker *sm; + int i; + int sumsize; + uint32_t ino; + uint32_t crc; + struct jffs2_inode_sum_node *summary; + struct jffs2_inode_sum_record *sum_rec; + int bad_sum = 0; +#endif ofs = jeb->offset; prevofs = jeb->offset - 1; @@ -314,10 +340,217 @@ } } #endif + +#ifdef CONFIG_JFFS2_FS_SUMMARY + /* Looking for summary marker */ + sm = (sum_marker *)kmalloc(sizeof(*sm), GFP_KERNEL); + if (!sm) { + return -ENOMEM; + } + + err = jffs2_fill_scan_buf(c, (unsigned char *) sm, jeb->offset + c->sector_size - 8, 8); + + if (err) { + return err; + } + + if (je32_to_cpu(sm->magic) == JFFS2_SUM_MAGIC) { + ofs = je32_to_cpu(sm->offset); + sumsize = c->sector_size - ofs; + ofs += jeb->offset; + + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Inode summary information found at 0x%x (%d bytes)\n", ofs, sumsize)); + + summary = (struct jffs2_inode_sum_node *) kmalloc(sumsize, GFP_KERNEL); + + if (!summary) { + kfree(sm); + return -ENOMEM; + } + + err = jffs2_fill_scan_buf(c, (unsigned char *)summary, ofs, sumsize); + + if (err) { + kfree(sm); + kfree(summary); + return err; + } + + /* OK, now check for node validity and CRC */ + crcnode.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK); + crcnode.nodetype = cpu_to_je16(JFFS2_NODETYPE_INODE_SUM); + crcnode.totlen = summary->totlen; + hdr_crc = crc32(0, &crcnode, sizeof(crcnode)-4); + + if (je32_to_cpu(summary->hdr_crc) != hdr_crc) { + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node header is corrupt (bad CRC or no summary at all)\n")); + bad_sum = 1; + } + + if ((!bad_sum) && (je32_to_cpu(summary->totlen) != sumsize)) { + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (wrong erasesize?)\n")); + bad_sum = 1; + } + + crc = crc32(0, summary, sizeof(struct jffs2_inode_sum_node)-8); + + if ((!bad_sum) && (je32_to_cpu(summary->node_crc) != crc)) { + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (bad CRC)\n")); + bad_sum = 1; + } + + sum_rec = (struct jffs2_inode_sum_record *) &(summary->sum[0]); + crc = crc32(0, sum_rec, sumsize - sizeof(struct jffs2_inode_sum_node)); + + if ((!bad_sum) && (je32_to_cpu(summary->sum_crc) != crc)) { + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node data is corrupt (bad CRC)\n")); + bad_sum = 1; + } + + if (!bad_sum) { + + if ( je32_to_cpu(summary->cln_mkr) ){ + + D1(printk(KERN_DEBUG "Summary : CLEANMARKER node \n")); + + if (je32_to_cpu(summary->cln_mkr) != c->cleanmarker_size) { + printk(KERN_DEBUG "CLEANMARKER node has totlen 0x%x != normal 0x%x\n", + je32_to_cpu(summary->cln_mkr), c->cleanmarker_size); + UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) ); + } + else if (jeb->first_node) { + printk(KERN_DEBUG "CLEANMARKER node not first node in block (0x%08x)\n", jeb->offset); + UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) ); + } + else { + struct jffs2_raw_node_ref *marker_ref = jffs2_alloc_raw_node_ref(); + + if (!marker_ref) { + printk(KERN_NOTICE "Failed to allocate node ref for clean marker\n"); + return -ENOMEM; + } + + marker_ref->next_in_ino = NULL; + marker_ref->next_phys = NULL; + marker_ref->flash_offset = jeb->offset | REF_NORMAL; + marker_ref->__totlen = je32_to_cpu(summary->cln_mkr); + jeb->first_node = jeb->last_node = marker_ref; + + USED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) ); + + } + } + + for(i = 0; i < je16_to_cpu(summary->sum_num); i++) { + + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Processing summary information %d\n", i)); + + //JFFS2_NODETYPE_INODE: + ino = je32_to_cpu(sum_rec->inode); + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Inode at 0x%08x\n", jeb->offset + je32_to_cpu(sum_rec->offset))); + raw = jffs2_alloc_raw_node_ref(); + if (!raw) { + printk(KERN_NOTICE "jffs2_scan_eraseblock(): allocation of node reference failed\n"); + kfree(sm); + kfree(summary); + return -ENOMEM; + } + + ic = jffs2_get_ino_cache(c, ino); + if (!ic) { + ic = jffs2_scan_make_ino_cache(c, ino); + if (!ic) { + printk(KERN_NOTICE "jffs2_scan_eraseblock(): scan_make_ino_cache failed\n"); + jffs2_free_raw_node_ref(raw); + kfree(sm); + kfree(summary); + return -ENOMEM; + } + } + + raw->flash_offset = (jeb->offset + je32_to_cpu(sum_rec->offset)) | REF_UNCHECKED; + raw->__totlen = PAD(je32_to_cpu(sum_rec->totlen)); + raw->next_phys = NULL; + raw->next_in_ino = ic->nodes; + + ic->nodes = raw; + if (!jeb->first_node) + jeb->first_node = raw; + if (jeb->last_node) + jeb->last_node->next_phys = raw; + jeb->last_node = raw; + + /* do we need this? this requires storing another 4 bytes per record in the cache or an expensive reading */ + pseudo_random += je32_to_cpu(sum_rec->version); + + UNCHECKED_SPACE(PAD(je32_to_cpu(sum_rec->totlen))); + + sum_rec++; + } + + kfree(sm); + kfree(summary); + + /* for ACCT_PARANOIA_CHECK */ + cache_ref = jffs2_alloc_raw_node_ref(); + + if (!cache_ref) { + printk(KERN_NOTICE "Failed to allocate node ref for cache\n"); + return -ENOMEM; + } + + cache_ref->next_in_ino = NULL; + cache_ref->next_phys = NULL; + cache_ref->flash_offset = ofs | REF_NORMAL; + cache_ref->__totlen = sumsize; + + if (!jeb->first_node) + jeb->first_node = cache_ref; + if (jeb->last_node) + jeb->last_node->next_phys = cache_ref; + jeb->last_node = cache_ref; + + USED_SPACE(sumsize); + + /* somebody check this and all of space accounting in summary support */ + + if ((jeb->used_size + jeb->unchecked_size) == PAD(c->cleanmarker_size) && !jeb->dirty_size + && (!jeb->first_node || !jeb->first_node->next_in_ino) ) { + return BLK_STATE_CLEANMARKER; + } + /* move blocks with max 4 byte dirty space to cleanlist */ + else if (!ISDIRTY(c->sector_size - (jeb->used_size + jeb->unchecked_size))) { + c->dirty_size -= jeb->dirty_size; + c->wasted_size += jeb->dirty_size; + jeb->wasted_size += jeb->dirty_size; + jeb->dirty_size = 0; + return BLK_STATE_CLEAN; + } + else if (jeb->used_size || jeb->unchecked_size) { + return BLK_STATE_PARTDIRTY; + } + else { + return BLK_STATE_ALLDIRTY; + } + } + } + D1(printk(KERN_DEBUG "Summary end\n")); + + ofs = jeb->offset; + prevofs = jeb->offset - 1; + +#endif + buf_ofs = jeb->offset; if (!buf_size) { buf_len = c->sector_size; +#ifdef CONFIG_JFFS2_FS_SUMMARY + /* must reread because of summary test */ + err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len); + if (err) + return err; +#endif } else { buf_len = EMPTY_SCAN_SIZE; err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len); diff --unified --recursive --new-file mtd2/include/linux/jffs2.h mtd/include/linux/jffs2.h --- mtd2/include/linux/jffs2.h 2004-05-25 13:31:55.000000000 +0200 +++ mtd/include/linux/jffs2.h 2004-10-20 14:53:52.000000000 +0200 @@ -28,6 +28,9 @@ #define JFFS2_EMPTY_BITMASK 0xffff #define JFFS2_DIRTY_BITMASK 0x0000 +/* Summary node MAGIC marker */ +#define JFFS2_SUM_MAGIC 0x02851885 + /* We only allow a single char for length, and 0xFF is empty flash so we don't want it confused with a real length. Hence max 254. */ @@ -61,6 +64,7 @@ #define JFFS2_NODETYPE_INODE (JFFS2_FEATURE_INCOMPAT | JFFS2_NODE_ACCURATE | 2) #define JFFS2_NODETYPE_CLEANMARKER (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3) #define JFFS2_NODETYPE_PADDING (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 4) +#define JFFS2_NODETYPE_INODE_SUM (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 6) // Maybe later... //#define JFFS2_NODETYPE_CHECKPOINT (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3) @@ -148,10 +152,31 @@ uint8_t data[0]; } __attribute__((packed)); +struct jffs2_inode_sum_node{ + jint16_t magic; + jint16_t nodetype; /* = JFFS2_NODETYPE_INODE_SUM */ + jint32_t totlen; + jint32_t hdr_crc; + jint16_t sum_num; /* number of sum entries*/ + jint32_t cln_mkr; /* clean marker size, 0 = no cleanmarker */ + jint32_t sum_crc; /* summary information crc */ + jint32_t node_crc; /* node crc */ + jint32_t sum[0]; /* inode summary info */ +} __attribute__((packed)); + +struct jffs2_inode_sum_record{ + jint32_t inode; + jint32_t version; + jint32_t offset; + jint32_t totlen; +} __attribute__((packed)); + + union jffs2_node_union { struct jffs2_raw_inode i; struct jffs2_raw_dirent d; struct jffs2_unknown_node u; + struct jffs2_inode_sum_node s; }; #endif /* __LINUX_JFFS2_H__ */ diff --unified --recursive --new-file mtd2/util/Makefile mtd/util/Makefile --- mtd2/util/Makefile 2004-07-13 19:49:43.000000000 +0200 +++ mtd/util/Makefile 2004-10-19 15:11:02.000000000 +0200 @@ -13,7 +13,7 @@ TARGETS = ftl_format flash_erase flash_eraseall nanddump doc_loadbios \ mkfs.jffs ftl_check mkfs.jffs2 flash_lock flash_unlock \ flash_info mtd_debug flashcp nandwrite jffs2dump \ - nftldump nftl_format docfdisk #jffs2reader + nftldump nftl_format docfdisk sumtool #jffs2reader SYMLINKS = compr_lzari.c compr_lzo.c @@ -48,6 +48,9 @@ jffs2dump: jffs2dump.o crc32.o $(CC) $(LDFLAGS) -o $@ $^ +sumtool: sumtool.o crc32.o + $(CC) $(LDFLAGS) -o $@ $^ + install: ${TARGETS} mkdir -p ${DESTDIR}/${SBINDIR} install -m0755 -oroot -groot ${TARGETS} ${DESTDIR}/${SBINDIR}/ diff --unified --recursive --new-file mtd2/util/jffs2dump.c mtd/util/jffs2dump.c --- mtd2/util/jffs2dump.c 2004-06-19 00:11:48.000000000 +0200 +++ mtd/util/jffs2dump.c 2004-10-20 14:55:28.000000000 +0200 @@ -3,7 +3,7 @@ * * Copyright (C) 2003 Thomas Gleixner (tglx@linutronix.de) * - * $Id: jffs2dump.c,v 1.6 2004/06/18 22:11:48 gleixner Exp $ + * $Id: jffs2dump.c,v 1.1 2004/10/19 07:19:55 weth Exp $ * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License version 2 as @@ -277,7 +277,63 @@ p += PAD(je32_to_cpu (node->d.totlen)); break; - + + case JFFS2_NODETYPE_INODE_SUM:{ + + int i; + jint32_t *offset,*magic; + + printf ("%8s Inode Sum node at 0x%08x, totlen 0x%08x, sum_num %5d, cleanmarker size %5d\n", + obsolete ? "Obsolete" : "", + p - data, + je32_to_cpu (node->s.totlen), + je16_to_cpu (node->s.sum_num), + je32_to_cpu (node->s.cln_mkr)); + + crc = crc32 (0, node, sizeof (struct jffs2_inode_sum_node) - 8); + if (crc != je32_to_cpu (node->s.node_crc)) { + printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.node_crc), crc); + p += PAD(je32_to_cpu (node->s.totlen)); + dirty += PAD(je32_to_cpu (node->s.totlen));; + continue; + } + + crc = crc32(0, p + sizeof (struct jffs2_inode_sum_node), je32_to_cpu (node->s.totlen) - sizeof(struct jffs2_inode_sum_node)); + if (crc != je32_to_cpu(node->s.sum_crc)) { + printf ("Wrong data_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.sum_crc), crc); + p += PAD(je32_to_cpu (node->s.totlen)); + dirty += PAD(je32_to_cpu (node->s.totlen));; + continue; + } + + if(verbose){ + for(i = 0; i < je16_to_cpu (node->s.sum_num); i++){ + struct jffs2_inode_sum_record *sp; + sp = (struct jffs2_inode_sum_record *) (p + sizeof (struct jffs2_inode_sum_node)); + + printf ("%14s #ino %5d, version %5d, offset %8d, totlen 0x%08x\n", + "", + je32_to_cpu (sp[i].inode), + je32_to_cpu (sp[i].version), + je32_to_cpu (sp[i].offset), + je32_to_cpu (sp[i].totlen)); + + } + + offset = (jint32_t *)((char *)p + je32_to_cpu(node->s.totlen) - 8); + magic = (jint32_t *)((char *)p + je32_to_cpu(node->s.totlen) - 4); + + printf("%14s Sum Node Offset 0x%08x, Magic 0x%08x\n", + "", + je32_to_cpu(*offset), + je32_to_cpu(*magic)); + } + + p += PAD(je32_to_cpu (node->s.totlen)); + break; + + } + case JFFS2_NODETYPE_CLEANMARKER: if (verbose) { printf ("%8s Cleanmarker at 0x%08x, totlen 0x%08x\n", @@ -418,9 +474,9 @@ write (fd, &newnode, sizeof (struct jffs2_raw_dirent)); write (fd, p + sizeof (struct jffs2_raw_dirent), PAD (je32_to_cpu (node->d.totlen) - sizeof (struct jffs2_raw_dirent))); - p += PAD(je32_to_cpu (node->d.totlen)); + p += PAD(je32_to_cpu (node->d.totlen)); break; - + case JFFS2_NODETYPE_CLEANMARKER: case JFFS2_NODETYPE_PADDING: newnode.u.magic = cnv_e16 (node->u.magic); diff --unified --recursive --new-file mtd2/util/sumtool.c mtd/util/sumtool.c --- mtd2/util/sumtool.c 1970-01-01 01:00:00.000000000 +0100 +++ mtd/util/sumtool.c 2004-10-20 11:56:08.000000000 +0200 @@ -0,0 +1,800 @@ +/* + * sumtool.c + * + * Copyright (C) 2004 Zoltan Sogor <weth@inf.u-szeged.hu>, + * Ferenc Havasi <havasi@inf.u-szeged.hu>, + * Patrik Kluba <pajko@halom.u-szeged.hu>, + * University of Szeged, Hungary + * + * $Id: sumtool.c,v 1.2 2004/10/20 09:56:08 hafy Exp $ + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version 2 + * of the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + * Overview: + * This is a utility to reorder nodes and insert inode summary information + * into JFFS2 image for faster mount time - specially on NAND. + * + */ + +#include <errno.h> +#include <stdint.h> +#include <stdio.h> +#include <stdlib.h> +#include <stdarg.h> +#include <string.h> +#include <unistd.h> +#include <fcntl.h> +#include <time.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <sys/param.h> +#include <asm/types.h> +#include <dirent.h> +#include <mtd/jffs2-user.h> +#include <endian.h> +#include <byteswap.h> +#include <getopt.h> +#include "crc32.h" + +#define PAD(x) (((x)+3)&~3) + +#define SINODE 0 /* Inode Type*/ +#define SONODE 1 /* Other type*/ + +static const char *const app_name = "sumtool"; + +typedef struct sum_storage { + jint32_t inode; + jint32_t version; + jint32_t offset; + jint32_t totlen; + struct sum_storage *next; +} sum_storage; + +static sum_storage *sum_collected = NULL; /* summary info list */ +static int sum_records = 0; /* number of sumary records */ + + +static int verbose = 0; +static int add_cleanmarkers = 1; /* add cleanmarker to output */ +static int use_input_cleanmarker_size = 1; /* use input file's cleanmarker size (default) */ +static int found_cleanmarkers = 0; /* cleanmarker found in input file */ +static struct jffs2_unknown_node cleanmarker; +static int cleanmarker_size = sizeof(cleanmarker); +static const char *short_options = "o:i:e:hvVblnc:"; +static int erase_block_size = 65536; +static int target_endian = __BYTE_ORDER; +static int out_fd = -1; +static int in_fd = -1; + +static uint8_t *inode_buffer = NULL; /* buffer for inodes */ +static unsigned int ino_ofs = 0; /* inode buffer offset */ + +static uint8_t *dirent_buffer = NULL; /* buffer for directory entries and other (symlink, spec. device files, etc.)*/ +static unsigned int dent_ofs = 0; /* directory enrty buffer offset*/ + +static uint8_t *file_buffer = NULL; /* file buffer contains the actual erase block*/ +static unsigned int file_ofs = 0; /* position in the buffer */ + +static struct option long_options[] = { + {"output", 1, NULL, 'o'}, + {"input", 1, NULL, 'i'}, + {"eraseblock", 1, NULL, 'e'}, + {"help", 0, NULL, 'h'}, + {"verbose", 0, NULL, 'v'}, + {"version", 0, NULL, 'V'}, + {"bigendian", 0, NULL, 'b'}, + {"littleendian", 0, NULL, 'l'}, + {"no-cleanmarkers", 0, NULL, 'n'}, + {"cleanmarker", 1, NULL, 'c'}, + {NULL, 0, NULL, 0} +}; + +static char *helptext = + "Usage: sumtool [OPTIONS] -i inputfile -o outputfile\n" + "Convert the input JFFS2 file to a SUM-ed JFFS2 file\n\n" + "Options:\n" + " -e, --eraseblock=SIZE Use erase block size SIZE (default: 64KiB)\n" + " (usually 16KiB on NAND)\n" + " -c, --cleanmarker=SIZE Size of cleanmarker (default 12).\n" + " (usually 16 bytes on NAND, and will be set to\n" + " this value if left at the default 12). Will be\n" + " stored in OOB after each physical page composing\n" + " a physical erase block.\n" + " -n, --no-cleanmarkers Don't add a cleanmarker to every eraseblock\n" + " -o, --output=FILE Output to FILE \n" + " -i, --input=FILE Input from FILE \n" + " -b, --bigendian Image is big endian\n" + " -l --littleendian Image is little endian\n" + " -h, --help Display this help text\n" + " -v, --verbose Verbose operation\n" + " -V, --version Display version information\n\n"; + + +static char *revtext = "$Revision: 1.2 $"; + +static unsigned char ffbuf[16] = { + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff +}; + +static void verror_msg(const char *s, va_list p) { + fflush(stdout); + fprintf(stderr, "%s: ", app_name); + vfprintf(stderr, s, p); +} + +static void error_msg_and_die(const char *s, ...) { + va_list p; + + va_start(p, s); + verror_msg(s, p); + va_end(p); + putc('\n', stderr); + exit(EXIT_FAILURE); +} + +static void vperror_msg(const char *s, va_list p) { + int err = errno; + + if (s == 0) + s = ""; + verror_msg(s, p); + if (*s) + s = ": "; + fprintf(stderr, "%s%s\n", s, strerror(err)); +} + +static void perror_msg_and_die(const char *s, ...) { + va_list p; + + va_start(p, s); + vperror_msg(s, p); + va_end(p); + exit(EXIT_FAILURE); +} + + + +static void full_write(void *target_buff, const void *buf, int len, int nd); + +void setup_cleanmarker() { + + cleanmarker.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK); + cleanmarker.nodetype = cpu_to_je16(JFFS2_NODETYPE_CLEANMARKER); + cleanmarker.totlen = cpu_to_je32(cleanmarker_size); + cleanmarker.hdr_crc = cpu_to_je32(crc32(0, &cleanmarker, sizeof(struct jffs2_unknown_node)-4)); +} + +void process_options (int argc, char **argv){ + int opt,c; + + while ((opt = getopt_long(argc, argv, short_options, long_options, &c)) >= 0) + { + switch (opt) + { + case 'o': + if (out_fd != -1) { + error_msg_and_die("output filename specified more than once"); + } + out_fd = open(optarg, O_CREAT | O_TRUNC | O_RDWR, 0644); + if (out_fd == -1) { + perror_msg_and_die("open output file"); + } + break; + + case 'i': + if (in_fd != -1) { + error_msg_and_die("input filename specified more than once"); + } + in_fd = open(optarg, O_RDONLY); + if (in_fd == -1) { + perror_msg_and_die("open input file"); + } + break; + case 'b': + target_endian = __BIG_ENDIAN; + break; + case 'l': + target_endian = __LITTLE_ENDIAN; + break; + case 'h': + case '?': + error_msg_and_die(helptext); + + case 'v': + verbose = 1; + break; + + case 'V': + error_msg_and_die("revision %.*s\n", + (int) strlen(revtext) - 13, revtext + 11); + + case 'e': { + char *next; + unsigned units = 0; + erase_block_size = strtol(optarg, &next, 0); + if (!erase_block_size) + error_msg_and_die("Unrecognisable erase size\n"); + + if (*next) { + if (!strcmp(next, "KiB")) { + units = 1024; + } else if (!strcmp(next, "MiB")) { + units = 1024 * 1024; + } else { + error_msg_and_die("Unknown units in erasesize\n"); + } + } else { + if (erase_block_size < 0x1000) + units = 1024; + else + units = 1; + } + erase_block_size *= units; + + /* If it's less than 8KiB, they're not allowed */ + if (erase_block_size < 0x2000) { + fprintf(stderr, "Erase size 0x%x too small. Increasing to 8KiB minimum\n", + erase_block_size); + erase_block_size = 0x2000; + } + break; + } + + case 'n': + add_cleanmarkers = 0; + break; + case 'c': + cleanmarker_size = strtol(optarg, NULL, 0); + + if (cleanmarker_size < sizeof(cleanmarker)) { + error_msg_and_die("cleanmarker size must be >= 12"); + } + if (cleanmarker_size >= erase_block_size) { + error_msg_and_die("cleanmarker size must be < eraseblock size"); + } + + use_input_cleanmarker_size = 0; + found_cleanmarkers = 1; + setup_cleanmarker(); + + break; + + } + } +} + + +void init_buffers() { + + inode_buffer = malloc(erase_block_size); + + if (!inode_buffer) { + perror("out of memory"); + close (in_fd); + close (out_fd); + exit(1); + } + + dirent_buffer = malloc(erase_block_size); + + if (!dirent_buffer) { + perror("out of memory"); + close (in_fd); + close (out_fd); + exit(1); + } + + file_buffer = malloc(erase_block_size); + + if (!file_buffer) { + perror("out of memory"); + close (in_fd); + close (out_fd); + exit(1); + } +} + +void clean_buffers() { + + if (inode_buffer) + free(inode_buffer); + if (dirent_buffer) + free(dirent_buffer); + if (file_buffer) + free(file_buffer); +} + +int load_next_block() { + + int ret; + ret = read(in_fd, file_buffer, erase_block_size); + file_ofs = 0; + + if(verbose) + printf("Load next block : %d bytes read\n",ret); + + return ret; +} + +void write_buff_to_file(int nd) { + + int ret; + int len = erase_block_size; + uint8_t *buf = NULL; + + if (!nd) { + buf = inode_buffer; + while (len > 0) { + ret = write(out_fd, buf, len); + + if (ret < 0) + perror_msg_and_die("write"); + + if (ret == 0) + perror_msg_and_die("write returned zero"); + + len -= ret; + buf += ret; + } + ino_ofs = 0; + } + else { + buf = dirent_buffer; + while (len > 0) { + ret = write(out_fd, buf, len); + + if (ret < 0) + perror_msg_and_die("write"); + + if (ret == 0) + perror_msg_and_die("write returned zero"); + + len -= ret; + buf += ret; + } + dent_ofs = 0; + } +} + +void dump_sum_records() { + + struct jffs2_inode_sum_node isum; + struct sum_storage *temp; + jint32_t offset; + jint32_t *wpage; + int datasize; + int infosize; + int padsize; + jint32_t magic = cpu_to_je32(JFFS2_SUM_MAGIC); + + if (!sum_records) + return; + + datasize = sum_records * sizeof(struct jffs2_inode_sum_record) + 8; + infosize = sizeof(struct jffs2_inode_sum_node) + datasize; + padsize = erase_block_size - ino_ofs - infosize; + infosize += padsize; datasize += padsize; + offset = cpu_to_je32(ino_ofs); + jint32_t *tpage = (jint32_t *) malloc(datasize); + + if(!tpage) + error_msg_and_die("Can't allocate memory to dump summary information!\n"); + + memset(tpage, 0xff, datasize); + memset(&isum, 0, sizeof(isum)); + + isum.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK); + isum.nodetype = cpu_to_je16(JFFS2_NODETYPE_INODE_SUM); + isum.totlen = cpu_to_je32(infosize); + isum.hdr_crc = cpu_to_je32(crc32(0, &isum, sizeof(struct jffs2_unknown_node) - 4)); + + if (add_cleanmarkers && found_cleanmarkers) { + isum.cln_mkr = cpu_to_je32(cleanmarker_size); + } + else{ + isum.cln_mkr = cpu_to_je32(0); + } + + isum.sum_num = cpu_to_je16(sum_records); + wpage = tpage; + + while (sum_records) { + *(wpage++) = sum_collected->inode; + *(wpage++) = sum_collected->version; + *(wpage++) = sum_collected->offset; + *(wpage++) = sum_collected->totlen; + temp = sum_collected; + sum_collected = sum_collected->next; + free(temp); + sum_records--; + } + + ((char *)wpage) += padsize; + *(wpage++) = offset; + *(wpage++) = magic; + isum.sum_crc = cpu_to_je32(crc32(0, tpage, datasize)); + isum.node_crc = cpu_to_je32(crc32(0, &isum, sizeof(isum) - 8)); + + full_write(inode_buffer + ino_ofs, &isum, sizeof(isum), SINODE); + full_write(inode_buffer + ino_ofs, tpage, datasize, SINODE); + + free(tpage); +} + +static void full_write(void *target_buff, const void *buf, int len, int nd) { + memcpy(target_buff, buf, len); + + if (!nd) + ino_ofs += len; + else + dent_ofs += len; +} + +static void pad(int req, int nd) { + if (!nd) { + + while (req) { + if (req > sizeof(ffbuf)) { + full_write(inode_buffer + ino_ofs, ffbuf, sizeof(ffbuf), nd); + req -= sizeof(ffbuf); + } else { + full_write(inode_buffer + ino_ofs, ffbuf, req, nd); + req = 0; + } + } + } + else { + while (req) { + if (req > sizeof(ffbuf)) { + full_write(dirent_buffer + dent_ofs, ffbuf, sizeof(ffbuf), nd); + req -= sizeof(ffbuf); + } + else { + full_write(dirent_buffer + dent_ofs, ffbuf, req, nd); + req = 0; + } + } + } +} + +static inline void padword(int nd) { + + if (!nd){ + if (ino_ofs % 4) { + full_write(inode_buffer + ino_ofs, ffbuf, 4 - (ino_ofs % 4), nd); + } + } + else { + if (dent_ofs % 4) { + full_write(dirent_buffer + dent_ofs, ffbuf, 4 - (dent_ofs % 4), nd); + } + } +} + +static inline void pad_block_if_less_than(int req, int nd) { + if (!nd) { + int datasize = ((sum_records + 1) * sizeof(struct jffs2_inode_sum_record)) + sizeof(struct jffs2_inode_sum_node) + 8; + datasize += (4 - (datasize % 4)) % 4; + if (ino_ofs + req > erase_block_size - datasize) { + dump_sum_records(); + write_buff_to_file(nd); + } + + if (add_cleanmarkers && found_cleanmarkers) { + if (!ino_ofs) { + full_write(inode_buffer, &cleanmarker, sizeof(cleanmarker), nd); + pad(cleanmarker_size - sizeof(cleanmarker), nd); + padword(nd); + } + } + + } + else { + if (dent_ofs + req > erase_block_size) { + pad(erase_block_size - dent_ofs, nd); + write_buff_to_file(nd); + } + + if (add_cleanmarkers && found_cleanmarkers) { + if (!dent_ofs) { + full_write(dirent_buffer, &cleanmarker, sizeof(cleanmarker), nd); + pad(cleanmarker_size - sizeof(cleanmarker), nd); + padword(nd); + } + } + } +} + +void flush_buffers() { + + if ((add_cleanmarkers == 1) && (found_cleanmarkers == 1)) { /* CLEANMARKER */ + if (ino_ofs != cleanmarker_size) { /* INODE BUFFER */ + + int datasize = ((sum_records + 1) * sizeof(struct jffs2_inode_sum_record)) + sizeof(struct jffs2_inode_sum_node) + 8; + datasize += (4 - (datasize % 4)) % 4; + + /* If we have a full inode buffer, then write out inode and summary data */ + if (ino_ofs + sizeof(struct jffs2_raw_inode) + JFFS2_MIN_DATA_LEN > erase_block_size - datasize) { + dump_sum_records(); + write_buff_to_file(SINODE); + } + /* else just write out inode data */ + else{ + pad(erase_block_size - ino_ofs, SINODE); + write_buff_to_file(SINODE); + } + } + + if (dent_ofs != cleanmarker_size) { /* DIRENT AND OTHERS BUFFER */ + pad(erase_block_size - dent_ofs, SONODE); + write_buff_to_file(SONODE); + } + + } + else { /* NO CLEANMARKER */ + if (ino_ofs != 0) { /* INODE BUFFER */ + + int datasize = ((sum_records + 1) * sizeof(struct jffs2_inode_sum_record)) + sizeof(struct jffs2_inode_sum_node) + 8; + datasize += (4 - (datasize % 4)) % 4; + + /* If we have a full inode buffer, then write out inode and summary data */ + if (ino_ofs + sizeof(struct jffs2_raw_inode) + JFFS2_MIN_DATA_LEN > erase_block_size - datasize) { + dump_sum_records(); + write_buff_to_file(SINODE); + } + /* Else just write out inode data */ + else{ + pad(erase_block_size - ino_ofs, SINODE); + write_buff_to_file(SINODE); + } + } + + if (dent_ofs != 0) { /* DIRENT AND OTHER BUFFER */ + pad(erase_block_size - dent_ofs, SONODE); + write_buff_to_file(SONODE); + } + } +} + + +void write_dirent_to_buff(union jffs2_node_union *node) { + + pad_block_if_less_than(je32_to_cpu (node->d.totlen), SONODE); + full_write(dirent_buffer + dent_ofs, &(node->d), je32_to_cpu (node->d.totlen), SONODE); + padword(SONODE); +} + +void add_sum_entry(union jffs2_node_union *node) { + + sum_storage *walk; + sum_storage *temp = (sum_storage *) malloc(sizeof(sum_storage)); + + if (!temp) + error_msg_and_die("Can't allocate memory for summary information!\n"); + + temp->inode = node->i.ino; + temp->version = node->i.version; + temp->offset = cpu_to_je32(ino_ofs); + temp->totlen = node->i.totlen; + temp->next = NULL; + + if (!sum_collected) { + sum_collected = temp; + } + else { + walk = sum_collected; + + while (walk->next) { + walk = walk->next; + } + walk->next = temp; + } + sum_records++; +} + +void write_inode_to_buff(union jffs2_node_union *node) { + + pad_block_if_less_than(je32_to_cpu (node->i.totlen), SINODE); + add_sum_entry(node); /* Add inode summary entry to summary list */ + full_write(inode_buffer + ino_ofs, &(node->i), je32_to_cpu (node->i.totlen), SINODE); /* Write out the inode to inode_buffer */ + padword(SINODE); + +} + + +void create_summed_image(int inp_size) { + uint8_t *p = file_buffer; + union jffs2_node_union *node; + uint32_t crc; + uint16_t type; + int bitchbitmask = 0; + int obsolete; + + char name[256]; + + while ( p < (file_buffer + inp_size)) { + + node = (union jffs2_node_union*) p; + + /* Skip empty space */ + if (je16_to_cpu (node->u.magic) == 0xFFFF && je16_to_cpu (node->u.nodetype) == 0xFFFF) { + p += 4; + continue; + } + + if (je16_to_cpu (node->u.magic) != JFFS2_MAGIC_BITMASK) { + if (!bitchbitmask++) + printf ("Wrong bitmask at 0x%08x, 0x%04x\n", p - file_buffer, je16_to_cpu (node->u.magic)); + p += 4; + continue; + } + + bitchbitmask = 0; + + type = je16_to_cpu(node->u.nodetype); + if ((type & JFFS2_NODE_ACCURATE) != JFFS2_NODE_ACCURATE) { + obsolete = 1; + type |= JFFS2_NODE_ACCURATE; + } else + obsolete = 0; + + node->u.nodetype = cpu_to_je16(type); + + crc = crc32 (0, node, sizeof (struct jffs2_unknown_node) - 4); + if (crc != je32_to_cpu (node->u.hdr_crc)) { + printf ("Wrong hdr_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->u.hdr_crc), crc); + p += 4; + continue; + } + + switch(je16_to_cpu(node->u.nodetype)) { + + case JFFS2_NODETYPE_INODE: + if(verbose) + printf ("%8s Inode node at 0x%08x, totlen 0x%08x, #ino %5d, version %5d, isize %8d, csize %8d, dsize %8d, offset %8d\n", + obsolete ? "Obsolete" : "", + p - file_buffer, je32_to_cpu (node->i.totlen), je32_to_cpu (node->i.ino), + je32_to_cpu ( node->i.version), je32_to_cpu (node->i.isize), + je32_to_cpu (node->i.csize), je32_to_cpu (node->i.dsize), je32_to_cpu (node->i.offset)); + + crc = crc32 (0, node, sizeof (struct jffs2_raw_inode) - 8); + if (crc != je32_to_cpu (node->i.node_crc)) { + printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.node_crc), crc); + p += PAD(je32_to_cpu (node->i.totlen)); + continue; + } + + crc = crc32(0, p + sizeof (struct jffs2_raw_inode), je32_to_cpu(node->i.csize)); + if (crc != je32_to_cpu(node->i.data_crc)) { + printf ("Wrong data_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.data_crc), crc); + p += PAD(je32_to_cpu (node->i.totlen)); + continue; + } + + write_inode_to_buff(node); + + p += PAD(je32_to_cpu (node->i.totlen)); + break; + + case JFFS2_NODETYPE_DIRENT: + memcpy (name, node->d.name, node->d.nsize); + name [node->d.nsize] = 0x0; + + if(verbose) + printf ("%8s Dirent node at 0x%08x, totlen 0x%08x, #pino %5d, version %5d, #ino %8d, nsize %8d, name %s\n", + obsolete ? "Obsolete" : "", + p - file_buffer, je32_to_cpu (node->d.totlen), je32_to_cpu (node->d.pino), + je32_to_cpu ( node->d.version), je32_to_cpu (node->d.ino), + node->d.nsize, name); + + crc = crc32 (0, node, sizeof (struct jffs2_raw_dirent) - 8); + if (crc != je32_to_cpu (node->d.node_crc)) { + printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.node_crc), crc); + p += PAD(je32_to_cpu (node->d.totlen)); + continue; + } + + crc = crc32(0, p + sizeof (struct jffs2_raw_dirent), node->d.nsize); + if (crc != je32_to_cpu(node->d.name_crc)) { + printf ("Wrong name_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.name_crc), crc); + p += PAD(je32_to_cpu (node->d.totlen)); + continue; + } + + write_dirent_to_buff(node); + + p += PAD(je32_to_cpu (node->d.totlen)); + break; + + case JFFS2_NODETYPE_CLEANMARKER: + if (verbose) { + printf ("%8s Cleanmarker at 0x%08x, totlen 0x%08x\n", + obsolete ? "Obsolete" : "", + p - file_buffer, je32_to_cpu (node->u.totlen)); + } + + if(!found_cleanmarkers){ + found_cleanmarkers = 1; + + if(add_cleanmarkers == 1 && use_input_cleanmarker_size == 1){ + cleanmarker_size = je32_to_cpu (node->u.totlen); + setup_cleanmarker(); + } + } + + p += PAD(je32_to_cpu (node->u.totlen)); + break; + + case JFFS2_NODETYPE_PADDING: + if (verbose) { + printf ("%8s Padding node at 0x%08x, totlen 0x%08x\n", + obsolete ? "Obsolete" : "", + p - file_buffer, je32_to_cpu (node->u.totlen)); + } + p += PAD(je32_to_cpu (node->u.totlen)); + break; + + case 0xffff: + p += 4; + break; + + default: + if (verbose) { + printf ("%8s Unknown node at 0x%08x, totlen 0x%08x\n", + obsolete ? "Obsolete" : "", + p - file_buffer, je32_to_cpu (node->u.totlen)); + } + + write_dirent_to_buff(node); + + p += PAD(je32_to_cpu (node->u.totlen)); + } + } +} + +int main(int argc, char **argv) { + + int ret; + + process_options(argc,argv); + + if ((in_fd == -1) || (out_fd == -1)) { + + if(in_fd != -1) + close(in_fd); + if(out_fd != -1) + close(out_fd); + + error_msg_and_die("You must specify input and output files!\n"); + } + + init_buffers(); + + while ((ret = load_next_block())) { + create_summed_image(ret); + } + + flush_buffers(); + clean_buffers(); + + if (in_fd != -1) + close(in_fd); + if (out_fd != -1) + close(out_fd); + + return 0; +} ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: [OBORONA-SPAM] JFFS2 mount time 2004-10-20 14:26 Ferenc Havasi @ 2004-10-20 15:26 ` Artem B. Bityuckiy 2004-10-20 15:49 ` Ferenc Havasi 2004-10-21 6:29 ` Artem B. Bityuckiy ` (2 subsequent siblings) 3 siblings, 1 reply; 82+ messages in thread From: Artem B. Bityuckiy @ 2004-10-20 15:26 UTC (permalink / raw) To: Ferenc Havasi; +Cc: dwmw2, linux-mtd Ferenc, I didn't investigate your patch well yet, but after 3 minuets of looking to it I have two questions: 1. Why did you introduce the JFFS2_SUM_MAGIC constant? As I understand, the node's magic field is needed to identify the *beginning of node*, *not the node type*. The type of node is defined by the next field, called 'nodetype'. You use it (JFFS2_NODETYPE_INODE_SUM). So, IMHO, the JFFS2_SUM_MAGIC constant doesn't fit into the common rules... 2. This is very minor of course, just a remark. IMHO, its better to avoid too many ifdefs, so, I think it is unnecessary to place the function prototype under ifdef. I mead: +#ifdef CONFIG_JFFS2_FS_SUMMARY +static struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino); +#endif Ferenc Havasi wrote: > Dear All, > > Here is the latest version of our mount time improvement. > > Using of it: > - apply this patch on the latest version of MTD > - compile sumtool (make command in mtd/util) > - make your JFFS2 image as before (or you can use already created images > as well) > - run sumtool to insert summary information, for example: > ./sumtool -i original.jffs2 -o new.jffs2 -e128KiB > - recompile your kernel with "JFFS2 inode summary support" > > Jarkko made a measurement on a real NAND device: his JFFS2 image was > 120819928 (115M), after running sumtool the new image was 123338752 (117M). > > Using the original mount time was 55 sec, with the new image it is only > 8.5 sec. > > It works very similar as our previous improvement: stores special > information at the end of the erase blocks, and at mount time if there > is this kind of information the scaning of the erase block is unneccessary. > > New things compared to our previous improvement: > - it was fully rewritten > - we separated the user space tool from mkfs. (sumtool) > - sumtool now not only inserts the summary information but also make > some node-reordering. There will be two kind of erase blocks: in the > "first type" there will be only jffs2_raw_inodes, and all other node > (jffs2_raw_dirent) will be stored in the "second type". It generates > summary at the end of all "fist type" eraseblock. (the "second type" > will be scanned as before, because all information is needed in > jffs_raw_dirent at mount time) > > Ceratinly all of these things are optional (as you can see above you > have to select it from kernel config). The JFFS2 image produced by > sumtool is also usable with previous kernel because the summary node is > JFFS2_FEATURE_RWCOMPAT_DELETE. > > I think it can be usefull not only for us. David, may I commit it to the > CVS? > > Regards, > Ferenc -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-20 15:26 ` [OBORONA-SPAM] " Artem B. Bityuckiy @ 2004-10-20 15:49 ` Ferenc Havasi 2004-10-20 15:53 ` Artem B. Bityuckiy 0 siblings, 1 reply; 82+ messages in thread From: Ferenc Havasi @ 2004-10-20 15:49 UTC (permalink / raw) To: Artem B. Bityuckiy; +Cc: dwmw2, linux-mtd Hi Artem, > 1. Why did you introduce the JFFS2_SUM_MAGIC constant? As I understand, > the node's magic field is needed to identify the *beginning of node*, > *not the node type*. The type of node is defined by the next field, > called 'nodetype'. You use it (JFFS2_NODETYPE_INODE_SUM). So, IMHO, the > JFFS2_SUM_MAGIC constant doesn't fit into the common rules... The reason is the following: the summary node is at the end of the erase block, and it has not fixed size (its size depends on the information it stores). The main advantage of using summary node is to avoid the original scanning method. So we cannot use the original full-scanning method to determine the begining of the summary node (using only JFFS2_NODETYPE_INODE_SUM). Our method is the following: - read some bytes at the end of the erase block - if the last word is JFFS2_SUM_MAGIC than we will almost sure that it is an erase block which has summary - the word before this magic is the length of the node - using this length we can check that it is really a JFFS2_NODETYPE_INODE_SUM node, and process it I can't image more effective method to determine the begining of the summary node. (if you have better suggestion...) And because the magic is inside of the summary node I think it is fit to the philosophy of JFFS2 - but a little bit tricky. > 2. This is very minor of course, just a remark. IMHO, its better to > avoid too many ifdefs, so, I think it is unnecessary to place the > function prototype under ifdef. I mead: > > +#ifdef CONFIG_JFFS2_FS_SUMMARY > +static struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct > jffs2_sb_info *c, uint32_t ino); > +#endif Yes, I aggree. I will modify it. Bye, Ferenc ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-20 15:49 ` Ferenc Havasi @ 2004-10-20 15:53 ` Artem B. Bityuckiy 0 siblings, 0 replies; 82+ messages in thread From: Artem B. Bityuckiy @ 2004-10-20 15:53 UTC (permalink / raw) To: Ferenc Havasi; +Cc: linux-mtd Ferenc Havasi wrote: > The reason is the following: the summary node is at the end of the erase > block, and it has not fixed size (its size depends on the information it > stores). > > The main advantage of using summary node is to avoid the original > scanning method. So we cannot use the original full-scanning method to > determine the begining of the summary node (using only > JFFS2_NODETYPE_INODE_SUM). > > Our method is the following: > - read some bytes at the end of the erase block > - if the last word is JFFS2_SUM_MAGIC than we will almost sure that it > is an erase block which has summary > - the word before this magic is the length of the node > - using this length we can check that it is really a > JFFS2_NODETYPE_INODE_SUM node, and process it > > I can't image more effective method to determine the begining of the > summary node. (if you have better suggestion...) And because the magic > is inside of the summary node I think it is fit to the philosophy of > JFFS2 - but a little bit tricky. Ok, I got it. I was wrong, sorry. -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-20 14:26 Ferenc Havasi 2004-10-20 15:26 ` [OBORONA-SPAM] " Artem B. Bityuckiy @ 2004-10-21 6:29 ` Artem B. Bityuckiy 2004-10-21 6:54 ` Ferenc Havasi 2004-10-21 12:49 ` Jarkko Lavinen 2004-10-21 13:24 ` David Woodhouse 3 siblings, 1 reply; 82+ messages in thread From: Artem B. Bityuckiy @ 2004-10-21 6:29 UTC (permalink / raw) To: Ferenc Havasi; +Cc: dwmw2, linux-mtd, jffs-dev Hello Ferenc, As I understand, you only prepare JFFS2 image with summaries. This is great until we do not change anything. For read-only file-systems this is OK. But what if files/direntries are changed/deleted ? Do you write summary information dynamically? How are you going to place nodes/direntries to different blocks dynamically? Ferenc Havasi wrote: > Dear All, > > Here is the latest version of our mount time improvement. > > Using of it: > - apply this patch on the latest version of MTD > - compile sumtool (make command in mtd/util) > - make your JFFS2 image as before (or you can use already created images > as well) > - run sumtool to insert summary information, for example: > ./sumtool -i original.jffs2 -o new.jffs2 -e128KiB > - recompile your kernel with "JFFS2 inode summary support" > > Jarkko made a measurement on a real NAND device: his JFFS2 image was > 120819928 (115M), after running sumtool the new image was 123338752 (117M). > > Using the original mount time was 55 sec, with the new image it is only > 8.5 sec. > > It works very similar as our previous improvement: stores special > information at the end of the erase blocks, and at mount time if there > is this kind of information the scaning of the erase block is unneccessary. > > New things compared to our previous improvement: > - it was fully rewritten > - we separated the user space tool from mkfs. (sumtool) > - sumtool now not only inserts the summary information but also make > some node-reordering. There will be two kind of erase blocks: in the > "first type" there will be only jffs2_raw_inodes, and all other node > (jffs2_raw_dirent) will be stored in the "second type". It generates > summary at the end of all "fist type" eraseblock. (the "second type" > will be scanned as before, because all information is needed in > jffs_raw_dirent at mount time) > > Ceratinly all of these things are optional (as you can see above you > have to select it from kernel config). The JFFS2 image produced by > sumtool is also usable with previous kernel because the summary node is > JFFS2_FEATURE_RWCOMPAT_DELETE. > > I think it can be usefull not only for us. David, may I commit it to the > CVS? > > Regards, > Ferenc -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-21 6:29 ` Artem B. Bityuckiy @ 2004-10-21 6:54 ` Ferenc Havasi 2004-10-21 7:16 ` Artem B. Bityuckiy 0 siblings, 1 reply; 82+ messages in thread From: Ferenc Havasi @ 2004-10-21 6:54 UTC (permalink / raw) To: Artem B. Bityuckiy; +Cc: dwmw2, linux-mtd, jffs-dev Hi Artem, > As I understand, you only prepare JFFS2 image with summaries. This is > great until we do not change anything. For read-only file-systems this > is OK. > > But what if files/direntries are changed/deleted ? Do you write summary > information dynamically? How are you going to place nodes/direntries to > different blocks dynamically? You are right, there is a small change which is really important (and will be ready very soon) to extend jffs2_mark_node_obsolete() to mark not only the node but also its entry in the summary. Any other improvement can be done later, because after it the filesystem will be always coherent, because we write summary only at the of the erasy blocks, when it is fully "finished" - so if there is a summary somewhere we will not need to extend it, only to mark the obscolated nodes. We also plan in the near future to implement the ability of generating summary dinamically when the filesystem finishes an erase block - which keep this "fast mount time" permament. Bye, Ferenc ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-21 6:54 ` Ferenc Havasi @ 2004-10-21 7:16 ` Artem B. Bityuckiy 2004-10-21 19:50 ` Ferenc Havasi 0 siblings, 1 reply; 82+ messages in thread From: Artem B. Bityuckiy @ 2004-10-21 7:16 UTC (permalink / raw) To: Ferenc Havasi; +Cc: dwmw2, linux-mtd, jffs-dev > You are right, there is a small change which is really important (and > will be ready very soon) to extend jffs2_mark_node_obsolete() to mark > not only the node but also its entry in the summary. Unfortunately, you can not mark entries as obsoleted in your summary node in case of NAND. If you write your summary only for *full* blocks, you will not need to mark entries obsoleted, even if you have NOR flash (but you can on NOR). The partially filled blocks must not have the summary node (you can introduce special marker and write it to OOB of the last page of NAND/last word of sector on NOR which tells if there is the summary node present). So, fully filled block will have summary and will be scanned very quickly, partially filled ones will have no summary and will be fully scanned, free blocks will have cleanmarkers and will not be scanned, other blocks will be either erased or considered free. > > Any other improvement can be done later, because after it the filesystem > will be always coherent, because we write summary only at the of the > erasy blocks, when it is fully "finished" - so if there is a summary > somewhere we will not need to extend it, only to mark the obscolated nodes. Yes, nice, but why do you need to mark obsoleted nodes in summary ??? When you insert node to the fragtree or dirents to the list, JFFS2 code will detect obsoleted nodes automatically, no need to mark them physically. > > We also plan in the near future to implement the ability of generating > summary dinamically when the filesystem finishes an erase block - which > keep this "fast mount time" permament. This would be perfect. -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-21 7:16 ` Artem B. Bityuckiy @ 2004-10-21 19:50 ` Ferenc Havasi 0 siblings, 0 replies; 82+ messages in thread From: Ferenc Havasi @ 2004-10-21 19:50 UTC (permalink / raw) To: Artem B. Bityuckiy; +Cc: dwmw2, linux-mtd, jffs-dev Hi Artem, > Unfortunately, you can not mark entries as obsoleted in your summary > node in case of NAND. > > If you write your summary only for *full* blocks, you will not need to > mark entries obsoleted, even if you have NOR flash (but you can on NOR). > The partially filled blocks must not have the summary node (you can > introduce special marker and write it to OOB of the last page of > NAND/last word of sector on NOR which tells if there is the summary node > present). Really, you are right. So we only have to solve this problem on NOR. I think the easiest solution is to set jffs2_can_mark_obsolete() to false if the summary support is enabled. Bye, Ferenc ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-20 14:26 Ferenc Havasi 2004-10-20 15:26 ` [OBORONA-SPAM] " Artem B. Bityuckiy 2004-10-21 6:29 ` Artem B. Bityuckiy @ 2004-10-21 12:49 ` Jarkko Lavinen 2004-10-21 19:11 ` Ferenc Havasi 2004-10-22 9:58 ` Ferenc Havasi 2004-10-21 13:24 ` David Woodhouse 3 siblings, 2 replies; 82+ messages in thread From: Jarkko Lavinen @ 2004-10-21 12:49 UTC (permalink / raw) Cc: dwmw2, linux-mtd On Wed, Oct 20, 2004 at 04:26:27PM +0200, ext Ferenc Havasi wrote: > Jarkko made a measurement on a real NAND device: his JFFS2 image was > 120819928 (115M), after running sumtool the new image was 123338752 (117M). > > Using the original mount time was 55 sec, with the new image it is only > 8.5 sec. My initial test was only about the mount time. I have now also tried to exercise the patched file system and with very little testing I get CRC or ECC errors. # mount /dev/mtdblock2 /mnt -t jffs2 # mkdir /mnt/testdir # umount /mnt jffs2_flush_wbuf(): Write failed with -5 # mount /dev/mtdblock2 /mnt -t jffs2 mtd->read(0x1fbec bytes from 0x1fc0414) returned ECC error Empty flash at 0x01fc2f1c ends at 0x01fc3000 # With plain 2.6.9-rc4-omap1 with fresh CVS MTD code, I don't see anything weird occuring. Jarkko Lavinen ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-21 12:49 ` Jarkko Lavinen @ 2004-10-21 19:11 ` Ferenc Havasi 2004-10-22 9:58 ` Ferenc Havasi 1 sibling, 0 replies; 82+ messages in thread From: Ferenc Havasi @ 2004-10-21 19:11 UTC (permalink / raw) To: Jarkko Lavinen; +Cc: dwmw2, Kluba Patrik, linux-mtd Hi Jarkko, > My initial test was only about the mount time. I have now also tried > to exercise the patched file system and with very little testing I get > CRC or ECC errors. > > # mount /dev/mtdblock2 /mnt -t jffs2 > # mkdir /mnt/testdir > # umount /mnt > jffs2_flush_wbuf(): Write failed with -5 > # mount /dev/mtdblock2 /mnt -t jffs2 > mtd->read(0x1fbec bytes from 0x1fc0414) returned ECC error > Empty flash at 0x01fc2f1c ends at 0x01fc3000 > # > > With plain 2.6.9-rc4-omap1 with fresh CVS MTD code, I don't see anything > weird occuring. Can you send me the full kernel log file? Another interesting test would be to test the new image (sumtool) with the CVS MTD code. Because the summary is RWCOMPAT_DELETE node it should works well. It would be nice to now if there is CRC/ECC errors in this case. Thanks, Ferenc ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-21 12:49 ` Jarkko Lavinen 2004-10-21 19:11 ` Ferenc Havasi @ 2004-10-22 9:58 ` Ferenc Havasi 1 sibling, 0 replies; 82+ messages in thread From: Ferenc Havasi @ 2004-10-22 9:58 UTC (permalink / raw) To: Jarkko Lavinen; +Cc: dwmw2, linux-mtd Jarkko Lavinen wrote: > My initial test was only about the mount time. I have now also tried > to exercise the patched file system and with very little testing I get > CRC or ECC errors. > > # mount /dev/mtdblock2 /mnt -t jffs2 > # mkdir /mnt/testdir > # umount /mnt > jffs2_flush_wbuf(): Write failed with -5 > # mount /dev/mtdblock2 /mnt -t jffs2 > mtd->read(0x1fbec bytes from 0x1fc0414) returned ECC error > Empty flash at 0x01fc2f1c ends at 0x01fc3000 > # > > With plain 2.6.9-rc4-omap1 with fresh CVS MTD code, I don't see anything > weird occuring. We are tring to find out what happens here... Jarkko previously sent me some more detail, the logs starts with: > sh-2.05b# /rootfstest.sh > Mounting file system: Ok > Creating a test directory: Ok > Creating a test file: mtd->read(0x44 bytes from 0x1fa344c) returned ECC error > Data CRC 6d0b1da8 != calculated CRC 9c4f3838 for node at 01fa344c > mtd->read(0x44 bytes from 0x1fa3e20) returned ECC error > Data CRC 5127cb7f != calculated CRC 057e127c for node at 01fa3e20 > mtd->read(0x44 bytes from 0x1fa4388) returned ECC error > mtd->read(0x44 bytes from 0x1fa5cb8) returned ECC error > mtd->read(0x44 bytes from 0x1fa6748) returned ECC error > mtd->read(0x44 bytes from 0x1fa7b30) returned ECC error > Data CRC 72b41a04 != calculated CRC ebc121db for node at 01fa7b30 > mtd->read(0x44 bytes from 0x1fa866c) returned ECC error > Data CRC 9ff1d419 != calculated CRC cb2cce56 for node at 01fa866c It means the mounting is done successfully. The first problem is when the filesystem try to read jffs2_raw_inode nodes (if I am right the 0x44 is the size of that). It is not successfull (I don't know why), and it cause CRC errors, too. The only one differences should be only the the original version already read this 0x44 before (during mount time), the summary version did not read yet, just know where it is from the summary. Jarkko, one more interesing thing can be (if you have that image) to see what is at the place 0x1fa344c, 01fa3e20, ... with the tool jffs2dump. If anyone have any idea that is welcome. Unfortunatelly we don't have real NAND device, and it works with our emulator. Thanks, Ferenc ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-20 14:26 Ferenc Havasi ` (2 preceding siblings ...) 2004-10-21 12:49 ` Jarkko Lavinen @ 2004-10-21 13:24 ` David Woodhouse 2004-10-21 20:05 ` Ferenc Havasi 3 siblings, 1 reply; 82+ messages in thread From: David Woodhouse @ 2004-10-21 13:24 UTC (permalink / raw) To: Ferenc Havasi; +Cc: linux-mtd, jffs-dev On Wed, 2004-10-20 at 16:26 +0200, Ferenc Havasi wrote: > Dear All, > > Here is the latest version of our mount time improvement. It's looking good, but the kernel really needs to be able to write these summaries for _itself_ in order to give a real improvement over the long term. If the file system has to be read-only we might as well be using cramfs, and if the summary becomes obsolete over time we might as well not bother in a lot of cases. -- dwmw2 ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-21 13:24 ` David Woodhouse @ 2004-10-21 20:05 ` Ferenc Havasi 2004-10-22 12:44 ` Artem Bityuckiy 0 siblings, 1 reply; 82+ messages in thread From: Ferenc Havasi @ 2004-10-21 20:05 UTC (permalink / raw) To: David Woodhouse; +Cc: linux-mtd, jffs-dev David Woodhouse wrote: > It's looking good, but the kernel really needs to be able to write these > summaries for _itself_ in order to give a real improvement over the long > term. If the file system has to be read-only we might as well be using > cramfs, and if the summary becomes obsolete over time we might as well > not bother in a lot of cases. Our plan for it: We would like to store some additional information in jeb struct: - a type information, where there this type can be INODE_ONLY and ANYTHING_OTHER. This information is easy to detect during mount time. - a predicted summary size (calculated dinamically). It will be used to decide when to generate the summary. Ceratinly only for INODE_ONLY erase blocks. If I am right every node allocation is done by jffs2_reserve_space(). We would like to modify it, and introduce a new interface for it called jffs2_reserve_space_for_inode() function. Every inode storing function (there is no too much I think) should call jffs2_reserve_space_for_inode() with some extra information (inode number...). jffs2_reserve_space() should use only ANYTHING_OTHER eraseblocks, as jffs2_reserve_space_for_inode() use only INODE_ONLY ones. If there is no free space in them it should use the usual technique to find a clean eraseblock and start to store the new node in it. The generating of summary is also the task of jffs2_reserve_space_for_inode(), if the new inode (+summary) is not fit in the erase block, it will generates summary. What do you think? Regards, Ferenc ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-21 20:05 ` Ferenc Havasi @ 2004-10-22 12:44 ` Artem Bityuckiy 2004-10-25 9:36 ` Ferenc Havasi 2004-10-26 9:29 ` Jarkko Lavinen 0 siblings, 2 replies; 82+ messages in thread From: Artem Bityuckiy @ 2004-10-22 12:44 UTC (permalink / raw) To: Ferenc Havasi; +Cc: linux-mtd, David Woodhouse, jffs-dev Hello Ferenc, At first, please, let me describe your design shortly to be sure I understand it and we both thinking the same way. Essentially, your design is based on the fact that you do not want to refer directory entries in the summary nodes. Motivation that you will keep almost the copy of direntries in the summary, thus: 1. duplicating too many information. 2. you suppose there will not be the mount speed acceleration. So, for this purpose you are going to distribute the inode nodes and other (including direntry nodes) by different blocks. Those blocks, who contain only the inode nodes, will have summaries, other blocks - will not. I think this is not the best solution. Why? In general, because I do not like the following: A. Your idea to distribute inode nodes and other nodes between different blocks. B. Your assumption that the directory information in summaries will not affect the mount time. The following are reasons concerning the item A. 1. Your change will affect JFFS2 very heavily. You will introduce restriction into JFFS2. Another improvements may not work with such restriction. Now all the blocks are equivalent. But you want to distinguish between two kins of blocks. Don't you think it is too complicated decision? 2. Think about the wear-leveling. In JFFS it was ideal. In JFFS2 it is good, but not so ideal. I average, the inode nodes are changed more often (just think about FIFOs, we told about them in this list recently). So, you will need to Garbage Collect the NODE_ONLY blocks more often. So, I afraid the wear-leveling will suffer from your improvement. 3. Imagine the file system with *lots* of very small files. I this case, the direntries portion on the media will be large enough. And the mount time of such file system will not be improved very well. 4. It seems for me you will need to increase the number of blocks which are reserved for the garbage collection (double ?). This is also minor drawback. The following are reasons concerning the item B. I believe that if we have directory references in summaries, this will increase the mount speed. 1. At first, we will store fewer data! We don't need to keep the common headers, CRCs and mctimes. 2. At the second, we may compress summary (direntries aren't compressed)! 3. And the third, on NAND there is difference between reading lots of different pages or few pages. I propose the another design. 1. Keep direntry references in summaries too and hence, do not distinguish between blocks with inode nodes and direntries. 2. Compress summaries. So, you will avoid a lot of problems related to teaching the GC distinguish between different blocks. This will be more natural. I believe, summaries must refer *any* node in block. This is more simple and clean design. Why you do not like this? I see only one potential problem: direntries may have long names (up to 255 symbols). this may lead to large summaries. But in this case we may do: 1. Improve the JFFS2 itself. Keep, say, only 20, characters in the full_dirent structure. Most of direntries will fit. For other, we will just read the flash. 2. We may not touch JFFS2, and keep only 20 characters in summaries. For other direntries, we may read them from flash (keeping theirs flash offsets instead of names). Comments? -- Best regards, Artem B. Bityuckiy Oktet Labs (St. Petersburg), Software Engineer. +78124286709 (office) +79112449030 (mobile) E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-22 12:44 ` Artem Bityuckiy @ 2004-10-25 9:36 ` Ferenc Havasi 2004-10-25 10:56 ` Artem Bityuckiy 2004-10-25 11:21 ` Artem Bityuckiy 2004-10-26 9:29 ` Jarkko Lavinen 1 sibling, 2 replies; 82+ messages in thread From: Ferenc Havasi @ 2004-10-25 9:36 UTC (permalink / raw) To: dedekind, David Woodhouse; +Cc: linux-mtd, jffs-dev Hi Artem, > So, for this purpose you are going to distribute the inode nodes and > other (including direntry nodes) by different blocks. Those blocks, who > contain only the inode nodes, will have summaries, other blocks - will not. Yes, I think there are three kinds of nodes: - type A contains relevant amount of data which is not needed at mount time (jffs2_raw_inode) - type B is (almost) fully needed at mount time (jffs2_raw_dirent) - type C is any other (unkown, developements in the future...) To achieve as much mount time speed up as possible I think we should distinguish them. Using summary the really relevant speed up will be only at node type A. We can also generate summary for type B, but that (as you wrote) relevant ratio of the information will be duplicated. So we whould like to intorduce two kinds of erase blocks: - erase blocks with summary: it will store (now only) type A nodes, maybe later some of type B - erase block without summary: it will store all of type C and B nodes which is not stored before > 1. Your change will affect JFFS2 very heavily. You will introduce > restriction into JFFS2. Another improvements may not work with such > restriction. Now all the blocks are equivalent. But you want to > distinguish between two kins of blocks. Don't you think it is too > complicated decision? What kind of restriction do you mean? We don't introduce any restrictions. The "type C" kind of nodes are processed as before, using the usual scanning method. If you what to force for every node to make their represenation in the summary, that whould be a restriction. I think for some kinds of node summary is meaningful, and for some kinds not. If we mix them that can be a very big slow down, if you what to process them only with making a reference in the summary to its offset, because if you (for example) what to read only 50 bytes (size of the node) you will have to read 512/2048 bytes depening on the flash. (where mostly there will be inode nodes which is not neccesery to read because that is int he summary) But if all of this "not summarized, small" nodes are stored in a "seperated" erase block than the this 512/2048 byte reading will not be unnecessary (because on the remaining 462-1998 bytes will store also relevant information, which is not in the summary). > 2. Think about the wear-leveling. In JFFS it was ideal. In JFFS2 it is > good, but not so ideal. I average, the inode nodes are changed more > often (just think about FIFOs, we told about them in this list > recently). So, you will need to Garbage Collect the NODE_ONLY blocks > more often. So, I afraid the wear-leveling will suffer from your > improvement. I think the GC solves it "automaticly". This mark (SUMMARIZED/NOT_SUMMARIZED) is not a premament thing, it is done "pseudo randomly". I aggree that it cause some different behavior in wear-leveling but I don't think it makes it relevantly worse. > 4. It seems for me you will need to increase the number of blocks which > are reserved for the garbage collection (double ?). This is also minor > drawback. I don't understand what do you mean here. > I believe that if we have directory references in summaries, this will > increase the mount speed. > 1. At first, we will store fewer data! We don't need to keep the common > headers, CRCs and mctimes. > 2. At the second, we may compress summary (direntries aren't compressed)! > 3. And the third, on NAND there is difference between reading lots of > different pages or few pages. Yes, we should try it - to store dirents in SUMMARIZED erase blocks. But it can be a improvement later, for first we need a well working stable system - and this is urgent for us now. > 2. Compress summaries. It makes harder to determine the optimal time of summary generation (it is easy to see the summary size, but here the compressed size of it the relevant). It can cause smaller image but may cause some slow down, too. We may introduce it later as an option. So now we have two open discussion: - is the SUMMARIZED / NOT_SUMMARIZED distiguishment good or not - in the first version do we need dirents in the summary or not Fortunatelly the effects (and side effects) of this improvements will be active only if the new kernel option is enabled, and don't kill any other future improvements. I curious about (at least) David's optinion about these topics. Bye, Ferenc ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-25 9:36 ` Ferenc Havasi @ 2004-10-25 10:56 ` Artem Bityuckiy 2004-10-25 15:30 ` Ferenc Havasi 2004-10-25 11:21 ` Artem Bityuckiy 1 sibling, 1 reply; 82+ messages in thread From: Artem Bityuckiy @ 2004-10-25 10:56 UTC (permalink / raw) To: Ferenc Havasi; +Cc: linux-mtd, David Woodhouse, jffs-dev Hello Ferenc, > Yes, I think there are three kinds of nodes: > - type A contains relevant amount of data which is not needed at mount > time (jffs2_raw_inode) > - type B is (almost) fully needed at mount time (jffs2_raw_dirent) > - type C is any other (unkown, developements in the future...) > > To achieve as much mount time speed up as possible I think we should > distinguish them. This is what I really do not like. Ok, let us discuss now only this topic. Lt I explain why I believe it is vad and very *unnatural* to introduce two or more kinds of blocks. The example of JFFS2 change that I consider natural is the introduction of new node type. It is natural, because of when JFFS2 was designed, this possibility was foreseen and taken into account. It is relatively easy to do this. It is possible to do this and do not affect other things in the JFFS2. Conversely, the introducing several block types was not foreseen in the JFFS2 design. And all things in the JFFS2 are coded with the assumption all the blocks are equivalent. This is my point view on the issue in general. Now I will try to illustrate why I think so. 1. In JFFS2 there are several lists of blocks - clean_list, dirty_list, very_dirty_list?. Are you going to introduce clean_list_typeA, dirty_list_typeA, very_dirty_list_typeA, clean_list_typeB, dirty_list_typeB, very_dirty_list_typeB ? 2. Just do 'grep "_list" * | grep -e "\(dirty\)\|\(very\)"' and see how many places in JFFS2 where these lists are changed. Do you think it is natural to introduce 3 more lists? I believe not. What if somebody else will introduce one more type of block? 3. There is write buffer in the JFFS2 which is used in case of NAND. Are you going to have two wbufs? This is also significant change. 4. Now the GC just gives one block, and moves all the valid nodes to another one. In your case (if you have the JFFS2 image which was created by older code, without your patch, where all node types are mixed), you will need to move one type of nodes to one block, another to the another block. So, I think you will be needed to change many things in JFFS2. You have a risk to hit on a can of worms. So, do you agree that this change is *unnatural* ? =================================================================== >> 4. It seems for me you will need to increase the number of blocks >> which are reserved for the garbage collection (double ?). This is also >> minor drawback. > I don't understand what do you mean here I mean the sb->resv_blocks_gcmerge and related. You will need to increase it, which is not very good. -- Best regards, Artem B. Bityuckiy Oktet Labs (St. Petersburg), Software Engineer. +78124286709 (office) +79112449030 (mobile) E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-25 10:56 ` Artem Bityuckiy @ 2004-10-25 15:30 ` Ferenc Havasi 2004-10-26 9:59 ` Artem Bityuckiy 0 siblings, 1 reply; 82+ messages in thread From: Ferenc Havasi @ 2004-10-25 15:30 UTC (permalink / raw) To: dedekind; +Cc: linux-mtd, David Woodhouse, jffs-dev Hi Artem, > > To achieve as much mount time speed up as possible I think we should > > distinguish them. > This is what I really do not like. > > Ok, let us discuss now only this topic. Lt I explain why I believe it is > vad and very *unnatural* to introduce two or more kinds of blocks. You are right, it can be unnatural in point of the original design of the JFFS2. But I think in point of the connection of this optimization and JFFS2 it is more natural than simple store offsets in the summary, or copy all the information into it. Our plan was modify wbuf (make a second one) and modify jffs2_reserve_space to select the right wbuf and generate summary. Never planded to introduce new clean_*, dirty_*, ... lists, thats really too difficult. > 3. There is write buffer in the JFFS2 which is used in case of NAND. Are > you going to have two wbufs? This is also significant change. Yes, we started to implement it yesterday and now agree. It is really not easy, and we don't write to rewrite the NAND handling part of JFFS2 whithout a real NAND device. Maybe at the design of JFFS3 :) So you convinced me. We will change the design of summary. The inodes and dirents will be also in the summary. All other nodes will be copied as itself into the summary and cause a warning. The summary support will be a required thing for new node types, too. In the kernel we will have to modify 1. jffs2_scan_eraseblock(), as it is already in our patch 2. jeb struct to store generated the summary dinamically (one plus field) 3. jffs2_reserve_space(), which will have a new parameter (summary size), which can be JFFS2_SUMMARY_INODE_SIZE or JFFS2_SUMMARY_DIRENT_SIZE(namelen). It can decide when to generate summary and it can do this generation. 4. jffs2_flash_writev(), which is used to write info to flash. It can parse the node (similar to sumtool) and store the summary of it in its jeb. If it works we'll check the effect of compressing the summary. (size and speed) Comments? Bye, Ferenc P.s.: Thanks for this good conversation. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-25 15:30 ` Ferenc Havasi @ 2004-10-26 9:59 ` Artem Bityuckiy 2004-10-26 10:21 ` Ferenc Havasi 0 siblings, 1 reply; 82+ messages in thread From: Artem Bityuckiy @ 2004-10-26 9:59 UTC (permalink / raw) To: Ferenc Havasi; +Cc: David Woodhouse, linux-mtd, jffs-dev Hello Ferenc, Ferenc Havasi wrote: > In the kernel we will have to modify > 1. jffs2_scan_eraseblock(), as it is already in our patch > 2. jeb struct to store generated the summary dinamically (one plus field) IMHO, since the summary relates only to one block, the current block, it is logical to refer the summary from the jffs2_sb_info, not from jffs2_erase_blocks. It is also not very nice to store it in the jffs2_erase_blocks since it will increase the size of array of JFFS2 blocks (c->blocks[]). > 3. jffs2_reserve_space(), which will have a new parameter (summary > size), which can be JFFS2_SUMMARY_INODE_SIZE or > JFFS2_SUMMARY_DIRENT_SIZE(namelen). It can decide when to generate > summary and it can do this generation. Yes, I also think so. Currently the jffs2_do_reserve_space() do (as I understand): 1. If the current block (c->nextblock) have space and it is sufficient for request, it reserves it. 2. If the c->nextblock has fewer size, than requested, the c->nextblock is wasted, put to the correspondent list (dirty_list, etc), free block is taken and reserved. Thus, the jffs2_do_reserve_space() should be improved to be able to save some space for summary. And, some function like jffs2_write_summary() which will be called before jffs2_do_reserve_space() takes new block from the free_list. > 4. jffs2_flash_writev(), which is used to write info to flash. It can > parse the node (similar to sumtool) and store the summary of it in its jeb. May be write here... Didn't think a lot... May be as I wrote, in jffs2_do_reserve_space()... I also offer you to include direntries in summaries and compress them. See: sizeof(struct jffs2_raw_dirent) = 40 (without name) you will need to store in your summary only: totlen pino version ino nsize type name which is 24 bytes. You don't store all data! Of course, in case of long names things are not so good... If you also compress them, they will be smaller (minus 50-70%)! So, if there are few direntries in block, why not to store them in summary? Did you measured the time of summary uncompress on your system? I can't know for sure, but I suspect that if you have, say, 200MHz system, the time of uncompression = o(time of block read)! There is one more issue: if there are too many direntries in block, summary may become too large (the compression helps here). In this case you may not write summary or don't mention direntries in summary. -- Best regards, Artem B. Bityuckiy Oktet Labs (St. Petersburg), Software Engineer. +78124286709 (office) +79112449030 (mobile) E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-26 9:59 ` Artem Bityuckiy @ 2004-10-26 10:21 ` Ferenc Havasi 2004-10-26 11:05 ` Artem Bityuckiy 0 siblings, 1 reply; 82+ messages in thread From: Ferenc Havasi @ 2004-10-26 10:21 UTC (permalink / raw) To: dedekind; +Cc: David Woodhouse, linux-mtd, jffs-dev Hi Artem, > IMHO, since the summary relates only to one block, the current block, it > is logical to refer the summary from the jffs2_sb_info, not from > jffs2_erase_blocks. It is also not very nice to store it in the > jffs2_erase_blocks since it will increase the size of array of JFFS2 > blocks (c->blocks[]). Is it sure than only one non-full erase block is in the filesystem? Non-full means here that there is some nodes already in that, but also there is some free space at the end of it. >> 4. jffs2_flash_writev(), which is used to write info to flash. It can >> parse the node (similar to sumtool) and store the summary of it in its >> jeb. > > May be write here... Didn't think a lot... May be as I wrote, in > jffs2_do_reserve_space()... As I see jffs2_do_reserve space is called before inode/... allocation in most cases. So at that time the summary information is not know - but at writing it have to be known certainly. > So, if there are few direntries in block, why not to store them in summary? You may misunderstood me. In the previous letter I wrote: "So you convinced me. We will change the design of summary. The inodes and dirents will be also in the summary." So now we do plan to store dirents in the summary. :) > Did you measured the time of summary uncompress on your system? I can't > know for sure, but I suspect that if you have, say, 200MHz system, the > time of uncompression = o(time of block read)! It depends on the compressor. We will test it with zlib/rtime. I whould like to implement as an optional feature. > There is one more issue: if there are too many direntries in block, > summary may become too large (the compression helps here). In this case > you may not write summary or don't mention direntries in summary. Let see how it work, and after we can make it more optimal :) Bye, Ferenc ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-26 10:21 ` Ferenc Havasi @ 2004-10-26 11:05 ` Artem Bityuckiy 2004-10-26 13:52 ` Ferenc Havasi 0 siblings, 1 reply; 82+ messages in thread From: Artem Bityuckiy @ 2004-10-26 11:05 UTC (permalink / raw) To: Ferenc Havasi; +Cc: linux-mtd, David Woodhouse, jffs-dev Ferenc, > Is it sure than only one non-full erase block is in the filesystem? > Non-full means here that there is some nodes already in that, but also > there is some free space at the end of it. I didn't analyse this accurately, but my vision is that there is one current block (c->nextblock). Even GC moves nodes to it. This is because the jffs2_do_reserve_space() is always used (even by GC), and the jffs2_do_reserve_space() always uses c->nextblock. > As I see jffs2_do_reserve space is called before inode/... allocation in > most cases. So at that time the summary information is not know - but at > writing it have to be known certainly. May be... From another hand you may write summary every time the jffs2_reserve_space() fetches new block from the free_list... Anyway, this is not fundamental... > You may misunderstood me. In the previous letter I wrote: "So you > convinced me. We will change the design of summary. The inodes and > dirents will be also in the summary." > > So now we do plan to store dirents in the summary. :) OK, sorry. :-) > Let see how it work, and after we can make it more optimal :) Agree :-) Also, please, take into account that there may be checkpoint nodes (I'm implementing this). So, I think you need to have a generic mechanism to add new node types to your summary. Also, I think it is good to have a generic mechanism to just refer some nodes from summaries (for example, direntries with long names or something else). Thank you for conversation too. :-) -- Best regards, Artem B. Bityuckiy Oktet Labs (St. Petersburg), Software Engineer. +78124286709 (office) +79112449030 (mobile) E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-26 11:05 ` Artem Bityuckiy @ 2004-10-26 13:52 ` Ferenc Havasi 0 siblings, 0 replies; 82+ messages in thread From: Ferenc Havasi @ 2004-10-26 13:52 UTC (permalink / raw) To: dedekind; +Cc: linux-mtd, David Woodhouse, jffs-dev Hi Artem, > Also, please, take into account that there may be checkpoint nodes (I'm > implementing this). So, I think you need to have a generic mechanism to > add new node types to your summary. > > Also, I think it is good to have a generic mechanism to just refer some > nodes from summaries (for example, direntries with long names or > something else). Yes, it will be easy to extend. We also need a this general support - because we will introduce a new node type, too, becauseof the model file support, which will start to commit when David finishes his patch for Linus. Bye, Ferenc ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-25 9:36 ` Ferenc Havasi 2004-10-25 10:56 ` Artem Bityuckiy @ 2004-10-25 11:21 ` Artem Bityuckiy 1 sibling, 0 replies; 82+ messages in thread From: Artem Bityuckiy @ 2004-10-25 11:21 UTC (permalink / raw) To: Ferenc Havasi; +Cc: linux-mtd, jffs-dev > I curious about (at least) David's optinion about these topics. I also wonder why people are not very active :-) -- Best regards, Artem B. Bityuckiy Oktet Labs (St. Petersburg), Software Engineer. +78124286709 (office) +79112449030 (mobile) E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-22 12:44 ` Artem Bityuckiy 2004-10-25 9:36 ` Ferenc Havasi @ 2004-10-26 9:29 ` Jarkko Lavinen 2004-10-26 10:24 ` Ferenc Havasi 2004-10-26 10:34 ` Artem Bityuckiy 1 sibling, 2 replies; 82+ messages in thread From: Jarkko Lavinen @ 2004-10-26 9:29 UTC (permalink / raw) To: linux-mtd, jffs-dev; +Cc: ext Artem Bityuckiy, David Woodhouse [-- Attachment #1: Type: text/plain, Size: 2655 bytes --] I tried to see with jffs2dump how much Inodes and Dirents I have on root filesystem on Arm testbed. Quick and dirty Perl script attached. This isn't accurate as the calculated total image size misses at least the final padding on the last erase block. The size of the plain JFFS2 image is 31.1 MiB. The root fs consists of all applications and libraries and no user data. $ jffs2dump -c rootfs.jffs2 | perl jffs2stats.pl Number of dirents: 6144. Total dirent node space: 304911 (0.9%) Average dirent len: 49.6 Total dirent name space: 76671 Average name len: 12.5 Number of Inodes: 21197 Total Inode space: 32254866 (99.1%) Average Inode size: 1521.7 Padding: 37326 0.1% Total image size: 32559777 $ ls -l rootfs.jffs2 -rw-r--r-- 1 root root 32597104 Oct 20 15:11 rootfs.jffs2 With sumtool the image size grows to 31.8 MiB $ jffs2dump -c rootfs-sum.jffs2 | perl jffs2stats.pl Number of dirents: 6144. Total dirent node space: 304911 (0.9%) Average dirent len: 49.6 Total dirent name space: 76671 Average name len: 12.5 Number of Inodes: 21197 Total Inode space: 32254866 (97.2%) Average Inode size: 1521.7 Number of Inode Summary nodes: 251 Total Inode Sum space: 631524, (1.9%) Average Sum node size: 2516.0 Padding: 153063 0.5% Total image size: 33191301 $ ls -l rootfs-sum.jffs2 -rw-r--r-- 1 root root 33423360 Oct 20 15:23 rootfs-sum.jffs2 If dentries were stored just as they are (unstripped and uncompressed) in the summary, the summary size would grow by 50% to about 3% of the whole image size. On Fri, Oct 22, 2004 at 04:44:13PM +0400, ext Artem Bityuckiy wrote: > I believe that if we have directory references in summaries, this will > increase the mount speed. > > 1. At first, we will store fewer data! We don't need to keep the common > headers, CRCs and mctimes. > 2. At the second, we may compress summary (direntries aren't compressed)! > 3. And the third, on NAND there is difference between reading lots of > different pages or few pages. I tried Ferenc's earlier mount time patch in August and the 52s mount time dropped then to 14s. If I understand right, inodes and dentries were then mixed in the erase block and the summary was for inodes only. This shows reading dentries from semirandom places is expensive. Ferenc's latest patch put dentries on their own erase block in consecutive order. Considering only the read efficiency from the media, reading consecutive, uncompressed, and unstripped dentries from a summary should cost no more than reading them from dedicated erase block. Jarkko Lavinen [-- Attachment #2: jffs2stats.pl --] [-- Type: text/x-perl, Size: 3230 bytes --] #! /usr/bin/perl $EBLOCKSIZE=131072; $dirents = $totdirentlen = $totnamelen = 0; $inodes = $totinodelen = 0; $sumnodes = $totsumnodelen = 0; $totpadlen = 0; $gaps = $totgaplen = 0; $nextaddr = 0; sub checkpadding { my ($addr, $totlen) = @_; my $len = hex($addr) - $nextaddr; if ($len > 0) { if (hex($addr) % $EBLOCKSIZE == 0 || $len <= 3) { $totalpadlen += $len; } else { print sprintf "Gap seen at %08x .. $addr, length $len\n", $nextaddr; $gaps++; $totgaplen += hex($addr) - $nextaddr; } } $nextaddr = hex($addr) + hex($totlen); } while(<>) { chop; if (/^\s+Dirent/) { die "Cannot parse $_" if (! /^ \s+ Dirent \s+ node \s at \s+ (\w+), \s+ totlen \s+ (\w+), \s+ \#pino \s+ (\w+), \s+ version \s+ (\w+), \s+ \#ino \s+ (\w+), \s+ nsize \s+ (\w+), \s+ name \s+ (.*) $/x); my ($addr, $totlen, $pino, $version, $ino, $nsize, $name) = ($1, $2, $3, $4, $5, $6, $7); &checkpadding($addr, $totlen); $dirents++; $totdirentlen += hex($totlen); $totnamelen += hex($nsize); } elsif (/^\s+Inode Sum/) { die "Cannot parse $_" if (! /^ \s+ Inode \s Sum \s+ node \s at \s+ (\w+), \s+ totlen \s+ (\w+), \s+ sum_num \s+ (\w+), \s+ cleanmarker \s size \s+ (\w+) \s* $/x); my ($addr, $totlen, $sum_num, $cleanmarksize) = ($1, $2, $3, $4); &checkpadding($addr, $totlen); $sumnodes++; $totsumnodelen += hex($totlen); } elsif (/^\s+Inode/) { die "Cannot parse $_" if (! /^ \s+ Inode \s+ node \s at \s+ (\w+), \s+ totlen \s+ (\w+), \s+ \#ino \s+ (\w+), \s+ version \s+ (\w+), \s+ isize \s+ (\w+), \s+ csize \s+ (\w+), \s+ dsize \s+ (\w+), \s+ offset \s+ (\w+) \s* $/x); my ($addr, $totlen, $ino, $version, $isize, $csize, $dsize, $offset) = ($1, $2, $3, $4, $5, $6, $7, $8); &checkpadding($addr, $totlen); $inodes++; $totinodelen += hex($totlen); } else { die "Cannot parse $_"; } } $totalsize = $totdirentlen + $totinodelen + $totsumnodelen + $totpadlen + $totgaplen; print "Number of dirents:\t$dirents.\n"; print sprintf " Total dirent node space:\t$totdirentlen (%.1f%%)\n", 100.0*$totdirentlen/$totalsize; print " Average dirent len:\t", sprintf("%.1f", $totdirentlen/$dirents), "\n" if ($dirents > 0); print " Total dirent name space:\t$totnamelen\n"; print sprintf(" Average name len:\t%.1f\n", $totnamelen/$dirents) if ($dirents > 0); print "\n"; print "Number of Inodes:\t$inodes\n"; print sprintf " Total Inode space:\t$totinodelen (%.1f%%)\n", 100.0*$totinodelen/$totalsize; print " Average Inode size:\t", sprintf("%.1f", $totinodelen/$inodes), "\n" if ($inodes > 0); if ($sumnodes) { print "\n"; print "Number of Inode Summary nodes:\t$sumnodes\n"; print sprintf " Total Inode Sum space:\t$totsumnodelen, (%.1f%%)\n", 100.0*$totsumnodelen/$totalsize; print " Average Sum node size:\t", sprintf("%.1f", $totsumnodelen/$sumnodes), "\n"; } print sprintf "\nPadding:\t$totalpadlen %.1f%%\n", 100.0*$totalpadlen/$totalsize; print "Total image size: $totalsize\n"; ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-26 9:29 ` Jarkko Lavinen @ 2004-10-26 10:24 ` Ferenc Havasi 2004-10-26 10:34 ` Artem Bityuckiy 1 sibling, 0 replies; 82+ messages in thread From: Ferenc Havasi @ 2004-10-26 10:24 UTC (permalink / raw) To: Jarkko Lavinen; +Cc: ext Artem Bityuckiy, linux-mtd, David Woodhouse, jffs-dev Hi Jarkko, > If dentries were stored just as they are (unstripped and uncompressed) > in the summary, the summary size would grow by 50% to about 3% of the > whole image size. Thanks, good to know it. Did you got ECC/CRC errors? The most interest test for me whould be to test the new (sumtool) image with the original kernel (because the summary nodes are compatibles it should work), and see if there is ECC/CRC errors or not. Bye, Ferenc ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-10-26 9:29 ` Jarkko Lavinen 2004-10-26 10:24 ` Ferenc Havasi @ 2004-10-26 10:34 ` Artem Bityuckiy 1 sibling, 0 replies; 82+ messages in thread From: Artem Bityuckiy @ 2004-10-26 10:34 UTC (permalink / raw) To: Jarkko Lavinen; +Cc: David Woodhouse, linux-mtd, jffs-dev Hello Jarkko, > I tried Ferenc's earlier mount time patch in August and the 52s mount > time dropped then to 14s. If I understand right, inodes and dentries > were then mixed in the erase block and the summary was for inodes > only. This shows reading dentries from semirandom places is > expensive. This is very good that direntries are distributed more or less uniformly in average. > > Ferenc's latest patch put dentries on their own erase block in > consecutive order. Considering only the read efficiency from the > media, reading consecutive, uncompressed, and unstripped dentries from > a summary should cost no more than reading them from dedicated erase > block. > Definitely true - the second patch must be better than the first one. But unfortunately, it hard to do this dinamically :-( Ferenc tried... But in my proposition, we will also refer direntries in the summary - this is not the same as to read direntries from where they are placed, this is another thing, especially in case of NAND! There is difference (if we have NAND) - whether to read one 512 NAND page containing compressed information about 20-25 direntries or to read 20-25 *different* NAND pages. So, I think, new design will also better than the early Ferenc's patch :-) -- Best regards, Artem B. Bityuckiy Oktet Labs (St. Petersburg), Software Engineer. +78124286709 (office) +79112449030 (mobile) E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru ^ permalink raw reply [flat|nested] 82+ messages in thread
* JFFS2 mount time @ 2004-12-15 23:19 Gareth Bult (Encryptec) 2004-12-16 0:15 ` Josh Boyer 2004-12-16 13:43 ` Ferenc Havasi 0 siblings, 2 replies; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-15 23:19 UTC (permalink / raw) To: Linux MTD Hi, I've been working on the MTD block driver for JFFS2 and now seem to have a relatively stable environment with a JFFS2 root filesystem on a USB flash key device. (mtd block device speed now approximates actual device speed on both read and write) It seems to be much quicker (generally) than the previous ext2 incarnation. However, I'm looking at 30secs to mount the root filesystem. I would be very grateful of some advice on the following before I start digging through the JFFS2 code itself; a. Is it possible (conceptually) to background the mount scan so the system can boot while the scan happens ? b. Is there an easy way to bypass the scan and does JFFS3 do this ? Any help/advice would be much appreciated. Many thanks, Gareth. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-15 23:19 Gareth Bult (Encryptec) @ 2004-12-16 0:15 ` Josh Boyer 2004-12-16 1:02 ` Gareth Bult (Encryptec) 2004-12-16 13:43 ` Ferenc Havasi 1 sibling, 1 reply; 82+ messages in thread From: Josh Boyer @ 2004-12-16 0:15 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD Gareth Bult (Encryptec) wrote: > Hi, > > I've been working on the MTD block driver for JFFS2 and now seem to have > a relatively stable environment with a JFFS2 root filesystem on a USB > flash key device. > (mtd block device speed now approximates actual device speed on both > read and write) I assume you mean blkmtd here. The mtdblock driver is used to present an IDE type interface to flash, whereas blkmtd is used to present a flash interface to an IDE type device. > > It seems to be much quicker (generally) than the previous ext2 > incarnation. > > However, I'm looking at 30secs to mount the root filesystem. How big of a device? > a. Is it possible (conceptually) to background the mount scan so the > system can boot while the scan happens ? Which version of the code are you using? Mount time has been significantly improved in newer versions of the code. > b. Is there an easy way to bypass the scan and does JFFS3 do this ? Not that I know of, because the code needs to read the node information in order to get the versioning of inodes, etc. correct. JFFS3 is essentially the exact same code base as JFFS2 right now. It was just forked about a week ago. josh ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-16 0:15 ` Josh Boyer @ 2004-12-16 1:02 ` Gareth Bult (Encryptec) 2004-12-16 12:53 ` Josh Boyer 0 siblings, 1 reply; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-16 1:02 UTC (permalink / raw) To: Josh Boyer; +Cc: Linux MTD Hi, Sure, the file I've been changing is blkmtd.c. The device I'm aiming for and testing on is 256Mb with a 128k erase size. Currently I'm using the code included in the 2.6.9 kernel source tree with tools from cvs as of about a week ago. Is this likely to contain the latest speedups or should I be looking elsewhere? tia Gareth. On Wed, 2004-12-15 at 18:15 -0600, Josh Boyer wrote: > Gareth Bult (Encryptec) wrote: > > Hi, > > > > I've been working on the MTD block driver for JFFS2 and now seem to have > > a relatively stable environment with a JFFS2 root filesystem on a USB > > flash key device. > > (mtd block device speed now approximates actual device speed on both > > read and write) > > I assume you mean blkmtd here. The mtdblock driver is used to present > an IDE type interface to flash, whereas blkmtd is used to present a > flash interface to an IDE type device. > > > > > It seems to be much quicker (generally) than the previous ext2 > > incarnation. > > > > However, I'm looking at 30secs to mount the root filesystem. > > How big of a device? > > > a. Is it possible (conceptually) to background the mount scan so the > > system can boot while the scan happens ? > > Which version of the code are you using? Mount time has been > significantly improved in newer versions of the code. > > > b. Is there an easy way to bypass the scan and does JFFS3 do this ? > > Not that I know of, because the code needs to read the node information > in order to get the versioning of inodes, etc. correct. > > JFFS3 is essentially the exact same code base as JFFS2 right now. It > was just forked about a week ago. > > josh > ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-16 1:02 ` Gareth Bult (Encryptec) @ 2004-12-16 12:53 ` Josh Boyer 2004-12-16 21:22 ` Gareth Bult (Encryptec) 0 siblings, 1 reply; 82+ messages in thread From: Josh Boyer @ 2004-12-16 12:53 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD On Wed, 2004-12-15 at 19:02, Gareth Bult (Encryptec) wrote: > Hi, Quick FYI, don't top post. The email gremlin will chase you down and make you read http://david.woodhou.se/email.html if you break too many rules ;) > > Sure, the file I've been changing is blkmtd.c. Ok, cool. > > The device I'm aiming for and testing on is 256Mb with a 128k erase > size. > > Currently I'm using the code included in the 2.6.9 kernel source tree > with tools from cvs as of about a week ago. Ok, good. > Is this likely to contain the latest speedups or should I be looking > elsewhere? That should have the speedups I was referring to. Basically checking the node CRCs is deferred until after mount time so the mount succeeds much more quickly. There is one more feature pending that I know of. Ferenc Havasi has some eraseblock summary changes that should help. I believe the intention was to commit that to JFFS3, but I haven't seen it go in yet. josh ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-16 12:53 ` Josh Boyer @ 2004-12-16 21:22 ` Gareth Bult (Encryptec) 2004-12-16 21:28 ` Josh Boyer 0 siblings, 1 reply; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-16 21:22 UTC (permalink / raw) To: Josh Boyer; +Cc: Linux MTD Hi, > There is one more feature pending that I know of. Ferenc Havasi has > some eraseblock summary changes that should help. I believe the > intention was to commit that to JFFS3, but I haven't seen it go in yet. Ok, I have these and will be trying them presently .. just wondered if there was any way I could avoid running "patch" .. :) Tvm Gareth. > josh > ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-16 21:22 ` Gareth Bult (Encryptec) @ 2004-12-16 21:28 ` Josh Boyer 2004-12-16 21:47 ` Gareth Bult (Encryptec) 0 siblings, 1 reply; 82+ messages in thread From: Josh Boyer @ 2004-12-16 21:28 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD Gareth Bult (Encryptec) wrote: > Hi, > >>There is one more feature pending that I know of. Ferenc Havasi has >>some eraseblock summary changes that should help. I believe the >>intention was to commit that to JFFS3, but I haven't seen it go in yet. > > > Ok, I have these and will be trying them presently .. > > just wondered if there was any way I could avoid running "patch" .. :) Bug Ferenc to commit ;). I'm not sure if he subscribes to this list or not. Also, Artem Bityuckiy is working on an inode checkpoint feature for JFFS3. Once he gets it working, it should increase overall performance. If you have any ideas or patches, feel free to submit them to the list. That's what JFFS3 was cloned for, and there isn't too much concern with backwards compatibility. josh ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-16 21:28 ` Josh Boyer @ 2004-12-16 21:47 ` Gareth Bult (Encryptec) 2004-12-17 12:54 ` Josh Boyer 2004-12-18 16:19 ` Jörn Engel 0 siblings, 2 replies; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-16 21:47 UTC (permalink / raw) To: Josh Boyer; +Cc: Linux MTD If you have any ideas or patches, feel free to submit them to the list. > That's what JFFS3 was cloned for, and there isn't too much concern > with backwards compatibility. Mmm, blkmtd.c is pretty much a rewrite, less than half the size and uses the system buffer cache instead of reading / writing directly to the device. This boosts performance x 3 on read x lots on write. [I get x 100 faster] (albeit you lose bad block detection on write) If anyone's interested in the code, let me know and I'll clean it up and post it .. however it'll be a new file and not a patch. I've only tested it on USB key flash, but I'm assuming it'll be similar for any block device / hard disk / whatever .. Gareth. On Thu, 2004-12-16 at 15:28 -0600, Josh Boyer wrote: > Gareth Bult (Encryptec) wrote: > > Hi, > > > >>There is one more feature pending that I know of. Ferenc Havasi has > >>some eraseblock summary changes that should help. I believe the > >>intention was to commit that to JFFS3, but I haven't seen it go in yet. > > > > > > Ok, I have these and will be trying them presently .. > > > > just wondered if there was any way I could avoid running "patch" .. :) > > Bug Ferenc to commit ;). I'm not sure if he subscribes to this list or not. > > Also, Artem Bityuckiy is working on an inode checkpoint feature for > JFFS3. Once he gets it working, it should increase overall performance. > > If you have any ideas or patches, feel free to submit them to the list. > That's what JFFS3 was cloned for, and there isn't too much concern > with backwards compatibility. > > josh ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-16 21:47 ` Gareth Bult (Encryptec) @ 2004-12-17 12:54 ` Josh Boyer 2004-12-17 15:33 ` Gareth Bult (Encryptec) 2004-12-18 16:19 ` Jörn Engel 1 sibling, 1 reply; 82+ messages in thread From: Josh Boyer @ 2004-12-17 12:54 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD On Thu, 2004-12-16 at 15:47, Gareth Bult (Encryptec) wrote: > > blkmtd.c is pretty much a rewrite, less than half the size and uses the > system buffer cache instead of reading / writing directly to the device. > This boosts performance x 3 on read x lots on write. [I get x 100 > faster] (albeit you lose bad block detection on write) Hm, ok. An alternative to using the system buffer would be to make JFFS[23] use a write buffer for those devices. It's already done on NAND which has some special handling. The support for ECC'd NOR probably matches more closely what you'd want to do. Take a look at wbuf.c. > > If anyone's interested in the code, let me know and I'll clean it up and > post it .. however it'll be a new file and not a patch. Well, if it's better than the original we can always take yours and replace what's there. Send it on, performance gains are always welcome. I'm sure there are others interested in what you've done that will review the code. josh ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-17 12:54 ` Josh Boyer @ 2004-12-17 15:33 ` Gareth Bult (Encryptec) 2004-12-17 16:02 ` Josh Boyer 0 siblings, 1 reply; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-17 15:33 UTC (permalink / raw) To: Josh Boyer; +Cc: Linux MTD Hi, On Fri, 2004-12-17 at 06:54 -0600, Josh Boyer wrote: > Hm, ok. An alternative to using the system buffer would be to make > JFFS[23] use a write buffer for those devices. Forgive my ignorance .. but what's the difference between using a write buffer and using the system buffer ? >From my pov, the system buffer is there, already 100% reliable and somewhat larger than any [potential] write buffer ??? (Given the system buffer uses the underlying block devices read / write routines and the block device emulator will typically sit on a block device such as a hard drive or USB flash key .. [?]) > It's already done on > NAND which has some special handling. The support for ECC'd NOR > probably matches more closely what you'd want to do. Take a look at > wbuf.c. Mmm, after looking at wbuf.c I'm none the wiser as to what it does or how it relates .. > Well, if it's better than the original we can always take yours and > replace what's there. Send it on, performance gains are always > welcome. I'm sure there are others interested in what you've done that > will review the code. I think once I get this FL release out the way I'll have a a more in-depth look at JFFS*. It seems to be written almost exclusively for MTD devices .. which is great .. for MTD devices .. but for block devices and flash (USB for example) presented through a standard interface (SCSI for example) it seems there is scope for a much "thinner" filesystem generic devices with a subset of JFFS's feature set. Gareth. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-17 15:33 ` Gareth Bult (Encryptec) @ 2004-12-17 16:02 ` Josh Boyer 2004-12-17 16:46 ` Gareth Bult (Encryptec) 0 siblings, 1 reply; 82+ messages in thread From: Josh Boyer @ 2004-12-17 16:02 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD Gareth Bult (Encryptec) wrote:> > > Forgive my ignorance .. but what's the difference between using a write > buffer and using the system buffer ? > >>From my pov, the system buffer is there, already 100% reliable and > somewhat larger than any [potential] write buffer ??? Perhaps alternative was the wrong word to use on my part. I'll just wait until I see your code. I'm probably confusing myself by making assumptions on what it's doing :). > > Mmm, after looking at wbuf.c I'm none the wiser as to what it does or > how it relates .. It might not relate. IIRC there were some discussions on buffering the writes done by JFFS[23] to the size of a sector on the block device so that partial writes aren't done unnecessarily. This is what the wbuf.c file does. NAND and ECC'd NOR require this because you can only write to a page once (usually) before an erase is needed to write again. For disk devices I don't think it's required, but it could help. Or maybe your code does this at the MTD level. I dunno. I could be spouting nonsense again ;). > > > I think once I get this FL release out the way I'll have a a more > in-depth look at JFFS*. It seems to be written almost exclusively for > MTD devices .. which is great .. for MTD devices .. but for block > devices and flash (USB for example) presented through a standard > interface (SCSI for example) it seems there is scope for a much > "thinner" filesystem generic devices with a subset of JFFS's feature > set. Yes, it is a Journaled _Flash_ File System. So of course it's written for MTD devices. Not sure why blkmtd.c was originally written. Test vehicle for JFFS2, "why not", "I'm bored", etc. Who knows? :) josh ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-17 16:02 ` Josh Boyer @ 2004-12-17 16:46 ` Gareth Bult (Encryptec) 2004-12-17 17:08 ` Artem B. Bityuckiy 2004-12-17 17:10 ` Josh Boyer 0 siblings, 2 replies; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-17 16:46 UTC (permalink / raw) To: Josh Boyer; +Cc: Linux MTD On Fri, 2004-12-17 at 10:02 -0600, Josh Boyer wrote: > For disk devices I don't think it's required, but it could help. Or > maybe your code does this at the MTD level. I dunno. I could be > spouting nonsense again ;). Mmm, as long as writes are aligned on 2k pages and are linear / unidirectional, I'm not sure anything else is really a problem for USB flash drives .. > Yes, it is a Journaled _Flash_ File System. So of course it's written > for MTD devices. Not sure why blkmtd.c was originally written. Test > vehicle for JFFS2, "why not", "I'm bored", etc. Who knows? :) Mm, my point exactly , Journaled _Flash_ File System - as opposed to Journaled _MTD_ _Flash_ File System .. !! USB flash is still flash! - just presented via a generic interface as opposed to an embedded one .. (AFAIK) I guess I was sort of hoping blkmtd.c was written to enable JFFS2 to be more generic and embedded device independent. I suspect however it was written purely for testing purposes .. :( As I understand it, the Flash in USB keys is identical to the MTD type embedded devices, except that the USB keys always come with micro-controllers that handle and optimise read/write/erase operations and present them via a generic PC/USB based disk interface. I'm not sure how fast MTD can be driven, but key manufacturers seem to think that keys should be able to run at 20Mb/sec read and 10Mb/sec write [for large block read/writes] which is much faster than is required for a key to replace a hard disk in a workstation. I'm thinking that JFFS2 could draw in a huge additional user base if it catered for (or at least supported) such devices .. (!) Gareth. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-17 16:46 ` Gareth Bult (Encryptec) @ 2004-12-17 17:08 ` Artem B. Bityuckiy 2004-12-17 17:10 ` Josh Boyer 1 sibling, 0 replies; 82+ messages in thread From: Artem B. Bityuckiy @ 2004-12-17 17:08 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD On Fri, 17 Dec 2004, Gareth Bult (Encryptec) wrote: > On Fri, 2004-12-17 at 10:02 -0600, Josh Boyer wrote: > > For disk devices I don't think it's required, but it could help. Or > > maybe your code does this at the MTD level. I dunno. I could be > > spouting nonsense again ;). > > Mmm, as long as writes are aligned on 2k pages and are linear / > unidirectional, I'm not sure anything else is really a problem > for USB flash drives .. > > > Yes, it is a Journaled _Flash_ File System. So of course it's written > > for MTD devices. Not sure why blkmtd.c was originally written. Test > > vehicle for JFFS2, "why not", "I'm bored", etc. Who knows? :) > > Mm, my point exactly , Journaled _Flash_ File System - as opposed to > Journaled _MTD_ _Flash_ File System .. !! > > USB flash is still flash! - just presented via a generic interface as > opposed to an embedded one .. (AFAIK) > > I guess I was sort of hoping blkmtd.c was written to enable JFFS2 to be > more generic and embedded device independent. I suspect however it was > written purely for testing purposes .. :( > > As I understand it, the Flash in USB keys is identical to the MTD type > embedded devices, except that the USB keys always come with > micro-controllers that handle and optimise read/write/erase operations > and present them via a generic PC/USB based disk interface. I'm not sure > how fast MTD can be driven, but key manufacturers seem to think that > keys should be able to run at 20Mb/sec read and 10Mb/sec write [for > large block read/writes] which is much faster than is required for a key > to replace a hard disk in a workstation. > > I'm thinking that JFFS2 could draw in a huge additional user base if it > catered for (or at least supported) such devices .. (! My understanding was that it is only seen as SCSI block device from outside. E.g. it is not possible, say, to erase sector, to write one page, etc. If so, just foget about JFFS2 on that, IMHO. > > Gareth. > > > ______________________________________________________ > Linux MTD discussion mailing list > http://lists.infradead.org/mailman/listinfo/linux-mtd/ > -- Best Regards, Artem B. Bityuckiy, St.-Petersburg, Russia. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-17 16:46 ` Gareth Bult (Encryptec) 2004-12-17 17:08 ` Artem B. Bityuckiy @ 2004-12-17 17:10 ` Josh Boyer 2004-12-17 17:26 ` Gareth Bult (Encryptec) 1 sibling, 1 reply; 82+ messages in thread From: Josh Boyer @ 2004-12-17 17:10 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD Gareth Bult (Encryptec) wrote: > > Mm, my point exactly , Journaled _Flash_ File System - as opposed to > Journaled _MTD_ _Flash_ File System .. !! > > USB flash is still flash! - just presented via a generic interface as > opposed to an embedded one .. (AFAIK) No, it's really not. There is no access to the low level flash, so MTD cannot directly drive it. Same principle as the Compact Flash devices, which MTD doesn't deal with either (other than via blkmtd). > > I guess I was sort of hoping blkmtd.c was written to enable JFFS2 to be > more generic and embedded device independent. I suspect however it was > written purely for testing purposes .. :( Yes, most likely. > > As I understand it, the Flash in USB keys is identical to the MTD type > embedded devices, except that the USB keys always come with > micro-controllers that handle and optimise read/write/erase operations > and present them via a generic PC/USB based disk interface. I'm not sure > how fast MTD can be driven, but key manufacturers seem to think that > keys should be able to run at 20Mb/sec read and 10Mb/sec write [for > large block read/writes] which is much faster than is required for a key > to replace a hard disk in a workstation. The flash used in the devices is identical, but MTD has no access to it. > > I'm thinking that JFFS2 could draw in a huge additional user base if it > catered for (or at least supported) such devices .. (!) That's the problem. There is no filesystem for these "abominations" really. They pretend to be IDE, SCSI, etc. but are really flash underneath. Neither the MTD driver or the IDE drivers (for example) deal with them really well, but I'm not sure that they can. Did you try ext3 on the device with a native driver (IDE, SCSI, USB specific) instead of via blkmtd.c? josh ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-17 17:10 ` Josh Boyer @ 2004-12-17 17:26 ` Gareth Bult (Encryptec) 2004-12-17 17:35 ` Josh Boyer 0 siblings, 1 reply; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-17 17:26 UTC (permalink / raw) To: Josh Boyer; +Cc: Linux MTD On Fri, 2004-12-17 at 11:10 -0600, Josh Boyer wrote: > That's the problem. There is no filesystem for these "abominations" > really. They pretend to be IDE, SCSI, etc. but are really flash > underneath. Neither the MTD driver or the IDE drivers (for example) > deal with them really well, but I'm not sure that they can. > > Did you try ext3 on the device with a native driver (IDE, SCSI, USB > specific) instead of via blkmtd.c? Sure, ext2 and ext3 work just fine .. however .. as far as I'm aware, although ext3 is log structured, it's not optimised to run in a linear fashion and doesn't worry about repeatedly writing to the same block .. which is likely to cause the USB flash controller to execute an erase prior to write .. which will quickly wear down the device [?!] (Am I wrong? in which case why isn't everyone using ext3 instead of JFFS?) The other requirement of course is inline compression, which last time I looked ext3 didn't do out of the box (?) I'm going to deliberately wear down some blocks on a key a little later (at block level) to see what the OS makes of it ... (and whether it takes 100,000 or 1,000,000 cycles .. I see conflicting specs .. I guess the basic spec for a USB flash key (you may call them abominations, but you should see the sales stats!) would be; a. purely log structured fs with linear / circular operation b. write blocks aligned to 2k (or 4k might be smarter to fit with buffer page sizes) c. erase blocks @ 128k d. inline compression Strikes me JFFS2 does all this, possibly bar the generic device support .. Gareth. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-17 17:26 ` Gareth Bult (Encryptec) @ 2004-12-17 17:35 ` Josh Boyer 2004-12-17 18:09 ` Gareth Bult (Encryptec) 2004-12-18 16:02 ` Jörn Engel 0 siblings, 2 replies; 82+ messages in thread From: Josh Boyer @ 2004-12-17 17:35 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD Gareth Bult (Encryptec) wrote: > > > Sure, ext2 and ext3 work just fine .. however .. as far as I'm aware, > although ext3 is log structured, it's not optimised to run in a linear > fashion and doesn't worry about repeatedly writing to the same block .. > which is likely to cause the USB flash controller to execute an erase > prior to write .. which will quickly wear down the device [?!] > > (Am I wrong? in which case why isn't everyone using ext3 instead of > JFFS?) Most of the controllers for these devices do the wear leveling themselves. At least that was my understanding. We don't deal much with these devices, so I'm not sure what everyone else uses. Alot of the CF devices are FAT formatted to work with Windows, etc. > > The other requirement of course is inline compression, which last time I > looked ext3 didn't do out of the box (?) No, it doesn't. There are patches for ext2 to make it use compression (e2compr), but then you lose the journaling. > > I'm going to deliberately wear down some blocks on a key a little later > (at block level) to see what the OS makes of it ... (and whether it > takes 100,000 or 1,000,000 cycles .. I see conflicting specs .. That probably depends on the actual flash chips used within the device. Not all of them use the same type of flash. > > I guess the basic spec for a USB flash key (you may call them > abominations, but you should see the sales stats!) would be; Hey, they certainly are handy. I'm not arguing with that :). > > a. purely log structured fs with linear / circular operation > b. write blocks aligned to 2k (or 4k might be smarter to fit with buffer > page sizes) > c. erase blocks @ 128k > d. inline compression Again, most controllers do all the wear leveling, erasing for you. So you are left with compression, right? And that is certainly a lacking thing. A block-based, writeable, compressed filesystem that does journalling. Sounds like a good opportunity to me :). josh ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-17 17:35 ` Josh Boyer @ 2004-12-17 18:09 ` Gareth Bult (Encryptec) 2004-12-17 19:14 ` jasmine 2004-12-18 16:02 ` Jörn Engel 1 sibling, 1 reply; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-17 18:09 UTC (permalink / raw) To: Josh Boyer; +Cc: Linux MTD On Fri, 2004-12-17 at 11:35 -0600, Josh Boyer wrote: > Gareth Bult (Encryptec) wrote: > Again, most controllers do all the wear leveling, erasing for you. So > you are left with compression, right? > > And that is certainly a lacking thing. A block-based, writeable, > compressed filesystem that does journalling. Sounds like a good > opportunity to me :). Mmm, sounds a bit like JFFS2 on a block emulator , yeah ? ;-) Gareth. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-17 18:09 ` Gareth Bult (Encryptec) @ 2004-12-17 19:14 ` jasmine 2004-12-17 20:55 ` Gareth Bult (Encryptec) 0 siblings, 1 reply; 82+ messages in thread From: jasmine @ 2004-12-17 19:14 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD On Fri, 17 Dec 2004, Gareth Bult (Encryptec) wrote: > Mmm, sounds a bit like JFFS2 on a block emulator , yeah ? Only if you actively enjoy wasting processor cycles (and microjoules) and ROM footprint... -J. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-17 19:14 ` jasmine @ 2004-12-17 20:55 ` Gareth Bult (Encryptec) 0 siblings, 0 replies; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-17 20:55 UTC (permalink / raw) To: jasmine; +Cc: Linux MTD On Fri, 2004-12-17 at 19:14 +0000, jasmine@linuxgrrls.org wrote: > On Fri, 17 Dec 2004, Gareth Bult (Encryptec) wrote: > > > Mmm, sounds a bit like JFFS2 on a block emulator , yeah ? > > Only if you actively enjoy wasting processor cycles (and microjoules) and > ROM footprint... Processor cycles, got em to spare .. Microjoules coming out of my ears .. ROM footprint .. don't use 'em .. What's the alternative then .. :) Gareth. ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-17 17:35 ` Josh Boyer 2004-12-17 18:09 ` Gareth Bult (Encryptec) @ 2004-12-18 16:02 ` Jörn Engel 2004-12-20 16:34 ` Josh Boyer 1 sibling, 1 reply; 82+ messages in thread From: Jörn Engel @ 2004-12-18 16:02 UTC (permalink / raw) To: Josh Boyer; +Cc: Linux MTD, Gareth Bult (Encryptec) On Fri, 17 December 2004 11:35:04 -0600, Josh Boyer wrote: > > Again, most controllers do all the wear leveling, erasing for you. So > you are left with compression, right? If you have enough trust in your El Cheapo hardware supplier. There is no standard that requires wear leveling and there are reports that basically tell you to do your own wear leveling on some devices. With no good way to distinguish "some" devices from "others", you can read that as "all". ;) USB keys and all the various other consumer-type flash devices are good enough to beat 3 1/2" floppies, that's about it. But that's quite good enough for most people. Floppies sold like crazy after all. Jörn -- Simplicity is prerequisite for reliability. -- Edsger W. Dijkstra ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-18 16:02 ` Jörn Engel @ 2004-12-20 16:34 ` Josh Boyer 2004-12-20 17:12 ` Gareth Bult (Encryptec) 0 siblings, 1 reply; 82+ messages in thread From: Josh Boyer @ 2004-12-20 16:34 UTC (permalink / raw) To: Jörn Engel; +Cc: Linux MTD, Gareth Bult (Encryptec) On Sat, 2004-12-18 at 10:02, Jörn Engel wrote: > On Fri, 17 December 2004 11:35:04 -0600, Josh Boyer wrote: > > > > Again, most controllers do all the wear leveling, erasing for you. So > > you are left with compression, right? > > If you have enough trust in your El Cheapo hardware supplier. There > is no standard that requires wear leveling and there are reports that > basically tell you to do your own wear leveling on some devices. With > no good way to distinguish "some" devices from "others", you can read > that as "all". ;) Ok, that's why I said "most". And if your El Cheapo hardware has built in wear leveling, then doing wear leveling on top of that is always questionable. It might not make things worse, but it's probably not very efficient. Now if you have El Super Cheapo hardware that explicitly states you need to do wear leveling, that's a different story ;). > > USB keys and all the various other consumer-type flash devices are > good enough to beat 3 1/2" floppies, that's about it. But that's > quite good enough for most people. Floppies sold like crazy after > all. That's because it was the only removable media for PCs for a long time. At least to the non-geeks. I can remember buying spare hard drives and carrying those from machine to machine if I needed to do big transfers ;). josh ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-20 16:34 ` Josh Boyer @ 2004-12-20 17:12 ` Gareth Bult (Encryptec) 2004-12-21 13:09 ` Jörn Engel 0 siblings, 1 reply; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-20 17:12 UTC (permalink / raw) To: Josh Boyer; +Cc: Linux MTD On Mon, 2004-12-20 at 10:34 -0600, Josh Boyer wrote: > Ok, that's why I said "most". And if your El Cheapo hardware has built > in wear leveling, then doing wear leveling on top of that is always > questionable. It might not make things worse, but it's probably not > very efficient. Mmm, if the USB flash micro-controllers have build-in wear levelling, they'd be quite something .. (!) > Now if you have El Super Cheapo hardware that explicitly states you need > to do wear leveling, that's a different story ;). Here's a quote from a "Kingston" advert; "Controllers automatically lock out bad memory cells and move the data to avoid corruption. Controllers also automatically distribute write cycles across the flash cells to extend the life of the flash memory card" What this means exactly in real terms .. [?] (for example, do you lose some of your key to a bad block table, if so how much ???) Here's a bit from "Verbatim"; "For ultra-reliability, Verbatim’s new Store ‘n’ Go Pro drives feature an on-board 32 bit ARM-7 microprocessor that manages I/O operations and many of the drive’s technical features. An advanced wear-leveling algorithm is used to distribute writes evenly among flash storage cells. By evenly distributing the writing, Store ‘n’ Go Pro achieves an unprecedented reliability rating of over one million write/erase cycles. Advanced Error Detection Code/Error Correction Code furthers the Store ‘n’ Go Pro’s immunity to failure." Interesting articles / links; http://www.digitalmediadesigner.com/articles/viewarticle.jsp?id=27727 http://www.techworld.com/features/index.cfm?fuseaction=displayfeature&FeatureID=413 In summary: lots of manufacturers seem to be quoting virtual / physical block mapping on the fly with integrated wear levelling ... (!) So maybe wear levelling is out of date at filesystem level ? The techworld link to the M-Systems chips looks interesting .. Can anyone with detailed flash experience comment ? Gareth. > That's because it was the only removable media for PCs for a long time. > At least to the non-geeks. I can remember buying spare hard drives and > carrying those from machine to machine if I needed to do big transfers > ;). > > josh > ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-20 17:12 ` Gareth Bult (Encryptec) @ 2004-12-21 13:09 ` Jörn Engel 2004-12-21 13:24 ` Gareth Bult (Encryptec) 0 siblings, 1 reply; 82+ messages in thread From: Jörn Engel @ 2004-12-21 13:09 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD On Mon, 20 December 2004 17:12:28 +0000, Gareth Bult (Encryptec) wrote: > On Mon, 2004-12-20 at 10:34 -0600, Josh Boyer wrote: > > Ok, that's why I said "most". And if your El Cheapo hardware has built > > in wear leveling, then doing wear leveling on top of that is always > > questionable. It might not make things worse, but it's probably not > > very efficient. Imo jffs2-style wear leveling is very efficient. So efficient, it doesn't really matter whether the hardware does the same or not. Difference between no wear leveling and what jffs2 does is - at most - 1% additional writes on the jffs2 side. That's not much. Agreed, it can actually cost latency, but from a life-time or overall performance point of view, it doesn't really matter. On the other hand, it gives you absolute condidence that your El Cheapo hardware doesn't need to have anything it may or may not promise to have. No nasty surprised when you really don't need them. > > Now if you have El Super Cheapo hardware that explicitly states you need > > to do wear leveling, that's a different story ;). Noone will state that. It's not exactly good marketing. You might remember so interesting stories about what marketing promised and what actually got delivered, no? > In summary: lots of manufacturers seem to be quoting virtual / physical > block mapping on the fly with integrated wear levelling ... (!) > > So maybe wear levelling is out of date at filesystem level ? > The techworld link to the M-Systems chips looks interesting .. > > Can anyone with detailed flash experience comment ? The endorsed features sound quite nice, agreed. But I have still to read a spec that convinces me. Jörn -- My second remark is that our intellectual powers are rather geared to master static relations and that our powers to visualize processes evolving in time are relatively poorly developed. -- Edsger W. Dijkstra ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-21 13:09 ` Jörn Engel @ 2004-12-21 13:24 ` Gareth Bult (Encryptec) 2004-12-21 13:34 ` Jörn Engel 0 siblings, 1 reply; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-21 13:24 UTC (permalink / raw) To: Jörn Engel; +Cc: Linux MTD [-- Attachment #1: Type: text/plain, Size: 1335 bytes --] Hi, On Tue, 2004-12-21 at 14:09 +0100, Jörn Engel wrote: > Imo jffs2-style wear leveling is very efficient. So efficient, it > doesn't really matter whether the hardware does the same or not. :) I was of the same opinion but didn't really have any evidence to back it up .. > Difference between no wear leveling and what jffs2 does is - at most - > 1% additional writes on the jffs2 side. That's not much. Agreed, it > can actually cost latency, but from a life-time or overall performance > point of view, it doesn't really matter. Sure. > On the other hand, it gives you absolute condidence that your El > Cheapo hardware doesn't need to have anything it may or may not > promise to have. No nasty surprised when you really don't need them. :) > Noone will state that. It's not exactly good marketing. You might > remember so interesting stories about what marketing promised and what > actually got delivered, no? Sure, although I have noticed that some people shout more loudly than others with regards to the wear levelling technology in their devices .. > The endorsed features sound quite nice, agreed. But I have still to > read a spec that convinces me. Mmm .. anyone know a magazine editor who'd like to acquire a bunch of keys and runs some tests for us .. ;-) Gareth. [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-21 13:24 ` Gareth Bult (Encryptec) @ 2004-12-21 13:34 ` Jörn Engel 0 siblings, 0 replies; 82+ messages in thread From: Jörn Engel @ 2004-12-21 13:34 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD On Tue, 21 December 2004 13:24:35 +0000, Gareth Bult (Encryptec) wrote: > > Mmm .. anyone know a magazine editor who'd like to acquire a bunch of > keys and runs some tests for us .. ;-) Somewhere in my lkml-mailbox was a report of some device not doing decent wear leveling. It should be in the archives somewhere. Ext3 killed the devices rather quickly, jffs2 didn't. Jörn -- Data dominates. If you've chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data structures, not algorithms, are central to programming. -- Rob Pike ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-16 21:47 ` Gareth Bult (Encryptec) 2004-12-17 12:54 ` Josh Boyer @ 2004-12-18 16:19 ` Jörn Engel 2004-12-18 17:32 ` Gareth Bult (Encryptec) 1 sibling, 1 reply; 82+ messages in thread From: Jörn Engel @ 2004-12-18 16:19 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD On Thu, 16 December 2004 21:47:49 +0000, Gareth Bult (Encryptec) wrote: > > blkmtd.c is pretty much a rewrite, less than half the size and uses the > system buffer cache instead of reading / writing directly to the device. > This boosts performance x 3 on read x lots on write. [I get x 100 > faster] (albeit you lose bad block detection on write) > > If anyone's interested in the code, let me know and I'll clean it up and > post it .. however it'll be a new file and not a patch. Sure am! If the code lives up to your promises I'd like to read it (might learn something new) and possibly use it sometime soon. Jörn -- The competent programmer is fully aware of the strictly limited size of his own skull; therefore he approaches the programming task in full humility, and among other things he avoids clever tricks like the plague. -- Edsger W. Dijkstra ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-18 16:19 ` Jörn Engel @ 2004-12-18 17:32 ` Gareth Bult (Encryptec) 2004-12-18 17:52 ` Jörn Engel 0 siblings, 1 reply; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-18 17:32 UTC (permalink / raw) To: Jörn Engel; +Cc: Linux MTD [-- Attachment #1: Type: text/plain, Size: 1098 bytes --] Ok, There's a public health warning attached to this code .. I've not cleaned it up and I've not added any comments .. (!) fyi; I don't use mkfs.jffs2 .. I flash_eraseall the device then work from there with "tar".. there seems to be no benefit [for me] to using mkfs and it seems a little more reliant on the device size than the erase program. Gareth. On Sat, 2004-12-18 at 17:19 +0100, Jörn Engel wrote: > On Thu, 16 December 2004 21:47:49 +0000, Gareth Bult (Encryptec) wrote: > > > > blkmtd.c is pretty much a rewrite, less than half the size and uses the > > system buffer cache instead of reading / writing directly to the device. > > This boosts performance x 3 on read x lots on write. [I get x 100 > > faster] (albeit you lose bad block detection on write) > > > > If anyone's interested in the code, let me know and I'll clean it up and > > post it .. however it'll be a new file and not a patch. > > Sure am! If the code lives up to your promises I'd like to read it > (might learn something new) and possibly use it sometime soon. > > Jörn > [-- Attachment #2: blkmtd.c --] [-- Type: text/x-csrc, Size: 16190 bytes --] /* * $Id: blkmtd.c,v 1.23 2004/08/09 14:03:19 dwmw2 Exp $ * * blkmtd.c - use a block device as a fake MTD * * Author: Simon Evans <spse@secret.org.uk> * * Copyright (C) 2001,2002 Simon Evans * * Licence: GPL * * How it works: * The driver uses raw/io to read/write the device and the page * cache to cache access. Writes update the page cache with the * new data and mark it dirty and add the page into a BIO which * is then written out. * * It can be loaded Read-Only to prevent erases and writes to the * medium. * */ #include <linux/config.h> #include <linux/module.h> #include <linux/fs.h> #include <linux/blkdev.h> #include <linux/bio.h> #include <linux/pagemap.h> #include <linux/list.h> #include <linux/init.h> #include <linux/mtd/mtd.h> #include <linux/buffer_head.h> #define err(format, arg...) printk(KERN_ERR "blkmtd: " format "\n" , ## arg) #define info(format, arg...) printk(KERN_INFO "blkmtd: " format "\n" , ## arg) #define warn(format, arg...) printk(KERN_WARNING "blkmtd: " format "\n" , ## arg) #define crit(format, arg...) printk(KERN_CRIT "blkmtd: " format "\n" , ## arg) /* Default erase size in K, always make it a multiple of PAGE_SIZE */ #define CONFIG_MTD_BLKDEV_ERASESIZE (128 << 10) /* 128KiB */ #define VERSION "$Revision: 1.23 $" /* Info for the block device */ struct blkmtd_dev { struct list_head list; struct block_device *blkdev; struct mtd_info mtd_info; struct semaphore wrbuf_mutex; }; /* Static info about the MTD, used in cleanup_module */ static LIST_HEAD(blkmtd_device_list); static void blkmtd_sync(struct mtd_info *mtd); #define MAX_DEVICES 4 /* Module parameters passed by insmod/modprobe */ char *device[MAX_DEVICES]; /* the block device to use */ int erasesz[MAX_DEVICES]; /* optional default erase size */ int ro[MAX_DEVICES]; /* optional read only flag */ int sync; MODULE_LICENSE("GPL"); MODULE_AUTHOR("Simon Evans <spse@secret.org.uk>"); MODULE_DESCRIPTION("Emulate an MTD using a block device"); MODULE_PARM(device, "1-4s"); MODULE_PARM_DESC(device, "block device to use"); MODULE_PARM(erasesz, "1-4i"); MODULE_PARM_DESC(erasesz, "optional erase size to use in KiB. eg 4=4KiB."); MODULE_PARM(ro, "1-4i"); MODULE_PARM_DESC(ro, "1=Read only, writes and erases cause errors"); MODULE_PARM(sync, "i"); MODULE_PARM_DESC(sync, "1=Synchronous writes"); #define PAGE_READAHEAD 64 void cache_readahead(struct address_space *mapping, int index) { filler_t *filler = (filler_t*)(mapping->a_ops->readpage); int i,pagei; unsigned ret = 0; unsigned long end_index; struct page *page; LIST_HEAD(page_pool); struct inode *inode = mapping->host; loff_t isize = i_size_read(inode); if(!isize) { printk(KERN_INFO "iSize=0 in cache_readahead\n"); return; } end_index = ((isize - 1) >> PAGE_CACHE_SHIFT); spin_lock_irq(&mapping->tree_lock); for(i=0;i<PAGE_READAHEAD;i++) { pagei=index+i; if (pagei > end_index) { printk(KERN_INFO "Overrun end of disk in cache readahead\n"); break; } page = radix_tree_lookup(&mapping->page_tree, pagei); if(page && (!i)) break; if(page) continue; spin_unlock_irq(&mapping->tree_lock); page = page_cache_alloc_cold(mapping); spin_lock_irq(&mapping->tree_lock); if(!page) break; page->index=pagei; list_add(&page->lru, &page_pool); ret++; } spin_unlock_irq(&mapping->tree_lock); if(ret) { read_cache_pages(mapping, &page_pool, filler, NULL); //printk(KERN_INFO "Readahead [%d/%d] pages from [%d]\n",ret,PAGE_READAHEAD,index); } } static struct page* page_readahead(struct address_space *mapping, int index) { filler_t *filler = (filler_t*)(mapping->a_ops->readpage); cache_readahead(mapping,index); return read_cache_page(mapping, index, filler, NULL); } /** * write_pages - write block of data to device via the page cache * @dev: device to write to * @buf: data source or NULL if erase (output is set to 0xff) * @to: offset into output device * @len: amount to data to write * @retlen: amount of data written * * Grab pages from the page cache and fill them with the source data. * Non page aligned start and end result in a readin of the page and * part of the page being modified. Pages are added to the bio and then written * out. */ static int write_pages(struct blkmtd_dev *dev, const u_char *buf, loff_t to, size_t len, size_t *retlen) { struct page *page; struct address_space *mapping = dev->blkdev->bd_inode->i_mapping; // int err = 0; // return status int index = to >> PAGE_SHIFT; // page index int offset = to & ~PAGE_MASK; // page offset // static int my_read() { err=0; page = page_readahead(mapping,index); if( unlikely(!page) ) { crit("unable to read page (%d) in write_pages\n",index); err=-ENOMEM; } //if(IS_ERR(page)) err=-EIO; if(err) printk("READ ERROR\n"); return err; } // static int my_erase() { int pages = len >> PAGE_SHIFT; unsigned char *p; unsigned char *max; int format; while(pages) { //printk(KERN_INFO "readahead on index (%d)\n",index); if(my_read()) return err; max = ((char*)page_address(page)+PAGE_SIZE); format = 0; for(p=(unsigned char*)page_address(page); p<max; p++) if(*p != 0xff) { format=1; break; } if(format) { //DEBUG(3, "write_pages: erasing\n"); lock_page(page); memset(page_address(page), 0xff, PAGE_SIZE); set_page_dirty(page); unlock_page(page); } page_cache_release(page); pages--; index++; } *retlen = len; return 0; } static int my_write() { int cpylen; if(retlen) *retlen = 0; //printk(KERN_INFO "Write req: %d\n",len); while(len) { if((offset+len) > PAGE_SIZE) cpylen = PAGE_SIZE - offset; // multiple pages else cpylen = len; // this page len = len - cpylen; // // Get page // if(my_read()) return err; // if(memcmp(page_address(page)+offset,buf,cpylen)) { lock_page(page); memcpy(page_address(page) + offset,buf, cpylen); set_page_dirty(page); unlock_page(page); } page_cache_release(page); if(retlen) *retlen += cpylen; // buf+=cpylen; offset = 0; index++; // } //printk(KERN_INFO "Write act: %d\n",*retlen); return 0; } err=0; down(&dev->wrbuf_mutex); err = buf ? my_write() : my_erase(); up(&dev->wrbuf_mutex); return err; } /* erase a specified part of the device */ static int blkmtd_erase(struct mtd_info *mtd, struct erase_info *instr) { struct blkmtd_dev *dev = mtd->priv; struct mtd_erase_region_info *einfo = mtd->eraseregions; int numregions = mtd->numeraseregions; size_t from; u_long len; int err = -EIO; size_t retlen; instr->state = MTD_ERASING; from = instr->addr; len = instr->len; /* check erase region has valid start and length */ //DEBUG(2, "blkmtd: erase: dev = `%s' from = 0x%zx len = 0x%lx\n", mtd->name+9, from, len); while(numregions) { DEBUG(3, "blkmtd: checking erase region = 0x%08X size = 0x%X num = 0x%x\n", einfo->offset, einfo->erasesize, einfo->numblocks); if(from >= einfo->offset && from < einfo->offset + (einfo->erasesize * einfo->numblocks)) { if(len == einfo->erasesize && ( (from - einfo->offset) % einfo->erasesize == 0)) break; } numregions--; einfo++; } if(!numregions) { /* Not a valid erase block */ err("erase: invalid erase request 0x%lX @ 0x%08zX", len, from); instr->state = MTD_ERASE_FAILED; err = -EIO; } if(instr->state != MTD_ERASE_FAILED) { /* do the erase */ DEBUG(3, "Doing erase from = %zd len = %ld\n", from, len); err = write_pages(dev, NULL, from, len, &retlen); if(err || retlen != len) { err("erase failed err = %d", err); instr->state = MTD_ERASE_FAILED; } else { instr->state = MTD_ERASE_DONE; } } DEBUG(3, "blkmtd: erase: checking callback\n"); mtd_erase_callback(instr); DEBUG(2, "blkmtd: erase: finished (err = %d)\n", err); return err; } static int blkmtd_read(struct mtd_info *mtd, loff_t from, size_t len, size_t *retlen, u_char *buf) { struct blkmtd_dev *dev = mtd->priv; struct page *page; // int index = from >> PAGE_SHIFT; int offset = from - (index << PAGE_SHIFT); int cpylen; if(from > mtd->size) return -EINVAL; if(from + len > mtd->size) len = mtd->size - from; if(retlen) *retlen = 0; while(len) { if((offset+len) > PAGE_SIZE) cpylen = PAGE_SIZE - offset; // multiple pages else cpylen = len; // this page len = len - cpylen; // // Get page // page = page_readahead(dev->blkdev->bd_inode->i_mapping,index); if(!page) return -ENOMEM; if(IS_ERR(page)) return -EIO; // memcpy(buf, page_address(page) + offset, cpylen); page_cache_release(page); // if(retlen) *retlen += cpylen; buf+=cpylen; offset = 0; index++; // } return 0; } /* write data to the underlying device */ static int blkmtd_write(struct mtd_info *mtd, loff_t to, size_t len, size_t *retlen, const u_char *buf) { struct blkmtd_dev *dev = mtd->priv; int err; if(!len) return 0; if(to >= mtd->size) return -ENOSPC; if(to + len > mtd->size) len = mtd->size - to; err = write_pages(dev, buf, to, len, retlen); if(err > 0) err = 0; return err; } /* sync the device - wait until the write queue is empty */ static void blkmtd_sync(struct mtd_info *mtd) { sync_blockdev(((struct blkmtd_dev*)mtd->priv)->blkdev); return; } static void free_device(struct blkmtd_dev *dev) { DEBUG(2, "blkmtd: free_device() dev = %p\n", dev); if(dev) { if(dev->mtd_info.eraseregions) kfree(dev->mtd_info.eraseregions); if(dev->mtd_info.name) kfree(dev->mtd_info.name); if(dev->blkdev) { invalidate_inode_pages(dev->blkdev->bd_inode->i_mapping); close_bdev_excl(dev->blkdev); } kfree(dev); } } /* For a given size and initial erase size, calculate the number * and size of each erase region. Goes round the loop twice, * once to find out how many regions, then allocates space, * then round the loop again to fill it in. */ static struct mtd_erase_region_info *calc_erase_regions( size_t erase_size, size_t total_size, int *regions) { struct mtd_erase_region_info *info = NULL; DEBUG(2, "calc_erase_regions, es = %zd size = %zd regions = %d\n", erase_size, total_size, *regions); /* Make any user specified erasesize be a power of 2 and at least PAGE_SIZE */ if(erase_size) { int es = erase_size; erase_size = 1; while(es != 1) { es >>= 1; erase_size <<= 1; } if(erase_size < PAGE_SIZE) erase_size = PAGE_SIZE; } else { erase_size = CONFIG_MTD_BLKDEV_ERASESIZE; } *regions = 0; do { int tot_size = total_size; int er_size = erase_size; int count = 0, offset = 0, regcnt = 0; while(tot_size) { count = tot_size / er_size; if(count) { tot_size = tot_size % er_size; if(info) { DEBUG(2, "adding to erase info off=%d er=%d cnt=%d\n", offset, er_size, count); (info+regcnt)->offset = offset; (info+regcnt)->erasesize = er_size; (info+regcnt)->numblocks = count; (*regions)++; } regcnt++; offset += (count * er_size); } while(er_size > tot_size) er_size >>= 1; } if(info == NULL) { info = kmalloc(regcnt * sizeof(struct mtd_erase_region_info), GFP_KERNEL); if(!info) break; } } while(!(*regions)); DEBUG(2, "calc_erase_regions done, es = %zd size = %zd regions = %d\n", erase_size, total_size, *regions); return info; } extern dev_t __init name_to_dev_t(const char *line); static struct blkmtd_dev *add_device(char *devname, int readonly, int erase_size) { struct block_device *bdev; int mode; struct blkmtd_dev *dev; if(!devname) return NULL; /* Get a handle on the device */ #ifdef MODULE mode = (readonly) ? O_RDONLY : O_RDWR; bdev = open_bdev_excl(devname, mode, NULL); #else mode = (readonly) ? FMODE_READ : FMODE_WRITE; bdev = open_by_devnum(name_to_dev_t(devname), mode); #endif if(IS_ERR(bdev)) { err("error: cannot open device %s", devname); DEBUG(2, "blkmtd: opening bdev returned %ld\n", PTR_ERR(bdev)); return NULL; } DEBUG(1, "blkmtd: found a block device major = %d, minor = %d\n", MAJOR(bdev->bd_dev), MINOR(bdev->bd_dev)); if(MAJOR(bdev->bd_dev) == MTD_BLOCK_MAJOR) { err("attempting to use an MTD device as a block device"); blkdev_put(bdev); return NULL; } dev = kmalloc(sizeof(struct blkmtd_dev), GFP_KERNEL); if(dev == NULL) { blkdev_put(bdev); return NULL; } memset(dev, 0, sizeof(struct blkmtd_dev)); dev->blkdev = bdev; atomic_set(&(dev->blkdev->bd_inode->i_mapping->truncate_count), 0); if(!readonly) { init_MUTEX(&dev->wrbuf_mutex); } dev->mtd_info.size = dev->blkdev->bd_inode->i_size & PAGE_MASK; /* Setup the MTD structure */ /* make the name contain the block device in */ dev->mtd_info.name = kmalloc(sizeof("blkmtd: ") + strlen(devname), GFP_KERNEL); if(dev->mtd_info.name == NULL) goto devinit_err; sprintf(dev->mtd_info.name, "blkmtd: %s", devname); dev->mtd_info.eraseregions = calc_erase_regions(erase_size, dev->mtd_info.size, &dev->mtd_info.numeraseregions); if(dev->mtd_info.eraseregions == NULL) goto devinit_err; dev->mtd_info.erasesize = dev->mtd_info.eraseregions->erasesize; DEBUG(1, "blkmtd: init: found %d erase regions\n", dev->mtd_info.numeraseregions); if(readonly) { dev->mtd_info.type = MTD_ROM; dev->mtd_info.flags = MTD_CAP_ROM; } else { dev->mtd_info.type = MTD_RAM; dev->mtd_info.flags = MTD_CAP_RAM; dev->mtd_info.erase = blkmtd_erase; dev->mtd_info.write = blkmtd_write; dev->mtd_info.writev = default_mtd_writev; dev->mtd_info.sync = blkmtd_sync; } dev->mtd_info.read = blkmtd_read; dev->mtd_info.readv = default_mtd_readv; dev->mtd_info.priv = dev; dev->mtd_info.owner = THIS_MODULE; list_add(&dev->list, &blkmtd_device_list); if (add_mtd_device(&dev->mtd_info)) { /* Device didnt get added, so free the entry */ list_del(&dev->list); goto devinit_err; } else { info("mtd%d: [%s] erase_size = %dKiB %s [%ld]", dev->mtd_info.index, dev->mtd_info.name + strlen("blkmtd: "), dev->mtd_info.erasesize >> 10, readonly ? "(read-only)" : "", PAGE_SIZE); } return dev; devinit_err: free_device(dev); return NULL; } /* Cleanup and exit - sync the device and kill of the kernel thread */ static void __devexit cleanup_blkmtd(void) { struct list_head *temp1, *temp2; /* Remove the MTD devices */ list_for_each_safe(temp1, temp2, &blkmtd_device_list) { struct blkmtd_dev *dev = list_entry(temp1, struct blkmtd_dev, list); blkmtd_sync(&dev->mtd_info); del_mtd_device(&dev->mtd_info); info("mtd%d: [%s] removed", dev->mtd_info.index, dev->mtd_info.name + strlen("blkmtd: ")); list_del(&dev->list); free_device(dev); } } #ifndef MODULE /* Handle kernel boot params */ static int __init param_blkmtd_device(char *str) { int i; for(i = 0; i < MAX_DEVICES; i++) { device[i] = str; DEBUG(2, "blkmtd: device setup: %d = %s\n", i, device[i]); strsep(&str, ","); } return 1; } static int __init param_blkmtd_erasesz(char *str) { int i; for(i = 0; i < MAX_DEVICES; i++) { char *val = strsep(&str, ","); if(val) erasesz[i] = simple_strtoul(val, NULL, 0); DEBUG(2, "blkmtd: erasesz setup: %d = %d\n", i, erasesz[i]); } return 1; } static int __init param_blkmtd_ro(char *str) { int i; for(i = 0; i < MAX_DEVICES; i++) { char *val = strsep(&str, ","); if(val) ro[i] = simple_strtoul(val, NULL, 0); DEBUG(2, "blkmtd: ro setup: %d = %d\n", i, ro[i]); } return 1; } static int __init param_blkmtd_sync(char *str) { if(str[0] == '1') sync = 1; return 1; } __setup("blkmtd_device=", param_blkmtd_device); __setup("blkmtd_erasesz=", param_blkmtd_erasesz); __setup("blkmtd_ro=", param_blkmtd_ro); __setup("blkmtd_sync=", param_blkmtd_sync); #endif /* Startup */ static int __init init_blkmtd(void) { int i; info("version " VERSION); /* Check args - device[0] is the bare minimum*/ if(!device[0]) { err("error: missing `device' name\n"); return -EINVAL; } for(i = 0; i < MAX_DEVICES; i++) add_device(device[i], ro[i], erasesz[i] << 10); if(list_empty(&blkmtd_device_list)) return -EINVAL; return 0; } module_init(init_blkmtd); module_exit(cleanup_blkmtd); ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-18 17:32 ` Gareth Bult (Encryptec) @ 2004-12-18 17:52 ` Jörn Engel 2004-12-18 18:11 ` Jörn Engel 0 siblings, 1 reply; 82+ messages in thread From: Jörn Engel @ 2004-12-18 17:52 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD On Sat, 18 December 2004 17:32:35 +0000, Gareth Bult (Encryptec) wrote: > > There's a public health warning attached to this code .. > > I've not cleaned it up and I've not added any comments .. I'll wear protective gloves then. Thanks! > fyi; I don't use mkfs.jffs2 .. I flash_eraseall the device then work > from there with "tar".. there seems to be no benefit [for me] to using > mkfs and it seems a little more reliant on the device size than the > erase program. Makes a lot of sense. Both approaches have some advantages. Jörn -- The story so far: In the beginning the Universe was created. This has made a lot of people very angry and been widely regarded as a bad move. -- Douglas Adams? ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-18 17:52 ` Jörn Engel @ 2004-12-18 18:11 ` Jörn Engel 2004-12-18 20:48 ` Gareth Bult (Encryptec) 0 siblings, 1 reply; 82+ messages in thread From: Jörn Engel @ 2004-12-18 18:11 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD On Sat, 18 December 2004 18:52:22 +0100, Jörn Engel wrote: > On Sat, 18 December 2004 17:32:35 +0000, Gareth Bult (Encryptec) wrote: > > > > There's a public health warning attached to this code .. > > > > I've not cleaned it up and I've not added any comments .. > > I'll wear protective gloves then. Thanks! Not a pretty. Looks like quite a few cleanups from phram.c can be applied to this one as well. From a first glance: o Coding style (Lindent?) o Function length o #ifdef MODULE o parameters have an insane interface o ro-devices. the same can be achieved with chmod. But you started the work, which is a good thing. Jörn -- Public Domain - Free as in Beer General Public - Free as in Speech BSD License - Free as in Enterprise Shared Source - Free as in "Work will make you..." ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-18 18:11 ` Jörn Engel @ 2004-12-18 20:48 ` Gareth Bult (Encryptec) 2004-12-19 2:44 ` Jörn Engel 2004-12-21 13:30 ` Jörn Engel 0 siblings, 2 replies; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-18 20:48 UTC (permalink / raw) To: Jörn Engel; +Cc: Linux MTD Like I said, hacked to work [for me] and not to look pretty. To be honest what's there will do me for now .. although I'd like to clean it up a little when I get time. fyi; don't confuse the old and new code. > o Coding style (Lindent?) Whatever was there already. > o Function length My functions are shorter than the ones that were there, lots can be cleaned up here. > o #ifdef MODULE .. ? I only use it as a module, not of this code was touched. > o parameters have an insane interface .. again , parameters and interfaces are all from the original with the exception of a couple of new routines which mirror existing parameter lists. > o ro-devices. the same can be achieved with chmod. Again, not touched this. Gareth. On Sat, 2004-12-18 at 19:11 +0100, Jörn Engel wrote: > On Sat, 18 December 2004 18:52:22 +0100, Jörn Engel wrote: > > On Sat, 18 December 2004 17:32:35 +0000, Gareth Bult (Encryptec) wrote: > > > > > > There's a public health warning attached to this code .. > > > > > > I've not cleaned it up and I've not added any comments .. > > > > I'll wear protective gloves then. Thanks! > > Not a pretty. Looks like quite a few cleanups from phram.c can be > applied to this one as well. From a first glance: > o Coding style (Lindent?) > o Function length > o #ifdef MODULE > o parameters have an insane interface > o ro-devices. the same can be achieved with chmod. > > But you started the work, which is a good thing. > > Jörn > ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-18 20:48 ` Gareth Bult (Encryptec) @ 2004-12-19 2:44 ` Jörn Engel 2004-12-21 13:30 ` Jörn Engel 1 sibling, 0 replies; 82+ messages in thread From: Jörn Engel @ 2004-12-19 2:44 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD On Sat, 18 December 2004 20:48:50 +0000, Gareth Bult (Encryptec) wrote: > > Like I said, hacked to work [for me] and not to look pretty. > > To be honest what's there will do me for now .. although I'd like to > clean it up a little when I get time. > > fyi; don't confuse the old and new code. Sure. I had a look at tkdiff and there was ~50% unchanged code. "This code ain't pretty." didn't mean "You sh*thead are stupid and cannot create decent code." It was more like "It still ain't pretty, although you already improved it." > > o Coding style (Lindent?) > > Whatever was there already. > > > o Function length > > My functions are shorter than the ones that were there, lots can be > cleaned up here. > > > o #ifdef MODULE > > .. ? I only use it as a module, not of this code was touched. > > > o parameters have an insane interface > > .. again , parameters and interfaces are all from the original with the > exception of a couple of new routines which mirror existing parameter > lists. > > > o ro-devices. the same can be achieved with chmod. > > Again, not touched this. Yep. Those five point were the obvious stuff I could find in five minutes. Fixing them takes a lot more than five minutes and changing the parameter interface basically means copying the thing and fixing the new one. I did the same with slram/phram. The slram interface was different for module/non-module built and insane once you had more than a single device. Mtdblock looks quite similar and seems to need the same cure. So whatever you can improve, please do. Jörn -- It's just what we asked for, but not what we want! -- anonymous ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-18 20:48 ` Gareth Bult (Encryptec) 2004-12-19 2:44 ` Jörn Engel @ 2004-12-21 13:30 ` Jörn Engel 2004-12-21 13:40 ` Gareth Bult (Encryptec) 1 sibling, 1 reply; 82+ messages in thread From: Jörn Engel @ 2004-12-21 13:30 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD On Sat, 18 December 2004 20:48:50 +0000, Gareth Bult (Encryptec) wrote: > > To be honest what's there will do me for now .. although I'd like to > clean it up a little when I get time. Guess you've never been patch-bombed before. I took the liberty to do some untested cleanups in ~20 patches. Most likely the end result will not work anymore, but the direction is where I'd like to go. If you find the time, could you test the patches and see which work and which don't? The former will go into cvs then and I'll try to fix the latter. Jörn -- It does not matter how slowly you go, so long as you do not stop. -- Confucius ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-21 13:30 ` Jörn Engel @ 2004-12-21 13:40 ` Gareth Bult (Encryptec) 2004-12-21 15:00 ` David Woodhouse 0 siblings, 1 reply; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-21 13:40 UTC (permalink / raw) To: Jörn Engel; +Cc: Linux MTD [-- Attachment #1: Type: text/plain, Size: 686 bytes --] .. and I find the patches where ? On Tue, 2004-12-21 at 14:30 +0100, Jörn Engel wrote: > On Sat, 18 December 2004 20:48:50 +0000, Gareth Bult (Encryptec) wrote: > > > > To be honest what's there will do me for now .. although I'd like to > > clean it up a little when I get time. > > Guess you've never been patch-bombed before. I took the liberty to do > some untested cleanups in ~20 patches. Most likely the end result > will not work anymore, but the direction is where I'd like to go. > > If you find the time, could you test the patches and see which work > and which don't? The former will go into cvs then and I'll try to fix > the latter. > > Jörn > [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-21 13:40 ` Gareth Bult (Encryptec) @ 2004-12-21 15:00 ` David Woodhouse [not found] ` <1103644945.10792.175.camel@squizzey.bult.co.uk> 0 siblings, 1 reply; 82+ messages in thread From: David Woodhouse @ 2004-12-21 15:00 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD On Tue, 2004-12-21 at 13:40 +0000, Gareth Bult (Encryptec) wrote: > .. and I find the patches where ? On the list after I approved all 22 of them; they were trapped for moderation because they had attachments. After looking at one or two of yours which were also trapped, I approved them all because I thought they were all due to the GPG signature. Unfortunately I let through an HTML mail. Please don't send those -- and please don't top-post either. -- dwmw2 ^ permalink raw reply [flat|nested] 82+ messages in thread
[parent not found: <1103644945.10792.175.camel@squizzey.bult.co.uk>]
* Re: JFFS2 mount time [not found] ` <1103644945.10792.175.camel@squizzey.bult.co.uk> @ 2004-12-21 16:04 ` Jörn Engel 0 siblings, 0 replies; 82+ messages in thread From: Jörn Engel @ 2004-12-21 16:04 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD, David Woodhouse On Tue, 21 December 2004 16:02:25 +0000, Gareth Bult (Encryptec) wrote: > > Ahhhhhh! I get that a lot. ;) Jörn -- Fantasy is more important than knowledge. Knowledge is limited, while fantasy embraces the whole world. -- Albert Einstein ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-15 23:19 Gareth Bult (Encryptec) 2004-12-16 0:15 ` Josh Boyer @ 2004-12-16 13:43 ` Ferenc Havasi 2004-12-20 16:01 ` Gareth Bult (Encryptec) 1 sibling, 1 reply; 82+ messages in thread From: Ferenc Havasi @ 2004-12-16 13:43 UTC (permalink / raw) To: Gareth Bult; +Cc: Linux MTD [-- Attachment #1: Type: text/plain, Size: 343 bytes --] Hi Gareth, > However, I'm looking at 30secs to mount the root filesystem. Did you try our patch? I send it now again, we made it fresh for the current CVS. See its usage in the archive (16th November). It will part of JFFS3. This is designed for NAND devices - I don't know too much about your environment, but it may help. Bye, Ferenc [-- Attachment #2: jffs2-summary-20041207.patch --] [-- Type: text/plain, Size: 88751 bytes --] diff --unified --new-file --recursive mtd/fs/Kconfig mtd-dec07/fs/Kconfig --- mtd/fs/Kconfig 2004-11-04 00:00:11.000000000 +0100 +++ mtd-dec07/fs/Kconfig 2004-12-07 16:03:02.000000000 +0100 @@ -77,6 +77,19 @@ ECC for JFFS2. This type of flash chip is not common, however it is available from STMicro. +config JFFS2_SUMMARY + bool "JFFS2 summary support (EXPERIMENTAL)" + depends on JFFS2_FS + default n + help + This feature makes it possible to use summary information + for faster filesystem mount - specially on NAND. + + The summary information can be inserted into a filesystem image + by the utility 'sumtool'. + + If unsure, say 'N'. + config JFFS2_COMPRESSION_OPTIONS bool "Advanced compression options for JFFS2" default n diff --unified --new-file --recursive mtd/fs/jffs2/Makefile.common mtd-dec07/fs/jffs2/Makefile.common --- mtd/fs/jffs2/Makefile.common 2004-11-04 00:00:11.000000000 +0100 +++ mtd-dec07/fs/jffs2/Makefile.common 2004-12-07 16:03:02.000000000 +0100 @@ -16,3 +16,4 @@ jffs2-$(CONFIG_JFFS2_RUBIN) += compr_rubin.o jffs2-$(CONFIG_JFFS2_RTIME) += compr_rtime.o jffs2-$(CONFIG_JFFS2_ZLIB) += compr_zlib.o +jffs2-$(CONFIG_JFFS2_SUMMARY) += summary.o diff --unified --new-file --recursive mtd/fs/jffs2/build.c mtd-dec07/fs/jffs2/build.c --- mtd/fs/jffs2/build.c 2004-11-28 00:00:13.000000000 +0100 +++ mtd-dec07/fs/jffs2/build.c 2004-12-07 16:08:31.000000000 +0100 @@ -331,6 +331,10 @@ c->blocks[i].first_node = NULL; c->blocks[i].last_node = NULL; c->blocks[i].bad_count = 0; + #ifdef CONFIG_JFFS2_SUMMARY + c->blocks[i].sum_collected = NULL; + #endif + } init_MUTEX(&c->alloc_sem); diff --unified --new-file --recursive mtd/fs/jffs2/dir.c mtd-dec07/fs/jffs2/dir.c --- mtd/fs/jffs2/dir.c 2004-11-17 00:00:14.000000000 +0100 +++ mtd-dec07/fs/jffs2/dir.c 2004-12-07 16:03:02.000000000 +0100 @@ -22,6 +22,7 @@ #include <linux/time.h> #include "nodelist.h" + /* Urgh. Please tell me there's a nicer way of doing these. */ #include <linux/version.h> #if LINUX_VERSION_CODE < KERNEL_VERSION(2,5,48) @@ -314,7 +315,7 @@ * Just the node will do for now, though */ namelen = dentry->d_name.len; - ret = jffs2_reserve_space(c, sizeof(*ri) + strlen(target), &phys_ofs, &alloclen, ALLOC_NORMAL); + ret = jffs2_reserve_space(c, sizeof(*ri) + strlen(target), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE); if (ret) { jffs2_free_raw_inode(ri); @@ -360,7 +361,7 @@ up(&f->sem); jffs2_complete_reservation(c); - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL); + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen)); if (ret) { /* Eep. */ jffs2_clear_inode(inode); @@ -445,7 +446,7 @@ * Just the node will do for now, though */ namelen = dentry->d_name.len; - ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL); + ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE); if (ret) { jffs2_free_raw_inode(ri); @@ -488,7 +489,7 @@ up(&f->sem); jffs2_complete_reservation(c); - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL); + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen)); if (ret) { /* Eep. */ jffs2_clear_inode(inode); @@ -597,7 +598,7 @@ * Just the node will do for now, though */ namelen = dentry->d_name.len; - ret = jffs2_reserve_space(c, sizeof(*ri) + devlen, &phys_ofs, &alloclen, ALLOC_NORMAL); + ret = jffs2_reserve_space(c, sizeof(*ri) + devlen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE); if (ret) { jffs2_free_raw_inode(ri); @@ -642,7 +643,7 @@ up(&f->sem); jffs2_complete_reservation(c); - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL); + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen)); if (ret) { /* Eep. */ jffs2_clear_inode(inode); @@ -796,4 +797,3 @@ return 0; } - diff --unified --new-file --recursive mtd/fs/jffs2/file.c mtd-dec07/fs/jffs2/file.c --- mtd/fs/jffs2/file.c 2004-11-17 00:00:14.000000000 +0100 +++ mtd-dec07/fs/jffs2/file.c 2004-12-07 16:03:02.000000000 +0100 @@ -22,6 +22,7 @@ #include <linux/jffs2.h> #include "nodelist.h" + extern int generic_file_open(struct inode *, struct file *) __attribute__((weak)); extern loff_t generic_file_llseek(struct file *file, loff_t offset, int origin) __attribute__((weak)); @@ -135,7 +136,7 @@ D1(printk(KERN_DEBUG "Writing new hole frag 0x%x-0x%x between current EOF and new page\n", (unsigned int)inode->i_size, pageofs)); - ret = jffs2_reserve_space(c, sizeof(ri), &phys_ofs, &alloc_len, ALLOC_NORMAL); + ret = jffs2_reserve_space(c, sizeof(ri), &phys_ofs, &alloc_len, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE); if (ret) return ret; diff --unified --new-file --recursive mtd/fs/jffs2/fs.c mtd-dec07/fs/jffs2/fs.c --- mtd/fs/jffs2/fs.c 2004-11-29 00:00:09.000000000 +0100 +++ mtd-dec07/fs/jffs2/fs.c 2004-12-07 16:03:02.000000000 +0100 @@ -74,7 +74,7 @@ return -ENOMEM; } - ret = jffs2_reserve_space(c, sizeof(*ri) + mdatalen, &phys_ofs, &alloclen, ALLOC_NORMAL); + ret = jffs2_reserve_space(c, sizeof(*ri) + mdatalen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE); if (ret) { jffs2_free_raw_inode(ri); if (S_ISLNK(inode->i_mode & S_IFMT)) diff --unified --new-file --recursive mtd/fs/jffs2/gc.c mtd-dec07/fs/jffs2/gc.c --- mtd/fs/jffs2/gc.c 2004-11-17 00:00:14.000000000 +0100 +++ mtd-dec07/fs/jffs2/gc.c 2004-12-07 16:03:02.000000000 +0100 @@ -506,7 +506,7 @@ don't want to force wastage of the end of a block if splitting would work. */ ret = jffs2_reserve_space_gc(c, min_t(uint32_t, sizeof(struct jffs2_raw_inode) + JFFS2_MIN_DATA_LEN, - rawlen), &phys_ofs, &alloclen); + rawlen), &phys_ofs, &alloclen, rawlen); /* this is not optimal yet */ if (ret) return ret; @@ -614,7 +614,7 @@ ACCT_SANITY_CHECK(c,jeb); D1(ACCT_PARANOIA_CHECK(jeb)); - ret = jffs2_reserve_space_gc(c, rawlen, &phys_ofs, &dummy); + ret = jffs2_reserve_space_gc(c, rawlen, &phys_ofs, &dummy, rawlen); /* this is not optimal yet */ if (!ret) { D1(printk(KERN_DEBUG "Allocated space at 0x%08x to retry failed write.\n", phys_ofs)); @@ -692,7 +692,7 @@ } - ret = jffs2_reserve_space_gc(c, sizeof(ri) + mdatalen, &phys_ofs, &alloclen); + ret = jffs2_reserve_space_gc(c, sizeof(ri) + mdatalen, &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE); if (ret) { printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_metadata failed: %d\n", sizeof(ri)+ mdatalen, ret); @@ -759,7 +759,7 @@ rd.node_crc = cpu_to_je32(crc32(0, &rd, sizeof(rd)-8)); rd.name_crc = cpu_to_je32(crc32(0, fd->name, rd.nsize)); - ret = jffs2_reserve_space_gc(c, sizeof(rd)+rd.nsize, &phys_ofs, &alloclen); + ret = jffs2_reserve_space_gc(c, sizeof(rd)+rd.nsize, &phys_ofs, &alloclen, JFFS2_SUMMARY_DIRENT_SIZE(rd.nsize)); if (ret) { printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_dirent failed: %d\n", sizeof(rd)+rd.nsize, ret); @@ -961,7 +961,7 @@ ri.data_crc = cpu_to_je32(0); ri.node_crc = cpu_to_je32(crc32(0, &ri, sizeof(ri)-8)); - ret = jffs2_reserve_space_gc(c, sizeof(ri), &phys_ofs, &alloclen); + ret = jffs2_reserve_space_gc(c, sizeof(ri), &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE); if (ret) { printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_hole failed: %d\n", sizeof(ri), ret); @@ -1186,7 +1186,7 @@ uint32_t cdatalen; uint16_t comprtype = JFFS2_COMPR_NONE; - ret = jffs2_reserve_space_gc(c, sizeof(ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen); + ret = jffs2_reserve_space_gc(c, sizeof(ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE); if (ret) { printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_dnode failed: %d\n", @@ -1243,4 +1243,3 @@ jffs2_gc_release_page(c, pg_ptr, &pg); return ret; } - diff --unified --new-file --recursive mtd/fs/jffs2/nodelist.h mtd-dec07/fs/jffs2/nodelist.h --- mtd/fs/jffs2/nodelist.h 2004-11-26 15:08:31.000000000 +0100 +++ mtd-dec07/fs/jffs2/nodelist.h 2004-12-07 16:03:02.000000000 +0100 @@ -20,6 +20,7 @@ #include <linux/jffs2.h> #include <linux/jffs2_fs_sb.h> #include <linux/jffs2_fs_i.h> +#include "summary.h" #ifdef __ECOS #include "os-ecos.h" @@ -194,6 +195,10 @@ int bad_count; uint32_t offset; /* of this block in the MTD */ +#ifdef CONFIG_JFFS2_SUMMARY + struct jffs2_sum_info *sum_collected; +#endif + uint32_t unchecked_size; uint32_t used_size; uint32_t dirty_size; @@ -392,8 +397,8 @@ /* nodemgmt.c */ int jffs2_thread_should_wake(struct jffs2_sb_info *c); -int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio); -int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len); +int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio, uint32_t sumsize); +int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize); int jffs2_add_physical_node_ref(struct jffs2_sb_info *c, struct jffs2_raw_node_ref *new); void jffs2_complete_reservation(struct jffs2_sb_info *c); void jffs2_mark_node_obsolete(struct jffs2_sb_info *c, struct jffs2_raw_node_ref *raw); @@ -455,6 +460,10 @@ /* scan.c */ int jffs2_scan_medium(struct jffs2_sb_info *c); void jffs2_rotate_lists(struct jffs2_sb_info *c); +int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf, + uint32_t ofs, uint32_t len); +struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino); + /* build.c */ int jffs2_do_mount_fs(struct jffs2_sb_info *c); diff --unified --new-file --recursive mtd/fs/jffs2/nodemgmt.c mtd-dec07/fs/jffs2/nodemgmt.c --- mtd/fs/jffs2/nodemgmt.c 2004-11-26 15:08:31.000000000 +0100 +++ mtd-dec07/fs/jffs2/nodemgmt.c 2004-12-07 16:03:02.000000000 +0100 @@ -38,9 +38,9 @@ * for the requested allocation. */ -static int jffs2_do_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len); +static int jffs2_do_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize); -int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio) +int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio, uint32_t sumsize) { int ret = -EAGAIN; int blocksneeded = c->resv_blocks_write; @@ -129,7 +129,7 @@ spin_lock(&c->erase_completion_lock); } - ret = jffs2_do_reserve_space(c, minsize, ofs, len); + ret = jffs2_do_reserve_space(c, minsize, ofs, len, sumsize); if (ret) { D1(printk(KERN_DEBUG "jffs2_reserve_space: ret is %d\n", ret)); } @@ -140,7 +140,7 @@ return ret; } -int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len) +int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize) { int ret = -EAGAIN; minsize = PAD(minsize); @@ -149,7 +149,7 @@ spin_lock(&c->erase_completion_lock); while(ret == -EAGAIN) { - ret = jffs2_do_reserve_space(c, minsize, ofs, len); + ret = jffs2_do_reserve_space(c, minsize, ofs, len, sumsize); if (ret) { D1(printk(KERN_DEBUG "jffs2_reserve_space_gc: looping, ret is %d\n", ret)); } @@ -159,50 +159,112 @@ } /* Called with alloc sem _and_ erase_completion_lock */ -static int jffs2_do_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len) +static int jffs2_do_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize) { struct jffs2_eraseblock *jeb = c->nextblock; + uint32_t nofree_size; restart: - if (jeb && minsize > jeb->free_size) { - /* Skip the end of this block and file it as having some dirty space */ - /* If there's a pending write to it, flush now */ - if (jffs2_wbuf_dirty(c)) { - spin_unlock(&c->erase_completion_lock); - D1(printk(KERN_DEBUG "jffs2_do_reserve_space: Flushing write buffer\n")); - jffs2_flush_wbuf_pad(c); - spin_lock(&c->erase_completion_lock); - jeb = c->nextblock; - goto restart; - } - c->wasted_size += jeb->free_size; - c->free_size -= jeb->free_size; - jeb->wasted_size += jeb->free_size; - jeb->free_size = 0; + nofree_size = 0; + +#ifdef CONFIG_JFFS2_SUMMARY + + if (sumsize != JFFS2_SUMMARY_NOSUM_SIZE) { + int ret; + if (jeb) { + if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret; + nofree_size = sumsize + jeb->sum_collected->sum_size + JFFS2_SUMMARY_FRAME_SIZE; + } - /* Check, if we have a dirty block now, or if it was dirty already */ - if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) { - c->dirty_size += jeb->wasted_size; - c->wasted_size -= jeb->wasted_size; - jeb->dirty_size += jeb->wasted_size; - jeb->wasted_size = 0; - if (VERYDIRTY(c, jeb->dirty_size)) { - D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", + D1(printk(KERN_DEBUG "JFFS2: minsize %d , jeb->free(%d) , sum_collected->size(%d) , sumsize(%d)\n",minsize,jeb->free_size,jeb->sum_collected->sum_size,sumsize)); + + if (jeb && (minsize + jeb->sum_collected->sum_size + sumsize + JFFS2_SUMMARY_FRAME_SIZE > jeb->free_size)) { + D1(printk(KERN_DEBUG "JFFS2: generating summary for 0x%08x.\n", jeb->offset)); + if (jeb->sum_collected->sum_size == JFFS2_SUMMARY_NOSUM_SIZE) { + sumsize = JFFS2_SUMMARY_NOSUM_SIZE; + jffs2_sum_clean_collected(jeb); + goto restart; + } + + ret = jffs2_sum_write_sumnode(c); + + if (ret) + return ret; + + if (jeb->sum_collected->sum_size == JFFS2_SUMMARY_NOSUM_SIZE) { //jffs2_write_sumnode can't write out the summary information + sumsize = JFFS2_SUMMARY_NOSUM_SIZE; + jffs2_sum_clean_collected(jeb); + goto restart; + } + + /* Check, if we have a dirty block now, or if it was dirty already */ + if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) { + c->dirty_size += jeb->wasted_size; + c->wasted_size -= jeb->wasted_size; + jeb->dirty_size += jeb->wasted_size; + jeb->wasted_size = 0; + if (VERYDIRTY(c, jeb->dirty_size)) { + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", + jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size)); + list_add_tail(&jeb->list, &c->very_dirty_list); + } else { + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", + jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size)); + list_add_tail(&jeb->list, &c->dirty_list); + } + } else { + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size)); - list_add_tail(&jeb->list, &c->very_dirty_list); - } else { - D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", + list_add_tail(&jeb->list, &c->clean_list); + } + c->nextblock = jeb = NULL; + } + } + else { +#endif + if (jeb && minsize > jeb->free_size) { + /* Skip the end of this block and file it as having some dirty space */ + /* If there's a pending write to it, flush now */ + + if (jffs2_wbuf_dirty(c)) { + spin_unlock(&c->erase_completion_lock); + D1(printk(KERN_DEBUG "jffs2_do_reserve_space: Flushing write buffer\n")); + jffs2_flush_wbuf_pad(c); + spin_lock(&c->erase_completion_lock); + jeb = c->nextblock; + goto restart; + } + + c->wasted_size += jeb->free_size; + c->free_size -= jeb->free_size; + jeb->wasted_size += jeb->free_size; + jeb->free_size = 0; + + /* Check, if we have a dirty block now, or if it was dirty already */ + if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) { + c->dirty_size += jeb->wasted_size; + c->wasted_size -= jeb->wasted_size; + jeb->dirty_size += jeb->wasted_size; + jeb->wasted_size = 0; + if (VERYDIRTY(c, jeb->dirty_size)) { + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", + jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size)); + list_add_tail(&jeb->list, &c->very_dirty_list); + } else { + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", + jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size)); + list_add_tail(&jeb->list, &c->dirty_list); + } + } else { + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size)); - list_add_tail(&jeb->list, &c->dirty_list); + list_add_tail(&jeb->list, &c->clean_list); } - } else { - D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", - jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size)); - list_add_tail(&jeb->list, &c->clean_list); + c->nextblock = jeb = NULL; } - c->nextblock = jeb = NULL; +#ifdef CONFIG_JFFS2_SUMMARY } - +#endif if (!jeb) { struct list_head *next; /* Take the next block off the 'free' list */ @@ -266,7 +328,7 @@ /* OK, jeb (==c->nextblock) is now pointing at a block which definitely has enough space */ *ofs = jeb->offset + (c->sector_size - jeb->free_size); - *len = jeb->free_size; + *len = jeb->free_size - nofree_size; if (c->cleanmarker_size && jeb->used_size == c->cleanmarker_size && !jeb->first_node->next_in_ino) { diff --unified --new-file --recursive mtd/fs/jffs2/os-linux.h mtd-dec07/fs/jffs2/os-linux.h --- mtd/fs/jffs2/os-linux.h 2004-11-17 00:00:14.000000000 +0100 +++ mtd-dec07/fs/jffs2/os-linux.h 2004-12-07 16:03:02.000000000 +0100 @@ -100,7 +100,13 @@ #define jffs2_is_readonly(c) (OFNI_BS_2SFFJ(c)->s_flags & MS_RDONLY) #if (!defined CONFIG_JFFS2_FS_NAND && !defined CONFIG_JFFS2_FS_NOR_ECC) + +#ifndef CONFIG_JFFS2_SUMMARY #define jffs2_can_mark_obsolete(c) (1) +#else +#define jffs2_can_mark_obsolete(c) (0) +#endif + #define jffs2_cleanmarker_oob(c) (0) #define jffs2_write_nand_cleanmarker(c,jeb) (-EIO) @@ -121,7 +127,12 @@ #else /* NAND and/or ECC'd NOR support present */ +#ifndef CONFIG_JFFS2_SUMMARY #define jffs2_can_mark_obsolete(c) ((c->mtd->type == MTD_NORFLASH && !(c->mtd->flags & MTD_ECC)) || c->mtd->type == MTD_RAM) +#else +#define jffs2_can_mark_obsolete(c) (0) +#endif + #define jffs2_cleanmarker_oob(c) (c->mtd->type == MTD_NANDFLASH) #define jffs2_flash_write_oob(c, ofs, len, retlen, buf) ((c)->mtd->write_oob((c)->mtd, ofs, len, retlen, buf)) @@ -218,5 +229,3 @@ #endif /* __JFFS2_OS_LINUX_H__ */ - - diff --unified --new-file --recursive mtd/fs/jffs2/scan.c mtd-dec07/fs/jffs2/scan.c --- mtd/fs/jffs2/scan.c 2004-11-26 15:08:31.000000000 +0100 +++ mtd-dec07/fs/jffs2/scan.c 2004-12-07 16:03:02.000000000 +0100 @@ -18,22 +18,10 @@ #include <linux/crc32.h> #include <linux/compiler.h> #include "nodelist.h" +#include "summary.h" #define EMPTY_SCAN_SIZE 1024 -#define DIRTY_SPACE(x) do { typeof(x) _x = (x); \ - c->free_size -= _x; c->dirty_size += _x; \ - jeb->free_size -= _x ; jeb->dirty_size += _x; \ - }while(0) -#define USED_SPACE(x) do { typeof(x) _x = (x); \ - c->free_size -= _x; c->used_size += _x; \ - jeb->free_size -= _x ; jeb->used_size += _x; \ - }while(0) -#define UNCHECKED_SPACE(x) do { typeof(x) _x = (x); \ - c->free_size -= _x; c->unchecked_size += _x; \ - jeb->free_size -= _x ; jeb->unchecked_size += _x; \ - }while(0) - #define noisy_printk(noise, args...) do { \ if (*(noise)) { \ printk(KERN_NOTICE args); \ @@ -58,13 +46,6 @@ static int jffs2_scan_dirent_node(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb, struct jffs2_raw_dirent *rd, uint32_t ofs); -#define BLK_STATE_ALLFF 0 -#define BLK_STATE_CLEAN 1 -#define BLK_STATE_PARTDIRTY 2 -#define BLK_STATE_CLEANMARKER 3 -#define BLK_STATE_ALLDIRTY 4 -#define BLK_STATE_BADBLOCK 5 - static inline int min_free(struct jffs2_sb_info *c) { uint32_t min = 2 * sizeof(struct jffs2_raw_inode); @@ -257,7 +238,7 @@ return ret; } -static int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf, +int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf, uint32_t ofs, uint32_t len) { int ret; @@ -286,6 +267,11 @@ uint32_t hdr_crc, buf_ofs, buf_len; int err; int noise = 0; + +#ifdef CONFIG_JFFS2_SUMMARY + struct jffs2_sum_marker *sm; +#endif + #ifdef CONFIG_JFFS2_FS_NAND int cleanmarkerfound = 0; #endif @@ -311,10 +297,54 @@ } } #endif + +#ifdef CONFIG_JFFS2_SUMMARY + sm = (struct jffs2_sum_marker *)kmalloc(sizeof(struct jffs2_sum_marker), GFP_KERNEL); + if (!sm) { + return -ENOMEM; + } + + err = jffs2_fill_scan_buf(c, (unsigned char *) sm, jeb->offset + c->sector_size - sizeof(struct jffs2_sum_marker), sizeof(struct jffs2_sum_marker)); + + if (err) { + kfree(sm); + return err; + } + + if (je32_to_cpu(sm->magic) == JFFS2_SUM_MAGIC ) { + + if(je32_to_cpu(sm->erase_size) == c->sector_size) { + int ret = jffs2_sum_scan_sumnode(c,jeb,je32_to_cpu(sm->offset),&pseudo_random); + + if (ret) { + kfree(sm); + return ret; + } + } + + printk(KERN_WARNING "FS erase_block_size != JFFS2 erase_block_size => skipping summary information\n"); + + } + + kfree(sm); + + ofs = jeb->offset; + prevofs = jeb->offset - 1; + +#endif + buf_ofs = jeb->offset; if (!buf_size) { buf_len = c->sector_size; + +#ifdef CONFIG_JFFS2_SUMMARY + /* must reread because of summary test */ + err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len); + if (err) + return err; +#endif + } else { buf_len = EMPTY_SCAN_SIZE; err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len); @@ -356,6 +386,8 @@ noise = 10; + D1(printk(KERN_DEBUG "JFFS2: no summary found in jeb 0x%08x. Apply original scan.\n",jeb->offset)); + scan_more: while(ofs < jeb->offset + c->sector_size) { @@ -571,6 +603,9 @@ break; case JFFS2_NODETYPE_PADDING: +#ifdef CONFIG_JFFS2_SUMMARY + jffs2_sum_add_padding_mem(jeb,je32_to_cpu(node->totlen)); +#endif DIRTY_SPACE(PAD(je32_to_cpu(node->totlen))); ofs += PAD(je32_to_cpu(node->totlen)); break; @@ -634,7 +669,7 @@ return BLK_STATE_ALLDIRTY; } -static struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino) +struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino) { struct jffs2_inode_cache *ic; @@ -728,6 +763,11 @@ pseudo_random += je32_to_cpu(ri->version); UNCHECKED_SPACE(PAD(je32_to_cpu(ri->totlen))); + +#ifdef CONFIG_JFFS2_SUMMARY + jffs2_sum_add_inode_mem(jeb,ri,ofs); +#endif + return 0; } @@ -806,6 +846,10 @@ USED_SPACE(PAD(je32_to_cpu(rd->totlen))); jffs2_add_fd_to_list(c, fd, &ic->scan_dents); +#ifdef CONFIG_JFFS2_SUMMARY + jffs2_sum_add_dirent_mem(jeb,rd,ofs); +#endif + return 0; } diff --unified --new-file --recursive mtd/fs/jffs2/summary.c mtd-dec07/fs/jffs2/summary.c --- mtd/fs/jffs2/summary.c 1970-01-01 01:00:00.000000000 +0100 +++ mtd-dec07/fs/jffs2/summary.c 2004-12-07 16:03:02.000000000 +0100 @@ -0,0 +1,720 @@ +/* + * JFFS2 -- Journalling Flash File System, Version 2. + * + * Copyright (C) 2004 Ferenc Havasi <havasi@inf.u-szeged.hu>, + * Zoltan Sogor <weth@inf.u-szeged.hu>, + * Patrik Kluba <pajko@halom.u-szeged.hu>, + * University of Szeged, Hungary + * + * For licensing information, see the file 'LICENCE' in this directory. + * + * $Id$ + * + */ + +#include <linux/kernel.h> +#include <linux/sched.h> +#include <linux/slab.h> +#include <linux/mtd/mtd.h> +#include <linux/pagemap.h> +#include <linux/crc32.h> +#include <linux/compiler.h> +#include <linux/vmalloc.h> +#include "nodelist.h" + +int jffs2_sum_init(struct jffs2_sb_info *c) +{ + c->summary_buf = (jint32_t *) vmalloc(c->sector_size); + if (!c->summary_buf) { + printk(KERN_WARNING "JFFS2: can't allocate memory to dump summary information!\n"); + return 1; + } + return 0; +} + +void jffs2_sum_exit(struct jffs2_sb_info *c) +{ + if (c->summary_buf) { + vfree(c->summary_buf); + c->summary_buf = NULL; + } +} + +int jffs2_sum_care_sum_collected(struct jffs2_eraseblock *jeb) +{ + if (!jeb->sum_collected) { + jeb->sum_collected = (struct jffs2_sum_info *) kmalloc(sizeof(struct jffs2_sum_info), GFP_KERNEL); + + if (!jeb->sum_collected) + return -ENOMEM; + + jeb->sum_collected->sum_list = NULL; + jeb->sum_collected->sum_num = 0; + jeb->sum_collected->sum_size = 0; + jeb->sum_collected->sum_padded = 0; + } + return 0; +} + +static int jffs2_sum_add_mem(struct jffs2_eraseblock *jeb, union jffs2_sum_mem *item) +{ + + union jffs2_sum_mem *walk; + int ret; + + if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret; + + if (!jeb->sum_collected->sum_list) { + jeb->sum_collected->sum_list = (union jffs2_sum_mem *) item; + } + else { + walk = jeb->sum_collected->sum_list; + + while (walk->u.next) { + walk = walk->u.next; + } + walk->u.next = (union jffs2_sum_mem *) item; + } + switch (je16_to_cpu(item->u.nodetype)) { + case JFFS2_NODETYPE_INODE: + jeb->sum_collected->sum_size += JFFS2_SUMMARY_INODE_SIZE; + jeb->sum_collected->sum_num++; + break; + case JFFS2_NODETYPE_DIRENT: + jeb->sum_collected->sum_size += JFFS2_SUMMARY_DIRENT_SIZE(item->d.nsize); + jeb->sum_collected->sum_num++; + break; + default: + printk(KERN_WARNING "__jffs2_add_sum_mem(): UNKNOWN node type %d\n", je16_to_cpu(item->u.nodetype)); + return 1; + } + return 0; +} + +void jffs2_sum_clean_all_info(struct jffs2_sb_info *c) +{ + int i; + + for (i=0; i<c->nr_blocks; i++) { + struct jffs2_eraseblock *jeb = &c->blocks[i]; + + jffs2_sum_clean_collected(jeb); + kfree(jeb->sum_collected); + jeb->sum_collected = NULL; + } +} + +/* These 3 functions are called from scan.c to collect summary info for not closed jeb */ + +int jffs2_sum_add_padding_mem(struct jffs2_eraseblock *jeb, uint32_t size) +{ + int ret; + + if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret; + jeb->sum_collected->sum_padded += size; + return 0; +} + +int jffs2_sum_add_inode_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_inode *ri, uint32_t ofs) +{ + + struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *) kmalloc(sizeof(struct jffs2_sum_inode_mem), GFP_KERNEL); + + if (!temp) + return -ENOMEM; + + ofs -= jeb->offset; + + temp->nodetype = ri->nodetype; + temp->inode = ri->ino; + temp->version = ri->version; + temp->offset = cpu_to_je32(ofs); + temp->totlen = ri->totlen; + temp->next = NULL; + + return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp); +} + +int jffs2_sum_add_dirent_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_dirent *rd, uint32_t ofs) +{ + + struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *) + kmalloc(sizeof(struct jffs2_sum_dirent_mem) + rd->nsize, GFP_KERNEL); + + if (!temp) + return -ENOMEM; + + ofs -= jeb->offset; + + temp->nodetype = rd->nodetype; + temp->totlen = rd->totlen; + temp->offset = cpu_to_je32(ofs); + temp->pino = rd->pino; + temp->version = rd->version; + temp->ino = rd->ino; + temp->nsize = rd->nsize; + temp->type = rd->type; + temp->next = NULL; + + memcpy(temp->name, rd->name, rd->nsize); + + return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp); +} + +/* Cleanup every collected summary information */ + +void jffs2_sum_clean_collected(struct jffs2_eraseblock *jeb) +{ + + union jffs2_sum_mem *temp; + + if(jeb && jeb->sum_collected){ + + while(jeb->sum_collected->sum_list){ + temp = jeb->sum_collected->sum_list; + jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next; + kfree(temp); + jeb->sum_collected->sum_num--; + } + + if(jeb->sum_collected->sum_num != 0){ + printk(KERN_WARNING "Ooops, something wrong happened! sum_num != 0, but sum_list = null ???"); + jeb->sum_collected->sum_num = 0; + } + } +} + +/* Called from wbuf.c to collect writed node info */ + +int jffs2_sum_add_kvec(struct jffs2_sb_info *c, const struct kvec *invecs, unsigned long count, uint32_t ofs) +{ + union jffs2_node_union *node; + struct jffs2_eraseblock *jeb; + int ret; + + node = (union jffs2_node_union *) invecs[0].iov_base; + jeb = &c->blocks[ofs / c->sector_size]; + ofs -= jeb->offset; + + if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret; + + switch(je16_to_cpu(node->u.nodetype)){ + case JFFS2_NODETYPE_INODE : { + struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *) + kmalloc(sizeof(struct jffs2_sum_inode_mem), GFP_KERNEL); + + if (!temp) + return -ENOMEM; + + temp->nodetype = node->i.nodetype; + temp->inode = node->i.ino; + temp->version = node->i.version; + temp->offset = cpu_to_je32(ofs); + temp->totlen = node->i.totlen; + temp->next = NULL; + + return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp); + + break; + } + + case JFFS2_NODETYPE_DIRENT : { + struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *) + kmalloc(sizeof(struct jffs2_sum_dirent_mem) + node->d.nsize, GFP_KERNEL); + + if (!temp) + return -ENOMEM; + + temp->nodetype = node->d.nodetype; + temp->totlen = node->d.totlen; + temp->offset = cpu_to_je32(ofs); + temp->pino = node->d.pino; + temp->version = node->d.version; + temp->ino = node->d.ino; + temp->nsize = node->d.nsize; + temp->type = node->d.type; + temp->next = NULL; + + memcpy(temp->name,invecs[1].iov_base,node->d.nsize); + + return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp); + + break; + } + + case JFFS2_NODETYPE_PADDING : { + D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node PADDING\n")); + jeb->sum_collected->sum_padded += je32_to_cpu(node->u.totlen); + break; + } + + case JFFS2_NODETYPE_CLEANMARKER : { + D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node CLEANMARKER\n")); + break; + } + + case JFFS2_NODETYPE_SUMMARY : { + D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node SUMMARY\n")); + break; + } + + default : { + printk(KERN_WARNING "jffs2_sum_add_kvec(): Node not supported\n"); + BUG(); + break; + } + } + + return 0; +} + +/* Process the summary information - called from jffs2_scan_eraseblock() */ + +int jffs2_sum_scan_sumnode(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb, uint32_t ofs, uint32_t *pseudo_random) +{ + + struct jffs2_unknown_node crcnode; + struct jffs2_raw_node_ref *raw; + struct jffs2_raw_node_ref *cache_ref; + struct jffs2_inode_cache *ic; + struct jffs2_full_dirent *fd; + + int i, err; + int bad_sum = 0; + int sumsize; + uint32_t ino; + uint32_t crc; + struct jffs2_summary_node *summary; + + sumsize = c->sector_size - ofs; + ofs += jeb->offset; + + D1(printk(KERN_DEBUG "JFFS2: summary found for 0x%08x at 0x%08x (0x%x bytes)\n", jeb->offset, ofs, sumsize)); + + summary = (struct jffs2_summary_node *) kmalloc(sumsize, GFP_KERNEL); + + if (!summary) { + return -ENOMEM; + } + + err = jffs2_fill_scan_buf(c, (unsigned char *)summary, ofs, sumsize); + + if (err) { + kfree(summary); + return err; + } + + /* OK, now check for node validity and CRC */ + crcnode.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK); + crcnode.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY); + crcnode.totlen = summary->totlen; + crc = crc32(0, &crcnode, sizeof(crcnode)-4); + + if (je32_to_cpu(summary->hdr_crc) != crc) { + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node header is corrupt (bad CRC or no summary at all)\n")); + bad_sum = 1; + } + + if ((!bad_sum) && (je32_to_cpu(summary->totlen) != sumsize)) { + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (wrong erasesize?)\n")); + bad_sum = 1; + } + + crc = crc32(0, summary, sizeof(struct jffs2_summary_node)-8); + + if ((!bad_sum) && (je32_to_cpu(summary->node_crc) != crc)) { + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (bad CRC)\n")); + bad_sum = 1; + } + + crc = crc32(0, summary->sum, sumsize - sizeof(struct jffs2_summary_node)); + + if ((!bad_sum) && (je32_to_cpu(summary->sum_crc) != crc)) { + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node data is corrupt (bad CRC)\n")); + bad_sum = 1; + } + + if (!bad_sum) { + + struct jffs2_sum_unknown_flash *sp; + sp = (struct jffs2_sum_unknown_flash *) summary->sum; + + if ( je32_to_cpu(summary->cln_mkr) ) { + + D1(printk(KERN_DEBUG "Summary : CLEANMARKER node \n")); + + if (je32_to_cpu(summary->cln_mkr) != c->cleanmarker_size) { + D1(printk(KERN_DEBUG "CLEANMARKER node has totlen 0x%x != normal 0x%x\n", + je32_to_cpu(summary->cln_mkr), c->cleanmarker_size)); + UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) ); + } + else if (jeb->first_node) { + D1(printk(KERN_DEBUG "CLEANMARKER node not first node in block (0x%08x)\n", jeb->offset); + UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)))); + } + else { + struct jffs2_raw_node_ref *marker_ref = jffs2_alloc_raw_node_ref(); + + if (!marker_ref) { + D1(printk(KERN_NOTICE "Failed to allocate node ref for clean marker\n")); + kfree(summary); + return -ENOMEM; + } + + marker_ref->next_in_ino = NULL; + marker_ref->next_phys = NULL; + marker_ref->flash_offset = jeb->offset | REF_NORMAL; + marker_ref->__totlen = je32_to_cpu(summary->cln_mkr); + jeb->first_node = jeb->last_node = marker_ref; + + USED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) ); + + } + } + + if ( je32_to_cpu(summary->padded) ) { + DIRTY_SPACE(je32_to_cpu(summary->padded)); + } + + for(i = 0; i < je16_to_cpu(summary->sum_num); i++) { + + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Processing summary information %d\n", i)); + uint8_t *temp8ptr = NULL; + + switch (je16_to_cpu(sp->nodetype)) { + + case JFFS2_NODETYPE_INODE : { + struct jffs2_sum_inode_flash *spi; + spi = (struct jffs2_sum_inode_flash *) sp; + + ino = je32_to_cpu(spi->inode); + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Inode at 0x%08x\n", jeb->offset + je32_to_cpu(spi->offset))); + raw = jffs2_alloc_raw_node_ref(); + if (!raw) { + printk(KERN_NOTICE "jffs2_scan_eraseblock(): allocation of node reference failed\n"); + kfree(summary); + return -ENOMEM; + } + + ic = jffs2_get_ino_cache(c, ino); + if (!ic) { + ic = jffs2_scan_make_ino_cache(c, ino); + if (!ic) { + printk(KERN_NOTICE "jffs2_scan_eraseblock(): scan_make_ino_cache failed\n"); + jffs2_free_raw_node_ref(raw); + kfree(summary); + return -ENOMEM; + } + } + + raw->flash_offset = (jeb->offset + je32_to_cpu(spi->offset)) | REF_UNCHECKED; + raw->__totlen = PAD(je32_to_cpu(spi->totlen)); + raw->next_phys = NULL; + raw->next_in_ino = ic->nodes; + + ic->nodes = raw; + if (!jeb->first_node) + jeb->first_node = raw; + if (jeb->last_node) + jeb->last_node->next_phys = raw; + jeb->last_node = raw; + + *pseudo_random += je32_to_cpu(spi->version); + UNCHECKED_SPACE(PAD(je32_to_cpu(spi->totlen))); + + temp8ptr = (uint8_t *) sp; + temp8ptr += JFFS2_SUMMARY_INODE_SIZE; + sp = (struct jffs2_sum_unknown_flash *) temp8ptr; + + break; + } + + case JFFS2_NODETYPE_DIRENT : { + struct jffs2_sum_dirent_flash *spd; + spd = (struct jffs2_sum_dirent_flash *) sp; + + fd = jffs2_alloc_full_dirent(spd->nsize+1); + if (!fd) { + kfree(summary); + return -ENOMEM; + } + + memcpy(&fd->name, spd->name, spd->nsize); + fd->name[spd->nsize] = 0; + + raw = jffs2_alloc_raw_node_ref(); + if (!raw) { + jffs2_free_full_dirent(fd); + printk(KERN_NOTICE "jffs2_scan_dirent_node(): allocation of node reference failed\n"); + kfree(summary); + return -ENOMEM; + } + + ic = jffs2_scan_make_ino_cache(c, je32_to_cpu(spd->pino)); + if (!ic) { + jffs2_free_full_dirent(fd); + jffs2_free_raw_node_ref(raw); + kfree(summary); + return -ENOMEM; + } + + raw->__totlen = PAD(je32_to_cpu(spd->totlen)); + raw->flash_offset = (jeb->offset + je32_to_cpu(spd->offset)) | REF_PRISTINE; + raw->next_phys = NULL; + raw->next_in_ino = ic->nodes; + ic->nodes = raw; + if (!jeb->first_node) + jeb->first_node = raw; + if (jeb->last_node) + jeb->last_node->next_phys = raw; + jeb->last_node = raw; + + fd->raw = raw; + fd->next = NULL; + fd->version = je32_to_cpu(spd->version); + fd->ino = je32_to_cpu(spd->ino); + fd->nhash = full_name_hash(fd->name, spd->nsize); + fd->type = spd->type; + USED_SPACE(PAD(je32_to_cpu(spd->totlen))); + jffs2_add_fd_to_list(c, fd, &ic->scan_dents); + + *pseudo_random += je32_to_cpu(spd->version); + + temp8ptr = (uint8_t *) sp; + temp8ptr += JFFS2_SUMMARY_DIRENT_SIZE(spd->nsize); + sp = (struct jffs2_sum_unknown_flash *) temp8ptr; + + break; + } + + default : { + printk(KERN_WARNING "Kernel doesn't support this type of node !!! Exiting"); + return -EIO; + break; + } + } + } + + kfree(summary); + + /* for ACCT_PARANOIA_CHECK */ + cache_ref = jffs2_alloc_raw_node_ref(); + + if (!cache_ref) { + printk(KERN_NOTICE "Failed to allocate node ref for cache\n"); + return -ENOMEM; + } + + cache_ref->next_in_ino = NULL; + cache_ref->next_phys = NULL; + cache_ref->flash_offset = ofs | REF_NORMAL; + cache_ref->__totlen = sumsize; + + if (!jeb->first_node) + jeb->first_node = cache_ref; + if (jeb->last_node) + jeb->last_node->next_phys = cache_ref; + jeb->last_node = cache_ref; + + USED_SPACE(sumsize); + + jeb->wasted_size += jeb->free_size; + c->wasted_size += jeb->free_size; + c->free_size -= jeb->free_size; + jeb->free_size = 0; + + /* somebody check this and all of space accounting in summary support */ + + if ((jeb->used_size + jeb->unchecked_size) == PAD(c->cleanmarker_size) && !jeb->dirty_size + && (!jeb->first_node || !jeb->first_node->next_in_ino) ) { + return BLK_STATE_CLEANMARKER; + } + /* move blocks with max 4 byte dirty space to cleanlist */ + else if (!ISDIRTY(c->sector_size - (jeb->used_size + jeb->unchecked_size))) { + c->dirty_size -= jeb->dirty_size; + c->wasted_size += jeb->dirty_size; + jeb->wasted_size += jeb->dirty_size; + jeb->dirty_size = 0; + return BLK_STATE_CLEAN; + } + else if (jeb->used_size || jeb->unchecked_size) { + return BLK_STATE_PARTDIRTY; + } + else { + return BLK_STATE_ALLDIRTY; + } + } + + return 0; +} + +/* Write out summary information - called from jffs2_do_reserve_space */ + +int jffs2_sum_write_sumnode(struct jffs2_sb_info *c) +{ + struct jffs2_summary_node isum; + union jffs2_sum_mem *temp; + jint32_t offset; + jint32_t *wpage; + uint8_t *tempptr; + int datasize; + int infosize; + int padsize; + size_t retlen; + int ret; + struct jffs2_eraseblock *jeb; + struct kvec vecs[2]; + jint32_t magic = cpu_to_je32(JFFS2_SUM_MAGIC); + + D2(printk(KERN_DEBUG "jffs2_sum_write_sumnode()\n")); + + jeb = c->nextblock; + + if (!jeb->sum_collected->sum_num || !jeb->sum_collected->sum_list) { + printk(KERN_WARNING "JFFS2: jffs2_sum_write_sumnode(): empty summary info!!!\n"); + BUG(); + } + + datasize = jeb->sum_collected->sum_size + sizeof(struct jffs2_sum_marker); + infosize = sizeof(struct jffs2_summary_node) + datasize; + padsize = jeb->free_size - infosize; + infosize += padsize; datasize += padsize; + offset = cpu_to_je32(c->sector_size - jeb->free_size); + + if (padsize < 0) { // if jeb hasn't got enought free space for summary + + union jffs2_sum_mem *temp; + + while(jeb->sum_collected->sum_list){ //cleanup sum_list + temp = jeb->sum_collected->sum_list; + jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next; + kfree(temp); + jeb->sum_collected->sum_num--; + } + + jeb->sum_collected->sum_list = NULL; + jeb->sum_collected->sum_num = 0; + jeb->sum_collected->sum_size = JFFS2_SUMMARY_NOSUM_SIZE; // don't try to write out summary for this node + + printk(KERN_WARNING "JFFS2: not enough space for summary, padsize = %d\n",padsize); + return 0; + } + + memset(c->summary_buf, 0xff, datasize); + memset(&isum, 0, sizeof(isum)); + + isum.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK); + isum.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY); + isum.totlen = cpu_to_je32(infosize); + isum.hdr_crc = cpu_to_je32(crc32(0, &isum, sizeof(struct jffs2_unknown_node) - 4)); + isum.padded = cpu_to_je32(jeb->sum_collected->sum_padded); + + if (c->cleanmarker_size) { + isum.cln_mkr = cpu_to_je32(c->cleanmarker_size); + } + else{ + isum.cln_mkr = cpu_to_je32(0); + } + + isum.sum_num = cpu_to_je16(jeb->sum_collected->sum_num); + wpage = c->summary_buf; + + + while (jeb->sum_collected->sum_num) { // write sum_data + + + switch(je16_to_cpu(jeb->sum_collected->sum_list->u.nodetype)){ + + case JFFS2_NODETYPE_INODE : { + jint16_t *temp16ptr = (jint16_t *)wpage; + + *(temp16ptr++) = jeb->sum_collected->sum_list->i.nodetype; + wpage = (jint32_t *) temp16ptr; + + *(wpage++) = jeb->sum_collected->sum_list->i.inode; + *(wpage++) = jeb->sum_collected->sum_list->i.version; + *(wpage++) = jeb->sum_collected->sum_list->i.offset; + *(wpage++) = jeb->sum_collected->sum_list->i.totlen; + break; + } + + case JFFS2_NODETYPE_DIRENT : { + jint16_t *temp16ptr = (jint16_t *) wpage; + uint8_t *temp8ptr = NULL; + + *(temp16ptr++) = jeb->sum_collected->sum_list->d.nodetype; + wpage = (jint32_t *) temp16ptr; + + *(wpage++) = jeb->sum_collected->sum_list->d.totlen; + *(wpage++) = jeb->sum_collected->sum_list->d.offset; + *(wpage++) = jeb->sum_collected->sum_list->d.pino; + *(wpage++) = jeb->sum_collected->sum_list->d.version; + *(wpage++) = jeb->sum_collected->sum_list->d.ino; + + temp8ptr = (uint8_t *) wpage; + *(temp8ptr++) = jeb->sum_collected->sum_list->d.nsize; + *(temp8ptr++) = jeb->sum_collected->sum_list->d.type; + + memcpy(temp8ptr,jeb->sum_collected->sum_list->d.name,jeb->sum_collected->sum_list->d.nsize); + temp8ptr += jeb->sum_collected->sum_list->d.nsize; + wpage = (jint32_t *) temp8ptr; + + break; + } + + default : { + printk(KERN_WARNING "Unknown node in summary information!!!\n"); + BUG(); + } + } + + temp = jeb->sum_collected->sum_list; + jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next; + kfree(temp); + + jeb->sum_collected->sum_num--; + } + + jeb->sum_collected->sum_size = 0; + + tempptr = (uint8_t *) wpage; + tempptr += padsize; + wpage = (jint32_t *) tempptr; + + *(wpage++) = offset; + *(wpage++) = cpu_to_je32(c->sector_size); + *(wpage++) = magic; + isum.sum_crc = cpu_to_je32(crc32(0, c->summary_buf, datasize)); + isum.node_crc = cpu_to_je32(crc32(0, &isum, sizeof(isum) - 8)); + + vecs[0].iov_base = &isum; + vecs[0].iov_len = sizeof(isum); + vecs[1].iov_base = c->summary_buf; + vecs[1].iov_len = datasize; + + D1(printk(KERN_DEBUG "JFFS2: writing out data to flash to pos : 0x%08x\n",jeb->offset + c->sector_size - jeb->free_size)); + + spin_unlock(&c->erase_completion_lock); + ret = jffs2_flash_writev(c, vecs, 2, jeb->offset + c->sector_size - jeb->free_size, &retlen, 0); + spin_lock(&c->erase_completion_lock); + + + if (ret || (retlen != infosize)) { + printk(KERN_WARNING "JFFS2: write of %zd bytes at 0x%08x failed. returned %d, retlen %zd\n", + infosize, jeb->offset + c->sector_size - jeb->free_size, ret, retlen); + + jeb->sum_collected->sum_size = JFFS2_SUMMARY_NOSUM_SIZE; + + WASTED_SPACE(infosize); + + return 0; + } + + /*spin_unlock(&c->erase_completion_lock); + jffs2_flush_wbuf_pad(c); // summary filled the wbuf + spin_lock(&c->erase_completion_lock);*/ + + WASTED_SPACE(infosize); + + return 0; +} diff --unified --new-file --recursive mtd/fs/jffs2/summary.h mtd-dec07/fs/jffs2/summary.h --- mtd/fs/jffs2/summary.h 1970-01-01 01:00:00.000000000 +0100 +++ mtd-dec07/fs/jffs2/summary.h 2004-12-07 16:03:02.000000000 +0100 @@ -0,0 +1,158 @@ +/* + * JFFS2 -- Journalling Flash File System, Version 2. + * + * Copyright (C) 2004 Ferenc Havasi <havasi@inf.u-szeged.hu>, + * Zoltan Sogor <weth@inf.u-szeged.hu>, + * Patrik Kluba <pajko@halom.u-szeged.hu>, + * University of Szeged, Hungary + * + * For licensing information, see the file 'LICENCE' in this directory. + * + * $Id$ + * + */ + +#ifndef JFFS2_SUMMARY_H +#define JFFS2_SUMMARY_H + +#include <linux/uio.h> +#include <linux/jffs2.h> + +#define DIRTY_SPACE(x) do { typeof(x) _x = (x); \ + c->free_size -= _x; c->dirty_size += _x; \ + jeb->free_size -= _x ; jeb->dirty_size += _x; \ + }while(0) +#define USED_SPACE(x) do { typeof(x) _x = (x); \ + c->free_size -= _x; c->used_size += _x; \ + jeb->free_size -= _x ; jeb->used_size += _x; \ + }while(0) +#define WASTED_SPACE(x) do { typeof(x) _x = (x); \ + c->free_size -= _x; c->wasted_size += _x; \ + jeb->free_size -= _x ; jeb->wasted_size += _x; \ + }while(0) +#define UNCHECKED_SPACE(x) do { typeof(x) _x = (x); \ + c->free_size -= _x; c->unchecked_size += _x; \ + jeb->free_size -= _x ; jeb->unchecked_size += _x; \ + }while(0) + +#define BLK_STATE_ALLFF 0 +#define BLK_STATE_CLEAN 1 +#define BLK_STATE_PARTDIRTY 2 +#define BLK_STATE_CLEANMARKER 3 +#define BLK_STATE_ALLDIRTY 4 +#define BLK_STATE_BADBLOCK 5 + +#define JFFS2_SUMMARY_NOSUM_SIZE 0xffffffff +#define JFFS2_SUMMARY_INODE_SIZE (sizeof(struct jffs2_sum_inode_flash)) +#define JFFS2_SUMMARY_DIRENT_SIZE(x) (sizeof(struct jffs2_sum_dirent_flash) + (x)) + +struct jffs2_sum_unknown_flash +{ + jint16_t nodetype; /* node type */ +}; + +struct jffs2_sum_inode_flash +{ + jint16_t nodetype; /* node type */ + jint32_t inode; /* inode number */ + jint32_t version; /* inode version */ + jint32_t offset; /* offset on jeb */ + jint32_t totlen; /* record length */ +} __attribute__((packed)); + +struct jffs2_sum_dirent_flash +{ + jint16_t nodetype; /* == JFFS_NODETYPE_DIRENT */ + jint32_t totlen; /* record length */ + jint32_t offset; /* ofset on jeb */ + jint32_t pino; /* parent inode */ + jint32_t version; /* dirent version */ + jint32_t ino; /* == zero for unlink */ + uint8_t nsize; /* dirent name size */ + uint8_t type; /* dirent type */ + uint8_t name[0]; /* dirent name */ +} __attribute__((packed)); + +union jffs2_sum_flash{ + struct jffs2_sum_unknown_flash u; + struct jffs2_sum_inode_flash i; + struct jffs2_sum_dirent_flash d; +}; + +/* list version of jffs2_sum_*flash for kernel and sumtool */ +struct jffs2_sum_unknown_mem +{ + union jffs2_sum_mem *next; + jint16_t nodetype; /* node type */ + +}; + +struct jffs2_sum_inode_mem +{ + union jffs2_sum_mem *next; + jint16_t nodetype; /* node type */ + jint32_t inode; /* inode number */ + jint32_t version; /* inode version */ + jint32_t offset; /* offset on jeb */ + jint32_t totlen; /* record length */ +} __attribute__((packed)); + +struct jffs2_sum_dirent_mem +{ + union jffs2_sum_mem *next; + jint16_t nodetype; /* == JFFS_NODETYPE_DIRENT */ + jint32_t totlen; /* record length */ + jint32_t offset; /* ofset on jeb */ + jint32_t pino; /* parent inode */ + jint32_t version; /* dirent version */ + jint32_t ino; /* == zero for unlink */ + uint8_t nsize; /* dirent name size */ + uint8_t type; /* dirent type */ + uint8_t name[0]; /* dirent name */ +} __attribute__((packed)); + +union jffs2_sum_mem +{ + struct jffs2_sum_unknown_mem u; + struct jffs2_sum_inode_mem i; + struct jffs2_sum_dirent_mem d; +}; + +struct jffs2_sum_info +{ + uint32_t sum_size; + uint32_t sum_num; + uint32_t sum_padded; + union jffs2_sum_mem *sum_list; +}; + +struct jffs2_sum_marker +{ + jint32_t offset; + jint32_t erase_size; + jint32_t magic; +}; + +#define JFFS2_SUMMARY_FRAME_SIZE (sizeof(struct jffs2_summary_node)+sizeof(struct jffs2_sum_marker)+3) + +#if !(defined(SUM_TOOL) || defined(JFFS2DUMP)) + +int jffs2_sum_init(struct jffs2_sb_info *c); +void jffs2_sum_exit(struct jffs2_sb_info *c); + +int jffs2_sum_care_sum_collected(struct jffs2_eraseblock *jeb); + +void jffs2_sum_clean_collected(struct jffs2_eraseblock *jeb); +void jffs2_sum_clean_all_info(struct jffs2_sb_info *c); /* clean up all summary information in all jeb (umount) */ + +int jffs2_sum_add_padding_mem(struct jffs2_eraseblock *jeb, uint32_t size); +int jffs2_sum_add_inode_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_inode *ri, uint32_t ofs); +int jffs2_sum_add_dirent_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_dirent *rd, uint32_t ofs); +int jffs2_sum_add_kvec(struct jffs2_sb_info *c, const struct kvec *invecs, unsigned long count, uint32_t to); + +int jffs2_sum_scan_sumnode(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb, uint32_t ofs, uint32_t *pseudo_random); +int jffs2_sum_write_sumnode(struct jffs2_sb_info *c); + +#endif + +#endif /* JFFS2_SUMMARY_H */ diff --unified --new-file --recursive mtd/fs/jffs2/super-v24.c mtd-dec07/fs/jffs2/super-v24.c --- mtd/fs/jffs2/super-v24.c 2004-11-17 00:00:14.000000000 +0100 +++ mtd-dec07/fs/jffs2/super-v24.c 2004-12-07 16:03:02.000000000 +0100 @@ -24,6 +24,7 @@ #include <linux/mtd/mtd.h> #include "compr.h" #include "nodelist.h" +#include "summary.h" #ifndef MTD_BLOCK_MAJOR #define MTD_BLOCK_MAJOR 31 @@ -72,6 +73,9 @@ put_mtd_device(c->mtd); return NULL; } +#ifdef CONFIG_JFFS2_SUMMARY + jffs2_sum_init(c); +#endif return sb; } @@ -88,6 +92,10 @@ down(&c->alloc_sem); jffs2_flush_wbuf_pad(c); up(&c->alloc_sem); +#ifdef CONFIG_JFFS2_SUMMARY + jffs2_sum_clean_all_info(c); + jffs2_sum_exit(c); +#endif jffs2_free_ino_caches(c); jffs2_free_raw_node_refs(c); kfree(c->blocks); @@ -110,6 +118,9 @@ #ifdef CONFIG_JFFS2_FS_NAND " (NAND)" #endif +#ifdef CONFIG_JFFS2_SUMMARY + " (SUMMARY)" +#endif " (C) 2001-2003 Red Hat, Inc.\n"); #ifdef JFFS2_OUT_OF_KERNEL diff --unified --new-file --recursive mtd/fs/jffs2/super.c mtd-dec07/fs/jffs2/super.c --- mtd/fs/jffs2/super.c 2004-11-26 15:08:31.000000000 +0100 +++ mtd-dec07/fs/jffs2/super.c 2004-12-07 16:03:02.000000000 +0100 @@ -152,6 +152,10 @@ return ERR_PTR(ret); } +#ifdef CONFIG_JFFS2_SUMMARY + jffs2_sum_init(c); +#endif + sb->s_flags |= MS_ACTIVE; return sb; @@ -275,6 +279,10 @@ down(&c->alloc_sem); jffs2_flush_wbuf_pad(c); up(&c->alloc_sem); +#ifdef CONFIG_JFFS2_SUMMARY + jffs2_sum_clean_all_info(c); + jffs2_sum_exit(c); +#endif jffs2_free_ino_caches(c); jffs2_free_raw_node_refs(c); if (c->mtd->flags & MTD_NO_VIRTBLOCKS) @@ -312,6 +320,9 @@ #ifdef CONFIG_JFFS2_FS_NAND " (NAND)" #endif +#ifdef CONFIG_JFFS2_SUMMARY + " (SUMMARY) " +#endif " (C) 2001-2003 Red Hat, Inc.\n"); jffs2_inode_cachep = kmem_cache_create("jffs2_i", diff --unified --new-file --recursive mtd/fs/jffs2/wbuf.c mtd-dec07/fs/jffs2/wbuf.c --- mtd/fs/jffs2/wbuf.c 2004-11-26 15:08:31.000000000 +0100 +++ mtd-dec07/fs/jffs2/wbuf.c 2004-12-07 16:17:11.000000000 +0100 @@ -261,7 +261,7 @@ /* ... and get an allocation of space from a shiny new block instead */ - ret = jffs2_reserve_space_gc(c, end-start, &ofs, &len); + ret = jffs2_reserve_space_gc(c, end-start, &ofs, &len, JFFS2_SUMMARY_NOSUM_SIZE); if (ret) { printk(KERN_WARNING "Failed to allocate space for wbuf recovery. Data loss ensues.\n"); if (buf) @@ -601,8 +601,15 @@ uint32_t outvec_to = to; /* If not NAND flash, don't bother */ - if (!c->wbuf) + if (!c->wbuf){ +#ifdef CONFIG_JFFS2_SUMMARY + D1(printk("JFFS2.summary: NOT NAND MEMORY\n")); + if(jffs2_sum_add_kvec(c,invecs,count,(uint32_t) to)){ + printk("jffs2_sum_add_kvec(): MEMORY ALLOCATION ERROR!"); + } +#endif return jffs2_flash_direct_writev(c, invecs, count, to, retlen); + } down_write(&c->wbuf_sem); @@ -801,6 +808,12 @@ alldone: *retlen = donelen; +#ifdef CONFIG_JFFS2_SUMMARY + if(jffs2_sum_add_kvec(c,invecs,count,(uint32_t) to)){ + printk("jffs2_sum_add_kvec(): MEMORY ALLOCATION ERROR!"); + } +#endif + if (c->wbuf_len && ino) jffs2_wbuf_dirties_inode(c, ino); diff --unified --new-file --recursive mtd/fs/jffs2/write.c mtd-dec07/fs/jffs2/write.c --- mtd/fs/jffs2/write.c 2004-11-17 00:00:14.000000000 +0100 +++ mtd-dec07/fs/jffs2/write.c 2004-12-07 16:03:02.000000000 +0100 @@ -173,13 +173,13 @@ D1(ACCT_PARANOIA_CHECK(jeb)); if (alloc_mode == ALLOC_GC) { - ret = jffs2_reserve_space_gc(c, sizeof(*ri) + datalen, &flash_ofs, &dummy); + ret = jffs2_reserve_space_gc(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, JFFS2_SUMMARY_INODE_SIZE); } else { /* Locking pain */ up(&f->sem); jffs2_complete_reservation(c); - ret = jffs2_reserve_space(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, alloc_mode); + ret = jffs2_reserve_space(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, alloc_mode, JFFS2_SUMMARY_INODE_SIZE); down(&f->sem); } @@ -308,13 +308,13 @@ D1(ACCT_PARANOIA_CHECK(jeb)); if (alloc_mode == ALLOC_GC) { - ret = jffs2_reserve_space_gc(c, sizeof(*rd) + namelen, &flash_ofs, &dummy); + ret = jffs2_reserve_space_gc(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, JFFS2_SUMMARY_DIRENT_SIZE(namelen)); } else { /* Locking pain */ up(&f->sem); jffs2_complete_reservation(c); - ret = jffs2_reserve_space(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, alloc_mode); + ret = jffs2_reserve_space(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, alloc_mode, JFFS2_SUMMARY_DIRENT_SIZE(namelen)); down(&f->sem); } @@ -371,7 +371,7 @@ retry: D2(printk(KERN_DEBUG "jffs2_commit_write() loop: 0x%x to write to 0x%x\n", writelen, offset)); - ret = jffs2_reserve_space(c, sizeof(*ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, ALLOC_NORMAL); + ret = jffs2_reserve_space(c, sizeof(*ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE); if (ret) { D1(printk(KERN_DEBUG "jffs2_reserve_space returned %d\n", ret)); break; @@ -458,7 +458,7 @@ /* Try to reserve enough space for both node and dirent. * Just the node will do for now, though */ - ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL); + ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE); D1(printk(KERN_DEBUG "jffs2_do_create(): reserved 0x%x bytes\n", alloclen)); if (ret) { up(&f->sem); @@ -487,7 +487,7 @@ up(&f->sem); jffs2_complete_reservation(c); - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL); + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen)); if (ret) { /* Eep. */ @@ -557,7 +557,7 @@ if (!rd) return -ENOMEM; - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_DELETION); + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_DELETION, JFFS2_SUMMARY_DIRENT_SIZE(namelen)); if (ret) { jffs2_free_raw_dirent(rd); return ret; @@ -663,7 +663,7 @@ if (!rd) return -ENOMEM; - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL); + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen)); if (ret) { jffs2_free_raw_dirent(rd); return ret; diff --unified --new-file --recursive mtd/include/linux/jffs2.h mtd-dec07/include/linux/jffs2.h --- mtd/include/linux/jffs2.h 2004-11-17 00:00:15.000000000 +0100 +++ mtd-dec07/include/linux/jffs2.h 2004-12-07 16:03:02.000000000 +0100 @@ -28,6 +28,9 @@ #define JFFS2_EMPTY_BITMASK 0xffff #define JFFS2_DIRTY_BITMASK 0x0000 +/* Summary node MAGIC marker */ +#define JFFS2_SUM_MAGIC 0x02851885 + /* We only allow a single char for length, and 0xFF is empty flash so we don't want it confused with a real length. Hence max 254. */ @@ -62,6 +65,8 @@ #define JFFS2_NODETYPE_CLEANMARKER (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3) #define JFFS2_NODETYPE_PADDING (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 4) +#define JFFS2_NODETYPE_SUMMARY (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 6) + // Maybe later... //#define JFFS2_NODETYPE_CHECKPOINT (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3) //#define JFFS2_NODETYPE_OPTIONS (JFFS2_FEATURE_RWCOMPAT_COPY | JFFS2_NODE_ACCURATE | 4) @@ -148,10 +153,24 @@ uint8_t data[0]; } __attribute__((packed)); +struct jffs2_summary_node{ + jint16_t magic; + jint16_t nodetype; /* = JFFS2_NODETYPE_INODE_SUM */ + jint32_t totlen; + jint32_t hdr_crc; + jint16_t sum_num; /* number of sum entries*/ + jint32_t cln_mkr; /* clean marker size, 0 = no cleanmarker */ + jint32_t padded; /* sum of the size of padding nodes */ + jint32_t sum_crc; /* summary information crc */ + jint32_t node_crc; /* node crc */ + jint32_t sum[0]; /* inode summary info */ +} __attribute__((packed)); + union jffs2_node_union { struct jffs2_raw_inode i; struct jffs2_raw_dirent d; struct jffs2_unknown_node u; + struct jffs2_summary_node s; }; #endif /* __LINUX_JFFS2_H__ */ diff --unified --new-file --recursive mtd/include/linux/jffs2_fs_sb.h mtd-dec07/include/linux/jffs2_fs_sb.h --- mtd/include/linux/jffs2_fs_sb.h 2004-11-26 15:08:31.000000000 +0100 +++ mtd-dec07/include/linux/jffs2_fs_sb.h 2004-12-07 16:03:02.000000000 +0100 @@ -110,6 +110,9 @@ uint32_t fsdata_pos; uint32_t fsdata_len; #endif +#ifdef CONFIG_JFFS2_SUMMARY + jint32_t *summary_buf; +#endif /* OS-private pointer for getting back to master superblock info */ void *os_priv; diff --unified --new-file --recursive mtd/util/Makefile mtd-dec07/util/Makefile --- mtd/util/Makefile 2004-07-14 00:00:18.000000000 +0200 +++ mtd-dec07/util/Makefile 2004-12-07 16:03:02.000000000 +0100 @@ -13,7 +13,7 @@ TARGETS = ftl_format flash_erase flash_eraseall nanddump doc_loadbios \ mkfs.jffs ftl_check mkfs.jffs2 flash_lock flash_unlock \ flash_info mtd_debug flashcp nandwrite jffs2dump \ - nftldump nftl_format docfdisk #jffs2reader + nftldump nftl_format docfdisk sumtool #jffs2reader SYMLINKS = compr_lzari.c compr_lzo.c @@ -48,6 +48,10 @@ jffs2dump: jffs2dump.o crc32.o $(CC) $(LDFLAGS) -o $@ $^ +sumtool: sumtool.o crc32.o + $(CC) $(LDFLAGS) -o $@ $^ + + install: ${TARGETS} mkdir -p ${DESTDIR}/${SBINDIR} install -m0755 -oroot -groot ${TARGETS} ${DESTDIR}/${SBINDIR}/ diff --unified --new-file --recursive mtd/util/jffs2dump.c mtd-dec07/util/jffs2dump.c --- mtd/util/jffs2dump.c 2004-12-03 00:00:10.000000000 +0100 +++ mtd-dec07/util/jffs2dump.c 2004-12-07 16:03:02.000000000 +0100 @@ -35,6 +35,10 @@ #include <getopt.h> #include "crc32.h" +#define CONFIG_JFFS2_SUMMARY +#define JFFS2DUMP +#include "../fs/jffs2/summary.h" + #define PROGRAM "jffs2dump" #define VERSION "$Revision: 1.7 $" @@ -177,7 +181,7 @@ */ void do_dumpcontent (void) { - char *p = data; + char *p = data, *p_free_begin; union jffs2_node_union *node; int empty = 0, dirty = 0; char name[256]; @@ -186,16 +190,21 @@ int bitchbitmask = 0; int obsolete; + p_free_begin = NULL; while ( p < (data + imglen)) { node = (union jffs2_node_union*) p; /* Skip empty space */ + if (!p_free_begin) p_free_begin = p; if (je16_to_cpu (node->u.magic) == 0xFFFF && je16_to_cpu (node->u.nodetype) == 0xFFFF) { p += 4; empty += 4; continue; } - + + if (p != p_free_begin) printf("Empty space found from 0x%08x to 0x%08x\n", p_free_begin-data, p-data); + p_free_begin = NULL; + if (je16_to_cpu (node->u.magic) != JFFS2_MAGIC_BITMASK) { if (!bitchbitmask++) printf ("Wrong bitmask at 0x%08x, 0x%04x\n", p - data, je16_to_cpu (node->u.magic)); @@ -277,7 +286,110 @@ p += PAD(je32_to_cpu (node->d.totlen)); break; - + + case JFFS2_NODETYPE_SUMMARY:{ + + int i; + struct jffs2_sum_marker * sm; + + printf ("%8s Inode Sum node at 0x%08x, totlen 0x%08x, sum_num %5d, cleanmarker size %5d\n", + obsolete ? "Obsolete" : "", + p - data, + je32_to_cpu (node->s.totlen), + je16_to_cpu (node->s.sum_num), + je32_to_cpu (node->s.cln_mkr)); + + crc = crc32 (0, node, sizeof (struct jffs2_summary_node) - 8); + if (crc != je32_to_cpu (node->s.node_crc)) { + printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.node_crc), crc); + p += PAD(je32_to_cpu (node->s.totlen)); + dirty += PAD(je32_to_cpu (node->s.totlen));; + continue; + } + + crc = crc32(0, p + sizeof (struct jffs2_summary_node), je32_to_cpu (node->s.totlen) - sizeof(struct jffs2_summary_node)); + if (crc != je32_to_cpu(node->s.sum_crc)) { + printf ("Wrong data_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.sum_crc), crc); + p += PAD(je32_to_cpu (node->s.totlen)); + dirty += PAD(je32_to_cpu (node->s.totlen));; + continue; + } + + if(verbose){ + struct jffs2_sum_unknown_flash *sp; + uint8_t *temp8ptr = NULL; + sp = (struct jffs2_sum_unknown_flash *) (p + sizeof (struct jffs2_summary_node)); + + for(i = 0; i < je16_to_cpu (node->s.sum_num); i++){ + + switch(je16_to_cpu(sp->nodetype)){ + case JFFS2_NODETYPE_INODE : { + + struct jffs2_sum_inode_flash *spi; + spi = (struct jffs2_sum_inode_flash *) sp; + + printf ("%14s #ino %5d, version %5d, offset 0x%08x, totlen 0x%08x\n", + "", + je32_to_cpu (spi->inode), + je32_to_cpu (spi->version), + je32_to_cpu (spi->offset), + je32_to_cpu (spi->totlen)); + + temp8ptr = (uint8_t *) sp; + temp8ptr += JFFS2_SUMMARY_INODE_SIZE; + sp = (struct jffs2_sum_unknown_flash *) temp8ptr; + + break; + } + + case JFFS2_NODETYPE_DIRENT : { + + char name[255]; + struct jffs2_sum_dirent_flash *spd; + spd = (struct jffs2_sum_dirent_flash *) sp; + + memcpy(name,spd->name,spd->nsize); + name [spd->nsize] = 0x0; + + printf ("%14s dirent offset 0x%08x, totlen 0x%08x, #pino %5d, version %5d, #ino %8d, nsize %8d, name %s \n", + "", + je32_to_cpu (spd->offset), + je32_to_cpu (spd->totlen), + je32_to_cpu (spd->pino), + je32_to_cpu (spd->version), + je32_to_cpu (spd->ino), + spd->nsize, + name); + + temp8ptr = (uint8_t *) sp; + temp8ptr += JFFS2_SUMMARY_DIRENT_SIZE(spd->nsize); + sp = (struct jffs2_sum_unknown_flash *) temp8ptr; + + break; + } + + default : { + break; + } + } + } + + + sm = (struct jffs2_sum_marker *) ((char *)p + je32_to_cpu(node->s.totlen) - sizeof(struct jffs2_sum_marker)); + + printf("%14s Sum Node Offset 0x%08x, Erase_block_size 0x%08x, Magic 0x%08x, Padded size 0x%08x\n" , + "", + je32_to_cpu(sm->offset), + je32_to_cpu(sm->erase_size), + je32_to_cpu(sm->magic), + je32_to_cpu(node->s.padded)); + } + + p += PAD(je32_to_cpu (node->s.totlen)); + break; + + } + case JFFS2_NODETYPE_CLEANMARKER: if (verbose) { printf ("%8s Cleanmarker at 0x%08x, totlen 0x%08x\n", diff --unified --new-file --recursive mtd/util/sumtool.c mtd-dec07/util/sumtool.c --- mtd/util/sumtool.c 1970-01-01 01:00:00.000000000 +0100 +++ mtd-dec07/util/sumtool.c 2004-12-07 17:47:31.000000000 +0100 @@ -0,0 +1,857 @@ +/* + * sumtool.c + * + * Copyright (C) 2004 Zoltan Sogor <weth@inf.u-szeged.hu>, + * Ferenc Havasi <havasi@inf.u-szeged.hu>, + * Patrik Kluba <pajko@halom.u-szeged.hu>, + * University of Szeged, Hungary + * + * $Id: sumtool.c,v 1.2 2004/10/20 09:56:08 hafy Exp $ + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version 2 + * of the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. + * + * Overview: + * This is a utility to reorder nodes and insert inode summary information + * into JFFS2 image for faster mount time - specially on NAND. + * + */ + +#include <errno.h> +#include <stdint.h> +#include <stdio.h> +#include <stdlib.h> +#include <stdarg.h> +#include <string.h> +#include <unistd.h> +#include <fcntl.h> +#include <time.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <sys/param.h> +#include <asm/types.h> +#include <dirent.h> +#include <mtd/jffs2-user.h> +#include <endian.h> +#include <byteswap.h> +#include <getopt.h> +#include "crc32.h" + +#define CONFIG_JFFS2_SUMMARY +#define SUM_TOOL +#include "../fs/jffs2/summary.h" + + +#define PAD(x) (((x)+3)&~3) + +static const char *const app_name = "sumtool"; + +static struct jffs2_sum_info *sum_collected = NULL;/* summary info list */ + +static int verbose = 0; +static int padto = 0; /* pad the output with 0xFF to the end of the final eraseblock */ +static int add_cleanmarkers = 1; /* add cleanmarker to output */ +static int use_input_cleanmarker_size = 1; /* use input file's cleanmarker size (default) */ +static int found_cleanmarkers = 0; /* cleanmarker found in input file */ +static struct jffs2_unknown_node cleanmarker; +static int cleanmarker_size = sizeof(cleanmarker); +static const char *short_options = "o:i:e:hvVblnc:p"; +static int erase_block_size = 65536; +static int target_endian = __BYTE_ORDER; +static int out_fd = -1; +static int in_fd = -1; + +static uint8_t *data_buffer = NULL; /* buffer for inodes */ +static unsigned int data_ofs = 0; /* inode buffer offset */ + +static uint8_t *file_buffer = NULL; /* file buffer contains the actual erase block*/ +static unsigned int file_ofs = 0; /* position in the buffer */ + +static struct option long_options[] = { + {"output", 1, NULL, 'o'}, + {"input", 1, NULL, 'i'}, + {"eraseblock", 1, NULL, 'e'}, + {"help", 0, NULL, 'h'}, + {"verbose", 0, NULL, 'v'}, + {"version", 0, NULL, 'V'}, + {"bigendian", 0, NULL, 'b'}, + {"littleendian", 0, NULL, 'l'}, + {"no-cleanmarkers", 0, NULL, 'n'}, + {"cleanmarker", 1, NULL, 'c'}, + {"pad", 0, NULL, 'p'}, + {NULL, 0, NULL, 0} +}; + +static char *helptext = + "Usage: sumtool [OPTIONS] -i inputfile -o outputfile\n" + "Convert the input JFFS2 file to a SUM-ed JFFS2 file\n\n" + "Options:\n" + " -e, --eraseblock=SIZE Use erase block size SIZE (default: 64KiB)\n" + " (usually 16KiB on NAND)\n" + " -c, --cleanmarker=SIZE Size of cleanmarker (default 12).\n" + " (usually 16 bytes on NAND, and will be set to\n" + " this value if left at the default 12). Will be\n" + " stored in OOB after each physical page composing\n" + " a physical erase block.\n" + " -n, --no-cleanmarkers Don't add a cleanmarker to every eraseblock\n" + " -o, --output=FILE Output to FILE \n" + " -i, --input=FILE Input from FILE \n" + " -b, --bigendian Image is big endian\n" + " -l --littleendian Image is little endian\n" + " -h, --help Display this help text\n" + " -v, --verbose Verbose operation\n" + " -V, --version Display version information\n" + " -p, --pad Pad the OUTPUT with 0xFF to the end of the final\n" + " eraseblock\n\n"; + + +static char *revtext = "$Revision: 1.3 $"; + +static unsigned char ffbuf[16] = { + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff +}; + +static void verror_msg(const char *s, va_list p) { + fflush(stdout); + fprintf(stderr, "%s: ", app_name); + vfprintf(stderr, s, p); +} + +static void error_msg_and_die(const char *s, ...) { + va_list p; + + va_start(p, s); + verror_msg(s, p); + va_end(p); + putc('\n', stderr); + exit(EXIT_FAILURE); +} + +static void vperror_msg(const char *s, va_list p) { + int err = errno; + + if (s == 0) + s = ""; + verror_msg(s, p); + if (*s) + s = ": "; + fprintf(stderr, "%s%s\n", s, strerror(err)); +} + +static void perror_msg_and_die(const char *s, ...) { + va_list p; + + va_start(p, s); + vperror_msg(s, p); + va_end(p); + exit(EXIT_FAILURE); +} + + + +static void full_write(void *target_buff, const void *buf, int len); + +void setup_cleanmarker() { + + cleanmarker.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK); + cleanmarker.nodetype = cpu_to_je16(JFFS2_NODETYPE_CLEANMARKER); + cleanmarker.totlen = cpu_to_je32(cleanmarker_size); + cleanmarker.hdr_crc = cpu_to_je32(crc32(0, &cleanmarker, sizeof(struct jffs2_unknown_node)-4)); +} + +void process_options (int argc, char **argv){ + int opt,c; + + while ((opt = getopt_long(argc, argv, short_options, long_options, &c)) >= 0) + { + switch (opt) + { + case 'o': + if (out_fd != -1) { + error_msg_and_die("output filename specified more than once"); + } + out_fd = open(optarg, O_CREAT | O_TRUNC | O_RDWR, 0644); + if (out_fd == -1) { + perror_msg_and_die("open output file"); + } + break; + + case 'i': + if (in_fd != -1) { + error_msg_and_die("input filename specified more than once"); + } + in_fd = open(optarg, O_RDONLY); + if (in_fd == -1) { + perror_msg_and_die("open input file"); + } + break; + case 'b': + target_endian = __BIG_ENDIAN; + break; + case 'l': + target_endian = __LITTLE_ENDIAN; + break; + case 'h': + case '?': + error_msg_and_die(helptext); + + case 'v': + verbose = 1; + break; + + case 'V': + error_msg_and_die("revision %.*s\n", + (int) strlen(revtext) - 13, revtext + 11); + + case 'e': { + char *next; + unsigned units = 0; + erase_block_size = strtol(optarg, &next, 0); + if (!erase_block_size) + error_msg_and_die("Unrecognisable erase size\n"); + + if (*next) { + if (!strcmp(next, "KiB")) { + units = 1024; + } else if (!strcmp(next, "MiB")) { + units = 1024 * 1024; + } else { + error_msg_and_die("Unknown units in erasesize\n"); + } + } else { + if (erase_block_size < 0x1000) + units = 1024; + else + units = 1; + } + erase_block_size *= units; + + /* If it's less than 8KiB, they're not allowed */ + if (erase_block_size < 0x2000) { + fprintf(stderr, "Erase size 0x%x too small. Increasing to 8KiB minimum\n", + erase_block_size); + erase_block_size = 0x2000; + } + break; + } + + case 'n': + add_cleanmarkers = 0; + break; + case 'c': + cleanmarker_size = strtol(optarg, NULL, 0); + + if (cleanmarker_size < sizeof(cleanmarker)) { + error_msg_and_die("cleanmarker size must be >= 12"); + } + if (cleanmarker_size >= erase_block_size) { + error_msg_and_die("cleanmarker size must be < eraseblock size"); + } + + use_input_cleanmarker_size = 0; + found_cleanmarkers = 1; + setup_cleanmarker(); + + break; + case 'p': + padto = 1; + break; + + } + } +} + + +void init_buffers() { + + data_buffer = malloc(erase_block_size); + + if (!data_buffer) { + perror("out of memory"); + close (in_fd); + close (out_fd); + exit(1); + } + + file_buffer = malloc(erase_block_size); + + if (!file_buffer) { + perror("out of memory"); + close (in_fd); + close (out_fd); + exit(1); + } +} + +void init_sumlist(){ + + sum_collected = (struct jffs2_sum_info *) malloc (sizeof(struct jffs2_sum_info)); + + if (!sum_collected) + error_msg_and_die("Can't allocate memory for jffs2_sum_info!\n"); + + sum_collected->sum_list = NULL; + sum_collected->sum_num = 0; + sum_collected->sum_size = 0; + +} + +void clean_buffers() { + + if (data_buffer) + free(data_buffer); + if (file_buffer) + free(file_buffer); +} + +void clean_sumlist() { + + union jffs2_sum_mem *temp; + + if(sum_collected){ + + while(sum_collected->sum_list){ + temp = sum_collected->sum_list; + sum_collected->sum_list = sum_collected->sum_list->u.next; + free(temp); + sum_collected->sum_num--; + } + + if(sum_collected->sum_num != 0) + printf("Ooops, something wrong happened! sum_num != 0, but sum_list = null ???"); + + free(sum_collected); + } +} + +int load_next_block() { + + int ret; + ret = read(in_fd, file_buffer, erase_block_size); + file_ofs = 0; + + if(verbose) + printf("Load next block : %d bytes read\n",ret); + + return ret; +} + +void write_buff_to_file() { + + int ret; + int len = data_ofs; + + uint8_t *buf = NULL; + + buf = data_buffer; + while (len > 0) { + ret = write(out_fd, buf, len); + + if (ret < 0) + perror_msg_and_die("write"); + + if (ret == 0) + perror_msg_and_die("write returned zero"); + + len -= ret; + buf += ret; + } + + data_ofs = 0; +} + +void dump_sum_records() { + + struct jffs2_summary_node isum; + union jffs2_sum_mem *temp; + jint32_t offset; + jint32_t *wpage; + uint8_t *tempptr; + int datasize; + int infosize; + int padsize; + jint32_t magic = cpu_to_je32(JFFS2_SUM_MAGIC); + + if (!sum_collected->sum_num || !sum_collected->sum_list) + return; + + datasize = sum_collected->sum_size + sizeof(struct jffs2_sum_marker); + infosize = sizeof(struct jffs2_summary_node) + datasize; + padsize = erase_block_size - data_ofs - infosize; + infosize += padsize; datasize += padsize; + offset = cpu_to_je32(data_ofs); + + jint32_t *tpage = (jint32_t *) malloc(datasize); + + if(!tpage) + error_msg_and_die("Can't allocate memory to dump summary information!\n"); + + memset(tpage, 0xff, datasize); + memset(&isum, 0, sizeof(isum)); + + isum.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK); + isum.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY); + isum.totlen = cpu_to_je32(infosize); + isum.hdr_crc = cpu_to_je32(crc32(0, &isum, sizeof(struct jffs2_unknown_node) - 4)); + isum.padded = cpu_to_je32(0); + + if (add_cleanmarkers && found_cleanmarkers) { + isum.cln_mkr = cpu_to_je32(cleanmarker_size); + } + else{ + isum.cln_mkr = cpu_to_je32(0); + } + + isum.sum_num = cpu_to_je16(sum_collected->sum_num); + wpage = tpage; + + + while (sum_collected->sum_num) { + + + switch(je16_to_cpu(sum_collected->sum_list->u.nodetype)){ + + case JFFS2_NODETYPE_INODE : { + jint16_t *temp16ptr = (jint16_t *)wpage; + + *(temp16ptr++) = sum_collected->sum_list->i.nodetype; + wpage = (jint32_t *) temp16ptr; + + *(wpage++) = sum_collected->sum_list->i.inode; + *(wpage++) = sum_collected->sum_list->i.version; + *(wpage++) = sum_collected->sum_list->i.offset; + *(wpage++) = sum_collected->sum_list->i.totlen; + break; + } + + case JFFS2_NODETYPE_DIRENT : { + jint16_t *temp16ptr = (jint16_t *) wpage; + uint8_t *temp8ptr = NULL; + + *(temp16ptr++) = sum_collected->sum_list->d.nodetype; + wpage = (jint32_t *) temp16ptr; + + *(wpage++) = sum_collected->sum_list->d.totlen; + *(wpage++) = sum_collected->sum_list->d.offset; + *(wpage++) = sum_collected->sum_list->d.pino; + *(wpage++) = sum_collected->sum_list->d.version; + *(wpage++) = sum_collected->sum_list->d.ino; + + temp8ptr = (uint8_t *) wpage; + *(temp8ptr++) = sum_collected->sum_list->d.nsize; + *(temp8ptr++) = sum_collected->sum_list->d.type; + + memcpy(temp8ptr,sum_collected->sum_list->d.name,sum_collected->sum_list->d.nsize); + temp8ptr += sum_collected->sum_list->d.nsize; + wpage = (jint32_t *) temp8ptr; + + break; + } + + default : { + printf("This is not good for me!!!!\n"); + } + } + + temp = sum_collected->sum_list; + sum_collected->sum_list = sum_collected->sum_list->u.next; + free(temp); + + sum_collected->sum_num--; + } + + sum_collected->sum_size = 0; + + tempptr = (uint8_t *) wpage; + tempptr += padsize; + wpage = (jint32_t *) tempptr; + + *(wpage++) = offset; + *(wpage++) = cpu_to_je32(erase_block_size); + *(wpage++) = magic; + isum.sum_crc = cpu_to_je32(crc32(0, tpage, datasize)); + isum.node_crc = cpu_to_je32(crc32(0, &isum, sizeof(isum) - 8)); + + full_write(data_buffer + data_ofs, &isum, sizeof(isum)); + full_write(data_buffer + data_ofs, tpage, datasize); + + free(tpage); + +} + +static void full_write(void *target_buff, const void *buf, int len) { + + memcpy(target_buff, buf, len); + data_ofs += len; +} + +static void pad(int req) { + while (req) { + if (req > sizeof(ffbuf)) { + full_write(data_buffer + data_ofs, ffbuf, sizeof(ffbuf)); + req -= sizeof(ffbuf); + } else { + full_write(data_buffer + data_ofs, ffbuf, req); + req = 0; + } + } +} + +static inline void padword() { + + if (data_ofs % 4) { + full_write(data_buffer + data_ofs, ffbuf, 4 - (data_ofs % 4)); + } +} + + +static inline void pad_block_if_less_than(int req,int plus) { + + int datasize = req + plus + sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8; + datasize += (4 - (datasize % 4)) % 4; + + if (data_ofs + req > erase_block_size - datasize) { + dump_sum_records(); + write_buff_to_file(); + } + + if (add_cleanmarkers && found_cleanmarkers) { + if (!data_ofs) { + full_write(data_buffer, &cleanmarker, sizeof(cleanmarker)); + pad(cleanmarker_size - sizeof(cleanmarker)); + padword(); + } + } +} + +void flush_buffers() { + + if ((add_cleanmarkers == 1) && (found_cleanmarkers == 1)) { /* CLEANMARKER */ + if (data_ofs != cleanmarker_size) { /* INODE BUFFER */ + + int datasize = sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8; + datasize += (4 - (datasize % 4)) % 4; + + /* If we have a full inode buffer, then write out inode and summary data */ + if (data_ofs + sizeof(struct jffs2_raw_inode) + 2*JFFS2_MIN_DATA_LEN > erase_block_size - datasize) { + dump_sum_records(); + write_buff_to_file(); + } + /* else just write out inode data */ + else{ + if(padto) + pad(erase_block_size - data_ofs); + write_buff_to_file(); + } + } + } + else { /* NO CLEANMARKER */ + if (data_ofs != 0) { /* INODE BUFFER */ + + int datasize = sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8; + datasize += (4 - (datasize % 4)) % 4; + + /* If we have a full inode buffer, then write out inode and summary data */ + if (data_ofs + sizeof(struct jffs2_raw_inode) + 2*JFFS2_MIN_DATA_LEN > erase_block_size - datasize) { + dump_sum_records(); + write_buff_to_file(); + } + /* Else just write out inode data */ + else{ + if(padto) + pad(erase_block_size - data_ofs); + write_buff_to_file(); + } + } + } +} + +int add_sum_mem(union jffs2_sum_mem *item) { + + union jffs2_sum_mem *walk; + + if (!sum_collected->sum_list) { + sum_collected->sum_list = (union jffs2_sum_mem *) item; + } + else { + walk = sum_collected->sum_list; + + while (walk->u.next) { + walk = walk->u.next; + } + walk->u.next = (union jffs2_sum_mem *) item; + } + switch (je16_to_cpu(item->u.nodetype)) { + case JFFS2_NODETYPE_INODE: + sum_collected->sum_size += JFFS2_SUMMARY_INODE_SIZE; + sum_collected->sum_num++; + break; + + case JFFS2_NODETYPE_DIRENT: + sum_collected->sum_size += JFFS2_SUMMARY_DIRENT_SIZE(item->d.nsize); + sum_collected->sum_num++; + break; + + default: + error_msg_and_die("__jffs2_add_sum_mem(): UNKNOWN node type %d\n", je16_to_cpu(item->u.nodetype)); + } + return 0; +} + +void add_sum_inode_mem(union jffs2_node_union *node) { + + struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *) malloc(sizeof(struct jffs2_sum_inode_mem)); + + if (!temp) + error_msg_and_die("Can't allocate memory for summary information!\n"); + + temp->nodetype = node->i.nodetype; + temp->inode = node->i.ino; + temp->version = node->i.version; + temp->offset = cpu_to_je32(data_ofs); + temp->totlen = node->i.totlen; + temp->next = NULL; + + add_sum_mem((union jffs2_sum_mem *) temp); + +} + +void add_sum_dirent_mem(union jffs2_node_union *node) { + + struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *) + malloc(sizeof(struct jffs2_sum_dirent_mem) + node->d.nsize); + + if (!temp) + error_msg_and_die("Can't allocate memory for summary information!\n"); + + temp->nodetype = node->d.nodetype; + temp->totlen = node->d.totlen; + temp->offset = cpu_to_je32(data_ofs); + temp->pino = node->d.pino; + temp->version = node->d.version; + temp->ino = node->d.ino; + temp->nsize = node->d.nsize; + temp->type = node->d.type; + temp->next = NULL; + + memcpy(temp->name,node->d.name,node->d.nsize); + + add_sum_mem((union jffs2_sum_mem *) temp); + +} + +void write_dirent_to_buff(union jffs2_node_union *node) { + + pad_block_if_less_than(je32_to_cpu (node->d.totlen),JFFS2_SUMMARY_DIRENT_SIZE(node->d.nsize)); + add_sum_dirent_mem(node); + full_write(data_buffer + data_ofs, &(node->d), je32_to_cpu (node->d.totlen)); + padword(); +} + + +void write_inode_to_buff(union jffs2_node_union *node) { + + pad_block_if_less_than(je32_to_cpu (node->i.totlen),JFFS2_SUMMARY_INODE_SIZE); + add_sum_inode_mem(node); /* Add inode summary mem to summary list */ + full_write(data_buffer + data_ofs, &(node->i), je32_to_cpu (node->i.totlen)); /* Write out the inode to inode_buffer */ + padword(); + +} + + +void create_summed_image(int inp_size) { + uint8_t *p = file_buffer; + union jffs2_node_union *node; + uint32_t crc; + uint16_t type; + int bitchbitmask = 0; + int obsolete; + + char name[256]; + + while ( p < (file_buffer + inp_size)) { + + node = (union jffs2_node_union *) p; + + /* Skip empty space */ + if (je16_to_cpu (node->u.magic) == 0xFFFF && je16_to_cpu (node->u.nodetype) == 0xFFFF) { + p += 4; + continue; + } + + if (je16_to_cpu (node->u.magic) != JFFS2_MAGIC_BITMASK) { + if (!bitchbitmask++) + printf ("Wrong bitmask at 0x%08x, 0x%04x\n", p - file_buffer, je16_to_cpu (node->u.magic)); + p += 4; + continue; + } + + bitchbitmask = 0; + + type = je16_to_cpu(node->u.nodetype); + if ((type & JFFS2_NODE_ACCURATE) != JFFS2_NODE_ACCURATE) { + obsolete = 1; + type |= JFFS2_NODE_ACCURATE; + } else + obsolete = 0; + + node->u.nodetype = cpu_to_je16(type); + + crc = crc32 (0, node, sizeof (struct jffs2_unknown_node) - 4); + if (crc != je32_to_cpu (node->u.hdr_crc)) { + printf ("Wrong hdr_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->u.hdr_crc), crc); + p += 4; + continue; + } + + switch(je16_to_cpu(node->u.nodetype)) { + + case JFFS2_NODETYPE_INODE: + if(verbose) + printf ("%8s Inode node at 0x%08x, totlen 0x%08x, #ino %5d, version %5d, isize %8d, csize %8d, dsize %8d, offset %8d\n", + obsolete ? "Obsolete" : "", + p - file_buffer, je32_to_cpu (node->i.totlen), je32_to_cpu (node->i.ino), + je32_to_cpu ( node->i.version), je32_to_cpu (node->i.isize), + je32_to_cpu (node->i.csize), je32_to_cpu (node->i.dsize), je32_to_cpu (node->i.offset)); + + crc = crc32 (0, node, sizeof (struct jffs2_raw_inode) - 8); + if (crc != je32_to_cpu (node->i.node_crc)) { + printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.node_crc), crc); + p += PAD(je32_to_cpu (node->i.totlen)); + continue; + } + + crc = crc32(0, p + sizeof (struct jffs2_raw_inode), je32_to_cpu(node->i.csize)); + if (crc != je32_to_cpu(node->i.data_crc)) { + printf ("Wrong data_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.data_crc), crc); + p += PAD(je32_to_cpu (node->i.totlen)); + continue; + } + + write_inode_to_buff(node); + + p += PAD(je32_to_cpu (node->i.totlen)); + break; + + case JFFS2_NODETYPE_DIRENT: + memcpy (name, node->d.name, node->d.nsize); + name [node->d.nsize] = 0x0; + + if(verbose) + printf ("%8s Dirent node at 0x%08x, totlen 0x%08x, #pino %5d, version %5d, #ino %8d, nsize %8d, name %s\n", + obsolete ? "Obsolete" : "", + p - file_buffer, je32_to_cpu (node->d.totlen), je32_to_cpu (node->d.pino), + je32_to_cpu ( node->d.version), je32_to_cpu (node->d.ino), + node->d.nsize, name); + + crc = crc32 (0, node, sizeof (struct jffs2_raw_dirent) - 8); + if (crc != je32_to_cpu (node->d.node_crc)) { + printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.node_crc), crc); + p += PAD(je32_to_cpu (node->d.totlen)); + continue; + } + + crc = crc32(0, p + sizeof (struct jffs2_raw_dirent), node->d.nsize); + if (crc != je32_to_cpu(node->d.name_crc)) { + printf ("Wrong name_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.name_crc), crc); + p += PAD(je32_to_cpu (node->d.totlen)); + continue; + } + + write_dirent_to_buff(node); + + p += PAD(je32_to_cpu (node->d.totlen)); + break; + + case JFFS2_NODETYPE_CLEANMARKER: + if (verbose) { + printf ("%8s Cleanmarker at 0x%08x, totlen 0x%08x\n", + obsolete ? "Obsolete" : "", + p - file_buffer, je32_to_cpu (node->u.totlen)); + } + + if(!found_cleanmarkers){ + found_cleanmarkers = 1; + + if(add_cleanmarkers == 1 && use_input_cleanmarker_size == 1){ + cleanmarker_size = je32_to_cpu (node->u.totlen); + setup_cleanmarker(); + } + } + + p += PAD(je32_to_cpu (node->u.totlen)); + break; + + case JFFS2_NODETYPE_PADDING: + if (verbose) { + printf ("%8s Padding node at 0x%08x, totlen 0x%08x\n", + obsolete ? "Obsolete" : "", + p - file_buffer, je32_to_cpu (node->u.totlen)); + } + p += PAD(je32_to_cpu (node->u.totlen)); + break; + + case 0xffff: + p += 4; + break; + + default: + if (verbose) { + printf ("%8s Unknown node at 0x%08x, totlen 0x%08x\n", + obsolete ? "Obsolete" : "", + p - file_buffer, je32_to_cpu (node->u.totlen)); + } + + //write_unknown_to_buff(node); + + p += PAD(je32_to_cpu (node->u.totlen)); + } + } +} + +int main(int argc, char **argv) { + + int ret; + + process_options(argc,argv); + + if ((in_fd == -1) || (out_fd == -1)) { + + if(in_fd != -1) + close(in_fd); + if(out_fd != -1) + close(out_fd); + + error_msg_and_die("You must specify input and output files!\n"); + } + + init_buffers(); + init_sumlist(); + + while ((ret = load_next_block())) { + create_summed_image(ret); + } + + flush_buffers(); + clean_buffers(); + clean_sumlist(); + + if (in_fd != -1) + close(in_fd); + if (out_fd != -1) + close(out_fd); + + return 0; +} ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-16 13:43 ` Ferenc Havasi @ 2004-12-20 16:01 ` Gareth Bult (Encryptec) 2004-12-20 16:09 ` Ferenc Havasi 0 siblings, 1 reply; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-20 16:01 UTC (permalink / raw) To: Ferenc Havasi; +Cc: Linux MTD Hi, I'm afraid this patch seems to be quite a long way from applying cleanly to my source. I'm using 2.6.9 .. at least half a dozen failures or more ... :( Will look into it in more detail later ... Gareth. On Thu, 2004-12-16 at 14:43 +0100, Ferenc Havasi wrote: > Hi Gareth, > > > However, I'm looking at 30secs to mount the root filesystem. > > Did you try our patch? I send it now again, we made it fresh for the > current CVS. See its usage in the archive (16th November). > > It will part of JFFS3. > > This is designed for NAND devices - I don't know too much about your > environment, but it may help. > > Bye, > Ferenc > plain text document attachment (jffs2-summary-20041207.patch) > diff --unified --new-file --recursive mtd/fs/Kconfig mtd-dec07/fs/Kconfig > --- mtd/fs/Kconfig 2004-11-04 00:00:11.000000000 +0100 > +++ mtd-dec07/fs/Kconfig 2004-12-07 16:03:02.000000000 +0100 > @@ -77,6 +77,19 @@ > ECC for JFFS2. This type of flash chip is not common, however it is > available from STMicro. > > +config JFFS2_SUMMARY > + bool "JFFS2 summary support (EXPERIMENTAL)" > + depends on JFFS2_FS > + default n > + help > + This feature makes it possible to use summary information > + for faster filesystem mount - specially on NAND. > + > + The summary information can be inserted into a filesystem image > + by the utility 'sumtool'. > + > + If unsure, say 'N'. > + > config JFFS2_COMPRESSION_OPTIONS > bool "Advanced compression options for JFFS2" > default n > diff --unified --new-file --recursive mtd/fs/jffs2/Makefile.common mtd-dec07/fs/jffs2/Makefile.common > --- mtd/fs/jffs2/Makefile.common 2004-11-04 00:00:11.000000000 +0100 > +++ mtd-dec07/fs/jffs2/Makefile.common 2004-12-07 16:03:02.000000000 +0100 > @@ -16,3 +16,4 @@ > jffs2-$(CONFIG_JFFS2_RUBIN) += compr_rubin.o > jffs2-$(CONFIG_JFFS2_RTIME) += compr_rtime.o > jffs2-$(CONFIG_JFFS2_ZLIB) += compr_zlib.o > +jffs2-$(CONFIG_JFFS2_SUMMARY) += summary.o > diff --unified --new-file --recursive mtd/fs/jffs2/build.c mtd-dec07/fs/jffs2/build.c > --- mtd/fs/jffs2/build.c 2004-11-28 00:00:13.000000000 +0100 > +++ mtd-dec07/fs/jffs2/build.c 2004-12-07 16:08:31.000000000 +0100 > @@ -331,6 +331,10 @@ > c->blocks[i].first_node = NULL; > c->blocks[i].last_node = NULL; > c->blocks[i].bad_count = 0; > + #ifdef CONFIG_JFFS2_SUMMARY > + c->blocks[i].sum_collected = NULL; > + #endif > + > } > > init_MUTEX(&c->alloc_sem); > diff --unified --new-file --recursive mtd/fs/jffs2/dir.c mtd-dec07/fs/jffs2/dir.c > --- mtd/fs/jffs2/dir.c 2004-11-17 00:00:14.000000000 +0100 > +++ mtd-dec07/fs/jffs2/dir.c 2004-12-07 16:03:02.000000000 +0100 > @@ -22,6 +22,7 @@ > #include <linux/time.h> > #include "nodelist.h" > > + > /* Urgh. Please tell me there's a nicer way of doing these. */ > #include <linux/version.h> > #if LINUX_VERSION_CODE < KERNEL_VERSION(2,5,48) > @@ -314,7 +315,7 @@ > * Just the node will do for now, though > */ > namelen = dentry->d_name.len; > - ret = jffs2_reserve_space(c, sizeof(*ri) + strlen(target), &phys_ofs, &alloclen, ALLOC_NORMAL); > + ret = jffs2_reserve_space(c, sizeof(*ri) + strlen(target), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE); > > if (ret) { > jffs2_free_raw_inode(ri); > @@ -360,7 +361,7 @@ > up(&f->sem); > > jffs2_complete_reservation(c); > - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL); > + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen)); > if (ret) { > /* Eep. */ > jffs2_clear_inode(inode); > @@ -445,7 +446,7 @@ > * Just the node will do for now, though > */ > namelen = dentry->d_name.len; > - ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL); > + ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE); > > if (ret) { > jffs2_free_raw_inode(ri); > @@ -488,7 +489,7 @@ > up(&f->sem); > > jffs2_complete_reservation(c); > - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL); > + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen)); > if (ret) { > /* Eep. */ > jffs2_clear_inode(inode); > @@ -597,7 +598,7 @@ > * Just the node will do for now, though > */ > namelen = dentry->d_name.len; > - ret = jffs2_reserve_space(c, sizeof(*ri) + devlen, &phys_ofs, &alloclen, ALLOC_NORMAL); > + ret = jffs2_reserve_space(c, sizeof(*ri) + devlen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE); > > if (ret) { > jffs2_free_raw_inode(ri); > @@ -642,7 +643,7 @@ > up(&f->sem); > > jffs2_complete_reservation(c); > - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL); > + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen)); > if (ret) { > /* Eep. */ > jffs2_clear_inode(inode); > @@ -796,4 +797,3 @@ > > return 0; > } > - > diff --unified --new-file --recursive mtd/fs/jffs2/file.c mtd-dec07/fs/jffs2/file.c > --- mtd/fs/jffs2/file.c 2004-11-17 00:00:14.000000000 +0100 > +++ mtd-dec07/fs/jffs2/file.c 2004-12-07 16:03:02.000000000 +0100 > @@ -22,6 +22,7 @@ > #include <linux/jffs2.h> > #include "nodelist.h" > > + > extern int generic_file_open(struct inode *, struct file *) __attribute__((weak)); > extern loff_t generic_file_llseek(struct file *file, loff_t offset, int origin) __attribute__((weak)); > > @@ -135,7 +136,7 @@ > D1(printk(KERN_DEBUG "Writing new hole frag 0x%x-0x%x between current EOF and new page\n", > (unsigned int)inode->i_size, pageofs)); > > - ret = jffs2_reserve_space(c, sizeof(ri), &phys_ofs, &alloc_len, ALLOC_NORMAL); > + ret = jffs2_reserve_space(c, sizeof(ri), &phys_ofs, &alloc_len, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE); > if (ret) > return ret; > > diff --unified --new-file --recursive mtd/fs/jffs2/fs.c mtd-dec07/fs/jffs2/fs.c > --- mtd/fs/jffs2/fs.c 2004-11-29 00:00:09.000000000 +0100 > +++ mtd-dec07/fs/jffs2/fs.c 2004-12-07 16:03:02.000000000 +0100 > @@ -74,7 +74,7 @@ > return -ENOMEM; > } > > - ret = jffs2_reserve_space(c, sizeof(*ri) + mdatalen, &phys_ofs, &alloclen, ALLOC_NORMAL); > + ret = jffs2_reserve_space(c, sizeof(*ri) + mdatalen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE); > if (ret) { > jffs2_free_raw_inode(ri); > if (S_ISLNK(inode->i_mode & S_IFMT)) > diff --unified --new-file --recursive mtd/fs/jffs2/gc.c mtd-dec07/fs/jffs2/gc.c > --- mtd/fs/jffs2/gc.c 2004-11-17 00:00:14.000000000 +0100 > +++ mtd-dec07/fs/jffs2/gc.c 2004-12-07 16:03:02.000000000 +0100 > @@ -506,7 +506,7 @@ > don't want to force wastage of the end of a block if splitting would > work. */ > ret = jffs2_reserve_space_gc(c, min_t(uint32_t, sizeof(struct jffs2_raw_inode) + JFFS2_MIN_DATA_LEN, > - rawlen), &phys_ofs, &alloclen); > + rawlen), &phys_ofs, &alloclen, rawlen); /* this is not optimal yet */ > if (ret) > return ret; > > @@ -614,7 +614,7 @@ > ACCT_SANITY_CHECK(c,jeb); > D1(ACCT_PARANOIA_CHECK(jeb)); > > - ret = jffs2_reserve_space_gc(c, rawlen, &phys_ofs, &dummy); > + ret = jffs2_reserve_space_gc(c, rawlen, &phys_ofs, &dummy, rawlen); /* this is not optimal yet */ > > if (!ret) { > D1(printk(KERN_DEBUG "Allocated space at 0x%08x to retry failed write.\n", phys_ofs)); > @@ -692,7 +692,7 @@ > > } > > - ret = jffs2_reserve_space_gc(c, sizeof(ri) + mdatalen, &phys_ofs, &alloclen); > + ret = jffs2_reserve_space_gc(c, sizeof(ri) + mdatalen, &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE); > if (ret) { > printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_metadata failed: %d\n", > sizeof(ri)+ mdatalen, ret); > @@ -759,7 +759,7 @@ > rd.node_crc = cpu_to_je32(crc32(0, &rd, sizeof(rd)-8)); > rd.name_crc = cpu_to_je32(crc32(0, fd->name, rd.nsize)); > > - ret = jffs2_reserve_space_gc(c, sizeof(rd)+rd.nsize, &phys_ofs, &alloclen); > + ret = jffs2_reserve_space_gc(c, sizeof(rd)+rd.nsize, &phys_ofs, &alloclen, JFFS2_SUMMARY_DIRENT_SIZE(rd.nsize)); > if (ret) { > printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_dirent failed: %d\n", > sizeof(rd)+rd.nsize, ret); > @@ -961,7 +961,7 @@ > ri.data_crc = cpu_to_je32(0); > ri.node_crc = cpu_to_je32(crc32(0, &ri, sizeof(ri)-8)); > > - ret = jffs2_reserve_space_gc(c, sizeof(ri), &phys_ofs, &alloclen); > + ret = jffs2_reserve_space_gc(c, sizeof(ri), &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE); > if (ret) { > printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_hole failed: %d\n", > sizeof(ri), ret); > @@ -1186,7 +1186,7 @@ > uint32_t cdatalen; > uint16_t comprtype = JFFS2_COMPR_NONE; > > - ret = jffs2_reserve_space_gc(c, sizeof(ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen); > + ret = jffs2_reserve_space_gc(c, sizeof(ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE); > > if (ret) { > printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_dnode failed: %d\n", > @@ -1243,4 +1243,3 @@ > jffs2_gc_release_page(c, pg_ptr, &pg); > return ret; > } > - > diff --unified --new-file --recursive mtd/fs/jffs2/nodelist.h mtd-dec07/fs/jffs2/nodelist.h > --- mtd/fs/jffs2/nodelist.h 2004-11-26 15:08:31.000000000 +0100 > +++ mtd-dec07/fs/jffs2/nodelist.h 2004-12-07 16:03:02.000000000 +0100 > @@ -20,6 +20,7 @@ > #include <linux/jffs2.h> > #include <linux/jffs2_fs_sb.h> > #include <linux/jffs2_fs_i.h> > +#include "summary.h" > > #ifdef __ECOS > #include "os-ecos.h" > @@ -194,6 +195,10 @@ > int bad_count; > uint32_t offset; /* of this block in the MTD */ > > +#ifdef CONFIG_JFFS2_SUMMARY > + struct jffs2_sum_info *sum_collected; > +#endif > + > uint32_t unchecked_size; > uint32_t used_size; > uint32_t dirty_size; > @@ -392,8 +397,8 @@ > > /* nodemgmt.c */ > int jffs2_thread_should_wake(struct jffs2_sb_info *c); > -int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio); > -int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len); > +int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio, uint32_t sumsize); > +int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize); > int jffs2_add_physical_node_ref(struct jffs2_sb_info *c, struct jffs2_raw_node_ref *new); > void jffs2_complete_reservation(struct jffs2_sb_info *c); > void jffs2_mark_node_obsolete(struct jffs2_sb_info *c, struct jffs2_raw_node_ref *raw); > @@ -455,6 +460,10 @@ > /* scan.c */ > int jffs2_scan_medium(struct jffs2_sb_info *c); > void jffs2_rotate_lists(struct jffs2_sb_info *c); > +int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf, > + uint32_t ofs, uint32_t len); > +struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino); > + > > /* build.c */ > int jffs2_do_mount_fs(struct jffs2_sb_info *c); > diff --unified --new-file --recursive mtd/fs/jffs2/nodemgmt.c mtd-dec07/fs/jffs2/nodemgmt.c > --- mtd/fs/jffs2/nodemgmt.c 2004-11-26 15:08:31.000000000 +0100 > +++ mtd-dec07/fs/jffs2/nodemgmt.c 2004-12-07 16:03:02.000000000 +0100 > @@ -38,9 +38,9 @@ > * for the requested allocation. > */ > > -static int jffs2_do_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len); > +static int jffs2_do_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize); > > -int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio) > +int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio, uint32_t sumsize) > { > int ret = -EAGAIN; > int blocksneeded = c->resv_blocks_write; > @@ -129,7 +129,7 @@ > spin_lock(&c->erase_completion_lock); > } > > - ret = jffs2_do_reserve_space(c, minsize, ofs, len); > + ret = jffs2_do_reserve_space(c, minsize, ofs, len, sumsize); > if (ret) { > D1(printk(KERN_DEBUG "jffs2_reserve_space: ret is %d\n", ret)); > } > @@ -140,7 +140,7 @@ > return ret; > } > > -int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len) > +int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize) > { > int ret = -EAGAIN; > minsize = PAD(minsize); > @@ -149,7 +149,7 @@ > > spin_lock(&c->erase_completion_lock); > while(ret == -EAGAIN) { > - ret = jffs2_do_reserve_space(c, minsize, ofs, len); > + ret = jffs2_do_reserve_space(c, minsize, ofs, len, sumsize); > if (ret) { > D1(printk(KERN_DEBUG "jffs2_reserve_space_gc: looping, ret is %d\n", ret)); > } > @@ -159,50 +159,112 @@ > } > > /* Called with alloc sem _and_ erase_completion_lock */ > -static int jffs2_do_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len) > +static int jffs2_do_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize) > { > struct jffs2_eraseblock *jeb = c->nextblock; > + uint32_t nofree_size; > > restart: > - if (jeb && minsize > jeb->free_size) { > - /* Skip the end of this block and file it as having some dirty space */ > - /* If there's a pending write to it, flush now */ > - if (jffs2_wbuf_dirty(c)) { > - spin_unlock(&c->erase_completion_lock); > - D1(printk(KERN_DEBUG "jffs2_do_reserve_space: Flushing write buffer\n")); > - jffs2_flush_wbuf_pad(c); > - spin_lock(&c->erase_completion_lock); > - jeb = c->nextblock; > - goto restart; > - } > - c->wasted_size += jeb->free_size; > - c->free_size -= jeb->free_size; > - jeb->wasted_size += jeb->free_size; > - jeb->free_size = 0; > + nofree_size = 0; > + > +#ifdef CONFIG_JFFS2_SUMMARY > + > + if (sumsize != JFFS2_SUMMARY_NOSUM_SIZE) { > + int ret; > + if (jeb) { > + if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret; > + nofree_size = sumsize + jeb->sum_collected->sum_size + JFFS2_SUMMARY_FRAME_SIZE; > + } > > - /* Check, if we have a dirty block now, or if it was dirty already */ > - if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) { > - c->dirty_size += jeb->wasted_size; > - c->wasted_size -= jeb->wasted_size; > - jeb->dirty_size += jeb->wasted_size; > - jeb->wasted_size = 0; > - if (VERYDIRTY(c, jeb->dirty_size)) { > - D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", > + D1(printk(KERN_DEBUG "JFFS2: minsize %d , jeb->free(%d) , sum_collected->size(%d) , sumsize(%d)\n",minsize,jeb->free_size,jeb->sum_collected->sum_size,sumsize)); > + > + if (jeb && (minsize + jeb->sum_collected->sum_size + sumsize + JFFS2_SUMMARY_FRAME_SIZE > jeb->free_size)) { > + D1(printk(KERN_DEBUG "JFFS2: generating summary for 0x%08x.\n", jeb->offset)); > + if (jeb->sum_collected->sum_size == JFFS2_SUMMARY_NOSUM_SIZE) { > + sumsize = JFFS2_SUMMARY_NOSUM_SIZE; > + jffs2_sum_clean_collected(jeb); > + goto restart; > + } > + > + ret = jffs2_sum_write_sumnode(c); > + > + if (ret) > + return ret; > + > + if (jeb->sum_collected->sum_size == JFFS2_SUMMARY_NOSUM_SIZE) { //jffs2_write_sumnode can't write out the summary information > + sumsize = JFFS2_SUMMARY_NOSUM_SIZE; > + jffs2_sum_clean_collected(jeb); > + goto restart; > + } > + > + /* Check, if we have a dirty block now, or if it was dirty already */ > + if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) { > + c->dirty_size += jeb->wasted_size; > + c->wasted_size -= jeb->wasted_size; > + jeb->dirty_size += jeb->wasted_size; > + jeb->wasted_size = 0; > + if (VERYDIRTY(c, jeb->dirty_size)) { > + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", > + jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size)); > + list_add_tail(&jeb->list, &c->very_dirty_list); > + } else { > + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", > + jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size)); > + list_add_tail(&jeb->list, &c->dirty_list); > + } > + } else { > + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", > jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size)); > - list_add_tail(&jeb->list, &c->very_dirty_list); > - } else { > - D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", > + list_add_tail(&jeb->list, &c->clean_list); > + } > + c->nextblock = jeb = NULL; > + } > + } > + else { > +#endif > + if (jeb && minsize > jeb->free_size) { > + /* Skip the end of this block and file it as having some dirty space */ > + /* If there's a pending write to it, flush now */ > + > + if (jffs2_wbuf_dirty(c)) { > + spin_unlock(&c->erase_completion_lock); > + D1(printk(KERN_DEBUG "jffs2_do_reserve_space: Flushing write buffer\n")); > + jffs2_flush_wbuf_pad(c); > + spin_lock(&c->erase_completion_lock); > + jeb = c->nextblock; > + goto restart; > + } > + > + c->wasted_size += jeb->free_size; > + c->free_size -= jeb->free_size; > + jeb->wasted_size += jeb->free_size; > + jeb->free_size = 0; > + > + /* Check, if we have a dirty block now, or if it was dirty already */ > + if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) { > + c->dirty_size += jeb->wasted_size; > + c->wasted_size -= jeb->wasted_size; > + jeb->dirty_size += jeb->wasted_size; > + jeb->wasted_size = 0; > + if (VERYDIRTY(c, jeb->dirty_size)) { > + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", > + jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size)); > + list_add_tail(&jeb->list, &c->very_dirty_list); > + } else { > + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", > + jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size)); > + list_add_tail(&jeb->list, &c->dirty_list); > + } > + } else { > + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", > jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size)); > - list_add_tail(&jeb->list, &c->dirty_list); > + list_add_tail(&jeb->list, &c->clean_list); > } > - } else { > - D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n", > - jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size)); > - list_add_tail(&jeb->list, &c->clean_list); > + c->nextblock = jeb = NULL; > } > - c->nextblock = jeb = NULL; > +#ifdef CONFIG_JFFS2_SUMMARY > } > - > +#endif > if (!jeb) { > struct list_head *next; > /* Take the next block off the 'free' list */ > @@ -266,7 +328,7 @@ > /* OK, jeb (==c->nextblock) is now pointing at a block which definitely has > enough space */ > *ofs = jeb->offset + (c->sector_size - jeb->free_size); > - *len = jeb->free_size; > + *len = jeb->free_size - nofree_size; > > if (c->cleanmarker_size && jeb->used_size == c->cleanmarker_size && > !jeb->first_node->next_in_ino) { > diff --unified --new-file --recursive mtd/fs/jffs2/os-linux.h mtd-dec07/fs/jffs2/os-linux.h > --- mtd/fs/jffs2/os-linux.h 2004-11-17 00:00:14.000000000 +0100 > +++ mtd-dec07/fs/jffs2/os-linux.h 2004-12-07 16:03:02.000000000 +0100 > @@ -100,7 +100,13 @@ > #define jffs2_is_readonly(c) (OFNI_BS_2SFFJ(c)->s_flags & MS_RDONLY) > > #if (!defined CONFIG_JFFS2_FS_NAND && !defined CONFIG_JFFS2_FS_NOR_ECC) > + > +#ifndef CONFIG_JFFS2_SUMMARY > #define jffs2_can_mark_obsolete(c) (1) > +#else > +#define jffs2_can_mark_obsolete(c) (0) > +#endif > + > #define jffs2_cleanmarker_oob(c) (0) > #define jffs2_write_nand_cleanmarker(c,jeb) (-EIO) > > @@ -121,7 +127,12 @@ > > #else /* NAND and/or ECC'd NOR support present */ > > +#ifndef CONFIG_JFFS2_SUMMARY > #define jffs2_can_mark_obsolete(c) ((c->mtd->type == MTD_NORFLASH && !(c->mtd->flags & MTD_ECC)) || c->mtd->type == MTD_RAM) > +#else > +#define jffs2_can_mark_obsolete(c) (0) > +#endif > + > #define jffs2_cleanmarker_oob(c) (c->mtd->type == MTD_NANDFLASH) > > #define jffs2_flash_write_oob(c, ofs, len, retlen, buf) ((c)->mtd->write_oob((c)->mtd, ofs, len, retlen, buf)) > @@ -218,5 +229,3 @@ > > > #endif /* __JFFS2_OS_LINUX_H__ */ > - > - > diff --unified --new-file --recursive mtd/fs/jffs2/scan.c mtd-dec07/fs/jffs2/scan.c > --- mtd/fs/jffs2/scan.c 2004-11-26 15:08:31.000000000 +0100 > +++ mtd-dec07/fs/jffs2/scan.c 2004-12-07 16:03:02.000000000 +0100 > @@ -18,22 +18,10 @@ > #include <linux/crc32.h> > #include <linux/compiler.h> > #include "nodelist.h" > +#include "summary.h" > > #define EMPTY_SCAN_SIZE 1024 > > -#define DIRTY_SPACE(x) do { typeof(x) _x = (x); \ > - c->free_size -= _x; c->dirty_size += _x; \ > - jeb->free_size -= _x ; jeb->dirty_size += _x; \ > - }while(0) > -#define USED_SPACE(x) do { typeof(x) _x = (x); \ > - c->free_size -= _x; c->used_size += _x; \ > - jeb->free_size -= _x ; jeb->used_size += _x; \ > - }while(0) > -#define UNCHECKED_SPACE(x) do { typeof(x) _x = (x); \ > - c->free_size -= _x; c->unchecked_size += _x; \ > - jeb->free_size -= _x ; jeb->unchecked_size += _x; \ > - }while(0) > - > #define noisy_printk(noise, args...) do { \ > if (*(noise)) { \ > printk(KERN_NOTICE args); \ > @@ -58,13 +46,6 @@ > static int jffs2_scan_dirent_node(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb, > struct jffs2_raw_dirent *rd, uint32_t ofs); > > -#define BLK_STATE_ALLFF 0 > -#define BLK_STATE_CLEAN 1 > -#define BLK_STATE_PARTDIRTY 2 > -#define BLK_STATE_CLEANMARKER 3 > -#define BLK_STATE_ALLDIRTY 4 > -#define BLK_STATE_BADBLOCK 5 > - > static inline int min_free(struct jffs2_sb_info *c) > { > uint32_t min = 2 * sizeof(struct jffs2_raw_inode); > @@ -257,7 +238,7 @@ > return ret; > } > > -static int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf, > +int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf, > uint32_t ofs, uint32_t len) > { > int ret; > @@ -286,6 +267,11 @@ > uint32_t hdr_crc, buf_ofs, buf_len; > int err; > int noise = 0; > + > +#ifdef CONFIG_JFFS2_SUMMARY > + struct jffs2_sum_marker *sm; > +#endif > + > #ifdef CONFIG_JFFS2_FS_NAND > int cleanmarkerfound = 0; > #endif > @@ -311,10 +297,54 @@ > } > } > #endif > + > +#ifdef CONFIG_JFFS2_SUMMARY > + sm = (struct jffs2_sum_marker *)kmalloc(sizeof(struct jffs2_sum_marker), GFP_KERNEL); > + if (!sm) { > + return -ENOMEM; > + } > + > + err = jffs2_fill_scan_buf(c, (unsigned char *) sm, jeb->offset + c->sector_size - sizeof(struct jffs2_sum_marker), sizeof(struct jffs2_sum_marker)); > + > + if (err) { > + kfree(sm); > + return err; > + } > + > + if (je32_to_cpu(sm->magic) == JFFS2_SUM_MAGIC ) { > + > + if(je32_to_cpu(sm->erase_size) == c->sector_size) { > + int ret = jffs2_sum_scan_sumnode(c,jeb,je32_to_cpu(sm->offset),&pseudo_random); > + > + if (ret) { > + kfree(sm); > + return ret; > + } > + } > + > + printk(KERN_WARNING "FS erase_block_size != JFFS2 erase_block_size => skipping summary information\n"); > + > + } > + > + kfree(sm); > + > + ofs = jeb->offset; > + prevofs = jeb->offset - 1; > + > +#endif > + > buf_ofs = jeb->offset; > > if (!buf_size) { > buf_len = c->sector_size; > + > +#ifdef CONFIG_JFFS2_SUMMARY > + /* must reread because of summary test */ > + err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len); > + if (err) > + return err; > +#endif > + > } else { > buf_len = EMPTY_SCAN_SIZE; > err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len); > @@ -356,6 +386,8 @@ > > noise = 10; > > + D1(printk(KERN_DEBUG "JFFS2: no summary found in jeb 0x%08x. Apply original scan.\n",jeb->offset)); > + > scan_more: > while(ofs < jeb->offset + c->sector_size) { > > @@ -571,6 +603,9 @@ > break; > > case JFFS2_NODETYPE_PADDING: > +#ifdef CONFIG_JFFS2_SUMMARY > + jffs2_sum_add_padding_mem(jeb,je32_to_cpu(node->totlen)); > +#endif > DIRTY_SPACE(PAD(je32_to_cpu(node->totlen))); > ofs += PAD(je32_to_cpu(node->totlen)); > break; > @@ -634,7 +669,7 @@ > return BLK_STATE_ALLDIRTY; > } > > -static struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino) > +struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino) > { > struct jffs2_inode_cache *ic; > > @@ -728,6 +763,11 @@ > pseudo_random += je32_to_cpu(ri->version); > > UNCHECKED_SPACE(PAD(je32_to_cpu(ri->totlen))); > + > +#ifdef CONFIG_JFFS2_SUMMARY > + jffs2_sum_add_inode_mem(jeb,ri,ofs); > +#endif > + > return 0; > } > > @@ -806,6 +846,10 @@ > USED_SPACE(PAD(je32_to_cpu(rd->totlen))); > jffs2_add_fd_to_list(c, fd, &ic->scan_dents); > > +#ifdef CONFIG_JFFS2_SUMMARY > + jffs2_sum_add_dirent_mem(jeb,rd,ofs); > +#endif > + > return 0; > } > > diff --unified --new-file --recursive mtd/fs/jffs2/summary.c mtd-dec07/fs/jffs2/summary.c > --- mtd/fs/jffs2/summary.c 1970-01-01 01:00:00.000000000 +0100 > +++ mtd-dec07/fs/jffs2/summary.c 2004-12-07 16:03:02.000000000 +0100 > @@ -0,0 +1,720 @@ > +/* > + * JFFS2 -- Journalling Flash File System, Version 2. > + * > + * Copyright (C) 2004 Ferenc Havasi <havasi@inf.u-szeged.hu>, > + * Zoltan Sogor <weth@inf.u-szeged.hu>, > + * Patrik Kluba <pajko@halom.u-szeged.hu>, > + * University of Szeged, Hungary > + * > + * For licensing information, see the file 'LICENCE' in this directory. > + * > + * $Id$ > + * > + */ > + > +#include <linux/kernel.h> > +#include <linux/sched.h> > +#include <linux/slab.h> > +#include <linux/mtd/mtd.h> > +#include <linux/pagemap.h> > +#include <linux/crc32.h> > +#include <linux/compiler.h> > +#include <linux/vmalloc.h> > +#include "nodelist.h" > + > +int jffs2_sum_init(struct jffs2_sb_info *c) > +{ > + c->summary_buf = (jint32_t *) vmalloc(c->sector_size); > + if (!c->summary_buf) { > + printk(KERN_WARNING "JFFS2: can't allocate memory to dump summary information!\n"); > + return 1; > + } > + return 0; > +} > + > +void jffs2_sum_exit(struct jffs2_sb_info *c) > +{ > + if (c->summary_buf) { > + vfree(c->summary_buf); > + c->summary_buf = NULL; > + } > +} > + > +int jffs2_sum_care_sum_collected(struct jffs2_eraseblock *jeb) > +{ > + if (!jeb->sum_collected) { > + jeb->sum_collected = (struct jffs2_sum_info *) kmalloc(sizeof(struct jffs2_sum_info), GFP_KERNEL); > + > + if (!jeb->sum_collected) > + return -ENOMEM; > + > + jeb->sum_collected->sum_list = NULL; > + jeb->sum_collected->sum_num = 0; > + jeb->sum_collected->sum_size = 0; > + jeb->sum_collected->sum_padded = 0; > + } > + return 0; > +} > + > +static int jffs2_sum_add_mem(struct jffs2_eraseblock *jeb, union jffs2_sum_mem *item) > +{ > + > + union jffs2_sum_mem *walk; > + int ret; > + > + if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret; > + > + if (!jeb->sum_collected->sum_list) { > + jeb->sum_collected->sum_list = (union jffs2_sum_mem *) item; > + } > + else { > + walk = jeb->sum_collected->sum_list; > + > + while (walk->u.next) { > + walk = walk->u.next; > + } > + walk->u.next = (union jffs2_sum_mem *) item; > + } > + switch (je16_to_cpu(item->u.nodetype)) { > + case JFFS2_NODETYPE_INODE: > + jeb->sum_collected->sum_size += JFFS2_SUMMARY_INODE_SIZE; > + jeb->sum_collected->sum_num++; > + break; > + case JFFS2_NODETYPE_DIRENT: > + jeb->sum_collected->sum_size += JFFS2_SUMMARY_DIRENT_SIZE(item->d.nsize); > + jeb->sum_collected->sum_num++; > + break; > + default: > + printk(KERN_WARNING "__jffs2_add_sum_mem(): UNKNOWN node type %d\n", je16_to_cpu(item->u.nodetype)); > + return 1; > + } > + return 0; > +} > + > +void jffs2_sum_clean_all_info(struct jffs2_sb_info *c) > +{ > + int i; > + > + for (i=0; i<c->nr_blocks; i++) { > + struct jffs2_eraseblock *jeb = &c->blocks[i]; > + > + jffs2_sum_clean_collected(jeb); > + kfree(jeb->sum_collected); > + jeb->sum_collected = NULL; > + } > +} > + > +/* These 3 functions are called from scan.c to collect summary info for not closed jeb */ > + > +int jffs2_sum_add_padding_mem(struct jffs2_eraseblock *jeb, uint32_t size) > +{ > + int ret; > + > + if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret; > + jeb->sum_collected->sum_padded += size; > + return 0; > +} > + > +int jffs2_sum_add_inode_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_inode *ri, uint32_t ofs) > +{ > + > + struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *) kmalloc(sizeof(struct jffs2_sum_inode_mem), GFP_KERNEL); > + > + if (!temp) > + return -ENOMEM; > + > + ofs -= jeb->offset; > + > + temp->nodetype = ri->nodetype; > + temp->inode = ri->ino; > + temp->version = ri->version; > + temp->offset = cpu_to_je32(ofs); > + temp->totlen = ri->totlen; > + temp->next = NULL; > + > + return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp); > +} > + > +int jffs2_sum_add_dirent_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_dirent *rd, uint32_t ofs) > +{ > + > + struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *) > + kmalloc(sizeof(struct jffs2_sum_dirent_mem) + rd->nsize, GFP_KERNEL); > + > + if (!temp) > + return -ENOMEM; > + > + ofs -= jeb->offset; > + > + temp->nodetype = rd->nodetype; > + temp->totlen = rd->totlen; > + temp->offset = cpu_to_je32(ofs); > + temp->pino = rd->pino; > + temp->version = rd->version; > + temp->ino = rd->ino; > + temp->nsize = rd->nsize; > + temp->type = rd->type; > + temp->next = NULL; > + > + memcpy(temp->name, rd->name, rd->nsize); > + > + return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp); > +} > + > +/* Cleanup every collected summary information */ > + > +void jffs2_sum_clean_collected(struct jffs2_eraseblock *jeb) > +{ > + > + union jffs2_sum_mem *temp; > + > + if(jeb && jeb->sum_collected){ > + > + while(jeb->sum_collected->sum_list){ > + temp = jeb->sum_collected->sum_list; > + jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next; > + kfree(temp); > + jeb->sum_collected->sum_num--; > + } > + > + if(jeb->sum_collected->sum_num != 0){ > + printk(KERN_WARNING "Ooops, something wrong happened! sum_num != 0, but sum_list = null ???"); > + jeb->sum_collected->sum_num = 0; > + } > + } > +} > + > +/* Called from wbuf.c to collect writed node info */ > + > +int jffs2_sum_add_kvec(struct jffs2_sb_info *c, const struct kvec *invecs, unsigned long count, uint32_t ofs) > +{ > + union jffs2_node_union *node; > + struct jffs2_eraseblock *jeb; > + int ret; > + > + node = (union jffs2_node_union *) invecs[0].iov_base; > + jeb = &c->blocks[ofs / c->sector_size]; > + ofs -= jeb->offset; > + > + if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret; > + > + switch(je16_to_cpu(node->u.nodetype)){ > + case JFFS2_NODETYPE_INODE : { > + struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *) > + kmalloc(sizeof(struct jffs2_sum_inode_mem), GFP_KERNEL); > + > + if (!temp) > + return -ENOMEM; > + > + temp->nodetype = node->i.nodetype; > + temp->inode = node->i.ino; > + temp->version = node->i.version; > + temp->offset = cpu_to_je32(ofs); > + temp->totlen = node->i.totlen; > + temp->next = NULL; > + > + return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp); > + > + break; > + } > + > + case JFFS2_NODETYPE_DIRENT : { > + struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *) > + kmalloc(sizeof(struct jffs2_sum_dirent_mem) + node->d.nsize, GFP_KERNEL); > + > + if (!temp) > + return -ENOMEM; > + > + temp->nodetype = node->d.nodetype; > + temp->totlen = node->d.totlen; > + temp->offset = cpu_to_je32(ofs); > + temp->pino = node->d.pino; > + temp->version = node->d.version; > + temp->ino = node->d.ino; > + temp->nsize = node->d.nsize; > + temp->type = node->d.type; > + temp->next = NULL; > + > + memcpy(temp->name,invecs[1].iov_base,node->d.nsize); > + > + return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp); > + > + break; > + } > + > + case JFFS2_NODETYPE_PADDING : { > + D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node PADDING\n")); > + jeb->sum_collected->sum_padded += je32_to_cpu(node->u.totlen); > + break; > + } > + > + case JFFS2_NODETYPE_CLEANMARKER : { > + D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node CLEANMARKER\n")); > + break; > + } > + > + case JFFS2_NODETYPE_SUMMARY : { > + D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node SUMMARY\n")); > + break; > + } > + > + default : { > + printk(KERN_WARNING "jffs2_sum_add_kvec(): Node not supported\n"); > + BUG(); > + break; > + } > + } > + > + return 0; > +} > + > +/* Process the summary information - called from jffs2_scan_eraseblock() */ > + > +int jffs2_sum_scan_sumnode(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb, uint32_t ofs, uint32_t *pseudo_random) > +{ > + > + struct jffs2_unknown_node crcnode; > + struct jffs2_raw_node_ref *raw; > + struct jffs2_raw_node_ref *cache_ref; > + struct jffs2_inode_cache *ic; > + struct jffs2_full_dirent *fd; > + > + int i, err; > + int bad_sum = 0; > + int sumsize; > + uint32_t ino; > + uint32_t crc; > + struct jffs2_summary_node *summary; > + > + sumsize = c->sector_size - ofs; > + ofs += jeb->offset; > + > + D1(printk(KERN_DEBUG "JFFS2: summary found for 0x%08x at 0x%08x (0x%x bytes)\n", jeb->offset, ofs, sumsize)); > + > + summary = (struct jffs2_summary_node *) kmalloc(sumsize, GFP_KERNEL); > + > + if (!summary) { > + return -ENOMEM; > + } > + > + err = jffs2_fill_scan_buf(c, (unsigned char *)summary, ofs, sumsize); > + > + if (err) { > + kfree(summary); > + return err; > + } > + > + /* OK, now check for node validity and CRC */ > + crcnode.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK); > + crcnode.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY); > + crcnode.totlen = summary->totlen; > + crc = crc32(0, &crcnode, sizeof(crcnode)-4); > + > + if (je32_to_cpu(summary->hdr_crc) != crc) { > + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node header is corrupt (bad CRC or no summary at all)\n")); > + bad_sum = 1; > + } > + > + if ((!bad_sum) && (je32_to_cpu(summary->totlen) != sumsize)) { > + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (wrong erasesize?)\n")); > + bad_sum = 1; > + } > + > + crc = crc32(0, summary, sizeof(struct jffs2_summary_node)-8); > + > + if ((!bad_sum) && (je32_to_cpu(summary->node_crc) != crc)) { > + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (bad CRC)\n")); > + bad_sum = 1; > + } > + > + crc = crc32(0, summary->sum, sumsize - sizeof(struct jffs2_summary_node)); > + > + if ((!bad_sum) && (je32_to_cpu(summary->sum_crc) != crc)) { > + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node data is corrupt (bad CRC)\n")); > + bad_sum = 1; > + } > + > + if (!bad_sum) { > + > + struct jffs2_sum_unknown_flash *sp; > + sp = (struct jffs2_sum_unknown_flash *) summary->sum; > + > + if ( je32_to_cpu(summary->cln_mkr) ) { > + > + D1(printk(KERN_DEBUG "Summary : CLEANMARKER node \n")); > + > + if (je32_to_cpu(summary->cln_mkr) != c->cleanmarker_size) { > + D1(printk(KERN_DEBUG "CLEANMARKER node has totlen 0x%x != normal 0x%x\n", > + je32_to_cpu(summary->cln_mkr), c->cleanmarker_size)); > + UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) ); > + } > + else if (jeb->first_node) { > + D1(printk(KERN_DEBUG "CLEANMARKER node not first node in block (0x%08x)\n", jeb->offset); > + UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)))); > + } > + else { > + struct jffs2_raw_node_ref *marker_ref = jffs2_alloc_raw_node_ref(); > + > + if (!marker_ref) { > + D1(printk(KERN_NOTICE "Failed to allocate node ref for clean marker\n")); > + kfree(summary); > + return -ENOMEM; > + } > + > + marker_ref->next_in_ino = NULL; > + marker_ref->next_phys = NULL; > + marker_ref->flash_offset = jeb->offset | REF_NORMAL; > + marker_ref->__totlen = je32_to_cpu(summary->cln_mkr); > + jeb->first_node = jeb->last_node = marker_ref; > + > + USED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) ); > + > + } > + } > + > + if ( je32_to_cpu(summary->padded) ) { > + DIRTY_SPACE(je32_to_cpu(summary->padded)); > + } > + > + for(i = 0; i < je16_to_cpu(summary->sum_num); i++) { > + > + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Processing summary information %d\n", i)); > + uint8_t *temp8ptr = NULL; > + > + switch (je16_to_cpu(sp->nodetype)) { > + > + case JFFS2_NODETYPE_INODE : { > + struct jffs2_sum_inode_flash *spi; > + spi = (struct jffs2_sum_inode_flash *) sp; > + > + ino = je32_to_cpu(spi->inode); > + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Inode at 0x%08x\n", jeb->offset + je32_to_cpu(spi->offset))); > + raw = jffs2_alloc_raw_node_ref(); > + if (!raw) { > + printk(KERN_NOTICE "jffs2_scan_eraseblock(): allocation of node reference failed\n"); > + kfree(summary); > + return -ENOMEM; > + } > + > + ic = jffs2_get_ino_cache(c, ino); > + if (!ic) { > + ic = jffs2_scan_make_ino_cache(c, ino); > + if (!ic) { > + printk(KERN_NOTICE "jffs2_scan_eraseblock(): scan_make_ino_cache failed\n"); > + jffs2_free_raw_node_ref(raw); > + kfree(summary); > + return -ENOMEM; > + } > + } > + > + raw->flash_offset = (jeb->offset + je32_to_cpu(spi->offset)) | REF_UNCHECKED; > + raw->__totlen = PAD(je32_to_cpu(spi->totlen)); > + raw->next_phys = NULL; > + raw->next_in_ino = ic->nodes; > + > + ic->nodes = raw; > + if (!jeb->first_node) > + jeb->first_node = raw; > + if (jeb->last_node) > + jeb->last_node->next_phys = raw; > + jeb->last_node = raw; > + > + *pseudo_random += je32_to_cpu(spi->version); > + UNCHECKED_SPACE(PAD(je32_to_cpu(spi->totlen))); > + > + temp8ptr = (uint8_t *) sp; > + temp8ptr += JFFS2_SUMMARY_INODE_SIZE; > + sp = (struct jffs2_sum_unknown_flash *) temp8ptr; > + > + break; > + } > + > + case JFFS2_NODETYPE_DIRENT : { > + struct jffs2_sum_dirent_flash *spd; > + spd = (struct jffs2_sum_dirent_flash *) sp; > + > + fd = jffs2_alloc_full_dirent(spd->nsize+1); > + if (!fd) { > + kfree(summary); > + return -ENOMEM; > + } > + > + memcpy(&fd->name, spd->name, spd->nsize); > + fd->name[spd->nsize] = 0; > + > + raw = jffs2_alloc_raw_node_ref(); > + if (!raw) { > + jffs2_free_full_dirent(fd); > + printk(KERN_NOTICE "jffs2_scan_dirent_node(): allocation of node reference failed\n"); > + kfree(summary); > + return -ENOMEM; > + } > + > + ic = jffs2_scan_make_ino_cache(c, je32_to_cpu(spd->pino)); > + if (!ic) { > + jffs2_free_full_dirent(fd); > + jffs2_free_raw_node_ref(raw); > + kfree(summary); > + return -ENOMEM; > + } > + > + raw->__totlen = PAD(je32_to_cpu(spd->totlen)); > + raw->flash_offset = (jeb->offset + je32_to_cpu(spd->offset)) | REF_PRISTINE; > + raw->next_phys = NULL; > + raw->next_in_ino = ic->nodes; > + ic->nodes = raw; > + if (!jeb->first_node) > + jeb->first_node = raw; > + if (jeb->last_node) > + jeb->last_node->next_phys = raw; > + jeb->last_node = raw; > + > + fd->raw = raw; > + fd->next = NULL; > + fd->version = je32_to_cpu(spd->version); > + fd->ino = je32_to_cpu(spd->ino); > + fd->nhash = full_name_hash(fd->name, spd->nsize); > + fd->type = spd->type; > + USED_SPACE(PAD(je32_to_cpu(spd->totlen))); > + jffs2_add_fd_to_list(c, fd, &ic->scan_dents); > + > + *pseudo_random += je32_to_cpu(spd->version); > + > + temp8ptr = (uint8_t *) sp; > + temp8ptr += JFFS2_SUMMARY_DIRENT_SIZE(spd->nsize); > + sp = (struct jffs2_sum_unknown_flash *) temp8ptr; > + > + break; > + } > + > + default : { > + printk(KERN_WARNING "Kernel doesn't support this type of node !!! Exiting"); > + return -EIO; > + break; > + } > + } > + } > + > + kfree(summary); > + > + /* for ACCT_PARANOIA_CHECK */ > + cache_ref = jffs2_alloc_raw_node_ref(); > + > + if (!cache_ref) { > + printk(KERN_NOTICE "Failed to allocate node ref for cache\n"); > + return -ENOMEM; > + } > + > + cache_ref->next_in_ino = NULL; > + cache_ref->next_phys = NULL; > + cache_ref->flash_offset = ofs | REF_NORMAL; > + cache_ref->__totlen = sumsize; > + > + if (!jeb->first_node) > + jeb->first_node = cache_ref; > + if (jeb->last_node) > + jeb->last_node->next_phys = cache_ref; > + jeb->last_node = cache_ref; > + > + USED_SPACE(sumsize); > + > + jeb->wasted_size += jeb->free_size; > + c->wasted_size += jeb->free_size; > + c->free_size -= jeb->free_size; > + jeb->free_size = 0; > + > + /* somebody check this and all of space accounting in summary support */ > + > + if ((jeb->used_size + jeb->unchecked_size) == PAD(c->cleanmarker_size) && !jeb->dirty_size > + && (!jeb->first_node || !jeb->first_node->next_in_ino) ) { > + return BLK_STATE_CLEANMARKER; > + } > + /* move blocks with max 4 byte dirty space to cleanlist */ > + else if (!ISDIRTY(c->sector_size - (jeb->used_size + jeb->unchecked_size))) { > + c->dirty_size -= jeb->dirty_size; > + c->wasted_size += jeb->dirty_size; > + jeb->wasted_size += jeb->dirty_size; > + jeb->dirty_size = 0; > + return BLK_STATE_CLEAN; > + } > + else if (jeb->used_size || jeb->unchecked_size) { > + return BLK_STATE_PARTDIRTY; > + } > + else { > + return BLK_STATE_ALLDIRTY; > + } > + } > + > + return 0; > +} > + > +/* Write out summary information - called from jffs2_do_reserve_space */ > + > +int jffs2_sum_write_sumnode(struct jffs2_sb_info *c) > +{ > + struct jffs2_summary_node isum; > + union jffs2_sum_mem *temp; > + jint32_t offset; > + jint32_t *wpage; > + uint8_t *tempptr; > + int datasize; > + int infosize; > + int padsize; > + size_t retlen; > + int ret; > + struct jffs2_eraseblock *jeb; > + struct kvec vecs[2]; > + jint32_t magic = cpu_to_je32(JFFS2_SUM_MAGIC); > + > + D2(printk(KERN_DEBUG "jffs2_sum_write_sumnode()\n")); > + > + jeb = c->nextblock; > + > + if (!jeb->sum_collected->sum_num || !jeb->sum_collected->sum_list) { > + printk(KERN_WARNING "JFFS2: jffs2_sum_write_sumnode(): empty summary info!!!\n"); > + BUG(); > + } > + > + datasize = jeb->sum_collected->sum_size + sizeof(struct jffs2_sum_marker); > + infosize = sizeof(struct jffs2_summary_node) + datasize; > + padsize = jeb->free_size - infosize; > + infosize += padsize; datasize += padsize; > + offset = cpu_to_je32(c->sector_size - jeb->free_size); > + > + if (padsize < 0) { // if jeb hasn't got enought free space for summary > + > + union jffs2_sum_mem *temp; > + > + while(jeb->sum_collected->sum_list){ //cleanup sum_list > + temp = jeb->sum_collected->sum_list; > + jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next; > + kfree(temp); > + jeb->sum_collected->sum_num--; > + } > + > + jeb->sum_collected->sum_list = NULL; > + jeb->sum_collected->sum_num = 0; > + jeb->sum_collected->sum_size = JFFS2_SUMMARY_NOSUM_SIZE; // don't try to write out summary for this node > + > + printk(KERN_WARNING "JFFS2: not enough space for summary, padsize = %d\n",padsize); > + return 0; > + } > + > + memset(c->summary_buf, 0xff, datasize); > + memset(&isum, 0, sizeof(isum)); > + > + isum.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK); > + isum.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY); > + isum.totlen = cpu_to_je32(infosize); > + isum.hdr_crc = cpu_to_je32(crc32(0, &isum, sizeof(struct jffs2_unknown_node) - 4)); > + isum.padded = cpu_to_je32(jeb->sum_collected->sum_padded); > + > + if (c->cleanmarker_size) { > + isum.cln_mkr = cpu_to_je32(c->cleanmarker_size); > + } > + else{ > + isum.cln_mkr = cpu_to_je32(0); > + } > + > + isum.sum_num = cpu_to_je16(jeb->sum_collected->sum_num); > + wpage = c->summary_buf; > + > + > + while (jeb->sum_collected->sum_num) { // write sum_data > + > + > + switch(je16_to_cpu(jeb->sum_collected->sum_list->u.nodetype)){ > + > + case JFFS2_NODETYPE_INODE : { > + jint16_t *temp16ptr = (jint16_t *)wpage; > + > + *(temp16ptr++) = jeb->sum_collected->sum_list->i.nodetype; > + wpage = (jint32_t *) temp16ptr; > + > + *(wpage++) = jeb->sum_collected->sum_list->i.inode; > + *(wpage++) = jeb->sum_collected->sum_list->i.version; > + *(wpage++) = jeb->sum_collected->sum_list->i.offset; > + *(wpage++) = jeb->sum_collected->sum_list->i.totlen; > + break; > + } > + > + case JFFS2_NODETYPE_DIRENT : { > + jint16_t *temp16ptr = (jint16_t *) wpage; > + uint8_t *temp8ptr = NULL; > + > + *(temp16ptr++) = jeb->sum_collected->sum_list->d.nodetype; > + wpage = (jint32_t *) temp16ptr; > + > + *(wpage++) = jeb->sum_collected->sum_list->d.totlen; > + *(wpage++) = jeb->sum_collected->sum_list->d.offset; > + *(wpage++) = jeb->sum_collected->sum_list->d.pino; > + *(wpage++) = jeb->sum_collected->sum_list->d.version; > + *(wpage++) = jeb->sum_collected->sum_list->d.ino; > + > + temp8ptr = (uint8_t *) wpage; > + *(temp8ptr++) = jeb->sum_collected->sum_list->d.nsize; > + *(temp8ptr++) = jeb->sum_collected->sum_list->d.type; > + > + memcpy(temp8ptr,jeb->sum_collected->sum_list->d.name,jeb->sum_collected->sum_list->d.nsize); > + temp8ptr += jeb->sum_collected->sum_list->d.nsize; > + wpage = (jint32_t *) temp8ptr; > + > + break; > + } > + > + default : { > + printk(KERN_WARNING "Unknown node in summary information!!!\n"); > + BUG(); > + } > + } > + > + temp = jeb->sum_collected->sum_list; > + jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next; > + kfree(temp); > + > + jeb->sum_collected->sum_num--; > + } > + > + jeb->sum_collected->sum_size = 0; > + > + tempptr = (uint8_t *) wpage; > + tempptr += padsize; > + wpage = (jint32_t *) tempptr; > + > + *(wpage++) = offset; > + *(wpage++) = cpu_to_je32(c->sector_size); > + *(wpage++) = magic; > + isum.sum_crc = cpu_to_je32(crc32(0, c->summary_buf, datasize)); > + isum.node_crc = cpu_to_je32(crc32(0, &isum, sizeof(isum) - 8)); > + > + vecs[0].iov_base = &isum; > + vecs[0].iov_len = sizeof(isum); > + vecs[1].iov_base = c->summary_buf; > + vecs[1].iov_len = datasize; > + > + D1(printk(KERN_DEBUG "JFFS2: writing out data to flash to pos : 0x%08x\n",jeb->offset + c->sector_size - jeb->free_size)); > + > + spin_unlock(&c->erase_completion_lock); > + ret = jffs2_flash_writev(c, vecs, 2, jeb->offset + c->sector_size - jeb->free_size, &retlen, 0); > + spin_lock(&c->erase_completion_lock); > + > + > + if (ret || (retlen != infosize)) { > + printk(KERN_WARNING "JFFS2: write of %zd bytes at 0x%08x failed. returned %d, retlen %zd\n", > + infosize, jeb->offset + c->sector_size - jeb->free_size, ret, retlen); > + > + jeb->sum_collected->sum_size = JFFS2_SUMMARY_NOSUM_SIZE; > + > + WASTED_SPACE(infosize); > + > + return 0; > + } > + > + /*spin_unlock(&c->erase_completion_lock); > + jffs2_flush_wbuf_pad(c); // summary filled the wbuf > + spin_lock(&c->erase_completion_lock);*/ > + > + WASTED_SPACE(infosize); > + > + return 0; > +} > diff --unified --new-file --recursive mtd/fs/jffs2/summary.h mtd-dec07/fs/jffs2/summary.h > --- mtd/fs/jffs2/summary.h 1970-01-01 01:00:00.000000000 +0100 > +++ mtd-dec07/fs/jffs2/summary.h 2004-12-07 16:03:02.000000000 +0100 > @@ -0,0 +1,158 @@ > +/* > + * JFFS2 -- Journalling Flash File System, Version 2. > + * > + * Copyright (C) 2004 Ferenc Havasi <havasi@inf.u-szeged.hu>, > + * Zoltan Sogor <weth@inf.u-szeged.hu>, > + * Patrik Kluba <pajko@halom.u-szeged.hu>, > + * University of Szeged, Hungary > + * > + * For licensing information, see the file 'LICENCE' in this directory. > + * > + * $Id$ > + * > + */ > + > +#ifndef JFFS2_SUMMARY_H > +#define JFFS2_SUMMARY_H > + > +#include <linux/uio.h> > +#include <linux/jffs2.h> > + > +#define DIRTY_SPACE(x) do { typeof(x) _x = (x); \ > + c->free_size -= _x; c->dirty_size += _x; \ > + jeb->free_size -= _x ; jeb->dirty_size += _x; \ > + }while(0) > +#define USED_SPACE(x) do { typeof(x) _x = (x); \ > + c->free_size -= _x; c->used_size += _x; \ > + jeb->free_size -= _x ; jeb->used_size += _x; \ > + }while(0) > +#define WASTED_SPACE(x) do { typeof(x) _x = (x); \ > + c->free_size -= _x; c->wasted_size += _x; \ > + jeb->free_size -= _x ; jeb->wasted_size += _x; \ > + }while(0) > +#define UNCHECKED_SPACE(x) do { typeof(x) _x = (x); \ > + c->free_size -= _x; c->unchecked_size += _x; \ > + jeb->free_size -= _x ; jeb->unchecked_size += _x; \ > + }while(0) > + > +#define BLK_STATE_ALLFF 0 > +#define BLK_STATE_CLEAN 1 > +#define BLK_STATE_PARTDIRTY 2 > +#define BLK_STATE_CLEANMARKER 3 > +#define BLK_STATE_ALLDIRTY 4 > +#define BLK_STATE_BADBLOCK 5 > + > +#define JFFS2_SUMMARY_NOSUM_SIZE 0xffffffff > +#define JFFS2_SUMMARY_INODE_SIZE (sizeof(struct jffs2_sum_inode_flash)) > +#define JFFS2_SUMMARY_DIRENT_SIZE(x) (sizeof(struct jffs2_sum_dirent_flash) + (x)) > + > +struct jffs2_sum_unknown_flash > +{ > + jint16_t nodetype; /* node type */ > +}; > + > +struct jffs2_sum_inode_flash > +{ > + jint16_t nodetype; /* node type */ > + jint32_t inode; /* inode number */ > + jint32_t version; /* inode version */ > + jint32_t offset; /* offset on jeb */ > + jint32_t totlen; /* record length */ > +} __attribute__((packed)); > + > +struct jffs2_sum_dirent_flash > +{ > + jint16_t nodetype; /* == JFFS_NODETYPE_DIRENT */ > + jint32_t totlen; /* record length */ > + jint32_t offset; /* ofset on jeb */ > + jint32_t pino; /* parent inode */ > + jint32_t version; /* dirent version */ > + jint32_t ino; /* == zero for unlink */ > + uint8_t nsize; /* dirent name size */ > + uint8_t type; /* dirent type */ > + uint8_t name[0]; /* dirent name */ > +} __attribute__((packed)); > + > +union jffs2_sum_flash{ > + struct jffs2_sum_unknown_flash u; > + struct jffs2_sum_inode_flash i; > + struct jffs2_sum_dirent_flash d; > +}; > + > +/* list version of jffs2_sum_*flash for kernel and sumtool */ > +struct jffs2_sum_unknown_mem > +{ > + union jffs2_sum_mem *next; > + jint16_t nodetype; /* node type */ > + > +}; > + > +struct jffs2_sum_inode_mem > +{ > + union jffs2_sum_mem *next; > + jint16_t nodetype; /* node type */ > + jint32_t inode; /* inode number */ > + jint32_t version; /* inode version */ > + jint32_t offset; /* offset on jeb */ > + jint32_t totlen; /* record length */ > +} __attribute__((packed)); > + > +struct jffs2_sum_dirent_mem > +{ > + union jffs2_sum_mem *next; > + jint16_t nodetype; /* == JFFS_NODETYPE_DIRENT */ > + jint32_t totlen; /* record length */ > + jint32_t offset; /* ofset on jeb */ > + jint32_t pino; /* parent inode */ > + jint32_t version; /* dirent version */ > + jint32_t ino; /* == zero for unlink */ > + uint8_t nsize; /* dirent name size */ > + uint8_t type; /* dirent type */ > + uint8_t name[0]; /* dirent name */ > +} __attribute__((packed)); > + > +union jffs2_sum_mem > +{ > + struct jffs2_sum_unknown_mem u; > + struct jffs2_sum_inode_mem i; > + struct jffs2_sum_dirent_mem d; > +}; > + > +struct jffs2_sum_info > +{ > + uint32_t sum_size; > + uint32_t sum_num; > + uint32_t sum_padded; > + union jffs2_sum_mem *sum_list; > +}; > + > +struct jffs2_sum_marker > +{ > + jint32_t offset; > + jint32_t erase_size; > + jint32_t magic; > +}; > + > +#define JFFS2_SUMMARY_FRAME_SIZE (sizeof(struct jffs2_summary_node)+sizeof(struct jffs2_sum_marker)+3) > + > +#if !(defined(SUM_TOOL) || defined(JFFS2DUMP)) > + > +int jffs2_sum_init(struct jffs2_sb_info *c); > +void jffs2_sum_exit(struct jffs2_sb_info *c); > + > +int jffs2_sum_care_sum_collected(struct jffs2_eraseblock *jeb); > + > +void jffs2_sum_clean_collected(struct jffs2_eraseblock *jeb); > +void jffs2_sum_clean_all_info(struct jffs2_sb_info *c); /* clean up all summary information in all jeb (umount) */ > + > +int jffs2_sum_add_padding_mem(struct jffs2_eraseblock *jeb, uint32_t size); > +int jffs2_sum_add_inode_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_inode *ri, uint32_t ofs); > +int jffs2_sum_add_dirent_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_dirent *rd, uint32_t ofs); > +int jffs2_sum_add_kvec(struct jffs2_sb_info *c, const struct kvec *invecs, unsigned long count, uint32_t to); > + > +int jffs2_sum_scan_sumnode(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb, uint32_t ofs, uint32_t *pseudo_random); > +int jffs2_sum_write_sumnode(struct jffs2_sb_info *c); > + > +#endif > + > +#endif /* JFFS2_SUMMARY_H */ > diff --unified --new-file --recursive mtd/fs/jffs2/super-v24.c mtd-dec07/fs/jffs2/super-v24.c > --- mtd/fs/jffs2/super-v24.c 2004-11-17 00:00:14.000000000 +0100 > +++ mtd-dec07/fs/jffs2/super-v24.c 2004-12-07 16:03:02.000000000 +0100 > @@ -24,6 +24,7 @@ > #include <linux/mtd/mtd.h> > #include "compr.h" > #include "nodelist.h" > +#include "summary.h" > > #ifndef MTD_BLOCK_MAJOR > #define MTD_BLOCK_MAJOR 31 > @@ -72,6 +73,9 @@ > put_mtd_device(c->mtd); > return NULL; > } > +#ifdef CONFIG_JFFS2_SUMMARY > + jffs2_sum_init(c); > +#endif > > return sb; > } > @@ -88,6 +92,10 @@ > down(&c->alloc_sem); > jffs2_flush_wbuf_pad(c); > up(&c->alloc_sem); > +#ifdef CONFIG_JFFS2_SUMMARY > + jffs2_sum_clean_all_info(c); > + jffs2_sum_exit(c); > +#endif > jffs2_free_ino_caches(c); > jffs2_free_raw_node_refs(c); > kfree(c->blocks); > @@ -110,6 +118,9 @@ > #ifdef CONFIG_JFFS2_FS_NAND > " (NAND)" > #endif > +#ifdef CONFIG_JFFS2_SUMMARY > + " (SUMMARY)" > +#endif > " (C) 2001-2003 Red Hat, Inc.\n"); > > #ifdef JFFS2_OUT_OF_KERNEL > diff --unified --new-file --recursive mtd/fs/jffs2/super.c mtd-dec07/fs/jffs2/super.c > --- mtd/fs/jffs2/super.c 2004-11-26 15:08:31.000000000 +0100 > +++ mtd-dec07/fs/jffs2/super.c 2004-12-07 16:03:02.000000000 +0100 > @@ -152,6 +152,10 @@ > return ERR_PTR(ret); > } > > +#ifdef CONFIG_JFFS2_SUMMARY > + jffs2_sum_init(c); > +#endif > + > sb->s_flags |= MS_ACTIVE; > return sb; > > @@ -275,6 +279,10 @@ > down(&c->alloc_sem); > jffs2_flush_wbuf_pad(c); > up(&c->alloc_sem); > +#ifdef CONFIG_JFFS2_SUMMARY > + jffs2_sum_clean_all_info(c); > + jffs2_sum_exit(c); > +#endif > jffs2_free_ino_caches(c); > jffs2_free_raw_node_refs(c); > if (c->mtd->flags & MTD_NO_VIRTBLOCKS) > @@ -312,6 +320,9 @@ > #ifdef CONFIG_JFFS2_FS_NAND > " (NAND)" > #endif > +#ifdef CONFIG_JFFS2_SUMMARY > + " (SUMMARY) " > +#endif > " (C) 2001-2003 Red Hat, Inc.\n"); > > jffs2_inode_cachep = kmem_cache_create("jffs2_i", > diff --unified --new-file --recursive mtd/fs/jffs2/wbuf.c mtd-dec07/fs/jffs2/wbuf.c > --- mtd/fs/jffs2/wbuf.c 2004-11-26 15:08:31.000000000 +0100 > +++ mtd-dec07/fs/jffs2/wbuf.c 2004-12-07 16:17:11.000000000 +0100 > @@ -261,7 +261,7 @@ > > > /* ... and get an allocation of space from a shiny new block instead */ > - ret = jffs2_reserve_space_gc(c, end-start, &ofs, &len); > + ret = jffs2_reserve_space_gc(c, end-start, &ofs, &len, JFFS2_SUMMARY_NOSUM_SIZE); > if (ret) { > printk(KERN_WARNING "Failed to allocate space for wbuf recovery. Data loss ensues.\n"); > if (buf) > @@ -601,8 +601,15 @@ > uint32_t outvec_to = to; > > /* If not NAND flash, don't bother */ > - if (!c->wbuf) > + if (!c->wbuf){ > +#ifdef CONFIG_JFFS2_SUMMARY > + D1(printk("JFFS2.summary: NOT NAND MEMORY\n")); > + if(jffs2_sum_add_kvec(c,invecs,count,(uint32_t) to)){ > + printk("jffs2_sum_add_kvec(): MEMORY ALLOCATION ERROR!"); > + } > +#endif > return jffs2_flash_direct_writev(c, invecs, count, to, retlen); > + } > > down_write(&c->wbuf_sem); > > @@ -801,6 +808,12 @@ > alldone: > *retlen = donelen; > > +#ifdef CONFIG_JFFS2_SUMMARY > + if(jffs2_sum_add_kvec(c,invecs,count,(uint32_t) to)){ > + printk("jffs2_sum_add_kvec(): MEMORY ALLOCATION ERROR!"); > + } > +#endif > + > if (c->wbuf_len && ino) > jffs2_wbuf_dirties_inode(c, ino); > > diff --unified --new-file --recursive mtd/fs/jffs2/write.c mtd-dec07/fs/jffs2/write.c > --- mtd/fs/jffs2/write.c 2004-11-17 00:00:14.000000000 +0100 > +++ mtd-dec07/fs/jffs2/write.c 2004-12-07 16:03:02.000000000 +0100 > @@ -173,13 +173,13 @@ > D1(ACCT_PARANOIA_CHECK(jeb)); > > if (alloc_mode == ALLOC_GC) { > - ret = jffs2_reserve_space_gc(c, sizeof(*ri) + datalen, &flash_ofs, &dummy); > + ret = jffs2_reserve_space_gc(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, JFFS2_SUMMARY_INODE_SIZE); > } else { > /* Locking pain */ > up(&f->sem); > jffs2_complete_reservation(c); > > - ret = jffs2_reserve_space(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, alloc_mode); > + ret = jffs2_reserve_space(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, alloc_mode, JFFS2_SUMMARY_INODE_SIZE); > down(&f->sem); > } > > @@ -308,13 +308,13 @@ > D1(ACCT_PARANOIA_CHECK(jeb)); > > if (alloc_mode == ALLOC_GC) { > - ret = jffs2_reserve_space_gc(c, sizeof(*rd) + namelen, &flash_ofs, &dummy); > + ret = jffs2_reserve_space_gc(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, JFFS2_SUMMARY_DIRENT_SIZE(namelen)); > } else { > /* Locking pain */ > up(&f->sem); > jffs2_complete_reservation(c); > > - ret = jffs2_reserve_space(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, alloc_mode); > + ret = jffs2_reserve_space(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, alloc_mode, JFFS2_SUMMARY_DIRENT_SIZE(namelen)); > down(&f->sem); > } > > @@ -371,7 +371,7 @@ > retry: > D2(printk(KERN_DEBUG "jffs2_commit_write() loop: 0x%x to write to 0x%x\n", writelen, offset)); > > - ret = jffs2_reserve_space(c, sizeof(*ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, ALLOC_NORMAL); > + ret = jffs2_reserve_space(c, sizeof(*ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE); > if (ret) { > D1(printk(KERN_DEBUG "jffs2_reserve_space returned %d\n", ret)); > break; > @@ -458,7 +458,7 @@ > /* Try to reserve enough space for both node and dirent. > * Just the node will do for now, though > */ > - ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL); > + ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE); > D1(printk(KERN_DEBUG "jffs2_do_create(): reserved 0x%x bytes\n", alloclen)); > if (ret) { > up(&f->sem); > @@ -487,7 +487,7 @@ > > up(&f->sem); > jffs2_complete_reservation(c); > - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL); > + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen)); > > if (ret) { > /* Eep. */ > @@ -557,7 +557,7 @@ > if (!rd) > return -ENOMEM; > > - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_DELETION); > + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_DELETION, JFFS2_SUMMARY_DIRENT_SIZE(namelen)); > if (ret) { > jffs2_free_raw_dirent(rd); > return ret; > @@ -663,7 +663,7 @@ > if (!rd) > return -ENOMEM; > > - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL); > + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen)); > if (ret) { > jffs2_free_raw_dirent(rd); > return ret; > diff --unified --new-file --recursive mtd/include/linux/jffs2.h mtd-dec07/include/linux/jffs2.h > --- mtd/include/linux/jffs2.h 2004-11-17 00:00:15.000000000 +0100 > +++ mtd-dec07/include/linux/jffs2.h 2004-12-07 16:03:02.000000000 +0100 > @@ -28,6 +28,9 @@ > #define JFFS2_EMPTY_BITMASK 0xffff > #define JFFS2_DIRTY_BITMASK 0x0000 > > +/* Summary node MAGIC marker */ > +#define JFFS2_SUM_MAGIC 0x02851885 > + > /* We only allow a single char for length, and 0xFF is empty flash so > we don't want it confused with a real length. Hence max 254. > */ > @@ -62,6 +65,8 @@ > #define JFFS2_NODETYPE_CLEANMARKER (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3) > #define JFFS2_NODETYPE_PADDING (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 4) > > +#define JFFS2_NODETYPE_SUMMARY (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 6) > + > // Maybe later... > //#define JFFS2_NODETYPE_CHECKPOINT (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3) > //#define JFFS2_NODETYPE_OPTIONS (JFFS2_FEATURE_RWCOMPAT_COPY | JFFS2_NODE_ACCURATE | 4) > @@ -148,10 +153,24 @@ > uint8_t data[0]; > } __attribute__((packed)); > > +struct jffs2_summary_node{ > + jint16_t magic; > + jint16_t nodetype; /* = JFFS2_NODETYPE_INODE_SUM */ > + jint32_t totlen; > + jint32_t hdr_crc; > + jint16_t sum_num; /* number of sum entries*/ > + jint32_t cln_mkr; /* clean marker size, 0 = no cleanmarker */ > + jint32_t padded; /* sum of the size of padding nodes */ > + jint32_t sum_crc; /* summary information crc */ > + jint32_t node_crc; /* node crc */ > + jint32_t sum[0]; /* inode summary info */ > +} __attribute__((packed)); > + > union jffs2_node_union { > struct jffs2_raw_inode i; > struct jffs2_raw_dirent d; > struct jffs2_unknown_node u; > + struct jffs2_summary_node s; > }; > > #endif /* __LINUX_JFFS2_H__ */ > diff --unified --new-file --recursive mtd/include/linux/jffs2_fs_sb.h mtd-dec07/include/linux/jffs2_fs_sb.h > --- mtd/include/linux/jffs2_fs_sb.h 2004-11-26 15:08:31.000000000 +0100 > +++ mtd-dec07/include/linux/jffs2_fs_sb.h 2004-12-07 16:03:02.000000000 +0100 > @@ -110,6 +110,9 @@ > uint32_t fsdata_pos; > uint32_t fsdata_len; > #endif > +#ifdef CONFIG_JFFS2_SUMMARY > + jint32_t *summary_buf; > +#endif > > /* OS-private pointer for getting back to master superblock info */ > void *os_priv; > diff --unified --new-file --recursive mtd/util/Makefile mtd-dec07/util/Makefile > --- mtd/util/Makefile 2004-07-14 00:00:18.000000000 +0200 > +++ mtd-dec07/util/Makefile 2004-12-07 16:03:02.000000000 +0100 > @@ -13,7 +13,7 @@ > TARGETS = ftl_format flash_erase flash_eraseall nanddump doc_loadbios \ > mkfs.jffs ftl_check mkfs.jffs2 flash_lock flash_unlock \ > flash_info mtd_debug flashcp nandwrite jffs2dump \ > - nftldump nftl_format docfdisk #jffs2reader > + nftldump nftl_format docfdisk sumtool #jffs2reader > > SYMLINKS = compr_lzari.c compr_lzo.c > > @@ -48,6 +48,10 @@ > jffs2dump: jffs2dump.o crc32.o > $(CC) $(LDFLAGS) -o $@ $^ > > +sumtool: sumtool.o crc32.o > + $(CC) $(LDFLAGS) -o $@ $^ > + > + > install: ${TARGETS} > mkdir -p ${DESTDIR}/${SBINDIR} > install -m0755 -oroot -groot ${TARGETS} ${DESTDIR}/${SBINDIR}/ > diff --unified --new-file --recursive mtd/util/jffs2dump.c mtd-dec07/util/jffs2dump.c > --- mtd/util/jffs2dump.c 2004-12-03 00:00:10.000000000 +0100 > +++ mtd-dec07/util/jffs2dump.c 2004-12-07 16:03:02.000000000 +0100 > @@ -35,6 +35,10 @@ > #include <getopt.h> > #include "crc32.h" > > +#define CONFIG_JFFS2_SUMMARY > +#define JFFS2DUMP > +#include "../fs/jffs2/summary.h" > + > #define PROGRAM "jffs2dump" > #define VERSION "$Revision: 1.7 $" > > @@ -177,7 +181,7 @@ > */ > void do_dumpcontent (void) > { > - char *p = data; > + char *p = data, *p_free_begin; > union jffs2_node_union *node; > int empty = 0, dirty = 0; > char name[256]; > @@ -186,16 +190,21 @@ > int bitchbitmask = 0; > int obsolete; > > + p_free_begin = NULL; > while ( p < (data + imglen)) { > node = (union jffs2_node_union*) p; > > /* Skip empty space */ > + if (!p_free_begin) p_free_begin = p; > if (je16_to_cpu (node->u.magic) == 0xFFFF && je16_to_cpu (node->u.nodetype) == 0xFFFF) { > p += 4; > empty += 4; > continue; > } > - > + > + if (p != p_free_begin) printf("Empty space found from 0x%08x to 0x%08x\n", p_free_begin-data, p-data); > + p_free_begin = NULL; > + > if (je16_to_cpu (node->u.magic) != JFFS2_MAGIC_BITMASK) { > if (!bitchbitmask++) > printf ("Wrong bitmask at 0x%08x, 0x%04x\n", p - data, je16_to_cpu (node->u.magic)); > @@ -277,7 +286,110 @@ > > p += PAD(je32_to_cpu (node->d.totlen)); > break; > - > + > + case JFFS2_NODETYPE_SUMMARY:{ > + > + int i; > + struct jffs2_sum_marker * sm; > + > + printf ("%8s Inode Sum node at 0x%08x, totlen 0x%08x, sum_num %5d, cleanmarker size %5d\n", > + obsolete ? "Obsolete" : "", > + p - data, > + je32_to_cpu (node->s.totlen), > + je16_to_cpu (node->s.sum_num), > + je32_to_cpu (node->s.cln_mkr)); > + > + crc = crc32 (0, node, sizeof (struct jffs2_summary_node) - 8); > + if (crc != je32_to_cpu (node->s.node_crc)) { > + printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.node_crc), crc); > + p += PAD(je32_to_cpu (node->s.totlen)); > + dirty += PAD(je32_to_cpu (node->s.totlen));; > + continue; > + } > + > + crc = crc32(0, p + sizeof (struct jffs2_summary_node), je32_to_cpu (node->s.totlen) - sizeof(struct jffs2_summary_node)); > + if (crc != je32_to_cpu(node->s.sum_crc)) { > + printf ("Wrong data_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.sum_crc), crc); > + p += PAD(je32_to_cpu (node->s.totlen)); > + dirty += PAD(je32_to_cpu (node->s.totlen));; > + continue; > + } > + > + if(verbose){ > + struct jffs2_sum_unknown_flash *sp; > + uint8_t *temp8ptr = NULL; > + sp = (struct jffs2_sum_unknown_flash *) (p + sizeof (struct jffs2_summary_node)); > + > + for(i = 0; i < je16_to_cpu (node->s.sum_num); i++){ > + > + switch(je16_to_cpu(sp->nodetype)){ > + case JFFS2_NODETYPE_INODE : { > + > + struct jffs2_sum_inode_flash *spi; > + spi = (struct jffs2_sum_inode_flash *) sp; > + > + printf ("%14s #ino %5d, version %5d, offset 0x%08x, totlen 0x%08x\n", > + "", > + je32_to_cpu (spi->inode), > + je32_to_cpu (spi->version), > + je32_to_cpu (spi->offset), > + je32_to_cpu (spi->totlen)); > + > + temp8ptr = (uint8_t *) sp; > + temp8ptr += JFFS2_SUMMARY_INODE_SIZE; > + sp = (struct jffs2_sum_unknown_flash *) temp8ptr; > + > + break; > + } > + > + case JFFS2_NODETYPE_DIRENT : { > + > + char name[255]; > + struct jffs2_sum_dirent_flash *spd; > + spd = (struct jffs2_sum_dirent_flash *) sp; > + > + memcpy(name,spd->name,spd->nsize); > + name [spd->nsize] = 0x0; > + > + printf ("%14s dirent offset 0x%08x, totlen 0x%08x, #pino %5d, version %5d, #ino %8d, nsize %8d, name %s \n", > + "", > + je32_to_cpu (spd->offset), > + je32_to_cpu (spd->totlen), > + je32_to_cpu (spd->pino), > + je32_to_cpu (spd->version), > + je32_to_cpu (spd->ino), > + spd->nsize, > + name); > + > + temp8ptr = (uint8_t *) sp; > + temp8ptr += JFFS2_SUMMARY_DIRENT_SIZE(spd->nsize); > + sp = (struct jffs2_sum_unknown_flash *) temp8ptr; > + > + break; > + } > + > + default : { > + break; > + } > + } > + } > + > + > + sm = (struct jffs2_sum_marker *) ((char *)p + je32_to_cpu(node->s.totlen) - sizeof(struct jffs2_sum_marker)); > + > + printf("%14s Sum Node Offset 0x%08x, Erase_block_size 0x%08x, Magic 0x%08x, Padded size 0x%08x\n" , > + "", > + je32_to_cpu(sm->offset), > + je32_to_cpu(sm->erase_size), > + je32_to_cpu(sm->magic), > + je32_to_cpu(node->s.padded)); > + } > + > + p += PAD(je32_to_cpu (node->s.totlen)); > + break; > + > + } > + > case JFFS2_NODETYPE_CLEANMARKER: > if (verbose) { > printf ("%8s Cleanmarker at 0x%08x, totlen 0x%08x\n", > diff --unified --new-file --recursive mtd/util/sumtool.c mtd-dec07/util/sumtool.c > --- mtd/util/sumtool.c 1970-01-01 01:00:00.000000000 +0100 > +++ mtd-dec07/util/sumtool.c 2004-12-07 17:47:31.000000000 +0100 > @@ -0,0 +1,857 @@ > +/* > + * sumtool.c > + * > + * Copyright (C) 2004 Zoltan Sogor <weth@inf.u-szeged.hu>, > + * Ferenc Havasi <havasi@inf.u-szeged.hu>, > + * Patrik Kluba <pajko@halom.u-szeged.hu>, > + * University of Szeged, Hungary > + * > + * $Id: sumtool.c,v 1.2 2004/10/20 09:56:08 hafy Exp $ > + * > + * This program is free software; you can redistribute it and/or > + * modify it under the terms of the GNU General Public License > + * as published by the Free Software Foundation; either version 2 > + * of the License, or (at your option) any later version. > + * > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program; if not, write to the Free Software > + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. > + * > + * Overview: > + * This is a utility to reorder nodes and insert inode summary information > + * into JFFS2 image for faster mount time - specially on NAND. > + * > + */ > + > +#include <errno.h> > +#include <stdint.h> > +#include <stdio.h> > +#include <stdlib.h> > +#include <stdarg.h> > +#include <string.h> > +#include <unistd.h> > +#include <fcntl.h> > +#include <time.h> > +#include <sys/types.h> > +#include <sys/stat.h> > +#include <sys/param.h> > +#include <asm/types.h> > +#include <dirent.h> > +#include <mtd/jffs2-user.h> > +#include <endian.h> > +#include <byteswap.h> > +#include <getopt.h> > +#include "crc32.h" > + > +#define CONFIG_JFFS2_SUMMARY > +#define SUM_TOOL > +#include "../fs/jffs2/summary.h" > + > + > +#define PAD(x) (((x)+3)&~3) > + > +static const char *const app_name = "sumtool"; > + > +static struct jffs2_sum_info *sum_collected = NULL;/* summary info list */ > + > +static int verbose = 0; > +static int padto = 0; /* pad the output with 0xFF to the end of the final eraseblock */ > +static int add_cleanmarkers = 1; /* add cleanmarker to output */ > +static int use_input_cleanmarker_size = 1; /* use input file's cleanmarker size (default) */ > +static int found_cleanmarkers = 0; /* cleanmarker found in input file */ > +static struct jffs2_unknown_node cleanmarker; > +static int cleanmarker_size = sizeof(cleanmarker); > +static const char *short_options = "o:i:e:hvVblnc:p"; > +static int erase_block_size = 65536; > +static int target_endian = __BYTE_ORDER; > +static int out_fd = -1; > +static int in_fd = -1; > + > +static uint8_t *data_buffer = NULL; /* buffer for inodes */ > +static unsigned int data_ofs = 0; /* inode buffer offset */ > + > +static uint8_t *file_buffer = NULL; /* file buffer contains the actual erase block*/ > +static unsigned int file_ofs = 0; /* position in the buffer */ > + > +static struct option long_options[] = { > + {"output", 1, NULL, 'o'}, > + {"input", 1, NULL, 'i'}, > + {"eraseblock", 1, NULL, 'e'}, > + {"help", 0, NULL, 'h'}, > + {"verbose", 0, NULL, 'v'}, > + {"version", 0, NULL, 'V'}, > + {"bigendian", 0, NULL, 'b'}, > + {"littleendian", 0, NULL, 'l'}, > + {"no-cleanmarkers", 0, NULL, 'n'}, > + {"cleanmarker", 1, NULL, 'c'}, > + {"pad", 0, NULL, 'p'}, > + {NULL, 0, NULL, 0} > +}; > + > +static char *helptext = > + "Usage: sumtool [OPTIONS] -i inputfile -o outputfile\n" > + "Convert the input JFFS2 file to a SUM-ed JFFS2 file\n\n" > + "Options:\n" > + " -e, --eraseblock=SIZE Use erase block size SIZE (default: 64KiB)\n" > + " (usually 16KiB on NAND)\n" > + " -c, --cleanmarker=SIZE Size of cleanmarker (default 12).\n" > + " (usually 16 bytes on NAND, and will be set to\n" > + " this value if left at the default 12). Will be\n" > + " stored in OOB after each physical page composing\n" > + " a physical erase block.\n" > + " -n, --no-cleanmarkers Don't add a cleanmarker to every eraseblock\n" > + " -o, --output=FILE Output to FILE \n" > + " -i, --input=FILE Input from FILE \n" > + " -b, --bigendian Image is big endian\n" > + " -l --littleendian Image is little endian\n" > + " -h, --help Display this help text\n" > + " -v, --verbose Verbose operation\n" > + " -V, --version Display version information\n" > + " -p, --pad Pad the OUTPUT with 0xFF to the end of the final\n" > + " eraseblock\n\n"; > + > + > +static char *revtext = "$Revision: 1.3 $"; > + > +static unsigned char ffbuf[16] = { > + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, > + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff > +}; > + > +static void verror_msg(const char *s, va_list p) { > + fflush(stdout); > + fprintf(stderr, "%s: ", app_name); > + vfprintf(stderr, s, p); > +} > + > +static void error_msg_and_die(const char *s, ...) { > + va_list p; > + > + va_start(p, s); > + verror_msg(s, p); > + va_end(p); > + putc('\n', stderr); > + exit(EXIT_FAILURE); > +} > + > +static void vperror_msg(const char *s, va_list p) { > + int err = errno; > + > + if (s == 0) > + s = ""; > + verror_msg(s, p); > + if (*s) > + s = ": "; > + fprintf(stderr, "%s%s\n", s, strerror(err)); > +} > + > +static void perror_msg_and_die(const char *s, ...) { > + va_list p; > + > + va_start(p, s); > + vperror_msg(s, p); > + va_end(p); > + exit(EXIT_FAILURE); > +} > + > + > + > +static void full_write(void *target_buff, const void *buf, int len); > + > +void setup_cleanmarker() { > + > + cleanmarker.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK); > + cleanmarker.nodetype = cpu_to_je16(JFFS2_NODETYPE_CLEANMARKER); > + cleanmarker.totlen = cpu_to_je32(cleanmarker_size); > + cleanmarker.hdr_crc = cpu_to_je32(crc32(0, &cleanmarker, sizeof(struct jffs2_unknown_node)-4)); > +} > + > +void process_options (int argc, char **argv){ > + int opt,c; > + > + while ((opt = getopt_long(argc, argv, short_options, long_options, &c)) >= 0) > + { > + switch (opt) > + { > + case 'o': > + if (out_fd != -1) { > + error_msg_and_die("output filename specified more than once"); > + } > + out_fd = open(optarg, O_CREAT | O_TRUNC | O_RDWR, 0644); > + if (out_fd == -1) { > + perror_msg_and_die("open output file"); > + } > + break; > + > + case 'i': > + if (in_fd != -1) { > + error_msg_and_die("input filename specified more than once"); > + } > + in_fd = open(optarg, O_RDONLY); > + if (in_fd == -1) { > + perror_msg_and_die("open input file"); > + } > + break; > + case 'b': > + target_endian = __BIG_ENDIAN; > + break; > + case 'l': > + target_endian = __LITTLE_ENDIAN; > + break; > + case 'h': > + case '?': > + error_msg_and_die(helptext); > + > + case 'v': > + verbose = 1; > + break; > + > + case 'V': > + error_msg_and_die("revision %.*s\n", > + (int) strlen(revtext) - 13, revtext + 11); > + > + case 'e': { > + char *next; > + unsigned units = 0; > + erase_block_size = strtol(optarg, &next, 0); > + if (!erase_block_size) > + error_msg_and_die("Unrecognisable erase size\n"); > + > + if (*next) { > + if (!strcmp(next, "KiB")) { > + units = 1024; > + } else if (!strcmp(next, "MiB")) { > + units = 1024 * 1024; > + } else { > + error_msg_and_die("Unknown units in erasesize\n"); > + } > + } else { > + if (erase_block_size < 0x1000) > + units = 1024; > + else > + units = 1; > + } > + erase_block_size *= units; > + > + /* If it's less than 8KiB, they're not allowed */ > + if (erase_block_size < 0x2000) { > + fprintf(stderr, "Erase size 0x%x too small. Increasing to 8KiB minimum\n", > + erase_block_size); > + erase_block_size = 0x2000; > + } > + break; > + } > + > + case 'n': > + add_cleanmarkers = 0; > + break; > + case 'c': > + cleanmarker_size = strtol(optarg, NULL, 0); > + > + if (cleanmarker_size < sizeof(cleanmarker)) { > + error_msg_and_die("cleanmarker size must be >= 12"); > + } > + if (cleanmarker_size >= erase_block_size) { > + error_msg_and_die("cleanmarker size must be < eraseblock size"); > + } > + > + use_input_cleanmarker_size = 0; > + found_cleanmarkers = 1; > + setup_cleanmarker(); > + > + break; > + case 'p': > + padto = 1; > + break; > + > + } > + } > +} > + > + > +void init_buffers() { > + > + data_buffer = malloc(erase_block_size); > + > + if (!data_buffer) { > + perror("out of memory"); > + close (in_fd); > + close (out_fd); > + exit(1); > + } > + > + file_buffer = malloc(erase_block_size); > + > + if (!file_buffer) { > + perror("out of memory"); > + close (in_fd); > + close (out_fd); > + exit(1); > + } > +} > + > +void init_sumlist(){ > + > + sum_collected = (struct jffs2_sum_info *) malloc (sizeof(struct jffs2_sum_info)); > + > + if (!sum_collected) > + error_msg_and_die("Can't allocate memory for jffs2_sum_info!\n"); > + > + sum_collected->sum_list = NULL; > + sum_collected->sum_num = 0; > + sum_collected->sum_size = 0; > + > +} > + > +void clean_buffers() { > + > + if (data_buffer) > + free(data_buffer); > + if (file_buffer) > + free(file_buffer); > +} > + > +void clean_sumlist() { > + > + union jffs2_sum_mem *temp; > + > + if(sum_collected){ > + > + while(sum_collected->sum_list){ > + temp = sum_collected->sum_list; > + sum_collected->sum_list = sum_collected->sum_list->u.next; > + free(temp); > + sum_collected->sum_num--; > + } > + > + if(sum_collected->sum_num != 0) > + printf("Ooops, something wrong happened! sum_num != 0, but sum_list = null ???"); > + > + free(sum_collected); > + } > +} > + > +int load_next_block() { > + > + int ret; > + ret = read(in_fd, file_buffer, erase_block_size); > + file_ofs = 0; > + > + if(verbose) > + printf("Load next block : %d bytes read\n",ret); > + > + return ret; > +} > + > +void write_buff_to_file() { > + > + int ret; > + int len = data_ofs; > + > + uint8_t *buf = NULL; > + > + buf = data_buffer; > + while (len > 0) { > + ret = write(out_fd, buf, len); > + > + if (ret < 0) > + perror_msg_and_die("write"); > + > + if (ret == 0) > + perror_msg_and_die("write returned zero"); > + > + len -= ret; > + buf += ret; > + } > + > + data_ofs = 0; > +} > + > +void dump_sum_records() { > + > + struct jffs2_summary_node isum; > + union jffs2_sum_mem *temp; > + jint32_t offset; > + jint32_t *wpage; > + uint8_t *tempptr; > + int datasize; > + int infosize; > + int padsize; > + jint32_t magic = cpu_to_je32(JFFS2_SUM_MAGIC); > + > + if (!sum_collected->sum_num || !sum_collected->sum_list) > + return; > + > + datasize = sum_collected->sum_size + sizeof(struct jffs2_sum_marker); > + infosize = sizeof(struct jffs2_summary_node) + datasize; > + padsize = erase_block_size - data_ofs - infosize; > + infosize += padsize; datasize += padsize; > + offset = cpu_to_je32(data_ofs); > + > + jint32_t *tpage = (jint32_t *) malloc(datasize); > + > + if(!tpage) > + error_msg_and_die("Can't allocate memory to dump summary information!\n"); > + > + memset(tpage, 0xff, datasize); > + memset(&isum, 0, sizeof(isum)); > + > + isum.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK); > + isum.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY); > + isum.totlen = cpu_to_je32(infosize); > + isum.hdr_crc = cpu_to_je32(crc32(0, &isum, sizeof(struct jffs2_unknown_node) - 4)); > + isum.padded = cpu_to_je32(0); > + > + if (add_cleanmarkers && found_cleanmarkers) { > + isum.cln_mkr = cpu_to_je32(cleanmarker_size); > + } > + else{ > + isum.cln_mkr = cpu_to_je32(0); > + } > + > + isum.sum_num = cpu_to_je16(sum_collected->sum_num); > + wpage = tpage; > + > + > + while (sum_collected->sum_num) { > + > + > + switch(je16_to_cpu(sum_collected->sum_list->u.nodetype)){ > + > + case JFFS2_NODETYPE_INODE : { > + jint16_t *temp16ptr = (jint16_t *)wpage; > + > + *(temp16ptr++) = sum_collected->sum_list->i.nodetype; > + wpage = (jint32_t *) temp16ptr; > + > + *(wpage++) = sum_collected->sum_list->i.inode; > + *(wpage++) = sum_collected->sum_list->i.version; > + *(wpage++) = sum_collected->sum_list->i.offset; > + *(wpage++) = sum_collected->sum_list->i.totlen; > + break; > + } > + > + case JFFS2_NODETYPE_DIRENT : { > + jint16_t *temp16ptr = (jint16_t *) wpage; > + uint8_t *temp8ptr = NULL; > + > + *(temp16ptr++) = sum_collected->sum_list->d.nodetype; > + wpage = (jint32_t *) temp16ptr; > + > + *(wpage++) = sum_collected->sum_list->d.totlen; > + *(wpage++) = sum_collected->sum_list->d.offset; > + *(wpage++) = sum_collected->sum_list->d.pino; > + *(wpage++) = sum_collected->sum_list->d.version; > + *(wpage++) = sum_collected->sum_list->d.ino; > + > + temp8ptr = (uint8_t *) wpage; > + *(temp8ptr++) = sum_collected->sum_list->d.nsize; > + *(temp8ptr++) = sum_collected->sum_list->d.type; > + > + memcpy(temp8ptr,sum_collected->sum_list->d.name,sum_collected->sum_list->d.nsize); > + temp8ptr += sum_collected->sum_list->d.nsize; > + wpage = (jint32_t *) temp8ptr; > + > + break; > + } > + > + default : { > + printf("This is not good for me!!!!\n"); > + } > + } > + > + temp = sum_collected->sum_list; > + sum_collected->sum_list = sum_collected->sum_list->u.next; > + free(temp); > + > + sum_collected->sum_num--; > + } > + > + sum_collected->sum_size = 0; > + > + tempptr = (uint8_t *) wpage; > + tempptr += padsize; > + wpage = (jint32_t *) tempptr; > + > + *(wpage++) = offset; > + *(wpage++) = cpu_to_je32(erase_block_size); > + *(wpage++) = magic; > + isum.sum_crc = cpu_to_je32(crc32(0, tpage, datasize)); > + isum.node_crc = cpu_to_je32(crc32(0, &isum, sizeof(isum) - 8)); > + > + full_write(data_buffer + data_ofs, &isum, sizeof(isum)); > + full_write(data_buffer + data_ofs, tpage, datasize); > + > + free(tpage); > + > +} > + > +static void full_write(void *target_buff, const void *buf, int len) { > + > + memcpy(target_buff, buf, len); > + data_ofs += len; > +} > + > +static void pad(int req) { > + while (req) { > + if (req > sizeof(ffbuf)) { > + full_write(data_buffer + data_ofs, ffbuf, sizeof(ffbuf)); > + req -= sizeof(ffbuf); > + } else { > + full_write(data_buffer + data_ofs, ffbuf, req); > + req = 0; > + } > + } > +} > + > +static inline void padword() { > + > + if (data_ofs % 4) { > + full_write(data_buffer + data_ofs, ffbuf, 4 - (data_ofs % 4)); > + } > +} > + > + > +static inline void pad_block_if_less_than(int req,int plus) { > + > + int datasize = req + plus + sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8; > + datasize += (4 - (datasize % 4)) % 4; > + > + if (data_ofs + req > erase_block_size - datasize) { > + dump_sum_records(); > + write_buff_to_file(); > + } > + > + if (add_cleanmarkers && found_cleanmarkers) { > + if (!data_ofs) { > + full_write(data_buffer, &cleanmarker, sizeof(cleanmarker)); > + pad(cleanmarker_size - sizeof(cleanmarker)); > + padword(); > + } > + } > +} > + > +void flush_buffers() { > + > + if ((add_cleanmarkers == 1) && (found_cleanmarkers == 1)) { /* CLEANMARKER */ > + if (data_ofs != cleanmarker_size) { /* INODE BUFFER */ > + > + int datasize = sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8; > + datasize += (4 - (datasize % 4)) % 4; > + > + /* If we have a full inode buffer, then write out inode and summary data */ > + if (data_ofs + sizeof(struct jffs2_raw_inode) + 2*JFFS2_MIN_DATA_LEN > erase_block_size - datasize) { > + dump_sum_records(); > + write_buff_to_file(); > + } > + /* else just write out inode data */ > + else{ > + if(padto) > + pad(erase_block_size - data_ofs); > + write_buff_to_file(); > + } > + } > + } > + else { /* NO CLEANMARKER */ > + if (data_ofs != 0) { /* INODE BUFFER */ > + > + int datasize = sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8; > + datasize += (4 - (datasize % 4)) % 4; > + > + /* If we have a full inode buffer, then write out inode and summary data */ > + if (data_ofs + sizeof(struct jffs2_raw_inode) + 2*JFFS2_MIN_DATA_LEN > erase_block_size - datasize) { > + dump_sum_records(); > + write_buff_to_file(); > + } > + /* Else just write out inode data */ > + else{ > + if(padto) > + pad(erase_block_size - data_ofs); > + write_buff_to_file(); > + } > + } > + } > +} > + > +int add_sum_mem(union jffs2_sum_mem *item) { > + > + union jffs2_sum_mem *walk; > + > + if (!sum_collected->sum_list) { > + sum_collected->sum_list = (union jffs2_sum_mem *) item; > + } > + else { > + walk = sum_collected->sum_list; > + > + while (walk->u.next) { > + walk = walk->u.next; > + } > + walk->u.next = (union jffs2_sum_mem *) item; > + } > + switch (je16_to_cpu(item->u.nodetype)) { > + case JFFS2_NODETYPE_INODE: > + sum_collected->sum_size += JFFS2_SUMMARY_INODE_SIZE; > + sum_collected->sum_num++; > + break; > + > + case JFFS2_NODETYPE_DIRENT: > + sum_collected->sum_size += JFFS2_SUMMARY_DIRENT_SIZE(item->d.nsize); > + sum_collected->sum_num++; > + break; > + > + default: > + error_msg_and_die("__jffs2_add_sum_mem(): UNKNOWN node type %d\n", je16_to_cpu(item->u.nodetype)); > + } > + return 0; > +} > + > +void add_sum_inode_mem(union jffs2_node_union *node) { > + > + struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *) malloc(sizeof(struct jffs2_sum_inode_mem)); > + > + if (!temp) > + error_msg_and_die("Can't allocate memory for summary information!\n"); > + > + temp->nodetype = node->i.nodetype; > + temp->inode = node->i.ino; > + temp->version = node->i.version; > + temp->offset = cpu_to_je32(data_ofs); > + temp->totlen = node->i.totlen; > + temp->next = NULL; > + > + add_sum_mem((union jffs2_sum_mem *) temp); > + > +} > + > +void add_sum_dirent_mem(union jffs2_node_union *node) { > + > + struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *) > + malloc(sizeof(struct jffs2_sum_dirent_mem) + node->d.nsize); > + > + if (!temp) > + error_msg_and_die("Can't allocate memory for summary information!\n"); > + > + temp->nodetype = node->d.nodetype; > + temp->totlen = node->d.totlen; > + temp->offset = cpu_to_je32(data_ofs); > + temp->pino = node->d.pino; > + temp->version = node->d.version; > + temp->ino = node->d.ino; > + temp->nsize = node->d.nsize; > + temp->type = node->d.type; > + temp->next = NULL; > + > + memcpy(temp->name,node->d.name,node->d.nsize); > + > + add_sum_mem((union jffs2_sum_mem *) temp); > + > +} > + > +void write_dirent_to_buff(union jffs2_node_union *node) { > + > + pad_block_if_less_than(je32_to_cpu (node->d.totlen),JFFS2_SUMMARY_DIRENT_SIZE(node->d.nsize)); > + add_sum_dirent_mem(node); > + full_write(data_buffer + data_ofs, &(node->d), je32_to_cpu (node->d.totlen)); > + padword(); > +} > + > + > +void write_inode_to_buff(union jffs2_node_union *node) { > + > + pad_block_if_less_than(je32_to_cpu (node->i.totlen),JFFS2_SUMMARY_INODE_SIZE); > + add_sum_inode_mem(node); /* Add inode summary mem to summary list */ > + full_write(data_buffer + data_ofs, &(node->i), je32_to_cpu (node->i.totlen)); /* Write out the inode to inode_buffer */ > + padword(); > + > +} > + > + > +void create_summed_image(int inp_size) { > + uint8_t *p = file_buffer; > + union jffs2_node_union *node; > + uint32_t crc; > + uint16_t type; > + int bitchbitmask = 0; > + int obsolete; > + > + char name[256]; > + > + while ( p < (file_buffer + inp_size)) { > + > + node = (union jffs2_node_union *) p; > + > + /* Skip empty space */ > + if (je16_to_cpu (node->u.magic) == 0xFFFF && je16_to_cpu (node->u.nodetype) == 0xFFFF) { > + p += 4; > + continue; > + } > + > + if (je16_to_cpu (node->u.magic) != JFFS2_MAGIC_BITMASK) { > + if (!bitchbitmask++) > + printf ("Wrong bitmask at 0x%08x, 0x%04x\n", p - file_buffer, je16_to_cpu (node->u.magic)); > + p += 4; > + continue; > + } > + > + bitchbitmask = 0; > + > + type = je16_to_cpu(node->u.nodetype); > + if ((type & JFFS2_NODE_ACCURATE) != JFFS2_NODE_ACCURATE) { > + obsolete = 1; > + type |= JFFS2_NODE_ACCURATE; > + } else > + obsolete = 0; > + > + node->u.nodetype = cpu_to_je16(type); > + > + crc = crc32 (0, node, sizeof (struct jffs2_unknown_node) - 4); > + if (crc != je32_to_cpu (node->u.hdr_crc)) { > + printf ("Wrong hdr_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->u.hdr_crc), crc); > + p += 4; > + continue; > + } > + > + switch(je16_to_cpu(node->u.nodetype)) { > + > + case JFFS2_NODETYPE_INODE: > + if(verbose) > + printf ("%8s Inode node at 0x%08x, totlen 0x%08x, #ino %5d, version %5d, isize %8d, csize %8d, dsize %8d, offset %8d\n", > + obsolete ? "Obsolete" : "", > + p - file_buffer, je32_to_cpu (node->i.totlen), je32_to_cpu (node->i.ino), > + je32_to_cpu ( node->i.version), je32_to_cpu (node->i.isize), > + je32_to_cpu (node->i.csize), je32_to_cpu (node->i.dsize), je32_to_cpu (node->i.offset)); > + > + crc = crc32 (0, node, sizeof (struct jffs2_raw_inode) - 8); > + if (crc != je32_to_cpu (node->i.node_crc)) { > + printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.node_crc), crc); > + p += PAD(je32_to_cpu (node->i.totlen)); > + continue; > + } > + > + crc = crc32(0, p + sizeof (struct jffs2_raw_inode), je32_to_cpu(node->i.csize)); > + if (crc != je32_to_cpu(node->i.data_crc)) { > + printf ("Wrong data_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.data_crc), crc); > + p += PAD(je32_to_cpu (node->i.totlen)); > + continue; > + } > + > + write_inode_to_buff(node); > + > + p += PAD(je32_to_cpu (node->i.totlen)); > + break; > + > + case JFFS2_NODETYPE_DIRENT: > + memcpy (name, node->d.name, node->d.nsize); > + name [node->d.nsize] = 0x0; > + > + if(verbose) > + printf ("%8s Dirent node at 0x%08x, totlen 0x%08x, #pino %5d, version %5d, #ino %8d, nsize %8d, name %s\n", > + obsolete ? "Obsolete" : "", > + p - file_buffer, je32_to_cpu (node->d.totlen), je32_to_cpu (node->d.pino), > + je32_to_cpu ( node->d.version), je32_to_cpu (node->d.ino), > + node->d.nsize, name); > + > + crc = crc32 (0, node, sizeof (struct jffs2_raw_dirent) - 8); > + if (crc != je32_to_cpu (node->d.node_crc)) { > + printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.node_crc), crc); > + p += PAD(je32_to_cpu (node->d.totlen)); > + continue; > + } > + > + crc = crc32(0, p + sizeof (struct jffs2_raw_dirent), node->d.nsize); > + if (crc != je32_to_cpu(node->d.name_crc)) { > + printf ("Wrong name_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.name_crc), crc); > + p += PAD(je32_to_cpu (node->d.totlen)); > + continue; > + } > + > + write_dirent_to_buff(node); > + > + p += PAD(je32_to_cpu (node->d.totlen)); > + break; > + > + case JFFS2_NODETYPE_CLEANMARKER: > + if (verbose) { > + printf ("%8s Cleanmarker at 0x%08x, totlen 0x%08x\n", > + obsolete ? "Obsolete" : "", > + p - file_buffer, je32_to_cpu (node->u.totlen)); > + } > + > + if(!found_cleanmarkers){ > + found_cleanmarkers = 1; > + > + if(add_cleanmarkers == 1 && use_input_cleanmarker_size == 1){ > + cleanmarker_size = je32_to_cpu (node->u.totlen); > + setup_cleanmarker(); > + } > + } > + > + p += PAD(je32_to_cpu (node->u.totlen)); > + break; > + > + case JFFS2_NODETYPE_PADDING: > + if (verbose) { > + printf ("%8s Padding node at 0x%08x, totlen 0x%08x\n", > + obsolete ? "Obsolete" : "", > + p - file_buffer, je32_to_cpu (node->u.totlen)); > + } > + p += PAD(je32_to_cpu (node->u.totlen)); > + break; > + > + case 0xffff: > + p += 4; > + break; > + > + default: > + if (verbose) { > + printf ("%8s Unknown node at 0x%08x, totlen 0x%08x\n", > + obsolete ? "Obsolete" : "", > + p - file_buffer, je32_to_cpu (node->u.totlen)); > + } > + > + //write_unknown_to_buff(node); > + > + p += PAD(je32_to_cpu (node->u.totlen)); > + } > + } > +} > + > +int main(int argc, char **argv) { > + > + int ret; > + > + process_options(argc,argv); > + > + if ((in_fd == -1) || (out_fd == -1)) { > + > + if(in_fd != -1) > + close(in_fd); > + if(out_fd != -1) > + close(out_fd); > + > + error_msg_and_die("You must specify input and output files!\n"); > + } > + > + init_buffers(); > + init_sumlist(); > + > + while ((ret = load_next_block())) { > + create_summed_image(ret); > + } > + > + flush_buffers(); > + clean_buffers(); > + clean_sumlist(); > + > + if (in_fd != -1) > + close(in_fd); > + if (out_fd != -1) > + close(out_fd); > + > + return 0; > +} > ______________________________________________________ > Linux MTD discussion mailing list > http://lists.infradead.org/mailman/listinfo/linux-mtd/ ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-20 16:01 ` Gareth Bult (Encryptec) @ 2004-12-20 16:09 ` Ferenc Havasi 2004-12-20 16:39 ` Gareth Bult (Encryptec) 2004-12-20 17:48 ` Gareth Bult (Encryptec) 0 siblings, 2 replies; 82+ messages in thread From: Ferenc Havasi @ 2004-12-20 16:09 UTC (permalink / raw) To: Gareth Bult (Encryptec); +Cc: Linux MTD Hi Gareth, > I'm afraid this patch seems to be quite a long way from applying cleanly > to my source. I'm using 2.6.9 .. at least half a dozen failures or > more ... It works with JFFS2 in the latest MTD CVS. JFFS3 will contains this feature soon. Regards, Ferenc ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-20 16:09 ` Ferenc Havasi @ 2004-12-20 16:39 ` Gareth Bult (Encryptec) 2004-12-20 17:48 ` Gareth Bult (Encryptec) 1 sibling, 0 replies; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-20 16:39 UTC (permalink / raw) To: Ferenc Havasi; +Cc: Linux MTD [-- Attachment #1: Type: text/plain, Size: 656 bytes --] Yup, I've cvs'd the latest source and patched ... kernel seems to build Ok .. Just trying a full build from clean to make sure, then I'll give it a go .. Is there an example of how to use sumtool anywhere ? If I eraseall the flash then mount JFFS2 and copy files on, do I need to use sumtool ? Gareth. On Mon, 2004-12-20 at 17:09 +0100, Ferenc Havasi wrote: > Hi Gareth, > > > I'm afraid this patch seems to be quite a long way from applying cleanly > > to my source. I'm using 2.6.9 .. at least half a dozen failures or > > more ... > > It works with JFFS2 in the latest MTD CVS. > > JFFS3 will contains this feature soon. > > Regards, > Ferenc [-- Attachment #2: Type: text/html, Size: 1194 bytes --] ^ permalink raw reply [flat|nested] 82+ messages in thread
* Re: JFFS2 mount time 2004-12-20 16:09 ` Ferenc Havasi 2004-12-20 16:39 ` Gareth Bult (Encryptec) @ 2004-12-20 17:48 ` Gareth Bult (Encryptec) 1 sibling, 0 replies; 82+ messages in thread From: Gareth Bult (Encryptec) @ 2004-12-20 17:48 UTC (permalink / raw) To: Ferenc Havasi; +Cc: Linux MTD Hi, Patches applied and it seems to be working .. After dd'ing a live FS off the key, sumtool'ing it and dd'ing it back on, it seems to have cut mount time from 31 seconds to 11 seconds ... This is certainly much better .. any way of making it quicker .. ;-) Gareth. On Mon, 2004-12-20 at 17:09 +0100, Ferenc Havasi wrote: > Hi Gareth, > > > I'm afraid this patch seems to be quite a long way from applying cleanly > > to my source. I'm using 2.6.9 .. at least half a dozen failures or > > more ... > > It works with JFFS2 in the latest MTD CVS. > > JFFS3 will contains this feature soon. > > Regards, > Ferenc ^ permalink raw reply [flat|nested] 82+ messages in thread
end of thread, other threads:[~2005-01-11 8:34 UTC | newest] Thread overview: 82+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2004-01-13 12:50 JFFS2 mount time Jarkko Lavinen (NMP/Helsinki) 2004-01-13 13:19 ` Joakim Tjernlund 2004-01-13 13:39 ` David Woodhouse 2004-01-13 15:25 ` Kenneth Johansson 2004-01-13 15:30 ` David Woodhouse 2004-01-13 16:21 ` Kenneth Johansson 2004-01-13 17:29 ` Jörn Engel 2004-01-13 17:40 ` Kenneth Johansson 2004-01-13 18:43 ` Jörn Engel 2005-01-04 15:00 ` Steven Scholz 2005-01-05 10:45 ` Artem B. Bityuckiy 2005-01-05 11:10 ` Ferenc Havasi 2005-01-05 11:24 ` Steven Scholz 2005-01-05 11:45 ` Artem B. Bityuckiy 2005-01-06 10:50 ` Ferenc Havasi 2005-01-10 15:14 ` Steven Scholz 2005-01-10 15:25 ` Steven Scholz 2005-01-11 8:26 ` Ferenc Havasi 2005-01-11 8:34 ` Steven Scholz 2004-10-20 14:26 Ferenc Havasi 2004-10-20 15:26 ` [OBORONA-SPAM] " Artem B. Bityuckiy 2004-10-20 15:49 ` Ferenc Havasi 2004-10-20 15:53 ` Artem B. Bityuckiy 2004-10-21 6:29 ` Artem B. Bityuckiy 2004-10-21 6:54 ` Ferenc Havasi 2004-10-21 7:16 ` Artem B. Bityuckiy 2004-10-21 19:50 ` Ferenc Havasi 2004-10-21 12:49 ` Jarkko Lavinen 2004-10-21 19:11 ` Ferenc Havasi 2004-10-22 9:58 ` Ferenc Havasi 2004-10-21 13:24 ` David Woodhouse 2004-10-21 20:05 ` Ferenc Havasi 2004-10-22 12:44 ` Artem Bityuckiy 2004-10-25 9:36 ` Ferenc Havasi 2004-10-25 10:56 ` Artem Bityuckiy 2004-10-25 15:30 ` Ferenc Havasi 2004-10-26 9:59 ` Artem Bityuckiy 2004-10-26 10:21 ` Ferenc Havasi 2004-10-26 11:05 ` Artem Bityuckiy 2004-10-26 13:52 ` Ferenc Havasi 2004-10-25 11:21 ` Artem Bityuckiy 2004-10-26 9:29 ` Jarkko Lavinen 2004-10-26 10:24 ` Ferenc Havasi 2004-10-26 10:34 ` Artem Bityuckiy 2004-12-15 23:19 Gareth Bult (Encryptec) 2004-12-16 0:15 ` Josh Boyer 2004-12-16 1:02 ` Gareth Bult (Encryptec) 2004-12-16 12:53 ` Josh Boyer 2004-12-16 21:22 ` Gareth Bult (Encryptec) 2004-12-16 21:28 ` Josh Boyer 2004-12-16 21:47 ` Gareth Bult (Encryptec) 2004-12-17 12:54 ` Josh Boyer 2004-12-17 15:33 ` Gareth Bult (Encryptec) 2004-12-17 16:02 ` Josh Boyer 2004-12-17 16:46 ` Gareth Bult (Encryptec) 2004-12-17 17:08 ` Artem B. Bityuckiy 2004-12-17 17:10 ` Josh Boyer 2004-12-17 17:26 ` Gareth Bult (Encryptec) 2004-12-17 17:35 ` Josh Boyer 2004-12-17 18:09 ` Gareth Bult (Encryptec) 2004-12-17 19:14 ` jasmine 2004-12-17 20:55 ` Gareth Bult (Encryptec) 2004-12-18 16:02 ` Jörn Engel 2004-12-20 16:34 ` Josh Boyer 2004-12-20 17:12 ` Gareth Bult (Encryptec) 2004-12-21 13:09 ` Jörn Engel 2004-12-21 13:24 ` Gareth Bult (Encryptec) 2004-12-21 13:34 ` Jörn Engel 2004-12-18 16:19 ` Jörn Engel 2004-12-18 17:32 ` Gareth Bult (Encryptec) 2004-12-18 17:52 ` Jörn Engel 2004-12-18 18:11 ` Jörn Engel 2004-12-18 20:48 ` Gareth Bult (Encryptec) 2004-12-19 2:44 ` Jörn Engel 2004-12-21 13:30 ` Jörn Engel 2004-12-21 13:40 ` Gareth Bult (Encryptec) 2004-12-21 15:00 ` David Woodhouse [not found] ` <1103644945.10792.175.camel@squizzey.bult.co.uk> 2004-12-21 16:04 ` Jörn Engel 2004-12-16 13:43 ` Ferenc Havasi 2004-12-20 16:01 ` Gareth Bult (Encryptec) 2004-12-20 16:09 ` Ferenc Havasi 2004-12-20 16:39 ` Gareth Bult (Encryptec) 2004-12-20 17:48 ` Gareth Bult (Encryptec)
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.