* JFFS2 mount time
@ 2004-12-15 23:19 Gareth Bult (Encryptec)
2004-12-16 0:15 ` Josh Boyer
2004-12-16 13:43 ` Ferenc Havasi
0 siblings, 2 replies; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-15 23:19 UTC (permalink / raw)
To: Linux MTD
Hi,
I've been working on the MTD block driver for JFFS2 and now seem to have
a relatively stable environment with a JFFS2 root filesystem on a USB
flash key device.
(mtd block device speed now approximates actual device speed on both
read and write)
It seems to be much quicker (generally) than the previous ext2
incarnation.
However, I'm looking at 30secs to mount the root filesystem.
I would be very grateful of some advice on the following before I start
digging through the JFFS2 code itself;
a. Is it possible (conceptually) to background the mount scan so the
system can boot while the scan happens ?
b. Is there an easy way to bypass the scan and does JFFS3 do this ?
Any help/advice would be much appreciated.
Many thanks,
Gareth.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-15 23:19 JFFS2 mount time Gareth Bult (Encryptec)
@ 2004-12-16 0:15 ` Josh Boyer
2004-12-16 1:02 ` Gareth Bult (Encryptec)
2004-12-16 13:43 ` Ferenc Havasi
1 sibling, 1 reply; 113+ messages in thread
From: Josh Boyer @ 2004-12-16 0:15 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Gareth Bult (Encryptec) wrote:
> Hi,
>
> I've been working on the MTD block driver for JFFS2 and now seem to have
> a relatively stable environment with a JFFS2 root filesystem on a USB
> flash key device.
> (mtd block device speed now approximates actual device speed on both
> read and write)
I assume you mean blkmtd here. The mtdblock driver is used to present
an IDE type interface to flash, whereas blkmtd is used to present a
flash interface to an IDE type device.
>
> It seems to be much quicker (generally) than the previous ext2
> incarnation.
>
> However, I'm looking at 30secs to mount the root filesystem.
How big of a device?
> a. Is it possible (conceptually) to background the mount scan so the
> system can boot while the scan happens ?
Which version of the code are you using? Mount time has been
significantly improved in newer versions of the code.
> b. Is there an easy way to bypass the scan and does JFFS3 do this ?
Not that I know of, because the code needs to read the node information
in order to get the versioning of inodes, etc. correct.
JFFS3 is essentially the exact same code base as JFFS2 right now. It
was just forked about a week ago.
josh
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-16 0:15 ` Josh Boyer
@ 2004-12-16 1:02 ` Gareth Bult (Encryptec)
2004-12-16 12:53 ` Josh Boyer
0 siblings, 1 reply; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-16 1:02 UTC (permalink / raw)
To: Josh Boyer; +Cc: Linux MTD
Hi,
Sure, the file I've been changing is blkmtd.c.
The device I'm aiming for and testing on is 256Mb with a 128k erase
size.
Currently I'm using the code included in the 2.6.9 kernel source tree
with tools from cvs as of about a week ago.
Is this likely to contain the latest speedups or should I be looking
elsewhere?
tia
Gareth.
On Wed, 2004-12-15 at 18:15 -0600, Josh Boyer wrote:
> Gareth Bult (Encryptec) wrote:
> > Hi,
> >
> > I've been working on the MTD block driver for JFFS2 and now seem to have
> > a relatively stable environment with a JFFS2 root filesystem on a USB
> > flash key device.
> > (mtd block device speed now approximates actual device speed on both
> > read and write)
>
> I assume you mean blkmtd here. The mtdblock driver is used to present
> an IDE type interface to flash, whereas blkmtd is used to present a
> flash interface to an IDE type device.
>
> >
> > It seems to be much quicker (generally) than the previous ext2
> > incarnation.
> >
> > However, I'm looking at 30secs to mount the root filesystem.
>
> How big of a device?
>
> > a. Is it possible (conceptually) to background the mount scan so the
> > system can boot while the scan happens ?
>
> Which version of the code are you using? Mount time has been
> significantly improved in newer versions of the code.
>
> > b. Is there an easy way to bypass the scan and does JFFS3 do this ?
>
> Not that I know of, because the code needs to read the node information
> in order to get the versioning of inodes, etc. correct.
>
> JFFS3 is essentially the exact same code base as JFFS2 right now. It
> was just forked about a week ago.
>
> josh
>
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-16 1:02 ` Gareth Bult (Encryptec)
@ 2004-12-16 12:53 ` Josh Boyer
2004-12-16 21:22 ` Gareth Bult (Encryptec)
0 siblings, 1 reply; 113+ messages in thread
From: Josh Boyer @ 2004-12-16 12:53 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
On Wed, 2004-12-15 at 19:02, Gareth Bult (Encryptec) wrote:
> Hi,
Quick FYI, don't top post. The email gremlin will chase you down and
make you read http://david.woodhou.se/email.html if you break too many
rules ;)
>
> Sure, the file I've been changing is blkmtd.c.
Ok, cool.
>
> The device I'm aiming for and testing on is 256Mb with a 128k erase
> size.
>
> Currently I'm using the code included in the 2.6.9 kernel source tree
> with tools from cvs as of about a week ago.
Ok, good.
> Is this likely to contain the latest speedups or should I be looking
> elsewhere?
That should have the speedups I was referring to. Basically checking
the node CRCs is deferred until after mount time so the mount succeeds
much more quickly.
There is one more feature pending that I know of. Ferenc Havasi has
some eraseblock summary changes that should help. I believe the
intention was to commit that to JFFS3, but I haven't seen it go in yet.
josh
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-15 23:19 JFFS2 mount time Gareth Bult (Encryptec)
2004-12-16 0:15 ` Josh Boyer
@ 2004-12-16 13:43 ` Ferenc Havasi
2004-12-20 16:01 ` Gareth Bult (Encryptec)
1 sibling, 1 reply; 113+ messages in thread
From: Ferenc Havasi @ 2004-12-16 13:43 UTC (permalink / raw)
To: Gareth Bult; +Cc: Linux MTD
[-- Attachment #1: Type: text/plain, Size: 343 bytes --]
Hi Gareth,
> However, I'm looking at 30secs to mount the root filesystem.
Did you try our patch? I send it now again, we made it fresh for the
current CVS. See its usage in the archive (16th November).
It will part of JFFS3.
This is designed for NAND devices - I don't know too much about your
environment, but it may help.
Bye,
Ferenc
[-- Attachment #2: jffs2-summary-20041207.patch --]
[-- Type: text/plain, Size: 88751 bytes --]
diff --unified --new-file --recursive mtd/fs/Kconfig mtd-dec07/fs/Kconfig
--- mtd/fs/Kconfig 2004-11-04 00:00:11.000000000 +0100
+++ mtd-dec07/fs/Kconfig 2004-12-07 16:03:02.000000000 +0100
@@ -77,6 +77,19 @@
ECC for JFFS2. This type of flash chip is not common, however it is
available from STMicro.
+config JFFS2_SUMMARY
+ bool "JFFS2 summary support (EXPERIMENTAL)"
+ depends on JFFS2_FS
+ default n
+ help
+ This feature makes it possible to use summary information
+ for faster filesystem mount - specially on NAND.
+
+ The summary information can be inserted into a filesystem image
+ by the utility 'sumtool'.
+
+ If unsure, say 'N'.
+
config JFFS2_COMPRESSION_OPTIONS
bool "Advanced compression options for JFFS2"
default n
diff --unified --new-file --recursive mtd/fs/jffs2/Makefile.common mtd-dec07/fs/jffs2/Makefile.common
--- mtd/fs/jffs2/Makefile.common 2004-11-04 00:00:11.000000000 +0100
+++ mtd-dec07/fs/jffs2/Makefile.common 2004-12-07 16:03:02.000000000 +0100
@@ -16,3 +16,4 @@
jffs2-$(CONFIG_JFFS2_RUBIN) += compr_rubin.o
jffs2-$(CONFIG_JFFS2_RTIME) += compr_rtime.o
jffs2-$(CONFIG_JFFS2_ZLIB) += compr_zlib.o
+jffs2-$(CONFIG_JFFS2_SUMMARY) += summary.o
diff --unified --new-file --recursive mtd/fs/jffs2/build.c mtd-dec07/fs/jffs2/build.c
--- mtd/fs/jffs2/build.c 2004-11-28 00:00:13.000000000 +0100
+++ mtd-dec07/fs/jffs2/build.c 2004-12-07 16:08:31.000000000 +0100
@@ -331,6 +331,10 @@
c->blocks[i].first_node = NULL;
c->blocks[i].last_node = NULL;
c->blocks[i].bad_count = 0;
+ #ifdef CONFIG_JFFS2_SUMMARY
+ c->blocks[i].sum_collected = NULL;
+ #endif
+
}
init_MUTEX(&c->alloc_sem);
diff --unified --new-file --recursive mtd/fs/jffs2/dir.c mtd-dec07/fs/jffs2/dir.c
--- mtd/fs/jffs2/dir.c 2004-11-17 00:00:14.000000000 +0100
+++ mtd-dec07/fs/jffs2/dir.c 2004-12-07 16:03:02.000000000 +0100
@@ -22,6 +22,7 @@
#include <linux/time.h>
#include "nodelist.h"
+
/* Urgh. Please tell me there's a nicer way of doing these. */
#include <linux/version.h>
#if LINUX_VERSION_CODE < KERNEL_VERSION(2,5,48)
@@ -314,7 +315,7 @@
* Just the node will do for now, though
*/
namelen = dentry->d_name.len;
- ret = jffs2_reserve_space(c, sizeof(*ri) + strlen(target), &phys_ofs, &alloclen, ALLOC_NORMAL);
+ ret = jffs2_reserve_space(c, sizeof(*ri) + strlen(target), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
if (ret) {
jffs2_free_raw_inode(ri);
@@ -360,7 +361,7 @@
up(&f->sem);
jffs2_complete_reservation(c);
- ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
+ ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
if (ret) {
/* Eep. */
jffs2_clear_inode(inode);
@@ -445,7 +446,7 @@
* Just the node will do for now, though
*/
namelen = dentry->d_name.len;
- ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL);
+ ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
if (ret) {
jffs2_free_raw_inode(ri);
@@ -488,7 +489,7 @@
up(&f->sem);
jffs2_complete_reservation(c);
- ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
+ ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
if (ret) {
/* Eep. */
jffs2_clear_inode(inode);
@@ -597,7 +598,7 @@
* Just the node will do for now, though
*/
namelen = dentry->d_name.len;
- ret = jffs2_reserve_space(c, sizeof(*ri) + devlen, &phys_ofs, &alloclen, ALLOC_NORMAL);
+ ret = jffs2_reserve_space(c, sizeof(*ri) + devlen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
if (ret) {
jffs2_free_raw_inode(ri);
@@ -642,7 +643,7 @@
up(&f->sem);
jffs2_complete_reservation(c);
- ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
+ ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
if (ret) {
/* Eep. */
jffs2_clear_inode(inode);
@@ -796,4 +797,3 @@
return 0;
}
-
diff --unified --new-file --recursive mtd/fs/jffs2/file.c mtd-dec07/fs/jffs2/file.c
--- mtd/fs/jffs2/file.c 2004-11-17 00:00:14.000000000 +0100
+++ mtd-dec07/fs/jffs2/file.c 2004-12-07 16:03:02.000000000 +0100
@@ -22,6 +22,7 @@
#include <linux/jffs2.h>
#include "nodelist.h"
+
extern int generic_file_open(struct inode *, struct file *) __attribute__((weak));
extern loff_t generic_file_llseek(struct file *file, loff_t offset, int origin) __attribute__((weak));
@@ -135,7 +136,7 @@
D1(printk(KERN_DEBUG "Writing new hole frag 0x%x-0x%x between current EOF and new page\n",
(unsigned int)inode->i_size, pageofs));
- ret = jffs2_reserve_space(c, sizeof(ri), &phys_ofs, &alloc_len, ALLOC_NORMAL);
+ ret = jffs2_reserve_space(c, sizeof(ri), &phys_ofs, &alloc_len, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
if (ret)
return ret;
diff --unified --new-file --recursive mtd/fs/jffs2/fs.c mtd-dec07/fs/jffs2/fs.c
--- mtd/fs/jffs2/fs.c 2004-11-29 00:00:09.000000000 +0100
+++ mtd-dec07/fs/jffs2/fs.c 2004-12-07 16:03:02.000000000 +0100
@@ -74,7 +74,7 @@
return -ENOMEM;
}
- ret = jffs2_reserve_space(c, sizeof(*ri) + mdatalen, &phys_ofs, &alloclen, ALLOC_NORMAL);
+ ret = jffs2_reserve_space(c, sizeof(*ri) + mdatalen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
if (ret) {
jffs2_free_raw_inode(ri);
if (S_ISLNK(inode->i_mode & S_IFMT))
diff --unified --new-file --recursive mtd/fs/jffs2/gc.c mtd-dec07/fs/jffs2/gc.c
--- mtd/fs/jffs2/gc.c 2004-11-17 00:00:14.000000000 +0100
+++ mtd-dec07/fs/jffs2/gc.c 2004-12-07 16:03:02.000000000 +0100
@@ -506,7 +506,7 @@
don't want to force wastage of the end of a block if splitting would
work. */
ret = jffs2_reserve_space_gc(c, min_t(uint32_t, sizeof(struct jffs2_raw_inode) + JFFS2_MIN_DATA_LEN,
- rawlen), &phys_ofs, &alloclen);
+ rawlen), &phys_ofs, &alloclen, rawlen); /* this is not optimal yet */
if (ret)
return ret;
@@ -614,7 +614,7 @@
ACCT_SANITY_CHECK(c,jeb);
D1(ACCT_PARANOIA_CHECK(jeb));
- ret = jffs2_reserve_space_gc(c, rawlen, &phys_ofs, &dummy);
+ ret = jffs2_reserve_space_gc(c, rawlen, &phys_ofs, &dummy, rawlen); /* this is not optimal yet */
if (!ret) {
D1(printk(KERN_DEBUG "Allocated space at 0x%08x to retry failed write.\n", phys_ofs));
@@ -692,7 +692,7 @@
}
- ret = jffs2_reserve_space_gc(c, sizeof(ri) + mdatalen, &phys_ofs, &alloclen);
+ ret = jffs2_reserve_space_gc(c, sizeof(ri) + mdatalen, &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE);
if (ret) {
printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_metadata failed: %d\n",
sizeof(ri)+ mdatalen, ret);
@@ -759,7 +759,7 @@
rd.node_crc = cpu_to_je32(crc32(0, &rd, sizeof(rd)-8));
rd.name_crc = cpu_to_je32(crc32(0, fd->name, rd.nsize));
- ret = jffs2_reserve_space_gc(c, sizeof(rd)+rd.nsize, &phys_ofs, &alloclen);
+ ret = jffs2_reserve_space_gc(c, sizeof(rd)+rd.nsize, &phys_ofs, &alloclen, JFFS2_SUMMARY_DIRENT_SIZE(rd.nsize));
if (ret) {
printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_dirent failed: %d\n",
sizeof(rd)+rd.nsize, ret);
@@ -961,7 +961,7 @@
ri.data_crc = cpu_to_je32(0);
ri.node_crc = cpu_to_je32(crc32(0, &ri, sizeof(ri)-8));
- ret = jffs2_reserve_space_gc(c, sizeof(ri), &phys_ofs, &alloclen);
+ ret = jffs2_reserve_space_gc(c, sizeof(ri), &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE);
if (ret) {
printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_hole failed: %d\n",
sizeof(ri), ret);
@@ -1186,7 +1186,7 @@
uint32_t cdatalen;
uint16_t comprtype = JFFS2_COMPR_NONE;
- ret = jffs2_reserve_space_gc(c, sizeof(ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen);
+ ret = jffs2_reserve_space_gc(c, sizeof(ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE);
if (ret) {
printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_dnode failed: %d\n",
@@ -1243,4 +1243,3 @@
jffs2_gc_release_page(c, pg_ptr, &pg);
return ret;
}
-
diff --unified --new-file --recursive mtd/fs/jffs2/nodelist.h mtd-dec07/fs/jffs2/nodelist.h
--- mtd/fs/jffs2/nodelist.h 2004-11-26 15:08:31.000000000 +0100
+++ mtd-dec07/fs/jffs2/nodelist.h 2004-12-07 16:03:02.000000000 +0100
@@ -20,6 +20,7 @@
#include <linux/jffs2.h>
#include <linux/jffs2_fs_sb.h>
#include <linux/jffs2_fs_i.h>
+#include "summary.h"
#ifdef __ECOS
#include "os-ecos.h"
@@ -194,6 +195,10 @@
int bad_count;
uint32_t offset; /* of this block in the MTD */
+#ifdef CONFIG_JFFS2_SUMMARY
+ struct jffs2_sum_info *sum_collected;
+#endif
+
uint32_t unchecked_size;
uint32_t used_size;
uint32_t dirty_size;
@@ -392,8 +397,8 @@
/* nodemgmt.c */
int jffs2_thread_should_wake(struct jffs2_sb_info *c);
-int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio);
-int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len);
+int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio, uint32_t sumsize);
+int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize);
int jffs2_add_physical_node_ref(struct jffs2_sb_info *c, struct jffs2_raw_node_ref *new);
void jffs2_complete_reservation(struct jffs2_sb_info *c);
void jffs2_mark_node_obsolete(struct jffs2_sb_info *c, struct jffs2_raw_node_ref *raw);
@@ -455,6 +460,10 @@
/* scan.c */
int jffs2_scan_medium(struct jffs2_sb_info *c);
void jffs2_rotate_lists(struct jffs2_sb_info *c);
+int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf,
+ uint32_t ofs, uint32_t len);
+struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino);
+
/* build.c */
int jffs2_do_mount_fs(struct jffs2_sb_info *c);
diff --unified --new-file --recursive mtd/fs/jffs2/nodemgmt.c mtd-dec07/fs/jffs2/nodemgmt.c
--- mtd/fs/jffs2/nodemgmt.c 2004-11-26 15:08:31.000000000 +0100
+++ mtd-dec07/fs/jffs2/nodemgmt.c 2004-12-07 16:03:02.000000000 +0100
@@ -38,9 +38,9 @@
* for the requested allocation.
*/
-static int jffs2_do_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len);
+static int jffs2_do_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize);
-int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio)
+int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio, uint32_t sumsize)
{
int ret = -EAGAIN;
int blocksneeded = c->resv_blocks_write;
@@ -129,7 +129,7 @@
spin_lock(&c->erase_completion_lock);
}
- ret = jffs2_do_reserve_space(c, minsize, ofs, len);
+ ret = jffs2_do_reserve_space(c, minsize, ofs, len, sumsize);
if (ret) {
D1(printk(KERN_DEBUG "jffs2_reserve_space: ret is %d\n", ret));
}
@@ -140,7 +140,7 @@
return ret;
}
-int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len)
+int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize)
{
int ret = -EAGAIN;
minsize = PAD(minsize);
@@ -149,7 +149,7 @@
spin_lock(&c->erase_completion_lock);
while(ret == -EAGAIN) {
- ret = jffs2_do_reserve_space(c, minsize, ofs, len);
+ ret = jffs2_do_reserve_space(c, minsize, ofs, len, sumsize);
if (ret) {
D1(printk(KERN_DEBUG "jffs2_reserve_space_gc: looping, ret is %d\n", ret));
}
@@ -159,50 +159,112 @@
}
/* Called with alloc sem _and_ erase_completion_lock */
-static int jffs2_do_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len)
+static int jffs2_do_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize)
{
struct jffs2_eraseblock *jeb = c->nextblock;
+ uint32_t nofree_size;
restart:
- if (jeb && minsize > jeb->free_size) {
- /* Skip the end of this block and file it as having some dirty space */
- /* If there's a pending write to it, flush now */
- if (jffs2_wbuf_dirty(c)) {
- spin_unlock(&c->erase_completion_lock);
- D1(printk(KERN_DEBUG "jffs2_do_reserve_space: Flushing write buffer\n"));
- jffs2_flush_wbuf_pad(c);
- spin_lock(&c->erase_completion_lock);
- jeb = c->nextblock;
- goto restart;
- }
- c->wasted_size += jeb->free_size;
- c->free_size -= jeb->free_size;
- jeb->wasted_size += jeb->free_size;
- jeb->free_size = 0;
+ nofree_size = 0;
+
+#ifdef CONFIG_JFFS2_SUMMARY
+
+ if (sumsize != JFFS2_SUMMARY_NOSUM_SIZE) {
+ int ret;
+ if (jeb) {
+ if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret;
+ nofree_size = sumsize + jeb->sum_collected->sum_size + JFFS2_SUMMARY_FRAME_SIZE;
+ }
- /* Check, if we have a dirty block now, or if it was dirty already */
- if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) {
- c->dirty_size += jeb->wasted_size;
- c->wasted_size -= jeb->wasted_size;
- jeb->dirty_size += jeb->wasted_size;
- jeb->wasted_size = 0;
- if (VERYDIRTY(c, jeb->dirty_size)) {
- D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
+ D1(printk(KERN_DEBUG "JFFS2: minsize %d , jeb->free(%d) , sum_collected->size(%d) , sumsize(%d)\n",minsize,jeb->free_size,jeb->sum_collected->sum_size,sumsize));
+
+ if (jeb && (minsize + jeb->sum_collected->sum_size + sumsize + JFFS2_SUMMARY_FRAME_SIZE > jeb->free_size)) {
+ D1(printk(KERN_DEBUG "JFFS2: generating summary for 0x%08x.\n", jeb->offset));
+ if (jeb->sum_collected->sum_size == JFFS2_SUMMARY_NOSUM_SIZE) {
+ sumsize = JFFS2_SUMMARY_NOSUM_SIZE;
+ jffs2_sum_clean_collected(jeb);
+ goto restart;
+ }
+
+ ret = jffs2_sum_write_sumnode(c);
+
+ if (ret)
+ return ret;
+
+ if (jeb->sum_collected->sum_size == JFFS2_SUMMARY_NOSUM_SIZE) { //jffs2_write_sumnode can't write out the summary information
+ sumsize = JFFS2_SUMMARY_NOSUM_SIZE;
+ jffs2_sum_clean_collected(jeb);
+ goto restart;
+ }
+
+ /* Check, if we have a dirty block now, or if it was dirty already */
+ if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) {
+ c->dirty_size += jeb->wasted_size;
+ c->wasted_size -= jeb->wasted_size;
+ jeb->dirty_size += jeb->wasted_size;
+ jeb->wasted_size = 0;
+ if (VERYDIRTY(c, jeb->dirty_size)) {
+ D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
+ jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
+ list_add_tail(&jeb->list, &c->very_dirty_list);
+ } else {
+ D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
+ jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
+ list_add_tail(&jeb->list, &c->dirty_list);
+ }
+ } else {
+ D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
- list_add_tail(&jeb->list, &c->very_dirty_list);
- } else {
- D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
+ list_add_tail(&jeb->list, &c->clean_list);
+ }
+ c->nextblock = jeb = NULL;
+ }
+ }
+ else {
+#endif
+ if (jeb && minsize > jeb->free_size) {
+ /* Skip the end of this block and file it as having some dirty space */
+ /* If there's a pending write to it, flush now */
+
+ if (jffs2_wbuf_dirty(c)) {
+ spin_unlock(&c->erase_completion_lock);
+ D1(printk(KERN_DEBUG "jffs2_do_reserve_space: Flushing write buffer\n"));
+ jffs2_flush_wbuf_pad(c);
+ spin_lock(&c->erase_completion_lock);
+ jeb = c->nextblock;
+ goto restart;
+ }
+
+ c->wasted_size += jeb->free_size;
+ c->free_size -= jeb->free_size;
+ jeb->wasted_size += jeb->free_size;
+ jeb->free_size = 0;
+
+ /* Check, if we have a dirty block now, or if it was dirty already */
+ if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) {
+ c->dirty_size += jeb->wasted_size;
+ c->wasted_size -= jeb->wasted_size;
+ jeb->dirty_size += jeb->wasted_size;
+ jeb->wasted_size = 0;
+ if (VERYDIRTY(c, jeb->dirty_size)) {
+ D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
+ jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
+ list_add_tail(&jeb->list, &c->very_dirty_list);
+ } else {
+ D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
+ jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
+ list_add_tail(&jeb->list, &c->dirty_list);
+ }
+ } else {
+ D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
- list_add_tail(&jeb->list, &c->dirty_list);
+ list_add_tail(&jeb->list, &c->clean_list);
}
- } else {
- D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
- jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
- list_add_tail(&jeb->list, &c->clean_list);
+ c->nextblock = jeb = NULL;
}
- c->nextblock = jeb = NULL;
+#ifdef CONFIG_JFFS2_SUMMARY
}
-
+#endif
if (!jeb) {
struct list_head *next;
/* Take the next block off the 'free' list */
@@ -266,7 +328,7 @@
/* OK, jeb (==c->nextblock) is now pointing at a block which definitely has
enough space */
*ofs = jeb->offset + (c->sector_size - jeb->free_size);
- *len = jeb->free_size;
+ *len = jeb->free_size - nofree_size;
if (c->cleanmarker_size && jeb->used_size == c->cleanmarker_size &&
!jeb->first_node->next_in_ino) {
diff --unified --new-file --recursive mtd/fs/jffs2/os-linux.h mtd-dec07/fs/jffs2/os-linux.h
--- mtd/fs/jffs2/os-linux.h 2004-11-17 00:00:14.000000000 +0100
+++ mtd-dec07/fs/jffs2/os-linux.h 2004-12-07 16:03:02.000000000 +0100
@@ -100,7 +100,13 @@
#define jffs2_is_readonly(c) (OFNI_BS_2SFFJ(c)->s_flags & MS_RDONLY)
#if (!defined CONFIG_JFFS2_FS_NAND && !defined CONFIG_JFFS2_FS_NOR_ECC)
+
+#ifndef CONFIG_JFFS2_SUMMARY
#define jffs2_can_mark_obsolete(c) (1)
+#else
+#define jffs2_can_mark_obsolete(c) (0)
+#endif
+
#define jffs2_cleanmarker_oob(c) (0)
#define jffs2_write_nand_cleanmarker(c,jeb) (-EIO)
@@ -121,7 +127,12 @@
#else /* NAND and/or ECC'd NOR support present */
+#ifndef CONFIG_JFFS2_SUMMARY
#define jffs2_can_mark_obsolete(c) ((c->mtd->type == MTD_NORFLASH && !(c->mtd->flags & MTD_ECC)) || c->mtd->type == MTD_RAM)
+#else
+#define jffs2_can_mark_obsolete(c) (0)
+#endif
+
#define jffs2_cleanmarker_oob(c) (c->mtd->type == MTD_NANDFLASH)
#define jffs2_flash_write_oob(c, ofs, len, retlen, buf) ((c)->mtd->write_oob((c)->mtd, ofs, len, retlen, buf))
@@ -218,5 +229,3 @@
#endif /* __JFFS2_OS_LINUX_H__ */
-
-
diff --unified --new-file --recursive mtd/fs/jffs2/scan.c mtd-dec07/fs/jffs2/scan.c
--- mtd/fs/jffs2/scan.c 2004-11-26 15:08:31.000000000 +0100
+++ mtd-dec07/fs/jffs2/scan.c 2004-12-07 16:03:02.000000000 +0100
@@ -18,22 +18,10 @@
#include <linux/crc32.h>
#include <linux/compiler.h>
#include "nodelist.h"
+#include "summary.h"
#define EMPTY_SCAN_SIZE 1024
-#define DIRTY_SPACE(x) do { typeof(x) _x = (x); \
- c->free_size -= _x; c->dirty_size += _x; \
- jeb->free_size -= _x ; jeb->dirty_size += _x; \
- }while(0)
-#define USED_SPACE(x) do { typeof(x) _x = (x); \
- c->free_size -= _x; c->used_size += _x; \
- jeb->free_size -= _x ; jeb->used_size += _x; \
- }while(0)
-#define UNCHECKED_SPACE(x) do { typeof(x) _x = (x); \
- c->free_size -= _x; c->unchecked_size += _x; \
- jeb->free_size -= _x ; jeb->unchecked_size += _x; \
- }while(0)
-
#define noisy_printk(noise, args...) do { \
if (*(noise)) { \
printk(KERN_NOTICE args); \
@@ -58,13 +46,6 @@
static int jffs2_scan_dirent_node(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb,
struct jffs2_raw_dirent *rd, uint32_t ofs);
-#define BLK_STATE_ALLFF 0
-#define BLK_STATE_CLEAN 1
-#define BLK_STATE_PARTDIRTY 2
-#define BLK_STATE_CLEANMARKER 3
-#define BLK_STATE_ALLDIRTY 4
-#define BLK_STATE_BADBLOCK 5
-
static inline int min_free(struct jffs2_sb_info *c)
{
uint32_t min = 2 * sizeof(struct jffs2_raw_inode);
@@ -257,7 +238,7 @@
return ret;
}
-static int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf,
+int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf,
uint32_t ofs, uint32_t len)
{
int ret;
@@ -286,6 +267,11 @@
uint32_t hdr_crc, buf_ofs, buf_len;
int err;
int noise = 0;
+
+#ifdef CONFIG_JFFS2_SUMMARY
+ struct jffs2_sum_marker *sm;
+#endif
+
#ifdef CONFIG_JFFS2_FS_NAND
int cleanmarkerfound = 0;
#endif
@@ -311,10 +297,54 @@
}
}
#endif
+
+#ifdef CONFIG_JFFS2_SUMMARY
+ sm = (struct jffs2_sum_marker *)kmalloc(sizeof(struct jffs2_sum_marker), GFP_KERNEL);
+ if (!sm) {
+ return -ENOMEM;
+ }
+
+ err = jffs2_fill_scan_buf(c, (unsigned char *) sm, jeb->offset + c->sector_size - sizeof(struct jffs2_sum_marker), sizeof(struct jffs2_sum_marker));
+
+ if (err) {
+ kfree(sm);
+ return err;
+ }
+
+ if (je32_to_cpu(sm->magic) == JFFS2_SUM_MAGIC ) {
+
+ if(je32_to_cpu(sm->erase_size) == c->sector_size) {
+ int ret = jffs2_sum_scan_sumnode(c,jeb,je32_to_cpu(sm->offset),&pseudo_random);
+
+ if (ret) {
+ kfree(sm);
+ return ret;
+ }
+ }
+
+ printk(KERN_WARNING "FS erase_block_size != JFFS2 erase_block_size => skipping summary information\n");
+
+ }
+
+ kfree(sm);
+
+ ofs = jeb->offset;
+ prevofs = jeb->offset - 1;
+
+#endif
+
buf_ofs = jeb->offset;
if (!buf_size) {
buf_len = c->sector_size;
+
+#ifdef CONFIG_JFFS2_SUMMARY
+ /* must reread because of summary test */
+ err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len);
+ if (err)
+ return err;
+#endif
+
} else {
buf_len = EMPTY_SCAN_SIZE;
err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len);
@@ -356,6 +386,8 @@
noise = 10;
+ D1(printk(KERN_DEBUG "JFFS2: no summary found in jeb 0x%08x. Apply original scan.\n",jeb->offset));
+
scan_more:
while(ofs < jeb->offset + c->sector_size) {
@@ -571,6 +603,9 @@
break;
case JFFS2_NODETYPE_PADDING:
+#ifdef CONFIG_JFFS2_SUMMARY
+ jffs2_sum_add_padding_mem(jeb,je32_to_cpu(node->totlen));
+#endif
DIRTY_SPACE(PAD(je32_to_cpu(node->totlen)));
ofs += PAD(je32_to_cpu(node->totlen));
break;
@@ -634,7 +669,7 @@
return BLK_STATE_ALLDIRTY;
}
-static struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino)
+struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino)
{
struct jffs2_inode_cache *ic;
@@ -728,6 +763,11 @@
pseudo_random += je32_to_cpu(ri->version);
UNCHECKED_SPACE(PAD(je32_to_cpu(ri->totlen)));
+
+#ifdef CONFIG_JFFS2_SUMMARY
+ jffs2_sum_add_inode_mem(jeb,ri,ofs);
+#endif
+
return 0;
}
@@ -806,6 +846,10 @@
USED_SPACE(PAD(je32_to_cpu(rd->totlen)));
jffs2_add_fd_to_list(c, fd, &ic->scan_dents);
+#ifdef CONFIG_JFFS2_SUMMARY
+ jffs2_sum_add_dirent_mem(jeb,rd,ofs);
+#endif
+
return 0;
}
diff --unified --new-file --recursive mtd/fs/jffs2/summary.c mtd-dec07/fs/jffs2/summary.c
--- mtd/fs/jffs2/summary.c 1970-01-01 01:00:00.000000000 +0100
+++ mtd-dec07/fs/jffs2/summary.c 2004-12-07 16:03:02.000000000 +0100
@@ -0,0 +1,720 @@
+/*
+ * JFFS2 -- Journalling Flash File System, Version 2.
+ *
+ * Copyright (C) 2004 Ferenc Havasi <havasi@inf.u-szeged.hu>,
+ * Zoltan Sogor <weth@inf.u-szeged.hu>,
+ * Patrik Kluba <pajko@halom.u-szeged.hu>,
+ * University of Szeged, Hungary
+ *
+ * For licensing information, see the file 'LICENCE' in this directory.
+ *
+ * $Id$
+ *
+ */
+
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/mtd/mtd.h>
+#include <linux/pagemap.h>
+#include <linux/crc32.h>
+#include <linux/compiler.h>
+#include <linux/vmalloc.h>
+#include "nodelist.h"
+
+int jffs2_sum_init(struct jffs2_sb_info *c)
+{
+ c->summary_buf = (jint32_t *) vmalloc(c->sector_size);
+ if (!c->summary_buf) {
+ printk(KERN_WARNING "JFFS2: can't allocate memory to dump summary information!\n");
+ return 1;
+ }
+ return 0;
+}
+
+void jffs2_sum_exit(struct jffs2_sb_info *c)
+{
+ if (c->summary_buf) {
+ vfree(c->summary_buf);
+ c->summary_buf = NULL;
+ }
+}
+
+int jffs2_sum_care_sum_collected(struct jffs2_eraseblock *jeb)
+{
+ if (!jeb->sum_collected) {
+ jeb->sum_collected = (struct jffs2_sum_info *) kmalloc(sizeof(struct jffs2_sum_info), GFP_KERNEL);
+
+ if (!jeb->sum_collected)
+ return -ENOMEM;
+
+ jeb->sum_collected->sum_list = NULL;
+ jeb->sum_collected->sum_num = 0;
+ jeb->sum_collected->sum_size = 0;
+ jeb->sum_collected->sum_padded = 0;
+ }
+ return 0;
+}
+
+static int jffs2_sum_add_mem(struct jffs2_eraseblock *jeb, union jffs2_sum_mem *item)
+{
+
+ union jffs2_sum_mem *walk;
+ int ret;
+
+ if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret;
+
+ if (!jeb->sum_collected->sum_list) {
+ jeb->sum_collected->sum_list = (union jffs2_sum_mem *) item;
+ }
+ else {
+ walk = jeb->sum_collected->sum_list;
+
+ while (walk->u.next) {
+ walk = walk->u.next;
+ }
+ walk->u.next = (union jffs2_sum_mem *) item;
+ }
+ switch (je16_to_cpu(item->u.nodetype)) {
+ case JFFS2_NODETYPE_INODE:
+ jeb->sum_collected->sum_size += JFFS2_SUMMARY_INODE_SIZE;
+ jeb->sum_collected->sum_num++;
+ break;
+ case JFFS2_NODETYPE_DIRENT:
+ jeb->sum_collected->sum_size += JFFS2_SUMMARY_DIRENT_SIZE(item->d.nsize);
+ jeb->sum_collected->sum_num++;
+ break;
+ default:
+ printk(KERN_WARNING "__jffs2_add_sum_mem(): UNKNOWN node type %d\n", je16_to_cpu(item->u.nodetype));
+ return 1;
+ }
+ return 0;
+}
+
+void jffs2_sum_clean_all_info(struct jffs2_sb_info *c)
+{
+ int i;
+
+ for (i=0; i<c->nr_blocks; i++) {
+ struct jffs2_eraseblock *jeb = &c->blocks[i];
+
+ jffs2_sum_clean_collected(jeb);
+ kfree(jeb->sum_collected);
+ jeb->sum_collected = NULL;
+ }
+}
+
+/* These 3 functions are called from scan.c to collect summary info for not closed jeb */
+
+int jffs2_sum_add_padding_mem(struct jffs2_eraseblock *jeb, uint32_t size)
+{
+ int ret;
+
+ if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret;
+ jeb->sum_collected->sum_padded += size;
+ return 0;
+}
+
+int jffs2_sum_add_inode_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_inode *ri, uint32_t ofs)
+{
+
+ struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *) kmalloc(sizeof(struct jffs2_sum_inode_mem), GFP_KERNEL);
+
+ if (!temp)
+ return -ENOMEM;
+
+ ofs -= jeb->offset;
+
+ temp->nodetype = ri->nodetype;
+ temp->inode = ri->ino;
+ temp->version = ri->version;
+ temp->offset = cpu_to_je32(ofs);
+ temp->totlen = ri->totlen;
+ temp->next = NULL;
+
+ return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp);
+}
+
+int jffs2_sum_add_dirent_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_dirent *rd, uint32_t ofs)
+{
+
+ struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *)
+ kmalloc(sizeof(struct jffs2_sum_dirent_mem) + rd->nsize, GFP_KERNEL);
+
+ if (!temp)
+ return -ENOMEM;
+
+ ofs -= jeb->offset;
+
+ temp->nodetype = rd->nodetype;
+ temp->totlen = rd->totlen;
+ temp->offset = cpu_to_je32(ofs);
+ temp->pino = rd->pino;
+ temp->version = rd->version;
+ temp->ino = rd->ino;
+ temp->nsize = rd->nsize;
+ temp->type = rd->type;
+ temp->next = NULL;
+
+ memcpy(temp->name, rd->name, rd->nsize);
+
+ return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp);
+}
+
+/* Cleanup every collected summary information */
+
+void jffs2_sum_clean_collected(struct jffs2_eraseblock *jeb)
+{
+
+ union jffs2_sum_mem *temp;
+
+ if(jeb && jeb->sum_collected){
+
+ while(jeb->sum_collected->sum_list){
+ temp = jeb->sum_collected->sum_list;
+ jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next;
+ kfree(temp);
+ jeb->sum_collected->sum_num--;
+ }
+
+ if(jeb->sum_collected->sum_num != 0){
+ printk(KERN_WARNING "Ooops, something wrong happened! sum_num != 0, but sum_list = null ???");
+ jeb->sum_collected->sum_num = 0;
+ }
+ }
+}
+
+/* Called from wbuf.c to collect writed node info */
+
+int jffs2_sum_add_kvec(struct jffs2_sb_info *c, const struct kvec *invecs, unsigned long count, uint32_t ofs)
+{
+ union jffs2_node_union *node;
+ struct jffs2_eraseblock *jeb;
+ int ret;
+
+ node = (union jffs2_node_union *) invecs[0].iov_base;
+ jeb = &c->blocks[ofs / c->sector_size];
+ ofs -= jeb->offset;
+
+ if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret;
+
+ switch(je16_to_cpu(node->u.nodetype)){
+ case JFFS2_NODETYPE_INODE : {
+ struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *)
+ kmalloc(sizeof(struct jffs2_sum_inode_mem), GFP_KERNEL);
+
+ if (!temp)
+ return -ENOMEM;
+
+ temp->nodetype = node->i.nodetype;
+ temp->inode = node->i.ino;
+ temp->version = node->i.version;
+ temp->offset = cpu_to_je32(ofs);
+ temp->totlen = node->i.totlen;
+ temp->next = NULL;
+
+ return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp);
+
+ break;
+ }
+
+ case JFFS2_NODETYPE_DIRENT : {
+ struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *)
+ kmalloc(sizeof(struct jffs2_sum_dirent_mem) + node->d.nsize, GFP_KERNEL);
+
+ if (!temp)
+ return -ENOMEM;
+
+ temp->nodetype = node->d.nodetype;
+ temp->totlen = node->d.totlen;
+ temp->offset = cpu_to_je32(ofs);
+ temp->pino = node->d.pino;
+ temp->version = node->d.version;
+ temp->ino = node->d.ino;
+ temp->nsize = node->d.nsize;
+ temp->type = node->d.type;
+ temp->next = NULL;
+
+ memcpy(temp->name,invecs[1].iov_base,node->d.nsize);
+
+ return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp);
+
+ break;
+ }
+
+ case JFFS2_NODETYPE_PADDING : {
+ D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node PADDING\n"));
+ jeb->sum_collected->sum_padded += je32_to_cpu(node->u.totlen);
+ break;
+ }
+
+ case JFFS2_NODETYPE_CLEANMARKER : {
+ D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node CLEANMARKER\n"));
+ break;
+ }
+
+ case JFFS2_NODETYPE_SUMMARY : {
+ D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node SUMMARY\n"));
+ break;
+ }
+
+ default : {
+ printk(KERN_WARNING "jffs2_sum_add_kvec(): Node not supported\n");
+ BUG();
+ break;
+ }
+ }
+
+ return 0;
+}
+
+/* Process the summary information - called from jffs2_scan_eraseblock() */
+
+int jffs2_sum_scan_sumnode(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb, uint32_t ofs, uint32_t *pseudo_random)
+{
+
+ struct jffs2_unknown_node crcnode;
+ struct jffs2_raw_node_ref *raw;
+ struct jffs2_raw_node_ref *cache_ref;
+ struct jffs2_inode_cache *ic;
+ struct jffs2_full_dirent *fd;
+
+ int i, err;
+ int bad_sum = 0;
+ int sumsize;
+ uint32_t ino;
+ uint32_t crc;
+ struct jffs2_summary_node *summary;
+
+ sumsize = c->sector_size - ofs;
+ ofs += jeb->offset;
+
+ D1(printk(KERN_DEBUG "JFFS2: summary found for 0x%08x at 0x%08x (0x%x bytes)\n", jeb->offset, ofs, sumsize));
+
+ summary = (struct jffs2_summary_node *) kmalloc(sumsize, GFP_KERNEL);
+
+ if (!summary) {
+ return -ENOMEM;
+ }
+
+ err = jffs2_fill_scan_buf(c, (unsigned char *)summary, ofs, sumsize);
+
+ if (err) {
+ kfree(summary);
+ return err;
+ }
+
+ /* OK, now check for node validity and CRC */
+ crcnode.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
+ crcnode.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY);
+ crcnode.totlen = summary->totlen;
+ crc = crc32(0, &crcnode, sizeof(crcnode)-4);
+
+ if (je32_to_cpu(summary->hdr_crc) != crc) {
+ D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node header is corrupt (bad CRC or no summary at all)\n"));
+ bad_sum = 1;
+ }
+
+ if ((!bad_sum) && (je32_to_cpu(summary->totlen) != sumsize)) {
+ D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (wrong erasesize?)\n"));
+ bad_sum = 1;
+ }
+
+ crc = crc32(0, summary, sizeof(struct jffs2_summary_node)-8);
+
+ if ((!bad_sum) && (je32_to_cpu(summary->node_crc) != crc)) {
+ D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (bad CRC)\n"));
+ bad_sum = 1;
+ }
+
+ crc = crc32(0, summary->sum, sumsize - sizeof(struct jffs2_summary_node));
+
+ if ((!bad_sum) && (je32_to_cpu(summary->sum_crc) != crc)) {
+ D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node data is corrupt (bad CRC)\n"));
+ bad_sum = 1;
+ }
+
+ if (!bad_sum) {
+
+ struct jffs2_sum_unknown_flash *sp;
+ sp = (struct jffs2_sum_unknown_flash *) summary->sum;
+
+ if ( je32_to_cpu(summary->cln_mkr) ) {
+
+ D1(printk(KERN_DEBUG "Summary : CLEANMARKER node \n"));
+
+ if (je32_to_cpu(summary->cln_mkr) != c->cleanmarker_size) {
+ D1(printk(KERN_DEBUG "CLEANMARKER node has totlen 0x%x != normal 0x%x\n",
+ je32_to_cpu(summary->cln_mkr), c->cleanmarker_size));
+ UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) );
+ }
+ else if (jeb->first_node) {
+ D1(printk(KERN_DEBUG "CLEANMARKER node not first node in block (0x%08x)\n", jeb->offset);
+ UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr))));
+ }
+ else {
+ struct jffs2_raw_node_ref *marker_ref = jffs2_alloc_raw_node_ref();
+
+ if (!marker_ref) {
+ D1(printk(KERN_NOTICE "Failed to allocate node ref for clean marker\n"));
+ kfree(summary);
+ return -ENOMEM;
+ }
+
+ marker_ref->next_in_ino = NULL;
+ marker_ref->next_phys = NULL;
+ marker_ref->flash_offset = jeb->offset | REF_NORMAL;
+ marker_ref->__totlen = je32_to_cpu(summary->cln_mkr);
+ jeb->first_node = jeb->last_node = marker_ref;
+
+ USED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) );
+
+ }
+ }
+
+ if ( je32_to_cpu(summary->padded) ) {
+ DIRTY_SPACE(je32_to_cpu(summary->padded));
+ }
+
+ for(i = 0; i < je16_to_cpu(summary->sum_num); i++) {
+
+ D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Processing summary information %d\n", i));
+ uint8_t *temp8ptr = NULL;
+
+ switch (je16_to_cpu(sp->nodetype)) {
+
+ case JFFS2_NODETYPE_INODE : {
+ struct jffs2_sum_inode_flash *spi;
+ spi = (struct jffs2_sum_inode_flash *) sp;
+
+ ino = je32_to_cpu(spi->inode);
+ D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Inode at 0x%08x\n", jeb->offset + je32_to_cpu(spi->offset)));
+ raw = jffs2_alloc_raw_node_ref();
+ if (!raw) {
+ printk(KERN_NOTICE "jffs2_scan_eraseblock(): allocation of node reference failed\n");
+ kfree(summary);
+ return -ENOMEM;
+ }
+
+ ic = jffs2_get_ino_cache(c, ino);
+ if (!ic) {
+ ic = jffs2_scan_make_ino_cache(c, ino);
+ if (!ic) {
+ printk(KERN_NOTICE "jffs2_scan_eraseblock(): scan_make_ino_cache failed\n");
+ jffs2_free_raw_node_ref(raw);
+ kfree(summary);
+ return -ENOMEM;
+ }
+ }
+
+ raw->flash_offset = (jeb->offset + je32_to_cpu(spi->offset)) | REF_UNCHECKED;
+ raw->__totlen = PAD(je32_to_cpu(spi->totlen));
+ raw->next_phys = NULL;
+ raw->next_in_ino = ic->nodes;
+
+ ic->nodes = raw;
+ if (!jeb->first_node)
+ jeb->first_node = raw;
+ if (jeb->last_node)
+ jeb->last_node->next_phys = raw;
+ jeb->last_node = raw;
+
+ *pseudo_random += je32_to_cpu(spi->version);
+ UNCHECKED_SPACE(PAD(je32_to_cpu(spi->totlen)));
+
+ temp8ptr = (uint8_t *) sp;
+ temp8ptr += JFFS2_SUMMARY_INODE_SIZE;
+ sp = (struct jffs2_sum_unknown_flash *) temp8ptr;
+
+ break;
+ }
+
+ case JFFS2_NODETYPE_DIRENT : {
+ struct jffs2_sum_dirent_flash *spd;
+ spd = (struct jffs2_sum_dirent_flash *) sp;
+
+ fd = jffs2_alloc_full_dirent(spd->nsize+1);
+ if (!fd) {
+ kfree(summary);
+ return -ENOMEM;
+ }
+
+ memcpy(&fd->name, spd->name, spd->nsize);
+ fd->name[spd->nsize] = 0;
+
+ raw = jffs2_alloc_raw_node_ref();
+ if (!raw) {
+ jffs2_free_full_dirent(fd);
+ printk(KERN_NOTICE "jffs2_scan_dirent_node(): allocation of node reference failed\n");
+ kfree(summary);
+ return -ENOMEM;
+ }
+
+ ic = jffs2_scan_make_ino_cache(c, je32_to_cpu(spd->pino));
+ if (!ic) {
+ jffs2_free_full_dirent(fd);
+ jffs2_free_raw_node_ref(raw);
+ kfree(summary);
+ return -ENOMEM;
+ }
+
+ raw->__totlen = PAD(je32_to_cpu(spd->totlen));
+ raw->flash_offset = (jeb->offset + je32_to_cpu(spd->offset)) | REF_PRISTINE;
+ raw->next_phys = NULL;
+ raw->next_in_ino = ic->nodes;
+ ic->nodes = raw;
+ if (!jeb->first_node)
+ jeb->first_node = raw;
+ if (jeb->last_node)
+ jeb->last_node->next_phys = raw;
+ jeb->last_node = raw;
+
+ fd->raw = raw;
+ fd->next = NULL;
+ fd->version = je32_to_cpu(spd->version);
+ fd->ino = je32_to_cpu(spd->ino);
+ fd->nhash = full_name_hash(fd->name, spd->nsize);
+ fd->type = spd->type;
+ USED_SPACE(PAD(je32_to_cpu(spd->totlen)));
+ jffs2_add_fd_to_list(c, fd, &ic->scan_dents);
+
+ *pseudo_random += je32_to_cpu(spd->version);
+
+ temp8ptr = (uint8_t *) sp;
+ temp8ptr += JFFS2_SUMMARY_DIRENT_SIZE(spd->nsize);
+ sp = (struct jffs2_sum_unknown_flash *) temp8ptr;
+
+ break;
+ }
+
+ default : {
+ printk(KERN_WARNING "Kernel doesn't support this type of node !!! Exiting");
+ return -EIO;
+ break;
+ }
+ }
+ }
+
+ kfree(summary);
+
+ /* for ACCT_PARANOIA_CHECK */
+ cache_ref = jffs2_alloc_raw_node_ref();
+
+ if (!cache_ref) {
+ printk(KERN_NOTICE "Failed to allocate node ref for cache\n");
+ return -ENOMEM;
+ }
+
+ cache_ref->next_in_ino = NULL;
+ cache_ref->next_phys = NULL;
+ cache_ref->flash_offset = ofs | REF_NORMAL;
+ cache_ref->__totlen = sumsize;
+
+ if (!jeb->first_node)
+ jeb->first_node = cache_ref;
+ if (jeb->last_node)
+ jeb->last_node->next_phys = cache_ref;
+ jeb->last_node = cache_ref;
+
+ USED_SPACE(sumsize);
+
+ jeb->wasted_size += jeb->free_size;
+ c->wasted_size += jeb->free_size;
+ c->free_size -= jeb->free_size;
+ jeb->free_size = 0;
+
+ /* somebody check this and all of space accounting in summary support */
+
+ if ((jeb->used_size + jeb->unchecked_size) == PAD(c->cleanmarker_size) && !jeb->dirty_size
+ && (!jeb->first_node || !jeb->first_node->next_in_ino) ) {
+ return BLK_STATE_CLEANMARKER;
+ }
+ /* move blocks with max 4 byte dirty space to cleanlist */
+ else if (!ISDIRTY(c->sector_size - (jeb->used_size + jeb->unchecked_size))) {
+ c->dirty_size -= jeb->dirty_size;
+ c->wasted_size += jeb->dirty_size;
+ jeb->wasted_size += jeb->dirty_size;
+ jeb->dirty_size = 0;
+ return BLK_STATE_CLEAN;
+ }
+ else if (jeb->used_size || jeb->unchecked_size) {
+ return BLK_STATE_PARTDIRTY;
+ }
+ else {
+ return BLK_STATE_ALLDIRTY;
+ }
+ }
+
+ return 0;
+}
+
+/* Write out summary information - called from jffs2_do_reserve_space */
+
+int jffs2_sum_write_sumnode(struct jffs2_sb_info *c)
+{
+ struct jffs2_summary_node isum;
+ union jffs2_sum_mem *temp;
+ jint32_t offset;
+ jint32_t *wpage;
+ uint8_t *tempptr;
+ int datasize;
+ int infosize;
+ int padsize;
+ size_t retlen;
+ int ret;
+ struct jffs2_eraseblock *jeb;
+ struct kvec vecs[2];
+ jint32_t magic = cpu_to_je32(JFFS2_SUM_MAGIC);
+
+ D2(printk(KERN_DEBUG "jffs2_sum_write_sumnode()\n"));
+
+ jeb = c->nextblock;
+
+ if (!jeb->sum_collected->sum_num || !jeb->sum_collected->sum_list) {
+ printk(KERN_WARNING "JFFS2: jffs2_sum_write_sumnode(): empty summary info!!!\n");
+ BUG();
+ }
+
+ datasize = jeb->sum_collected->sum_size + sizeof(struct jffs2_sum_marker);
+ infosize = sizeof(struct jffs2_summary_node) + datasize;
+ padsize = jeb->free_size - infosize;
+ infosize += padsize; datasize += padsize;
+ offset = cpu_to_je32(c->sector_size - jeb->free_size);
+
+ if (padsize < 0) { // if jeb hasn't got enought free space for summary
+
+ union jffs2_sum_mem *temp;
+
+ while(jeb->sum_collected->sum_list){ //cleanup sum_list
+ temp = jeb->sum_collected->sum_list;
+ jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next;
+ kfree(temp);
+ jeb->sum_collected->sum_num--;
+ }
+
+ jeb->sum_collected->sum_list = NULL;
+ jeb->sum_collected->sum_num = 0;
+ jeb->sum_collected->sum_size = JFFS2_SUMMARY_NOSUM_SIZE; // don't try to write out summary for this node
+
+ printk(KERN_WARNING "JFFS2: not enough space for summary, padsize = %d\n",padsize);
+ return 0;
+ }
+
+ memset(c->summary_buf, 0xff, datasize);
+ memset(&isum, 0, sizeof(isum));
+
+ isum.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
+ isum.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY);
+ isum.totlen = cpu_to_je32(infosize);
+ isum.hdr_crc = cpu_to_je32(crc32(0, &isum, sizeof(struct jffs2_unknown_node) - 4));
+ isum.padded = cpu_to_je32(jeb->sum_collected->sum_padded);
+
+ if (c->cleanmarker_size) {
+ isum.cln_mkr = cpu_to_je32(c->cleanmarker_size);
+ }
+ else{
+ isum.cln_mkr = cpu_to_je32(0);
+ }
+
+ isum.sum_num = cpu_to_je16(jeb->sum_collected->sum_num);
+ wpage = c->summary_buf;
+
+
+ while (jeb->sum_collected->sum_num) { // write sum_data
+
+
+ switch(je16_to_cpu(jeb->sum_collected->sum_list->u.nodetype)){
+
+ case JFFS2_NODETYPE_INODE : {
+ jint16_t *temp16ptr = (jint16_t *)wpage;
+
+ *(temp16ptr++) = jeb->sum_collected->sum_list->i.nodetype;
+ wpage = (jint32_t *) temp16ptr;
+
+ *(wpage++) = jeb->sum_collected->sum_list->i.inode;
+ *(wpage++) = jeb->sum_collected->sum_list->i.version;
+ *(wpage++) = jeb->sum_collected->sum_list->i.offset;
+ *(wpage++) = jeb->sum_collected->sum_list->i.totlen;
+ break;
+ }
+
+ case JFFS2_NODETYPE_DIRENT : {
+ jint16_t *temp16ptr = (jint16_t *) wpage;
+ uint8_t *temp8ptr = NULL;
+
+ *(temp16ptr++) = jeb->sum_collected->sum_list->d.nodetype;
+ wpage = (jint32_t *) temp16ptr;
+
+ *(wpage++) = jeb->sum_collected->sum_list->d.totlen;
+ *(wpage++) = jeb->sum_collected->sum_list->d.offset;
+ *(wpage++) = jeb->sum_collected->sum_list->d.pino;
+ *(wpage++) = jeb->sum_collected->sum_list->d.version;
+ *(wpage++) = jeb->sum_collected->sum_list->d.ino;
+
+ temp8ptr = (uint8_t *) wpage;
+ *(temp8ptr++) = jeb->sum_collected->sum_list->d.nsize;
+ *(temp8ptr++) = jeb->sum_collected->sum_list->d.type;
+
+ memcpy(temp8ptr,jeb->sum_collected->sum_list->d.name,jeb->sum_collected->sum_list->d.nsize);
+ temp8ptr += jeb->sum_collected->sum_list->d.nsize;
+ wpage = (jint32_t *) temp8ptr;
+
+ break;
+ }
+
+ default : {
+ printk(KERN_WARNING "Unknown node in summary information!!!\n");
+ BUG();
+ }
+ }
+
+ temp = jeb->sum_collected->sum_list;
+ jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next;
+ kfree(temp);
+
+ jeb->sum_collected->sum_num--;
+ }
+
+ jeb->sum_collected->sum_size = 0;
+
+ tempptr = (uint8_t *) wpage;
+ tempptr += padsize;
+ wpage = (jint32_t *) tempptr;
+
+ *(wpage++) = offset;
+ *(wpage++) = cpu_to_je32(c->sector_size);
+ *(wpage++) = magic;
+ isum.sum_crc = cpu_to_je32(crc32(0, c->summary_buf, datasize));
+ isum.node_crc = cpu_to_je32(crc32(0, &isum, sizeof(isum) - 8));
+
+ vecs[0].iov_base = &isum;
+ vecs[0].iov_len = sizeof(isum);
+ vecs[1].iov_base = c->summary_buf;
+ vecs[1].iov_len = datasize;
+
+ D1(printk(KERN_DEBUG "JFFS2: writing out data to flash to pos : 0x%08x\n",jeb->offset + c->sector_size - jeb->free_size));
+
+ spin_unlock(&c->erase_completion_lock);
+ ret = jffs2_flash_writev(c, vecs, 2, jeb->offset + c->sector_size - jeb->free_size, &retlen, 0);
+ spin_lock(&c->erase_completion_lock);
+
+
+ if (ret || (retlen != infosize)) {
+ printk(KERN_WARNING "JFFS2: write of %zd bytes at 0x%08x failed. returned %d, retlen %zd\n",
+ infosize, jeb->offset + c->sector_size - jeb->free_size, ret, retlen);
+
+ jeb->sum_collected->sum_size = JFFS2_SUMMARY_NOSUM_SIZE;
+
+ WASTED_SPACE(infosize);
+
+ return 0;
+ }
+
+ /*spin_unlock(&c->erase_completion_lock);
+ jffs2_flush_wbuf_pad(c); // summary filled the wbuf
+ spin_lock(&c->erase_completion_lock);*/
+
+ WASTED_SPACE(infosize);
+
+ return 0;
+}
diff --unified --new-file --recursive mtd/fs/jffs2/summary.h mtd-dec07/fs/jffs2/summary.h
--- mtd/fs/jffs2/summary.h 1970-01-01 01:00:00.000000000 +0100
+++ mtd-dec07/fs/jffs2/summary.h 2004-12-07 16:03:02.000000000 +0100
@@ -0,0 +1,158 @@
+/*
+ * JFFS2 -- Journalling Flash File System, Version 2.
+ *
+ * Copyright (C) 2004 Ferenc Havasi <havasi@inf.u-szeged.hu>,
+ * Zoltan Sogor <weth@inf.u-szeged.hu>,
+ * Patrik Kluba <pajko@halom.u-szeged.hu>,
+ * University of Szeged, Hungary
+ *
+ * For licensing information, see the file 'LICENCE' in this directory.
+ *
+ * $Id$
+ *
+ */
+
+#ifndef JFFS2_SUMMARY_H
+#define JFFS2_SUMMARY_H
+
+#include <linux/uio.h>
+#include <linux/jffs2.h>
+
+#define DIRTY_SPACE(x) do { typeof(x) _x = (x); \
+ c->free_size -= _x; c->dirty_size += _x; \
+ jeb->free_size -= _x ; jeb->dirty_size += _x; \
+ }while(0)
+#define USED_SPACE(x) do { typeof(x) _x = (x); \
+ c->free_size -= _x; c->used_size += _x; \
+ jeb->free_size -= _x ; jeb->used_size += _x; \
+ }while(0)
+#define WASTED_SPACE(x) do { typeof(x) _x = (x); \
+ c->free_size -= _x; c->wasted_size += _x; \
+ jeb->free_size -= _x ; jeb->wasted_size += _x; \
+ }while(0)
+#define UNCHECKED_SPACE(x) do { typeof(x) _x = (x); \
+ c->free_size -= _x; c->unchecked_size += _x; \
+ jeb->free_size -= _x ; jeb->unchecked_size += _x; \
+ }while(0)
+
+#define BLK_STATE_ALLFF 0
+#define BLK_STATE_CLEAN 1
+#define BLK_STATE_PARTDIRTY 2
+#define BLK_STATE_CLEANMARKER 3
+#define BLK_STATE_ALLDIRTY 4
+#define BLK_STATE_BADBLOCK 5
+
+#define JFFS2_SUMMARY_NOSUM_SIZE 0xffffffff
+#define JFFS2_SUMMARY_INODE_SIZE (sizeof(struct jffs2_sum_inode_flash))
+#define JFFS2_SUMMARY_DIRENT_SIZE(x) (sizeof(struct jffs2_sum_dirent_flash) + (x))
+
+struct jffs2_sum_unknown_flash
+{
+ jint16_t nodetype; /* node type */
+};
+
+struct jffs2_sum_inode_flash
+{
+ jint16_t nodetype; /* node type */
+ jint32_t inode; /* inode number */
+ jint32_t version; /* inode version */
+ jint32_t offset; /* offset on jeb */
+ jint32_t totlen; /* record length */
+} __attribute__((packed));
+
+struct jffs2_sum_dirent_flash
+{
+ jint16_t nodetype; /* == JFFS_NODETYPE_DIRENT */
+ jint32_t totlen; /* record length */
+ jint32_t offset; /* ofset on jeb */
+ jint32_t pino; /* parent inode */
+ jint32_t version; /* dirent version */
+ jint32_t ino; /* == zero for unlink */
+ uint8_t nsize; /* dirent name size */
+ uint8_t type; /* dirent type */
+ uint8_t name[0]; /* dirent name */
+} __attribute__((packed));
+
+union jffs2_sum_flash{
+ struct jffs2_sum_unknown_flash u;
+ struct jffs2_sum_inode_flash i;
+ struct jffs2_sum_dirent_flash d;
+};
+
+/* list version of jffs2_sum_*flash for kernel and sumtool */
+struct jffs2_sum_unknown_mem
+{
+ union jffs2_sum_mem *next;
+ jint16_t nodetype; /* node type */
+
+};
+
+struct jffs2_sum_inode_mem
+{
+ union jffs2_sum_mem *next;
+ jint16_t nodetype; /* node type */
+ jint32_t inode; /* inode number */
+ jint32_t version; /* inode version */
+ jint32_t offset; /* offset on jeb */
+ jint32_t totlen; /* record length */
+} __attribute__((packed));
+
+struct jffs2_sum_dirent_mem
+{
+ union jffs2_sum_mem *next;
+ jint16_t nodetype; /* == JFFS_NODETYPE_DIRENT */
+ jint32_t totlen; /* record length */
+ jint32_t offset; /* ofset on jeb */
+ jint32_t pino; /* parent inode */
+ jint32_t version; /* dirent version */
+ jint32_t ino; /* == zero for unlink */
+ uint8_t nsize; /* dirent name size */
+ uint8_t type; /* dirent type */
+ uint8_t name[0]; /* dirent name */
+} __attribute__((packed));
+
+union jffs2_sum_mem
+{
+ struct jffs2_sum_unknown_mem u;
+ struct jffs2_sum_inode_mem i;
+ struct jffs2_sum_dirent_mem d;
+};
+
+struct jffs2_sum_info
+{
+ uint32_t sum_size;
+ uint32_t sum_num;
+ uint32_t sum_padded;
+ union jffs2_sum_mem *sum_list;
+};
+
+struct jffs2_sum_marker
+{
+ jint32_t offset;
+ jint32_t erase_size;
+ jint32_t magic;
+};
+
+#define JFFS2_SUMMARY_FRAME_SIZE (sizeof(struct jffs2_summary_node)+sizeof(struct jffs2_sum_marker)+3)
+
+#if !(defined(SUM_TOOL) || defined(JFFS2DUMP))
+
+int jffs2_sum_init(struct jffs2_sb_info *c);
+void jffs2_sum_exit(struct jffs2_sb_info *c);
+
+int jffs2_sum_care_sum_collected(struct jffs2_eraseblock *jeb);
+
+void jffs2_sum_clean_collected(struct jffs2_eraseblock *jeb);
+void jffs2_sum_clean_all_info(struct jffs2_sb_info *c); /* clean up all summary information in all jeb (umount) */
+
+int jffs2_sum_add_padding_mem(struct jffs2_eraseblock *jeb, uint32_t size);
+int jffs2_sum_add_inode_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_inode *ri, uint32_t ofs);
+int jffs2_sum_add_dirent_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_dirent *rd, uint32_t ofs);
+int jffs2_sum_add_kvec(struct jffs2_sb_info *c, const struct kvec *invecs, unsigned long count, uint32_t to);
+
+int jffs2_sum_scan_sumnode(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb, uint32_t ofs, uint32_t *pseudo_random);
+int jffs2_sum_write_sumnode(struct jffs2_sb_info *c);
+
+#endif
+
+#endif /* JFFS2_SUMMARY_H */
diff --unified --new-file --recursive mtd/fs/jffs2/super-v24.c mtd-dec07/fs/jffs2/super-v24.c
--- mtd/fs/jffs2/super-v24.c 2004-11-17 00:00:14.000000000 +0100
+++ mtd-dec07/fs/jffs2/super-v24.c 2004-12-07 16:03:02.000000000 +0100
@@ -24,6 +24,7 @@
#include <linux/mtd/mtd.h>
#include "compr.h"
#include "nodelist.h"
+#include "summary.h"
#ifndef MTD_BLOCK_MAJOR
#define MTD_BLOCK_MAJOR 31
@@ -72,6 +73,9 @@
put_mtd_device(c->mtd);
return NULL;
}
+#ifdef CONFIG_JFFS2_SUMMARY
+ jffs2_sum_init(c);
+#endif
return sb;
}
@@ -88,6 +92,10 @@
down(&c->alloc_sem);
jffs2_flush_wbuf_pad(c);
up(&c->alloc_sem);
+#ifdef CONFIG_JFFS2_SUMMARY
+ jffs2_sum_clean_all_info(c);
+ jffs2_sum_exit(c);
+#endif
jffs2_free_ino_caches(c);
jffs2_free_raw_node_refs(c);
kfree(c->blocks);
@@ -110,6 +118,9 @@
#ifdef CONFIG_JFFS2_FS_NAND
" (NAND)"
#endif
+#ifdef CONFIG_JFFS2_SUMMARY
+ " (SUMMARY)"
+#endif
" (C) 2001-2003 Red Hat, Inc.\n");
#ifdef JFFS2_OUT_OF_KERNEL
diff --unified --new-file --recursive mtd/fs/jffs2/super.c mtd-dec07/fs/jffs2/super.c
--- mtd/fs/jffs2/super.c 2004-11-26 15:08:31.000000000 +0100
+++ mtd-dec07/fs/jffs2/super.c 2004-12-07 16:03:02.000000000 +0100
@@ -152,6 +152,10 @@
return ERR_PTR(ret);
}
+#ifdef CONFIG_JFFS2_SUMMARY
+ jffs2_sum_init(c);
+#endif
+
sb->s_flags |= MS_ACTIVE;
return sb;
@@ -275,6 +279,10 @@
down(&c->alloc_sem);
jffs2_flush_wbuf_pad(c);
up(&c->alloc_sem);
+#ifdef CONFIG_JFFS2_SUMMARY
+ jffs2_sum_clean_all_info(c);
+ jffs2_sum_exit(c);
+#endif
jffs2_free_ino_caches(c);
jffs2_free_raw_node_refs(c);
if (c->mtd->flags & MTD_NO_VIRTBLOCKS)
@@ -312,6 +320,9 @@
#ifdef CONFIG_JFFS2_FS_NAND
" (NAND)"
#endif
+#ifdef CONFIG_JFFS2_SUMMARY
+ " (SUMMARY) "
+#endif
" (C) 2001-2003 Red Hat, Inc.\n");
jffs2_inode_cachep = kmem_cache_create("jffs2_i",
diff --unified --new-file --recursive mtd/fs/jffs2/wbuf.c mtd-dec07/fs/jffs2/wbuf.c
--- mtd/fs/jffs2/wbuf.c 2004-11-26 15:08:31.000000000 +0100
+++ mtd-dec07/fs/jffs2/wbuf.c 2004-12-07 16:17:11.000000000 +0100
@@ -261,7 +261,7 @@
/* ... and get an allocation of space from a shiny new block instead */
- ret = jffs2_reserve_space_gc(c, end-start, &ofs, &len);
+ ret = jffs2_reserve_space_gc(c, end-start, &ofs, &len, JFFS2_SUMMARY_NOSUM_SIZE);
if (ret) {
printk(KERN_WARNING "Failed to allocate space for wbuf recovery. Data loss ensues.\n");
if (buf)
@@ -601,8 +601,15 @@
uint32_t outvec_to = to;
/* If not NAND flash, don't bother */
- if (!c->wbuf)
+ if (!c->wbuf){
+#ifdef CONFIG_JFFS2_SUMMARY
+ D1(printk("JFFS2.summary: NOT NAND MEMORY\n"));
+ if(jffs2_sum_add_kvec(c,invecs,count,(uint32_t) to)){
+ printk("jffs2_sum_add_kvec(): MEMORY ALLOCATION ERROR!");
+ }
+#endif
return jffs2_flash_direct_writev(c, invecs, count, to, retlen);
+ }
down_write(&c->wbuf_sem);
@@ -801,6 +808,12 @@
alldone:
*retlen = donelen;
+#ifdef CONFIG_JFFS2_SUMMARY
+ if(jffs2_sum_add_kvec(c,invecs,count,(uint32_t) to)){
+ printk("jffs2_sum_add_kvec(): MEMORY ALLOCATION ERROR!");
+ }
+#endif
+
if (c->wbuf_len && ino)
jffs2_wbuf_dirties_inode(c, ino);
diff --unified --new-file --recursive mtd/fs/jffs2/write.c mtd-dec07/fs/jffs2/write.c
--- mtd/fs/jffs2/write.c 2004-11-17 00:00:14.000000000 +0100
+++ mtd-dec07/fs/jffs2/write.c 2004-12-07 16:03:02.000000000 +0100
@@ -173,13 +173,13 @@
D1(ACCT_PARANOIA_CHECK(jeb));
if (alloc_mode == ALLOC_GC) {
- ret = jffs2_reserve_space_gc(c, sizeof(*ri) + datalen, &flash_ofs, &dummy);
+ ret = jffs2_reserve_space_gc(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, JFFS2_SUMMARY_INODE_SIZE);
} else {
/* Locking pain */
up(&f->sem);
jffs2_complete_reservation(c);
- ret = jffs2_reserve_space(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, alloc_mode);
+ ret = jffs2_reserve_space(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, alloc_mode, JFFS2_SUMMARY_INODE_SIZE);
down(&f->sem);
}
@@ -308,13 +308,13 @@
D1(ACCT_PARANOIA_CHECK(jeb));
if (alloc_mode == ALLOC_GC) {
- ret = jffs2_reserve_space_gc(c, sizeof(*rd) + namelen, &flash_ofs, &dummy);
+ ret = jffs2_reserve_space_gc(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
} else {
/* Locking pain */
up(&f->sem);
jffs2_complete_reservation(c);
- ret = jffs2_reserve_space(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, alloc_mode);
+ ret = jffs2_reserve_space(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, alloc_mode, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
down(&f->sem);
}
@@ -371,7 +371,7 @@
retry:
D2(printk(KERN_DEBUG "jffs2_commit_write() loop: 0x%x to write to 0x%x\n", writelen, offset));
- ret = jffs2_reserve_space(c, sizeof(*ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, ALLOC_NORMAL);
+ ret = jffs2_reserve_space(c, sizeof(*ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
if (ret) {
D1(printk(KERN_DEBUG "jffs2_reserve_space returned %d\n", ret));
break;
@@ -458,7 +458,7 @@
/* Try to reserve enough space for both node and dirent.
* Just the node will do for now, though
*/
- ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL);
+ ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
D1(printk(KERN_DEBUG "jffs2_do_create(): reserved 0x%x bytes\n", alloclen));
if (ret) {
up(&f->sem);
@@ -487,7 +487,7 @@
up(&f->sem);
jffs2_complete_reservation(c);
- ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
+ ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
if (ret) {
/* Eep. */
@@ -557,7 +557,7 @@
if (!rd)
return -ENOMEM;
- ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_DELETION);
+ ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_DELETION, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
if (ret) {
jffs2_free_raw_dirent(rd);
return ret;
@@ -663,7 +663,7 @@
if (!rd)
return -ENOMEM;
- ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
+ ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
if (ret) {
jffs2_free_raw_dirent(rd);
return ret;
diff --unified --new-file --recursive mtd/include/linux/jffs2.h mtd-dec07/include/linux/jffs2.h
--- mtd/include/linux/jffs2.h 2004-11-17 00:00:15.000000000 +0100
+++ mtd-dec07/include/linux/jffs2.h 2004-12-07 16:03:02.000000000 +0100
@@ -28,6 +28,9 @@
#define JFFS2_EMPTY_BITMASK 0xffff
#define JFFS2_DIRTY_BITMASK 0x0000
+/* Summary node MAGIC marker */
+#define JFFS2_SUM_MAGIC 0x02851885
+
/* We only allow a single char for length, and 0xFF is empty flash so
we don't want it confused with a real length. Hence max 254.
*/
@@ -62,6 +65,8 @@
#define JFFS2_NODETYPE_CLEANMARKER (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3)
#define JFFS2_NODETYPE_PADDING (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 4)
+#define JFFS2_NODETYPE_SUMMARY (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 6)
+
// Maybe later...
//#define JFFS2_NODETYPE_CHECKPOINT (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3)
//#define JFFS2_NODETYPE_OPTIONS (JFFS2_FEATURE_RWCOMPAT_COPY | JFFS2_NODE_ACCURATE | 4)
@@ -148,10 +153,24 @@
uint8_t data[0];
} __attribute__((packed));
+struct jffs2_summary_node{
+ jint16_t magic;
+ jint16_t nodetype; /* = JFFS2_NODETYPE_INODE_SUM */
+ jint32_t totlen;
+ jint32_t hdr_crc;
+ jint16_t sum_num; /* number of sum entries*/
+ jint32_t cln_mkr; /* clean marker size, 0 = no cleanmarker */
+ jint32_t padded; /* sum of the size of padding nodes */
+ jint32_t sum_crc; /* summary information crc */
+ jint32_t node_crc; /* node crc */
+ jint32_t sum[0]; /* inode summary info */
+} __attribute__((packed));
+
union jffs2_node_union {
struct jffs2_raw_inode i;
struct jffs2_raw_dirent d;
struct jffs2_unknown_node u;
+ struct jffs2_summary_node s;
};
#endif /* __LINUX_JFFS2_H__ */
diff --unified --new-file --recursive mtd/include/linux/jffs2_fs_sb.h mtd-dec07/include/linux/jffs2_fs_sb.h
--- mtd/include/linux/jffs2_fs_sb.h 2004-11-26 15:08:31.000000000 +0100
+++ mtd-dec07/include/linux/jffs2_fs_sb.h 2004-12-07 16:03:02.000000000 +0100
@@ -110,6 +110,9 @@
uint32_t fsdata_pos;
uint32_t fsdata_len;
#endif
+#ifdef CONFIG_JFFS2_SUMMARY
+ jint32_t *summary_buf;
+#endif
/* OS-private pointer for getting back to master superblock info */
void *os_priv;
diff --unified --new-file --recursive mtd/util/Makefile mtd-dec07/util/Makefile
--- mtd/util/Makefile 2004-07-14 00:00:18.000000000 +0200
+++ mtd-dec07/util/Makefile 2004-12-07 16:03:02.000000000 +0100
@@ -13,7 +13,7 @@
TARGETS = ftl_format flash_erase flash_eraseall nanddump doc_loadbios \
mkfs.jffs ftl_check mkfs.jffs2 flash_lock flash_unlock \
flash_info mtd_debug flashcp nandwrite jffs2dump \
- nftldump nftl_format docfdisk #jffs2reader
+ nftldump nftl_format docfdisk sumtool #jffs2reader
SYMLINKS = compr_lzari.c compr_lzo.c
@@ -48,6 +48,10 @@
jffs2dump: jffs2dump.o crc32.o
$(CC) $(LDFLAGS) -o $@ $^
+sumtool: sumtool.o crc32.o
+ $(CC) $(LDFLAGS) -o $@ $^
+
+
install: ${TARGETS}
mkdir -p ${DESTDIR}/${SBINDIR}
install -m0755 -oroot -groot ${TARGETS} ${DESTDIR}/${SBINDIR}/
diff --unified --new-file --recursive mtd/util/jffs2dump.c mtd-dec07/util/jffs2dump.c
--- mtd/util/jffs2dump.c 2004-12-03 00:00:10.000000000 +0100
+++ mtd-dec07/util/jffs2dump.c 2004-12-07 16:03:02.000000000 +0100
@@ -35,6 +35,10 @@
#include <getopt.h>
#include "crc32.h"
+#define CONFIG_JFFS2_SUMMARY
+#define JFFS2DUMP
+#include "../fs/jffs2/summary.h"
+
#define PROGRAM "jffs2dump"
#define VERSION "$Revision: 1.7 $"
@@ -177,7 +181,7 @@
*/
void do_dumpcontent (void)
{
- char *p = data;
+ char *p = data, *p_free_begin;
union jffs2_node_union *node;
int empty = 0, dirty = 0;
char name[256];
@@ -186,16 +190,21 @@
int bitchbitmask = 0;
int obsolete;
+ p_free_begin = NULL;
while ( p < (data + imglen)) {
node = (union jffs2_node_union*) p;
/* Skip empty space */
+ if (!p_free_begin) p_free_begin = p;
if (je16_to_cpu (node->u.magic) == 0xFFFF && je16_to_cpu (node->u.nodetype) == 0xFFFF) {
p += 4;
empty += 4;
continue;
}
-
+
+ if (p != p_free_begin) printf("Empty space found from 0x%08x to 0x%08x\n", p_free_begin-data, p-data);
+ p_free_begin = NULL;
+
if (je16_to_cpu (node->u.magic) != JFFS2_MAGIC_BITMASK) {
if (!bitchbitmask++)
printf ("Wrong bitmask at 0x%08x, 0x%04x\n", p - data, je16_to_cpu (node->u.magic));
@@ -277,7 +286,110 @@
p += PAD(je32_to_cpu (node->d.totlen));
break;
-
+
+ case JFFS2_NODETYPE_SUMMARY:{
+
+ int i;
+ struct jffs2_sum_marker * sm;
+
+ printf ("%8s Inode Sum node at 0x%08x, totlen 0x%08x, sum_num %5d, cleanmarker size %5d\n",
+ obsolete ? "Obsolete" : "",
+ p - data,
+ je32_to_cpu (node->s.totlen),
+ je16_to_cpu (node->s.sum_num),
+ je32_to_cpu (node->s.cln_mkr));
+
+ crc = crc32 (0, node, sizeof (struct jffs2_summary_node) - 8);
+ if (crc != je32_to_cpu (node->s.node_crc)) {
+ printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.node_crc), crc);
+ p += PAD(je32_to_cpu (node->s.totlen));
+ dirty += PAD(je32_to_cpu (node->s.totlen));;
+ continue;
+ }
+
+ crc = crc32(0, p + sizeof (struct jffs2_summary_node), je32_to_cpu (node->s.totlen) - sizeof(struct jffs2_summary_node));
+ if (crc != je32_to_cpu(node->s.sum_crc)) {
+ printf ("Wrong data_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.sum_crc), crc);
+ p += PAD(je32_to_cpu (node->s.totlen));
+ dirty += PAD(je32_to_cpu (node->s.totlen));;
+ continue;
+ }
+
+ if(verbose){
+ struct jffs2_sum_unknown_flash *sp;
+ uint8_t *temp8ptr = NULL;
+ sp = (struct jffs2_sum_unknown_flash *) (p + sizeof (struct jffs2_summary_node));
+
+ for(i = 0; i < je16_to_cpu (node->s.sum_num); i++){
+
+ switch(je16_to_cpu(sp->nodetype)){
+ case JFFS2_NODETYPE_INODE : {
+
+ struct jffs2_sum_inode_flash *spi;
+ spi = (struct jffs2_sum_inode_flash *) sp;
+
+ printf ("%14s #ino %5d, version %5d, offset 0x%08x, totlen 0x%08x\n",
+ "",
+ je32_to_cpu (spi->inode),
+ je32_to_cpu (spi->version),
+ je32_to_cpu (spi->offset),
+ je32_to_cpu (spi->totlen));
+
+ temp8ptr = (uint8_t *) sp;
+ temp8ptr += JFFS2_SUMMARY_INODE_SIZE;
+ sp = (struct jffs2_sum_unknown_flash *) temp8ptr;
+
+ break;
+ }
+
+ case JFFS2_NODETYPE_DIRENT : {
+
+ char name[255];
+ struct jffs2_sum_dirent_flash *spd;
+ spd = (struct jffs2_sum_dirent_flash *) sp;
+
+ memcpy(name,spd->name,spd->nsize);
+ name [spd->nsize] = 0x0;
+
+ printf ("%14s dirent offset 0x%08x, totlen 0x%08x, #pino %5d, version %5d, #ino %8d, nsize %8d, name %s \n",
+ "",
+ je32_to_cpu (spd->offset),
+ je32_to_cpu (spd->totlen),
+ je32_to_cpu (spd->pino),
+ je32_to_cpu (spd->version),
+ je32_to_cpu (spd->ino),
+ spd->nsize,
+ name);
+
+ temp8ptr = (uint8_t *) sp;
+ temp8ptr += JFFS2_SUMMARY_DIRENT_SIZE(spd->nsize);
+ sp = (struct jffs2_sum_unknown_flash *) temp8ptr;
+
+ break;
+ }
+
+ default : {
+ break;
+ }
+ }
+ }
+
+
+ sm = (struct jffs2_sum_marker *) ((char *)p + je32_to_cpu(node->s.totlen) - sizeof(struct jffs2_sum_marker));
+
+ printf("%14s Sum Node Offset 0x%08x, Erase_block_size 0x%08x, Magic 0x%08x, Padded size 0x%08x\n" ,
+ "",
+ je32_to_cpu(sm->offset),
+ je32_to_cpu(sm->erase_size),
+ je32_to_cpu(sm->magic),
+ je32_to_cpu(node->s.padded));
+ }
+
+ p += PAD(je32_to_cpu (node->s.totlen));
+ break;
+
+ }
+
case JFFS2_NODETYPE_CLEANMARKER:
if (verbose) {
printf ("%8s Cleanmarker at 0x%08x, totlen 0x%08x\n",
diff --unified --new-file --recursive mtd/util/sumtool.c mtd-dec07/util/sumtool.c
--- mtd/util/sumtool.c 1970-01-01 01:00:00.000000000 +0100
+++ mtd-dec07/util/sumtool.c 2004-12-07 17:47:31.000000000 +0100
@@ -0,0 +1,857 @@
+/*
+ * sumtool.c
+ *
+ * Copyright (C) 2004 Zoltan Sogor <weth@inf.u-szeged.hu>,
+ * Ferenc Havasi <havasi@inf.u-szeged.hu>,
+ * Patrik Kluba <pajko@halom.u-szeged.hu>,
+ * University of Szeged, Hungary
+ *
+ * $Id: sumtool.c,v 1.2 2004/10/20 09:56:08 hafy Exp $
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Overview:
+ * This is a utility to reorder nodes and insert inode summary information
+ * into JFFS2 image for faster mount time - specially on NAND.
+ *
+ */
+
+#include <errno.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <string.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <time.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/param.h>
+#include <asm/types.h>
+#include <dirent.h>
+#include <mtd/jffs2-user.h>
+#include <endian.h>
+#include <byteswap.h>
+#include <getopt.h>
+#include "crc32.h"
+
+#define CONFIG_JFFS2_SUMMARY
+#define SUM_TOOL
+#include "../fs/jffs2/summary.h"
+
+
+#define PAD(x) (((x)+3)&~3)
+
+static const char *const app_name = "sumtool";
+
+static struct jffs2_sum_info *sum_collected = NULL;/* summary info list */
+
+static int verbose = 0;
+static int padto = 0; /* pad the output with 0xFF to the end of the final eraseblock */
+static int add_cleanmarkers = 1; /* add cleanmarker to output */
+static int use_input_cleanmarker_size = 1; /* use input file's cleanmarker size (default) */
+static int found_cleanmarkers = 0; /* cleanmarker found in input file */
+static struct jffs2_unknown_node cleanmarker;
+static int cleanmarker_size = sizeof(cleanmarker);
+static const char *short_options = "o:i:e:hvVblnc:p";
+static int erase_block_size = 65536;
+static int target_endian = __BYTE_ORDER;
+static int out_fd = -1;
+static int in_fd = -1;
+
+static uint8_t *data_buffer = NULL; /* buffer for inodes */
+static unsigned int data_ofs = 0; /* inode buffer offset */
+
+static uint8_t *file_buffer = NULL; /* file buffer contains the actual erase block*/
+static unsigned int file_ofs = 0; /* position in the buffer */
+
+static struct option long_options[] = {
+ {"output", 1, NULL, 'o'},
+ {"input", 1, NULL, 'i'},
+ {"eraseblock", 1, NULL, 'e'},
+ {"help", 0, NULL, 'h'},
+ {"verbose", 0, NULL, 'v'},
+ {"version", 0, NULL, 'V'},
+ {"bigendian", 0, NULL, 'b'},
+ {"littleendian", 0, NULL, 'l'},
+ {"no-cleanmarkers", 0, NULL, 'n'},
+ {"cleanmarker", 1, NULL, 'c'},
+ {"pad", 0, NULL, 'p'},
+ {NULL, 0, NULL, 0}
+};
+
+static char *helptext =
+ "Usage: sumtool [OPTIONS] -i inputfile -o outputfile\n"
+ "Convert the input JFFS2 file to a SUM-ed JFFS2 file\n\n"
+ "Options:\n"
+ " -e, --eraseblock=SIZE Use erase block size SIZE (default: 64KiB)\n"
+ " (usually 16KiB on NAND)\n"
+ " -c, --cleanmarker=SIZE Size of cleanmarker (default 12).\n"
+ " (usually 16 bytes on NAND, and will be set to\n"
+ " this value if left at the default 12). Will be\n"
+ " stored in OOB after each physical page composing\n"
+ " a physical erase block.\n"
+ " -n, --no-cleanmarkers Don't add a cleanmarker to every eraseblock\n"
+ " -o, --output=FILE Output to FILE \n"
+ " -i, --input=FILE Input from FILE \n"
+ " -b, --bigendian Image is big endian\n"
+ " -l --littleendian Image is little endian\n"
+ " -h, --help Display this help text\n"
+ " -v, --verbose Verbose operation\n"
+ " -V, --version Display version information\n"
+ " -p, --pad Pad the OUTPUT with 0xFF to the end of the final\n"
+ " eraseblock\n\n";
+
+
+static char *revtext = "$Revision: 1.3 $";
+
+static unsigned char ffbuf[16] = {
+ 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
+ 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff
+};
+
+static void verror_msg(const char *s, va_list p) {
+ fflush(stdout);
+ fprintf(stderr, "%s: ", app_name);
+ vfprintf(stderr, s, p);
+}
+
+static void error_msg_and_die(const char *s, ...) {
+ va_list p;
+
+ va_start(p, s);
+ verror_msg(s, p);
+ va_end(p);
+ putc('\n', stderr);
+ exit(EXIT_FAILURE);
+}
+
+static void vperror_msg(const char *s, va_list p) {
+ int err = errno;
+
+ if (s == 0)
+ s = "";
+ verror_msg(s, p);
+ if (*s)
+ s = ": ";
+ fprintf(stderr, "%s%s\n", s, strerror(err));
+}
+
+static void perror_msg_and_die(const char *s, ...) {
+ va_list p;
+
+ va_start(p, s);
+ vperror_msg(s, p);
+ va_end(p);
+ exit(EXIT_FAILURE);
+}
+
+
+
+static void full_write(void *target_buff, const void *buf, int len);
+
+void setup_cleanmarker() {
+
+ cleanmarker.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
+ cleanmarker.nodetype = cpu_to_je16(JFFS2_NODETYPE_CLEANMARKER);
+ cleanmarker.totlen = cpu_to_je32(cleanmarker_size);
+ cleanmarker.hdr_crc = cpu_to_je32(crc32(0, &cleanmarker, sizeof(struct jffs2_unknown_node)-4));
+}
+
+void process_options (int argc, char **argv){
+ int opt,c;
+
+ while ((opt = getopt_long(argc, argv, short_options, long_options, &c)) >= 0)
+ {
+ switch (opt)
+ {
+ case 'o':
+ if (out_fd != -1) {
+ error_msg_and_die("output filename specified more than once");
+ }
+ out_fd = open(optarg, O_CREAT | O_TRUNC | O_RDWR, 0644);
+ if (out_fd == -1) {
+ perror_msg_and_die("open output file");
+ }
+ break;
+
+ case 'i':
+ if (in_fd != -1) {
+ error_msg_and_die("input filename specified more than once");
+ }
+ in_fd = open(optarg, O_RDONLY);
+ if (in_fd == -1) {
+ perror_msg_and_die("open input file");
+ }
+ break;
+ case 'b':
+ target_endian = __BIG_ENDIAN;
+ break;
+ case 'l':
+ target_endian = __LITTLE_ENDIAN;
+ break;
+ case 'h':
+ case '?':
+ error_msg_and_die(helptext);
+
+ case 'v':
+ verbose = 1;
+ break;
+
+ case 'V':
+ error_msg_and_die("revision %.*s\n",
+ (int) strlen(revtext) - 13, revtext + 11);
+
+ case 'e': {
+ char *next;
+ unsigned units = 0;
+ erase_block_size = strtol(optarg, &next, 0);
+ if (!erase_block_size)
+ error_msg_and_die("Unrecognisable erase size\n");
+
+ if (*next) {
+ if (!strcmp(next, "KiB")) {
+ units = 1024;
+ } else if (!strcmp(next, "MiB")) {
+ units = 1024 * 1024;
+ } else {
+ error_msg_and_die("Unknown units in erasesize\n");
+ }
+ } else {
+ if (erase_block_size < 0x1000)
+ units = 1024;
+ else
+ units = 1;
+ }
+ erase_block_size *= units;
+
+ /* If it's less than 8KiB, they're not allowed */
+ if (erase_block_size < 0x2000) {
+ fprintf(stderr, "Erase size 0x%x too small. Increasing to 8KiB minimum\n",
+ erase_block_size);
+ erase_block_size = 0x2000;
+ }
+ break;
+ }
+
+ case 'n':
+ add_cleanmarkers = 0;
+ break;
+ case 'c':
+ cleanmarker_size = strtol(optarg, NULL, 0);
+
+ if (cleanmarker_size < sizeof(cleanmarker)) {
+ error_msg_and_die("cleanmarker size must be >= 12");
+ }
+ if (cleanmarker_size >= erase_block_size) {
+ error_msg_and_die("cleanmarker size must be < eraseblock size");
+ }
+
+ use_input_cleanmarker_size = 0;
+ found_cleanmarkers = 1;
+ setup_cleanmarker();
+
+ break;
+ case 'p':
+ padto = 1;
+ break;
+
+ }
+ }
+}
+
+
+void init_buffers() {
+
+ data_buffer = malloc(erase_block_size);
+
+ if (!data_buffer) {
+ perror("out of memory");
+ close (in_fd);
+ close (out_fd);
+ exit(1);
+ }
+
+ file_buffer = malloc(erase_block_size);
+
+ if (!file_buffer) {
+ perror("out of memory");
+ close (in_fd);
+ close (out_fd);
+ exit(1);
+ }
+}
+
+void init_sumlist(){
+
+ sum_collected = (struct jffs2_sum_info *) malloc (sizeof(struct jffs2_sum_info));
+
+ if (!sum_collected)
+ error_msg_and_die("Can't allocate memory for jffs2_sum_info!\n");
+
+ sum_collected->sum_list = NULL;
+ sum_collected->sum_num = 0;
+ sum_collected->sum_size = 0;
+
+}
+
+void clean_buffers() {
+
+ if (data_buffer)
+ free(data_buffer);
+ if (file_buffer)
+ free(file_buffer);
+}
+
+void clean_sumlist() {
+
+ union jffs2_sum_mem *temp;
+
+ if(sum_collected){
+
+ while(sum_collected->sum_list){
+ temp = sum_collected->sum_list;
+ sum_collected->sum_list = sum_collected->sum_list->u.next;
+ free(temp);
+ sum_collected->sum_num--;
+ }
+
+ if(sum_collected->sum_num != 0)
+ printf("Ooops, something wrong happened! sum_num != 0, but sum_list = null ???");
+
+ free(sum_collected);
+ }
+}
+
+int load_next_block() {
+
+ int ret;
+ ret = read(in_fd, file_buffer, erase_block_size);
+ file_ofs = 0;
+
+ if(verbose)
+ printf("Load next block : %d bytes read\n",ret);
+
+ return ret;
+}
+
+void write_buff_to_file() {
+
+ int ret;
+ int len = data_ofs;
+
+ uint8_t *buf = NULL;
+
+ buf = data_buffer;
+ while (len > 0) {
+ ret = write(out_fd, buf, len);
+
+ if (ret < 0)
+ perror_msg_and_die("write");
+
+ if (ret == 0)
+ perror_msg_and_die("write returned zero");
+
+ len -= ret;
+ buf += ret;
+ }
+
+ data_ofs = 0;
+}
+
+void dump_sum_records() {
+
+ struct jffs2_summary_node isum;
+ union jffs2_sum_mem *temp;
+ jint32_t offset;
+ jint32_t *wpage;
+ uint8_t *tempptr;
+ int datasize;
+ int infosize;
+ int padsize;
+ jint32_t magic = cpu_to_je32(JFFS2_SUM_MAGIC);
+
+ if (!sum_collected->sum_num || !sum_collected->sum_list)
+ return;
+
+ datasize = sum_collected->sum_size + sizeof(struct jffs2_sum_marker);
+ infosize = sizeof(struct jffs2_summary_node) + datasize;
+ padsize = erase_block_size - data_ofs - infosize;
+ infosize += padsize; datasize += padsize;
+ offset = cpu_to_je32(data_ofs);
+
+ jint32_t *tpage = (jint32_t *) malloc(datasize);
+
+ if(!tpage)
+ error_msg_and_die("Can't allocate memory to dump summary information!\n");
+
+ memset(tpage, 0xff, datasize);
+ memset(&isum, 0, sizeof(isum));
+
+ isum.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
+ isum.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY);
+ isum.totlen = cpu_to_je32(infosize);
+ isum.hdr_crc = cpu_to_je32(crc32(0, &isum, sizeof(struct jffs2_unknown_node) - 4));
+ isum.padded = cpu_to_je32(0);
+
+ if (add_cleanmarkers && found_cleanmarkers) {
+ isum.cln_mkr = cpu_to_je32(cleanmarker_size);
+ }
+ else{
+ isum.cln_mkr = cpu_to_je32(0);
+ }
+
+ isum.sum_num = cpu_to_je16(sum_collected->sum_num);
+ wpage = tpage;
+
+
+ while (sum_collected->sum_num) {
+
+
+ switch(je16_to_cpu(sum_collected->sum_list->u.nodetype)){
+
+ case JFFS2_NODETYPE_INODE : {
+ jint16_t *temp16ptr = (jint16_t *)wpage;
+
+ *(temp16ptr++) = sum_collected->sum_list->i.nodetype;
+ wpage = (jint32_t *) temp16ptr;
+
+ *(wpage++) = sum_collected->sum_list->i.inode;
+ *(wpage++) = sum_collected->sum_list->i.version;
+ *(wpage++) = sum_collected->sum_list->i.offset;
+ *(wpage++) = sum_collected->sum_list->i.totlen;
+ break;
+ }
+
+ case JFFS2_NODETYPE_DIRENT : {
+ jint16_t *temp16ptr = (jint16_t *) wpage;
+ uint8_t *temp8ptr = NULL;
+
+ *(temp16ptr++) = sum_collected->sum_list->d.nodetype;
+ wpage = (jint32_t *) temp16ptr;
+
+ *(wpage++) = sum_collected->sum_list->d.totlen;
+ *(wpage++) = sum_collected->sum_list->d.offset;
+ *(wpage++) = sum_collected->sum_list->d.pino;
+ *(wpage++) = sum_collected->sum_list->d.version;
+ *(wpage++) = sum_collected->sum_list->d.ino;
+
+ temp8ptr = (uint8_t *) wpage;
+ *(temp8ptr++) = sum_collected->sum_list->d.nsize;
+ *(temp8ptr++) = sum_collected->sum_list->d.type;
+
+ memcpy(temp8ptr,sum_collected->sum_list->d.name,sum_collected->sum_list->d.nsize);
+ temp8ptr += sum_collected->sum_list->d.nsize;
+ wpage = (jint32_t *) temp8ptr;
+
+ break;
+ }
+
+ default : {
+ printf("This is not good for me!!!!\n");
+ }
+ }
+
+ temp = sum_collected->sum_list;
+ sum_collected->sum_list = sum_collected->sum_list->u.next;
+ free(temp);
+
+ sum_collected->sum_num--;
+ }
+
+ sum_collected->sum_size = 0;
+
+ tempptr = (uint8_t *) wpage;
+ tempptr += padsize;
+ wpage = (jint32_t *) tempptr;
+
+ *(wpage++) = offset;
+ *(wpage++) = cpu_to_je32(erase_block_size);
+ *(wpage++) = magic;
+ isum.sum_crc = cpu_to_je32(crc32(0, tpage, datasize));
+ isum.node_crc = cpu_to_je32(crc32(0, &isum, sizeof(isum) - 8));
+
+ full_write(data_buffer + data_ofs, &isum, sizeof(isum));
+ full_write(data_buffer + data_ofs, tpage, datasize);
+
+ free(tpage);
+
+}
+
+static void full_write(void *target_buff, const void *buf, int len) {
+
+ memcpy(target_buff, buf, len);
+ data_ofs += len;
+}
+
+static void pad(int req) {
+ while (req) {
+ if (req > sizeof(ffbuf)) {
+ full_write(data_buffer + data_ofs, ffbuf, sizeof(ffbuf));
+ req -= sizeof(ffbuf);
+ } else {
+ full_write(data_buffer + data_ofs, ffbuf, req);
+ req = 0;
+ }
+ }
+}
+
+static inline void padword() {
+
+ if (data_ofs % 4) {
+ full_write(data_buffer + data_ofs, ffbuf, 4 - (data_ofs % 4));
+ }
+}
+
+
+static inline void pad_block_if_less_than(int req,int plus) {
+
+ int datasize = req + plus + sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8;
+ datasize += (4 - (datasize % 4)) % 4;
+
+ if (data_ofs + req > erase_block_size - datasize) {
+ dump_sum_records();
+ write_buff_to_file();
+ }
+
+ if (add_cleanmarkers && found_cleanmarkers) {
+ if (!data_ofs) {
+ full_write(data_buffer, &cleanmarker, sizeof(cleanmarker));
+ pad(cleanmarker_size - sizeof(cleanmarker));
+ padword();
+ }
+ }
+}
+
+void flush_buffers() {
+
+ if ((add_cleanmarkers == 1) && (found_cleanmarkers == 1)) { /* CLEANMARKER */
+ if (data_ofs != cleanmarker_size) { /* INODE BUFFER */
+
+ int datasize = sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8;
+ datasize += (4 - (datasize % 4)) % 4;
+
+ /* If we have a full inode buffer, then write out inode and summary data */
+ if (data_ofs + sizeof(struct jffs2_raw_inode) + 2*JFFS2_MIN_DATA_LEN > erase_block_size - datasize) {
+ dump_sum_records();
+ write_buff_to_file();
+ }
+ /* else just write out inode data */
+ else{
+ if(padto)
+ pad(erase_block_size - data_ofs);
+ write_buff_to_file();
+ }
+ }
+ }
+ else { /* NO CLEANMARKER */
+ if (data_ofs != 0) { /* INODE BUFFER */
+
+ int datasize = sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8;
+ datasize += (4 - (datasize % 4)) % 4;
+
+ /* If we have a full inode buffer, then write out inode and summary data */
+ if (data_ofs + sizeof(struct jffs2_raw_inode) + 2*JFFS2_MIN_DATA_LEN > erase_block_size - datasize) {
+ dump_sum_records();
+ write_buff_to_file();
+ }
+ /* Else just write out inode data */
+ else{
+ if(padto)
+ pad(erase_block_size - data_ofs);
+ write_buff_to_file();
+ }
+ }
+ }
+}
+
+int add_sum_mem(union jffs2_sum_mem *item) {
+
+ union jffs2_sum_mem *walk;
+
+ if (!sum_collected->sum_list) {
+ sum_collected->sum_list = (union jffs2_sum_mem *) item;
+ }
+ else {
+ walk = sum_collected->sum_list;
+
+ while (walk->u.next) {
+ walk = walk->u.next;
+ }
+ walk->u.next = (union jffs2_sum_mem *) item;
+ }
+ switch (je16_to_cpu(item->u.nodetype)) {
+ case JFFS2_NODETYPE_INODE:
+ sum_collected->sum_size += JFFS2_SUMMARY_INODE_SIZE;
+ sum_collected->sum_num++;
+ break;
+
+ case JFFS2_NODETYPE_DIRENT:
+ sum_collected->sum_size += JFFS2_SUMMARY_DIRENT_SIZE(item->d.nsize);
+ sum_collected->sum_num++;
+ break;
+
+ default:
+ error_msg_and_die("__jffs2_add_sum_mem(): UNKNOWN node type %d\n", je16_to_cpu(item->u.nodetype));
+ }
+ return 0;
+}
+
+void add_sum_inode_mem(union jffs2_node_union *node) {
+
+ struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *) malloc(sizeof(struct jffs2_sum_inode_mem));
+
+ if (!temp)
+ error_msg_and_die("Can't allocate memory for summary information!\n");
+
+ temp->nodetype = node->i.nodetype;
+ temp->inode = node->i.ino;
+ temp->version = node->i.version;
+ temp->offset = cpu_to_je32(data_ofs);
+ temp->totlen = node->i.totlen;
+ temp->next = NULL;
+
+ add_sum_mem((union jffs2_sum_mem *) temp);
+
+}
+
+void add_sum_dirent_mem(union jffs2_node_union *node) {
+
+ struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *)
+ malloc(sizeof(struct jffs2_sum_dirent_mem) + node->d.nsize);
+
+ if (!temp)
+ error_msg_and_die("Can't allocate memory for summary information!\n");
+
+ temp->nodetype = node->d.nodetype;
+ temp->totlen = node->d.totlen;
+ temp->offset = cpu_to_je32(data_ofs);
+ temp->pino = node->d.pino;
+ temp->version = node->d.version;
+ temp->ino = node->d.ino;
+ temp->nsize = node->d.nsize;
+ temp->type = node->d.type;
+ temp->next = NULL;
+
+ memcpy(temp->name,node->d.name,node->d.nsize);
+
+ add_sum_mem((union jffs2_sum_mem *) temp);
+
+}
+
+void write_dirent_to_buff(union jffs2_node_union *node) {
+
+ pad_block_if_less_than(je32_to_cpu (node->d.totlen),JFFS2_SUMMARY_DIRENT_SIZE(node->d.nsize));
+ add_sum_dirent_mem(node);
+ full_write(data_buffer + data_ofs, &(node->d), je32_to_cpu (node->d.totlen));
+ padword();
+}
+
+
+void write_inode_to_buff(union jffs2_node_union *node) {
+
+ pad_block_if_less_than(je32_to_cpu (node->i.totlen),JFFS2_SUMMARY_INODE_SIZE);
+ add_sum_inode_mem(node); /* Add inode summary mem to summary list */
+ full_write(data_buffer + data_ofs, &(node->i), je32_to_cpu (node->i.totlen)); /* Write out the inode to inode_buffer */
+ padword();
+
+}
+
+
+void create_summed_image(int inp_size) {
+ uint8_t *p = file_buffer;
+ union jffs2_node_union *node;
+ uint32_t crc;
+ uint16_t type;
+ int bitchbitmask = 0;
+ int obsolete;
+
+ char name[256];
+
+ while ( p < (file_buffer + inp_size)) {
+
+ node = (union jffs2_node_union *) p;
+
+ /* Skip empty space */
+ if (je16_to_cpu (node->u.magic) == 0xFFFF && je16_to_cpu (node->u.nodetype) == 0xFFFF) {
+ p += 4;
+ continue;
+ }
+
+ if (je16_to_cpu (node->u.magic) != JFFS2_MAGIC_BITMASK) {
+ if (!bitchbitmask++)
+ printf ("Wrong bitmask at 0x%08x, 0x%04x\n", p - file_buffer, je16_to_cpu (node->u.magic));
+ p += 4;
+ continue;
+ }
+
+ bitchbitmask = 0;
+
+ type = je16_to_cpu(node->u.nodetype);
+ if ((type & JFFS2_NODE_ACCURATE) != JFFS2_NODE_ACCURATE) {
+ obsolete = 1;
+ type |= JFFS2_NODE_ACCURATE;
+ } else
+ obsolete = 0;
+
+ node->u.nodetype = cpu_to_je16(type);
+
+ crc = crc32 (0, node, sizeof (struct jffs2_unknown_node) - 4);
+ if (crc != je32_to_cpu (node->u.hdr_crc)) {
+ printf ("Wrong hdr_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->u.hdr_crc), crc);
+ p += 4;
+ continue;
+ }
+
+ switch(je16_to_cpu(node->u.nodetype)) {
+
+ case JFFS2_NODETYPE_INODE:
+ if(verbose)
+ printf ("%8s Inode node at 0x%08x, totlen 0x%08x, #ino %5d, version %5d, isize %8d, csize %8d, dsize %8d, offset %8d\n",
+ obsolete ? "Obsolete" : "",
+ p - file_buffer, je32_to_cpu (node->i.totlen), je32_to_cpu (node->i.ino),
+ je32_to_cpu ( node->i.version), je32_to_cpu (node->i.isize),
+ je32_to_cpu (node->i.csize), je32_to_cpu (node->i.dsize), je32_to_cpu (node->i.offset));
+
+ crc = crc32 (0, node, sizeof (struct jffs2_raw_inode) - 8);
+ if (crc != je32_to_cpu (node->i.node_crc)) {
+ printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.node_crc), crc);
+ p += PAD(je32_to_cpu (node->i.totlen));
+ continue;
+ }
+
+ crc = crc32(0, p + sizeof (struct jffs2_raw_inode), je32_to_cpu(node->i.csize));
+ if (crc != je32_to_cpu(node->i.data_crc)) {
+ printf ("Wrong data_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.data_crc), crc);
+ p += PAD(je32_to_cpu (node->i.totlen));
+ continue;
+ }
+
+ write_inode_to_buff(node);
+
+ p += PAD(je32_to_cpu (node->i.totlen));
+ break;
+
+ case JFFS2_NODETYPE_DIRENT:
+ memcpy (name, node->d.name, node->d.nsize);
+ name [node->d.nsize] = 0x0;
+
+ if(verbose)
+ printf ("%8s Dirent node at 0x%08x, totlen 0x%08x, #pino %5d, version %5d, #ino %8d, nsize %8d, name %s\n",
+ obsolete ? "Obsolete" : "",
+ p - file_buffer, je32_to_cpu (node->d.totlen), je32_to_cpu (node->d.pino),
+ je32_to_cpu ( node->d.version), je32_to_cpu (node->d.ino),
+ node->d.nsize, name);
+
+ crc = crc32 (0, node, sizeof (struct jffs2_raw_dirent) - 8);
+ if (crc != je32_to_cpu (node->d.node_crc)) {
+ printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.node_crc), crc);
+ p += PAD(je32_to_cpu (node->d.totlen));
+ continue;
+ }
+
+ crc = crc32(0, p + sizeof (struct jffs2_raw_dirent), node->d.nsize);
+ if (crc != je32_to_cpu(node->d.name_crc)) {
+ printf ("Wrong name_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.name_crc), crc);
+ p += PAD(je32_to_cpu (node->d.totlen));
+ continue;
+ }
+
+ write_dirent_to_buff(node);
+
+ p += PAD(je32_to_cpu (node->d.totlen));
+ break;
+
+ case JFFS2_NODETYPE_CLEANMARKER:
+ if (verbose) {
+ printf ("%8s Cleanmarker at 0x%08x, totlen 0x%08x\n",
+ obsolete ? "Obsolete" : "",
+ p - file_buffer, je32_to_cpu (node->u.totlen));
+ }
+
+ if(!found_cleanmarkers){
+ found_cleanmarkers = 1;
+
+ if(add_cleanmarkers == 1 && use_input_cleanmarker_size == 1){
+ cleanmarker_size = je32_to_cpu (node->u.totlen);
+ setup_cleanmarker();
+ }
+ }
+
+ p += PAD(je32_to_cpu (node->u.totlen));
+ break;
+
+ case JFFS2_NODETYPE_PADDING:
+ if (verbose) {
+ printf ("%8s Padding node at 0x%08x, totlen 0x%08x\n",
+ obsolete ? "Obsolete" : "",
+ p - file_buffer, je32_to_cpu (node->u.totlen));
+ }
+ p += PAD(je32_to_cpu (node->u.totlen));
+ break;
+
+ case 0xffff:
+ p += 4;
+ break;
+
+ default:
+ if (verbose) {
+ printf ("%8s Unknown node at 0x%08x, totlen 0x%08x\n",
+ obsolete ? "Obsolete" : "",
+ p - file_buffer, je32_to_cpu (node->u.totlen));
+ }
+
+ //write_unknown_to_buff(node);
+
+ p += PAD(je32_to_cpu (node->u.totlen));
+ }
+ }
+}
+
+int main(int argc, char **argv) {
+
+ int ret;
+
+ process_options(argc,argv);
+
+ if ((in_fd == -1) || (out_fd == -1)) {
+
+ if(in_fd != -1)
+ close(in_fd);
+ if(out_fd != -1)
+ close(out_fd);
+
+ error_msg_and_die("You must specify input and output files!\n");
+ }
+
+ init_buffers();
+ init_sumlist();
+
+ while ((ret = load_next_block())) {
+ create_summed_image(ret);
+ }
+
+ flush_buffers();
+ clean_buffers();
+ clean_sumlist();
+
+ if (in_fd != -1)
+ close(in_fd);
+ if (out_fd != -1)
+ close(out_fd);
+
+ return 0;
+}
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-16 12:53 ` Josh Boyer
@ 2004-12-16 21:22 ` Gareth Bult (Encryptec)
2004-12-16 21:28 ` Josh Boyer
0 siblings, 1 reply; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-16 21:22 UTC (permalink / raw)
To: Josh Boyer; +Cc: Linux MTD
Hi,
> There is one more feature pending that I know of. Ferenc Havasi has
> some eraseblock summary changes that should help. I believe the
> intention was to commit that to JFFS3, but I haven't seen it go in yet.
Ok, I have these and will be trying them presently ..
just wondered if there was any way I could avoid running "patch" .. :)
Tvm
Gareth.
> josh
>
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-16 21:22 ` Gareth Bult (Encryptec)
@ 2004-12-16 21:28 ` Josh Boyer
2004-12-16 21:47 ` Gareth Bult (Encryptec)
0 siblings, 1 reply; 113+ messages in thread
From: Josh Boyer @ 2004-12-16 21:28 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Gareth Bult (Encryptec) wrote:
> Hi,
>
>>There is one more feature pending that I know of. Ferenc Havasi has
>>some eraseblock summary changes that should help. I believe the
>>intention was to commit that to JFFS3, but I haven't seen it go in yet.
>
>
> Ok, I have these and will be trying them presently ..
>
> just wondered if there was any way I could avoid running "patch" .. :)
Bug Ferenc to commit ;). I'm not sure if he subscribes to this list or not.
Also, Artem Bityuckiy is working on an inode checkpoint feature for
JFFS3. Once he gets it working, it should increase overall performance.
If you have any ideas or patches, feel free to submit them to the list.
That's what JFFS3 was cloned for, and there isn't too much concern
with backwards compatibility.
josh
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-16 21:28 ` Josh Boyer
@ 2004-12-16 21:47 ` Gareth Bult (Encryptec)
2004-12-17 12:54 ` Josh Boyer
2004-12-18 16:19 ` Jörn Engel
0 siblings, 2 replies; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-16 21:47 UTC (permalink / raw)
To: Josh Boyer; +Cc: Linux MTD
If you have any ideas or patches, feel free to submit them to the list.
> That's what JFFS3 was cloned for, and there isn't too much concern
> with backwards compatibility.
Mmm,
blkmtd.c is pretty much a rewrite, less than half the size and uses the
system buffer cache instead of reading / writing directly to the device.
This boosts performance x 3 on read x lots on write. [I get x 100
faster] (albeit you lose bad block detection on write)
If anyone's interested in the code, let me know and I'll clean it up and
post it .. however it'll be a new file and not a patch.
I've only tested it on USB key flash, but I'm assuming it'll be similar
for any block device / hard disk / whatever ..
Gareth.
On Thu, 2004-12-16 at 15:28 -0600, Josh Boyer wrote:
> Gareth Bult (Encryptec) wrote:
> > Hi,
> >
> >>There is one more feature pending that I know of. Ferenc Havasi has
> >>some eraseblock summary changes that should help. I believe the
> >>intention was to commit that to JFFS3, but I haven't seen it go in yet.
> >
> >
> > Ok, I have these and will be trying them presently ..
> >
> > just wondered if there was any way I could avoid running "patch" .. :)
>
> Bug Ferenc to commit ;). I'm not sure if he subscribes to this list or not.
>
> Also, Artem Bityuckiy is working on an inode checkpoint feature for
> JFFS3. Once he gets it working, it should increase overall performance.
>
> If you have any ideas or patches, feel free to submit them to the list.
> That's what JFFS3 was cloned for, and there isn't too much concern
> with backwards compatibility.
>
> josh
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-16 21:47 ` Gareth Bult (Encryptec)
@ 2004-12-17 12:54 ` Josh Boyer
2004-12-17 15:33 ` Gareth Bult (Encryptec)
2004-12-18 16:19 ` Jörn Engel
1 sibling, 1 reply; 113+ messages in thread
From: Josh Boyer @ 2004-12-17 12:54 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
On Thu, 2004-12-16 at 15:47, Gareth Bult (Encryptec) wrote:
>
> blkmtd.c is pretty much a rewrite, less than half the size and uses the
> system buffer cache instead of reading / writing directly to the device.
> This boosts performance x 3 on read x lots on write. [I get x 100
> faster] (albeit you lose bad block detection on write)
Hm, ok. An alternative to using the system buffer would be to make
JFFS[23] use a write buffer for those devices. It's already done on
NAND which has some special handling. The support for ECC'd NOR
probably matches more closely what you'd want to do. Take a look at
wbuf.c.
>
> If anyone's interested in the code, let me know and I'll clean it up and
> post it .. however it'll be a new file and not a patch.
Well, if it's better than the original we can always take yours and
replace what's there. Send it on, performance gains are always
welcome. I'm sure there are others interested in what you've done that
will review the code.
josh
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-17 12:54 ` Josh Boyer
@ 2004-12-17 15:33 ` Gareth Bult (Encryptec)
2004-12-17 16:02 ` Josh Boyer
0 siblings, 1 reply; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-17 15:33 UTC (permalink / raw)
To: Josh Boyer; +Cc: Linux MTD
Hi,
On Fri, 2004-12-17 at 06:54 -0600, Josh Boyer wrote:
> Hm, ok. An alternative to using the system buffer would be to make
> JFFS[23] use a write buffer for those devices.
Forgive my ignorance .. but what's the difference between using a write
buffer and using the system buffer ?
>From my pov, the system buffer is there, already 100% reliable and
somewhat larger than any [potential] write buffer ???
(Given the system buffer uses the underlying block devices read / write
routines and the block device emulator will typically sit on a block
device such as a hard drive or USB flash key .. [?])
> It's already done on
> NAND which has some special handling. The support for ECC'd NOR
> probably matches more closely what you'd want to do. Take a look at
> wbuf.c.
Mmm, after looking at wbuf.c I'm none the wiser as to what it does or
how it relates ..
> Well, if it's better than the original we can always take yours and
> replace what's there. Send it on, performance gains are always
> welcome. I'm sure there are others interested in what you've done that
> will review the code.
I think once I get this FL release out the way I'll have a a more
in-depth look at JFFS*. It seems to be written almost exclusively for
MTD devices .. which is great .. for MTD devices .. but for block
devices and flash (USB for example) presented through a standard
interface (SCSI for example) it seems there is scope for a much
"thinner" filesystem generic devices with a subset of JFFS's feature
set.
Gareth.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-17 15:33 ` Gareth Bult (Encryptec)
@ 2004-12-17 16:02 ` Josh Boyer
2004-12-17 16:46 ` Gareth Bult (Encryptec)
0 siblings, 1 reply; 113+ messages in thread
From: Josh Boyer @ 2004-12-17 16:02 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Gareth Bult (Encryptec) wrote:>
>
> Forgive my ignorance .. but what's the difference between using a write
> buffer and using the system buffer ?
>
>>From my pov, the system buffer is there, already 100% reliable and
> somewhat larger than any [potential] write buffer ???
Perhaps alternative was the wrong word to use on my part. I'll just
wait until I see your code. I'm probably confusing myself by making
assumptions on what it's doing :).
>
> Mmm, after looking at wbuf.c I'm none the wiser as to what it does or
> how it relates ..
It might not relate. IIRC there were some discussions on buffering the
writes done by JFFS[23] to the size of a sector on the block device so
that partial writes aren't done unnecessarily. This is what the wbuf.c
file does. NAND and ECC'd NOR require this because you can only write
to a page once (usually) before an erase is needed to write again.
For disk devices I don't think it's required, but it could help. Or
maybe your code does this at the MTD level. I dunno. I could be
spouting nonsense again ;).
>
>
> I think once I get this FL release out the way I'll have a a more
> in-depth look at JFFS*. It seems to be written almost exclusively for
> MTD devices .. which is great .. for MTD devices .. but for block
> devices and flash (USB for example) presented through a standard
> interface (SCSI for example) it seems there is scope for a much
> "thinner" filesystem generic devices with a subset of JFFS's feature
> set.
Yes, it is a Journaled _Flash_ File System. So of course it's written
for MTD devices. Not sure why blkmtd.c was originally written. Test
vehicle for JFFS2, "why not", "I'm bored", etc. Who knows? :)
josh
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-17 16:02 ` Josh Boyer
@ 2004-12-17 16:46 ` Gareth Bult (Encryptec)
2004-12-17 17:08 ` Artem B. Bityuckiy
2004-12-17 17:10 ` Josh Boyer
0 siblings, 2 replies; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-17 16:46 UTC (permalink / raw)
To: Josh Boyer; +Cc: Linux MTD
On Fri, 2004-12-17 at 10:02 -0600, Josh Boyer wrote:
> For disk devices I don't think it's required, but it could help. Or
> maybe your code does this at the MTD level. I dunno. I could be
> spouting nonsense again ;).
Mmm, as long as writes are aligned on 2k pages and are linear /
unidirectional, I'm not sure anything else is really a problem
for USB flash drives ..
> Yes, it is a Journaled _Flash_ File System. So of course it's written
> for MTD devices. Not sure why blkmtd.c was originally written. Test
> vehicle for JFFS2, "why not", "I'm bored", etc. Who knows? :)
Mm, my point exactly , Journaled _Flash_ File System - as opposed to
Journaled _MTD_ _Flash_ File System .. !!
USB flash is still flash! - just presented via a generic interface as
opposed to an embedded one .. (AFAIK)
I guess I was sort of hoping blkmtd.c was written to enable JFFS2 to be
more generic and embedded device independent. I suspect however it was
written purely for testing purposes .. :(
As I understand it, the Flash in USB keys is identical to the MTD type
embedded devices, except that the USB keys always come with
micro-controllers that handle and optimise read/write/erase operations
and present them via a generic PC/USB based disk interface. I'm not sure
how fast MTD can be driven, but key manufacturers seem to think that
keys should be able to run at 20Mb/sec read and 10Mb/sec write [for
large block read/writes] which is much faster than is required for a key
to replace a hard disk in a workstation.
I'm thinking that JFFS2 could draw in a huge additional user base if it
catered for (or at least supported) such devices .. (!)
Gareth.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-17 16:46 ` Gareth Bult (Encryptec)
@ 2004-12-17 17:08 ` Artem B. Bityuckiy
2004-12-17 17:10 ` Josh Boyer
1 sibling, 0 replies; 113+ messages in thread
From: Artem B. Bityuckiy @ 2004-12-17 17:08 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
On Fri, 17 Dec 2004, Gareth Bult (Encryptec) wrote:
> On Fri, 2004-12-17 at 10:02 -0600, Josh Boyer wrote:
> > For disk devices I don't think it's required, but it could help. Or
> > maybe your code does this at the MTD level. I dunno. I could be
> > spouting nonsense again ;).
>
> Mmm, as long as writes are aligned on 2k pages and are linear /
> unidirectional, I'm not sure anything else is really a problem
> for USB flash drives ..
>
> > Yes, it is a Journaled _Flash_ File System. So of course it's written
> > for MTD devices. Not sure why blkmtd.c was originally written. Test
> > vehicle for JFFS2, "why not", "I'm bored", etc. Who knows? :)
>
> Mm, my point exactly , Journaled _Flash_ File System - as opposed to
> Journaled _MTD_ _Flash_ File System .. !!
>
> USB flash is still flash! - just presented via a generic interface as
> opposed to an embedded one .. (AFAIK)
>
> I guess I was sort of hoping blkmtd.c was written to enable JFFS2 to be
> more generic and embedded device independent. I suspect however it was
> written purely for testing purposes .. :(
>
> As I understand it, the Flash in USB keys is identical to the MTD type
> embedded devices, except that the USB keys always come with
> micro-controllers that handle and optimise read/write/erase operations
> and present them via a generic PC/USB based disk interface. I'm not sure
> how fast MTD can be driven, but key manufacturers seem to think that
> keys should be able to run at 20Mb/sec read and 10Mb/sec write [for
> large block read/writes] which is much faster than is required for a key
> to replace a hard disk in a workstation.
>
> I'm thinking that JFFS2 could draw in a huge additional user base if it
> catered for (or at least supported) such devices .. (!
My understanding was that it is only seen as SCSI block device from
outside. E.g. it is not possible, say, to erase sector, to write one
page, etc. If so, just foget about JFFS2 on that, IMHO.
>
> Gareth.
>
>
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
>
--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-17 16:46 ` Gareth Bult (Encryptec)
2004-12-17 17:08 ` Artem B. Bityuckiy
@ 2004-12-17 17:10 ` Josh Boyer
2004-12-17 17:26 ` Gareth Bult (Encryptec)
1 sibling, 1 reply; 113+ messages in thread
From: Josh Boyer @ 2004-12-17 17:10 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Gareth Bult (Encryptec) wrote:
>
> Mm, my point exactly , Journaled _Flash_ File System - as opposed to
> Journaled _MTD_ _Flash_ File System .. !!
>
> USB flash is still flash! - just presented via a generic interface as
> opposed to an embedded one .. (AFAIK)
No, it's really not. There is no access to the low level flash, so MTD
cannot directly drive it. Same principle as the Compact Flash devices,
which MTD doesn't deal with either (other than via blkmtd).
>
> I guess I was sort of hoping blkmtd.c was written to enable JFFS2 to be
> more generic and embedded device independent. I suspect however it was
> written purely for testing purposes .. :(
Yes, most likely.
>
> As I understand it, the Flash in USB keys is identical to the MTD type
> embedded devices, except that the USB keys always come with
> micro-controllers that handle and optimise read/write/erase operations
> and present them via a generic PC/USB based disk interface. I'm not sure
> how fast MTD can be driven, but key manufacturers seem to think that
> keys should be able to run at 20Mb/sec read and 10Mb/sec write [for
> large block read/writes] which is much faster than is required for a key
> to replace a hard disk in a workstation.
The flash used in the devices is identical, but MTD has no access to it.
>
> I'm thinking that JFFS2 could draw in a huge additional user base if it
> catered for (or at least supported) such devices .. (!)
That's the problem. There is no filesystem for these "abominations"
really. They pretend to be IDE, SCSI, etc. but are really flash
underneath. Neither the MTD driver or the IDE drivers (for example)
deal with them really well, but I'm not sure that they can.
Did you try ext3 on the device with a native driver (IDE, SCSI, USB
specific) instead of via blkmtd.c?
josh
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-17 17:10 ` Josh Boyer
@ 2004-12-17 17:26 ` Gareth Bult (Encryptec)
2004-12-17 17:35 ` Josh Boyer
0 siblings, 1 reply; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-17 17:26 UTC (permalink / raw)
To: Josh Boyer; +Cc: Linux MTD
On Fri, 2004-12-17 at 11:10 -0600, Josh Boyer wrote:
> That's the problem. There is no filesystem for these "abominations"
> really. They pretend to be IDE, SCSI, etc. but are really flash
> underneath. Neither the MTD driver or the IDE drivers (for example)
> deal with them really well, but I'm not sure that they can.
>
> Did you try ext3 on the device with a native driver (IDE, SCSI, USB
> specific) instead of via blkmtd.c?
Sure, ext2 and ext3 work just fine .. however .. as far as I'm aware,
although ext3 is log structured, it's not optimised to run in a linear
fashion and doesn't worry about repeatedly writing to the same block ..
which is likely to cause the USB flash controller to execute an erase
prior to write .. which will quickly wear down the device [?!]
(Am I wrong? in which case why isn't everyone using ext3 instead of
JFFS?)
The other requirement of course is inline compression, which last time I
looked ext3 didn't do out of the box (?)
I'm going to deliberately wear down some blocks on a key a little later
(at block level) to see what the OS makes of it ... (and whether it
takes 100,000 or 1,000,000 cycles .. I see conflicting specs ..
I guess the basic spec for a USB flash key (you may call them
abominations, but you should see the sales stats!) would be;
a. purely log structured fs with linear / circular operation
b. write blocks aligned to 2k (or 4k might be smarter to fit with buffer
page sizes)
c. erase blocks @ 128k
d. inline compression
Strikes me JFFS2 does all this, possibly bar the generic device
support ..
Gareth.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-17 17:26 ` Gareth Bult (Encryptec)
@ 2004-12-17 17:35 ` Josh Boyer
2004-12-17 18:09 ` Gareth Bult (Encryptec)
2004-12-18 16:02 ` Jörn Engel
0 siblings, 2 replies; 113+ messages in thread
From: Josh Boyer @ 2004-12-17 17:35 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Gareth Bult (Encryptec) wrote:
>
>
> Sure, ext2 and ext3 work just fine .. however .. as far as I'm aware,
> although ext3 is log structured, it's not optimised to run in a linear
> fashion and doesn't worry about repeatedly writing to the same block ..
> which is likely to cause the USB flash controller to execute an erase
> prior to write .. which will quickly wear down the device [?!]
>
> (Am I wrong? in which case why isn't everyone using ext3 instead of
> JFFS?)
Most of the controllers for these devices do the wear leveling
themselves. At least that was my understanding.
We don't deal much with these devices, so I'm not sure what everyone
else uses. Alot of the CF devices are FAT formatted to work with
Windows, etc.
>
> The other requirement of course is inline compression, which last time I
> looked ext3 didn't do out of the box (?)
No, it doesn't. There are patches for ext2 to make it use compression
(e2compr), but then you lose the journaling.
>
> I'm going to deliberately wear down some blocks on a key a little later
> (at block level) to see what the OS makes of it ... (and whether it
> takes 100,000 or 1,000,000 cycles .. I see conflicting specs ..
That probably depends on the actual flash chips used within the device.
Not all of them use the same type of flash.
>
> I guess the basic spec for a USB flash key (you may call them
> abominations, but you should see the sales stats!) would be;
Hey, they certainly are handy. I'm not arguing with that :).
>
> a. purely log structured fs with linear / circular operation
> b. write blocks aligned to 2k (or 4k might be smarter to fit with buffer
> page sizes)
> c. erase blocks @ 128k
> d. inline compression
Again, most controllers do all the wear leveling, erasing for you. So
you are left with compression, right?
And that is certainly a lacking thing. A block-based, writeable,
compressed filesystem that does journalling. Sounds like a good
opportunity to me :).
josh
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-17 17:35 ` Josh Boyer
@ 2004-12-17 18:09 ` Gareth Bult (Encryptec)
2004-12-17 19:14 ` jasmine
2004-12-18 16:02 ` Jörn Engel
1 sibling, 1 reply; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-17 18:09 UTC (permalink / raw)
To: Josh Boyer; +Cc: Linux MTD
On Fri, 2004-12-17 at 11:35 -0600, Josh Boyer wrote:
> Gareth Bult (Encryptec) wrote:
> Again, most controllers do all the wear leveling, erasing for you. So
> you are left with compression, right?
>
> And that is certainly a lacking thing. A block-based, writeable,
> compressed filesystem that does journalling. Sounds like a good
> opportunity to me :).
Mmm, sounds a bit like JFFS2 on a block emulator , yeah ?
;-)
Gareth.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-17 18:09 ` Gareth Bult (Encryptec)
@ 2004-12-17 19:14 ` jasmine
2004-12-17 20:55 ` Gareth Bult (Encryptec)
0 siblings, 1 reply; 113+ messages in thread
From: jasmine @ 2004-12-17 19:14 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
On Fri, 17 Dec 2004, Gareth Bult (Encryptec) wrote:
> Mmm, sounds a bit like JFFS2 on a block emulator , yeah ?
Only if you actively enjoy wasting processor cycles (and microjoules) and
ROM footprint...
-J.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-17 19:14 ` jasmine
@ 2004-12-17 20:55 ` Gareth Bult (Encryptec)
0 siblings, 0 replies; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-17 20:55 UTC (permalink / raw)
To: jasmine; +Cc: Linux MTD
On Fri, 2004-12-17 at 19:14 +0000, jasmine@linuxgrrls.org wrote:
> On Fri, 17 Dec 2004, Gareth Bult (Encryptec) wrote:
>
> > Mmm, sounds a bit like JFFS2 on a block emulator , yeah ?
>
> Only if you actively enjoy wasting processor cycles (and microjoules) and
> ROM footprint...
Processor cycles, got em to spare ..
Microjoules coming out of my ears ..
ROM footprint .. don't use 'em ..
What's the alternative then .. :)
Gareth.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-17 17:35 ` Josh Boyer
2004-12-17 18:09 ` Gareth Bult (Encryptec)
@ 2004-12-18 16:02 ` Jörn Engel
2004-12-20 16:34 ` Josh Boyer
1 sibling, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-18 16:02 UTC (permalink / raw)
To: Josh Boyer; +Cc: Linux MTD, Gareth Bult (Encryptec)
On Fri, 17 December 2004 11:35:04 -0600, Josh Boyer wrote:
>
> Again, most controllers do all the wear leveling, erasing for you. So
> you are left with compression, right?
If you have enough trust in your El Cheapo hardware supplier. There
is no standard that requires wear leveling and there are reports that
basically tell you to do your own wear leveling on some devices. With
no good way to distinguish "some" devices from "others", you can read
that as "all". ;)
USB keys and all the various other consumer-type flash devices are
good enough to beat 3 1/2" floppies, that's about it. But that's
quite good enough for most people. Floppies sold like crazy after
all.
Jörn
--
Simplicity is prerequisite for reliability.
-- Edsger W. Dijkstra
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-16 21:47 ` Gareth Bult (Encryptec)
2004-12-17 12:54 ` Josh Boyer
@ 2004-12-18 16:19 ` Jörn Engel
2004-12-18 17:32 ` Gareth Bult (Encryptec)
1 sibling, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-18 16:19 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
On Thu, 16 December 2004 21:47:49 +0000, Gareth Bult (Encryptec) wrote:
>
> blkmtd.c is pretty much a rewrite, less than half the size and uses the
> system buffer cache instead of reading / writing directly to the device.
> This boosts performance x 3 on read x lots on write. [I get x 100
> faster] (albeit you lose bad block detection on write)
>
> If anyone's interested in the code, let me know and I'll clean it up and
> post it .. however it'll be a new file and not a patch.
Sure am! If the code lives up to your promises I'd like to read it
(might learn something new) and possibly use it sometime soon.
Jörn
--
The competent programmer is fully aware of the strictly limited size of
his own skull; therefore he approaches the programming task in full
humility, and among other things he avoids clever tricks like the plague.
-- Edsger W. Dijkstra
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-18 16:19 ` Jörn Engel
@ 2004-12-18 17:32 ` Gareth Bult (Encryptec)
2004-12-18 17:52 ` Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-18 17:32 UTC (permalink / raw)
To: Jörn Engel; +Cc: Linux MTD
[-- Attachment #1: Type: text/plain, Size: 1098 bytes --]
Ok,
There's a public health warning attached to this code ..
I've not cleaned it up and I've not added any comments ..
(!)
fyi; I don't use mkfs.jffs2 .. I flash_eraseall the device then work
from there with "tar".. there seems to be no benefit [for me] to using
mkfs and it seems a little more reliant on the device size than the
erase program.
Gareth.
On Sat, 2004-12-18 at 17:19 +0100, Jörn Engel wrote:
> On Thu, 16 December 2004 21:47:49 +0000, Gareth Bult (Encryptec) wrote:
> >
> > blkmtd.c is pretty much a rewrite, less than half the size and uses the
> > system buffer cache instead of reading / writing directly to the device.
> > This boosts performance x 3 on read x lots on write. [I get x 100
> > faster] (albeit you lose bad block detection on write)
> >
> > If anyone's interested in the code, let me know and I'll clean it up and
> > post it .. however it'll be a new file and not a patch.
>
> Sure am! If the code lives up to your promises I'd like to read it
> (might learn something new) and possibly use it sometime soon.
>
> Jörn
>
[-- Attachment #2: blkmtd.c --]
[-- Type: text/x-csrc, Size: 16190 bytes --]
/*
* $Id: blkmtd.c,v 1.23 2004/08/09 14:03:19 dwmw2 Exp $
*
* blkmtd.c - use a block device as a fake MTD
*
* Author: Simon Evans <spse@secret.org.uk>
*
* Copyright (C) 2001,2002 Simon Evans
*
* Licence: GPL
*
* How it works:
* The driver uses raw/io to read/write the device and the page
* cache to cache access. Writes update the page cache with the
* new data and mark it dirty and add the page into a BIO which
* is then written out.
*
* It can be loaded Read-Only to prevent erases and writes to the
* medium.
*
*/
#include <linux/config.h>
#include <linux/module.h>
#include <linux/fs.h>
#include <linux/blkdev.h>
#include <linux/bio.h>
#include <linux/pagemap.h>
#include <linux/list.h>
#include <linux/init.h>
#include <linux/mtd/mtd.h>
#include <linux/buffer_head.h>
#define err(format, arg...) printk(KERN_ERR "blkmtd: " format "\n" , ## arg)
#define info(format, arg...) printk(KERN_INFO "blkmtd: " format "\n" , ## arg)
#define warn(format, arg...) printk(KERN_WARNING "blkmtd: " format "\n" , ## arg)
#define crit(format, arg...) printk(KERN_CRIT "blkmtd: " format "\n" , ## arg)
/* Default erase size in K, always make it a multiple of PAGE_SIZE */
#define CONFIG_MTD_BLKDEV_ERASESIZE (128 << 10) /* 128KiB */
#define VERSION "$Revision: 1.23 $"
/* Info for the block device */
struct blkmtd_dev {
struct list_head list;
struct block_device *blkdev;
struct mtd_info mtd_info;
struct semaphore wrbuf_mutex;
};
/* Static info about the MTD, used in cleanup_module */
static LIST_HEAD(blkmtd_device_list);
static void blkmtd_sync(struct mtd_info *mtd);
#define MAX_DEVICES 4
/* Module parameters passed by insmod/modprobe */
char *device[MAX_DEVICES]; /* the block device to use */
int erasesz[MAX_DEVICES]; /* optional default erase size */
int ro[MAX_DEVICES]; /* optional read only flag */
int sync;
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Simon Evans <spse@secret.org.uk>");
MODULE_DESCRIPTION("Emulate an MTD using a block device");
MODULE_PARM(device, "1-4s");
MODULE_PARM_DESC(device, "block device to use");
MODULE_PARM(erasesz, "1-4i");
MODULE_PARM_DESC(erasesz, "optional erase size to use in KiB. eg 4=4KiB.");
MODULE_PARM(ro, "1-4i");
MODULE_PARM_DESC(ro, "1=Read only, writes and erases cause errors");
MODULE_PARM(sync, "i");
MODULE_PARM_DESC(sync, "1=Synchronous writes");
#define PAGE_READAHEAD 64
void cache_readahead(struct address_space *mapping, int index)
{
filler_t *filler = (filler_t*)(mapping->a_ops->readpage);
int i,pagei;
unsigned ret = 0;
unsigned long end_index;
struct page *page;
LIST_HEAD(page_pool);
struct inode *inode = mapping->host;
loff_t isize = i_size_read(inode);
if(!isize) {
printk(KERN_INFO "iSize=0 in cache_readahead\n");
return;
}
end_index = ((isize - 1) >> PAGE_CACHE_SHIFT);
spin_lock_irq(&mapping->tree_lock);
for(i=0;i<PAGE_READAHEAD;i++) {
pagei=index+i;
if (pagei > end_index) {
printk(KERN_INFO "Overrun end of disk in cache readahead\n");
break;
}
page = radix_tree_lookup(&mapping->page_tree, pagei);
if(page && (!i)) break;
if(page) continue;
spin_unlock_irq(&mapping->tree_lock);
page = page_cache_alloc_cold(mapping);
spin_lock_irq(&mapping->tree_lock);
if(!page) break;
page->index=pagei;
list_add(&page->lru, &page_pool);
ret++;
}
spin_unlock_irq(&mapping->tree_lock);
if(ret) {
read_cache_pages(mapping, &page_pool, filler, NULL);
//printk(KERN_INFO "Readahead [%d/%d] pages from [%d]\n",ret,PAGE_READAHEAD,index);
}
}
static struct page* page_readahead(struct address_space *mapping, int index)
{
filler_t *filler = (filler_t*)(mapping->a_ops->readpage);
cache_readahead(mapping,index);
return read_cache_page(mapping, index, filler, NULL);
}
/**
* write_pages - write block of data to device via the page cache
* @dev: device to write to
* @buf: data source or NULL if erase (output is set to 0xff)
* @to: offset into output device
* @len: amount to data to write
* @retlen: amount of data written
*
* Grab pages from the page cache and fill them with the source data.
* Non page aligned start and end result in a readin of the page and
* part of the page being modified. Pages are added to the bio and then written
* out.
*/
static int write_pages(struct blkmtd_dev *dev, const u_char *buf, loff_t to, size_t len, size_t *retlen)
{
struct page *page;
struct address_space *mapping = dev->blkdev->bd_inode->i_mapping;
//
int err = 0; // return status
int index = to >> PAGE_SHIFT; // page index
int offset = to & ~PAGE_MASK; // page offset
//
static int my_read()
{
err=0;
page = page_readahead(mapping,index);
if( unlikely(!page) ) {
crit("unable to read page (%d) in write_pages\n",index);
err=-ENOMEM;
}
//if(IS_ERR(page)) err=-EIO;
if(err) printk("READ ERROR\n");
return err;
}
//
static int my_erase()
{
int pages = len >> PAGE_SHIFT;
unsigned char *p;
unsigned char *max;
int format;
while(pages) {
//printk(KERN_INFO "readahead on index (%d)\n",index);
if(my_read()) return err;
max = ((char*)page_address(page)+PAGE_SIZE);
format = 0;
for(p=(unsigned char*)page_address(page); p<max; p++)
if(*p != 0xff) {
format=1;
break;
}
if(format) {
//DEBUG(3, "write_pages: erasing\n");
lock_page(page);
memset(page_address(page), 0xff, PAGE_SIZE);
set_page_dirty(page);
unlock_page(page);
}
page_cache_release(page);
pages--;
index++;
}
*retlen = len;
return 0;
}
static int my_write()
{
int cpylen;
if(retlen) *retlen = 0;
//printk(KERN_INFO "Write req: %d\n",len);
while(len) {
if((offset+len) > PAGE_SIZE)
cpylen = PAGE_SIZE - offset; // multiple pages
else cpylen = len; // this page
len = len - cpylen;
//
// Get page
//
if(my_read()) return err;
//
if(memcmp(page_address(page)+offset,buf,cpylen)) {
lock_page(page);
memcpy(page_address(page) + offset,buf, cpylen);
set_page_dirty(page);
unlock_page(page);
}
page_cache_release(page);
if(retlen) *retlen += cpylen;
//
buf+=cpylen;
offset = 0;
index++;
//
}
//printk(KERN_INFO "Write act: %d\n",*retlen);
return 0;
}
err=0;
down(&dev->wrbuf_mutex);
err = buf ? my_write() : my_erase();
up(&dev->wrbuf_mutex);
return err;
}
/* erase a specified part of the device */
static int blkmtd_erase(struct mtd_info *mtd, struct erase_info *instr)
{
struct blkmtd_dev *dev = mtd->priv;
struct mtd_erase_region_info *einfo = mtd->eraseregions;
int numregions = mtd->numeraseregions;
size_t from;
u_long len;
int err = -EIO;
size_t retlen;
instr->state = MTD_ERASING;
from = instr->addr;
len = instr->len;
/* check erase region has valid start and length */
//DEBUG(2, "blkmtd: erase: dev = `%s' from = 0x%zx len = 0x%lx\n", mtd->name+9, from, len);
while(numregions) {
DEBUG(3, "blkmtd: checking erase region = 0x%08X size = 0x%X num = 0x%x\n",
einfo->offset, einfo->erasesize, einfo->numblocks);
if(from >= einfo->offset
&& from < einfo->offset + (einfo->erasesize * einfo->numblocks)) {
if(len == einfo->erasesize
&& ( (from - einfo->offset) % einfo->erasesize == 0))
break;
}
numregions--;
einfo++;
}
if(!numregions) {
/* Not a valid erase block */
err("erase: invalid erase request 0x%lX @ 0x%08zX", len, from);
instr->state = MTD_ERASE_FAILED;
err = -EIO;
}
if(instr->state != MTD_ERASE_FAILED) {
/* do the erase */
DEBUG(3, "Doing erase from = %zd len = %ld\n", from, len);
err = write_pages(dev, NULL, from, len, &retlen);
if(err || retlen != len) {
err("erase failed err = %d", err);
instr->state = MTD_ERASE_FAILED;
} else {
instr->state = MTD_ERASE_DONE;
}
}
DEBUG(3, "blkmtd: erase: checking callback\n");
mtd_erase_callback(instr);
DEBUG(2, "blkmtd: erase: finished (err = %d)\n", err);
return err;
}
static int blkmtd_read(struct mtd_info *mtd, loff_t from, size_t len, size_t *retlen, u_char *buf)
{
struct blkmtd_dev *dev = mtd->priv;
struct page *page;
//
int index = from >> PAGE_SHIFT;
int offset = from - (index << PAGE_SHIFT);
int cpylen;
if(from > mtd->size) return -EINVAL;
if(from + len > mtd->size) len = mtd->size - from;
if(retlen) *retlen = 0;
while(len) {
if((offset+len) > PAGE_SIZE)
cpylen = PAGE_SIZE - offset; // multiple pages
else cpylen = len; // this page
len = len - cpylen;
//
// Get page
//
page = page_readahead(dev->blkdev->bd_inode->i_mapping,index);
if(!page) return -ENOMEM;
if(IS_ERR(page)) return -EIO;
//
memcpy(buf, page_address(page) + offset, cpylen);
page_cache_release(page);
//
if(retlen) *retlen += cpylen;
buf+=cpylen;
offset = 0;
index++;
//
}
return 0;
}
/* write data to the underlying device */
static int blkmtd_write(struct mtd_info *mtd, loff_t to, size_t len,
size_t *retlen, const u_char *buf)
{
struct blkmtd_dev *dev = mtd->priv;
int err;
if(!len) return 0;
if(to >= mtd->size) return -ENOSPC;
if(to + len > mtd->size) len = mtd->size - to;
err = write_pages(dev, buf, to, len, retlen);
if(err > 0) err = 0;
return err;
}
/* sync the device - wait until the write queue is empty */
static void blkmtd_sync(struct mtd_info *mtd)
{
sync_blockdev(((struct blkmtd_dev*)mtd->priv)->blkdev);
return;
}
static void free_device(struct blkmtd_dev *dev)
{
DEBUG(2, "blkmtd: free_device() dev = %p\n", dev);
if(dev) {
if(dev->mtd_info.eraseregions)
kfree(dev->mtd_info.eraseregions);
if(dev->mtd_info.name)
kfree(dev->mtd_info.name);
if(dev->blkdev) {
invalidate_inode_pages(dev->blkdev->bd_inode->i_mapping);
close_bdev_excl(dev->blkdev);
}
kfree(dev);
}
}
/* For a given size and initial erase size, calculate the number
* and size of each erase region. Goes round the loop twice,
* once to find out how many regions, then allocates space,
* then round the loop again to fill it in.
*/
static struct mtd_erase_region_info *calc_erase_regions(
size_t erase_size, size_t total_size, int *regions)
{
struct mtd_erase_region_info *info = NULL;
DEBUG(2, "calc_erase_regions, es = %zd size = %zd regions = %d\n",
erase_size, total_size, *regions);
/* Make any user specified erasesize be a power of 2
and at least PAGE_SIZE */
if(erase_size) {
int es = erase_size;
erase_size = 1;
while(es != 1) {
es >>= 1;
erase_size <<= 1;
}
if(erase_size < PAGE_SIZE)
erase_size = PAGE_SIZE;
} else {
erase_size = CONFIG_MTD_BLKDEV_ERASESIZE;
}
*regions = 0;
do {
int tot_size = total_size;
int er_size = erase_size;
int count = 0, offset = 0, regcnt = 0;
while(tot_size) {
count = tot_size / er_size;
if(count) {
tot_size = tot_size % er_size;
if(info) {
DEBUG(2, "adding to erase info off=%d er=%d cnt=%d\n",
offset, er_size, count);
(info+regcnt)->offset = offset;
(info+regcnt)->erasesize = er_size;
(info+regcnt)->numblocks = count;
(*regions)++;
}
regcnt++;
offset += (count * er_size);
}
while(er_size > tot_size)
er_size >>= 1;
}
if(info == NULL) {
info = kmalloc(regcnt * sizeof(struct mtd_erase_region_info), GFP_KERNEL);
if(!info)
break;
}
} while(!(*regions));
DEBUG(2, "calc_erase_regions done, es = %zd size = %zd regions = %d\n",
erase_size, total_size, *regions);
return info;
}
extern dev_t __init name_to_dev_t(const char *line);
static struct blkmtd_dev *add_device(char *devname, int readonly, int erase_size)
{
struct block_device *bdev;
int mode;
struct blkmtd_dev *dev;
if(!devname)
return NULL;
/* Get a handle on the device */
#ifdef MODULE
mode = (readonly) ? O_RDONLY : O_RDWR;
bdev = open_bdev_excl(devname, mode, NULL);
#else
mode = (readonly) ? FMODE_READ : FMODE_WRITE;
bdev = open_by_devnum(name_to_dev_t(devname), mode);
#endif
if(IS_ERR(bdev)) {
err("error: cannot open device %s", devname);
DEBUG(2, "blkmtd: opening bdev returned %ld\n", PTR_ERR(bdev));
return NULL;
}
DEBUG(1, "blkmtd: found a block device major = %d, minor = %d\n",
MAJOR(bdev->bd_dev), MINOR(bdev->bd_dev));
if(MAJOR(bdev->bd_dev) == MTD_BLOCK_MAJOR) {
err("attempting to use an MTD device as a block device");
blkdev_put(bdev);
return NULL;
}
dev = kmalloc(sizeof(struct blkmtd_dev), GFP_KERNEL);
if(dev == NULL) {
blkdev_put(bdev);
return NULL;
}
memset(dev, 0, sizeof(struct blkmtd_dev));
dev->blkdev = bdev;
atomic_set(&(dev->blkdev->bd_inode->i_mapping->truncate_count), 0);
if(!readonly) {
init_MUTEX(&dev->wrbuf_mutex);
}
dev->mtd_info.size = dev->blkdev->bd_inode->i_size & PAGE_MASK;
/* Setup the MTD structure */
/* make the name contain the block device in */
dev->mtd_info.name = kmalloc(sizeof("blkmtd: ") + strlen(devname), GFP_KERNEL);
if(dev->mtd_info.name == NULL)
goto devinit_err;
sprintf(dev->mtd_info.name, "blkmtd: %s", devname);
dev->mtd_info.eraseregions = calc_erase_regions(erase_size, dev->mtd_info.size,
&dev->mtd_info.numeraseregions);
if(dev->mtd_info.eraseregions == NULL)
goto devinit_err;
dev->mtd_info.erasesize = dev->mtd_info.eraseregions->erasesize;
DEBUG(1, "blkmtd: init: found %d erase regions\n",
dev->mtd_info.numeraseregions);
if(readonly) {
dev->mtd_info.type = MTD_ROM;
dev->mtd_info.flags = MTD_CAP_ROM;
} else {
dev->mtd_info.type = MTD_RAM;
dev->mtd_info.flags = MTD_CAP_RAM;
dev->mtd_info.erase = blkmtd_erase;
dev->mtd_info.write = blkmtd_write;
dev->mtd_info.writev = default_mtd_writev;
dev->mtd_info.sync = blkmtd_sync;
}
dev->mtd_info.read = blkmtd_read;
dev->mtd_info.readv = default_mtd_readv;
dev->mtd_info.priv = dev;
dev->mtd_info.owner = THIS_MODULE;
list_add(&dev->list, &blkmtd_device_list);
if (add_mtd_device(&dev->mtd_info)) {
/* Device didnt get added, so free the entry */
list_del(&dev->list);
goto devinit_err;
} else {
info("mtd%d: [%s] erase_size = %dKiB %s [%ld]",
dev->mtd_info.index, dev->mtd_info.name + strlen("blkmtd: "),
dev->mtd_info.erasesize >> 10,
readonly ? "(read-only)" : "", PAGE_SIZE);
}
return dev;
devinit_err:
free_device(dev);
return NULL;
}
/* Cleanup and exit - sync the device and kill of the kernel thread */
static void __devexit cleanup_blkmtd(void)
{
struct list_head *temp1, *temp2;
/* Remove the MTD devices */
list_for_each_safe(temp1, temp2, &blkmtd_device_list) {
struct blkmtd_dev *dev = list_entry(temp1, struct blkmtd_dev,
list);
blkmtd_sync(&dev->mtd_info);
del_mtd_device(&dev->mtd_info);
info("mtd%d: [%s] removed", dev->mtd_info.index,
dev->mtd_info.name + strlen("blkmtd: "));
list_del(&dev->list);
free_device(dev);
}
}
#ifndef MODULE
/* Handle kernel boot params */
static int __init param_blkmtd_device(char *str)
{
int i;
for(i = 0; i < MAX_DEVICES; i++) {
device[i] = str;
DEBUG(2, "blkmtd: device setup: %d = %s\n", i, device[i]);
strsep(&str, ",");
}
return 1;
}
static int __init param_blkmtd_erasesz(char *str)
{
int i;
for(i = 0; i < MAX_DEVICES; i++) {
char *val = strsep(&str, ",");
if(val)
erasesz[i] = simple_strtoul(val, NULL, 0);
DEBUG(2, "blkmtd: erasesz setup: %d = %d\n", i, erasesz[i]);
}
return 1;
}
static int __init param_blkmtd_ro(char *str)
{
int i;
for(i = 0; i < MAX_DEVICES; i++) {
char *val = strsep(&str, ",");
if(val)
ro[i] = simple_strtoul(val, NULL, 0);
DEBUG(2, "blkmtd: ro setup: %d = %d\n", i, ro[i]);
}
return 1;
}
static int __init param_blkmtd_sync(char *str)
{
if(str[0] == '1')
sync = 1;
return 1;
}
__setup("blkmtd_device=", param_blkmtd_device);
__setup("blkmtd_erasesz=", param_blkmtd_erasesz);
__setup("blkmtd_ro=", param_blkmtd_ro);
__setup("blkmtd_sync=", param_blkmtd_sync);
#endif
/* Startup */
static int __init init_blkmtd(void)
{
int i;
info("version " VERSION);
/* Check args - device[0] is the bare minimum*/
if(!device[0]) {
err("error: missing `device' name\n");
return -EINVAL;
}
for(i = 0; i < MAX_DEVICES; i++)
add_device(device[i], ro[i], erasesz[i] << 10);
if(list_empty(&blkmtd_device_list))
return -EINVAL;
return 0;
}
module_init(init_blkmtd);
module_exit(cleanup_blkmtd);
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-18 17:32 ` Gareth Bult (Encryptec)
@ 2004-12-18 17:52 ` Jörn Engel
2004-12-18 18:11 ` Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-18 17:52 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
On Sat, 18 December 2004 17:32:35 +0000, Gareth Bult (Encryptec) wrote:
>
> There's a public health warning attached to this code ..
>
> I've not cleaned it up and I've not added any comments ..
I'll wear protective gloves then. Thanks!
> fyi; I don't use mkfs.jffs2 .. I flash_eraseall the device then work
> from there with "tar".. there seems to be no benefit [for me] to using
> mkfs and it seems a little more reliant on the device size than the
> erase program.
Makes a lot of sense. Both approaches have some advantages.
Jörn
--
The story so far:
In the beginning the Universe was created. This has made a lot
of people very angry and been widely regarded as a bad move.
-- Douglas Adams?
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-18 17:52 ` Jörn Engel
@ 2004-12-18 18:11 ` Jörn Engel
2004-12-18 20:48 ` Gareth Bult (Encryptec)
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-18 18:11 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
On Sat, 18 December 2004 18:52:22 +0100, Jörn Engel wrote:
> On Sat, 18 December 2004 17:32:35 +0000, Gareth Bult (Encryptec) wrote:
> >
> > There's a public health warning attached to this code ..
> >
> > I've not cleaned it up and I've not added any comments ..
>
> I'll wear protective gloves then. Thanks!
Not a pretty. Looks like quite a few cleanups from phram.c can be
applied to this one as well. From a first glance:
o Coding style (Lindent?)
o Function length
o #ifdef MODULE
o parameters have an insane interface
o ro-devices. the same can be achieved with chmod.
But you started the work, which is a good thing.
Jörn
--
Public Domain - Free as in Beer
General Public - Free as in Speech
BSD License - Free as in Enterprise
Shared Source - Free as in "Work will make you..."
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-18 18:11 ` Jörn Engel
@ 2004-12-18 20:48 ` Gareth Bult (Encryptec)
2004-12-19 2:44 ` Jörn Engel
2004-12-21 13:30 ` Jörn Engel
0 siblings, 2 replies; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-18 20:48 UTC (permalink / raw)
To: Jörn Engel; +Cc: Linux MTD
Like I said, hacked to work [for me] and not to look pretty.
To be honest what's there will do me for now .. although I'd like to
clean it up a little when I get time.
fyi; don't confuse the old and new code.
> o Coding style (Lindent?)
Whatever was there already.
> o Function length
My functions are shorter than the ones that were there, lots can be
cleaned up here.
> o #ifdef MODULE
.. ? I only use it as a module, not of this code was touched.
> o parameters have an insane interface
.. again , parameters and interfaces are all from the original with the
exception of a couple of new routines which mirror existing parameter
lists.
> o ro-devices. the same can be achieved with chmod.
Again, not touched this.
Gareth.
On Sat, 2004-12-18 at 19:11 +0100, Jörn Engel wrote:
> On Sat, 18 December 2004 18:52:22 +0100, Jörn Engel wrote:
> > On Sat, 18 December 2004 17:32:35 +0000, Gareth Bult (Encryptec) wrote:
> > >
> > > There's a public health warning attached to this code ..
> > >
> > > I've not cleaned it up and I've not added any comments ..
> >
> > I'll wear protective gloves then. Thanks!
>
> Not a pretty. Looks like quite a few cleanups from phram.c can be
> applied to this one as well. From a first glance:
> o Coding style (Lindent?)
> o Function length
> o #ifdef MODULE
> o parameters have an insane interface
> o ro-devices. the same can be achieved with chmod.
>
> But you started the work, which is a good thing.
>
> Jörn
>
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-18 20:48 ` Gareth Bult (Encryptec)
@ 2004-12-19 2:44 ` Jörn Engel
2004-12-21 13:30 ` Jörn Engel
1 sibling, 0 replies; 113+ messages in thread
From: Jörn Engel @ 2004-12-19 2:44 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
On Sat, 18 December 2004 20:48:50 +0000, Gareth Bult (Encryptec) wrote:
>
> Like I said, hacked to work [for me] and not to look pretty.
>
> To be honest what's there will do me for now .. although I'd like to
> clean it up a little when I get time.
>
> fyi; don't confuse the old and new code.
Sure. I had a look at tkdiff and there was ~50% unchanged code.
"This code ain't pretty." didn't mean "You sh*thead are stupid and
cannot create decent code." It was more like "It still ain't pretty,
although you already improved it."
> > o Coding style (Lindent?)
>
> Whatever was there already.
>
> > o Function length
>
> My functions are shorter than the ones that were there, lots can be
> cleaned up here.
>
> > o #ifdef MODULE
>
> .. ? I only use it as a module, not of this code was touched.
>
> > o parameters have an insane interface
>
> .. again , parameters and interfaces are all from the original with the
> exception of a couple of new routines which mirror existing parameter
> lists.
>
> > o ro-devices. the same can be achieved with chmod.
>
> Again, not touched this.
Yep. Those five point were the obvious stuff I could find in five
minutes. Fixing them takes a lot more than five minutes and changing
the parameter interface basically means copying the thing and fixing
the new one.
I did the same with slram/phram. The slram interface was different
for module/non-module built and insane once you had more than a single
device. Mtdblock looks quite similar and seems to need the same cure.
So whatever you can improve, please do.
Jörn
--
It's just what we asked for, but not what we want!
-- anonymous
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-16 13:43 ` Ferenc Havasi
@ 2004-12-20 16:01 ` Gareth Bult (Encryptec)
2004-12-20 16:09 ` Ferenc Havasi
0 siblings, 1 reply; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-20 16:01 UTC (permalink / raw)
To: Ferenc Havasi; +Cc: Linux MTD
Hi,
I'm afraid this patch seems to be quite a long way from applying cleanly
to my source. I'm using 2.6.9 .. at least half a dozen failures or
more ...
:(
Will look into it in more detail later ...
Gareth.
On Thu, 2004-12-16 at 14:43 +0100, Ferenc Havasi wrote:
> Hi Gareth,
>
> > However, I'm looking at 30secs to mount the root filesystem.
>
> Did you try our patch? I send it now again, we made it fresh for the
> current CVS. See its usage in the archive (16th November).
>
> It will part of JFFS3.
>
> This is designed for NAND devices - I don't know too much about your
> environment, but it may help.
>
> Bye,
> Ferenc
> plain text document attachment (jffs2-summary-20041207.patch)
> diff --unified --new-file --recursive mtd/fs/Kconfig mtd-dec07/fs/Kconfig
> --- mtd/fs/Kconfig 2004-11-04 00:00:11.000000000 +0100
> +++ mtd-dec07/fs/Kconfig 2004-12-07 16:03:02.000000000 +0100
> @@ -77,6 +77,19 @@
> ECC for JFFS2. This type of flash chip is not common, however it is
> available from STMicro.
>
> +config JFFS2_SUMMARY
> + bool "JFFS2 summary support (EXPERIMENTAL)"
> + depends on JFFS2_FS
> + default n
> + help
> + This feature makes it possible to use summary information
> + for faster filesystem mount - specially on NAND.
> +
> + The summary information can be inserted into a filesystem image
> + by the utility 'sumtool'.
> +
> + If unsure, say 'N'.
> +
> config JFFS2_COMPRESSION_OPTIONS
> bool "Advanced compression options for JFFS2"
> default n
> diff --unified --new-file --recursive mtd/fs/jffs2/Makefile.common mtd-dec07/fs/jffs2/Makefile.common
> --- mtd/fs/jffs2/Makefile.common 2004-11-04 00:00:11.000000000 +0100
> +++ mtd-dec07/fs/jffs2/Makefile.common 2004-12-07 16:03:02.000000000 +0100
> @@ -16,3 +16,4 @@
> jffs2-$(CONFIG_JFFS2_RUBIN) += compr_rubin.o
> jffs2-$(CONFIG_JFFS2_RTIME) += compr_rtime.o
> jffs2-$(CONFIG_JFFS2_ZLIB) += compr_zlib.o
> +jffs2-$(CONFIG_JFFS2_SUMMARY) += summary.o
> diff --unified --new-file --recursive mtd/fs/jffs2/build.c mtd-dec07/fs/jffs2/build.c
> --- mtd/fs/jffs2/build.c 2004-11-28 00:00:13.000000000 +0100
> +++ mtd-dec07/fs/jffs2/build.c 2004-12-07 16:08:31.000000000 +0100
> @@ -331,6 +331,10 @@
> c->blocks[i].first_node = NULL;
> c->blocks[i].last_node = NULL;
> c->blocks[i].bad_count = 0;
> + #ifdef CONFIG_JFFS2_SUMMARY
> + c->blocks[i].sum_collected = NULL;
> + #endif
> +
> }
>
> init_MUTEX(&c->alloc_sem);
> diff --unified --new-file --recursive mtd/fs/jffs2/dir.c mtd-dec07/fs/jffs2/dir.c
> --- mtd/fs/jffs2/dir.c 2004-11-17 00:00:14.000000000 +0100
> +++ mtd-dec07/fs/jffs2/dir.c 2004-12-07 16:03:02.000000000 +0100
> @@ -22,6 +22,7 @@
> #include <linux/time.h>
> #include "nodelist.h"
>
> +
> /* Urgh. Please tell me there's a nicer way of doing these. */
> #include <linux/version.h>
> #if LINUX_VERSION_CODE < KERNEL_VERSION(2,5,48)
> @@ -314,7 +315,7 @@
> * Just the node will do for now, though
> */
> namelen = dentry->d_name.len;
> - ret = jffs2_reserve_space(c, sizeof(*ri) + strlen(target), &phys_ofs, &alloclen, ALLOC_NORMAL);
> + ret = jffs2_reserve_space(c, sizeof(*ri) + strlen(target), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
>
> if (ret) {
> jffs2_free_raw_inode(ri);
> @@ -360,7 +361,7 @@
> up(&f->sem);
>
> jffs2_complete_reservation(c);
> - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
> + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
> if (ret) {
> /* Eep. */
> jffs2_clear_inode(inode);
> @@ -445,7 +446,7 @@
> * Just the node will do for now, though
> */
> namelen = dentry->d_name.len;
> - ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL);
> + ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
>
> if (ret) {
> jffs2_free_raw_inode(ri);
> @@ -488,7 +489,7 @@
> up(&f->sem);
>
> jffs2_complete_reservation(c);
> - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
> + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
> if (ret) {
> /* Eep. */
> jffs2_clear_inode(inode);
> @@ -597,7 +598,7 @@
> * Just the node will do for now, though
> */
> namelen = dentry->d_name.len;
> - ret = jffs2_reserve_space(c, sizeof(*ri) + devlen, &phys_ofs, &alloclen, ALLOC_NORMAL);
> + ret = jffs2_reserve_space(c, sizeof(*ri) + devlen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
>
> if (ret) {
> jffs2_free_raw_inode(ri);
> @@ -642,7 +643,7 @@
> up(&f->sem);
>
> jffs2_complete_reservation(c);
> - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
> + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
> if (ret) {
> /* Eep. */
> jffs2_clear_inode(inode);
> @@ -796,4 +797,3 @@
>
> return 0;
> }
> -
> diff --unified --new-file --recursive mtd/fs/jffs2/file.c mtd-dec07/fs/jffs2/file.c
> --- mtd/fs/jffs2/file.c 2004-11-17 00:00:14.000000000 +0100
> +++ mtd-dec07/fs/jffs2/file.c 2004-12-07 16:03:02.000000000 +0100
> @@ -22,6 +22,7 @@
> #include <linux/jffs2.h>
> #include "nodelist.h"
>
> +
> extern int generic_file_open(struct inode *, struct file *) __attribute__((weak));
> extern loff_t generic_file_llseek(struct file *file, loff_t offset, int origin) __attribute__((weak));
>
> @@ -135,7 +136,7 @@
> D1(printk(KERN_DEBUG "Writing new hole frag 0x%x-0x%x between current EOF and new page\n",
> (unsigned int)inode->i_size, pageofs));
>
> - ret = jffs2_reserve_space(c, sizeof(ri), &phys_ofs, &alloc_len, ALLOC_NORMAL);
> + ret = jffs2_reserve_space(c, sizeof(ri), &phys_ofs, &alloc_len, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
> if (ret)
> return ret;
>
> diff --unified --new-file --recursive mtd/fs/jffs2/fs.c mtd-dec07/fs/jffs2/fs.c
> --- mtd/fs/jffs2/fs.c 2004-11-29 00:00:09.000000000 +0100
> +++ mtd-dec07/fs/jffs2/fs.c 2004-12-07 16:03:02.000000000 +0100
> @@ -74,7 +74,7 @@
> return -ENOMEM;
> }
>
> - ret = jffs2_reserve_space(c, sizeof(*ri) + mdatalen, &phys_ofs, &alloclen, ALLOC_NORMAL);
> + ret = jffs2_reserve_space(c, sizeof(*ri) + mdatalen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
> if (ret) {
> jffs2_free_raw_inode(ri);
> if (S_ISLNK(inode->i_mode & S_IFMT))
> diff --unified --new-file --recursive mtd/fs/jffs2/gc.c mtd-dec07/fs/jffs2/gc.c
> --- mtd/fs/jffs2/gc.c 2004-11-17 00:00:14.000000000 +0100
> +++ mtd-dec07/fs/jffs2/gc.c 2004-12-07 16:03:02.000000000 +0100
> @@ -506,7 +506,7 @@
> don't want to force wastage of the end of a block if splitting would
> work. */
> ret = jffs2_reserve_space_gc(c, min_t(uint32_t, sizeof(struct jffs2_raw_inode) + JFFS2_MIN_DATA_LEN,
> - rawlen), &phys_ofs, &alloclen);
> + rawlen), &phys_ofs, &alloclen, rawlen); /* this is not optimal yet */
> if (ret)
> return ret;
>
> @@ -614,7 +614,7 @@
> ACCT_SANITY_CHECK(c,jeb);
> D1(ACCT_PARANOIA_CHECK(jeb));
>
> - ret = jffs2_reserve_space_gc(c, rawlen, &phys_ofs, &dummy);
> + ret = jffs2_reserve_space_gc(c, rawlen, &phys_ofs, &dummy, rawlen); /* this is not optimal yet */
>
> if (!ret) {
> D1(printk(KERN_DEBUG "Allocated space at 0x%08x to retry failed write.\n", phys_ofs));
> @@ -692,7 +692,7 @@
>
> }
>
> - ret = jffs2_reserve_space_gc(c, sizeof(ri) + mdatalen, &phys_ofs, &alloclen);
> + ret = jffs2_reserve_space_gc(c, sizeof(ri) + mdatalen, &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE);
> if (ret) {
> printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_metadata failed: %d\n",
> sizeof(ri)+ mdatalen, ret);
> @@ -759,7 +759,7 @@
> rd.node_crc = cpu_to_je32(crc32(0, &rd, sizeof(rd)-8));
> rd.name_crc = cpu_to_je32(crc32(0, fd->name, rd.nsize));
>
> - ret = jffs2_reserve_space_gc(c, sizeof(rd)+rd.nsize, &phys_ofs, &alloclen);
> + ret = jffs2_reserve_space_gc(c, sizeof(rd)+rd.nsize, &phys_ofs, &alloclen, JFFS2_SUMMARY_DIRENT_SIZE(rd.nsize));
> if (ret) {
> printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_dirent failed: %d\n",
> sizeof(rd)+rd.nsize, ret);
> @@ -961,7 +961,7 @@
> ri.data_crc = cpu_to_je32(0);
> ri.node_crc = cpu_to_je32(crc32(0, &ri, sizeof(ri)-8));
>
> - ret = jffs2_reserve_space_gc(c, sizeof(ri), &phys_ofs, &alloclen);
> + ret = jffs2_reserve_space_gc(c, sizeof(ri), &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE);
> if (ret) {
> printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_hole failed: %d\n",
> sizeof(ri), ret);
> @@ -1186,7 +1186,7 @@
> uint32_t cdatalen;
> uint16_t comprtype = JFFS2_COMPR_NONE;
>
> - ret = jffs2_reserve_space_gc(c, sizeof(ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen);
> + ret = jffs2_reserve_space_gc(c, sizeof(ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE);
>
> if (ret) {
> printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_dnode failed: %d\n",
> @@ -1243,4 +1243,3 @@
> jffs2_gc_release_page(c, pg_ptr, &pg);
> return ret;
> }
> -
> diff --unified --new-file --recursive mtd/fs/jffs2/nodelist.h mtd-dec07/fs/jffs2/nodelist.h
> --- mtd/fs/jffs2/nodelist.h 2004-11-26 15:08:31.000000000 +0100
> +++ mtd-dec07/fs/jffs2/nodelist.h 2004-12-07 16:03:02.000000000 +0100
> @@ -20,6 +20,7 @@
> #include <linux/jffs2.h>
> #include <linux/jffs2_fs_sb.h>
> #include <linux/jffs2_fs_i.h>
> +#include "summary.h"
>
> #ifdef __ECOS
> #include "os-ecos.h"
> @@ -194,6 +195,10 @@
> int bad_count;
> uint32_t offset; /* of this block in the MTD */
>
> +#ifdef CONFIG_JFFS2_SUMMARY
> + struct jffs2_sum_info *sum_collected;
> +#endif
> +
> uint32_t unchecked_size;
> uint32_t used_size;
> uint32_t dirty_size;
> @@ -392,8 +397,8 @@
>
> /* nodemgmt.c */
> int jffs2_thread_should_wake(struct jffs2_sb_info *c);
> -int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio);
> -int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len);
> +int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio, uint32_t sumsize);
> +int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize);
> int jffs2_add_physical_node_ref(struct jffs2_sb_info *c, struct jffs2_raw_node_ref *new);
> void jffs2_complete_reservation(struct jffs2_sb_info *c);
> void jffs2_mark_node_obsolete(struct jffs2_sb_info *c, struct jffs2_raw_node_ref *raw);
> @@ -455,6 +460,10 @@
> /* scan.c */
> int jffs2_scan_medium(struct jffs2_sb_info *c);
> void jffs2_rotate_lists(struct jffs2_sb_info *c);
> +int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf,
> + uint32_t ofs, uint32_t len);
> +struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino);
> +
>
> /* build.c */
> int jffs2_do_mount_fs(struct jffs2_sb_info *c);
> diff --unified --new-file --recursive mtd/fs/jffs2/nodemgmt.c mtd-dec07/fs/jffs2/nodemgmt.c
> --- mtd/fs/jffs2/nodemgmt.c 2004-11-26 15:08:31.000000000 +0100
> +++ mtd-dec07/fs/jffs2/nodemgmt.c 2004-12-07 16:03:02.000000000 +0100
> @@ -38,9 +38,9 @@
> * for the requested allocation.
> */
>
> -static int jffs2_do_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len);
> +static int jffs2_do_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize);
>
> -int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio)
> +int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio, uint32_t sumsize)
> {
> int ret = -EAGAIN;
> int blocksneeded = c->resv_blocks_write;
> @@ -129,7 +129,7 @@
> spin_lock(&c->erase_completion_lock);
> }
>
> - ret = jffs2_do_reserve_space(c, minsize, ofs, len);
> + ret = jffs2_do_reserve_space(c, minsize, ofs, len, sumsize);
> if (ret) {
> D1(printk(KERN_DEBUG "jffs2_reserve_space: ret is %d\n", ret));
> }
> @@ -140,7 +140,7 @@
> return ret;
> }
>
> -int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len)
> +int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize)
> {
> int ret = -EAGAIN;
> minsize = PAD(minsize);
> @@ -149,7 +149,7 @@
>
> spin_lock(&c->erase_completion_lock);
> while(ret == -EAGAIN) {
> - ret = jffs2_do_reserve_space(c, minsize, ofs, len);
> + ret = jffs2_do_reserve_space(c, minsize, ofs, len, sumsize);
> if (ret) {
> D1(printk(KERN_DEBUG "jffs2_reserve_space_gc: looping, ret is %d\n", ret));
> }
> @@ -159,50 +159,112 @@
> }
>
> /* Called with alloc sem _and_ erase_completion_lock */
> -static int jffs2_do_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len)
> +static int jffs2_do_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize)
> {
> struct jffs2_eraseblock *jeb = c->nextblock;
> + uint32_t nofree_size;
>
> restart:
> - if (jeb && minsize > jeb->free_size) {
> - /* Skip the end of this block and file it as having some dirty space */
> - /* If there's a pending write to it, flush now */
> - if (jffs2_wbuf_dirty(c)) {
> - spin_unlock(&c->erase_completion_lock);
> - D1(printk(KERN_DEBUG "jffs2_do_reserve_space: Flushing write buffer\n"));
> - jffs2_flush_wbuf_pad(c);
> - spin_lock(&c->erase_completion_lock);
> - jeb = c->nextblock;
> - goto restart;
> - }
> - c->wasted_size += jeb->free_size;
> - c->free_size -= jeb->free_size;
> - jeb->wasted_size += jeb->free_size;
> - jeb->free_size = 0;
> + nofree_size = 0;
> +
> +#ifdef CONFIG_JFFS2_SUMMARY
> +
> + if (sumsize != JFFS2_SUMMARY_NOSUM_SIZE) {
> + int ret;
> + if (jeb) {
> + if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret;
> + nofree_size = sumsize + jeb->sum_collected->sum_size + JFFS2_SUMMARY_FRAME_SIZE;
> + }
>
> - /* Check, if we have a dirty block now, or if it was dirty already */
> - if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) {
> - c->dirty_size += jeb->wasted_size;
> - c->wasted_size -= jeb->wasted_size;
> - jeb->dirty_size += jeb->wasted_size;
> - jeb->wasted_size = 0;
> - if (VERYDIRTY(c, jeb->dirty_size)) {
> - D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
> + D1(printk(KERN_DEBUG "JFFS2: minsize %d , jeb->free(%d) , sum_collected->size(%d) , sumsize(%d)\n",minsize,jeb->free_size,jeb->sum_collected->sum_size,sumsize));
> +
> + if (jeb && (minsize + jeb->sum_collected->sum_size + sumsize + JFFS2_SUMMARY_FRAME_SIZE > jeb->free_size)) {
> + D1(printk(KERN_DEBUG "JFFS2: generating summary for 0x%08x.\n", jeb->offset));
> + if (jeb->sum_collected->sum_size == JFFS2_SUMMARY_NOSUM_SIZE) {
> + sumsize = JFFS2_SUMMARY_NOSUM_SIZE;
> + jffs2_sum_clean_collected(jeb);
> + goto restart;
> + }
> +
> + ret = jffs2_sum_write_sumnode(c);
> +
> + if (ret)
> + return ret;
> +
> + if (jeb->sum_collected->sum_size == JFFS2_SUMMARY_NOSUM_SIZE) { //jffs2_write_sumnode can't write out the summary information
> + sumsize = JFFS2_SUMMARY_NOSUM_SIZE;
> + jffs2_sum_clean_collected(jeb);
> + goto restart;
> + }
> +
> + /* Check, if we have a dirty block now, or if it was dirty already */
> + if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) {
> + c->dirty_size += jeb->wasted_size;
> + c->wasted_size -= jeb->wasted_size;
> + jeb->dirty_size += jeb->wasted_size;
> + jeb->wasted_size = 0;
> + if (VERYDIRTY(c, jeb->dirty_size)) {
> + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
> + jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
> + list_add_tail(&jeb->list, &c->very_dirty_list);
> + } else {
> + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
> + jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
> + list_add_tail(&jeb->list, &c->dirty_list);
> + }
> + } else {
> + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
> jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
> - list_add_tail(&jeb->list, &c->very_dirty_list);
> - } else {
> - D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
> + list_add_tail(&jeb->list, &c->clean_list);
> + }
> + c->nextblock = jeb = NULL;
> + }
> + }
> + else {
> +#endif
> + if (jeb && minsize > jeb->free_size) {
> + /* Skip the end of this block and file it as having some dirty space */
> + /* If there's a pending write to it, flush now */
> +
> + if (jffs2_wbuf_dirty(c)) {
> + spin_unlock(&c->erase_completion_lock);
> + D1(printk(KERN_DEBUG "jffs2_do_reserve_space: Flushing write buffer\n"));
> + jffs2_flush_wbuf_pad(c);
> + spin_lock(&c->erase_completion_lock);
> + jeb = c->nextblock;
> + goto restart;
> + }
> +
> + c->wasted_size += jeb->free_size;
> + c->free_size -= jeb->free_size;
> + jeb->wasted_size += jeb->free_size;
> + jeb->free_size = 0;
> +
> + /* Check, if we have a dirty block now, or if it was dirty already */
> + if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) {
> + c->dirty_size += jeb->wasted_size;
> + c->wasted_size -= jeb->wasted_size;
> + jeb->dirty_size += jeb->wasted_size;
> + jeb->wasted_size = 0;
> + if (VERYDIRTY(c, jeb->dirty_size)) {
> + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
> + jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
> + list_add_tail(&jeb->list, &c->very_dirty_list);
> + } else {
> + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
> + jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
> + list_add_tail(&jeb->list, &c->dirty_list);
> + }
> + } else {
> + D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
> jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
> - list_add_tail(&jeb->list, &c->dirty_list);
> + list_add_tail(&jeb->list, &c->clean_list);
> }
> - } else {
> - D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
> - jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
> - list_add_tail(&jeb->list, &c->clean_list);
> + c->nextblock = jeb = NULL;
> }
> - c->nextblock = jeb = NULL;
> +#ifdef CONFIG_JFFS2_SUMMARY
> }
> -
> +#endif
> if (!jeb) {
> struct list_head *next;
> /* Take the next block off the 'free' list */
> @@ -266,7 +328,7 @@
> /* OK, jeb (==c->nextblock) is now pointing at a block which definitely has
> enough space */
> *ofs = jeb->offset + (c->sector_size - jeb->free_size);
> - *len = jeb->free_size;
> + *len = jeb->free_size - nofree_size;
>
> if (c->cleanmarker_size && jeb->used_size == c->cleanmarker_size &&
> !jeb->first_node->next_in_ino) {
> diff --unified --new-file --recursive mtd/fs/jffs2/os-linux.h mtd-dec07/fs/jffs2/os-linux.h
> --- mtd/fs/jffs2/os-linux.h 2004-11-17 00:00:14.000000000 +0100
> +++ mtd-dec07/fs/jffs2/os-linux.h 2004-12-07 16:03:02.000000000 +0100
> @@ -100,7 +100,13 @@
> #define jffs2_is_readonly(c) (OFNI_BS_2SFFJ(c)->s_flags & MS_RDONLY)
>
> #if (!defined CONFIG_JFFS2_FS_NAND && !defined CONFIG_JFFS2_FS_NOR_ECC)
> +
> +#ifndef CONFIG_JFFS2_SUMMARY
> #define jffs2_can_mark_obsolete(c) (1)
> +#else
> +#define jffs2_can_mark_obsolete(c) (0)
> +#endif
> +
> #define jffs2_cleanmarker_oob(c) (0)
> #define jffs2_write_nand_cleanmarker(c,jeb) (-EIO)
>
> @@ -121,7 +127,12 @@
>
> #else /* NAND and/or ECC'd NOR support present */
>
> +#ifndef CONFIG_JFFS2_SUMMARY
> #define jffs2_can_mark_obsolete(c) ((c->mtd->type == MTD_NORFLASH && !(c->mtd->flags & MTD_ECC)) || c->mtd->type == MTD_RAM)
> +#else
> +#define jffs2_can_mark_obsolete(c) (0)
> +#endif
> +
> #define jffs2_cleanmarker_oob(c) (c->mtd->type == MTD_NANDFLASH)
>
> #define jffs2_flash_write_oob(c, ofs, len, retlen, buf) ((c)->mtd->write_oob((c)->mtd, ofs, len, retlen, buf))
> @@ -218,5 +229,3 @@
>
>
> #endif /* __JFFS2_OS_LINUX_H__ */
> -
> -
> diff --unified --new-file --recursive mtd/fs/jffs2/scan.c mtd-dec07/fs/jffs2/scan.c
> --- mtd/fs/jffs2/scan.c 2004-11-26 15:08:31.000000000 +0100
> +++ mtd-dec07/fs/jffs2/scan.c 2004-12-07 16:03:02.000000000 +0100
> @@ -18,22 +18,10 @@
> #include <linux/crc32.h>
> #include <linux/compiler.h>
> #include "nodelist.h"
> +#include "summary.h"
>
> #define EMPTY_SCAN_SIZE 1024
>
> -#define DIRTY_SPACE(x) do { typeof(x) _x = (x); \
> - c->free_size -= _x; c->dirty_size += _x; \
> - jeb->free_size -= _x ; jeb->dirty_size += _x; \
> - }while(0)
> -#define USED_SPACE(x) do { typeof(x) _x = (x); \
> - c->free_size -= _x; c->used_size += _x; \
> - jeb->free_size -= _x ; jeb->used_size += _x; \
> - }while(0)
> -#define UNCHECKED_SPACE(x) do { typeof(x) _x = (x); \
> - c->free_size -= _x; c->unchecked_size += _x; \
> - jeb->free_size -= _x ; jeb->unchecked_size += _x; \
> - }while(0)
> -
> #define noisy_printk(noise, args...) do { \
> if (*(noise)) { \
> printk(KERN_NOTICE args); \
> @@ -58,13 +46,6 @@
> static int jffs2_scan_dirent_node(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb,
> struct jffs2_raw_dirent *rd, uint32_t ofs);
>
> -#define BLK_STATE_ALLFF 0
> -#define BLK_STATE_CLEAN 1
> -#define BLK_STATE_PARTDIRTY 2
> -#define BLK_STATE_CLEANMARKER 3
> -#define BLK_STATE_ALLDIRTY 4
> -#define BLK_STATE_BADBLOCK 5
> -
> static inline int min_free(struct jffs2_sb_info *c)
> {
> uint32_t min = 2 * sizeof(struct jffs2_raw_inode);
> @@ -257,7 +238,7 @@
> return ret;
> }
>
> -static int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf,
> +int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf,
> uint32_t ofs, uint32_t len)
> {
> int ret;
> @@ -286,6 +267,11 @@
> uint32_t hdr_crc, buf_ofs, buf_len;
> int err;
> int noise = 0;
> +
> +#ifdef CONFIG_JFFS2_SUMMARY
> + struct jffs2_sum_marker *sm;
> +#endif
> +
> #ifdef CONFIG_JFFS2_FS_NAND
> int cleanmarkerfound = 0;
> #endif
> @@ -311,10 +297,54 @@
> }
> }
> #endif
> +
> +#ifdef CONFIG_JFFS2_SUMMARY
> + sm = (struct jffs2_sum_marker *)kmalloc(sizeof(struct jffs2_sum_marker), GFP_KERNEL);
> + if (!sm) {
> + return -ENOMEM;
> + }
> +
> + err = jffs2_fill_scan_buf(c, (unsigned char *) sm, jeb->offset + c->sector_size - sizeof(struct jffs2_sum_marker), sizeof(struct jffs2_sum_marker));
> +
> + if (err) {
> + kfree(sm);
> + return err;
> + }
> +
> + if (je32_to_cpu(sm->magic) == JFFS2_SUM_MAGIC ) {
> +
> + if(je32_to_cpu(sm->erase_size) == c->sector_size) {
> + int ret = jffs2_sum_scan_sumnode(c,jeb,je32_to_cpu(sm->offset),&pseudo_random);
> +
> + if (ret) {
> + kfree(sm);
> + return ret;
> + }
> + }
> +
> + printk(KERN_WARNING "FS erase_block_size != JFFS2 erase_block_size => skipping summary information\n");
> +
> + }
> +
> + kfree(sm);
> +
> + ofs = jeb->offset;
> + prevofs = jeb->offset - 1;
> +
> +#endif
> +
> buf_ofs = jeb->offset;
>
> if (!buf_size) {
> buf_len = c->sector_size;
> +
> +#ifdef CONFIG_JFFS2_SUMMARY
> + /* must reread because of summary test */
> + err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len);
> + if (err)
> + return err;
> +#endif
> +
> } else {
> buf_len = EMPTY_SCAN_SIZE;
> err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len);
> @@ -356,6 +386,8 @@
>
> noise = 10;
>
> + D1(printk(KERN_DEBUG "JFFS2: no summary found in jeb 0x%08x. Apply original scan.\n",jeb->offset));
> +
> scan_more:
> while(ofs < jeb->offset + c->sector_size) {
>
> @@ -571,6 +603,9 @@
> break;
>
> case JFFS2_NODETYPE_PADDING:
> +#ifdef CONFIG_JFFS2_SUMMARY
> + jffs2_sum_add_padding_mem(jeb,je32_to_cpu(node->totlen));
> +#endif
> DIRTY_SPACE(PAD(je32_to_cpu(node->totlen)));
> ofs += PAD(je32_to_cpu(node->totlen));
> break;
> @@ -634,7 +669,7 @@
> return BLK_STATE_ALLDIRTY;
> }
>
> -static struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino)
> +struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino)
> {
> struct jffs2_inode_cache *ic;
>
> @@ -728,6 +763,11 @@
> pseudo_random += je32_to_cpu(ri->version);
>
> UNCHECKED_SPACE(PAD(je32_to_cpu(ri->totlen)));
> +
> +#ifdef CONFIG_JFFS2_SUMMARY
> + jffs2_sum_add_inode_mem(jeb,ri,ofs);
> +#endif
> +
> return 0;
> }
>
> @@ -806,6 +846,10 @@
> USED_SPACE(PAD(je32_to_cpu(rd->totlen)));
> jffs2_add_fd_to_list(c, fd, &ic->scan_dents);
>
> +#ifdef CONFIG_JFFS2_SUMMARY
> + jffs2_sum_add_dirent_mem(jeb,rd,ofs);
> +#endif
> +
> return 0;
> }
>
> diff --unified --new-file --recursive mtd/fs/jffs2/summary.c mtd-dec07/fs/jffs2/summary.c
> --- mtd/fs/jffs2/summary.c 1970-01-01 01:00:00.000000000 +0100
> +++ mtd-dec07/fs/jffs2/summary.c 2004-12-07 16:03:02.000000000 +0100
> @@ -0,0 +1,720 @@
> +/*
> + * JFFS2 -- Journalling Flash File System, Version 2.
> + *
> + * Copyright (C) 2004 Ferenc Havasi <havasi@inf.u-szeged.hu>,
> + * Zoltan Sogor <weth@inf.u-szeged.hu>,
> + * Patrik Kluba <pajko@halom.u-szeged.hu>,
> + * University of Szeged, Hungary
> + *
> + * For licensing information, see the file 'LICENCE' in this directory.
> + *
> + * $Id$
> + *
> + */
> +
> +#include <linux/kernel.h>
> +#include <linux/sched.h>
> +#include <linux/slab.h>
> +#include <linux/mtd/mtd.h>
> +#include <linux/pagemap.h>
> +#include <linux/crc32.h>
> +#include <linux/compiler.h>
> +#include <linux/vmalloc.h>
> +#include "nodelist.h"
> +
> +int jffs2_sum_init(struct jffs2_sb_info *c)
> +{
> + c->summary_buf = (jint32_t *) vmalloc(c->sector_size);
> + if (!c->summary_buf) {
> + printk(KERN_WARNING "JFFS2: can't allocate memory to dump summary information!\n");
> + return 1;
> + }
> + return 0;
> +}
> +
> +void jffs2_sum_exit(struct jffs2_sb_info *c)
> +{
> + if (c->summary_buf) {
> + vfree(c->summary_buf);
> + c->summary_buf = NULL;
> + }
> +}
> +
> +int jffs2_sum_care_sum_collected(struct jffs2_eraseblock *jeb)
> +{
> + if (!jeb->sum_collected) {
> + jeb->sum_collected = (struct jffs2_sum_info *) kmalloc(sizeof(struct jffs2_sum_info), GFP_KERNEL);
> +
> + if (!jeb->sum_collected)
> + return -ENOMEM;
> +
> + jeb->sum_collected->sum_list = NULL;
> + jeb->sum_collected->sum_num = 0;
> + jeb->sum_collected->sum_size = 0;
> + jeb->sum_collected->sum_padded = 0;
> + }
> + return 0;
> +}
> +
> +static int jffs2_sum_add_mem(struct jffs2_eraseblock *jeb, union jffs2_sum_mem *item)
> +{
> +
> + union jffs2_sum_mem *walk;
> + int ret;
> +
> + if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret;
> +
> + if (!jeb->sum_collected->sum_list) {
> + jeb->sum_collected->sum_list = (union jffs2_sum_mem *) item;
> + }
> + else {
> + walk = jeb->sum_collected->sum_list;
> +
> + while (walk->u.next) {
> + walk = walk->u.next;
> + }
> + walk->u.next = (union jffs2_sum_mem *) item;
> + }
> + switch (je16_to_cpu(item->u.nodetype)) {
> + case JFFS2_NODETYPE_INODE:
> + jeb->sum_collected->sum_size += JFFS2_SUMMARY_INODE_SIZE;
> + jeb->sum_collected->sum_num++;
> + break;
> + case JFFS2_NODETYPE_DIRENT:
> + jeb->sum_collected->sum_size += JFFS2_SUMMARY_DIRENT_SIZE(item->d.nsize);
> + jeb->sum_collected->sum_num++;
> + break;
> + default:
> + printk(KERN_WARNING "__jffs2_add_sum_mem(): UNKNOWN node type %d\n", je16_to_cpu(item->u.nodetype));
> + return 1;
> + }
> + return 0;
> +}
> +
> +void jffs2_sum_clean_all_info(struct jffs2_sb_info *c)
> +{
> + int i;
> +
> + for (i=0; i<c->nr_blocks; i++) {
> + struct jffs2_eraseblock *jeb = &c->blocks[i];
> +
> + jffs2_sum_clean_collected(jeb);
> + kfree(jeb->sum_collected);
> + jeb->sum_collected = NULL;
> + }
> +}
> +
> +/* These 3 functions are called from scan.c to collect summary info for not closed jeb */
> +
> +int jffs2_sum_add_padding_mem(struct jffs2_eraseblock *jeb, uint32_t size)
> +{
> + int ret;
> +
> + if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret;
> + jeb->sum_collected->sum_padded += size;
> + return 0;
> +}
> +
> +int jffs2_sum_add_inode_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_inode *ri, uint32_t ofs)
> +{
> +
> + struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *) kmalloc(sizeof(struct jffs2_sum_inode_mem), GFP_KERNEL);
> +
> + if (!temp)
> + return -ENOMEM;
> +
> + ofs -= jeb->offset;
> +
> + temp->nodetype = ri->nodetype;
> + temp->inode = ri->ino;
> + temp->version = ri->version;
> + temp->offset = cpu_to_je32(ofs);
> + temp->totlen = ri->totlen;
> + temp->next = NULL;
> +
> + return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp);
> +}
> +
> +int jffs2_sum_add_dirent_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_dirent *rd, uint32_t ofs)
> +{
> +
> + struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *)
> + kmalloc(sizeof(struct jffs2_sum_dirent_mem) + rd->nsize, GFP_KERNEL);
> +
> + if (!temp)
> + return -ENOMEM;
> +
> + ofs -= jeb->offset;
> +
> + temp->nodetype = rd->nodetype;
> + temp->totlen = rd->totlen;
> + temp->offset = cpu_to_je32(ofs);
> + temp->pino = rd->pino;
> + temp->version = rd->version;
> + temp->ino = rd->ino;
> + temp->nsize = rd->nsize;
> + temp->type = rd->type;
> + temp->next = NULL;
> +
> + memcpy(temp->name, rd->name, rd->nsize);
> +
> + return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp);
> +}
> +
> +/* Cleanup every collected summary information */
> +
> +void jffs2_sum_clean_collected(struct jffs2_eraseblock *jeb)
> +{
> +
> + union jffs2_sum_mem *temp;
> +
> + if(jeb && jeb->sum_collected){
> +
> + while(jeb->sum_collected->sum_list){
> + temp = jeb->sum_collected->sum_list;
> + jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next;
> + kfree(temp);
> + jeb->sum_collected->sum_num--;
> + }
> +
> + if(jeb->sum_collected->sum_num != 0){
> + printk(KERN_WARNING "Ooops, something wrong happened! sum_num != 0, but sum_list = null ???");
> + jeb->sum_collected->sum_num = 0;
> + }
> + }
> +}
> +
> +/* Called from wbuf.c to collect writed node info */
> +
> +int jffs2_sum_add_kvec(struct jffs2_sb_info *c, const struct kvec *invecs, unsigned long count, uint32_t ofs)
> +{
> + union jffs2_node_union *node;
> + struct jffs2_eraseblock *jeb;
> + int ret;
> +
> + node = (union jffs2_node_union *) invecs[0].iov_base;
> + jeb = &c->blocks[ofs / c->sector_size];
> + ofs -= jeb->offset;
> +
> + if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret;
> +
> + switch(je16_to_cpu(node->u.nodetype)){
> + case JFFS2_NODETYPE_INODE : {
> + struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *)
> + kmalloc(sizeof(struct jffs2_sum_inode_mem), GFP_KERNEL);
> +
> + if (!temp)
> + return -ENOMEM;
> +
> + temp->nodetype = node->i.nodetype;
> + temp->inode = node->i.ino;
> + temp->version = node->i.version;
> + temp->offset = cpu_to_je32(ofs);
> + temp->totlen = node->i.totlen;
> + temp->next = NULL;
> +
> + return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp);
> +
> + break;
> + }
> +
> + case JFFS2_NODETYPE_DIRENT : {
> + struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *)
> + kmalloc(sizeof(struct jffs2_sum_dirent_mem) + node->d.nsize, GFP_KERNEL);
> +
> + if (!temp)
> + return -ENOMEM;
> +
> + temp->nodetype = node->d.nodetype;
> + temp->totlen = node->d.totlen;
> + temp->offset = cpu_to_je32(ofs);
> + temp->pino = node->d.pino;
> + temp->version = node->d.version;
> + temp->ino = node->d.ino;
> + temp->nsize = node->d.nsize;
> + temp->type = node->d.type;
> + temp->next = NULL;
> +
> + memcpy(temp->name,invecs[1].iov_base,node->d.nsize);
> +
> + return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp);
> +
> + break;
> + }
> +
> + case JFFS2_NODETYPE_PADDING : {
> + D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node PADDING\n"));
> + jeb->sum_collected->sum_padded += je32_to_cpu(node->u.totlen);
> + break;
> + }
> +
> + case JFFS2_NODETYPE_CLEANMARKER : {
> + D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node CLEANMARKER\n"));
> + break;
> + }
> +
> + case JFFS2_NODETYPE_SUMMARY : {
> + D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node SUMMARY\n"));
> + break;
> + }
> +
> + default : {
> + printk(KERN_WARNING "jffs2_sum_add_kvec(): Node not supported\n");
> + BUG();
> + break;
> + }
> + }
> +
> + return 0;
> +}
> +
> +/* Process the summary information - called from jffs2_scan_eraseblock() */
> +
> +int jffs2_sum_scan_sumnode(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb, uint32_t ofs, uint32_t *pseudo_random)
> +{
> +
> + struct jffs2_unknown_node crcnode;
> + struct jffs2_raw_node_ref *raw;
> + struct jffs2_raw_node_ref *cache_ref;
> + struct jffs2_inode_cache *ic;
> + struct jffs2_full_dirent *fd;
> +
> + int i, err;
> + int bad_sum = 0;
> + int sumsize;
> + uint32_t ino;
> + uint32_t crc;
> + struct jffs2_summary_node *summary;
> +
> + sumsize = c->sector_size - ofs;
> + ofs += jeb->offset;
> +
> + D1(printk(KERN_DEBUG "JFFS2: summary found for 0x%08x at 0x%08x (0x%x bytes)\n", jeb->offset, ofs, sumsize));
> +
> + summary = (struct jffs2_summary_node *) kmalloc(sumsize, GFP_KERNEL);
> +
> + if (!summary) {
> + return -ENOMEM;
> + }
> +
> + err = jffs2_fill_scan_buf(c, (unsigned char *)summary, ofs, sumsize);
> +
> + if (err) {
> + kfree(summary);
> + return err;
> + }
> +
> + /* OK, now check for node validity and CRC */
> + crcnode.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
> + crcnode.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY);
> + crcnode.totlen = summary->totlen;
> + crc = crc32(0, &crcnode, sizeof(crcnode)-4);
> +
> + if (je32_to_cpu(summary->hdr_crc) != crc) {
> + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node header is corrupt (bad CRC or no summary at all)\n"));
> + bad_sum = 1;
> + }
> +
> + if ((!bad_sum) && (je32_to_cpu(summary->totlen) != sumsize)) {
> + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (wrong erasesize?)\n"));
> + bad_sum = 1;
> + }
> +
> + crc = crc32(0, summary, sizeof(struct jffs2_summary_node)-8);
> +
> + if ((!bad_sum) && (je32_to_cpu(summary->node_crc) != crc)) {
> + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (bad CRC)\n"));
> + bad_sum = 1;
> + }
> +
> + crc = crc32(0, summary->sum, sumsize - sizeof(struct jffs2_summary_node));
> +
> + if ((!bad_sum) && (je32_to_cpu(summary->sum_crc) != crc)) {
> + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node data is corrupt (bad CRC)\n"));
> + bad_sum = 1;
> + }
> +
> + if (!bad_sum) {
> +
> + struct jffs2_sum_unknown_flash *sp;
> + sp = (struct jffs2_sum_unknown_flash *) summary->sum;
> +
> + if ( je32_to_cpu(summary->cln_mkr) ) {
> +
> + D1(printk(KERN_DEBUG "Summary : CLEANMARKER node \n"));
> +
> + if (je32_to_cpu(summary->cln_mkr) != c->cleanmarker_size) {
> + D1(printk(KERN_DEBUG "CLEANMARKER node has totlen 0x%x != normal 0x%x\n",
> + je32_to_cpu(summary->cln_mkr), c->cleanmarker_size));
> + UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) );
> + }
> + else if (jeb->first_node) {
> + D1(printk(KERN_DEBUG "CLEANMARKER node not first node in block (0x%08x)\n", jeb->offset);
> + UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr))));
> + }
> + else {
> + struct jffs2_raw_node_ref *marker_ref = jffs2_alloc_raw_node_ref();
> +
> + if (!marker_ref) {
> + D1(printk(KERN_NOTICE "Failed to allocate node ref for clean marker\n"));
> + kfree(summary);
> + return -ENOMEM;
> + }
> +
> + marker_ref->next_in_ino = NULL;
> + marker_ref->next_phys = NULL;
> + marker_ref->flash_offset = jeb->offset | REF_NORMAL;
> + marker_ref->__totlen = je32_to_cpu(summary->cln_mkr);
> + jeb->first_node = jeb->last_node = marker_ref;
> +
> + USED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) );
> +
> + }
> + }
> +
> + if ( je32_to_cpu(summary->padded) ) {
> + DIRTY_SPACE(je32_to_cpu(summary->padded));
> + }
> +
> + for(i = 0; i < je16_to_cpu(summary->sum_num); i++) {
> +
> + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Processing summary information %d\n", i));
> + uint8_t *temp8ptr = NULL;
> +
> + switch (je16_to_cpu(sp->nodetype)) {
> +
> + case JFFS2_NODETYPE_INODE : {
> + struct jffs2_sum_inode_flash *spi;
> + spi = (struct jffs2_sum_inode_flash *) sp;
> +
> + ino = je32_to_cpu(spi->inode);
> + D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Inode at 0x%08x\n", jeb->offset + je32_to_cpu(spi->offset)));
> + raw = jffs2_alloc_raw_node_ref();
> + if (!raw) {
> + printk(KERN_NOTICE "jffs2_scan_eraseblock(): allocation of node reference failed\n");
> + kfree(summary);
> + return -ENOMEM;
> + }
> +
> + ic = jffs2_get_ino_cache(c, ino);
> + if (!ic) {
> + ic = jffs2_scan_make_ino_cache(c, ino);
> + if (!ic) {
> + printk(KERN_NOTICE "jffs2_scan_eraseblock(): scan_make_ino_cache failed\n");
> + jffs2_free_raw_node_ref(raw);
> + kfree(summary);
> + return -ENOMEM;
> + }
> + }
> +
> + raw->flash_offset = (jeb->offset + je32_to_cpu(spi->offset)) | REF_UNCHECKED;
> + raw->__totlen = PAD(je32_to_cpu(spi->totlen));
> + raw->next_phys = NULL;
> + raw->next_in_ino = ic->nodes;
> +
> + ic->nodes = raw;
> + if (!jeb->first_node)
> + jeb->first_node = raw;
> + if (jeb->last_node)
> + jeb->last_node->next_phys = raw;
> + jeb->last_node = raw;
> +
> + *pseudo_random += je32_to_cpu(spi->version);
> + UNCHECKED_SPACE(PAD(je32_to_cpu(spi->totlen)));
> +
> + temp8ptr = (uint8_t *) sp;
> + temp8ptr += JFFS2_SUMMARY_INODE_SIZE;
> + sp = (struct jffs2_sum_unknown_flash *) temp8ptr;
> +
> + break;
> + }
> +
> + case JFFS2_NODETYPE_DIRENT : {
> + struct jffs2_sum_dirent_flash *spd;
> + spd = (struct jffs2_sum_dirent_flash *) sp;
> +
> + fd = jffs2_alloc_full_dirent(spd->nsize+1);
> + if (!fd) {
> + kfree(summary);
> + return -ENOMEM;
> + }
> +
> + memcpy(&fd->name, spd->name, spd->nsize);
> + fd->name[spd->nsize] = 0;
> +
> + raw = jffs2_alloc_raw_node_ref();
> + if (!raw) {
> + jffs2_free_full_dirent(fd);
> + printk(KERN_NOTICE "jffs2_scan_dirent_node(): allocation of node reference failed\n");
> + kfree(summary);
> + return -ENOMEM;
> + }
> +
> + ic = jffs2_scan_make_ino_cache(c, je32_to_cpu(spd->pino));
> + if (!ic) {
> + jffs2_free_full_dirent(fd);
> + jffs2_free_raw_node_ref(raw);
> + kfree(summary);
> + return -ENOMEM;
> + }
> +
> + raw->__totlen = PAD(je32_to_cpu(spd->totlen));
> + raw->flash_offset = (jeb->offset + je32_to_cpu(spd->offset)) | REF_PRISTINE;
> + raw->next_phys = NULL;
> + raw->next_in_ino = ic->nodes;
> + ic->nodes = raw;
> + if (!jeb->first_node)
> + jeb->first_node = raw;
> + if (jeb->last_node)
> + jeb->last_node->next_phys = raw;
> + jeb->last_node = raw;
> +
> + fd->raw = raw;
> + fd->next = NULL;
> + fd->version = je32_to_cpu(spd->version);
> + fd->ino = je32_to_cpu(spd->ino);
> + fd->nhash = full_name_hash(fd->name, spd->nsize);
> + fd->type = spd->type;
> + USED_SPACE(PAD(je32_to_cpu(spd->totlen)));
> + jffs2_add_fd_to_list(c, fd, &ic->scan_dents);
> +
> + *pseudo_random += je32_to_cpu(spd->version);
> +
> + temp8ptr = (uint8_t *) sp;
> + temp8ptr += JFFS2_SUMMARY_DIRENT_SIZE(spd->nsize);
> + sp = (struct jffs2_sum_unknown_flash *) temp8ptr;
> +
> + break;
> + }
> +
> + default : {
> + printk(KERN_WARNING "Kernel doesn't support this type of node !!! Exiting");
> + return -EIO;
> + break;
> + }
> + }
> + }
> +
> + kfree(summary);
> +
> + /* for ACCT_PARANOIA_CHECK */
> + cache_ref = jffs2_alloc_raw_node_ref();
> +
> + if (!cache_ref) {
> + printk(KERN_NOTICE "Failed to allocate node ref for cache\n");
> + return -ENOMEM;
> + }
> +
> + cache_ref->next_in_ino = NULL;
> + cache_ref->next_phys = NULL;
> + cache_ref->flash_offset = ofs | REF_NORMAL;
> + cache_ref->__totlen = sumsize;
> +
> + if (!jeb->first_node)
> + jeb->first_node = cache_ref;
> + if (jeb->last_node)
> + jeb->last_node->next_phys = cache_ref;
> + jeb->last_node = cache_ref;
> +
> + USED_SPACE(sumsize);
> +
> + jeb->wasted_size += jeb->free_size;
> + c->wasted_size += jeb->free_size;
> + c->free_size -= jeb->free_size;
> + jeb->free_size = 0;
> +
> + /* somebody check this and all of space accounting in summary support */
> +
> + if ((jeb->used_size + jeb->unchecked_size) == PAD(c->cleanmarker_size) && !jeb->dirty_size
> + && (!jeb->first_node || !jeb->first_node->next_in_ino) ) {
> + return BLK_STATE_CLEANMARKER;
> + }
> + /* move blocks with max 4 byte dirty space to cleanlist */
> + else if (!ISDIRTY(c->sector_size - (jeb->used_size + jeb->unchecked_size))) {
> + c->dirty_size -= jeb->dirty_size;
> + c->wasted_size += jeb->dirty_size;
> + jeb->wasted_size += jeb->dirty_size;
> + jeb->dirty_size = 0;
> + return BLK_STATE_CLEAN;
> + }
> + else if (jeb->used_size || jeb->unchecked_size) {
> + return BLK_STATE_PARTDIRTY;
> + }
> + else {
> + return BLK_STATE_ALLDIRTY;
> + }
> + }
> +
> + return 0;
> +}
> +
> +/* Write out summary information - called from jffs2_do_reserve_space */
> +
> +int jffs2_sum_write_sumnode(struct jffs2_sb_info *c)
> +{
> + struct jffs2_summary_node isum;
> + union jffs2_sum_mem *temp;
> + jint32_t offset;
> + jint32_t *wpage;
> + uint8_t *tempptr;
> + int datasize;
> + int infosize;
> + int padsize;
> + size_t retlen;
> + int ret;
> + struct jffs2_eraseblock *jeb;
> + struct kvec vecs[2];
> + jint32_t magic = cpu_to_je32(JFFS2_SUM_MAGIC);
> +
> + D2(printk(KERN_DEBUG "jffs2_sum_write_sumnode()\n"));
> +
> + jeb = c->nextblock;
> +
> + if (!jeb->sum_collected->sum_num || !jeb->sum_collected->sum_list) {
> + printk(KERN_WARNING "JFFS2: jffs2_sum_write_sumnode(): empty summary info!!!\n");
> + BUG();
> + }
> +
> + datasize = jeb->sum_collected->sum_size + sizeof(struct jffs2_sum_marker);
> + infosize = sizeof(struct jffs2_summary_node) + datasize;
> + padsize = jeb->free_size - infosize;
> + infosize += padsize; datasize += padsize;
> + offset = cpu_to_je32(c->sector_size - jeb->free_size);
> +
> + if (padsize < 0) { // if jeb hasn't got enought free space for summary
> +
> + union jffs2_sum_mem *temp;
> +
> + while(jeb->sum_collected->sum_list){ //cleanup sum_list
> + temp = jeb->sum_collected->sum_list;
> + jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next;
> + kfree(temp);
> + jeb->sum_collected->sum_num--;
> + }
> +
> + jeb->sum_collected->sum_list = NULL;
> + jeb->sum_collected->sum_num = 0;
> + jeb->sum_collected->sum_size = JFFS2_SUMMARY_NOSUM_SIZE; // don't try to write out summary for this node
> +
> + printk(KERN_WARNING "JFFS2: not enough space for summary, padsize = %d\n",padsize);
> + return 0;
> + }
> +
> + memset(c->summary_buf, 0xff, datasize);
> + memset(&isum, 0, sizeof(isum));
> +
> + isum.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
> + isum.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY);
> + isum.totlen = cpu_to_je32(infosize);
> + isum.hdr_crc = cpu_to_je32(crc32(0, &isum, sizeof(struct jffs2_unknown_node) - 4));
> + isum.padded = cpu_to_je32(jeb->sum_collected->sum_padded);
> +
> + if (c->cleanmarker_size) {
> + isum.cln_mkr = cpu_to_je32(c->cleanmarker_size);
> + }
> + else{
> + isum.cln_mkr = cpu_to_je32(0);
> + }
> +
> + isum.sum_num = cpu_to_je16(jeb->sum_collected->sum_num);
> + wpage = c->summary_buf;
> +
> +
> + while (jeb->sum_collected->sum_num) { // write sum_data
> +
> +
> + switch(je16_to_cpu(jeb->sum_collected->sum_list->u.nodetype)){
> +
> + case JFFS2_NODETYPE_INODE : {
> + jint16_t *temp16ptr = (jint16_t *)wpage;
> +
> + *(temp16ptr++) = jeb->sum_collected->sum_list->i.nodetype;
> + wpage = (jint32_t *) temp16ptr;
> +
> + *(wpage++) = jeb->sum_collected->sum_list->i.inode;
> + *(wpage++) = jeb->sum_collected->sum_list->i.version;
> + *(wpage++) = jeb->sum_collected->sum_list->i.offset;
> + *(wpage++) = jeb->sum_collected->sum_list->i.totlen;
> + break;
> + }
> +
> + case JFFS2_NODETYPE_DIRENT : {
> + jint16_t *temp16ptr = (jint16_t *) wpage;
> + uint8_t *temp8ptr = NULL;
> +
> + *(temp16ptr++) = jeb->sum_collected->sum_list->d.nodetype;
> + wpage = (jint32_t *) temp16ptr;
> +
> + *(wpage++) = jeb->sum_collected->sum_list->d.totlen;
> + *(wpage++) = jeb->sum_collected->sum_list->d.offset;
> + *(wpage++) = jeb->sum_collected->sum_list->d.pino;
> + *(wpage++) = jeb->sum_collected->sum_list->d.version;
> + *(wpage++) = jeb->sum_collected->sum_list->d.ino;
> +
> + temp8ptr = (uint8_t *) wpage;
> + *(temp8ptr++) = jeb->sum_collected->sum_list->d.nsize;
> + *(temp8ptr++) = jeb->sum_collected->sum_list->d.type;
> +
> + memcpy(temp8ptr,jeb->sum_collected->sum_list->d.name,jeb->sum_collected->sum_list->d.nsize);
> + temp8ptr += jeb->sum_collected->sum_list->d.nsize;
> + wpage = (jint32_t *) temp8ptr;
> +
> + break;
> + }
> +
> + default : {
> + printk(KERN_WARNING "Unknown node in summary information!!!\n");
> + BUG();
> + }
> + }
> +
> + temp = jeb->sum_collected->sum_list;
> + jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next;
> + kfree(temp);
> +
> + jeb->sum_collected->sum_num--;
> + }
> +
> + jeb->sum_collected->sum_size = 0;
> +
> + tempptr = (uint8_t *) wpage;
> + tempptr += padsize;
> + wpage = (jint32_t *) tempptr;
> +
> + *(wpage++) = offset;
> + *(wpage++) = cpu_to_je32(c->sector_size);
> + *(wpage++) = magic;
> + isum.sum_crc = cpu_to_je32(crc32(0, c->summary_buf, datasize));
> + isum.node_crc = cpu_to_je32(crc32(0, &isum, sizeof(isum) - 8));
> +
> + vecs[0].iov_base = &isum;
> + vecs[0].iov_len = sizeof(isum);
> + vecs[1].iov_base = c->summary_buf;
> + vecs[1].iov_len = datasize;
> +
> + D1(printk(KERN_DEBUG "JFFS2: writing out data to flash to pos : 0x%08x\n",jeb->offset + c->sector_size - jeb->free_size));
> +
> + spin_unlock(&c->erase_completion_lock);
> + ret = jffs2_flash_writev(c, vecs, 2, jeb->offset + c->sector_size - jeb->free_size, &retlen, 0);
> + spin_lock(&c->erase_completion_lock);
> +
> +
> + if (ret || (retlen != infosize)) {
> + printk(KERN_WARNING "JFFS2: write of %zd bytes at 0x%08x failed. returned %d, retlen %zd\n",
> + infosize, jeb->offset + c->sector_size - jeb->free_size, ret, retlen);
> +
> + jeb->sum_collected->sum_size = JFFS2_SUMMARY_NOSUM_SIZE;
> +
> + WASTED_SPACE(infosize);
> +
> + return 0;
> + }
> +
> + /*spin_unlock(&c->erase_completion_lock);
> + jffs2_flush_wbuf_pad(c); // summary filled the wbuf
> + spin_lock(&c->erase_completion_lock);*/
> +
> + WASTED_SPACE(infosize);
> +
> + return 0;
> +}
> diff --unified --new-file --recursive mtd/fs/jffs2/summary.h mtd-dec07/fs/jffs2/summary.h
> --- mtd/fs/jffs2/summary.h 1970-01-01 01:00:00.000000000 +0100
> +++ mtd-dec07/fs/jffs2/summary.h 2004-12-07 16:03:02.000000000 +0100
> @@ -0,0 +1,158 @@
> +/*
> + * JFFS2 -- Journalling Flash File System, Version 2.
> + *
> + * Copyright (C) 2004 Ferenc Havasi <havasi@inf.u-szeged.hu>,
> + * Zoltan Sogor <weth@inf.u-szeged.hu>,
> + * Patrik Kluba <pajko@halom.u-szeged.hu>,
> + * University of Szeged, Hungary
> + *
> + * For licensing information, see the file 'LICENCE' in this directory.
> + *
> + * $Id$
> + *
> + */
> +
> +#ifndef JFFS2_SUMMARY_H
> +#define JFFS2_SUMMARY_H
> +
> +#include <linux/uio.h>
> +#include <linux/jffs2.h>
> +
> +#define DIRTY_SPACE(x) do { typeof(x) _x = (x); \
> + c->free_size -= _x; c->dirty_size += _x; \
> + jeb->free_size -= _x ; jeb->dirty_size += _x; \
> + }while(0)
> +#define USED_SPACE(x) do { typeof(x) _x = (x); \
> + c->free_size -= _x; c->used_size += _x; \
> + jeb->free_size -= _x ; jeb->used_size += _x; \
> + }while(0)
> +#define WASTED_SPACE(x) do { typeof(x) _x = (x); \
> + c->free_size -= _x; c->wasted_size += _x; \
> + jeb->free_size -= _x ; jeb->wasted_size += _x; \
> + }while(0)
> +#define UNCHECKED_SPACE(x) do { typeof(x) _x = (x); \
> + c->free_size -= _x; c->unchecked_size += _x; \
> + jeb->free_size -= _x ; jeb->unchecked_size += _x; \
> + }while(0)
> +
> +#define BLK_STATE_ALLFF 0
> +#define BLK_STATE_CLEAN 1
> +#define BLK_STATE_PARTDIRTY 2
> +#define BLK_STATE_CLEANMARKER 3
> +#define BLK_STATE_ALLDIRTY 4
> +#define BLK_STATE_BADBLOCK 5
> +
> +#define JFFS2_SUMMARY_NOSUM_SIZE 0xffffffff
> +#define JFFS2_SUMMARY_INODE_SIZE (sizeof(struct jffs2_sum_inode_flash))
> +#define JFFS2_SUMMARY_DIRENT_SIZE(x) (sizeof(struct jffs2_sum_dirent_flash) + (x))
> +
> +struct jffs2_sum_unknown_flash
> +{
> + jint16_t nodetype; /* node type */
> +};
> +
> +struct jffs2_sum_inode_flash
> +{
> + jint16_t nodetype; /* node type */
> + jint32_t inode; /* inode number */
> + jint32_t version; /* inode version */
> + jint32_t offset; /* offset on jeb */
> + jint32_t totlen; /* record length */
> +} __attribute__((packed));
> +
> +struct jffs2_sum_dirent_flash
> +{
> + jint16_t nodetype; /* == JFFS_NODETYPE_DIRENT */
> + jint32_t totlen; /* record length */
> + jint32_t offset; /* ofset on jeb */
> + jint32_t pino; /* parent inode */
> + jint32_t version; /* dirent version */
> + jint32_t ino; /* == zero for unlink */
> + uint8_t nsize; /* dirent name size */
> + uint8_t type; /* dirent type */
> + uint8_t name[0]; /* dirent name */
> +} __attribute__((packed));
> +
> +union jffs2_sum_flash{
> + struct jffs2_sum_unknown_flash u;
> + struct jffs2_sum_inode_flash i;
> + struct jffs2_sum_dirent_flash d;
> +};
> +
> +/* list version of jffs2_sum_*flash for kernel and sumtool */
> +struct jffs2_sum_unknown_mem
> +{
> + union jffs2_sum_mem *next;
> + jint16_t nodetype; /* node type */
> +
> +};
> +
> +struct jffs2_sum_inode_mem
> +{
> + union jffs2_sum_mem *next;
> + jint16_t nodetype; /* node type */
> + jint32_t inode; /* inode number */
> + jint32_t version; /* inode version */
> + jint32_t offset; /* offset on jeb */
> + jint32_t totlen; /* record length */
> +} __attribute__((packed));
> +
> +struct jffs2_sum_dirent_mem
> +{
> + union jffs2_sum_mem *next;
> + jint16_t nodetype; /* == JFFS_NODETYPE_DIRENT */
> + jint32_t totlen; /* record length */
> + jint32_t offset; /* ofset on jeb */
> + jint32_t pino; /* parent inode */
> + jint32_t version; /* dirent version */
> + jint32_t ino; /* == zero for unlink */
> + uint8_t nsize; /* dirent name size */
> + uint8_t type; /* dirent type */
> + uint8_t name[0]; /* dirent name */
> +} __attribute__((packed));
> +
> +union jffs2_sum_mem
> +{
> + struct jffs2_sum_unknown_mem u;
> + struct jffs2_sum_inode_mem i;
> + struct jffs2_sum_dirent_mem d;
> +};
> +
> +struct jffs2_sum_info
> +{
> + uint32_t sum_size;
> + uint32_t sum_num;
> + uint32_t sum_padded;
> + union jffs2_sum_mem *sum_list;
> +};
> +
> +struct jffs2_sum_marker
> +{
> + jint32_t offset;
> + jint32_t erase_size;
> + jint32_t magic;
> +};
> +
> +#define JFFS2_SUMMARY_FRAME_SIZE (sizeof(struct jffs2_summary_node)+sizeof(struct jffs2_sum_marker)+3)
> +
> +#if !(defined(SUM_TOOL) || defined(JFFS2DUMP))
> +
> +int jffs2_sum_init(struct jffs2_sb_info *c);
> +void jffs2_sum_exit(struct jffs2_sb_info *c);
> +
> +int jffs2_sum_care_sum_collected(struct jffs2_eraseblock *jeb);
> +
> +void jffs2_sum_clean_collected(struct jffs2_eraseblock *jeb);
> +void jffs2_sum_clean_all_info(struct jffs2_sb_info *c); /* clean up all summary information in all jeb (umount) */
> +
> +int jffs2_sum_add_padding_mem(struct jffs2_eraseblock *jeb, uint32_t size);
> +int jffs2_sum_add_inode_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_inode *ri, uint32_t ofs);
> +int jffs2_sum_add_dirent_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_dirent *rd, uint32_t ofs);
> +int jffs2_sum_add_kvec(struct jffs2_sb_info *c, const struct kvec *invecs, unsigned long count, uint32_t to);
> +
> +int jffs2_sum_scan_sumnode(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb, uint32_t ofs, uint32_t *pseudo_random);
> +int jffs2_sum_write_sumnode(struct jffs2_sb_info *c);
> +
> +#endif
> +
> +#endif /* JFFS2_SUMMARY_H */
> diff --unified --new-file --recursive mtd/fs/jffs2/super-v24.c mtd-dec07/fs/jffs2/super-v24.c
> --- mtd/fs/jffs2/super-v24.c 2004-11-17 00:00:14.000000000 +0100
> +++ mtd-dec07/fs/jffs2/super-v24.c 2004-12-07 16:03:02.000000000 +0100
> @@ -24,6 +24,7 @@
> #include <linux/mtd/mtd.h>
> #include "compr.h"
> #include "nodelist.h"
> +#include "summary.h"
>
> #ifndef MTD_BLOCK_MAJOR
> #define MTD_BLOCK_MAJOR 31
> @@ -72,6 +73,9 @@
> put_mtd_device(c->mtd);
> return NULL;
> }
> +#ifdef CONFIG_JFFS2_SUMMARY
> + jffs2_sum_init(c);
> +#endif
>
> return sb;
> }
> @@ -88,6 +92,10 @@
> down(&c->alloc_sem);
> jffs2_flush_wbuf_pad(c);
> up(&c->alloc_sem);
> +#ifdef CONFIG_JFFS2_SUMMARY
> + jffs2_sum_clean_all_info(c);
> + jffs2_sum_exit(c);
> +#endif
> jffs2_free_ino_caches(c);
> jffs2_free_raw_node_refs(c);
> kfree(c->blocks);
> @@ -110,6 +118,9 @@
> #ifdef CONFIG_JFFS2_FS_NAND
> " (NAND)"
> #endif
> +#ifdef CONFIG_JFFS2_SUMMARY
> + " (SUMMARY)"
> +#endif
> " (C) 2001-2003 Red Hat, Inc.\n");
>
> #ifdef JFFS2_OUT_OF_KERNEL
> diff --unified --new-file --recursive mtd/fs/jffs2/super.c mtd-dec07/fs/jffs2/super.c
> --- mtd/fs/jffs2/super.c 2004-11-26 15:08:31.000000000 +0100
> +++ mtd-dec07/fs/jffs2/super.c 2004-12-07 16:03:02.000000000 +0100
> @@ -152,6 +152,10 @@
> return ERR_PTR(ret);
> }
>
> +#ifdef CONFIG_JFFS2_SUMMARY
> + jffs2_sum_init(c);
> +#endif
> +
> sb->s_flags |= MS_ACTIVE;
> return sb;
>
> @@ -275,6 +279,10 @@
> down(&c->alloc_sem);
> jffs2_flush_wbuf_pad(c);
> up(&c->alloc_sem);
> +#ifdef CONFIG_JFFS2_SUMMARY
> + jffs2_sum_clean_all_info(c);
> + jffs2_sum_exit(c);
> +#endif
> jffs2_free_ino_caches(c);
> jffs2_free_raw_node_refs(c);
> if (c->mtd->flags & MTD_NO_VIRTBLOCKS)
> @@ -312,6 +320,9 @@
> #ifdef CONFIG_JFFS2_FS_NAND
> " (NAND)"
> #endif
> +#ifdef CONFIG_JFFS2_SUMMARY
> + " (SUMMARY) "
> +#endif
> " (C) 2001-2003 Red Hat, Inc.\n");
>
> jffs2_inode_cachep = kmem_cache_create("jffs2_i",
> diff --unified --new-file --recursive mtd/fs/jffs2/wbuf.c mtd-dec07/fs/jffs2/wbuf.c
> --- mtd/fs/jffs2/wbuf.c 2004-11-26 15:08:31.000000000 +0100
> +++ mtd-dec07/fs/jffs2/wbuf.c 2004-12-07 16:17:11.000000000 +0100
> @@ -261,7 +261,7 @@
>
>
> /* ... and get an allocation of space from a shiny new block instead */
> - ret = jffs2_reserve_space_gc(c, end-start, &ofs, &len);
> + ret = jffs2_reserve_space_gc(c, end-start, &ofs, &len, JFFS2_SUMMARY_NOSUM_SIZE);
> if (ret) {
> printk(KERN_WARNING "Failed to allocate space for wbuf recovery. Data loss ensues.\n");
> if (buf)
> @@ -601,8 +601,15 @@
> uint32_t outvec_to = to;
>
> /* If not NAND flash, don't bother */
> - if (!c->wbuf)
> + if (!c->wbuf){
> +#ifdef CONFIG_JFFS2_SUMMARY
> + D1(printk("JFFS2.summary: NOT NAND MEMORY\n"));
> + if(jffs2_sum_add_kvec(c,invecs,count,(uint32_t) to)){
> + printk("jffs2_sum_add_kvec(): MEMORY ALLOCATION ERROR!");
> + }
> +#endif
> return jffs2_flash_direct_writev(c, invecs, count, to, retlen);
> + }
>
> down_write(&c->wbuf_sem);
>
> @@ -801,6 +808,12 @@
> alldone:
> *retlen = donelen;
>
> +#ifdef CONFIG_JFFS2_SUMMARY
> + if(jffs2_sum_add_kvec(c,invecs,count,(uint32_t) to)){
> + printk("jffs2_sum_add_kvec(): MEMORY ALLOCATION ERROR!");
> + }
> +#endif
> +
> if (c->wbuf_len && ino)
> jffs2_wbuf_dirties_inode(c, ino);
>
> diff --unified --new-file --recursive mtd/fs/jffs2/write.c mtd-dec07/fs/jffs2/write.c
> --- mtd/fs/jffs2/write.c 2004-11-17 00:00:14.000000000 +0100
> +++ mtd-dec07/fs/jffs2/write.c 2004-12-07 16:03:02.000000000 +0100
> @@ -173,13 +173,13 @@
> D1(ACCT_PARANOIA_CHECK(jeb));
>
> if (alloc_mode == ALLOC_GC) {
> - ret = jffs2_reserve_space_gc(c, sizeof(*ri) + datalen, &flash_ofs, &dummy);
> + ret = jffs2_reserve_space_gc(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, JFFS2_SUMMARY_INODE_SIZE);
> } else {
> /* Locking pain */
> up(&f->sem);
> jffs2_complete_reservation(c);
>
> - ret = jffs2_reserve_space(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, alloc_mode);
> + ret = jffs2_reserve_space(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, alloc_mode, JFFS2_SUMMARY_INODE_SIZE);
> down(&f->sem);
> }
>
> @@ -308,13 +308,13 @@
> D1(ACCT_PARANOIA_CHECK(jeb));
>
> if (alloc_mode == ALLOC_GC) {
> - ret = jffs2_reserve_space_gc(c, sizeof(*rd) + namelen, &flash_ofs, &dummy);
> + ret = jffs2_reserve_space_gc(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
> } else {
> /* Locking pain */
> up(&f->sem);
> jffs2_complete_reservation(c);
>
> - ret = jffs2_reserve_space(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, alloc_mode);
> + ret = jffs2_reserve_space(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, alloc_mode, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
> down(&f->sem);
> }
>
> @@ -371,7 +371,7 @@
> retry:
> D2(printk(KERN_DEBUG "jffs2_commit_write() loop: 0x%x to write to 0x%x\n", writelen, offset));
>
> - ret = jffs2_reserve_space(c, sizeof(*ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, ALLOC_NORMAL);
> + ret = jffs2_reserve_space(c, sizeof(*ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
> if (ret) {
> D1(printk(KERN_DEBUG "jffs2_reserve_space returned %d\n", ret));
> break;
> @@ -458,7 +458,7 @@
> /* Try to reserve enough space for both node and dirent.
> * Just the node will do for now, though
> */
> - ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL);
> + ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
> D1(printk(KERN_DEBUG "jffs2_do_create(): reserved 0x%x bytes\n", alloclen));
> if (ret) {
> up(&f->sem);
> @@ -487,7 +487,7 @@
>
> up(&f->sem);
> jffs2_complete_reservation(c);
> - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
> + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
>
> if (ret) {
> /* Eep. */
> @@ -557,7 +557,7 @@
> if (!rd)
> return -ENOMEM;
>
> - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_DELETION);
> + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_DELETION, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
> if (ret) {
> jffs2_free_raw_dirent(rd);
> return ret;
> @@ -663,7 +663,7 @@
> if (!rd)
> return -ENOMEM;
>
> - ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
> + ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
> if (ret) {
> jffs2_free_raw_dirent(rd);
> return ret;
> diff --unified --new-file --recursive mtd/include/linux/jffs2.h mtd-dec07/include/linux/jffs2.h
> --- mtd/include/linux/jffs2.h 2004-11-17 00:00:15.000000000 +0100
> +++ mtd-dec07/include/linux/jffs2.h 2004-12-07 16:03:02.000000000 +0100
> @@ -28,6 +28,9 @@
> #define JFFS2_EMPTY_BITMASK 0xffff
> #define JFFS2_DIRTY_BITMASK 0x0000
>
> +/* Summary node MAGIC marker */
> +#define JFFS2_SUM_MAGIC 0x02851885
> +
> /* We only allow a single char for length, and 0xFF is empty flash so
> we don't want it confused with a real length. Hence max 254.
> */
> @@ -62,6 +65,8 @@
> #define JFFS2_NODETYPE_CLEANMARKER (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3)
> #define JFFS2_NODETYPE_PADDING (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 4)
>
> +#define JFFS2_NODETYPE_SUMMARY (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 6)
> +
> // Maybe later...
> //#define JFFS2_NODETYPE_CHECKPOINT (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3)
> //#define JFFS2_NODETYPE_OPTIONS (JFFS2_FEATURE_RWCOMPAT_COPY | JFFS2_NODE_ACCURATE | 4)
> @@ -148,10 +153,24 @@
> uint8_t data[0];
> } __attribute__((packed));
>
> +struct jffs2_summary_node{
> + jint16_t magic;
> + jint16_t nodetype; /* = JFFS2_NODETYPE_INODE_SUM */
> + jint32_t totlen;
> + jint32_t hdr_crc;
> + jint16_t sum_num; /* number of sum entries*/
> + jint32_t cln_mkr; /* clean marker size, 0 = no cleanmarker */
> + jint32_t padded; /* sum of the size of padding nodes */
> + jint32_t sum_crc; /* summary information crc */
> + jint32_t node_crc; /* node crc */
> + jint32_t sum[0]; /* inode summary info */
> +} __attribute__((packed));
> +
> union jffs2_node_union {
> struct jffs2_raw_inode i;
> struct jffs2_raw_dirent d;
> struct jffs2_unknown_node u;
> + struct jffs2_summary_node s;
> };
>
> #endif /* __LINUX_JFFS2_H__ */
> diff --unified --new-file --recursive mtd/include/linux/jffs2_fs_sb.h mtd-dec07/include/linux/jffs2_fs_sb.h
> --- mtd/include/linux/jffs2_fs_sb.h 2004-11-26 15:08:31.000000000 +0100
> +++ mtd-dec07/include/linux/jffs2_fs_sb.h 2004-12-07 16:03:02.000000000 +0100
> @@ -110,6 +110,9 @@
> uint32_t fsdata_pos;
> uint32_t fsdata_len;
> #endif
> +#ifdef CONFIG_JFFS2_SUMMARY
> + jint32_t *summary_buf;
> +#endif
>
> /* OS-private pointer for getting back to master superblock info */
> void *os_priv;
> diff --unified --new-file --recursive mtd/util/Makefile mtd-dec07/util/Makefile
> --- mtd/util/Makefile 2004-07-14 00:00:18.000000000 +0200
> +++ mtd-dec07/util/Makefile 2004-12-07 16:03:02.000000000 +0100
> @@ -13,7 +13,7 @@
> TARGETS = ftl_format flash_erase flash_eraseall nanddump doc_loadbios \
> mkfs.jffs ftl_check mkfs.jffs2 flash_lock flash_unlock \
> flash_info mtd_debug flashcp nandwrite jffs2dump \
> - nftldump nftl_format docfdisk #jffs2reader
> + nftldump nftl_format docfdisk sumtool #jffs2reader
>
> SYMLINKS = compr_lzari.c compr_lzo.c
>
> @@ -48,6 +48,10 @@
> jffs2dump: jffs2dump.o crc32.o
> $(CC) $(LDFLAGS) -o $@ $^
>
> +sumtool: sumtool.o crc32.o
> + $(CC) $(LDFLAGS) -o $@ $^
> +
> +
> install: ${TARGETS}
> mkdir -p ${DESTDIR}/${SBINDIR}
> install -m0755 -oroot -groot ${TARGETS} ${DESTDIR}/${SBINDIR}/
> diff --unified --new-file --recursive mtd/util/jffs2dump.c mtd-dec07/util/jffs2dump.c
> --- mtd/util/jffs2dump.c 2004-12-03 00:00:10.000000000 +0100
> +++ mtd-dec07/util/jffs2dump.c 2004-12-07 16:03:02.000000000 +0100
> @@ -35,6 +35,10 @@
> #include <getopt.h>
> #include "crc32.h"
>
> +#define CONFIG_JFFS2_SUMMARY
> +#define JFFS2DUMP
> +#include "../fs/jffs2/summary.h"
> +
> #define PROGRAM "jffs2dump"
> #define VERSION "$Revision: 1.7 $"
>
> @@ -177,7 +181,7 @@
> */
> void do_dumpcontent (void)
> {
> - char *p = data;
> + char *p = data, *p_free_begin;
> union jffs2_node_union *node;
> int empty = 0, dirty = 0;
> char name[256];
> @@ -186,16 +190,21 @@
> int bitchbitmask = 0;
> int obsolete;
>
> + p_free_begin = NULL;
> while ( p < (data + imglen)) {
> node = (union jffs2_node_union*) p;
>
> /* Skip empty space */
> + if (!p_free_begin) p_free_begin = p;
> if (je16_to_cpu (node->u.magic) == 0xFFFF && je16_to_cpu (node->u.nodetype) == 0xFFFF) {
> p += 4;
> empty += 4;
> continue;
> }
> -
> +
> + if (p != p_free_begin) printf("Empty space found from 0x%08x to 0x%08x\n", p_free_begin-data, p-data);
> + p_free_begin = NULL;
> +
> if (je16_to_cpu (node->u.magic) != JFFS2_MAGIC_BITMASK) {
> if (!bitchbitmask++)
> printf ("Wrong bitmask at 0x%08x, 0x%04x\n", p - data, je16_to_cpu (node->u.magic));
> @@ -277,7 +286,110 @@
>
> p += PAD(je32_to_cpu (node->d.totlen));
> break;
> -
> +
> + case JFFS2_NODETYPE_SUMMARY:{
> +
> + int i;
> + struct jffs2_sum_marker * sm;
> +
> + printf ("%8s Inode Sum node at 0x%08x, totlen 0x%08x, sum_num %5d, cleanmarker size %5d\n",
> + obsolete ? "Obsolete" : "",
> + p - data,
> + je32_to_cpu (node->s.totlen),
> + je16_to_cpu (node->s.sum_num),
> + je32_to_cpu (node->s.cln_mkr));
> +
> + crc = crc32 (0, node, sizeof (struct jffs2_summary_node) - 8);
> + if (crc != je32_to_cpu (node->s.node_crc)) {
> + printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.node_crc), crc);
> + p += PAD(je32_to_cpu (node->s.totlen));
> + dirty += PAD(je32_to_cpu (node->s.totlen));;
> + continue;
> + }
> +
> + crc = crc32(0, p + sizeof (struct jffs2_summary_node), je32_to_cpu (node->s.totlen) - sizeof(struct jffs2_summary_node));
> + if (crc != je32_to_cpu(node->s.sum_crc)) {
> + printf ("Wrong data_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.sum_crc), crc);
> + p += PAD(je32_to_cpu (node->s.totlen));
> + dirty += PAD(je32_to_cpu (node->s.totlen));;
> + continue;
> + }
> +
> + if(verbose){
> + struct jffs2_sum_unknown_flash *sp;
> + uint8_t *temp8ptr = NULL;
> + sp = (struct jffs2_sum_unknown_flash *) (p + sizeof (struct jffs2_summary_node));
> +
> + for(i = 0; i < je16_to_cpu (node->s.sum_num); i++){
> +
> + switch(je16_to_cpu(sp->nodetype)){
> + case JFFS2_NODETYPE_INODE : {
> +
> + struct jffs2_sum_inode_flash *spi;
> + spi = (struct jffs2_sum_inode_flash *) sp;
> +
> + printf ("%14s #ino %5d, version %5d, offset 0x%08x, totlen 0x%08x\n",
> + "",
> + je32_to_cpu (spi->inode),
> + je32_to_cpu (spi->version),
> + je32_to_cpu (spi->offset),
> + je32_to_cpu (spi->totlen));
> +
> + temp8ptr = (uint8_t *) sp;
> + temp8ptr += JFFS2_SUMMARY_INODE_SIZE;
> + sp = (struct jffs2_sum_unknown_flash *) temp8ptr;
> +
> + break;
> + }
> +
> + case JFFS2_NODETYPE_DIRENT : {
> +
> + char name[255];
> + struct jffs2_sum_dirent_flash *spd;
> + spd = (struct jffs2_sum_dirent_flash *) sp;
> +
> + memcpy(name,spd->name,spd->nsize);
> + name [spd->nsize] = 0x0;
> +
> + printf ("%14s dirent offset 0x%08x, totlen 0x%08x, #pino %5d, version %5d, #ino %8d, nsize %8d, name %s \n",
> + "",
> + je32_to_cpu (spd->offset),
> + je32_to_cpu (spd->totlen),
> + je32_to_cpu (spd->pino),
> + je32_to_cpu (spd->version),
> + je32_to_cpu (spd->ino),
> + spd->nsize,
> + name);
> +
> + temp8ptr = (uint8_t *) sp;
> + temp8ptr += JFFS2_SUMMARY_DIRENT_SIZE(spd->nsize);
> + sp = (struct jffs2_sum_unknown_flash *) temp8ptr;
> +
> + break;
> + }
> +
> + default : {
> + break;
> + }
> + }
> + }
> +
> +
> + sm = (struct jffs2_sum_marker *) ((char *)p + je32_to_cpu(node->s.totlen) - sizeof(struct jffs2_sum_marker));
> +
> + printf("%14s Sum Node Offset 0x%08x, Erase_block_size 0x%08x, Magic 0x%08x, Padded size 0x%08x\n" ,
> + "",
> + je32_to_cpu(sm->offset),
> + je32_to_cpu(sm->erase_size),
> + je32_to_cpu(sm->magic),
> + je32_to_cpu(node->s.padded));
> + }
> +
> + p += PAD(je32_to_cpu (node->s.totlen));
> + break;
> +
> + }
> +
> case JFFS2_NODETYPE_CLEANMARKER:
> if (verbose) {
> printf ("%8s Cleanmarker at 0x%08x, totlen 0x%08x\n",
> diff --unified --new-file --recursive mtd/util/sumtool.c mtd-dec07/util/sumtool.c
> --- mtd/util/sumtool.c 1970-01-01 01:00:00.000000000 +0100
> +++ mtd-dec07/util/sumtool.c 2004-12-07 17:47:31.000000000 +0100
> @@ -0,0 +1,857 @@
> +/*
> + * sumtool.c
> + *
> + * Copyright (C) 2004 Zoltan Sogor <weth@inf.u-szeged.hu>,
> + * Ferenc Havasi <havasi@inf.u-szeged.hu>,
> + * Patrik Kluba <pajko@halom.u-szeged.hu>,
> + * University of Szeged, Hungary
> + *
> + * $Id: sumtool.c,v 1.2 2004/10/20 09:56:08 hafy Exp $
> + *
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; either version 2
> + * of the License, or (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program; if not, write to the Free Software
> + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
> + *
> + * Overview:
> + * This is a utility to reorder nodes and insert inode summary information
> + * into JFFS2 image for faster mount time - specially on NAND.
> + *
> + */
> +
> +#include <errno.h>
> +#include <stdint.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <stdarg.h>
> +#include <string.h>
> +#include <unistd.h>
> +#include <fcntl.h>
> +#include <time.h>
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <sys/param.h>
> +#include <asm/types.h>
> +#include <dirent.h>
> +#include <mtd/jffs2-user.h>
> +#include <endian.h>
> +#include <byteswap.h>
> +#include <getopt.h>
> +#include "crc32.h"
> +
> +#define CONFIG_JFFS2_SUMMARY
> +#define SUM_TOOL
> +#include "../fs/jffs2/summary.h"
> +
> +
> +#define PAD(x) (((x)+3)&~3)
> +
> +static const char *const app_name = "sumtool";
> +
> +static struct jffs2_sum_info *sum_collected = NULL;/* summary info list */
> +
> +static int verbose = 0;
> +static int padto = 0; /* pad the output with 0xFF to the end of the final eraseblock */
> +static int add_cleanmarkers = 1; /* add cleanmarker to output */
> +static int use_input_cleanmarker_size = 1; /* use input file's cleanmarker size (default) */
> +static int found_cleanmarkers = 0; /* cleanmarker found in input file */
> +static struct jffs2_unknown_node cleanmarker;
> +static int cleanmarker_size = sizeof(cleanmarker);
> +static const char *short_options = "o:i:e:hvVblnc:p";
> +static int erase_block_size = 65536;
> +static int target_endian = __BYTE_ORDER;
> +static int out_fd = -1;
> +static int in_fd = -1;
> +
> +static uint8_t *data_buffer = NULL; /* buffer for inodes */
> +static unsigned int data_ofs = 0; /* inode buffer offset */
> +
> +static uint8_t *file_buffer = NULL; /* file buffer contains the actual erase block*/
> +static unsigned int file_ofs = 0; /* position in the buffer */
> +
> +static struct option long_options[] = {
> + {"output", 1, NULL, 'o'},
> + {"input", 1, NULL, 'i'},
> + {"eraseblock", 1, NULL, 'e'},
> + {"help", 0, NULL, 'h'},
> + {"verbose", 0, NULL, 'v'},
> + {"version", 0, NULL, 'V'},
> + {"bigendian", 0, NULL, 'b'},
> + {"littleendian", 0, NULL, 'l'},
> + {"no-cleanmarkers", 0, NULL, 'n'},
> + {"cleanmarker", 1, NULL, 'c'},
> + {"pad", 0, NULL, 'p'},
> + {NULL, 0, NULL, 0}
> +};
> +
> +static char *helptext =
> + "Usage: sumtool [OPTIONS] -i inputfile -o outputfile\n"
> + "Convert the input JFFS2 file to a SUM-ed JFFS2 file\n\n"
> + "Options:\n"
> + " -e, --eraseblock=SIZE Use erase block size SIZE (default: 64KiB)\n"
> + " (usually 16KiB on NAND)\n"
> + " -c, --cleanmarker=SIZE Size of cleanmarker (default 12).\n"
> + " (usually 16 bytes on NAND, and will be set to\n"
> + " this value if left at the default 12). Will be\n"
> + " stored in OOB after each physical page composing\n"
> + " a physical erase block.\n"
> + " -n, --no-cleanmarkers Don't add a cleanmarker to every eraseblock\n"
> + " -o, --output=FILE Output to FILE \n"
> + " -i, --input=FILE Input from FILE \n"
> + " -b, --bigendian Image is big endian\n"
> + " -l --littleendian Image is little endian\n"
> + " -h, --help Display this help text\n"
> + " -v, --verbose Verbose operation\n"
> + " -V, --version Display version information\n"
> + " -p, --pad Pad the OUTPUT with 0xFF to the end of the final\n"
> + " eraseblock\n\n";
> +
> +
> +static char *revtext = "$Revision: 1.3 $";
> +
> +static unsigned char ffbuf[16] = {
> + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
> + 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff
> +};
> +
> +static void verror_msg(const char *s, va_list p) {
> + fflush(stdout);
> + fprintf(stderr, "%s: ", app_name);
> + vfprintf(stderr, s, p);
> +}
> +
> +static void error_msg_and_die(const char *s, ...) {
> + va_list p;
> +
> + va_start(p, s);
> + verror_msg(s, p);
> + va_end(p);
> + putc('\n', stderr);
> + exit(EXIT_FAILURE);
> +}
> +
> +static void vperror_msg(const char *s, va_list p) {
> + int err = errno;
> +
> + if (s == 0)
> + s = "";
> + verror_msg(s, p);
> + if (*s)
> + s = ": ";
> + fprintf(stderr, "%s%s\n", s, strerror(err));
> +}
> +
> +static void perror_msg_and_die(const char *s, ...) {
> + va_list p;
> +
> + va_start(p, s);
> + vperror_msg(s, p);
> + va_end(p);
> + exit(EXIT_FAILURE);
> +}
> +
> +
> +
> +static void full_write(void *target_buff, const void *buf, int len);
> +
> +void setup_cleanmarker() {
> +
> + cleanmarker.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
> + cleanmarker.nodetype = cpu_to_je16(JFFS2_NODETYPE_CLEANMARKER);
> + cleanmarker.totlen = cpu_to_je32(cleanmarker_size);
> + cleanmarker.hdr_crc = cpu_to_je32(crc32(0, &cleanmarker, sizeof(struct jffs2_unknown_node)-4));
> +}
> +
> +void process_options (int argc, char **argv){
> + int opt,c;
> +
> + while ((opt = getopt_long(argc, argv, short_options, long_options, &c)) >= 0)
> + {
> + switch (opt)
> + {
> + case 'o':
> + if (out_fd != -1) {
> + error_msg_and_die("output filename specified more than once");
> + }
> + out_fd = open(optarg, O_CREAT | O_TRUNC | O_RDWR, 0644);
> + if (out_fd == -1) {
> + perror_msg_and_die("open output file");
> + }
> + break;
> +
> + case 'i':
> + if (in_fd != -1) {
> + error_msg_and_die("input filename specified more than once");
> + }
> + in_fd = open(optarg, O_RDONLY);
> + if (in_fd == -1) {
> + perror_msg_and_die("open input file");
> + }
> + break;
> + case 'b':
> + target_endian = __BIG_ENDIAN;
> + break;
> + case 'l':
> + target_endian = __LITTLE_ENDIAN;
> + break;
> + case 'h':
> + case '?':
> + error_msg_and_die(helptext);
> +
> + case 'v':
> + verbose = 1;
> + break;
> +
> + case 'V':
> + error_msg_and_die("revision %.*s\n",
> + (int) strlen(revtext) - 13, revtext + 11);
> +
> + case 'e': {
> + char *next;
> + unsigned units = 0;
> + erase_block_size = strtol(optarg, &next, 0);
> + if (!erase_block_size)
> + error_msg_and_die("Unrecognisable erase size\n");
> +
> + if (*next) {
> + if (!strcmp(next, "KiB")) {
> + units = 1024;
> + } else if (!strcmp(next, "MiB")) {
> + units = 1024 * 1024;
> + } else {
> + error_msg_and_die("Unknown units in erasesize\n");
> + }
> + } else {
> + if (erase_block_size < 0x1000)
> + units = 1024;
> + else
> + units = 1;
> + }
> + erase_block_size *= units;
> +
> + /* If it's less than 8KiB, they're not allowed */
> + if (erase_block_size < 0x2000) {
> + fprintf(stderr, "Erase size 0x%x too small. Increasing to 8KiB minimum\n",
> + erase_block_size);
> + erase_block_size = 0x2000;
> + }
> + break;
> + }
> +
> + case 'n':
> + add_cleanmarkers = 0;
> + break;
> + case 'c':
> + cleanmarker_size = strtol(optarg, NULL, 0);
> +
> + if (cleanmarker_size < sizeof(cleanmarker)) {
> + error_msg_and_die("cleanmarker size must be >= 12");
> + }
> + if (cleanmarker_size >= erase_block_size) {
> + error_msg_and_die("cleanmarker size must be < eraseblock size");
> + }
> +
> + use_input_cleanmarker_size = 0;
> + found_cleanmarkers = 1;
> + setup_cleanmarker();
> +
> + break;
> + case 'p':
> + padto = 1;
> + break;
> +
> + }
> + }
> +}
> +
> +
> +void init_buffers() {
> +
> + data_buffer = malloc(erase_block_size);
> +
> + if (!data_buffer) {
> + perror("out of memory");
> + close (in_fd);
> + close (out_fd);
> + exit(1);
> + }
> +
> + file_buffer = malloc(erase_block_size);
> +
> + if (!file_buffer) {
> + perror("out of memory");
> + close (in_fd);
> + close (out_fd);
> + exit(1);
> + }
> +}
> +
> +void init_sumlist(){
> +
> + sum_collected = (struct jffs2_sum_info *) malloc (sizeof(struct jffs2_sum_info));
> +
> + if (!sum_collected)
> + error_msg_and_die("Can't allocate memory for jffs2_sum_info!\n");
> +
> + sum_collected->sum_list = NULL;
> + sum_collected->sum_num = 0;
> + sum_collected->sum_size = 0;
> +
> +}
> +
> +void clean_buffers() {
> +
> + if (data_buffer)
> + free(data_buffer);
> + if (file_buffer)
> + free(file_buffer);
> +}
> +
> +void clean_sumlist() {
> +
> + union jffs2_sum_mem *temp;
> +
> + if(sum_collected){
> +
> + while(sum_collected->sum_list){
> + temp = sum_collected->sum_list;
> + sum_collected->sum_list = sum_collected->sum_list->u.next;
> + free(temp);
> + sum_collected->sum_num--;
> + }
> +
> + if(sum_collected->sum_num != 0)
> + printf("Ooops, something wrong happened! sum_num != 0, but sum_list = null ???");
> +
> + free(sum_collected);
> + }
> +}
> +
> +int load_next_block() {
> +
> + int ret;
> + ret = read(in_fd, file_buffer, erase_block_size);
> + file_ofs = 0;
> +
> + if(verbose)
> + printf("Load next block : %d bytes read\n",ret);
> +
> + return ret;
> +}
> +
> +void write_buff_to_file() {
> +
> + int ret;
> + int len = data_ofs;
> +
> + uint8_t *buf = NULL;
> +
> + buf = data_buffer;
> + while (len > 0) {
> + ret = write(out_fd, buf, len);
> +
> + if (ret < 0)
> + perror_msg_and_die("write");
> +
> + if (ret == 0)
> + perror_msg_and_die("write returned zero");
> +
> + len -= ret;
> + buf += ret;
> + }
> +
> + data_ofs = 0;
> +}
> +
> +void dump_sum_records() {
> +
> + struct jffs2_summary_node isum;
> + union jffs2_sum_mem *temp;
> + jint32_t offset;
> + jint32_t *wpage;
> + uint8_t *tempptr;
> + int datasize;
> + int infosize;
> + int padsize;
> + jint32_t magic = cpu_to_je32(JFFS2_SUM_MAGIC);
> +
> + if (!sum_collected->sum_num || !sum_collected->sum_list)
> + return;
> +
> + datasize = sum_collected->sum_size + sizeof(struct jffs2_sum_marker);
> + infosize = sizeof(struct jffs2_summary_node) + datasize;
> + padsize = erase_block_size - data_ofs - infosize;
> + infosize += padsize; datasize += padsize;
> + offset = cpu_to_je32(data_ofs);
> +
> + jint32_t *tpage = (jint32_t *) malloc(datasize);
> +
> + if(!tpage)
> + error_msg_and_die("Can't allocate memory to dump summary information!\n");
> +
> + memset(tpage, 0xff, datasize);
> + memset(&isum, 0, sizeof(isum));
> +
> + isum.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
> + isum.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY);
> + isum.totlen = cpu_to_je32(infosize);
> + isum.hdr_crc = cpu_to_je32(crc32(0, &isum, sizeof(struct jffs2_unknown_node) - 4));
> + isum.padded = cpu_to_je32(0);
> +
> + if (add_cleanmarkers && found_cleanmarkers) {
> + isum.cln_mkr = cpu_to_je32(cleanmarker_size);
> + }
> + else{
> + isum.cln_mkr = cpu_to_je32(0);
> + }
> +
> + isum.sum_num = cpu_to_je16(sum_collected->sum_num);
> + wpage = tpage;
> +
> +
> + while (sum_collected->sum_num) {
> +
> +
> + switch(je16_to_cpu(sum_collected->sum_list->u.nodetype)){
> +
> + case JFFS2_NODETYPE_INODE : {
> + jint16_t *temp16ptr = (jint16_t *)wpage;
> +
> + *(temp16ptr++) = sum_collected->sum_list->i.nodetype;
> + wpage = (jint32_t *) temp16ptr;
> +
> + *(wpage++) = sum_collected->sum_list->i.inode;
> + *(wpage++) = sum_collected->sum_list->i.version;
> + *(wpage++) = sum_collected->sum_list->i.offset;
> + *(wpage++) = sum_collected->sum_list->i.totlen;
> + break;
> + }
> +
> + case JFFS2_NODETYPE_DIRENT : {
> + jint16_t *temp16ptr = (jint16_t *) wpage;
> + uint8_t *temp8ptr = NULL;
> +
> + *(temp16ptr++) = sum_collected->sum_list->d.nodetype;
> + wpage = (jint32_t *) temp16ptr;
> +
> + *(wpage++) = sum_collected->sum_list->d.totlen;
> + *(wpage++) = sum_collected->sum_list->d.offset;
> + *(wpage++) = sum_collected->sum_list->d.pino;
> + *(wpage++) = sum_collected->sum_list->d.version;
> + *(wpage++) = sum_collected->sum_list->d.ino;
> +
> + temp8ptr = (uint8_t *) wpage;
> + *(temp8ptr++) = sum_collected->sum_list->d.nsize;
> + *(temp8ptr++) = sum_collected->sum_list->d.type;
> +
> + memcpy(temp8ptr,sum_collected->sum_list->d.name,sum_collected->sum_list->d.nsize);
> + temp8ptr += sum_collected->sum_list->d.nsize;
> + wpage = (jint32_t *) temp8ptr;
> +
> + break;
> + }
> +
> + default : {
> + printf("This is not good for me!!!!\n");
> + }
> + }
> +
> + temp = sum_collected->sum_list;
> + sum_collected->sum_list = sum_collected->sum_list->u.next;
> + free(temp);
> +
> + sum_collected->sum_num--;
> + }
> +
> + sum_collected->sum_size = 0;
> +
> + tempptr = (uint8_t *) wpage;
> + tempptr += padsize;
> + wpage = (jint32_t *) tempptr;
> +
> + *(wpage++) = offset;
> + *(wpage++) = cpu_to_je32(erase_block_size);
> + *(wpage++) = magic;
> + isum.sum_crc = cpu_to_je32(crc32(0, tpage, datasize));
> + isum.node_crc = cpu_to_je32(crc32(0, &isum, sizeof(isum) - 8));
> +
> + full_write(data_buffer + data_ofs, &isum, sizeof(isum));
> + full_write(data_buffer + data_ofs, tpage, datasize);
> +
> + free(tpage);
> +
> +}
> +
> +static void full_write(void *target_buff, const void *buf, int len) {
> +
> + memcpy(target_buff, buf, len);
> + data_ofs += len;
> +}
> +
> +static void pad(int req) {
> + while (req) {
> + if (req > sizeof(ffbuf)) {
> + full_write(data_buffer + data_ofs, ffbuf, sizeof(ffbuf));
> + req -= sizeof(ffbuf);
> + } else {
> + full_write(data_buffer + data_ofs, ffbuf, req);
> + req = 0;
> + }
> + }
> +}
> +
> +static inline void padword() {
> +
> + if (data_ofs % 4) {
> + full_write(data_buffer + data_ofs, ffbuf, 4 - (data_ofs % 4));
> + }
> +}
> +
> +
> +static inline void pad_block_if_less_than(int req,int plus) {
> +
> + int datasize = req + plus + sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8;
> + datasize += (4 - (datasize % 4)) % 4;
> +
> + if (data_ofs + req > erase_block_size - datasize) {
> + dump_sum_records();
> + write_buff_to_file();
> + }
> +
> + if (add_cleanmarkers && found_cleanmarkers) {
> + if (!data_ofs) {
> + full_write(data_buffer, &cleanmarker, sizeof(cleanmarker));
> + pad(cleanmarker_size - sizeof(cleanmarker));
> + padword();
> + }
> + }
> +}
> +
> +void flush_buffers() {
> +
> + if ((add_cleanmarkers == 1) && (found_cleanmarkers == 1)) { /* CLEANMARKER */
> + if (data_ofs != cleanmarker_size) { /* INODE BUFFER */
> +
> + int datasize = sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8;
> + datasize += (4 - (datasize % 4)) % 4;
> +
> + /* If we have a full inode buffer, then write out inode and summary data */
> + if (data_ofs + sizeof(struct jffs2_raw_inode) + 2*JFFS2_MIN_DATA_LEN > erase_block_size - datasize) {
> + dump_sum_records();
> + write_buff_to_file();
> + }
> + /* else just write out inode data */
> + else{
> + if(padto)
> + pad(erase_block_size - data_ofs);
> + write_buff_to_file();
> + }
> + }
> + }
> + else { /* NO CLEANMARKER */
> + if (data_ofs != 0) { /* INODE BUFFER */
> +
> + int datasize = sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8;
> + datasize += (4 - (datasize % 4)) % 4;
> +
> + /* If we have a full inode buffer, then write out inode and summary data */
> + if (data_ofs + sizeof(struct jffs2_raw_inode) + 2*JFFS2_MIN_DATA_LEN > erase_block_size - datasize) {
> + dump_sum_records();
> + write_buff_to_file();
> + }
> + /* Else just write out inode data */
> + else{
> + if(padto)
> + pad(erase_block_size - data_ofs);
> + write_buff_to_file();
> + }
> + }
> + }
> +}
> +
> +int add_sum_mem(union jffs2_sum_mem *item) {
> +
> + union jffs2_sum_mem *walk;
> +
> + if (!sum_collected->sum_list) {
> + sum_collected->sum_list = (union jffs2_sum_mem *) item;
> + }
> + else {
> + walk = sum_collected->sum_list;
> +
> + while (walk->u.next) {
> + walk = walk->u.next;
> + }
> + walk->u.next = (union jffs2_sum_mem *) item;
> + }
> + switch (je16_to_cpu(item->u.nodetype)) {
> + case JFFS2_NODETYPE_INODE:
> + sum_collected->sum_size += JFFS2_SUMMARY_INODE_SIZE;
> + sum_collected->sum_num++;
> + break;
> +
> + case JFFS2_NODETYPE_DIRENT:
> + sum_collected->sum_size += JFFS2_SUMMARY_DIRENT_SIZE(item->d.nsize);
> + sum_collected->sum_num++;
> + break;
> +
> + default:
> + error_msg_and_die("__jffs2_add_sum_mem(): UNKNOWN node type %d\n", je16_to_cpu(item->u.nodetype));
> + }
> + return 0;
> +}
> +
> +void add_sum_inode_mem(union jffs2_node_union *node) {
> +
> + struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *) malloc(sizeof(struct jffs2_sum_inode_mem));
> +
> + if (!temp)
> + error_msg_and_die("Can't allocate memory for summary information!\n");
> +
> + temp->nodetype = node->i.nodetype;
> + temp->inode = node->i.ino;
> + temp->version = node->i.version;
> + temp->offset = cpu_to_je32(data_ofs);
> + temp->totlen = node->i.totlen;
> + temp->next = NULL;
> +
> + add_sum_mem((union jffs2_sum_mem *) temp);
> +
> +}
> +
> +void add_sum_dirent_mem(union jffs2_node_union *node) {
> +
> + struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *)
> + malloc(sizeof(struct jffs2_sum_dirent_mem) + node->d.nsize);
> +
> + if (!temp)
> + error_msg_and_die("Can't allocate memory for summary information!\n");
> +
> + temp->nodetype = node->d.nodetype;
> + temp->totlen = node->d.totlen;
> + temp->offset = cpu_to_je32(data_ofs);
> + temp->pino = node->d.pino;
> + temp->version = node->d.version;
> + temp->ino = node->d.ino;
> + temp->nsize = node->d.nsize;
> + temp->type = node->d.type;
> + temp->next = NULL;
> +
> + memcpy(temp->name,node->d.name,node->d.nsize);
> +
> + add_sum_mem((union jffs2_sum_mem *) temp);
> +
> +}
> +
> +void write_dirent_to_buff(union jffs2_node_union *node) {
> +
> + pad_block_if_less_than(je32_to_cpu (node->d.totlen),JFFS2_SUMMARY_DIRENT_SIZE(node->d.nsize));
> + add_sum_dirent_mem(node);
> + full_write(data_buffer + data_ofs, &(node->d), je32_to_cpu (node->d.totlen));
> + padword();
> +}
> +
> +
> +void write_inode_to_buff(union jffs2_node_union *node) {
> +
> + pad_block_if_less_than(je32_to_cpu (node->i.totlen),JFFS2_SUMMARY_INODE_SIZE);
> + add_sum_inode_mem(node); /* Add inode summary mem to summary list */
> + full_write(data_buffer + data_ofs, &(node->i), je32_to_cpu (node->i.totlen)); /* Write out the inode to inode_buffer */
> + padword();
> +
> +}
> +
> +
> +void create_summed_image(int inp_size) {
> + uint8_t *p = file_buffer;
> + union jffs2_node_union *node;
> + uint32_t crc;
> + uint16_t type;
> + int bitchbitmask = 0;
> + int obsolete;
> +
> + char name[256];
> +
> + while ( p < (file_buffer + inp_size)) {
> +
> + node = (union jffs2_node_union *) p;
> +
> + /* Skip empty space */
> + if (je16_to_cpu (node->u.magic) == 0xFFFF && je16_to_cpu (node->u.nodetype) == 0xFFFF) {
> + p += 4;
> + continue;
> + }
> +
> + if (je16_to_cpu (node->u.magic) != JFFS2_MAGIC_BITMASK) {
> + if (!bitchbitmask++)
> + printf ("Wrong bitmask at 0x%08x, 0x%04x\n", p - file_buffer, je16_to_cpu (node->u.magic));
> + p += 4;
> + continue;
> + }
> +
> + bitchbitmask = 0;
> +
> + type = je16_to_cpu(node->u.nodetype);
> + if ((type & JFFS2_NODE_ACCURATE) != JFFS2_NODE_ACCURATE) {
> + obsolete = 1;
> + type |= JFFS2_NODE_ACCURATE;
> + } else
> + obsolete = 0;
> +
> + node->u.nodetype = cpu_to_je16(type);
> +
> + crc = crc32 (0, node, sizeof (struct jffs2_unknown_node) - 4);
> + if (crc != je32_to_cpu (node->u.hdr_crc)) {
> + printf ("Wrong hdr_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->u.hdr_crc), crc);
> + p += 4;
> + continue;
> + }
> +
> + switch(je16_to_cpu(node->u.nodetype)) {
> +
> + case JFFS2_NODETYPE_INODE:
> + if(verbose)
> + printf ("%8s Inode node at 0x%08x, totlen 0x%08x, #ino %5d, version %5d, isize %8d, csize %8d, dsize %8d, offset %8d\n",
> + obsolete ? "Obsolete" : "",
> + p - file_buffer, je32_to_cpu (node->i.totlen), je32_to_cpu (node->i.ino),
> + je32_to_cpu ( node->i.version), je32_to_cpu (node->i.isize),
> + je32_to_cpu (node->i.csize), je32_to_cpu (node->i.dsize), je32_to_cpu (node->i.offset));
> +
> + crc = crc32 (0, node, sizeof (struct jffs2_raw_inode) - 8);
> + if (crc != je32_to_cpu (node->i.node_crc)) {
> + printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.node_crc), crc);
> + p += PAD(je32_to_cpu (node->i.totlen));
> + continue;
> + }
> +
> + crc = crc32(0, p + sizeof (struct jffs2_raw_inode), je32_to_cpu(node->i.csize));
> + if (crc != je32_to_cpu(node->i.data_crc)) {
> + printf ("Wrong data_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.data_crc), crc);
> + p += PAD(je32_to_cpu (node->i.totlen));
> + continue;
> + }
> +
> + write_inode_to_buff(node);
> +
> + p += PAD(je32_to_cpu (node->i.totlen));
> + break;
> +
> + case JFFS2_NODETYPE_DIRENT:
> + memcpy (name, node->d.name, node->d.nsize);
> + name [node->d.nsize] = 0x0;
> +
> + if(verbose)
> + printf ("%8s Dirent node at 0x%08x, totlen 0x%08x, #pino %5d, version %5d, #ino %8d, nsize %8d, name %s\n",
> + obsolete ? "Obsolete" : "",
> + p - file_buffer, je32_to_cpu (node->d.totlen), je32_to_cpu (node->d.pino),
> + je32_to_cpu ( node->d.version), je32_to_cpu (node->d.ino),
> + node->d.nsize, name);
> +
> + crc = crc32 (0, node, sizeof (struct jffs2_raw_dirent) - 8);
> + if (crc != je32_to_cpu (node->d.node_crc)) {
> + printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.node_crc), crc);
> + p += PAD(je32_to_cpu (node->d.totlen));
> + continue;
> + }
> +
> + crc = crc32(0, p + sizeof (struct jffs2_raw_dirent), node->d.nsize);
> + if (crc != je32_to_cpu(node->d.name_crc)) {
> + printf ("Wrong name_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.name_crc), crc);
> + p += PAD(je32_to_cpu (node->d.totlen));
> + continue;
> + }
> +
> + write_dirent_to_buff(node);
> +
> + p += PAD(je32_to_cpu (node->d.totlen));
> + break;
> +
> + case JFFS2_NODETYPE_CLEANMARKER:
> + if (verbose) {
> + printf ("%8s Cleanmarker at 0x%08x, totlen 0x%08x\n",
> + obsolete ? "Obsolete" : "",
> + p - file_buffer, je32_to_cpu (node->u.totlen));
> + }
> +
> + if(!found_cleanmarkers){
> + found_cleanmarkers = 1;
> +
> + if(add_cleanmarkers == 1 && use_input_cleanmarker_size == 1){
> + cleanmarker_size = je32_to_cpu (node->u.totlen);
> + setup_cleanmarker();
> + }
> + }
> +
> + p += PAD(je32_to_cpu (node->u.totlen));
> + break;
> +
> + case JFFS2_NODETYPE_PADDING:
> + if (verbose) {
> + printf ("%8s Padding node at 0x%08x, totlen 0x%08x\n",
> + obsolete ? "Obsolete" : "",
> + p - file_buffer, je32_to_cpu (node->u.totlen));
> + }
> + p += PAD(je32_to_cpu (node->u.totlen));
> + break;
> +
> + case 0xffff:
> + p += 4;
> + break;
> +
> + default:
> + if (verbose) {
> + printf ("%8s Unknown node at 0x%08x, totlen 0x%08x\n",
> + obsolete ? "Obsolete" : "",
> + p - file_buffer, je32_to_cpu (node->u.totlen));
> + }
> +
> + //write_unknown_to_buff(node);
> +
> + p += PAD(je32_to_cpu (node->u.totlen));
> + }
> + }
> +}
> +
> +int main(int argc, char **argv) {
> +
> + int ret;
> +
> + process_options(argc,argv);
> +
> + if ((in_fd == -1) || (out_fd == -1)) {
> +
> + if(in_fd != -1)
> + close(in_fd);
> + if(out_fd != -1)
> + close(out_fd);
> +
> + error_msg_and_die("You must specify input and output files!\n");
> + }
> +
> + init_buffers();
> + init_sumlist();
> +
> + while ((ret = load_next_block())) {
> + create_summed_image(ret);
> + }
> +
> + flush_buffers();
> + clean_buffers();
> + clean_sumlist();
> +
> + if (in_fd != -1)
> + close(in_fd);
> + if (out_fd != -1)
> + close(out_fd);
> +
> + return 0;
> +}
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-20 16:01 ` Gareth Bult (Encryptec)
@ 2004-12-20 16:09 ` Ferenc Havasi
2004-12-20 16:39 ` Gareth Bult (Encryptec)
2004-12-20 17:48 ` Gareth Bult (Encryptec)
0 siblings, 2 replies; 113+ messages in thread
From: Ferenc Havasi @ 2004-12-20 16:09 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Hi Gareth,
> I'm afraid this patch seems to be quite a long way from applying cleanly
> to my source. I'm using 2.6.9 .. at least half a dozen failures or
> more ...
It works with JFFS2 in the latest MTD CVS.
JFFS3 will contains this feature soon.
Regards,
Ferenc
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-18 16:02 ` Jörn Engel
@ 2004-12-20 16:34 ` Josh Boyer
2004-12-20 17:12 ` Gareth Bult (Encryptec)
0 siblings, 1 reply; 113+ messages in thread
From: Josh Boyer @ 2004-12-20 16:34 UTC (permalink / raw)
To: Jörn Engel; +Cc: Linux MTD, Gareth Bult (Encryptec)
On Sat, 2004-12-18 at 10:02, Jörn Engel wrote:
> On Fri, 17 December 2004 11:35:04 -0600, Josh Boyer wrote:
> >
> > Again, most controllers do all the wear leveling, erasing for you. So
> > you are left with compression, right?
>
> If you have enough trust in your El Cheapo hardware supplier. There
> is no standard that requires wear leveling and there are reports that
> basically tell you to do your own wear leveling on some devices. With
> no good way to distinguish "some" devices from "others", you can read
> that as "all". ;)
Ok, that's why I said "most". And if your El Cheapo hardware has built
in wear leveling, then doing wear leveling on top of that is always
questionable. It might not make things worse, but it's probably not
very efficient.
Now if you have El Super Cheapo hardware that explicitly states you need
to do wear leveling, that's a different story ;).
>
> USB keys and all the various other consumer-type flash devices are
> good enough to beat 3 1/2" floppies, that's about it. But that's
> quite good enough for most people. Floppies sold like crazy after
> all.
That's because it was the only removable media for PCs for a long time.
At least to the non-geeks. I can remember buying spare hard drives and
carrying those from machine to machine if I needed to do big transfers
;).
josh
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-20 16:09 ` Ferenc Havasi
@ 2004-12-20 16:39 ` Gareth Bult (Encryptec)
2004-12-20 17:48 ` Gareth Bult (Encryptec)
1 sibling, 0 replies; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-20 16:39 UTC (permalink / raw)
To: Ferenc Havasi; +Cc: Linux MTD
[-- Attachment #1: Type: text/plain, Size: 656 bytes --]
Yup,
I've cvs'd the latest source and patched ... kernel seems to build Ok ..
Just trying a full build from clean to make sure, then I'll give it a
go ..
Is there an example of how to use sumtool anywhere ?
If I eraseall the flash then mount JFFS2 and copy files on, do I need to
use sumtool ?
Gareth.
On Mon, 2004-12-20 at 17:09 +0100, Ferenc Havasi wrote:
> Hi Gareth,
>
> > I'm afraid this patch seems to be quite a long way from applying cleanly
> > to my source. I'm using 2.6.9 .. at least half a dozen failures or
> > more ...
>
> It works with JFFS2 in the latest MTD CVS.
>
> JFFS3 will contains this feature soon.
>
> Regards,
> Ferenc
[-- Attachment #2: Type: text/html, Size: 1194 bytes --]
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-20 16:34 ` Josh Boyer
@ 2004-12-20 17:12 ` Gareth Bult (Encryptec)
2004-12-21 13:09 ` Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-20 17:12 UTC (permalink / raw)
To: Josh Boyer; +Cc: Linux MTD
On Mon, 2004-12-20 at 10:34 -0600, Josh Boyer wrote:
> Ok, that's why I said "most". And if your El Cheapo hardware has built
> in wear leveling, then doing wear leveling on top of that is always
> questionable. It might not make things worse, but it's probably not
> very efficient.
Mmm, if the USB flash micro-controllers have build-in wear levelling,
they'd be quite something .. (!)
> Now if you have El Super Cheapo hardware that explicitly states you need
> to do wear leveling, that's a different story ;).
Here's a quote from a "Kingston" advert;
"Controllers automatically lock out bad memory cells and move the data
to avoid corruption. Controllers also automatically distribute write
cycles across the flash cells to extend the life of the flash memory
card"
What this means exactly in real terms .. [?]
(for example, do you lose some of your key to a bad block table, if so
how much ???)
Here's a bit from "Verbatim";
"For ultra-reliability, Verbatim’s new Store ‘n’ Go Pro drives feature
an on-board 32 bit ARM-7 microprocessor that manages I/O operations and
many of the drive’s technical features. An advanced wear-leveling
algorithm is used to distribute writes evenly among flash storage cells.
By evenly distributing the writing, Store ‘n’ Go Pro achieves an
unprecedented reliability rating of over one million write/erase cycles.
Advanced Error Detection Code/Error Correction Code furthers the Store
‘n’ Go Pro’s immunity to failure."
Interesting articles / links;
http://www.digitalmediadesigner.com/articles/viewarticle.jsp?id=27727
http://www.techworld.com/features/index.cfm?fuseaction=displayfeature&FeatureID=413
In summary: lots of manufacturers seem to be quoting virtual / physical
block mapping on the fly with integrated wear levelling ... (!)
So maybe wear levelling is out of date at filesystem level ?
The techworld link to the M-Systems chips looks interesting ..
Can anyone with detailed flash experience comment ?
Gareth.
> That's because it was the only removable media for PCs for a long time.
> At least to the non-geeks. I can remember buying spare hard drives and
> carrying those from machine to machine if I needed to do big transfers
> ;).
>
> josh
>
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-20 16:09 ` Ferenc Havasi
2004-12-20 16:39 ` Gareth Bult (Encryptec)
@ 2004-12-20 17:48 ` Gareth Bult (Encryptec)
1 sibling, 0 replies; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-20 17:48 UTC (permalink / raw)
To: Ferenc Havasi; +Cc: Linux MTD
Hi,
Patches applied and it seems to be working ..
After dd'ing a live FS off the key, sumtool'ing it and dd'ing it back
on, it seems to have cut mount time from 31 seconds to 11 seconds ...
This is certainly much better .. any way of making it quicker .. ;-)
Gareth.
On Mon, 2004-12-20 at 17:09 +0100, Ferenc Havasi wrote:
> Hi Gareth,
>
> > I'm afraid this patch seems to be quite a long way from applying cleanly
> > to my source. I'm using 2.6.9 .. at least half a dozen failures or
> > more ...
>
> It works with JFFS2 in the latest MTD CVS.
>
> JFFS3 will contains this feature soon.
>
> Regards,
> Ferenc
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-20 17:12 ` Gareth Bult (Encryptec)
@ 2004-12-21 13:09 ` Jörn Engel
2004-12-21 13:24 ` Gareth Bult (Encryptec)
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 13:09 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
On Mon, 20 December 2004 17:12:28 +0000, Gareth Bult (Encryptec) wrote:
> On Mon, 2004-12-20 at 10:34 -0600, Josh Boyer wrote:
> > Ok, that's why I said "most". And if your El Cheapo hardware has built
> > in wear leveling, then doing wear leveling on top of that is always
> > questionable. It might not make things worse, but it's probably not
> > very efficient.
Imo jffs2-style wear leveling is very efficient. So efficient, it
doesn't really matter whether the hardware does the same or not.
Difference between no wear leveling and what jffs2 does is - at most -
1% additional writes on the jffs2 side. That's not much. Agreed, it
can actually cost latency, but from a life-time or overall performance
point of view, it doesn't really matter.
On the other hand, it gives you absolute condidence that your El
Cheapo hardware doesn't need to have anything it may or may not
promise to have. No nasty surprised when you really don't need them.
> > Now if you have El Super Cheapo hardware that explicitly states you need
> > to do wear leveling, that's a different story ;).
Noone will state that. It's not exactly good marketing. You might
remember so interesting stories about what marketing promised and what
actually got delivered, no?
> In summary: lots of manufacturers seem to be quoting virtual / physical
> block mapping on the fly with integrated wear levelling ... (!)
>
> So maybe wear levelling is out of date at filesystem level ?
> The techworld link to the M-Systems chips looks interesting ..
>
> Can anyone with detailed flash experience comment ?
The endorsed features sound quite nice, agreed. But I have still to
read a spec that convinces me.
Jörn
--
My second remark is that our intellectual powers are rather geared to
master static relations and that our powers to visualize processes
evolving in time are relatively poorly developed.
-- Edsger W. Dijkstra
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-21 13:09 ` Jörn Engel
@ 2004-12-21 13:24 ` Gareth Bult (Encryptec)
2004-12-21 13:34 ` Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-21 13:24 UTC (permalink / raw)
To: Jörn Engel; +Cc: Linux MTD
[-- Attachment #1: Type: text/plain, Size: 1335 bytes --]
Hi,
On Tue, 2004-12-21 at 14:09 +0100, Jörn Engel wrote:
> Imo jffs2-style wear leveling is very efficient. So efficient, it
> doesn't really matter whether the hardware does the same or not.
:) I was of the same opinion but didn't really have any evidence to back
it up ..
> Difference between no wear leveling and what jffs2 does is - at most -
> 1% additional writes on the jffs2 side. That's not much. Agreed, it
> can actually cost latency, but from a life-time or overall performance
> point of view, it doesn't really matter.
Sure.
> On the other hand, it gives you absolute condidence that your El
> Cheapo hardware doesn't need to have anything it may or may not
> promise to have. No nasty surprised when you really don't need them.
:)
> Noone will state that. It's not exactly good marketing. You might
> remember so interesting stories about what marketing promised and what
> actually got delivered, no?
Sure, although I have noticed that some people shout more loudly than
others with regards to the wear levelling technology in their devices ..
> The endorsed features sound quite nice, agreed. But I have still to
> read a spec that convinces me.
Mmm .. anyone know a magazine editor who'd like to acquire a bunch of
keys and runs some tests for us .. ;-)
Gareth.
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-18 20:48 ` Gareth Bult (Encryptec)
2004-12-19 2:44 ` Jörn Engel
@ 2004-12-21 13:30 ` Jörn Engel
2004-12-21 13:39 ` [PATCH 1/22] Add drivers/mtd/devices/blockmtd.c Jörn Engel
2004-12-21 13:40 ` JFFS2 mount time Gareth Bult (Encryptec)
1 sibling, 2 replies; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 13:30 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
On Sat, 18 December 2004 20:48:50 +0000, Gareth Bult (Encryptec) wrote:
>
> To be honest what's there will do me for now .. although I'd like to
> clean it up a little when I get time.
Guess you've never been patch-bombed before. I took the liberty to do
some untested cleanups in ~20 patches. Most likely the end result
will not work anymore, but the direction is where I'd like to go.
If you find the time, could you test the patches and see which work
and which don't? The former will go into cvs then and I'll try to fix
the latter.
Jörn
--
It does not matter how slowly you go, so long as you do not stop.
-- Confucius
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-21 13:24 ` Gareth Bult (Encryptec)
@ 2004-12-21 13:34 ` Jörn Engel
0 siblings, 0 replies; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 13:34 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
On Tue, 21 December 2004 13:24:35 +0000, Gareth Bult (Encryptec) wrote:
>
> Mmm .. anyone know a magazine editor who'd like to acquire a bunch of
> keys and runs some tests for us .. ;-)
Somewhere in my lkml-mailbox was a report of some device not doing
decent wear leveling. It should be in the archives somewhere. Ext3
killed the devices rather quickly, jffs2 didn't.
Jörn
--
Data dominates. If you've chosen the right data structures and organized
things well, the algorithms will almost always be self-evident. Data
structures, not algorithms, are central to programming.
-- Rob Pike
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 1/22] Add drivers/mtd/devices/blockmtd.c
2004-12-21 13:30 ` Jörn Engel
@ 2004-12-21 13:39 ` Jörn Engel
2004-12-21 13:41 ` [PATCH 2/22] Add copyrights Jörn Engel
2004-12-21 13:42 ` [PATCH 1/22] Add drivers/mtd/devices/blockmtd.c Gareth Bult (Encryptec)
2004-12-21 13:40 ` JFFS2 mount time Gareth Bult (Encryptec)
1 sibling, 2 replies; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 13:39 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Jörn
--
When you close your hand, you own nothing. When you open it up, you
own the whole world.
-- Li Mu Bai in Tiger & Dragon
Add your driver under a new name. There will be interface changes, so
it needs to be renamed anyway.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 647 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 647 insertions(+)
--- /dev/null 2004-08-21 20:14:47.000000000 +0200
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 18:56:51.000000000 +0100
@@ -0,0 +1,647 @@
+/*
+ * $Id: blkmtd.c,v 1.23 2004/08/09 14:03:19 dwmw2 Exp $
+ *
+ * blkmtd.c - use a block device as a fake MTD
+ *
+ * Author: Simon Evans <spse@secret.org.uk>
+ *
+ * Copyright (C) 2001,2002 Simon Evans
+ *
+ * Licence: GPL
+ *
+ * How it works:
+ * The driver uses raw/io to read/write the device and the page
+ * cache to cache access. Writes update the page cache with the
+ * new data and mark it dirty and add the page into a BIO which
+ * is then written out.
+ *
+ * It can be loaded Read-Only to prevent erases and writes to the
+ * medium.
+ *
+ */
+
+#include <linux/config.h>
+#include <linux/module.h>
+#include <linux/fs.h>
+#include <linux/blkdev.h>
+#include <linux/bio.h>
+#include <linux/pagemap.h>
+#include <linux/list.h>
+#include <linux/init.h>
+#include <linux/mtd/mtd.h>
+#include <linux/buffer_head.h>
+
+#define err(format, arg...) printk(KERN_ERR "blkmtd: " format "\n" , ## arg)
+#define info(format, arg...) printk(KERN_INFO "blkmtd: " format "\n" , ## arg)
+#define warn(format, arg...) printk(KERN_WARNING "blkmtd: " format "\n" , ## arg)
+#define crit(format, arg...) printk(KERN_CRIT "blkmtd: " format "\n" , ## arg)
+
+
+/* Default erase size in K, always make it a multiple of PAGE_SIZE */
+#define CONFIG_MTD_BLKDEV_ERASESIZE (128 << 10) /* 128KiB */
+#define VERSION "$Revision: 1.23 $"
+
+/* Info for the block device */
+struct blkmtd_dev {
+ struct list_head list;
+ struct block_device *blkdev;
+ struct mtd_info mtd_info;
+ struct semaphore wrbuf_mutex;
+};
+
+
+/* Static info about the MTD, used in cleanup_module */
+static LIST_HEAD(blkmtd_device_list);
+
+static void blkmtd_sync(struct mtd_info *mtd);
+
+#define MAX_DEVICES 4
+
+/* Module parameters passed by insmod/modprobe */
+char *device[MAX_DEVICES]; /* the block device to use */
+int erasesz[MAX_DEVICES]; /* optional default erase size */
+int ro[MAX_DEVICES]; /* optional read only flag */
+int sync;
+
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Simon Evans <spse@secret.org.uk>");
+MODULE_DESCRIPTION("Emulate an MTD using a block device");
+MODULE_PARM(device, "1-4s");
+MODULE_PARM_DESC(device, "block device to use");
+MODULE_PARM(erasesz, "1-4i");
+MODULE_PARM_DESC(erasesz, "optional erase size to use in KiB. eg 4=4KiB.");
+MODULE_PARM(ro, "1-4i");
+MODULE_PARM_DESC(ro, "1=Read only, writes and erases cause errors");
+MODULE_PARM(sync, "i");
+MODULE_PARM_DESC(sync, "1=Synchronous writes");
+
+#define PAGE_READAHEAD 64
+
+void cache_readahead(struct address_space *mapping, int index)
+{
+ filler_t *filler = (filler_t*)(mapping->a_ops->readpage);
+ int i,pagei;
+ unsigned ret = 0;
+ unsigned long end_index;
+ struct page *page;
+ LIST_HEAD(page_pool);
+ struct inode *inode = mapping->host;
+ loff_t isize = i_size_read(inode);
+
+ if(!isize) {
+ printk(KERN_INFO "iSize=0 in cache_readahead\n");
+ return;
+ }
+
+ end_index = ((isize - 1) >> PAGE_CACHE_SHIFT);
+
+ spin_lock_irq(&mapping->tree_lock);
+ for(i=0;i<PAGE_READAHEAD;i++) {
+ pagei=index+i;
+ if (pagei > end_index) {
+ printk(KERN_INFO "Overrun end of disk in cache readahead\n");
+ break;
+ }
+ page = radix_tree_lookup(&mapping->page_tree, pagei);
+ if(page && (!i)) break;
+ if(page) continue;
+ spin_unlock_irq(&mapping->tree_lock);
+ page = page_cache_alloc_cold(mapping);
+ spin_lock_irq(&mapping->tree_lock);
+ if(!page) break;
+ page->index=pagei;
+ list_add(&page->lru, &page_pool);
+ ret++;
+ }
+ spin_unlock_irq(&mapping->tree_lock);
+ if(ret) {
+ read_cache_pages(mapping, &page_pool, filler, NULL);
+ //printk(KERN_INFO "Readahead [%d/%d] pages from [%d]\n",ret,PAGE_READAHEAD,index);
+ }
+}
+
+static struct page* page_readahead(struct address_space *mapping, int index)
+{
+ filler_t *filler = (filler_t*)(mapping->a_ops->readpage);
+ cache_readahead(mapping,index);
+ return read_cache_page(mapping, index, filler, NULL);
+}
+/**
+ * write_pages - write block of data to device via the page cache
+ * @dev: device to write to
+ * @buf: data source or NULL if erase (output is set to 0xff)
+ * @to: offset into output device
+ * @len: amount to data to write
+ * @retlen: amount of data written
+ *
+ * Grab pages from the page cache and fill them with the source data.
+ * Non page aligned start and end result in a readin of the page and
+ * part of the page being modified. Pages are added to the bio and then written
+ * out.
+ */
+
+static int write_pages(struct blkmtd_dev *dev, const u_char *buf, loff_t to, size_t len, size_t *retlen)
+{
+ struct page *page;
+ struct address_space *mapping = dev->blkdev->bd_inode->i_mapping;
+ //
+ int err = 0; // return status
+ int index = to >> PAGE_SHIFT; // page index
+ int offset = to & ~PAGE_MASK; // page offset
+ //
+
+ static int my_read()
+ {
+ err=0;
+ page = page_readahead(mapping,index);
+ if( unlikely(!page) ) {
+ crit("unable to read page (%d) in write_pages\n",index);
+ err=-ENOMEM;
+ }
+ //if(IS_ERR(page)) err=-EIO;
+ if(err) printk("READ ERROR\n");
+ return err;
+ }
+ //
+ static int my_erase()
+ {
+ int pages = len >> PAGE_SHIFT;
+ unsigned char *p;
+ unsigned char *max;
+ int format;
+
+ while(pages) {
+ //printk(KERN_INFO "readahead on index (%d)\n",index);
+ if(my_read()) return err;
+
+ max = ((char*)page_address(page)+PAGE_SIZE);
+ format = 0;
+
+ for(p=(unsigned char*)page_address(page); p<max; p++)
+ if(*p != 0xff) {
+ format=1;
+ break;
+ }
+
+ if(format) {
+ //DEBUG(3, "write_pages: erasing\n");
+ lock_page(page);
+ memset(page_address(page), 0xff, PAGE_SIZE);
+ set_page_dirty(page);
+ unlock_page(page);
+ }
+ page_cache_release(page);
+ pages--;
+ index++;
+ }
+ *retlen = len;
+ return 0;
+ }
+
+ static int my_write()
+ {
+ int cpylen;
+
+ if(retlen) *retlen = 0;
+ //printk(KERN_INFO "Write req: %d\n",len);
+ while(len) {
+ if((offset+len) > PAGE_SIZE)
+ cpylen = PAGE_SIZE - offset; // multiple pages
+ else cpylen = len; // this page
+ len = len - cpylen;
+ //
+ // Get page
+ //
+ if(my_read()) return err;
+ //
+ if(memcmp(page_address(page)+offset,buf,cpylen)) {
+ lock_page(page);
+ memcpy(page_address(page) + offset,buf, cpylen);
+ set_page_dirty(page);
+ unlock_page(page);
+ }
+ page_cache_release(page);
+
+ if(retlen) *retlen += cpylen;
+ //
+ buf+=cpylen;
+ offset = 0;
+ index++;
+ //
+ }
+ //printk(KERN_INFO "Write act: %d\n",*retlen);
+ return 0;
+ }
+
+ err=0;
+ down(&dev->wrbuf_mutex);
+ err = buf ? my_write() : my_erase();
+ up(&dev->wrbuf_mutex);
+ return err;
+}
+
+/* erase a specified part of the device */
+static int blkmtd_erase(struct mtd_info *mtd, struct erase_info *instr)
+{
+ struct blkmtd_dev *dev = mtd->priv;
+ struct mtd_erase_region_info *einfo = mtd->eraseregions;
+ int numregions = mtd->numeraseregions;
+ size_t from;
+ u_long len;
+ int err = -EIO;
+ size_t retlen;
+
+ instr->state = MTD_ERASING;
+ from = instr->addr;
+ len = instr->len;
+
+ /* check erase region has valid start and length */
+ //DEBUG(2, "blkmtd: erase: dev = `%s' from = 0x%zx len = 0x%lx\n", mtd->name+9, from, len);
+ while(numregions) {
+ DEBUG(3, "blkmtd: checking erase region = 0x%08X size = 0x%X num = 0x%x\n",
+ einfo->offset, einfo->erasesize, einfo->numblocks);
+ if(from >= einfo->offset
+ && from < einfo->offset + (einfo->erasesize * einfo->numblocks)) {
+ if(len == einfo->erasesize
+ && ( (from - einfo->offset) % einfo->erasesize == 0))
+ break;
+ }
+ numregions--;
+ einfo++;
+ }
+
+ if(!numregions) {
+ /* Not a valid erase block */
+ err("erase: invalid erase request 0x%lX @ 0x%08zX", len, from);
+ instr->state = MTD_ERASE_FAILED;
+ err = -EIO;
+ }
+
+ if(instr->state != MTD_ERASE_FAILED) {
+ /* do the erase */
+ DEBUG(3, "Doing erase from = %zd len = %ld\n", from, len);
+ err = write_pages(dev, NULL, from, len, &retlen);
+ if(err || retlen != len) {
+ err("erase failed err = %d", err);
+ instr->state = MTD_ERASE_FAILED;
+ } else {
+ instr->state = MTD_ERASE_DONE;
+ }
+ }
+
+ DEBUG(3, "blkmtd: erase: checking callback\n");
+ mtd_erase_callback(instr);
+ DEBUG(2, "blkmtd: erase: finished (err = %d)\n", err);
+ return err;
+}
+
+static int blkmtd_read(struct mtd_info *mtd, loff_t from, size_t len, size_t *retlen, u_char *buf)
+{
+ struct blkmtd_dev *dev = mtd->priv;
+ struct page *page;
+ //
+ int index = from >> PAGE_SHIFT;
+ int offset = from - (index << PAGE_SHIFT);
+ int cpylen;
+
+ if(from > mtd->size) return -EINVAL;
+ if(from + len > mtd->size) len = mtd->size - from;
+
+ if(retlen) *retlen = 0;
+
+ while(len) {
+ if((offset+len) > PAGE_SIZE)
+ cpylen = PAGE_SIZE - offset; // multiple pages
+ else cpylen = len; // this page
+ len = len - cpylen;
+ //
+ // Get page
+ //
+ page = page_readahead(dev->blkdev->bd_inode->i_mapping,index);
+ if(!page) return -ENOMEM;
+ if(IS_ERR(page)) return -EIO;
+ //
+ memcpy(buf, page_address(page) + offset, cpylen);
+ page_cache_release(page);
+ //
+ if(retlen) *retlen += cpylen;
+ buf+=cpylen;
+ offset = 0;
+ index++;
+ //
+ }
+ return 0;
+}
+
+
+/* write data to the underlying device */
+static int blkmtd_write(struct mtd_info *mtd, loff_t to, size_t len,
+ size_t *retlen, const u_char *buf)
+{
+ struct blkmtd_dev *dev = mtd->priv;
+ int err;
+
+ if(!len) return 0;
+ if(to >= mtd->size) return -ENOSPC;
+ if(to + len > mtd->size) len = mtd->size - to;
+
+ err = write_pages(dev, buf, to, len, retlen);
+ if(err > 0) err = 0;
+ return err;
+}
+
+
+/* sync the device - wait until the write queue is empty */
+static void blkmtd_sync(struct mtd_info *mtd)
+{
+ sync_blockdev(((struct blkmtd_dev*)mtd->priv)->blkdev);
+ return;
+}
+
+static void free_device(struct blkmtd_dev *dev)
+{
+ DEBUG(2, "blkmtd: free_device() dev = %p\n", dev);
+ if(dev) {
+ if(dev->mtd_info.eraseregions)
+ kfree(dev->mtd_info.eraseregions);
+ if(dev->mtd_info.name)
+ kfree(dev->mtd_info.name);
+
+ if(dev->blkdev) {
+ invalidate_inode_pages(dev->blkdev->bd_inode->i_mapping);
+ close_bdev_excl(dev->blkdev);
+ }
+ kfree(dev);
+ }
+}
+
+
+/* For a given size and initial erase size, calculate the number
+ * and size of each erase region. Goes round the loop twice,
+ * once to find out how many regions, then allocates space,
+ * then round the loop again to fill it in.
+ */
+static struct mtd_erase_region_info *calc_erase_regions(
+ size_t erase_size, size_t total_size, int *regions)
+{
+ struct mtd_erase_region_info *info = NULL;
+
+ DEBUG(2, "calc_erase_regions, es = %zd size = %zd regions = %d\n",
+ erase_size, total_size, *regions);
+ /* Make any user specified erasesize be a power of 2
+ and at least PAGE_SIZE */
+ if(erase_size) {
+ int es = erase_size;
+ erase_size = 1;
+ while(es != 1) {
+ es >>= 1;
+ erase_size <<= 1;
+ }
+ if(erase_size < PAGE_SIZE)
+ erase_size = PAGE_SIZE;
+ } else {
+ erase_size = CONFIG_MTD_BLKDEV_ERASESIZE;
+ }
+
+ *regions = 0;
+
+ do {
+ int tot_size = total_size;
+ int er_size = erase_size;
+ int count = 0, offset = 0, regcnt = 0;
+
+ while(tot_size) {
+ count = tot_size / er_size;
+ if(count) {
+ tot_size = tot_size % er_size;
+ if(info) {
+ DEBUG(2, "adding to erase info off=%d er=%d cnt=%d\n",
+ offset, er_size, count);
+ (info+regcnt)->offset = offset;
+ (info+regcnt)->erasesize = er_size;
+ (info+regcnt)->numblocks = count;
+ (*regions)++;
+ }
+ regcnt++;
+ offset += (count * er_size);
+ }
+ while(er_size > tot_size)
+ er_size >>= 1;
+ }
+ if(info == NULL) {
+ info = kmalloc(regcnt * sizeof(struct mtd_erase_region_info), GFP_KERNEL);
+ if(!info)
+ break;
+ }
+ } while(!(*regions));
+ DEBUG(2, "calc_erase_regions done, es = %zd size = %zd regions = %d\n",
+ erase_size, total_size, *regions);
+ return info;
+}
+
+
+extern dev_t __init name_to_dev_t(const char *line);
+
+static struct blkmtd_dev *add_device(char *devname, int readonly, int erase_size)
+{
+ struct block_device *bdev;
+ int mode;
+ struct blkmtd_dev *dev;
+
+ if(!devname)
+ return NULL;
+
+ /* Get a handle on the device */
+
+
+#ifdef MODULE
+ mode = (readonly) ? O_RDONLY : O_RDWR;
+ bdev = open_bdev_excl(devname, mode, NULL);
+#else
+ mode = (readonly) ? FMODE_READ : FMODE_WRITE;
+ bdev = open_by_devnum(name_to_dev_t(devname), mode);
+#endif
+ if(IS_ERR(bdev)) {
+ err("error: cannot open device %s", devname);
+ DEBUG(2, "blkmtd: opening bdev returned %ld\n", PTR_ERR(bdev));
+ return NULL;
+ }
+
+ DEBUG(1, "blkmtd: found a block device major = %d, minor = %d\n",
+ MAJOR(bdev->bd_dev), MINOR(bdev->bd_dev));
+
+ if(MAJOR(bdev->bd_dev) == MTD_BLOCK_MAJOR) {
+ err("attempting to use an MTD device as a block device");
+ blkdev_put(bdev);
+ return NULL;
+ }
+
+ dev = kmalloc(sizeof(struct blkmtd_dev), GFP_KERNEL);
+ if(dev == NULL) {
+ blkdev_put(bdev);
+ return NULL;
+ }
+
+ memset(dev, 0, sizeof(struct blkmtd_dev));
+ dev->blkdev = bdev;
+ atomic_set(&(dev->blkdev->bd_inode->i_mapping->truncate_count), 0);
+ if(!readonly) {
+ init_MUTEX(&dev->wrbuf_mutex);
+ }
+
+ dev->mtd_info.size = dev->blkdev->bd_inode->i_size & PAGE_MASK;
+
+ /* Setup the MTD structure */
+ /* make the name contain the block device in */
+ dev->mtd_info.name = kmalloc(sizeof("blkmtd: ") + strlen(devname), GFP_KERNEL);
+ if(dev->mtd_info.name == NULL)
+ goto devinit_err;
+
+ sprintf(dev->mtd_info.name, "blkmtd: %s", devname);
+ dev->mtd_info.eraseregions = calc_erase_regions(erase_size, dev->mtd_info.size,
+ &dev->mtd_info.numeraseregions);
+ if(dev->mtd_info.eraseregions == NULL)
+ goto devinit_err;
+
+ dev->mtd_info.erasesize = dev->mtd_info.eraseregions->erasesize;
+ DEBUG(1, "blkmtd: init: found %d erase regions\n",
+ dev->mtd_info.numeraseregions);
+
+ if(readonly) {
+ dev->mtd_info.type = MTD_ROM;
+ dev->mtd_info.flags = MTD_CAP_ROM;
+ } else {
+ dev->mtd_info.type = MTD_RAM;
+ dev->mtd_info.flags = MTD_CAP_RAM;
+ dev->mtd_info.erase = blkmtd_erase;
+ dev->mtd_info.write = blkmtd_write;
+ dev->mtd_info.writev = default_mtd_writev;
+ dev->mtd_info.sync = blkmtd_sync;
+ }
+ dev->mtd_info.read = blkmtd_read;
+ dev->mtd_info.readv = default_mtd_readv;
+ dev->mtd_info.priv = dev;
+ dev->mtd_info.owner = THIS_MODULE;
+
+ list_add(&dev->list, &blkmtd_device_list);
+ if (add_mtd_device(&dev->mtd_info)) {
+ /* Device didnt get added, so free the entry */
+ list_del(&dev->list);
+ goto devinit_err;
+ } else {
+ info("mtd%d: [%s] erase_size = %dKiB %s [%ld]",
+ dev->mtd_info.index, dev->mtd_info.name + strlen("blkmtd: "),
+ dev->mtd_info.erasesize >> 10,
+ readonly ? "(read-only)" : "", PAGE_SIZE);
+ }
+ return dev;
+
+ devinit_err:
+ free_device(dev);
+ return NULL;
+}
+
+
+/* Cleanup and exit - sync the device and kill of the kernel thread */
+static void __devexit cleanup_blkmtd(void)
+{
+ struct list_head *temp1, *temp2;
+
+ /* Remove the MTD devices */
+ list_for_each_safe(temp1, temp2, &blkmtd_device_list) {
+ struct blkmtd_dev *dev = list_entry(temp1, struct blkmtd_dev,
+ list);
+ blkmtd_sync(&dev->mtd_info);
+ del_mtd_device(&dev->mtd_info);
+ info("mtd%d: [%s] removed", dev->mtd_info.index,
+ dev->mtd_info.name + strlen("blkmtd: "));
+ list_del(&dev->list);
+ free_device(dev);
+ }
+}
+
+#ifndef MODULE
+
+/* Handle kernel boot params */
+
+
+static int __init param_blkmtd_device(char *str)
+{
+ int i;
+
+ for(i = 0; i < MAX_DEVICES; i++) {
+ device[i] = str;
+ DEBUG(2, "blkmtd: device setup: %d = %s\n", i, device[i]);
+ strsep(&str, ",");
+ }
+ return 1;
+}
+
+
+static int __init param_blkmtd_erasesz(char *str)
+{
+ int i;
+ for(i = 0; i < MAX_DEVICES; i++) {
+ char *val = strsep(&str, ",");
+ if(val)
+ erasesz[i] = simple_strtoul(val, NULL, 0);
+ DEBUG(2, "blkmtd: erasesz setup: %d = %d\n", i, erasesz[i]);
+ }
+
+ return 1;
+}
+
+
+static int __init param_blkmtd_ro(char *str)
+{
+ int i;
+ for(i = 0; i < MAX_DEVICES; i++) {
+ char *val = strsep(&str, ",");
+ if(val)
+ ro[i] = simple_strtoul(val, NULL, 0);
+ DEBUG(2, "blkmtd: ro setup: %d = %d\n", i, ro[i]);
+ }
+
+ return 1;
+}
+
+
+static int __init param_blkmtd_sync(char *str)
+{
+ if(str[0] == '1')
+ sync = 1;
+ return 1;
+}
+
+__setup("blkmtd_device=", param_blkmtd_device);
+__setup("blkmtd_erasesz=", param_blkmtd_erasesz);
+__setup("blkmtd_ro=", param_blkmtd_ro);
+__setup("blkmtd_sync=", param_blkmtd_sync);
+
+#endif
+
+
+/* Startup */
+static int __init init_blkmtd(void)
+{
+ int i;
+
+ info("version " VERSION);
+ /* Check args - device[0] is the bare minimum*/
+ if(!device[0]) {
+ err("error: missing `device' name\n");
+ return -EINVAL;
+ }
+
+ for(i = 0; i < MAX_DEVICES; i++)
+ add_device(device[i], ro[i], erasesz[i] << 10);
+
+ if(list_empty(&blkmtd_device_list))
+ return -EINVAL;
+
+ return 0;
+}
+
+module_init(init_blkmtd);
+module_exit(cleanup_blkmtd);
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-21 13:30 ` Jörn Engel
2004-12-21 13:39 ` [PATCH 1/22] Add drivers/mtd/devices/blockmtd.c Jörn Engel
@ 2004-12-21 13:40 ` Gareth Bult (Encryptec)
2004-12-21 15:00 ` David Woodhouse
1 sibling, 1 reply; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-21 13:40 UTC (permalink / raw)
To: Jörn Engel; +Cc: Linux MTD
[-- Attachment #1: Type: text/plain, Size: 686 bytes --]
.. and I find the patches where ?
On Tue, 2004-12-21 at 14:30 +0100, Jörn Engel wrote:
> On Sat, 18 December 2004 20:48:50 +0000, Gareth Bult (Encryptec) wrote:
> >
> > To be honest what's there will do me for now .. although I'd like to
> > clean it up a little when I get time.
>
> Guess you've never been patch-bombed before. I took the liberty to do
> some untested cleanups in ~20 patches. Most likely the end result
> will not work anymore, but the direction is where I'd like to go.
>
> If you find the time, could you test the patches and see which work
> and which don't? The former will go into cvs then and I'll try to fix
> the latter.
>
> Jörn
>
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 2/22] Add copyrights
2004-12-21 13:39 ` [PATCH 1/22] Add drivers/mtd/devices/blockmtd.c Jörn Engel
@ 2004-12-21 13:41 ` Jörn Engel
2004-12-21 13:42 ` [PATCH 3/22] Remove read-only option Jörn Engel
2004-12-21 13:42 ` [PATCH 1/22] Add drivers/mtd/devices/blockmtd.c Gareth Bult (Encryptec)
1 sibling, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 13:41 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Add both our copyrights. Didn't have my mail archive at hands when
doing the patches. Shouldn't be hard to fix.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 6 ++++--
1 files changed, 4 insertions(+), 2 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_copyright 2004-12-20 18:33:22.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 18:34:22.000000000 +0100
@@ -1,11 +1,13 @@
/*
* $Id: blkmtd.c,v 1.23 2004/08/09 14:03:19 dwmw2 Exp $
*
- * blkmtd.c - use a block device as a fake MTD
+ * blockmtd.c - use a block device as a fake MTD
*
* Author: Simon Evans <spse@secret.org.uk>
*
- * Copyright (C) 2001,2002 Simon Evans
+ * Copyright (C) 2001,2002 Simon Evans
+ * Copyright (C) 2004
+ * Copyright (C) 2004 Jörn Engel <joern@wh.fh-wedel.de>
*
* Licence: GPL
*
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH 1/22] Add drivers/mtd/devices/blockmtd.c
2004-12-21 13:39 ` [PATCH 1/22] Add drivers/mtd/devices/blockmtd.c Jörn Engel
2004-12-21 13:41 ` [PATCH 2/22] Add copyrights Jörn Engel
@ 2004-12-21 13:42 ` Gareth Bult (Encryptec)
2004-12-21 14:15 ` Jörn Engel
1 sibling, 1 reply; 113+ messages in thread
From: Gareth Bult (Encryptec) @ 2004-12-21 13:42 UTC (permalink / raw)
To: Jörn Engel; +Cc: Linux MTD
[-- Attachment #1: Type: text/plain, Size: 19514 bytes --]
Ok,
Gimmie a clue - what did you change ?
Gareth.
On Tue, 2004-12-21 at 14:39 +0100, Jörn Engel wrote:
> Jörn
>
> --
> When you close your hand, you own nothing. When you open it up, you
> own the whole world.
> -- Li Mu Bai in Tiger & Dragon
>
> Add your driver under a new name. There will be interface changes, so
> it needs to be renamed anyway.
>
> Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
> ---
>
> blockmtd.c | 647 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 files changed, 647 insertions(+)
>
> --- /dev/null 2004-08-21 20:14:47.000000000 +0200
> +++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 18:56:51.000000000 +0100
> @@ -0,0 +1,647 @@
> +/*
> + * $Id: blkmtd.c,v 1.23 2004/08/09 14:03:19 dwmw2 Exp $
> + *
> + * blkmtd.c - use a block device as a fake MTD
> + *
> + * Author: Simon Evans <spse@secret.org.uk>
> + *
> + * Copyright (C) 2001,2002 Simon Evans
> + *
> + * Licence: GPL
> + *
> + * How it works:
> + * The driver uses raw/io to read/write the device and the page
> + * cache to cache access. Writes update the page cache with the
> + * new data and mark it dirty and add the page into a BIO which
> + * is then written out.
> + *
> + * It can be loaded Read-Only to prevent erases and writes to the
> + * medium.
> + *
> + */
> +
> +#include <linux/config.h>
> +#include <linux/module.h>
> +#include <linux/fs.h>
> +#include <linux/blkdev.h>
> +#include <linux/bio.h>
> +#include <linux/pagemap.h>
> +#include <linux/list.h>
> +#include <linux/init.h>
> +#include <linux/mtd/mtd.h>
> +#include <linux/buffer_head.h>
> +
> +#define err(format, arg...) printk(KERN_ERR "blkmtd: " format "\n" , ## arg)
> +#define info(format, arg...) printk(KERN_INFO "blkmtd: " format "\n" , ## arg)
> +#define warn(format, arg...) printk(KERN_WARNING "blkmtd: " format "\n" , ## arg)
> +#define crit(format, arg...) printk(KERN_CRIT "blkmtd: " format "\n" , ## arg)
> +
> +
> +/* Default erase size in K, always make it a multiple of PAGE_SIZE */
> +#define CONFIG_MTD_BLKDEV_ERASESIZE (128 << 10) /* 128KiB */
> +#define VERSION "$Revision: 1.23 $"
> +
> +/* Info for the block device */
> +struct blkmtd_dev {
> + struct list_head list;
> + struct block_device *blkdev;
> + struct mtd_info mtd_info;
> + struct semaphore wrbuf_mutex;
> +};
> +
> +
> +/* Static info about the MTD, used in cleanup_module */
> +static LIST_HEAD(blkmtd_device_list);
> +
> +static void blkmtd_sync(struct mtd_info *mtd);
> +
> +#define MAX_DEVICES 4
> +
> +/* Module parameters passed by insmod/modprobe */
> +char *device[MAX_DEVICES]; /* the block device to use */
> +int erasesz[MAX_DEVICES]; /* optional default erase size */
> +int ro[MAX_DEVICES]; /* optional read only flag */
> +int sync;
> +
> +
> +MODULE_LICENSE("GPL");
> +MODULE_AUTHOR("Simon Evans <spse@secret.org.uk>");
> +MODULE_DESCRIPTION("Emulate an MTD using a block device");
> +MODULE_PARM(device, "1-4s");
> +MODULE_PARM_DESC(device, "block device to use");
> +MODULE_PARM(erasesz, "1-4i");
> +MODULE_PARM_DESC(erasesz, "optional erase size to use in KiB. eg 4=4KiB.");
> +MODULE_PARM(ro, "1-4i");
> +MODULE_PARM_DESC(ro, "1=Read only, writes and erases cause errors");
> +MODULE_PARM(sync, "i");
> +MODULE_PARM_DESC(sync, "1=Synchronous writes");
> +
> +#define PAGE_READAHEAD 64
> +
> +void cache_readahead(struct address_space *mapping, int index)
> +{
> + filler_t *filler = (filler_t*)(mapping->a_ops->readpage);
> + int i,pagei;
> + unsigned ret = 0;
> + unsigned long end_index;
> + struct page *page;
> + LIST_HEAD(page_pool);
> + struct inode *inode = mapping->host;
> + loff_t isize = i_size_read(inode);
> +
> + if(!isize) {
> + printk(KERN_INFO "iSize=0 in cache_readahead\n");
> + return;
> + }
> +
> + end_index = ((isize - 1) >> PAGE_CACHE_SHIFT);
> +
> + spin_lock_irq(&mapping->tree_lock);
> + for(i=0;i<PAGE_READAHEAD;i++) {
> + pagei=index+i;
> + if (pagei > end_index) {
> + printk(KERN_INFO "Overrun end of disk in cache readahead\n");
> + break;
> + }
> + page = radix_tree_lookup(&mapping->page_tree, pagei);
> + if(page && (!i)) break;
> + if(page) continue;
> + spin_unlock_irq(&mapping->tree_lock);
> + page = page_cache_alloc_cold(mapping);
> + spin_lock_irq(&mapping->tree_lock);
> + if(!page) break;
> + page->index=pagei;
> + list_add(&page->lru, &page_pool);
> + ret++;
> + }
> + spin_unlock_irq(&mapping->tree_lock);
> + if(ret) {
> + read_cache_pages(mapping, &page_pool, filler, NULL);
> + //printk(KERN_INFO "Readahead [%d/%d] pages from [%d]\n",ret,PAGE_READAHEAD,index);
> + }
> +}
> +
> +static struct page* page_readahead(struct address_space *mapping, int index)
> +{
> + filler_t *filler = (filler_t*)(mapping->a_ops->readpage);
> + cache_readahead(mapping,index);
> + return read_cache_page(mapping, index, filler, NULL);
> +}
> +/**
> + * write_pages - write block of data to device via the page cache
> + * @dev: device to write to
> + * @buf: data source or NULL if erase (output is set to 0xff)
> + * @to: offset into output device
> + * @len: amount to data to write
> + * @retlen: amount of data written
> + *
> + * Grab pages from the page cache and fill them with the source data.
> + * Non page aligned start and end result in a readin of the page and
> + * part of the page being modified. Pages are added to the bio and then written
> + * out.
> + */
> +
> +static int write_pages(struct blkmtd_dev *dev, const u_char *buf, loff_t to, size_t len, size_t *retlen)
> +{
> + struct page *page;
> + struct address_space *mapping = dev->blkdev->bd_inode->i_mapping;
> + //
> + int err = 0; // return status
> + int index = to >> PAGE_SHIFT; // page index
> + int offset = to & ~PAGE_MASK; // page offset
> + //
> +
> + static int my_read()
> + {
> + err=0;
> + page = page_readahead(mapping,index);
> + if( unlikely(!page) ) {
> + crit("unable to read page (%d) in write_pages\n",index);
> + err=-ENOMEM;
> + }
> + //if(IS_ERR(page)) err=-EIO;
> + if(err) printk("READ ERROR\n");
> + return err;
> + }
> + //
> + static int my_erase()
> + {
> + int pages = len >> PAGE_SHIFT;
> + unsigned char *p;
> + unsigned char *max;
> + int format;
> +
> + while(pages) {
> + //printk(KERN_INFO "readahead on index (%d)\n",index);
> + if(my_read()) return err;
> +
> + max = ((char*)page_address(page)+PAGE_SIZE);
> + format = 0;
> +
> + for(p=(unsigned char*)page_address(page); p<max; p++)
> + if(*p != 0xff) {
> + format=1;
> + break;
> + }
> +
> + if(format) {
> + //DEBUG(3, "write_pages: erasing\n");
> + lock_page(page);
> + memset(page_address(page), 0xff, PAGE_SIZE);
> + set_page_dirty(page);
> + unlock_page(page);
> + }
> + page_cache_release(page);
> + pages--;
> + index++;
> + }
> + *retlen = len;
> + return 0;
> + }
> +
> + static int my_write()
> + {
> + int cpylen;
> +
> + if(retlen) *retlen = 0;
> + //printk(KERN_INFO "Write req: %d\n",len);
> + while(len) {
> + if((offset+len) > PAGE_SIZE)
> + cpylen = PAGE_SIZE - offset; // multiple pages
> + else cpylen = len; // this page
> + len = len - cpylen;
> + //
> + // Get page
> + //
> + if(my_read()) return err;
> + //
> + if(memcmp(page_address(page)+offset,buf,cpylen)) {
> + lock_page(page);
> + memcpy(page_address(page) + offset,buf, cpylen);
> + set_page_dirty(page);
> + unlock_page(page);
> + }
> + page_cache_release(page);
> +
> + if(retlen) *retlen += cpylen;
> + //
> + buf+=cpylen;
> + offset = 0;
> + index++;
> + //
> + }
> + //printk(KERN_INFO "Write act: %d\n",*retlen);
> + return 0;
> + }
> +
> + err=0;
> + down(&dev->wrbuf_mutex);
> + err = buf ? my_write() : my_erase();
> + up(&dev->wrbuf_mutex);
> + return err;
> +}
> +
> +/* erase a specified part of the device */
> +static int blkmtd_erase(struct mtd_info *mtd, struct erase_info *instr)
> +{
> + struct blkmtd_dev *dev = mtd->priv;
> + struct mtd_erase_region_info *einfo = mtd->eraseregions;
> + int numregions = mtd->numeraseregions;
> + size_t from;
> + u_long len;
> + int err = -EIO;
> + size_t retlen;
> +
> + instr->state = MTD_ERASING;
> + from = instr->addr;
> + len = instr->len;
> +
> + /* check erase region has valid start and length */
> + //DEBUG(2, "blkmtd: erase: dev = `%s' from = 0x%zx len = 0x%lx\n", mtd->name+9, from, len);
> + while(numregions) {
> + DEBUG(3, "blkmtd: checking erase region = 0x%08X size = 0x%X num = 0x%x\n",
> + einfo->offset, einfo->erasesize, einfo->numblocks);
> + if(from >= einfo->offset
> + && from < einfo->offset + (einfo->erasesize * einfo->numblocks)) {
> + if(len == einfo->erasesize
> + && ( (from - einfo->offset) % einfo->erasesize == 0))
> + break;
> + }
> + numregions--;
> + einfo++;
> + }
> +
> + if(!numregions) {
> + /* Not a valid erase block */
> + err("erase: invalid erase request 0x%lX @ 0x%08zX", len, from);
> + instr->state = MTD_ERASE_FAILED;
> + err = -EIO;
> + }
> +
> + if(instr->state != MTD_ERASE_FAILED) {
> + /* do the erase */
> + DEBUG(3, "Doing erase from = %zd len = %ld\n", from, len);
> + err = write_pages(dev, NULL, from, len, &retlen);
> + if(err || retlen != len) {
> + err("erase failed err = %d", err);
> + instr->state = MTD_ERASE_FAILED;
> + } else {
> + instr->state = MTD_ERASE_DONE;
> + }
> + }
> +
> + DEBUG(3, "blkmtd: erase: checking callback\n");
> + mtd_erase_callback(instr);
> + DEBUG(2, "blkmtd: erase: finished (err = %d)\n", err);
> + return err;
> +}
> +
> +static int blkmtd_read(struct mtd_info *mtd, loff_t from, size_t len, size_t *retlen, u_char *buf)
> +{
> + struct blkmtd_dev *dev = mtd->priv;
> + struct page *page;
> + //
> + int index = from >> PAGE_SHIFT;
> + int offset = from - (index << PAGE_SHIFT);
> + int cpylen;
> +
> + if(from > mtd->size) return -EINVAL;
> + if(from + len > mtd->size) len = mtd->size - from;
> +
> + if(retlen) *retlen = 0;
> +
> + while(len) {
> + if((offset+len) > PAGE_SIZE)
> + cpylen = PAGE_SIZE - offset; // multiple pages
> + else cpylen = len; // this page
> + len = len - cpylen;
> + //
> + // Get page
> + //
> + page = page_readahead(dev->blkdev->bd_inode->i_mapping,index);
> + if(!page) return -ENOMEM;
> + if(IS_ERR(page)) return -EIO;
> + //
> + memcpy(buf, page_address(page) + offset, cpylen);
> + page_cache_release(page);
> + //
> + if(retlen) *retlen += cpylen;
> + buf+=cpylen;
> + offset = 0;
> + index++;
> + //
> + }
> + return 0;
> +}
> +
> +
> +/* write data to the underlying device */
> +static int blkmtd_write(struct mtd_info *mtd, loff_t to, size_t len,
> + size_t *retlen, const u_char *buf)
> +{
> + struct blkmtd_dev *dev = mtd->priv;
> + int err;
> +
> + if(!len) return 0;
> + if(to >= mtd->size) return -ENOSPC;
> + if(to + len > mtd->size) len = mtd->size - to;
> +
> + err = write_pages(dev, buf, to, len, retlen);
> + if(err > 0) err = 0;
> + return err;
> +}
> +
> +
> +/* sync the device - wait until the write queue is empty */
> +static void blkmtd_sync(struct mtd_info *mtd)
> +{
> + sync_blockdev(((struct blkmtd_dev*)mtd->priv)->blkdev);
> + return;
> +}
> +
> +static void free_device(struct blkmtd_dev *dev)
> +{
> + DEBUG(2, "blkmtd: free_device() dev = %p\n", dev);
> + if(dev) {
> + if(dev->mtd_info.eraseregions)
> + kfree(dev->mtd_info.eraseregions);
> + if(dev->mtd_info.name)
> + kfree(dev->mtd_info.name);
> +
> + if(dev->blkdev) {
> + invalidate_inode_pages(dev->blkdev->bd_inode->i_mapping);
> + close_bdev_excl(dev->blkdev);
> + }
> + kfree(dev);
> + }
> +}
> +
> +
> +/* For a given size and initial erase size, calculate the number
> + * and size of each erase region. Goes round the loop twice,
> + * once to find out how many regions, then allocates space,
> + * then round the loop again to fill it in.
> + */
> +static struct mtd_erase_region_info *calc_erase_regions(
> + size_t erase_size, size_t total_size, int *regions)
> +{
> + struct mtd_erase_region_info *info = NULL;
> +
> + DEBUG(2, "calc_erase_regions, es = %zd size = %zd regions = %d\n",
> + erase_size, total_size, *regions);
> + /* Make any user specified erasesize be a power of 2
> + and at least PAGE_SIZE */
> + if(erase_size) {
> + int es = erase_size;
> + erase_size = 1;
> + while(es != 1) {
> + es >>= 1;
> + erase_size <<= 1;
> + }
> + if(erase_size < PAGE_SIZE)
> + erase_size = PAGE_SIZE;
> + } else {
> + erase_size = CONFIG_MTD_BLKDEV_ERASESIZE;
> + }
> +
> + *regions = 0;
> +
> + do {
> + int tot_size = total_size;
> + int er_size = erase_size;
> + int count = 0, offset = 0, regcnt = 0;
> +
> + while(tot_size) {
> + count = tot_size / er_size;
> + if(count) {
> + tot_size = tot_size % er_size;
> + if(info) {
> + DEBUG(2, "adding to erase info off=%d er=%d cnt=%d\n",
> + offset, er_size, count);
> + (info+regcnt)->offset = offset;
> + (info+regcnt)->erasesize = er_size;
> + (info+regcnt)->numblocks = count;
> + (*regions)++;
> + }
> + regcnt++;
> + offset += (count * er_size);
> + }
> + while(er_size > tot_size)
> + er_size >>= 1;
> + }
> + if(info == NULL) {
> + info = kmalloc(regcnt * sizeof(struct mtd_erase_region_info), GFP_KERNEL);
> + if(!info)
> + break;
> + }
> + } while(!(*regions));
> + DEBUG(2, "calc_erase_regions done, es = %zd size = %zd regions = %d\n",
> + erase_size, total_size, *regions);
> + return info;
> +}
> +
> +
> +extern dev_t __init name_to_dev_t(const char *line);
> +
> +static struct blkmtd_dev *add_device(char *devname, int readonly, int erase_size)
> +{
> + struct block_device *bdev;
> + int mode;
> + struct blkmtd_dev *dev;
> +
> + if(!devname)
> + return NULL;
> +
> + /* Get a handle on the device */
> +
> +
> +#ifdef MODULE
> + mode = (readonly) ? O_RDONLY : O_RDWR;
> + bdev = open_bdev_excl(devname, mode, NULL);
> +#else
> + mode = (readonly) ? FMODE_READ : FMODE_WRITE;
> + bdev = open_by_devnum(name_to_dev_t(devname), mode);
> +#endif
> + if(IS_ERR(bdev)) {
> + err("error: cannot open device %s", devname);
> + DEBUG(2, "blkmtd: opening bdev returned %ld\n", PTR_ERR(bdev));
> + return NULL;
> + }
> +
> + DEBUG(1, "blkmtd: found a block device major = %d, minor = %d\n",
> + MAJOR(bdev->bd_dev), MINOR(bdev->bd_dev));
> +
> + if(MAJOR(bdev->bd_dev) == MTD_BLOCK_MAJOR) {
> + err("attempting to use an MTD device as a block device");
> + blkdev_put(bdev);
> + return NULL;
> + }
> +
> + dev = kmalloc(sizeof(struct blkmtd_dev), GFP_KERNEL);
> + if(dev == NULL) {
> + blkdev_put(bdev);
> + return NULL;
> + }
> +
> + memset(dev, 0, sizeof(struct blkmtd_dev));
> + dev->blkdev = bdev;
> + atomic_set(&(dev->blkdev->bd_inode->i_mapping->truncate_count), 0);
> + if(!readonly) {
> + init_MUTEX(&dev->wrbuf_mutex);
> + }
> +
> + dev->mtd_info.size = dev->blkdev->bd_inode->i_size & PAGE_MASK;
> +
> + /* Setup the MTD structure */
> + /* make the name contain the block device in */
> + dev->mtd_info.name = kmalloc(sizeof("blkmtd: ") + strlen(devname), GFP_KERNEL);
> + if(dev->mtd_info.name == NULL)
> + goto devinit_err;
> +
> + sprintf(dev->mtd_info.name, "blkmtd: %s", devname);
> + dev->mtd_info.eraseregions = calc_erase_regions(erase_size, dev->mtd_info.size,
> + &dev->mtd_info.numeraseregions);
> + if(dev->mtd_info.eraseregions == NULL)
> + goto devinit_err;
> +
> + dev->mtd_info.erasesize = dev->mtd_info.eraseregions->erasesize;
> + DEBUG(1, "blkmtd: init: found %d erase regions\n",
> + dev->mtd_info.numeraseregions);
> +
> + if(readonly) {
> + dev->mtd_info.type = MTD_ROM;
> + dev->mtd_info.flags = MTD_CAP_ROM;
> + } else {
> + dev->mtd_info.type = MTD_RAM;
> + dev->mtd_info.flags = MTD_CAP_RAM;
> + dev->mtd_info.erase = blkmtd_erase;
> + dev->mtd_info.write = blkmtd_write;
> + dev->mtd_info.writev = default_mtd_writev;
> + dev->mtd_info.sync = blkmtd_sync;
> + }
> + dev->mtd_info.read = blkmtd_read;
> + dev->mtd_info.readv = default_mtd_readv;
> + dev->mtd_info.priv = dev;
> + dev->mtd_info.owner = THIS_MODULE;
> +
> + list_add(&dev->list, &blkmtd_device_list);
> + if (add_mtd_device(&dev->mtd_info)) {
> + /* Device didnt get added, so free the entry */
> + list_del(&dev->list);
> + goto devinit_err;
> + } else {
> + info("mtd%d: [%s] erase_size = %dKiB %s [%ld]",
> + dev->mtd_info.index, dev->mtd_info.name + strlen("blkmtd: "),
> + dev->mtd_info.erasesize >> 10,
> + readonly ? "(read-only)" : "", PAGE_SIZE);
> + }
> + return dev;
> +
> + devinit_err:
> + free_device(dev);
> + return NULL;
> +}
> +
> +
> +/* Cleanup and exit - sync the device and kill of the kernel thread */
> +static void __devexit cleanup_blkmtd(void)
> +{
> + struct list_head *temp1, *temp2;
> +
> + /* Remove the MTD devices */
> + list_for_each_safe(temp1, temp2, &blkmtd_device_list) {
> + struct blkmtd_dev *dev = list_entry(temp1, struct blkmtd_dev,
> + list);
> + blkmtd_sync(&dev->mtd_info);
> + del_mtd_device(&dev->mtd_info);
> + info("mtd%d: [%s] removed", dev->mtd_info.index,
> + dev->mtd_info.name + strlen("blkmtd: "));
> + list_del(&dev->list);
> + free_device(dev);
> + }
> +}
> +
> +#ifndef MODULE
> +
> +/* Handle kernel boot params */
> +
> +
> +static int __init param_blkmtd_device(char *str)
> +{
> + int i;
> +
> + for(i = 0; i < MAX_DEVICES; i++) {
> + device[i] = str;
> + DEBUG(2, "blkmtd: device setup: %d = %s\n", i, device[i]);
> + strsep(&str, ",");
> + }
> + return 1;
> +}
> +
> +
> +static int __init param_blkmtd_erasesz(char *str)
> +{
> + int i;
> + for(i = 0; i < MAX_DEVICES; i++) {
> + char *val = strsep(&str, ",");
> + if(val)
> + erasesz[i] = simple_strtoul(val, NULL, 0);
> + DEBUG(2, "blkmtd: erasesz setup: %d = %d\n", i, erasesz[i]);
> + }
> +
> + return 1;
> +}
> +
> +
> +static int __init param_blkmtd_ro(char *str)
> +{
> + int i;
> + for(i = 0; i < MAX_DEVICES; i++) {
> + char *val = strsep(&str, ",");
> + if(val)
> + ro[i] = simple_strtoul(val, NULL, 0);
> + DEBUG(2, "blkmtd: ro setup: %d = %d\n", i, ro[i]);
> + }
> +
> + return 1;
> +}
> +
> +
> +static int __init param_blkmtd_sync(char *str)
> +{
> + if(str[0] == '1')
> + sync = 1;
> + return 1;
> +}
> +
> +__setup("blkmtd_device=", param_blkmtd_device);
> +__setup("blkmtd_erasesz=", param_blkmtd_erasesz);
> +__setup("blkmtd_ro=", param_blkmtd_ro);
> +__setup("blkmtd_sync=", param_blkmtd_sync);
> +
> +#endif
> +
> +
> +/* Startup */
> +static int __init init_blkmtd(void)
> +{
> + int i;
> +
> + info("version " VERSION);
> + /* Check args - device[0] is the bare minimum*/
> + if(!device[0]) {
> + err("error: missing `device' name\n");
> + return -EINVAL;
> + }
> +
> + for(i = 0; i < MAX_DEVICES; i++)
> + add_device(device[i], ro[i], erasesz[i] << 10);
> +
> + if(list_empty(&blkmtd_device_list))
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +module_init(init_blkmtd);
> +module_exit(cleanup_blkmtd);
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 3/22] Remove read-only option
2004-12-21 13:41 ` [PATCH 2/22] Add copyrights Jörn Engel
@ 2004-12-21 13:42 ` Jörn Engel
2004-12-21 13:44 ` [PATCH 4/22] Change init/exit functions Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 13:42 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Read-only devices are an inherintly stupid idea. Get rid of them.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 51 +++++++++++++--------------------------------------
1 files changed, 13 insertions(+), 38 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_readonly 2004-12-20 18:34:22.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 18:37:15.000000000 +0100
@@ -62,7 +62,6 @@
/* Module parameters passed by insmod/modprobe */
char *device[MAX_DEVICES]; /* the block device to use */
int erasesz[MAX_DEVICES]; /* optional default erase size */
-int ro[MAX_DEVICES]; /* optional read only flag */
int sync;
@@ -73,8 +72,6 @@
MODULE_PARM_DESC(device, "block device to use");
MODULE_PARM(erasesz, "1-4i");
MODULE_PARM_DESC(erasesz, "optional erase size to use in KiB. eg 4=4KiB.");
-MODULE_PARM(ro, "1-4i");
-MODULE_PARM_DESC(ro, "1=Read only, writes and erases cause errors");
MODULE_PARM(sync, "i");
MODULE_PARM_DESC(sync, "1=Synchronous writes");
@@ -445,7 +442,7 @@
extern dev_t __init name_to_dev_t(const char *line);
-static struct blkmtd_dev *add_device(char *devname, int readonly, int erase_size)
+static struct blkmtd_dev *add_device(char *devname, int erase_size)
{
struct block_device *bdev;
int mode;
@@ -458,10 +455,10 @@
#ifdef MODULE
- mode = (readonly) ? O_RDONLY : O_RDWR;
+ mode = O_RDWR;
bdev = open_bdev_excl(devname, mode, NULL);
#else
- mode = (readonly) ? FMODE_READ : FMODE_WRITE;
+ mode = FMODE_WRITE;
bdev = open_by_devnum(name_to_dev_t(devname), mode);
#endif
if(IS_ERR(bdev)) {
@@ -488,9 +485,7 @@
memset(dev, 0, sizeof(struct blkmtd_dev));
dev->blkdev = bdev;
atomic_set(&(dev->blkdev->bd_inode->i_mapping->truncate_count), 0);
- if(!readonly) {
- init_MUTEX(&dev->wrbuf_mutex);
- }
+ init_MUTEX(&dev->wrbuf_mutex);
dev->mtd_info.size = dev->blkdev->bd_inode->i_size & PAGE_MASK;
@@ -510,17 +505,12 @@
DEBUG(1, "blkmtd: init: found %d erase regions\n",
dev->mtd_info.numeraseregions);
- if(readonly) {
- dev->mtd_info.type = MTD_ROM;
- dev->mtd_info.flags = MTD_CAP_ROM;
- } else {
- dev->mtd_info.type = MTD_RAM;
- dev->mtd_info.flags = MTD_CAP_RAM;
- dev->mtd_info.erase = blkmtd_erase;
- dev->mtd_info.write = blkmtd_write;
- dev->mtd_info.writev = default_mtd_writev;
- dev->mtd_info.sync = blkmtd_sync;
- }
+ dev->mtd_info.type = MTD_RAM;
+ dev->mtd_info.flags = MTD_CAP_RAM;
+ dev->mtd_info.erase = blkmtd_erase;
+ dev->mtd_info.write = blkmtd_write;
+ dev->mtd_info.writev = default_mtd_writev;
+ dev->mtd_info.sync = blkmtd_sync;
dev->mtd_info.read = blkmtd_read;
dev->mtd_info.readv = default_mtd_readv;
dev->mtd_info.priv = dev;
@@ -532,10 +522,10 @@
list_del(&dev->list);
goto devinit_err;
} else {
- info("mtd%d: [%s] erase_size = %dKiB %s [%ld]",
+ info("mtd%d: [%s] erase_size = %dKiB [%ld]",
dev->mtd_info.index, dev->mtd_info.name + strlen("blkmtd: "),
dev->mtd_info.erasesize >> 10,
- readonly ? "(read-only)" : "", PAGE_SIZE);
+ PAGE_SIZE);
}
return dev;
@@ -595,20 +585,6 @@
}
-static int __init param_blkmtd_ro(char *str)
-{
- int i;
- for(i = 0; i < MAX_DEVICES; i++) {
- char *val = strsep(&str, ",");
- if(val)
- ro[i] = simple_strtoul(val, NULL, 0);
- DEBUG(2, "blkmtd: ro setup: %d = %d\n", i, ro[i]);
- }
-
- return 1;
-}
-
-
static int __init param_blkmtd_sync(char *str)
{
if(str[0] == '1')
@@ -618,7 +594,6 @@
__setup("blkmtd_device=", param_blkmtd_device);
__setup("blkmtd_erasesz=", param_blkmtd_erasesz);
-__setup("blkmtd_ro=", param_blkmtd_ro);
__setup("blkmtd_sync=", param_blkmtd_sync);
#endif
@@ -637,7 +612,7 @@
}
for(i = 0; i < MAX_DEVICES; i++)
- add_device(device[i], ro[i], erasesz[i] << 10);
+ add_device(device[i], erasesz[i] << 10);
if(list_empty(&blkmtd_device_list))
return -EINVAL;
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 4/22] Change init/exit functions
2004-12-21 13:42 ` [PATCH 3/22] Remove read-only option Jörn Engel
@ 2004-12-21 13:44 ` Jörn Engel
2004-12-21 13:45 ` [PATCH 5/22] Remove gcc warnings Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 13:44 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Move/change the init and exit functions.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 53 ++++++++++++++++++++++++++---------------------------
1 files changed, 26 insertions(+), 27 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_init_exit 2004-12-20 18:37:15.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 18:39:30.000000000 +0100
@@ -534,25 +534,6 @@
return NULL;
}
-
-/* Cleanup and exit - sync the device and kill of the kernel thread */
-static void __devexit cleanup_blkmtd(void)
-{
- struct list_head *temp1, *temp2;
-
- /* Remove the MTD devices */
- list_for_each_safe(temp1, temp2, &blkmtd_device_list) {
- struct blkmtd_dev *dev = list_entry(temp1, struct blkmtd_dev,
- list);
- blkmtd_sync(&dev->mtd_info);
- del_mtd_device(&dev->mtd_info);
- info("mtd%d: [%s] removed", dev->mtd_info.index,
- dev->mtd_info.name + strlen("blkmtd: "));
- list_del(&dev->list);
- free_device(dev);
- }
-}
-
#ifndef MODULE
/* Handle kernel boot params */
@@ -599,26 +580,44 @@
#endif
-/* Startup */
-static int __init init_blkmtd(void)
+static int __init blockmtd_init(void)
{
int i;
info("version " VERSION);
- /* Check args - device[0] is the bare minimum*/
- if(!device[0]) {
+ /* Check args - device[0] is the bare minimum */
+ if (!device[0]) {
err("error: missing `device' name\n");
return -EINVAL;
}
- for(i = 0; i < MAX_DEVICES; i++)
+ for (i = 0; i < MAX_DEVICES; i++)
add_device(device[i], erasesz[i] << 10);
- if(list_empty(&blkmtd_device_list))
+ if (list_empty(&blkmtd_device_list))
return -EINVAL;
return 0;
}
-module_init(init_blkmtd);
-module_exit(cleanup_blkmtd);
+
+static void __devexit blockmtd_exit(void)
+{
+ struct list_head *temp1, *temp2;
+
+ /* Remove the MTD devices */
+ list_for_each_safe(temp1, temp2, &blkmtd_device_list) {
+ struct blkmtd_dev *dev = list_entry(temp1, struct blkmtd_dev,
+ list);
+ blkmtd_sync(&dev->mtd_info);
+ del_mtd_device(&dev->mtd_info);
+ info("mtd%d: [%s] removed", dev->mtd_info.index,
+ dev->mtd_info.name + strlen("blkmtd: "));
+ list_del(&dev->list);
+ free_device(dev);
+ }
+}
+
+
+module_init(blockmtd_init);
+module_exit(blockmtd_exit);
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 5/22] Remove gcc warnings
2004-12-21 13:44 ` [PATCH 4/22] Change init/exit functions Jörn Engel
@ 2004-12-21 13:45 ` Jörn Engel
2004-12-21 13:47 ` [PATCH 6/22] Remove debug macros Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 13:45 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Name sais enough.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 7 ++++---
1 files changed, 4 insertions(+), 3 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_prototypes 2004-12-20 18:57:18.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 19:00:27.000000000 +0100
@@ -150,7 +150,7 @@
int offset = to & ~PAGE_MASK; // page offset
//
- static int my_read()
+ static int my_read(void)
{
err=0;
page = page_readahead(mapping,index);
@@ -163,7 +163,7 @@
return err;
}
//
- static int my_erase()
+ static int my_erase(void)
{
int pages = len >> PAGE_SHIFT;
unsigned char *p;
@@ -198,7 +198,7 @@
return 0;
}
- static int my_write()
+ static int my_write(void)
{
int cpylen;
@@ -240,6 +240,7 @@
return err;
}
+
/* erase a specified part of the device */
static int blkmtd_erase(struct mtd_info *mtd, struct erase_info *instr)
{
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 6/22] Remove debug macros
2004-12-21 13:45 ` [PATCH 5/22] Remove gcc warnings Jörn Engel
@ 2004-12-21 13:47 ` Jörn Engel
2004-12-21 13:48 ` [PATCH 7/22] Lindent Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 13:47 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Remove debug output. I have absolutely no love for those things. If
you need them for debugging, add them, find the problem, fix the
problem, keep the fix and drop the debugging mess.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 24 +-----------------------
1 files changed, 1 insertion(+), 23 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_debug 2004-12-20 19:00:27.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 19:03:50.000000000 +0100
@@ -184,7 +184,6 @@
}
if(format) {
- //DEBUG(3, "write_pages: erasing\n");
lock_page(page);
memset(page_address(page), 0xff, PAGE_SIZE);
set_page_dirty(page);
@@ -257,10 +256,7 @@
len = instr->len;
/* check erase region has valid start and length */
- //DEBUG(2, "blkmtd: erase: dev = `%s' from = 0x%zx len = 0x%lx\n", mtd->name+9, from, len);
while(numregions) {
- DEBUG(3, "blkmtd: checking erase region = 0x%08X size = 0x%X num = 0x%x\n",
- einfo->offset, einfo->erasesize, einfo->numblocks);
if(from >= einfo->offset
&& from < einfo->offset + (einfo->erasesize * einfo->numblocks)) {
if(len == einfo->erasesize
@@ -280,7 +276,6 @@
if(instr->state != MTD_ERASE_FAILED) {
/* do the erase */
- DEBUG(3, "Doing erase from = %zd len = %ld\n", from, len);
err = write_pages(dev, NULL, from, len, &retlen);
if(err || retlen != len) {
err("erase failed err = %d", err);
@@ -290,9 +285,7 @@
}
}
- DEBUG(3, "blkmtd: erase: checking callback\n");
mtd_erase_callback(instr);
- DEBUG(2, "blkmtd: erase: finished (err = %d)\n", err);
return err;
}
@@ -361,7 +354,6 @@
static void free_device(struct blkmtd_dev *dev)
{
- DEBUG(2, "blkmtd: free_device() dev = %p\n", dev);
if(dev) {
if(dev->mtd_info.eraseregions)
kfree(dev->mtd_info.eraseregions);
@@ -387,8 +379,6 @@
{
struct mtd_erase_region_info *info = NULL;
- DEBUG(2, "calc_erase_regions, es = %zd size = %zd regions = %d\n",
- erase_size, total_size, *regions);
/* Make any user specified erasesize be a power of 2
and at least PAGE_SIZE */
if(erase_size) {
@@ -416,8 +406,6 @@
if(count) {
tot_size = tot_size % er_size;
if(info) {
- DEBUG(2, "adding to erase info off=%d er=%d cnt=%d\n",
- offset, er_size, count);
(info+regcnt)->offset = offset;
(info+regcnt)->erasesize = er_size;
(info+regcnt)->numblocks = count;
@@ -435,8 +423,7 @@
break;
}
} while(!(*regions));
- DEBUG(2, "calc_erase_regions done, es = %zd size = %zd regions = %d\n",
- erase_size, total_size, *regions);
+
return info;
}
@@ -464,13 +451,9 @@
#endif
if(IS_ERR(bdev)) {
err("error: cannot open device %s", devname);
- DEBUG(2, "blkmtd: opening bdev returned %ld\n", PTR_ERR(bdev));
return NULL;
}
- DEBUG(1, "blkmtd: found a block device major = %d, minor = %d\n",
- MAJOR(bdev->bd_dev), MINOR(bdev->bd_dev));
-
if(MAJOR(bdev->bd_dev) == MTD_BLOCK_MAJOR) {
err("attempting to use an MTD device as a block device");
blkdev_put(bdev);
@@ -503,9 +486,6 @@
goto devinit_err;
dev->mtd_info.erasesize = dev->mtd_info.eraseregions->erasesize;
- DEBUG(1, "blkmtd: init: found %d erase regions\n",
- dev->mtd_info.numeraseregions);
-
dev->mtd_info.type = MTD_RAM;
dev->mtd_info.flags = MTD_CAP_RAM;
dev->mtd_info.erase = blkmtd_erase;
@@ -546,7 +526,6 @@
for(i = 0; i < MAX_DEVICES; i++) {
device[i] = str;
- DEBUG(2, "blkmtd: device setup: %d = %s\n", i, device[i]);
strsep(&str, ",");
}
return 1;
@@ -560,7 +539,6 @@
char *val = strsep(&str, ",");
if(val)
erasesz[i] = simple_strtoul(val, NULL, 0);
- DEBUG(2, "blkmtd: erasesz setup: %d = %d\n", i, erasesz[i]);
}
return 1;
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 7/22] Lindent
2004-12-21 13:47 ` [PATCH 6/22] Remove debug macros Jörn Engel
@ 2004-12-21 13:48 ` Jörn Engel
2004-12-21 13:49 ` [PATCH 8/22] Remove sync interface Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 13:48 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Ran some functions through Lindent. Mostly mindless stuff, some small
manual changes as well.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 158 +++++++++++++++++++++++++++++++------------------------------
1 files changed, 82 insertions(+), 76 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_lindent 2004-12-20 19:03:50.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 19:10:28.000000000 +0100
@@ -79,8 +79,8 @@
void cache_readahead(struct address_space *mapping, int index)
{
- filler_t *filler = (filler_t*)(mapping->a_ops->readpage);
- int i,pagei;
+ filler_t *filler = (void*)mapping->a_ops->readpage;
+ int i, pagei;
unsigned ret = 0;
unsigned long end_index;
struct page *page;
@@ -88,7 +88,7 @@
struct inode *inode = mapping->host;
loff_t isize = i_size_read(inode);
- if(!isize) {
+ if (!isize) {
printk(KERN_INFO "iSize=0 in cache_readahead\n");
return;
}
@@ -96,25 +96,28 @@
end_index = ((isize - 1) >> PAGE_CACHE_SHIFT);
spin_lock_irq(&mapping->tree_lock);
- for(i=0;i<PAGE_READAHEAD;i++) {
- pagei=index+i;
+ for (i = 0; i < PAGE_READAHEAD; i++) {
+ pagei = index + i;
if (pagei > end_index) {
printk(KERN_INFO "Overrun end of disk in cache readahead\n");
break;
}
page = radix_tree_lookup(&mapping->page_tree, pagei);
- if(page && (!i)) break;
- if(page) continue;
+ if (page && (!i))
+ break;
+ if (page)
+ continue;
spin_unlock_irq(&mapping->tree_lock);
page = page_cache_alloc_cold(mapping);
spin_lock_irq(&mapping->tree_lock);
- if(!page) break;
- page->index=pagei;
+ if (!page)
+ break;
+ page->index = pagei;
list_add(&page->lru, &page_pool);
ret++;
}
spin_unlock_irq(&mapping->tree_lock);
- if(ret) {
+ if (ret) {
read_cache_pages(mapping, &page_pool, filler, NULL);
//printk(KERN_INFO "Readahead [%d/%d] pages from [%d]\n",ret,PAGE_READAHEAD,index);
}
@@ -256,28 +259,26 @@
len = instr->len;
/* check erase region has valid start and length */
- while(numregions) {
- if(from >= einfo->offset
- && from < einfo->offset + (einfo->erasesize * einfo->numblocks)) {
- if(len == einfo->erasesize
- && ( (from - einfo->offset) % einfo->erasesize == 0))
+ while (numregions) {
+ if (from >= einfo->offset && from < einfo->offset + (einfo->erasesize * einfo->numblocks)) {
+ if (len == einfo->erasesize && ((from - einfo->offset) % einfo->erasesize == 0))
break;
}
numregions--;
einfo++;
}
- if(!numregions) {
+ if (!numregions) {
/* Not a valid erase block */
err("erase: invalid erase request 0x%lX @ 0x%08zX", len, from);
instr->state = MTD_ERASE_FAILED;
err = -EIO;
}
- if(instr->state != MTD_ERASE_FAILED) {
+ if (instr->state != MTD_ERASE_FAILED) {
/* do the erase */
err = write_pages(dev, NULL, from, len, &retlen);
- if(err || retlen != len) {
+ if (err || retlen != len) {
err("erase failed err = %d", err);
instr->state = MTD_ERASE_FAILED;
} else {
@@ -294,35 +295,41 @@
struct blkmtd_dev *dev = mtd->priv;
struct page *page;
//
- int index = from >> PAGE_SHIFT;
- int offset = from - (index << PAGE_SHIFT);
+ int index = from >> PAGE_SHIFT;
+ int offset = from - (index << PAGE_SHIFT);
int cpylen;
- if(from > mtd->size) return -EINVAL;
- if(from + len > mtd->size) len = mtd->size - from;
+ if (from > mtd->size)
+ return -EINVAL;
+ if (from + len > mtd->size)
+ len = mtd->size - from;
- if(retlen) *retlen = 0;
+ if (retlen)
+ *retlen = 0;
- while(len) {
- if((offset+len) > PAGE_SIZE)
+ while (len) {
+ if ((offset + len) > PAGE_SIZE)
cpylen = PAGE_SIZE - offset; // multiple pages
- else cpylen = len; // this page
+ else
+ cpylen = len; // this page
len = len - cpylen;
- //
- // Get page
- //
- page = page_readahead(dev->blkdev->bd_inode->i_mapping,index);
- if(!page) return -ENOMEM;
- if(IS_ERR(page)) return -EIO;
- //
+
+ // Get page
+ page = page_readahead(dev->blkdev->bd_inode->i_mapping, index);
+ if (!page)
+ return -ENOMEM;
+ if (IS_ERR(page))
+ return -EIO;
+
memcpy(buf, page_address(page) + offset, cpylen);
page_cache_release(page);
- //
- if(retlen) *retlen += cpylen;
- buf+=cpylen;
+
+ if (retlen)
+ *retlen += cpylen;
+ buf += cpylen;
offset = 0;
index++;
- //
+
}
return 0;
}
@@ -335,12 +342,16 @@
struct blkmtd_dev *dev = mtd->priv;
int err;
- if(!len) return 0;
- if(to >= mtd->size) return -ENOSPC;
- if(to + len > mtd->size) len = mtd->size - to;
+ if (!len)
+ return 0;
+ if (to >= mtd->size)
+ return -ENOSPC;
+ if (to + len > mtd->size)
+ len = mtd->size - to;
err = write_pages(dev, buf, to, len, retlen);
- if(err > 0) err = 0;
+ if (err > 0)
+ err = 0;
return err;
}
@@ -354,13 +365,13 @@
static void free_device(struct blkmtd_dev *dev)
{
- if(dev) {
- if(dev->mtd_info.eraseregions)
+ if (dev) {
+ if (dev->mtd_info.eraseregions)
kfree(dev->mtd_info.eraseregions);
- if(dev->mtd_info.name)
+ if (dev->mtd_info.name)
kfree(dev->mtd_info.name);
- if(dev->blkdev) {
+ if (dev->blkdev) {
invalidate_inode_pages(dev->blkdev->bd_inode->i_mapping);
close_bdev_excl(dev->blkdev);
}
@@ -381,14 +392,14 @@
/* Make any user specified erasesize be a power of 2
and at least PAGE_SIZE */
- if(erase_size) {
+ if (erase_size) {
int es = erase_size;
erase_size = 1;
- while(es != 1) {
+ while (es != 1) {
es >>= 1;
erase_size <<= 1;
}
- if(erase_size < PAGE_SIZE)
+ if (erase_size < PAGE_SIZE)
erase_size = PAGE_SIZE;
} else {
erase_size = CONFIG_MTD_BLKDEV_ERASESIZE;
@@ -401,28 +412,28 @@
int er_size = erase_size;
int count = 0, offset = 0, regcnt = 0;
- while(tot_size) {
+ while (tot_size) {
count = tot_size / er_size;
- if(count) {
+ if (count) {
tot_size = tot_size % er_size;
- if(info) {
- (info+regcnt)->offset = offset;
- (info+regcnt)->erasesize = er_size;
- (info+regcnt)->numblocks = count;
+ if (info) {
+ (info + regcnt)->offset = offset;
+ (info + regcnt)->erasesize = er_size;
+ (info + regcnt)->numblocks = count;
(*regions)++;
}
regcnt++;
offset += (count * er_size);
}
- while(er_size > tot_size)
+ while (er_size > tot_size)
er_size >>= 1;
}
- if(info == NULL) {
+ if (info == NULL) {
info = kmalloc(regcnt * sizeof(struct mtd_erase_region_info), GFP_KERNEL);
- if(!info)
+ if (!info)
break;
}
- } while(!(*regions));
+ } while (!(*regions));
return info;
}
@@ -436,12 +447,11 @@
int mode;
struct blkmtd_dev *dev;
- if(!devname)
+ if (!devname)
return NULL;
/* Get a handle on the device */
-
#ifdef MODULE
mode = O_RDWR;
bdev = open_bdev_excl(devname, mode, NULL);
@@ -449,19 +459,19 @@
mode = FMODE_WRITE;
bdev = open_by_devnum(name_to_dev_t(devname), mode);
#endif
- if(IS_ERR(bdev)) {
+ if (IS_ERR(bdev)) {
err("error: cannot open device %s", devname);
return NULL;
}
- if(MAJOR(bdev->bd_dev) == MTD_BLOCK_MAJOR) {
+ if (MAJOR(bdev->bd_dev) == MTD_BLOCK_MAJOR) {
err("attempting to use an MTD device as a block device");
blkdev_put(bdev);
return NULL;
}
dev = kmalloc(sizeof(struct blkmtd_dev), GFP_KERNEL);
- if(dev == NULL) {
+ if (dev == NULL) {
blkdev_put(bdev);
return NULL;
}
@@ -476,13 +486,12 @@
/* Setup the MTD structure */
/* make the name contain the block device in */
dev->mtd_info.name = kmalloc(sizeof("blkmtd: ") + strlen(devname), GFP_KERNEL);
- if(dev->mtd_info.name == NULL)
+ if (dev->mtd_info.name == NULL)
goto devinit_err;
sprintf(dev->mtd_info.name, "blkmtd: %s", devname);
- dev->mtd_info.eraseregions = calc_erase_regions(erase_size, dev->mtd_info.size,
- &dev->mtd_info.numeraseregions);
- if(dev->mtd_info.eraseregions == NULL)
+ dev->mtd_info.eraseregions = calc_erase_regions(erase_size, dev->mtd_info.size, &dev->mtd_info.numeraseregions);
+ if (dev->mtd_info.eraseregions == NULL)
goto devinit_err;
dev->mtd_info.erasesize = dev->mtd_info.eraseregions->erasesize;
@@ -503,14 +512,11 @@
list_del(&dev->list);
goto devinit_err;
} else {
- info("mtd%d: [%s] erase_size = %dKiB [%ld]",
- dev->mtd_info.index, dev->mtd_info.name + strlen("blkmtd: "),
- dev->mtd_info.erasesize >> 10,
- PAGE_SIZE);
+ info("mtd%d: [%s] erase_size = %dKiB [%ld]", dev->mtd_info.index, dev->mtd_info.name + strlen("blkmtd: "), dev->mtd_info.erasesize >> 10, PAGE_SIZE);
}
return dev;
- devinit_err:
+devinit_err:
free_device(dev);
return NULL;
}
@@ -524,7 +530,7 @@
{
int i;
- for(i = 0; i < MAX_DEVICES; i++) {
+ for (i = 0; i < MAX_DEVICES; i++) {
device[i] = str;
strsep(&str, ",");
}
@@ -535,9 +541,9 @@
static int __init param_blkmtd_erasesz(char *str)
{
int i;
- for(i = 0; i < MAX_DEVICES; i++) {
+ for (i = 0; i < MAX_DEVICES; i++) {
char *val = strsep(&str, ",");
- if(val)
+ if (val)
erasesz[i] = simple_strtoul(val, NULL, 0);
}
@@ -547,7 +553,7 @@
static int __init param_blkmtd_sync(char *str)
{
- if(str[0] == '1')
+ if (str[0] == '1')
sync = 1;
return 1;
}
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 8/22] Remove sync interface
2004-12-21 13:48 ` [PATCH 7/22] Lindent Jörn Engel
@ 2004-12-21 13:49 ` Jörn Engel
2004-12-21 13:51 ` [PATCH 9/22] Change parameter interface to phram-style Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 13:49 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
The sync interface was unused. Not it's gone.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 13 -------------
1 files changed, 13 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_sync 2004-12-20 19:10:28.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 21:38:46.000000000 +0100
@@ -55,14 +55,11 @@
/* Static info about the MTD, used in cleanup_module */
static LIST_HEAD(blkmtd_device_list);
-static void blkmtd_sync(struct mtd_info *mtd);
-
#define MAX_DEVICES 4
/* Module parameters passed by insmod/modprobe */
char *device[MAX_DEVICES]; /* the block device to use */
int erasesz[MAX_DEVICES]; /* optional default erase size */
-int sync;
MODULE_LICENSE("GPL");
@@ -72,8 +69,6 @@
MODULE_PARM_DESC(device, "block device to use");
MODULE_PARM(erasesz, "1-4i");
MODULE_PARM_DESC(erasesz, "optional erase size to use in KiB. eg 4=4KiB.");
-MODULE_PARM(sync, "i");
-MODULE_PARM_DESC(sync, "1=Synchronous writes");
#define PAGE_READAHEAD 64
@@ -551,16 +546,8 @@
}
-static int __init param_blkmtd_sync(char *str)
-{
- if (str[0] == '1')
- sync = 1;
- return 1;
-}
-
__setup("blkmtd_device=", param_blkmtd_device);
__setup("blkmtd_erasesz=", param_blkmtd_erasesz);
-__setup("blkmtd_sync=", param_blkmtd_sync);
#endif
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 9/22] Change parameter interface to phram-style
2004-12-21 13:49 ` [PATCH 8/22] Remove sync interface Jörn Engel
@ 2004-12-21 13:51 ` Jörn Engel
2004-12-21 13:53 ` [PATCH 10/22] Cleanup macro usage Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 13:51 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Much nicer. Same interface at boot time, module load and runtime via
/sys/. It also groups by device, so you can add devices on the fly if
you wish to.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 141 +++++++++++++++++++++++++++++++++++++++++--------------------
1 files changed, 97 insertions(+), 44 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_interface 2004-12-20 21:38:46.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 22:07:09.000000000 +0100
@@ -33,6 +33,8 @@
#include <linux/mtd/mtd.h>
#include <linux/buffer_head.h>
+#define ERROR(fmt, args...) printk(KERN_ERR "blockmtd: " fmt , ## args)
+
#define err(format, arg...) printk(KERN_ERR "blkmtd: " format "\n" , ## arg)
#define info(format, arg...) printk(KERN_INFO "blkmtd: " format "\n" , ## arg)
#define warn(format, arg...) printk(KERN_WARNING "blkmtd: " format "\n" , ## arg)
@@ -57,18 +59,6 @@
#define MAX_DEVICES 4
-/* Module parameters passed by insmod/modprobe */
-char *device[MAX_DEVICES]; /* the block device to use */
-int erasesz[MAX_DEVICES]; /* optional default erase size */
-
-
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Simon Evans <spse@secret.org.uk>");
-MODULE_DESCRIPTION("Emulate an MTD using a block device");
-MODULE_PARM(device, "1-4s");
-MODULE_PARM_DESC(device, "block device to use");
-MODULE_PARM(erasesz, "1-4i");
-MODULE_PARM_DESC(erasesz, "optional erase size to use in KiB. eg 4=4KiB.");
#define PAGE_READAHEAD 64
@@ -516,63 +506,122 @@
return NULL;
}
-#ifndef MODULE
-/* Handle kernel boot params */
+static int ustrtoul(const char *cp, char **endp, unsigned int base)
+{
+ unsigned long result = simple_strtoul(cp, endp, base);
+ switch (**endp) {
+ case 'G' :
+ result *= 1024;
+ case 'M':
+ result *= 1024;
+ case 'k':
+ result *= 1024;
+ /* By dwmw2 editorial decree, "ki", "Mi" or "Gi" are to be used. */
+ if ((*endp)[1] == 'i')
+ (*endp) += 2;
+ }
+ return result;
+}
-static int __init param_blkmtd_device(char *str)
+static int parse_num32(u32 *num32, const char *token)
{
- int i;
+ char *endp;
+ unsigned long n;
- for (i = 0; i < MAX_DEVICES; i++) {
- device[i] = str;
- strsep(&str, ",");
- }
- return 1;
+ n = ustrtoul(token, &endp, 0);
+ if (*endp)
+ return -EINVAL;
+
+ *num32 = n;
+ return 0;
}
-static int __init param_blkmtd_erasesz(char *str)
+static int parse_name(char **pname, const char *token, size_t limit)
{
- int i;
- for (i = 0; i < MAX_DEVICES; i++) {
- char *val = strsep(&str, ",");
- if (val)
- erasesz[i] = simple_strtoul(val, NULL, 0);
- }
+ size_t len;
+ char *name;
- return 1;
-}
+ len = strlen(token) + 1;
+ if (len > limit)
+ return -ENOSPC;
+ name = kmalloc(len, GFP_KERNEL);
+ if (!name)
+ return -ENOMEM;
-__setup("blkmtd_device=", param_blkmtd_device);
-__setup("blkmtd_erasesz=", param_blkmtd_erasesz);
+ strcpy(name, token);
-#endif
+ *pname = name;
+ return 0;
+}
-static int __init blockmtd_init(void)
+#define parse_err(fmt, args...) do { \
+ ERROR("blockmtd: " fmt "\n", ## args); \
+ return 0; \
+} while (0)
+
+static int blockmtd_setup(const char *val, struct kernel_param *kp)
{
- int i;
+ char buf[80+12], *str=buf; /* 80 for device, 12 for erase size */
+ char *token[2];
+ char *name;
+ size_t erase_size = 0;
+ int i, ret;
- info("version " VERSION);
- /* Check args - device[0] is the bare minimum */
- if (!device[0]) {
- err("error: missing `device' name\n");
- return -EINVAL;
+ if (strnlen(val, sizeof(buf)) >= sizeof(buf))
+ parse_err("parameter too long");
+
+ strcpy(str, val);
+
+ for (i=0; i<2; i++)
+ token[i] = strsep(&str, ",");
+
+ { /* people dislike typing "echo -n". and it's simple enough */
+ char *newline = strrchr(token[1], '\n');
+ if (newline && !newline[1])
+ *newline = 0;
}
- for (i = 0; i < MAX_DEVICES; i++)
- add_device(device[i], erasesz[i] << 10);
+ if (str)
+ parse_err("too many arguments");
- if (list_empty(&blkmtd_device_list))
- return -EINVAL;
+ if (!token[0])
+ parse_err("no argument");
+
+ ret = parse_name(&name, token[0], 80);
+ if (ret == -ENOMEM)
+ parse_err("out of memory");
+ if (ret == -ENOSPC)
+ parse_err("name too long");
+ if (ret)
+ return 0;
+
+ if (token[1]) {
+ ret = parse_num32(&erase_size, token[1]);
+ if (ret)
+ parse_err("illegal erase size");
+ }
+
+ add_device(name, erase_size);
return 0;
}
+module_param_call(blockmtd, blockmtd_setup, NULL, NULL, 0200);
+MODULE_PARM_DESC(blockmtd, "Device to use. \"blockmtd=<dev>[,<erasesize>]\"");
+
+static int __init blockmtd_init(void)
+{
+ info("version " VERSION);
+ return 0;
+}
+
+
static void __devexit blockmtd_exit(void)
{
struct list_head *temp1, *temp2;
@@ -593,3 +642,7 @@
module_init(blockmtd_init);
module_exit(blockmtd_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Simon Evans <spse@secret.org.uk>");
+MODULE_DESCRIPTION("Emulate an MTD using a block device");
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 10/22] Cleanup macro usage
2004-12-21 13:51 ` [PATCH 9/22] Change parameter interface to phram-style Jörn Engel
@ 2004-12-21 13:53 ` Jörn Engel
2004-12-21 13:54 ` [PATCH 11/22] kfree simplifications Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 13:53 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Change some macros. "err" and "info" were both macros and variables,
which is at best confusing to the reader.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 26 ++++++++++----------------
1 files changed, 10 insertions(+), 16 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_macros 2004-12-20 22:07:09.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 22:12:07.000000000 +0100
@@ -33,12 +33,8 @@
#include <linux/mtd/mtd.h>
#include <linux/buffer_head.h>
-#define ERROR(fmt, args...) printk(KERN_ERR "blockmtd: " fmt , ## args)
-
-#define err(format, arg...) printk(KERN_ERR "blkmtd: " format "\n" , ## arg)
-#define info(format, arg...) printk(KERN_INFO "blkmtd: " format "\n" , ## arg)
-#define warn(format, arg...) printk(KERN_WARNING "blkmtd: " format "\n" , ## arg)
-#define crit(format, arg...) printk(KERN_CRIT "blkmtd: " format "\n" , ## arg)
+#define ERROR(fmt, args...) printk(KERN_ERR "blockmtd: " fmt "\n" , ## args)
+#define INFO(fmt, args...) printk(KERN_INFO "blockmtd: " fmt "\n" , ## args)
/* Default erase size in K, always make it a multiple of PAGE_SIZE */
@@ -57,8 +53,6 @@
/* Static info about the MTD, used in cleanup_module */
static LIST_HEAD(blkmtd_device_list);
-#define MAX_DEVICES 4
-
#define PAGE_READAHEAD 64
@@ -143,7 +137,7 @@
err=0;
page = page_readahead(mapping,index);
if( unlikely(!page) ) {
- crit("unable to read page (%d) in write_pages\n",index);
+ ERROR("unable to read page (%d) in write_pages\n", index);
err=-ENOMEM;
}
//if(IS_ERR(page)) err=-EIO;
@@ -255,7 +249,7 @@
if (!numregions) {
/* Not a valid erase block */
- err("erase: invalid erase request 0x%lX @ 0x%08zX", len, from);
+ ERROR("erase: invalid erase request 0x%lX @ 0x%08zX", len,from);
instr->state = MTD_ERASE_FAILED;
err = -EIO;
}
@@ -264,7 +258,7 @@
/* do the erase */
err = write_pages(dev, NULL, from, len, &retlen);
if (err || retlen != len) {
- err("erase failed err = %d", err);
+ ERROR("erase failed err = %d", err);
instr->state = MTD_ERASE_FAILED;
} else {
instr->state = MTD_ERASE_DONE;
@@ -445,12 +439,12 @@
bdev = open_by_devnum(name_to_dev_t(devname), mode);
#endif
if (IS_ERR(bdev)) {
- err("error: cannot open device %s", devname);
+ ERROR("error: cannot open device %s", devname);
return NULL;
}
if (MAJOR(bdev->bd_dev) == MTD_BLOCK_MAJOR) {
- err("attempting to use an MTD device as a block device");
+ ERROR("attempting to use an MTD device as a block device");
blkdev_put(bdev);
return NULL;
}
@@ -497,7 +491,7 @@
list_del(&dev->list);
goto devinit_err;
} else {
- info("mtd%d: [%s] erase_size = %dKiB [%ld]", dev->mtd_info.index, dev->mtd_info.name + strlen("blkmtd: "), dev->mtd_info.erasesize >> 10, PAGE_SIZE);
+ INFO("mtd%d: [%s] erase_size = %dKiB [%ld]", dev->mtd_info.index, dev->mtd_info.name + strlen("blkmtd: "), dev->mtd_info.erasesize >> 10, PAGE_SIZE);
}
return dev;
@@ -617,7 +611,7 @@
static int __init blockmtd_init(void)
{
- info("version " VERSION);
+ INFO("version " VERSION);
return 0;
}
@@ -632,7 +626,7 @@
list);
blkmtd_sync(&dev->mtd_info);
del_mtd_device(&dev->mtd_info);
- info("mtd%d: [%s] removed", dev->mtd_info.index,
+ INFO("mtd%d: [%s] removed", dev->mtd_info.index,
dev->mtd_info.name + strlen("blkmtd: "));
list_del(&dev->list);
free_device(dev);
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 11/22] kfree simplifications
2004-12-21 13:53 ` [PATCH 10/22] Cleanup macro usage Jörn Engel
@ 2004-12-21 13:54 ` Jörn Engel
2004-12-21 13:55 ` [PATCH 12/22] change blockmtd_sync Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 13:54 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
kfree(NULL) works just fine, which allows for much simpler code.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 26 +++++++++++++-------------
1 files changed, 13 insertions(+), 13 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_kfree 2004-12-20 22:12:07.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 22:14:47.000000000 +0100
@@ -342,20 +342,20 @@
return;
}
-static void free_device(struct blkmtd_dev *dev)
+static void blockmtd_free_device(struct blkmtd_dev *dev)
{
- if (dev) {
- if (dev->mtd_info.eraseregions)
- kfree(dev->mtd_info.eraseregions);
- if (dev->mtd_info.name)
- kfree(dev->mtd_info.name);
+ if (!dev)
+ return;
- if (dev->blkdev) {
- invalidate_inode_pages(dev->blkdev->bd_inode->i_mapping);
- close_bdev_excl(dev->blkdev);
- }
- kfree(dev);
+ kfree(dev->mtd_info.eraseregions);
+ kfree(dev->mtd_info.name);
+
+ if (dev->blkdev) {
+ invalidate_inode_pages(dev->blkdev->bd_inode->i_mapping);
+ close_bdev_excl(dev->blkdev);
}
+
+ kfree(dev);
}
@@ -496,7 +496,7 @@
return dev;
devinit_err:
- free_device(dev);
+ blockmtd_free_device(dev);
return NULL;
}
@@ -629,7 +629,7 @@
INFO("mtd%d: [%s] removed", dev->mtd_info.index,
dev->mtd_info.name + strlen("blkmtd: "));
list_del(&dev->list);
- free_device(dev);
+ blockmtd_free_device(dev);
}
}
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 12/22] change blockmtd_sync
2004-12-21 13:54 ` [PATCH 11/22] kfree simplifications Jörn Engel
@ 2004-12-21 13:55 ` Jörn Engel
2004-12-21 13:57 ` [PATCH 13/22] remove erase regions Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 13:55 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
I hate casts. This adds a line, but is much simpler to read. Also
renames the function.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 10 ++++++----
1 files changed, 6 insertions(+), 4 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_sync2 2004-12-20 22:14:47.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 22:16:31.000000000 +0100
@@ -336,12 +336,14 @@
/* sync the device - wait until the write queue is empty */
-static void blkmtd_sync(struct mtd_info *mtd)
+static void blockmtd_sync(struct mtd_info *mtd)
{
- sync_blockdev(((struct blkmtd_dev*)mtd->priv)->blkdev);
+ struct blkmtd_dev *dev = mtd->priv;
+ sync_blockdev(dev->blkdev);
return;
}
+
static void blockmtd_free_device(struct blkmtd_dev *dev)
{
if (!dev)
@@ -479,7 +481,7 @@
dev->mtd_info.erase = blkmtd_erase;
dev->mtd_info.write = blkmtd_write;
dev->mtd_info.writev = default_mtd_writev;
- dev->mtd_info.sync = blkmtd_sync;
+ dev->mtd_info.sync = blockmtd_sync;
dev->mtd_info.read = blkmtd_read;
dev->mtd_info.readv = default_mtd_readv;
dev->mtd_info.priv = dev;
@@ -624,7 +626,7 @@
list_for_each_safe(temp1, temp2, &blkmtd_device_list) {
struct blkmtd_dev *dev = list_entry(temp1, struct blkmtd_dev,
list);
- blkmtd_sync(&dev->mtd_info);
+ blockmtd_sync(&dev->mtd_info);
del_mtd_device(&dev->mtd_info);
INFO("mtd%d: [%s] removed", dev->mtd_info.index,
dev->mtd_info.name + strlen("blkmtd: "));
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 13/22] remove erase regions
2004-12-21 13:55 ` [PATCH 12/22] change blockmtd_sync Jörn Engel
@ 2004-12-21 13:57 ` Jörn Engel
2004-12-21 14:01 ` [PATCH 14/22] Change add_device Jörn Engel
2004-12-21 18:42 ` [PATCH 13/22] remove erase regions Christopher Hoover
0 siblings, 2 replies; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 13:57 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
I see absolutely no reason for complicated erase reagions. On the
user side, everyone but mtdchar effectively ignores it anyway.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 64 -------------------------------------------------------------
1 files changed, 1 insertion(+), 63 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_erase_regions 2004-12-20 22:16:31.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 22:21:50.000000000 +0100
@@ -361,65 +361,6 @@
}
-/* For a given size and initial erase size, calculate the number
- * and size of each erase region. Goes round the loop twice,
- * once to find out how many regions, then allocates space,
- * then round the loop again to fill it in.
- */
-static struct mtd_erase_region_info *calc_erase_regions(
- size_t erase_size, size_t total_size, int *regions)
-{
- struct mtd_erase_region_info *info = NULL;
-
- /* Make any user specified erasesize be a power of 2
- and at least PAGE_SIZE */
- if (erase_size) {
- int es = erase_size;
- erase_size = 1;
- while (es != 1) {
- es >>= 1;
- erase_size <<= 1;
- }
- if (erase_size < PAGE_SIZE)
- erase_size = PAGE_SIZE;
- } else {
- erase_size = CONFIG_MTD_BLKDEV_ERASESIZE;
- }
-
- *regions = 0;
-
- do {
- int tot_size = total_size;
- int er_size = erase_size;
- int count = 0, offset = 0, regcnt = 0;
-
- while (tot_size) {
- count = tot_size / er_size;
- if (count) {
- tot_size = tot_size % er_size;
- if (info) {
- (info + regcnt)->offset = offset;
- (info + regcnt)->erasesize = er_size;
- (info + regcnt)->numblocks = count;
- (*regions)++;
- }
- regcnt++;
- offset += (count * er_size);
- }
- while (er_size > tot_size)
- er_size >>= 1;
- }
- if (info == NULL) {
- info = kmalloc(regcnt * sizeof(struct mtd_erase_region_info), GFP_KERNEL);
- if (!info)
- break;
- }
- } while (!(*regions));
-
- return info;
-}
-
-
extern dev_t __init name_to_dev_t(const char *line);
static struct blkmtd_dev *add_device(char *devname, int erase_size)
@@ -471,11 +412,8 @@
goto devinit_err;
sprintf(dev->mtd_info.name, "blkmtd: %s", devname);
- dev->mtd_info.eraseregions = calc_erase_regions(erase_size, dev->mtd_info.size, &dev->mtd_info.numeraseregions);
- if (dev->mtd_info.eraseregions == NULL)
- goto devinit_err;
- dev->mtd_info.erasesize = dev->mtd_info.eraseregions->erasesize;
+ dev->mtd_info.erasesize = erase_size;
dev->mtd_info.type = MTD_RAM;
dev->mtd_info.flags = MTD_CAP_RAM;
dev->mtd_info.erase = blkmtd_erase;
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 14/22] Change add_device
2004-12-21 13:57 ` [PATCH 13/22] remove erase regions Jörn Engel
@ 2004-12-21 14:01 ` Jörn Engel
2004-12-21 14:02 ` [PATCH 15/22] Rename unreadable mutex Jörn Engel
2004-12-21 18:42 ` [PATCH 13/22] remove erase regions Christopher Hoover
1 sibling, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 14:01 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
o Remove the last remaining #ifdef.
o s/mtd_info/mtd/ makes nicer reading.
o General cleanup to add_device
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 86 ++++++++++++++++++++++++++-----------------------------------
1 files changed, 37 insertions(+), 49 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_ifdef 2004-12-20 22:21:50.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 22:38:34.000000000 +0100
@@ -45,7 +45,7 @@
struct blkmtd_dev {
struct list_head list;
struct block_device *blkdev;
- struct mtd_info mtd_info;
+ struct mtd_info mtd;
struct semaphore wrbuf_mutex;
};
@@ -349,8 +349,8 @@
if (!dev)
return;
- kfree(dev->mtd_info.eraseregions);
- kfree(dev->mtd_info.name);
+ kfree(dev->mtd.eraseregions);
+ kfree(dev->mtd.name);
if (dev->blkdev) {
invalidate_inode_pages(dev->blkdev->bd_inode->i_mapping);
@@ -361,78 +361,66 @@
}
-extern dev_t __init name_to_dev_t(const char *line);
-
static struct blkmtd_dev *add_device(char *devname, int erase_size)
{
struct block_device *bdev;
- int mode;
struct blkmtd_dev *dev;
if (!devname)
return NULL;
- /* Get a handle on the device */
+ dev = kmalloc(sizeof(struct blkmtd_dev), GFP_KERNEL);
+ if (!dev)
+ return NULL;
+ memset(dev, 0, sizeof(*dev));
-#ifdef MODULE
- mode = O_RDWR;
- bdev = open_bdev_excl(devname, mode, NULL);
-#else
- mode = FMODE_WRITE;
- bdev = open_by_devnum(name_to_dev_t(devname), mode);
-#endif
+ /* Get a handle on the device */
+ bdev = open_bdev_excl(devname, O_RDWR, NULL);
if (IS_ERR(bdev)) {
ERROR("error: cannot open device %s", devname);
- return NULL;
+ goto devinit_err;
}
+ dev->blkdev = bdev;
if (MAJOR(bdev->bd_dev) == MTD_BLOCK_MAJOR) {
ERROR("attempting to use an MTD device as a block device");
- blkdev_put(bdev);
- return NULL;
- }
-
- dev = kmalloc(sizeof(struct blkmtd_dev), GFP_KERNEL);
- if (dev == NULL) {
- blkdev_put(bdev);
- return NULL;
+ goto devinit_err;
}
- memset(dev, 0, sizeof(struct blkmtd_dev));
- dev->blkdev = bdev;
- atomic_set(&(dev->blkdev->bd_inode->i_mapping->truncate_count), 0);
+ atomic_set(&bdev->bd_inode->i_mapping->truncate_count, 0);
init_MUTEX(&dev->wrbuf_mutex);
- dev->mtd_info.size = dev->blkdev->bd_inode->i_size & PAGE_MASK;
-
/* Setup the MTD structure */
/* make the name contain the block device in */
- dev->mtd_info.name = kmalloc(sizeof("blkmtd: ") + strlen(devname), GFP_KERNEL);
- if (dev->mtd_info.name == NULL)
+ dev->mtd.name = kmalloc(sizeof("blockmtd: ") + strlen(devname),
+ GFP_KERNEL);
+ if (!dev->mtd.name)
goto devinit_err;
- sprintf(dev->mtd_info.name, "blkmtd: %s", devname);
+ sprintf(dev->mtd.name, "blockmtd: %s", devname);
- dev->mtd_info.erasesize = erase_size;
- dev->mtd_info.type = MTD_RAM;
- dev->mtd_info.flags = MTD_CAP_RAM;
- dev->mtd_info.erase = blkmtd_erase;
- dev->mtd_info.write = blkmtd_write;
- dev->mtd_info.writev = default_mtd_writev;
- dev->mtd_info.sync = blockmtd_sync;
- dev->mtd_info.read = blkmtd_read;
- dev->mtd_info.readv = default_mtd_readv;
- dev->mtd_info.priv = dev;
- dev->mtd_info.owner = THIS_MODULE;
+ dev->mtd.size = dev->blkdev->bd_inode->i_size & PAGE_MASK;
+ dev->mtd.erasesize = erase_size;
+ dev->mtd.type = MTD_RAM;
+ dev->mtd.flags = MTD_CAP_RAM;
+ dev->mtd.erase = blkmtd_erase;
+ dev->mtd.write = blkmtd_write;
+ dev->mtd.writev = default_mtd_writev;
+ dev->mtd.sync = blockmtd_sync;
+ dev->mtd.read = blkmtd_read;
+ dev->mtd.readv = default_mtd_readv;
+ dev->mtd.priv = dev;
+ dev->mtd.owner = THIS_MODULE;
list_add(&dev->list, &blkmtd_device_list);
- if (add_mtd_device(&dev->mtd_info)) {
+ if (add_mtd_device(&dev->mtd)) {
/* Device didnt get added, so free the entry */
list_del(&dev->list);
goto devinit_err;
- } else {
- INFO("mtd%d: [%s] erase_size = %dKiB [%ld]", dev->mtd_info.index, dev->mtd_info.name + strlen("blkmtd: "), dev->mtd_info.erasesize >> 10, PAGE_SIZE);
}
+ INFO("mtd%d: [%s] erase_size = %dKiB [%ld]", dev->mtd.index,
+ dev->mtd.name + strlen("blkmtd: "),
+ dev->mtd.erasesize >> 10, PAGE_SIZE);
return dev;
devinit_err:
@@ -564,10 +552,10 @@
list_for_each_safe(temp1, temp2, &blkmtd_device_list) {
struct blkmtd_dev *dev = list_entry(temp1, struct blkmtd_dev,
list);
- blockmtd_sync(&dev->mtd_info);
- del_mtd_device(&dev->mtd_info);
- INFO("mtd%d: [%s] removed", dev->mtd_info.index,
- dev->mtd_info.name + strlen("blkmtd: "));
+ blockmtd_sync(&dev->mtd);
+ del_mtd_device(&dev->mtd);
+ INFO("mtd%d: [%s] removed", dev->mtd.index,
+ dev->mtd.name + strlen("blkmtd: "));
list_del(&dev->list);
blockmtd_free_device(dev);
}
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 15/22] Rename unreadable mutex
2004-12-21 14:01 ` [PATCH 14/22] Change add_device Jörn Engel
@ 2004-12-21 14:02 ` Jörn Engel
2004-12-21 14:03 ` [PATCH 16/22] list changes Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 14:02 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
If I cannot pronounce a name, my reading abilities degrade.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 10 +++++-----
1 files changed, 5 insertions(+), 5 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_mutex 2004-12-20 22:38:34.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 22:41:15.000000000 +0100
@@ -46,7 +46,7 @@
struct list_head list;
struct block_device *blkdev;
struct mtd_info mtd;
- struct semaphore wrbuf_mutex;
+ struct semaphore write_mutex;
};
@@ -215,9 +215,9 @@
}
err=0;
- down(&dev->wrbuf_mutex);
+ down(&dev->write_mutex);
err = buf ? my_write() : my_erase();
- up(&dev->wrbuf_mutex);
+ up(&dev->write_mutex);
return err;
}
@@ -388,7 +388,7 @@
}
atomic_set(&bdev->bd_inode->i_mapping->truncate_count, 0);
- init_MUTEX(&dev->wrbuf_mutex);
+ init_MUTEX(&dev->write_mutex);
/* Setup the MTD structure */
/* make the name contain the block device in */
@@ -566,5 +566,5 @@
module_exit(blockmtd_exit);
MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Simon Evans <spse@secret.org.uk>");
+MODULE_AUTHOR("Simon Evans <spse@secret.org.uk> and others");
MODULE_DESCRIPTION("Emulate an MTD using a block device");
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 16/22] list changes
2004-12-21 14:02 ` [PATCH 15/22] Rename unreadable mutex Jörn Engel
@ 2004-12-21 14:03 ` Jörn Engel
2004-12-21 14:04 ` [PATCH 17/22] Rename central struct Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 14:03 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Change some list.h calls.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 10 ++++------
1 files changed, 4 insertions(+), 6 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_list 2004-12-20 22:41:15.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-20 22:53:00.000000000 +0100
@@ -412,12 +412,11 @@
dev->mtd.priv = dev;
dev->mtd.owner = THIS_MODULE;
- list_add(&dev->list, &blkmtd_device_list);
if (add_mtd_device(&dev->mtd)) {
/* Device didnt get added, so free the entry */
- list_del(&dev->list);
goto devinit_err;
}
+ list_add(&dev->list, &blkmtd_device_list);
INFO("mtd%d: [%s] erase_size = %dKiB [%ld]", dev->mtd.index,
dev->mtd.name + strlen("blkmtd: "),
dev->mtd.erasesize >> 10, PAGE_SIZE);
@@ -546,12 +545,11 @@
static void __devexit blockmtd_exit(void)
{
- struct list_head *temp1, *temp2;
+ struct list_head *pos, *next;
/* Remove the MTD devices */
- list_for_each_safe(temp1, temp2, &blkmtd_device_list) {
- struct blkmtd_dev *dev = list_entry(temp1, struct blkmtd_dev,
- list);
+ list_for_each_safe(pos, next, &blkmtd_device_list) {
+ struct blkmtd_dev *dev = list_entry(pos, typeof(*dev), list);
blockmtd_sync(&dev->mtd);
del_mtd_device(&dev->mtd);
INFO("mtd%d: [%s] removed", dev->mtd.index,
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 17/22] Rename central struct
2004-12-21 14:03 ` [PATCH 16/22] list changes Jörn Engel
@ 2004-12-21 14:04 ` Jörn Engel
2004-12-21 14:06 ` [PATCH 18/22] Function renaming Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 14:04 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
s/blkmtd_dev/blockmtd_dev/
Maybe "struct blockmtd" without the "_dev" would be even nicer.
Later.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 22 +++++++++++-----------
1 files changed, 11 insertions(+), 11 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_structname 2004-12-20 22:53:00.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-21 01:07:33.000000000 +0100
@@ -42,7 +42,7 @@
#define VERSION "$Revision: 1.23 $"
/* Info for the block device */
-struct blkmtd_dev {
+struct blockmtd_dev {
struct list_head list;
struct block_device *blkdev;
struct mtd_info mtd;
@@ -122,7 +122,7 @@
* out.
*/
-static int write_pages(struct blkmtd_dev *dev, const u_char *buf, loff_t to, size_t len, size_t *retlen)
+static int write_pages(struct blockmtd_dev *dev, const u_char *buf, loff_t to, size_t len, size_t *retlen)
{
struct page *page;
struct address_space *mapping = dev->blkdev->bd_inode->i_mapping;
@@ -225,7 +225,7 @@
/* erase a specified part of the device */
static int blkmtd_erase(struct mtd_info *mtd, struct erase_info *instr)
{
- struct blkmtd_dev *dev = mtd->priv;
+ struct blockmtd_dev *dev = mtd->priv;
struct mtd_erase_region_info *einfo = mtd->eraseregions;
int numregions = mtd->numeraseregions;
size_t from;
@@ -271,7 +271,7 @@
static int blkmtd_read(struct mtd_info *mtd, loff_t from, size_t len, size_t *retlen, u_char *buf)
{
- struct blkmtd_dev *dev = mtd->priv;
+ struct blockmtd_dev *dev = mtd->priv;
struct page *page;
//
int index = from >> PAGE_SHIFT;
@@ -318,7 +318,7 @@
static int blkmtd_write(struct mtd_info *mtd, loff_t to, size_t len,
size_t *retlen, const u_char *buf)
{
- struct blkmtd_dev *dev = mtd->priv;
+ struct blockmtd_dev *dev = mtd->priv;
int err;
if (!len)
@@ -338,13 +338,13 @@
/* sync the device - wait until the write queue is empty */
static void blockmtd_sync(struct mtd_info *mtd)
{
- struct blkmtd_dev *dev = mtd->priv;
+ struct blockmtd_dev *dev = mtd->priv;
sync_blockdev(dev->blkdev);
return;
}
-static void blockmtd_free_device(struct blkmtd_dev *dev)
+static void blockmtd_free_device(struct blockmtd_dev *dev)
{
if (!dev)
return;
@@ -361,15 +361,15 @@
}
-static struct blkmtd_dev *add_device(char *devname, int erase_size)
+static struct blockmtd_dev *add_device(char *devname, int erase_size)
{
struct block_device *bdev;
- struct blkmtd_dev *dev;
+ struct blockmtd_dev *dev;
if (!devname)
return NULL;
- dev = kmalloc(sizeof(struct blkmtd_dev), GFP_KERNEL);
+ dev = kmalloc(sizeof(struct blockmtd_dev), GFP_KERNEL);
if (!dev)
return NULL;
memset(dev, 0, sizeof(*dev));
@@ -549,7 +549,7 @@
/* Remove the MTD devices */
list_for_each_safe(pos, next, &blkmtd_device_list) {
- struct blkmtd_dev *dev = list_entry(pos, typeof(*dev), list);
+ struct blockmtd_dev *dev = list_entry(pos, typeof(*dev), list);
blockmtd_sync(&dev->mtd);
del_mtd_device(&dev->mtd);
INFO("mtd%d: [%s] removed", dev->mtd.index,
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 18/22] Function renaming
2004-12-21 14:04 ` [PATCH 17/22] Rename central struct Jörn Engel
@ 2004-12-21 14:06 ` Jörn Engel
2004-12-21 14:09 ` [PATCH 19/22] Fold various erase functions Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 14:06 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Renames _erase and _read functions and removes some leftover erase
region stuff.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 65 +++++++++++++++++++------------------------------------------
1 files changed, 21 insertions(+), 44 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_renames 2004-12-20 22:54:59.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-21 01:05:20.000000000 +0100
@@ -223,59 +223,36 @@
/* erase a specified part of the device */
-static int blkmtd_erase(struct mtd_info *mtd, struct erase_info *instr)
+static int blockmtd_erase(struct mtd_info *mtd, struct erase_info *instr)
{
struct blockmtd_dev *dev = mtd->priv;
- struct mtd_erase_region_info *einfo = mtd->eraseregions;
- int numregions = mtd->numeraseregions;
- size_t from;
- u_long len;
- int err = -EIO;
+ size_t from = instr->addr;
+ size_t len = instr->len;
size_t retlen;
+ int err;
instr->state = MTD_ERASING;
- from = instr->addr;
- len = instr->len;
-
- /* check erase region has valid start and length */
- while (numregions) {
- if (from >= einfo->offset && from < einfo->offset + (einfo->erasesize * einfo->numblocks)) {
- if (len == einfo->erasesize && ((from - einfo->offset) % einfo->erasesize == 0))
- break;
- }
- numregions--;
- einfo++;
- }
-
- if (!numregions) {
- /* Not a valid erase block */
- ERROR("erase: invalid erase request 0x%lX @ 0x%08zX", len,from);
+ err = write_pages(dev, NULL, from, len, &retlen);
+ if (err || retlen != len) {
+ ERROR("erase failed err = %d", err);
instr->state = MTD_ERASE_FAILED;
- err = -EIO;
- }
-
- if (instr->state != MTD_ERASE_FAILED) {
- /* do the erase */
- err = write_pages(dev, NULL, from, len, &retlen);
- if (err || retlen != len) {
- ERROR("erase failed err = %d", err);
- instr->state = MTD_ERASE_FAILED;
- } else {
- instr->state = MTD_ERASE_DONE;
- }
+ } else {
+ instr->state = MTD_ERASE_DONE;
}
+ instr->state = MTD_ERASE_DONE;
mtd_erase_callback(instr);
return err;
}
-static int blkmtd_read(struct mtd_info *mtd, loff_t from, size_t len, size_t *retlen, u_char *buf)
+
+static int blockmtd_read(struct mtd_info *mtd, loff_t from, size_t len,
+ size_t *retlen, u_char *buf)
{
struct blockmtd_dev *dev = mtd->priv;
struct page *page;
- //
int index = from >> PAGE_SHIFT;
- int offset = from - (index << PAGE_SHIFT);
+ int offset = from & (PAGE_SHIFT-1);
int cpylen;
if (from > mtd->size)
@@ -298,7 +275,7 @@
if (!page)
return -ENOMEM;
if (IS_ERR(page))
- return -EIO;
+ return PTR_ERR(page);
memcpy(buf, page_address(page) + offset, cpylen);
page_cache_release(page);
@@ -308,15 +285,14 @@
buf += cpylen;
offset = 0;
index++;
-
}
return 0;
}
/* write data to the underlying device */
-static int blkmtd_write(struct mtd_info *mtd, loff_t to, size_t len,
- size_t *retlen, const u_char *buf)
+static int blockmtd_write(struct mtd_info *mtd, loff_t to, size_t len,
+ size_t *retlen, const u_char *buf)
{
struct blockmtd_dev *dev = mtd->priv;
int err;
@@ -361,6 +337,7 @@
}
+/* FIXME: ensure that mtd->size % erase_size == 0 */
static struct blockmtd_dev *add_device(char *devname, int erase_size)
{
struct block_device *bdev;
@@ -403,11 +380,11 @@
dev->mtd.erasesize = erase_size;
dev->mtd.type = MTD_RAM;
dev->mtd.flags = MTD_CAP_RAM;
- dev->mtd.erase = blkmtd_erase;
- dev->mtd.write = blkmtd_write;
+ dev->mtd.erase = blockmtd_erase;
+ dev->mtd.write = blockmtd_write;
dev->mtd.writev = default_mtd_writev;
dev->mtd.sync = blockmtd_sync;
- dev->mtd.read = blkmtd_read;
+ dev->mtd.read = blockmtd_read;
dev->mtd.readv = default_mtd_readv;
dev->mtd.priv = dev;
dev->mtd.owner = THIS_MODULE;
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 19/22] Fold various erase functions
2004-12-21 14:06 ` [PATCH 18/22] Function renaming Jörn Engel
@ 2004-12-21 14:09 ` Jörn Engel
2004-12-21 14:10 ` [PATCH 20/22] Fold various write functions Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 14:09 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
After the previous 18 patches and your work, it becomes obvious that
write_pages() is effectively two functions with very little overlap.
This patch moves the erase part of that function out and simplifies
it.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 117 ++++++++++++++++++++++++++++++-------------------------------
1 files changed, 58 insertions(+), 59 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_erase 2004-12-21 01:52:12.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-21 02:12:15.000000000 +0100
@@ -121,70 +121,35 @@
* part of the page being modified. Pages are added to the bio and then written
* out.
*/
-
-static int write_pages(struct blockmtd_dev *dev, const u_char *buf, loff_t to, size_t len, size_t *retlen)
+static int my_read(struct page **page, struct address_space *mapping, int index)
+{
+ int err = 0;
+
+ *page = page_readahead(mapping, index);
+ if (unlikely(!*page)) {
+ ERROR("unable to read page (%d) in write_pages\n", index);
+ err = -ENOMEM;
+ }
+ if (IS_ERR(*page))
+ err = PTR_ERR(*page);
+ if (err)
+ printk("READ ERROR\n");
+ return err;
+}
+static int write_pages(struct blockmtd_dev *dev, const u_char *buf, loff_t to,
+ size_t len, size_t *retlen)
{
struct page *page;
struct address_space *mapping = dev->blkdev->bd_inode->i_mapping;
- //
int err = 0; // return status
int index = to >> PAGE_SHIFT; // page index
int offset = to & ~PAGE_MASK; // page offset
- //
-
- static int my_read(void)
- {
- err=0;
- page = page_readahead(mapping,index);
- if( unlikely(!page) ) {
- ERROR("unable to read page (%d) in write_pages\n", index);
- err=-ENOMEM;
- }
- //if(IS_ERR(page)) err=-EIO;
- if(err) printk("READ ERROR\n");
- return err;
- }
- //
- static int my_erase(void)
- {
- int pages = len >> PAGE_SHIFT;
- unsigned char *p;
- unsigned char *max;
- int format;
-
- while(pages) {
- //printk(KERN_INFO "readahead on index (%d)\n",index);
- if(my_read()) return err;
-
- max = ((char*)page_address(page)+PAGE_SIZE);
- format = 0;
-
- for(p=(unsigned char*)page_address(page); p<max; p++)
- if(*p != 0xff) {
- format=1;
- break;
- }
-
- if(format) {
- lock_page(page);
- memset(page_address(page), 0xff, PAGE_SIZE);
- set_page_dirty(page);
- unlock_page(page);
- }
- page_cache_release(page);
- pages--;
- index++;
- }
- *retlen = len;
- return 0;
- }
static int my_write(void)
{
int cpylen;
if(retlen) *retlen = 0;
- //printk(KERN_INFO "Write req: %d\n",len);
while(len) {
if((offset+len) > PAGE_SIZE)
cpylen = PAGE_SIZE - offset; // multiple pages
@@ -193,7 +158,9 @@
//
// Get page
//
- if(my_read()) return err;
+ err = my_read(&page, mapping, index);
+ if (err)
+ return err;
//
if(memcmp(page_address(page)+offset,buf,cpylen)) {
lock_page(page);
@@ -216,29 +183,61 @@
err=0;
down(&dev->write_mutex);
- err = buf ? my_write() : my_erase();
+ err = my_write();
up(&dev->write_mutex);
return err;
}
/* erase a specified part of the device */
+static int _blockmtd_erase(struct blockmtd_dev *dev, loff_t to, size_t len)
+{
+ struct address_space *mapping = dev->blkdev->bd_inode->i_mapping;
+ struct page *page;
+ int index = to >> PAGE_SHIFT; // page index
+ int pages = len >> PAGE_SHIFT;
+ u_long *p;
+ u_long *max;
+
+ while (pages) {
+ page = page_readahead(mapping, index);
+ if (!page)
+ return -ENOMEM;
+ if (IS_ERR(page))
+ return PTR_ERR(page);
+
+ max = (u_long*)page_address(page) + PAGE_SIZE;
+ for (p=(u_long*)page_address(page); p<max; p++)
+ if (*p != -1UL) {
+ lock_page(page);
+ memset(page_address(page), 0xff, PAGE_SIZE);
+ set_page_dirty(page);
+ unlock_page(page);
+ break;
+ }
+
+ page_cache_release(page);
+ pages--;
+ index++;
+ }
+ return 0;
+}
static int blockmtd_erase(struct mtd_info *mtd, struct erase_info *instr)
{
struct blockmtd_dev *dev = mtd->priv;
size_t from = instr->addr;
size_t len = instr->len;
- size_t retlen;
int err;
instr->state = MTD_ERASING;
- err = write_pages(dev, NULL, from, len, &retlen);
- if (err || retlen != len) {
+ down(&dev->write_mutex);
+ err = _blockmtd_erase(dev, from, len);
+ up(&dev->write_mutex);
+ if (err) {
ERROR("erase failed err = %d", err);
instr->state = MTD_ERASE_FAILED;
- } else {
+ } else
instr->state = MTD_ERASE_DONE;
- }
instr->state = MTD_ERASE_DONE;
mtd_erase_callback(instr);
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 20/22] Fold various write functions
2004-12-21 14:09 ` [PATCH 19/22] Fold various erase functions Jörn Engel
@ 2004-12-21 14:10 ` Jörn Engel
2004-12-21 14:11 ` [PATCH 21/22] Default erase size Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 14:10 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
With the erase half gone, the remaining write part can be folded into
blockmtd_write as well. Here we go.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 126 ++++++++++++++++++++++---------------------------------------
1 files changed, 46 insertions(+), 80 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_write 2004-12-21 02:12:15.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-21 02:19:58.000000000 +0100
@@ -102,91 +102,13 @@
}
}
+
static struct page* page_readahead(struct address_space *mapping, int index)
{
filler_t *filler = (filler_t*)(mapping->a_ops->readpage);
cache_readahead(mapping,index);
return read_cache_page(mapping, index, filler, NULL);
}
-/**
- * write_pages - write block of data to device via the page cache
- * @dev: device to write to
- * @buf: data source or NULL if erase (output is set to 0xff)
- * @to: offset into output device
- * @len: amount to data to write
- * @retlen: amount of data written
- *
- * Grab pages from the page cache and fill them with the source data.
- * Non page aligned start and end result in a readin of the page and
- * part of the page being modified. Pages are added to the bio and then written
- * out.
- */
-static int my_read(struct page **page, struct address_space *mapping, int index)
-{
- int err = 0;
-
- *page = page_readahead(mapping, index);
- if (unlikely(!*page)) {
- ERROR("unable to read page (%d) in write_pages\n", index);
- err = -ENOMEM;
- }
- if (IS_ERR(*page))
- err = PTR_ERR(*page);
- if (err)
- printk("READ ERROR\n");
- return err;
-}
-static int write_pages(struct blockmtd_dev *dev, const u_char *buf, loff_t to,
- size_t len, size_t *retlen)
-{
- struct page *page;
- struct address_space *mapping = dev->blkdev->bd_inode->i_mapping;
- int err = 0; // return status
- int index = to >> PAGE_SHIFT; // page index
- int offset = to & ~PAGE_MASK; // page offset
-
- static int my_write(void)
- {
- int cpylen;
-
- if(retlen) *retlen = 0;
- while(len) {
- if((offset+len) > PAGE_SIZE)
- cpylen = PAGE_SIZE - offset; // multiple pages
- else cpylen = len; // this page
- len = len - cpylen;
- //
- // Get page
- //
- err = my_read(&page, mapping, index);
- if (err)
- return err;
- //
- if(memcmp(page_address(page)+offset,buf,cpylen)) {
- lock_page(page);
- memcpy(page_address(page) + offset,buf, cpylen);
- set_page_dirty(page);
- unlock_page(page);
- }
- page_cache_release(page);
-
- if(retlen) *retlen += cpylen;
- //
- buf+=cpylen;
- offset = 0;
- index++;
- //
- }
- //printk(KERN_INFO "Write act: %d\n",*retlen);
- return 0;
- }
-
- err=0;
- down(&dev->write_mutex);
- err = my_write();
- up(&dev->write_mutex);
- return err;
-}
/* erase a specified part of the device */
@@ -290,6 +212,48 @@
/* write data to the underlying device */
+static int _blockmtd_write(struct blockmtd_dev *dev, const u_char *buf,
+ loff_t to, size_t len, size_t *retlen)
+{
+ struct page *page;
+ struct address_space *mapping = dev->blkdev->bd_inode->i_mapping;
+ int index = to >> PAGE_SHIFT; // page index
+ int offset = to & ~PAGE_MASK; // page offset
+ int cpylen;
+
+ if (retlen)
+ *retlen = 0;
+ while (len) {
+ if ((offset+len) > PAGE_SIZE)
+ cpylen = PAGE_SIZE - offset; // multiple pages
+ else
+ cpylen = len; // this page
+ len = len - cpylen;
+
+ // Get page
+ page = page_readahead(mapping, index);
+ if (!page)
+ return -ENOMEM;
+ if (IS_ERR(page))
+ return PTR_ERR(page);
+
+ if (memcmp(page_address(page)+offset, buf, cpylen)) {
+ lock_page(page);
+ memcpy(page_address(page) + offset, buf, cpylen);
+ set_page_dirty(page);
+ unlock_page(page);
+ }
+ page_cache_release(page);
+
+ if (retlen)
+ *retlen += cpylen;
+
+ buf += cpylen;
+ offset = 0;
+ index++;
+ }
+ return 0;
+}
static int blockmtd_write(struct mtd_info *mtd, loff_t to, size_t len,
size_t *retlen, const u_char *buf)
{
@@ -303,7 +267,9 @@
if (to + len > mtd->size)
len = mtd->size - to;
- err = write_pages(dev, buf, to, len, retlen);
+ down(&dev->write_mutex);
+ err = _blockmtd_write(dev, buf, to, len, retlen);
+ up(&dev->write_mutex);
if (err > 0)
err = 0;
return err;
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 21/22] Default erase size
2004-12-21 14:10 ` [PATCH 20/22] Fold various write functions Jörn Engel
@ 2004-12-21 14:11 ` Jörn Engel
2004-12-21 14:13 ` [PATCH 22/22] Readahead Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 14:11 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
For block devices, an erase size of 0 makes little sense. PAGE_SIZE
should be a sane default.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 4 +---
1 files changed, 1 insertion(+), 3 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_erasesize 2004-12-21 02:19:58.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-21 02:23:25.000000000 +0100
@@ -38,7 +38,6 @@
/* Default erase size in K, always make it a multiple of PAGE_SIZE */
-#define CONFIG_MTD_BLKDEV_ERASESIZE (128 << 10) /* 128KiB */
#define VERSION "$Revision: 1.23 $"
/* Info for the block device */
@@ -290,7 +289,6 @@
if (!dev)
return;
- kfree(dev->mtd.eraseregions);
kfree(dev->mtd.name);
if (dev->blkdev) {
@@ -432,7 +430,7 @@
char buf[80+12], *str=buf; /* 80 for device, 12 for erase size */
char *token[2];
char *name;
- size_t erase_size = 0;
+ size_t erase_size = PAGE_SIZE;
int i, ret;
if (strnlen(val, sizeof(buf)) >= sizeof(buf))
^ permalink raw reply [flat|nested] 113+ messages in thread
* [PATCH 22/22] Readahead
2004-12-21 14:11 ` [PATCH 21/22] Default erase size Jörn Engel
@ 2004-12-21 14:13 ` Jörn Engel
0 siblings, 0 replies; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 14:13 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
Small changes to readahead functions. I guess they can use some more,
but it was getting late and things looked too complicated.
Signed-off-by: Jörn Engel <joern@wohnheim.fh-wedel.de>
---
blockmtd.c | 11 ++++-------
1 files changed, 4 insertions(+), 7 deletions(-)
--- linux-2.6.9cow/drivers/mtd/devices/blockmtd.c~blockmtd_readahead 2004-12-21 02:24:35.000000000 +0100
+++ linux-2.6.9cow/drivers/mtd/devices/blockmtd.c 2004-12-21 02:30:03.000000000 +0100
@@ -54,10 +54,9 @@
#define PAGE_READAHEAD 64
-
void cache_readahead(struct address_space *mapping, int index)
{
- filler_t *filler = (void*)mapping->a_ops->readpage;
+ filler_t *filler = (filler_t*)mapping->a_ops->readpage;
int i, pagei;
unsigned ret = 0;
unsigned long end_index;
@@ -95,17 +94,15 @@
ret++;
}
spin_unlock_irq(&mapping->tree_lock);
- if (ret) {
+ if (ret)
read_cache_pages(mapping, &page_pool, filler, NULL);
- //printk(KERN_INFO "Readahead [%d/%d] pages from [%d]\n",ret,PAGE_READAHEAD,index);
- }
}
static struct page* page_readahead(struct address_space *mapping, int index)
{
- filler_t *filler = (filler_t*)(mapping->a_ops->readpage);
- cache_readahead(mapping,index);
+ filler_t *filler = (filler_t*)mapping->a_ops->readpage;
+ cache_readahead(mapping, index);
return read_cache_page(mapping, index, filler, NULL);
}
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH 1/22] Add drivers/mtd/devices/blockmtd.c
2004-12-21 13:42 ` [PATCH 1/22] Add drivers/mtd/devices/blockmtd.c Gareth Bult (Encryptec)
@ 2004-12-21 14:15 ` Jörn Engel
0 siblings, 0 replies; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 14:15 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
On Tue, 21 December 2004 13:42:41 +0000, Gareth Bult (Encryptec) wrote:
>
> Gimmie a clue - what did you change ?
Nothing yet. But you should have the other 21 patches by now. In
case you never dealt with inline patches, you can simply save the mail
(with all headers etc.) as a file and patch your preferred kernel with
that file. Patch is smart enough to ignore private chitchat.
Jörn
--
All art is but imitation of nature.
-- Lucius Annaeus Seneca
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-12-21 13:40 ` JFFS2 mount time Gareth Bult (Encryptec)
@ 2004-12-21 15:00 ` David Woodhouse
[not found] ` <1103644945.10792.175.camel@squizzey.bult.co.uk>
0 siblings, 1 reply; 113+ messages in thread
From: David Woodhouse @ 2004-12-21 15:00 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD
On Tue, 2004-12-21 at 13:40 +0000, Gareth Bult (Encryptec) wrote:
> .. and I find the patches where ?
On the list after I approved all 22 of them; they were trapped for
moderation because they had attachments. After looking at one or two of
yours which were also trapped, I approved them all because I thought
they were all due to the GPG signature. Unfortunately I let through an
HTML mail. Please don't send those -- and please don't top-post either.
--
dwmw2
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
[not found] ` <1103644945.10792.175.camel@squizzey.bult.co.uk>
@ 2004-12-21 16:04 ` Jörn Engel
0 siblings, 0 replies; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 16:04 UTC (permalink / raw)
To: Gareth Bult (Encryptec); +Cc: Linux MTD, David Woodhouse
On Tue, 21 December 2004 16:02:25 +0000, Gareth Bult (Encryptec) wrote:
>
> Ahhhhhh!
I get that a lot. ;)
Jörn
--
Fantasy is more important than knowledge. Knowledge is limited,
while fantasy embraces the whole world.
-- Albert Einstein
^ permalink raw reply [flat|nested] 113+ messages in thread
* RE: [PATCH 13/22] remove erase regions
2004-12-21 13:57 ` [PATCH 13/22] remove erase regions Jörn Engel
2004-12-21 14:01 ` [PATCH 14/22] Change add_device Jörn Engel
@ 2004-12-21 18:42 ` Christopher Hoover
2004-12-21 18:49 ` Jörn Engel
1 sibling, 1 reply; 113+ messages in thread
From: Christopher Hoover @ 2004-12-21 18:42 UTC (permalink / raw)
To: 'Jörn Engel'; +Cc: 'Linux MTD'
>From Jörn Engel -
> I see absolutely no reason for complicated erase reagions. On the
> user side, everyone but mtdchar effectively ignores it anyway.
I don't grok this. What about flash with variable-sized blocks? (I have a
board with such flash and code that uses eraseregions.)
-ch
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH 13/22] remove erase regions
2004-12-21 18:42 ` [PATCH 13/22] remove erase regions Christopher Hoover
@ 2004-12-21 18:49 ` Jörn Engel
2004-12-21 21:09 ` Christopher Hoover
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-21 18:49 UTC (permalink / raw)
To: Christopher Hoover; +Cc: 'Linux MTD'
On Tue, 21 December 2004 10:42:07 -0800, Christopher Hoover wrote:
> >From Jörn Engel -
> > I see absolutely no reason for complicated erase reagions. On the
> > user side, everyone but mtdchar effectively ignores it anyway.
>
> I don't grok this. What about flash with variable-sized blocks? (I have a
> board with such flash and code that uses eraseregions.)
Sure, from AMD or some other compatible manufacturer. The
variable-sized blocks were nice until there were better solutions to
the problem, like jffs2. Jffs2 exists, so they are largely useless.
Last time I looked, there were five users of mtd devices:
1. jffs
2. jffs2
3. yaffs
4. mtdblock
5. mtdchar
1 and 2 don't use the smaller blocks. 3 doesn't work with NOR-flash
anyway. 4 doesn't use the smaller blocks. 5 does, but is horribly
ugly and noone cares enough to clean it up. You do the math.
1, 2 and 4 effectively use the smaller blocks as if they were a single
larger block. So the space is not wasted, but that's about it.
Jörn
--
With a PC, I always felt limited by the software available. On Unix,
I am limited only by my knowledge.
-- Peter J. Schoenster
^ permalink raw reply [flat|nested] 113+ messages in thread
* RE: [PATCH 13/22] remove erase regions
2004-12-21 18:49 ` Jörn Engel
@ 2004-12-21 21:09 ` Christopher Hoover
2004-12-22 2:47 ` Eric W. Biederman
0 siblings, 1 reply; 113+ messages in thread
From: Christopher Hoover @ 2004-12-21 21:09 UTC (permalink / raw)
To: 'Jörn Engel'; +Cc: 'Linux MTD'
>From Jörn Engel [mailto:joern@wohnheim.fh-wedel.de] -
> On Tue, 21 December 2004 10:42:07 -0800, Christopher Hoover wrote:
> > >From Jörn Engel -
> > > I see absolutely no reason for complicated erase reagions. On the
> > > user side, everyone but mtdchar effectively ignores it anyway.
> >
> > I don't grok this. What about flash with variable-sized
> blocks? (I have a
> > board with such flash and code that uses eraseregions.)
>
> Sure, from AMD or some other compatible manufacturer. The
> variable-sized blocks were nice until there were better solutions to
> the problem, like jffs2. Jffs2 exists, so they are largely useless.
Intel C3 flash, too.
> 5. mtdchar
This is an important case.
The reason embedded systems use flash with variable sized blocks is for (in
the small blocks) parameter stores for bootloaders and applications.
I've got several deployed systems that use this technique. I've seen at
least one other.
> 5 does, but is horribly ugly and noone cares enough to clean it up.
This is not a reason to toss it. We don't capriciously break user space
interfaces in Linux.
Also this:
6. The hook that unlocks locked-on-power-up flash, such as (*surprise*) C3
flash. It needs to call unlock with the start address of each block. It
needs eraseergions to do that.
-ch
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH 13/22] remove erase regions
2004-12-21 21:09 ` Christopher Hoover
@ 2004-12-22 2:47 ` Eric W. Biederman
2004-12-22 8:59 ` Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Eric W. Biederman @ 2004-12-22 2:47 UTC (permalink / raw)
To: Christopher Hoover; +Cc: 'Linux, MTD'
"Christopher Hoover" <ch@murgatroid.com> writes:
> >From Jörn Engel [mailto:joern@wohnheim.fh-wedel.de] -
> > On Tue, 21 December 2004 10:42:07 -0800, Christopher Hoover wrote:
> > > >From Jörn Engel -
> > > > I see absolutely no reason for complicated erase reagions. On the
> > > > user side, everyone but mtdchar effectively ignores it anyway.
> > >
> > > I don't grok this. What about flash with variable-sized
> > blocks? (I have a
> > > board with such flash and code that uses eraseregions.)
> >
> > Sure, from AMD or some other compatible manufacturer. The
> > variable-sized blocks were nice until there were better solutions to
> > the problem, like jffs2. Jffs2 exists, so they are largely useless.
jffs2 is only a solution on large NOR flash parts.
However I find this conversation confusing. The patches appear to affect
just mtdblock.c. Which sounds like it is exclusively the mtd block device.
At which point I don't see a problem with simply removing variable erase
size for the silly block device emulation code.
Now if someone wants to remove something silly the block device emulation
sounds like a fine place to start. Just to place the silliness on the
other foot.
> Intel C3 flash, too.
>
> > 5. mtdchar
>
> This is an important case.
>
> The reason embedded systems use flash with variable sized blocks is for (in
> the small blocks) parameter stores for bootloaders and applications.
>
> I've got several deployed systems that use this technique. I've seen at
> least one other.
> > 5 does, but is horribly ugly and noone cares enough to clean it up.
If it ain't broke don't fix it. Besides I have trouble seeing how 500 lines
of code can be horribly ugly.
> This is not a reason to toss it. We don't capriciously break user space
> interfaces in Linux.
>
> Also this:
>
> 6. The hook that unlocks locked-on-power-up flash, such as (*surprise*) C3
> flash. It needs to call unlock with the start address of each block. It
> needs eraseergions to do that.
If Christopher is reading this right I agree that killing variable
erase sizes across the board is a very bad idea.
Eric
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH 13/22] remove erase regions
2004-12-22 2:47 ` Eric W. Biederman
@ 2004-12-22 8:59 ` Jörn Engel
2004-12-22 10:05 ` Eric W. Biederman
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-12-22 8:59 UTC (permalink / raw)
To: Eric W. Biederman; +Cc: Christopher Hoover, 'Linux MTD'
On Tue, 21 December 2004 19:47:00 -0700, Eric W. Biederman wrote:
> "Christopher Hoover" <ch@murgatroid.com> writes:
> > >From Jörn Engel [mailto:joern@wohnheim.fh-wedel.de] -
> > > On Tue, 21 December 2004 10:42:07 -0800, Christopher Hoover wrote:
> > > > >From Jörn Engel -
> > > > > I see absolutely no reason for complicated erase reagions. On the
> > > > > user side, everyone but mtdchar effectively ignores it anyway.
> > > >
> > > > I don't grok this. What about flash with variable-sized
> > > blocks? (I have a
> > > > board with such flash and code that uses eraseregions.)
> > >
> > > Sure, from AMD or some other compatible manufacturer. The
> > > variable-sized blocks were nice until there were better solutions to
> > > the problem, like jffs2. Jffs2 exists, so they are largely useless.
>
> jffs2 is only a solution on large NOR flash parts.
True. I've been meaning to scale jffs2 down, but never found the
time.
> However I find this conversation confusing. The patches appear to affect
> just mtdblock.c.
The other way around, blockmtd.c. Which is a cleaned up version of
blkmtd.c. It turns a block device into an mtd device, not vice versa.
Name is confusing, so if you have a better one...
> Which sounds like it is exclusively the mtd block device.
> At which point I don't see a problem with simply removing variable erase
> size for the silly block device emulation code.
>
> Now if someone wants to remove something silly the block device emulation
> sounds like a fine place to start. Just to place the silliness on the
> other foot.
Both emulations have valid uses:
mtd -> block device:
Very simple block devices. It is much simpler to write an mtd device
than a block device. Right now, all known-to-me users of phram use it
as a block device.
block device -> mtd:
Cheap flash media. Jffs2 helps on usb sticks as well, so turn those
puppies into mtds and be done.
> > > 5 does, but is horribly ugly and noone cares enough to clean it up.
>
> If it ain't broke don't fix it. Besides I have trouble seeing how 500 lines
> of code can be horribly ugly.
o read-only devices that confuse every single new user
o #define MAX_KMALLOC_SIZE 0x20000
o FIXME: This _really_ needs to die. In 2.5, we should ...
> > This is not a reason to toss it. We don't capriciously break user space
> > interfaces in Linux.
I don't. Instead I take a driver I'm unhappy with, copy it, and
change the parts I don't like in the copy. Nothing breaks on your
side.
> > Also this:
> >
> > 6. The hook that unlocks locked-on-power-up flash, such as (*surprise*) C3
> > flash. It needs to call unlock with the start address of each block. It
> > needs eraseergions to do that.
>
> If Christopher is reading this right I agree that killing variable
> erase sizes across the board is a very bad idea.
He isn't.
To finally lift all the confusion:
This is about the blockmtd driver, nothing else. Said driver takes a
block device and turns it into an mtd. And for an bd-based mtd, there
is _no_ point in having complicated erase regions.
For regular flash, I consider them to be silly as well, but there are
so many silly things that people like. Let them have fun, as long as
noone gets hurt.
Jörn
--
The only real mistake is the one from which we learn nothing.
-- John Powell
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH 13/22] remove erase regions
2004-12-22 8:59 ` Jörn Engel
@ 2004-12-22 10:05 ` Eric W. Biederman
2004-12-22 10:41 ` Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Eric W. Biederman @ 2004-12-22 10:05 UTC (permalink / raw)
To: Jörn Engel; +Cc: Christopher Hoover, 'Linux MTD'
Jörn Engel <joern@wohnheim.fh-wedel.de> writes:
> On Tue, 21 December 2004 19:47:00 -0700, Eric W. Biederman wrote:
> > jffs2 is only a solution on large NOR flash parts.
>
> True. I've been meaning to scale jffs2 down, but never found the
> time.
I keep trying to scale my flash chips up but that is non-trivial
on commodity motherboards :)
Until jffs2 is an effective solution on the very small flash
parts there will be a use for variable size flash blocks.
The flash parts I typically work with are 256KiB to 1MiB.
Usually with most blocks at 64KiB, although some variants have 4KiB
blocks.
On a 4KiB flash I could probably afford to not use a spare flash block
or two. For the common case of 64KiB I don't see how I could leave
an entire flash block unused. Which is the very hard part of scaling
jffs2 down to the tiniest systems.
> The other way around, blockmtd.c. Which is a cleaned up version of
> blkmtd.c. It turns a block device into an mtd device, not vice versa.
> Name is confusing, so if you have a better one...
block2mtd ?
> Both emulations have valid uses:
Agreed.
> > > > 5 does, but is horribly ugly and noone cares enough to clean it up.
> >
> > If it ain't broke don't fix it. Besides I have trouble seeing how 500 lines
> > of code can be horribly ugly.
>
> o read-only devices that confuse every single new user
Agreed. The read-only devices are silly.
> o #define MAX_KMALLOC_SIZE 0x20000
> o FIXME: This _really_ needs to die. In 2.5, we should ...
I see more problems with where the code wanted to go, as indicated
by those comments than with where the code is now. Probably
allocating a single buffer page and calling down into the mtd device
layer multiple times. I have trouble seeing the benefit
of making it a zero copy interface.
But none of this counts as horribly ugly in my book. Just
warts that may benefit from being cleaned up.
Some of the current IB stacks count get a lot closer to horribly ugly.
Several hundred lines of global variables in one file I count horribly
ugly. At 500 lines I can see ugly code, but it is hard to write
something that is a disaster that is that small. The obfuscated C
contest shows you can do it but we certainly don't have that.
> >
> > If Christopher is reading this right I agree that killing variable
> > erase sizes across the board is a very bad idea.
>
> He isn't.
>
> To finally lift all the confusion:
> This is about the blockmtd driver, nothing else. Said driver takes a
> block device and turns it into an mtd. And for an bd-based mtd, there
> is _no_ point in having complicated erase regions.
>
> For regular flash, I consider them to be silly as well, but there are
> so many silly things that people like. Let them have fun, as long as
> noone gets hurt.
Agreed.
So have your fun with block2mtd and I won't worry about it.
Eric
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: [PATCH 13/22] remove erase regions
2004-12-22 10:05 ` Eric W. Biederman
@ 2004-12-22 10:41 ` Jörn Engel
0 siblings, 0 replies; 113+ messages in thread
From: Jörn Engel @ 2004-12-22 10:41 UTC (permalink / raw)
To: Eric W. Biederman; +Cc: Christopher Hoover, 'Linux MTD'
On Wed, 22 December 2004 03:05:06 -0700, Eric W. Biederman wrote:
>
> On a 4KiB flash I could probably afford to not use a spare flash block
> or two. For the common case of 64KiB I don't see how I could leave
> an entire flash block unused. Which is the very hard part of scaling
> jffs2 down to the tiniest systems.
Yup. One block (plus a few bytes) is the lower bound. Below that,
you can read-modify-erase-update and hope the power won't fail before
you're done. Not fun.
> block2mtd ?
Deal.
> I see more problems with where the code wanted to go, as indicated
> by those comments than with where the code is now. Probably
> allocating a single buffer page and calling down into the mtd device
> layer multiple times. I have trouble seeing the benefit
> of making it a zero copy interface.
>
> But none of this counts as horribly ugly in my book. Just
> warts that may benefit from being cleaned up.
>
> Some of the current IB stacks count get a lot closer to horribly ugly.
> Several hundred lines of global variables in one file I count horribly
> ugly. At 500 lines I can see ugly code, but it is hard to write
> something that is a disaster that is that small. The obfuscated C
> contest shows you can do it but we certainly don't have that.
Agreed. Just ugly then.
> So have your fun with block2mtd and I won't worry about it.
;)
Jörn
--
Those who come seeking peace without a treaty are plotting.
-- Sun Tzu
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2005-01-11 8:26 ` Ferenc Havasi
@ 2005-01-11 8:34 ` Steven Scholz
0 siblings, 0 replies; 113+ messages in thread
From: Steven Scholz @ 2005-01-11 8:34 UTC (permalink / raw)
To: Ferenc Havasi; +Cc: MTD List
Hi Ferenc,
> If I am right CRCs are not only against the effect of unclean reboots
> but also to handle flash errors. On NAND flashes the ECC handles this
> problem but NOR doesn't have any error detection system.
>
>> Ferenc, what do you think about that?
>> No need to check CRC if we have a summary, right?
>
> Give us some days to invetigate it more deeply. I think we will be able
> to imrove the summary patch in some way.
I now get
Eep. Child "libutil.so" (ino #230) of dir ino #5 doesn't exist!
Eep. Child "libthread_db.so" (ino #227) of dir ino #5 doesn't exist!
Eep. Child "libutil.so.1" (ino #231) of dir ino #5 doesn't exist!
Eep. Child "libthread_db-1.0.so" (ino #226) of dir ino #5 doesn't exist!
Eep. Child "libutil-2.3.3.so" (ino #229) of dir ino #5 doesn't exist!
Eep. Child "libthread_db.so.1" (ino #228) of dir ino #5 doesn't exist!
Eep. Child "adjtimex" (ino #237) of dir ino #9 doesn't exist!
Eep. Child "backlight" (ino #238) of dir ino #9 doesn't exist!
Eep. Child "log" (ino #299) of dir ino #13 doesn't exist!
Eep. Child "demo_320_240_16.xml" (ino #234) of dir ino #232 doesn't exist!
Eep. Child "demo.xml" (ino #233) of dir ino #232 doesn't exist!
Eep. Child "fonttest" (ino #235) of dir ino #232 doesn't exist!
VFS: Mounted root (jffs2 filesystem).
Freeing init memory: 72K
while booting! :-(
I am not sure if this has to do with the summary patch. How can I find out?
--
Steven
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2005-01-10 15:14 ` Steven Scholz
2005-01-10 15:25 ` Steven Scholz
@ 2005-01-11 8:26 ` Ferenc Havasi
2005-01-11 8:34 ` Steven Scholz
1 sibling, 1 reply; 113+ messages in thread
From: Ferenc Havasi @ 2005-01-11 8:26 UTC (permalink / raw)
To: Steven Scholz; +Cc: MTD List
Hi Steven,
> As I wrote in another thread the summary patch reduced the mount time
> from 6 seconds to less than 1 second on my embedded ARM9 system with 8MB
> NAND flash.
> But now GC still slows down the start up of user applications as it
> checks all CRCs right after booting.
>> If we change
>> the patch a bit (asking not to check any CRC in the block has summary)
>> we may
>> have better mount speed (and iget()). And possibly in case of nor we
>> even doesn't
>> need to store any data in summary - only header. Header will show us
>> that the block
>> was successfully written (no unclean reboots) and we my trust it and
>> do not check CRC.
If I am right CRCs are not only against the effect of unclean reboots
but also to handle flash errors. On NAND flashes the ECC handles this
problem but NOR doesn't have any error detection system.
> Ferenc, what do you think about that?
> No need to check CRC if we have a summary, right?
Give us some days to invetigate it more deeply. I think we will be able
to imrove the summary patch in some way.
Bye,
Ferenc
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2005-01-10 15:14 ` Steven Scholz
@ 2005-01-10 15:25 ` Steven Scholz
2005-01-11 8:26 ` Ferenc Havasi
1 sibling, 0 replies; 113+ messages in thread
From: Steven Scholz @ 2005-01-10 15:25 UTC (permalink / raw)
To: MTD List
Steven Scholz wrote:
> As I wrote in another thread the summary patch reduced the mount time
> from 6 seconds to less than 1 second on my embedded ARM9 system with 8MB
> NAND flash.
Of course I meant the good old NOR flash.
--
Steven
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2005-01-05 11:45 ` Artem B. Bityuckiy
2005-01-06 10:50 ` Ferenc Havasi
@ 2005-01-10 15:14 ` Steven Scholz
2005-01-10 15:25 ` Steven Scholz
2005-01-11 8:26 ` Ferenc Havasi
1 sibling, 2 replies; 113+ messages in thread
From: Steven Scholz @ 2005-01-10 15:14 UTC (permalink / raw)
To: MTD List
Hi,
>> Thanks a million. I'll try that. I guess I can just use your Patch on
>> a vanilla linux-2.6.10!?
>>
>> BTW: On the link mentioned above I read "Speed up the mount time of
>> JFFS2 - specially when it uses NAND flash. " How about NOR flash?
>>
> My understanding of the problem is that summary may help in case of NOR,
> but in
> general it shouldn't increase the mount speed a lot (am I wrong?).
As I wrote in another thread the summary patch reduced the mount time from 6
seconds to less than 1 second on my embedded ARM9 system with 8MB NAND flash.
But now GC still slows down the start up of user applications as it checks all
CRCs right after booting.
> If we change
> the patch a bit (asking not to check any CRC in the block has summary)
> we may
> have better mount speed (and iget()). And possibly in case of nor we
> even doesn't
> need to store any data in summary - only header. Header will show us
> that the block
> was successfully written (no unclean reboots) and we my trust it and do
> not check CRC.
Makes sense to me.
Ferenc, what do you think about that?
No need to check CRC if we have a summary, right?
This could fix my GC problem mentioned above and in the thread
http://lists.infradead.org/pipermail/linux-mtd/2005-January/011411.html
--
Steven
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2005-01-05 11:45 ` Artem B. Bityuckiy
@ 2005-01-06 10:50 ` Ferenc Havasi
2005-01-10 15:14 ` Steven Scholz
1 sibling, 0 replies; 113+ messages in thread
From: Ferenc Havasi @ 2005-01-06 10:50 UTC (permalink / raw)
To: Artem B. Bityuckiy; +Cc: MTD List, Steven Scholz
Hi Steven,
>> Thanks a million. I'll try that. I guess I can just use your Patch on
>> a vanilla linux-2.6.10!?
>>
>> BTW: On the link mentioned above I read "Speed up the mount time of
>> JFFS2 - specially when it uses NAND flash. " How about NOR flash?
>>
> My understanding of the problem is that summary may help in case of NOR,
> but in
> general it shouldn't increase the mount speed a lot (am I wrong?).
Artem is right.
> Moreover, looking through the patch I suspect it wasn't tested with NOR,
> is it?
It should work with NOR, too. Now we tested it again, and noticed some
problems. One of them are "general", and an other is NOR specific. We
will fix it soon, so you may wait a little before testing it.
> If we change
> the patch a bit (asking not to check any CRC in the block has summary)
> we may
> have better mount speed (and iget()). And possibly in case of nor we
> even doesn't
> need to store any data in summary - only header. Header will show us
> that the block
> was successfully written (no unclean reboots) and we my trust it and
> do not check CRC.
Yes, if everything will be stable we should start to make it more optimal.
Bye,
Ferenc
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2005-01-05 11:24 ` Steven Scholz
@ 2005-01-05 11:45 ` Artem B. Bityuckiy
2005-01-06 10:50 ` Ferenc Havasi
2005-01-10 15:14 ` Steven Scholz
0 siblings, 2 replies; 113+ messages in thread
From: Artem B. Bityuckiy @ 2005-01-05 11:45 UTC (permalink / raw)
To: Steven Scholz; +Cc: MTD List
> Thanks a million. I'll try that. I guess I can just use your Patch on a
> vanilla linux-2.6.10!?
>
> BTW: On the link mentioned above I read "Speed up the mount time of
> JFFS2 - specially when it uses NAND flash. " How about NOR flash?
>
My understanding of the problem is that summary may help in case of NOR,
but in
general it shouldn't increase the mount speed a lot (am I wrong?). If we
change
the patch a bit (asking not to check any CRC in the block has summary)
we may
have better mount speed (and iget()). And possibly in case of nor we
even doesn't
need to store any data in summary - only header. Header will show us
that the block
was successfully written (no unclean reboots) and we my trust it and do
not check CRC.
Moreover, looking through the patch I suspect it wasn't tested with NOR,
is it?
--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2005-01-05 11:10 ` Ferenc Havasi
@ 2005-01-05 11:24 ` Steven Scholz
2005-01-05 11:45 ` Artem B. Bityuckiy
0 siblings, 1 reply; 113+ messages in thread
From: Steven Scholz @ 2005-01-05 11:24 UTC (permalink / raw)
To: MTD List
Hi Ferenc,
>> Note: JFFS3 development is in progress so be ready if it has bugs.
>
> Becauseof it, we maintain a summary patch for JFFS2. You can download
> the latest patch from http://www.inf.u-szeged.hu/jffs2/mount.php
> (I'm updating the page now, so you may wait a little.)
>
> Summary designed specially to speed up the mounttime, so it should help
> you.
Thanks a million. I'll try that. I guess I can just use your Patch on a vanilla
linux-2.6.10!?
BTW: On the link mentioned above I read "Speed up the mount time of JFFS2 -
specially when it uses NAND flash. " How about NOR flash?
--
Steven
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2005-01-05 10:45 ` Artem B. Bityuckiy
@ 2005-01-05 11:10 ` Ferenc Havasi
2005-01-05 11:24 ` Steven Scholz
0 siblings, 1 reply; 113+ messages in thread
From: Ferenc Havasi @ 2005-01-05 11:10 UTC (permalink / raw)
To: Steven Scholz; +Cc: MTD List
Hi Steven,
> Try JFFS3 with "summaries" enabled or Ferenc Havasi's JFFS2 "summary"
> patch. The mount speed should be increased with it.
>
> Note: JFFS3 development is in progress so be ready if it has bugs.
Becauseof it, we maintain a summary patch for JFFS2. You can download
the latest patch from http://www.inf.u-szeged.hu/jffs2/mount.php
(I'm updating the page now, so you may wait a little.)
Summary designed specially to speed up the mounttime, so it should help you.
Regards,
Ferenc
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2005-01-04 15:00 ` Steven Scholz
@ 2005-01-05 10:45 ` Artem B. Bityuckiy
2005-01-05 11:10 ` Ferenc Havasi
0 siblings, 1 reply; 113+ messages in thread
From: Artem B. Bityuckiy @ 2005-01-05 10:45 UTC (permalink / raw)
To: Steven Scholz; +Cc: MTD List
Hi
Try JFFS3 with "summaries" enabled or Ferenc Havasi's JFFS2 "summary"
patch. The mount speed should be increased with it.
Note: JFFS3 development is in progress so be ready if it has bugs.
Steven Scholz wrote:
> Hi,
>
>> Is there anything that could be done to reduce the mount time of large
>> JFFS2 partitions? Lazy mounting with prioritized scanning?
>
>
> I wonder if the current CVS head is much better (i.e. faster) in
> mounting a JFFS2 root fs then linux-2.6.10?
>
> Or in other words: does it make sense to use the current CVS instead of
> the code in linux-2.6.10?
>
> --
> Steven
>
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
>
--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-01-13 12:50 Jarkko Lavinen (NMP/Helsinki)
2004-01-13 13:19 ` Joakim Tjernlund
2004-01-13 13:39 ` David Woodhouse
@ 2005-01-04 15:00 ` Steven Scholz
2005-01-05 10:45 ` Artem B. Bityuckiy
2 siblings, 1 reply; 113+ messages in thread
From: Steven Scholz @ 2005-01-04 15:00 UTC (permalink / raw)
To: MTD List
Hi,
> Is there anything that could be done to reduce the mount time of large
> JFFS2 partitions? Lazy mounting with prioritized scanning?
I wonder if the current CVS head is much better (i.e. faster) in mounting a
JFFS2 root fs then linux-2.6.10?
Or in other words: does it make sense to use the current CVS instead of the code
in linux-2.6.10?
--
Steven
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-26 11:05 ` Artem Bityuckiy
@ 2004-10-26 13:52 ` Ferenc Havasi
0 siblings, 0 replies; 113+ messages in thread
From: Ferenc Havasi @ 2004-10-26 13:52 UTC (permalink / raw)
To: dedekind; +Cc: linux-mtd, David Woodhouse, jffs-dev
Hi Artem,
> Also, please, take into account that there may be checkpoint nodes (I'm
> implementing this). So, I think you need to have a generic mechanism to
> add new node types to your summary.
>
> Also, I think it is good to have a generic mechanism to just refer some
> nodes from summaries (for example, direntries with long names or
> something else).
Yes, it will be easy to extend.
We also need a this general support - because we will introduce a new
node type, too, becauseof the model file support, which will start to
commit when David finishes his patch for Linus.
Bye,
Ferenc
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-26 10:21 ` Ferenc Havasi
@ 2004-10-26 11:05 ` Artem Bityuckiy
2004-10-26 13:52 ` Ferenc Havasi
0 siblings, 1 reply; 113+ messages in thread
From: Artem Bityuckiy @ 2004-10-26 11:05 UTC (permalink / raw)
To: Ferenc Havasi; +Cc: linux-mtd, David Woodhouse, jffs-dev
Ferenc,
> Is it sure than only one non-full erase block is in the filesystem?
> Non-full means here that there is some nodes already in that, but also
> there is some free space at the end of it.
I didn't analyse this accurately, but my vision is that there is one
current block (c->nextblock). Even GC moves nodes to it. This is because
the jffs2_do_reserve_space() is always used (even by GC), and the
jffs2_do_reserve_space() always uses c->nextblock.
> As I see jffs2_do_reserve space is called before inode/... allocation in
> most cases. So at that time the summary information is not know - but at
> writing it have to be known certainly.
May be... From another hand you may write summary every time the
jffs2_reserve_space() fetches new block from the free_list...
Anyway, this is not fundamental...
> You may misunderstood me. In the previous letter I wrote: "So you
> convinced me. We will change the design of summary. The inodes and
> dirents will be also in the summary."
>
> So now we do plan to store dirents in the summary. :)
OK, sorry. :-)
> Let see how it work, and after we can make it more optimal :)
Agree :-)
Also, please, take into account that there may be checkpoint nodes (I'm
implementing this). So, I think you need to have a generic mechanism to
add new node types to your summary.
Also, I think it is good to have a generic mechanism to just refer some
nodes from summaries (for example, direntries with long names or
something else).
Thank you for conversation too.
:-)
--
Best regards, Artem B. Bityuckiy
Oktet Labs (St. Petersburg), Software Engineer.
+78124286709 (office) +79112449030 (mobile)
E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-26 9:29 ` Jarkko Lavinen
2004-10-26 10:24 ` Ferenc Havasi
@ 2004-10-26 10:34 ` Artem Bityuckiy
1 sibling, 0 replies; 113+ messages in thread
From: Artem Bityuckiy @ 2004-10-26 10:34 UTC (permalink / raw)
To: Jarkko Lavinen; +Cc: David Woodhouse, linux-mtd, jffs-dev
Hello Jarkko,
> I tried Ferenc's earlier mount time patch in August and the 52s mount
> time dropped then to 14s. If I understand right, inodes and dentries
> were then mixed in the erase block and the summary was for inodes
> only. This shows reading dentries from semirandom places is
> expensive.
This is very good that direntries are distributed more or less uniformly
in average.
>
> Ferenc's latest patch put dentries on their own erase block in
> consecutive order. Considering only the read efficiency from the
> media, reading consecutive, uncompressed, and unstripped dentries from
> a summary should cost no more than reading them from dedicated erase
> block.
>
Definitely true - the second patch must be better than the first one. But
unfortunately, it hard to do this dinamically :-( Ferenc tried...
But in my proposition, we will also refer direntries in the summary -
this is not the same as to read direntries from where they are placed,
this is another thing, especially in case of NAND! There is difference
(if we have NAND) - whether to read one 512 NAND page containing
compressed information about 20-25 direntries or to read 20-25
*different* NAND pages.
So, I think, new design will also better than the early Ferenc's patch :-)
--
Best regards, Artem B. Bityuckiy
Oktet Labs (St. Petersburg), Software Engineer.
+78124286709 (office) +79112449030 (mobile)
E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-26 9:29 ` Jarkko Lavinen
@ 2004-10-26 10:24 ` Ferenc Havasi
2004-10-26 10:34 ` Artem Bityuckiy
1 sibling, 0 replies; 113+ messages in thread
From: Ferenc Havasi @ 2004-10-26 10:24 UTC (permalink / raw)
To: Jarkko Lavinen; +Cc: ext Artem Bityuckiy, linux-mtd, David Woodhouse, jffs-dev
Hi Jarkko,
> If dentries were stored just as they are (unstripped and uncompressed)
> in the summary, the summary size would grow by 50% to about 3% of the
> whole image size.
Thanks, good to know it.
Did you got ECC/CRC errors? The most interest test for me whould be to
test the new (sumtool) image with the original kernel (because the
summary nodes are compatibles it should work), and see if there is
ECC/CRC errors or not.
Bye,
Ferenc
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-26 9:59 ` Artem Bityuckiy
@ 2004-10-26 10:21 ` Ferenc Havasi
2004-10-26 11:05 ` Artem Bityuckiy
0 siblings, 1 reply; 113+ messages in thread
From: Ferenc Havasi @ 2004-10-26 10:21 UTC (permalink / raw)
To: dedekind; +Cc: David Woodhouse, linux-mtd, jffs-dev
Hi Artem,
> IMHO, since the summary relates only to one block, the current block, it
> is logical to refer the summary from the jffs2_sb_info, not from
> jffs2_erase_blocks. It is also not very nice to store it in the
> jffs2_erase_blocks since it will increase the size of array of JFFS2
> blocks (c->blocks[]).
Is it sure than only one non-full erase block is in the filesystem?
Non-full means here that there is some nodes already in that, but also
there is some free space at the end of it.
>> 4. jffs2_flash_writev(), which is used to write info to flash. It can
>> parse the node (similar to sumtool) and store the summary of it in its
>> jeb.
>
> May be write here... Didn't think a lot... May be as I wrote, in
> jffs2_do_reserve_space()...
As I see jffs2_do_reserve space is called before inode/... allocation in
most cases. So at that time the summary information is not know - but at
writing it have to be known certainly.
> So, if there are few direntries in block, why not to store them in summary?
You may misunderstood me. In the previous letter I wrote: "So you
convinced me. We will change the design of summary. The inodes and
dirents will be also in the summary."
So now we do plan to store dirents in the summary. :)
> Did you measured the time of summary uncompress on your system? I can't
> know for sure, but I suspect that if you have, say, 200MHz system, the
> time of uncompression = o(time of block read)!
It depends on the compressor.
We will test it with zlib/rtime. I whould like to implement as an
optional feature.
> There is one more issue: if there are too many direntries in block,
> summary may become too large (the compression helps here). In this case
> you may not write summary or don't mention direntries in summary.
Let see how it work, and after we can make it more optimal :)
Bye,
Ferenc
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-25 15:30 ` Ferenc Havasi
@ 2004-10-26 9:59 ` Artem Bityuckiy
2004-10-26 10:21 ` Ferenc Havasi
0 siblings, 1 reply; 113+ messages in thread
From: Artem Bityuckiy @ 2004-10-26 9:59 UTC (permalink / raw)
To: Ferenc Havasi; +Cc: David Woodhouse, linux-mtd, jffs-dev
Hello Ferenc,
Ferenc Havasi wrote:
> In the kernel we will have to modify
> 1. jffs2_scan_eraseblock(), as it is already in our patch
> 2. jeb struct to store generated the summary dinamically (one plus field)
IMHO, since the summary relates only to one block, the current block, it
is logical to refer the summary from the jffs2_sb_info, not from
jffs2_erase_blocks. It is also not very nice to store it in the
jffs2_erase_blocks since it will increase the size of array of JFFS2
blocks (c->blocks[]).
> 3. jffs2_reserve_space(), which will have a new parameter (summary
> size), which can be JFFS2_SUMMARY_INODE_SIZE or
> JFFS2_SUMMARY_DIRENT_SIZE(namelen). It can decide when to generate
> summary and it can do this generation.
Yes, I also think so.
Currently the jffs2_do_reserve_space() do (as I understand):
1. If the current block (c->nextblock) have space and it is sufficient
for request, it reserves it.
2. If the c->nextblock has fewer size, than requested, the c->nextblock
is wasted, put to the correspondent list (dirty_list, etc), free block
is taken and reserved.
Thus, the jffs2_do_reserve_space() should be improved to be able to save
some space for summary. And, some function like jffs2_write_summary()
which will be called before jffs2_do_reserve_space() takes new block
from the free_list.
> 4. jffs2_flash_writev(), which is used to write info to flash. It can
> parse the node (similar to sumtool) and store the summary of it in its jeb.
May be write here... Didn't think a lot... May be as I wrote, in
jffs2_do_reserve_space()...
I also offer you to include direntries in summaries and compress them. See:
sizeof(struct jffs2_raw_dirent) = 40 (without name)
you will need to store in your summary only:
totlen
pino
version
ino
nsize
type
name
which is 24 bytes. You don't store all data! Of course, in case of long
names things are not so good...
If you also compress them, they will be smaller (minus 50-70%)!
So, if there are few direntries in block, why not to store them in summary?
Did you measured the time of summary uncompress on your system? I can't
know for sure, but I suspect that if you have, say, 200MHz system, the
time of uncompression = o(time of block read)!
There is one more issue: if there are too many direntries in block,
summary may become too large (the compression helps here). In this case
you may not write summary or don't mention direntries in summary.
--
Best regards, Artem B. Bityuckiy
Oktet Labs (St. Petersburg), Software Engineer.
+78124286709 (office) +79112449030 (mobile)
E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-22 12:44 ` Artem Bityuckiy
2004-10-25 9:36 ` Ferenc Havasi
@ 2004-10-26 9:29 ` Jarkko Lavinen
2004-10-26 10:24 ` Ferenc Havasi
2004-10-26 10:34 ` Artem Bityuckiy
1 sibling, 2 replies; 113+ messages in thread
From: Jarkko Lavinen @ 2004-10-26 9:29 UTC (permalink / raw)
To: linux-mtd, jffs-dev; +Cc: ext Artem Bityuckiy, David Woodhouse
[-- Attachment #1: Type: text/plain, Size: 2655 bytes --]
I tried to see with jffs2dump how much Inodes and Dirents I have on
root filesystem on Arm testbed. Quick and dirty Perl script attached.
This isn't accurate as the calculated total image size misses at least
the final padding on the last erase block.
The size of the plain JFFS2 image is 31.1 MiB. The root fs consists of
all applications and libraries and no user data.
$ jffs2dump -c rootfs.jffs2 | perl jffs2stats.pl
Number of dirents: 6144.
Total dirent node space: 304911 (0.9%)
Average dirent len: 49.6
Total dirent name space: 76671
Average name len: 12.5
Number of Inodes: 21197
Total Inode space: 32254866 (99.1%)
Average Inode size: 1521.7
Padding: 37326 0.1%
Total image size: 32559777
$ ls -l rootfs.jffs2
-rw-r--r-- 1 root root 32597104 Oct 20 15:11 rootfs.jffs2
With sumtool the image size grows to 31.8 MiB
$ jffs2dump -c rootfs-sum.jffs2 | perl jffs2stats.pl
Number of dirents: 6144.
Total dirent node space: 304911 (0.9%)
Average dirent len: 49.6
Total dirent name space: 76671
Average name len: 12.5
Number of Inodes: 21197
Total Inode space: 32254866 (97.2%)
Average Inode size: 1521.7
Number of Inode Summary nodes: 251
Total Inode Sum space: 631524, (1.9%)
Average Sum node size: 2516.0
Padding: 153063 0.5%
Total image size: 33191301
$ ls -l rootfs-sum.jffs2
-rw-r--r-- 1 root root 33423360 Oct 20 15:23 rootfs-sum.jffs2
If dentries were stored just as they are (unstripped and uncompressed)
in the summary, the summary size would grow by 50% to about 3% of the
whole image size.
On Fri, Oct 22, 2004 at 04:44:13PM +0400, ext Artem Bityuckiy wrote:
> I believe that if we have directory references in summaries, this will
> increase the mount speed.
>
> 1. At first, we will store fewer data! We don't need to keep the common
> headers, CRCs and mctimes.
> 2. At the second, we may compress summary (direntries aren't compressed)!
> 3. And the third, on NAND there is difference between reading lots of
> different pages or few pages.
I tried Ferenc's earlier mount time patch in August and the 52s mount
time dropped then to 14s. If I understand right, inodes and dentries
were then mixed in the erase block and the summary was for inodes
only. This shows reading dentries from semirandom places is
expensive.
Ferenc's latest patch put dentries on their own erase block in
consecutive order. Considering only the read efficiency from the
media, reading consecutive, uncompressed, and unstripped dentries from
a summary should cost no more than reading them from dedicated erase
block.
Jarkko Lavinen
[-- Attachment #2: jffs2stats.pl --]
[-- Type: text/x-perl, Size: 3230 bytes --]
#! /usr/bin/perl
$EBLOCKSIZE=131072;
$dirents = $totdirentlen = $totnamelen = 0;
$inodes = $totinodelen = 0;
$sumnodes = $totsumnodelen = 0;
$totpadlen = 0;
$gaps = $totgaplen = 0;
$nextaddr = 0;
sub checkpadding {
my ($addr, $totlen) = @_;
my $len = hex($addr) - $nextaddr;
if ($len > 0) {
if (hex($addr) % $EBLOCKSIZE == 0 || $len <= 3) {
$totalpadlen += $len;
} else {
print sprintf "Gap seen at %08x .. $addr, length $len\n", $nextaddr;
$gaps++;
$totgaplen += hex($addr) - $nextaddr;
}
}
$nextaddr = hex($addr) + hex($totlen);
}
while(<>) {
chop;
if (/^\s+Dirent/) {
die "Cannot parse $_" if (! /^ \s+
Dirent \s+
node \s at \s+ (\w+), \s+
totlen \s+ (\w+), \s+
\#pino \s+ (\w+), \s+
version \s+ (\w+), \s+
\#ino \s+ (\w+), \s+
nsize \s+ (\w+), \s+
name \s+ (.*)
$/x);
my ($addr, $totlen, $pino, $version, $ino, $nsize, $name) = ($1, $2, $3, $4, $5, $6, $7);
&checkpadding($addr, $totlen);
$dirents++;
$totdirentlen += hex($totlen);
$totnamelen += hex($nsize);
} elsif (/^\s+Inode Sum/) {
die "Cannot parse $_" if (! /^ \s+
Inode \s Sum \s+
node \s at \s+ (\w+), \s+
totlen \s+ (\w+), \s+
sum_num \s+ (\w+), \s+
cleanmarker \s size \s+ (\w+) \s*
$/x);
my ($addr, $totlen, $sum_num, $cleanmarksize) = ($1, $2, $3, $4);
&checkpadding($addr, $totlen);
$sumnodes++;
$totsumnodelen += hex($totlen);
} elsif (/^\s+Inode/) {
die "Cannot parse $_" if (! /^ \s+
Inode \s+
node \s at \s+ (\w+), \s+
totlen \s+ (\w+), \s+
\#ino \s+ (\w+), \s+
version \s+ (\w+), \s+
isize \s+ (\w+), \s+
csize \s+ (\w+), \s+
dsize \s+ (\w+), \s+
offset \s+ (\w+) \s*
$/x);
my ($addr, $totlen, $ino, $version, $isize, $csize, $dsize, $offset) = ($1, $2, $3, $4, $5, $6, $7, $8);
&checkpadding($addr, $totlen);
$inodes++;
$totinodelen += hex($totlen);
} else {
die "Cannot parse $_";
}
}
$totalsize = $totdirentlen + $totinodelen + $totsumnodelen + $totpadlen + $totgaplen;
print "Number of dirents:\t$dirents.\n";
print sprintf " Total dirent node space:\t$totdirentlen (%.1f%%)\n", 100.0*$totdirentlen/$totalsize;
print " Average dirent len:\t", sprintf("%.1f", $totdirentlen/$dirents), "\n" if ($dirents > 0);
print " Total dirent name space:\t$totnamelen\n";
print sprintf(" Average name len:\t%.1f\n", $totnamelen/$dirents) if ($dirents > 0);
print "\n";
print "Number of Inodes:\t$inodes\n";
print sprintf " Total Inode space:\t$totinodelen (%.1f%%)\n", 100.0*$totinodelen/$totalsize;
print " Average Inode size:\t", sprintf("%.1f", $totinodelen/$inodes), "\n" if ($inodes > 0);
if ($sumnodes) {
print "\n";
print "Number of Inode Summary nodes:\t$sumnodes\n";
print sprintf " Total Inode Sum space:\t$totsumnodelen, (%.1f%%)\n", 100.0*$totsumnodelen/$totalsize;
print " Average Sum node size:\t", sprintf("%.1f", $totsumnodelen/$sumnodes), "\n";
}
print sprintf "\nPadding:\t$totalpadlen %.1f%%\n", 100.0*$totalpadlen/$totalsize;
print "Total image size: $totalsize\n";
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-25 10:56 ` Artem Bityuckiy
@ 2004-10-25 15:30 ` Ferenc Havasi
2004-10-26 9:59 ` Artem Bityuckiy
0 siblings, 1 reply; 113+ messages in thread
From: Ferenc Havasi @ 2004-10-25 15:30 UTC (permalink / raw)
To: dedekind; +Cc: linux-mtd, David Woodhouse, jffs-dev
Hi Artem,
> > To achieve as much mount time speed up as possible I think we should
> > distinguish them.
> This is what I really do not like.
>
> Ok, let us discuss now only this topic. Lt I explain why I believe it is
> vad and very *unnatural* to introduce two or more kinds of blocks.
You are right, it can be unnatural in point of the original design of
the JFFS2. But I think in point of the connection of this optimization
and JFFS2 it is more natural than simple store offsets in the summary,
or copy all the information into it.
Our plan was modify wbuf (make a second one) and modify
jffs2_reserve_space to select the right wbuf and generate summary. Never
planded to introduce new clean_*, dirty_*, ... lists, thats really too
difficult.
> 3. There is write buffer in the JFFS2 which is used in case of NAND. Are
> you going to have two wbufs? This is also significant change.
Yes, we started to implement it yesterday and now agree. It is really
not easy, and we don't write to rewrite the NAND handling part of JFFS2
whithout a real NAND device. Maybe at the design of JFFS3 :)
So you convinced me. We will change the design of summary. The inodes
and dirents will be also in the summary. All other nodes will be copied
as itself into the summary and cause a warning. The summary support will
be a required thing for new node types, too.
In the kernel we will have to modify
1. jffs2_scan_eraseblock(), as it is already in our patch
2. jeb struct to store generated the summary dinamically (one plus field)
3. jffs2_reserve_space(), which will have a new parameter (summary
size), which can be JFFS2_SUMMARY_INODE_SIZE or
JFFS2_SUMMARY_DIRENT_SIZE(namelen). It can decide when to generate
summary and it can do this generation.
4. jffs2_flash_writev(), which is used to write info to flash. It can
parse the node (similar to sumtool) and store the summary of it in its jeb.
If it works we'll check the effect of compressing the summary. (size and
speed)
Comments?
Bye,
Ferenc
P.s.: Thanks for this good conversation.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-25 9:36 ` Ferenc Havasi
2004-10-25 10:56 ` Artem Bityuckiy
@ 2004-10-25 11:21 ` Artem Bityuckiy
1 sibling, 0 replies; 113+ messages in thread
From: Artem Bityuckiy @ 2004-10-25 11:21 UTC (permalink / raw)
To: Ferenc Havasi; +Cc: linux-mtd, jffs-dev
> I curious about (at least) David's optinion about these topics.
I also wonder why people are not very active :-)
--
Best regards, Artem B. Bityuckiy
Oktet Labs (St. Petersburg), Software Engineer.
+78124286709 (office) +79112449030 (mobile)
E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-25 9:36 ` Ferenc Havasi
@ 2004-10-25 10:56 ` Artem Bityuckiy
2004-10-25 15:30 ` Ferenc Havasi
2004-10-25 11:21 ` Artem Bityuckiy
1 sibling, 1 reply; 113+ messages in thread
From: Artem Bityuckiy @ 2004-10-25 10:56 UTC (permalink / raw)
To: Ferenc Havasi; +Cc: linux-mtd, David Woodhouse, jffs-dev
Hello Ferenc,
> Yes, I think there are three kinds of nodes:
> - type A contains relevant amount of data which is not needed at mount
> time (jffs2_raw_inode)
> - type B is (almost) fully needed at mount time (jffs2_raw_dirent)
> - type C is any other (unkown, developements in the future...)
>
> To achieve as much mount time speed up as possible I think we should
> distinguish them.
This is what I really do not like.
Ok, let us discuss now only this topic. Lt I explain why I believe it is
vad and very *unnatural* to introduce two or more kinds of blocks.
The example of JFFS2 change that I consider natural is the introduction
of new node type. It is natural, because of when JFFS2 was designed,
this possibility was foreseen and taken into account. It is relatively
easy to do this. It is possible to do this and do not affect other
things in the JFFS2.
Conversely, the introducing several block types was not foreseen in the
JFFS2 design. And all things in the JFFS2 are coded with the assumption
all the blocks are equivalent.
This is my point view on the issue in general.
Now I will try to illustrate why I think so.
1. In JFFS2 there are several lists of blocks - clean_list, dirty_list,
very_dirty_list?. Are you going to introduce clean_list_typeA,
dirty_list_typeA, very_dirty_list_typeA, clean_list_typeB,
dirty_list_typeB, very_dirty_list_typeB ?
2. Just do 'grep "_list" * | grep -e "\(dirty\)\|\(very\)"' and see how
many places in JFFS2 where these lists are changed. Do you think it is
natural to introduce 3 more lists? I believe not. What if somebody else
will introduce one more type of block?
3. There is write buffer in the JFFS2 which is used in case of NAND. Are
you going to have two wbufs? This is also significant change.
4. Now the GC just gives one block, and moves all the valid nodes to
another one. In your case (if you have the JFFS2 image which was created
by older code, without your patch, where all node types are mixed),
you will need to move one type of nodes to one block, another to the
another block.
So, I think you will be needed to change many things in JFFS2. You have
a risk to hit on a can of worms.
So, do you agree that this change is *unnatural* ?
===================================================================
>> 4. It seems for me you will need to increase the number of blocks
>> which are reserved for the garbage collection (double ?). This is also
>> minor drawback.
> I don't understand what do you mean here
I mean the sb->resv_blocks_gcmerge and related. You will need to
increase it, which is not very good.
--
Best regards, Artem B. Bityuckiy
Oktet Labs (St. Petersburg), Software Engineer.
+78124286709 (office) +79112449030 (mobile)
E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-22 12:44 ` Artem Bityuckiy
@ 2004-10-25 9:36 ` Ferenc Havasi
2004-10-25 10:56 ` Artem Bityuckiy
2004-10-25 11:21 ` Artem Bityuckiy
2004-10-26 9:29 ` Jarkko Lavinen
1 sibling, 2 replies; 113+ messages in thread
From: Ferenc Havasi @ 2004-10-25 9:36 UTC (permalink / raw)
To: dedekind, David Woodhouse; +Cc: linux-mtd, jffs-dev
Hi Artem,
> So, for this purpose you are going to distribute the inode nodes and
> other (including direntry nodes) by different blocks. Those blocks, who
> contain only the inode nodes, will have summaries, other blocks - will not.
Yes, I think there are three kinds of nodes:
- type A contains relevant amount of data which is not needed at mount
time (jffs2_raw_inode)
- type B is (almost) fully needed at mount time (jffs2_raw_dirent)
- type C is any other (unkown, developements in the future...)
To achieve as much mount time speed up as possible I think we should
distinguish them.
Using summary the really relevant speed up will be only at node type
A. We can also generate summary for type B, but that (as you wrote)
relevant ratio of the information will be duplicated.
So we whould like to intorduce two kinds of erase blocks:
- erase blocks with summary: it will store (now only) type A nodes,
maybe later some of type B
- erase block without summary: it will store all of type C and B nodes
which is not stored before
> 1. Your change will affect JFFS2 very heavily. You will introduce
> restriction into JFFS2. Another improvements may not work with such
> restriction. Now all the blocks are equivalent. But you want to
> distinguish between two kins of blocks. Don't you think it is too
> complicated decision?
What kind of restriction do you mean? We don't introduce any
restrictions. The "type C" kind of nodes are processed as before, using
the usual scanning method. If you what to force for every node to make
their represenation in the summary, that whould be a restriction.
I think for some kinds of node summary is meaningful, and for some kinds
not.
If we mix them that can be a very big slow down, if you what to process
them only with making a reference in the summary to its offset, because
if you (for example) what to read only 50 bytes (size of the node) you
will have to read 512/2048 bytes depening on the flash. (where mostly
there will be inode nodes which is not neccesery to read because that is
int he summary)
But if all of this "not summarized, small" nodes are stored in a
"seperated" erase block than the this 512/2048 byte reading will not be
unnecessary (because on the remaining 462-1998 bytes will store also
relevant information, which is not in the summary).
> 2. Think about the wear-leveling. In JFFS it was ideal. In JFFS2 it is
> good, but not so ideal. I average, the inode nodes are changed more
> often (just think about FIFOs, we told about them in this list
> recently). So, you will need to Garbage Collect the NODE_ONLY blocks
> more often. So, I afraid the wear-leveling will suffer from your
> improvement.
I think the GC solves it "automaticly". This mark
(SUMMARIZED/NOT_SUMMARIZED) is not a premament thing, it is done "pseudo
randomly".
I aggree that it cause some different behavior in wear-leveling but I
don't think it makes it relevantly worse.
> 4. It seems for me you will need to increase the number of blocks which
> are reserved for the garbage collection (double ?). This is also minor
> drawback.
I don't understand what do you mean here.
> I believe that if we have directory references in summaries, this will
> increase the mount speed.
> 1. At first, we will store fewer data! We don't need to keep the common
> headers, CRCs and mctimes.
> 2. At the second, we may compress summary (direntries aren't compressed)!
> 3. And the third, on NAND there is difference between reading lots of
> different pages or few pages.
Yes, we should try it - to store dirents in SUMMARIZED erase blocks. But
it can be a improvement later, for first we need a well working stable
system - and this is urgent for us now.
> 2. Compress summaries.
It makes harder to determine the optimal time of summary generation (it
is easy to see the summary size, but here the compressed size of it the
relevant). It can cause smaller image but may cause some slow down, too.
We may introduce it later as an option.
So now we have two open discussion:
- is the SUMMARIZED / NOT_SUMMARIZED distiguishment good or not
- in the first version do we need dirents in the summary or not
Fortunatelly the effects (and side effects) of this improvements will be
active only if the new kernel option is enabled, and don't kill any
other future improvements.
I curious about (at least) David's optinion about these topics.
Bye,
Ferenc
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-21 20:05 ` Ferenc Havasi
@ 2004-10-22 12:44 ` Artem Bityuckiy
2004-10-25 9:36 ` Ferenc Havasi
2004-10-26 9:29 ` Jarkko Lavinen
0 siblings, 2 replies; 113+ messages in thread
From: Artem Bityuckiy @ 2004-10-22 12:44 UTC (permalink / raw)
To: Ferenc Havasi; +Cc: linux-mtd, David Woodhouse, jffs-dev
Hello Ferenc,
At first, please, let me describe your design shortly to be sure I
understand it and we both thinking the same way.
Essentially, your design is based on the fact that you do not want to
refer directory entries in the summary nodes. Motivation that you will
keep almost the copy of direntries in the summary, thus:
1. duplicating too many information.
2. you suppose there will not be the mount speed acceleration.
So, for this purpose you are going to distribute the inode nodes and
other (including direntry nodes) by different blocks. Those blocks, who
contain only the inode nodes, will have summaries, other blocks - will not.
I think this is not the best solution. Why? In general, because I do not
like the following:
A. Your idea to distribute inode nodes and other nodes between different
blocks.
B. Your assumption that the directory information in summaries will not
affect the mount time.
The following are reasons concerning the item A.
1. Your change will affect JFFS2 very heavily. You will introduce
restriction into JFFS2. Another improvements may not work with such
restriction. Now all the blocks are equivalent. But you want to
distinguish between two kins of blocks. Don't you think it is too
complicated decision?
2. Think about the wear-leveling. In JFFS it was ideal. In JFFS2 it is
good, but not so ideal. I average, the inode nodes are changed more
often (just think about FIFOs, we told about them in this list
recently). So, you will need to Garbage Collect the NODE_ONLY blocks
more often. So, I afraid the wear-leveling will suffer from your
improvement.
3. Imagine the file system with *lots* of very small files. I this case,
the direntries portion on the media will be large enough. And the
mount time of such file system will not be improved very well.
4. It seems for me you will need to increase the number of blocks which
are reserved for the garbage collection (double ?). This is also minor
drawback.
The following are reasons concerning the item B.
I believe that if we have directory references in summaries, this will
increase the mount speed.
1. At first, we will store fewer data! We don't need to keep the common
headers, CRCs and mctimes.
2. At the second, we may compress summary (direntries aren't compressed)!
3. And the third, on NAND there is difference between reading lots of
different pages or few pages.
I propose the another design.
1. Keep direntry references in summaries too and hence, do not
distinguish between blocks with inode nodes and direntries.
2. Compress summaries.
So, you will avoid a lot of problems related to teaching the GC
distinguish between different blocks. This will be more natural. I
believe, summaries must refer *any* node in block. This is more simple
and clean design.
Why you do not like this?
I see only one potential problem: direntries may have long names (up to
255 symbols). this may lead to large summaries.
But in this case we may do:
1. Improve the JFFS2 itself. Keep, say, only 20, characters in the
full_dirent structure. Most of direntries will fit. For other, we will
just read the flash.
2. We may not touch JFFS2, and keep only 20 characters in summaries. For
other direntries, we may read them from flash (keeping theirs flash
offsets instead of names).
Comments?
--
Best regards, Artem B. Bityuckiy
Oktet Labs (St. Petersburg), Software Engineer.
+78124286709 (office) +79112449030 (mobile)
E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-21 12:49 ` Jarkko Lavinen
2004-10-21 19:11 ` Ferenc Havasi
@ 2004-10-22 9:58 ` Ferenc Havasi
1 sibling, 0 replies; 113+ messages in thread
From: Ferenc Havasi @ 2004-10-22 9:58 UTC (permalink / raw)
To: Jarkko Lavinen; +Cc: dwmw2, linux-mtd
Jarkko Lavinen wrote:
> My initial test was only about the mount time. I have now also tried
> to exercise the patched file system and with very little testing I get
> CRC or ECC errors.
>
> # mount /dev/mtdblock2 /mnt -t jffs2
> # mkdir /mnt/testdir
> # umount /mnt
> jffs2_flush_wbuf(): Write failed with -5
> # mount /dev/mtdblock2 /mnt -t jffs2
> mtd->read(0x1fbec bytes from 0x1fc0414) returned ECC error
> Empty flash at 0x01fc2f1c ends at 0x01fc3000
> #
>
> With plain 2.6.9-rc4-omap1 with fresh CVS MTD code, I don't see anything
> weird occuring.
We are tring to find out what happens here...
Jarkko previously sent me some more detail, the logs starts with:
> sh-2.05b# /rootfstest.sh
> Mounting file system: Ok
> Creating a test directory: Ok
> Creating a test file: mtd->read(0x44 bytes from 0x1fa344c) returned ECC error
> Data CRC 6d0b1da8 != calculated CRC 9c4f3838 for node at 01fa344c
> mtd->read(0x44 bytes from 0x1fa3e20) returned ECC error
> Data CRC 5127cb7f != calculated CRC 057e127c for node at 01fa3e20
> mtd->read(0x44 bytes from 0x1fa4388) returned ECC error
> mtd->read(0x44 bytes from 0x1fa5cb8) returned ECC error
> mtd->read(0x44 bytes from 0x1fa6748) returned ECC error
> mtd->read(0x44 bytes from 0x1fa7b30) returned ECC error
> Data CRC 72b41a04 != calculated CRC ebc121db for node at 01fa7b30
> mtd->read(0x44 bytes from 0x1fa866c) returned ECC error
> Data CRC 9ff1d419 != calculated CRC cb2cce56 for node at 01fa866c
It means the mounting is done successfully. The first problem is when
the filesystem try to read jffs2_raw_inode nodes (if I am right the 0x44
is the size of that). It is not successfull (I don't know why), and it
cause CRC errors, too. The only one differences should be only the the
original version already read this 0x44 before (during mount time), the
summary version did not read yet, just know where it is from the summary.
Jarkko, one more interesing thing can be (if you have that image) to see
what is at the place 0x1fa344c, 01fa3e20, ... with the tool jffs2dump.
If anyone have any idea that is welcome. Unfortunatelly we don't have
real NAND device, and it works with our emulator.
Thanks,
Ferenc
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-21 13:24 ` David Woodhouse
@ 2004-10-21 20:05 ` Ferenc Havasi
2004-10-22 12:44 ` Artem Bityuckiy
0 siblings, 1 reply; 113+ messages in thread
From: Ferenc Havasi @ 2004-10-21 20:05 UTC (permalink / raw)
To: David Woodhouse; +Cc: linux-mtd, jffs-dev
David Woodhouse wrote:
> It's looking good, but the kernel really needs to be able to write these
> summaries for _itself_ in order to give a real improvement over the long
> term. If the file system has to be read-only we might as well be using
> cramfs, and if the summary becomes obsolete over time we might as well
> not bother in a lot of cases.
Our plan for it:
We would like to store some additional information in jeb struct:
- a type information, where there this type can be INODE_ONLY and
ANYTHING_OTHER. This information is easy to detect during mount time.
- a predicted summary size (calculated dinamically). It will be used to
decide when to generate the summary. Ceratinly only for INODE_ONLY
erase blocks.
If I am right every node allocation is done by jffs2_reserve_space(). We
would like to modify it, and introduce a new interface for it called
jffs2_reserve_space_for_inode() function. Every inode storing function
(there is no too much I think) should call
jffs2_reserve_space_for_inode() with some extra information (inode
number...).
jffs2_reserve_space() should use only ANYTHING_OTHER eraseblocks, as
jffs2_reserve_space_for_inode() use only INODE_ONLY ones. If there is no
free space in them it should use the usual technique to find a clean
eraseblock and start to store the new node in it.
The generating of summary is also the task of
jffs2_reserve_space_for_inode(), if the new inode (+summary) is not fit
in the erase block, it will generates summary.
What do you think?
Regards,
Ferenc
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-21 7:16 ` Artem B. Bityuckiy
@ 2004-10-21 19:50 ` Ferenc Havasi
0 siblings, 0 replies; 113+ messages in thread
From: Ferenc Havasi @ 2004-10-21 19:50 UTC (permalink / raw)
To: Artem B. Bityuckiy; +Cc: dwmw2, linux-mtd, jffs-dev
Hi Artem,
> Unfortunately, you can not mark entries as obsoleted in your summary
> node in case of NAND.
>
> If you write your summary only for *full* blocks, you will not need to
> mark entries obsoleted, even if you have NOR flash (but you can on NOR).
> The partially filled blocks must not have the summary node (you can
> introduce special marker and write it to OOB of the last page of
> NAND/last word of sector on NOR which tells if there is the summary node
> present).
Really, you are right.
So we only have to solve this problem on NOR. I think the easiest
solution is to set jffs2_can_mark_obsolete() to false if the summary
support is enabled.
Bye,
Ferenc
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-21 12:49 ` Jarkko Lavinen
@ 2004-10-21 19:11 ` Ferenc Havasi
2004-10-22 9:58 ` Ferenc Havasi
1 sibling, 0 replies; 113+ messages in thread
From: Ferenc Havasi @ 2004-10-21 19:11 UTC (permalink / raw)
To: Jarkko Lavinen; +Cc: dwmw2, Kluba Patrik, linux-mtd
Hi Jarkko,
> My initial test was only about the mount time. I have now also tried
> to exercise the patched file system and with very little testing I get
> CRC or ECC errors.
>
> # mount /dev/mtdblock2 /mnt -t jffs2
> # mkdir /mnt/testdir
> # umount /mnt
> jffs2_flush_wbuf(): Write failed with -5
> # mount /dev/mtdblock2 /mnt -t jffs2
> mtd->read(0x1fbec bytes from 0x1fc0414) returned ECC error
> Empty flash at 0x01fc2f1c ends at 0x01fc3000
> #
>
> With plain 2.6.9-rc4-omap1 with fresh CVS MTD code, I don't see anything
> weird occuring.
Can you send me the full kernel log file?
Another interesting test would be to test the new image (sumtool) with
the CVS MTD code. Because the summary is RWCOMPAT_DELETE node it should
works well. It would be nice to now if there is CRC/ECC errors in this case.
Thanks,
Ferenc
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-20 14:26 Ferenc Havasi
` (2 preceding siblings ...)
2004-10-21 12:49 ` Jarkko Lavinen
@ 2004-10-21 13:24 ` David Woodhouse
2004-10-21 20:05 ` Ferenc Havasi
3 siblings, 1 reply; 113+ messages in thread
From: David Woodhouse @ 2004-10-21 13:24 UTC (permalink / raw)
To: Ferenc Havasi; +Cc: linux-mtd, jffs-dev
On Wed, 2004-10-20 at 16:26 +0200, Ferenc Havasi wrote:
> Dear All,
>
> Here is the latest version of our mount time improvement.
It's looking good, but the kernel really needs to be able to write these
summaries for _itself_ in order to give a real improvement over the long
term. If the file system has to be read-only we might as well be using
cramfs, and if the summary becomes obsolete over time we might as well
not bother in a lot of cases.
--
dwmw2
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-20 14:26 Ferenc Havasi
2004-10-20 15:26 ` [OBORONA-SPAM] " Artem B. Bityuckiy
2004-10-21 6:29 ` Artem B. Bityuckiy
@ 2004-10-21 12:49 ` Jarkko Lavinen
2004-10-21 19:11 ` Ferenc Havasi
2004-10-22 9:58 ` Ferenc Havasi
2004-10-21 13:24 ` David Woodhouse
3 siblings, 2 replies; 113+ messages in thread
From: Jarkko Lavinen @ 2004-10-21 12:49 UTC (permalink / raw)
Cc: dwmw2, linux-mtd
On Wed, Oct 20, 2004 at 04:26:27PM +0200, ext Ferenc Havasi wrote:
> Jarkko made a measurement on a real NAND device: his JFFS2 image was
> 120819928 (115M), after running sumtool the new image was 123338752 (117M).
>
> Using the original mount time was 55 sec, with the new image it is only
> 8.5 sec.
My initial test was only about the mount time. I have now also tried
to exercise the patched file system and with very little testing I get
CRC or ECC errors.
# mount /dev/mtdblock2 /mnt -t jffs2
# mkdir /mnt/testdir
# umount /mnt
jffs2_flush_wbuf(): Write failed with -5
# mount /dev/mtdblock2 /mnt -t jffs2
mtd->read(0x1fbec bytes from 0x1fc0414) returned ECC error
Empty flash at 0x01fc2f1c ends at 0x01fc3000
#
With plain 2.6.9-rc4-omap1 with fresh CVS MTD code, I don't see anything
weird occuring.
Jarkko Lavinen
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-21 6:54 ` Ferenc Havasi
@ 2004-10-21 7:16 ` Artem B. Bityuckiy
2004-10-21 19:50 ` Ferenc Havasi
0 siblings, 1 reply; 113+ messages in thread
From: Artem B. Bityuckiy @ 2004-10-21 7:16 UTC (permalink / raw)
To: Ferenc Havasi; +Cc: dwmw2, linux-mtd, jffs-dev
> You are right, there is a small change which is really important (and
> will be ready very soon) to extend jffs2_mark_node_obsolete() to mark
> not only the node but also its entry in the summary.
Unfortunately, you can not mark entries as obsoleted in your summary
node in case of NAND.
If you write your summary only for *full* blocks, you will not need to
mark entries obsoleted, even if you have NOR flash (but you can on NOR).
The partially filled blocks must not have the summary node (you can
introduce special marker and write it to OOB of the last page of
NAND/last word of sector on NOR which tells if there is the summary node
present).
So, fully filled block will have summary and will be scanned very
quickly, partially filled ones will have no summary and will be fully
scanned, free blocks will have cleanmarkers and will not be scanned,
other blocks will be either erased or considered free.
>
> Any other improvement can be done later, because after it the filesystem
> will be always coherent, because we write summary only at the of the
> erasy blocks, when it is fully "finished" - so if there is a summary
> somewhere we will not need to extend it, only to mark the obscolated nodes.
Yes, nice, but why do you need to mark obsoleted nodes in summary ???
When you insert node to the fragtree or dirents to the list, JFFS2 code
will detect obsoleted nodes automatically, no need to mark them physically.
>
> We also plan in the near future to implement the ability of generating
> summary dinamically when the filesystem finishes an erase block - which
> keep this "fast mount time" permament.
This would be perfect.
--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-21 6:29 ` Artem B. Bityuckiy
@ 2004-10-21 6:54 ` Ferenc Havasi
2004-10-21 7:16 ` Artem B. Bityuckiy
0 siblings, 1 reply; 113+ messages in thread
From: Ferenc Havasi @ 2004-10-21 6:54 UTC (permalink / raw)
To: Artem B. Bityuckiy; +Cc: dwmw2, linux-mtd, jffs-dev
Hi Artem,
> As I understand, you only prepare JFFS2 image with summaries. This is
> great until we do not change anything. For read-only file-systems this
> is OK.
>
> But what if files/direntries are changed/deleted ? Do you write summary
> information dynamically? How are you going to place nodes/direntries to
> different blocks dynamically?
You are right, there is a small change which is really important (and
will be ready very soon) to extend jffs2_mark_node_obsolete() to mark
not only the node but also its entry in the summary.
Any other improvement can be done later, because after it the filesystem
will be always coherent, because we write summary only at the of the
erasy blocks, when it is fully "finished" - so if there is a summary
somewhere we will not need to extend it, only to mark the obscolated nodes.
We also plan in the near future to implement the ability of generating
summary dinamically when the filesystem finishes an erase block - which
keep this "fast mount time" permament.
Bye,
Ferenc
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-20 14:26 Ferenc Havasi
2004-10-20 15:26 ` [OBORONA-SPAM] " Artem B. Bityuckiy
@ 2004-10-21 6:29 ` Artem B. Bityuckiy
2004-10-21 6:54 ` Ferenc Havasi
2004-10-21 12:49 ` Jarkko Lavinen
2004-10-21 13:24 ` David Woodhouse
3 siblings, 1 reply; 113+ messages in thread
From: Artem B. Bityuckiy @ 2004-10-21 6:29 UTC (permalink / raw)
To: Ferenc Havasi; +Cc: dwmw2, linux-mtd, jffs-dev
Hello Ferenc,
As I understand, you only prepare JFFS2 image with summaries. This is
great until we do not change anything. For read-only file-systems this
is OK.
But what if files/direntries are changed/deleted ? Do you write summary
information dynamically? How are you going to place nodes/direntries to
different blocks dynamically?
Ferenc Havasi wrote:
> Dear All,
>
> Here is the latest version of our mount time improvement.
>
> Using of it:
> - apply this patch on the latest version of MTD
> - compile sumtool (make command in mtd/util)
> - make your JFFS2 image as before (or you can use already created images
> as well)
> - run sumtool to insert summary information, for example:
> ./sumtool -i original.jffs2 -o new.jffs2 -e128KiB
> - recompile your kernel with "JFFS2 inode summary support"
>
> Jarkko made a measurement on a real NAND device: his JFFS2 image was
> 120819928 (115M), after running sumtool the new image was 123338752 (117M).
>
> Using the original mount time was 55 sec, with the new image it is only
> 8.5 sec.
>
> It works very similar as our previous improvement: stores special
> information at the end of the erase blocks, and at mount time if there
> is this kind of information the scaning of the erase block is unneccessary.
>
> New things compared to our previous improvement:
> - it was fully rewritten
> - we separated the user space tool from mkfs. (sumtool)
> - sumtool now not only inserts the summary information but also make
> some node-reordering. There will be two kind of erase blocks: in the
> "first type" there will be only jffs2_raw_inodes, and all other node
> (jffs2_raw_dirent) will be stored in the "second type". It generates
> summary at the end of all "fist type" eraseblock. (the "second type"
> will be scanned as before, because all information is needed in
> jffs_raw_dirent at mount time)
>
> Ceratinly all of these things are optional (as you can see above you
> have to select it from kernel config). The JFFS2 image produced by
> sumtool is also usable with previous kernel because the summary node is
> JFFS2_FEATURE_RWCOMPAT_DELETE.
>
> I think it can be usefull not only for us. David, may I commit it to the
> CVS?
>
> Regards,
> Ferenc
--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-20 15:49 ` Ferenc Havasi
@ 2004-10-20 15:53 ` Artem B. Bityuckiy
0 siblings, 0 replies; 113+ messages in thread
From: Artem B. Bityuckiy @ 2004-10-20 15:53 UTC (permalink / raw)
To: Ferenc Havasi; +Cc: linux-mtd
Ferenc Havasi wrote:
> The reason is the following: the summary node is at the end of the erase
> block, and it has not fixed size (its size depends on the information it
> stores).
>
> The main advantage of using summary node is to avoid the original
> scanning method. So we cannot use the original full-scanning method to
> determine the begining of the summary node (using only
> JFFS2_NODETYPE_INODE_SUM).
>
> Our method is the following:
> - read some bytes at the end of the erase block
> - if the last word is JFFS2_SUM_MAGIC than we will almost sure that it
> is an erase block which has summary
> - the word before this magic is the length of the node
> - using this length we can check that it is really a
> JFFS2_NODETYPE_INODE_SUM node, and process it
>
> I can't image more effective method to determine the begining of the
> summary node. (if you have better suggestion...) And because the magic
> is inside of the summary node I think it is fit to the philosophy of
> JFFS2 - but a little bit tricky.
Ok, I got it. I was wrong, sorry.
--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-10-20 15:26 ` [OBORONA-SPAM] " Artem B. Bityuckiy
@ 2004-10-20 15:49 ` Ferenc Havasi
2004-10-20 15:53 ` Artem B. Bityuckiy
0 siblings, 1 reply; 113+ messages in thread
From: Ferenc Havasi @ 2004-10-20 15:49 UTC (permalink / raw)
To: Artem B. Bityuckiy; +Cc: dwmw2, linux-mtd
Hi Artem,
> 1. Why did you introduce the JFFS2_SUM_MAGIC constant? As I understand,
> the node's magic field is needed to identify the *beginning of node*,
> *not the node type*. The type of node is defined by the next field,
> called 'nodetype'. You use it (JFFS2_NODETYPE_INODE_SUM). So, IMHO, the
> JFFS2_SUM_MAGIC constant doesn't fit into the common rules...
The reason is the following: the summary node is at the end of the erase
block, and it has not fixed size (its size depends on the information it
stores).
The main advantage of using summary node is to avoid the original
scanning method. So we cannot use the original full-scanning method to
determine the begining of the summary node (using only
JFFS2_NODETYPE_INODE_SUM).
Our method is the following:
- read some bytes at the end of the erase block
- if the last word is JFFS2_SUM_MAGIC than we will almost sure that it
is an erase block which has summary
- the word before this magic is the length of the node
- using this length we can check that it is really a
JFFS2_NODETYPE_INODE_SUM node, and process it
I can't image more effective method to determine the begining of the
summary node. (if you have better suggestion...) And because the magic
is inside of the summary node I think it is fit to the philosophy of
JFFS2 - but a little bit tricky.
> 2. This is very minor of course, just a remark. IMHO, its better to
> avoid too many ifdefs, so, I think it is unnecessary to place the
> function prototype under ifdef. I mead:
>
> +#ifdef CONFIG_JFFS2_FS_SUMMARY
> +static struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct
> jffs2_sb_info *c, uint32_t ino);
> +#endif
Yes, I aggree. I will modify it.
Bye,
Ferenc
^ permalink raw reply [flat|nested] 113+ messages in thread
* JFFS2 mount time
@ 2004-10-20 14:26 Ferenc Havasi
2004-10-20 15:26 ` [OBORONA-SPAM] " Artem B. Bityuckiy
` (3 more replies)
0 siblings, 4 replies; 113+ messages in thread
From: Ferenc Havasi @ 2004-10-20 14:26 UTC (permalink / raw)
To: linux-mtd, jffs-dev, dwmw2
[-- Attachment #1: Type: text/plain, Size: 1792 bytes --]
Dear All,
Here is the latest version of our mount time improvement.
Using of it:
- apply this patch on the latest version of MTD
- compile sumtool (make command in mtd/util)
- make your JFFS2 image as before (or you can use already created images
as well)
- run sumtool to insert summary information, for example:
./sumtool -i original.jffs2 -o new.jffs2 -e128KiB
- recompile your kernel with "JFFS2 inode summary support"
Jarkko made a measurement on a real NAND device: his JFFS2 image was
120819928 (115M), after running sumtool the new image was 123338752 (117M).
Using the original mount time was 55 sec, with the new image it is only
8.5 sec.
It works very similar as our previous improvement: stores special
information at the end of the erase blocks, and at mount time if there
is this kind of information the scaning of the erase block is unneccessary.
New things compared to our previous improvement:
- it was fully rewritten
- we separated the user space tool from mkfs. (sumtool)
- sumtool now not only inserts the summary information but also make
some node-reordering. There will be two kind of erase blocks: in the
"first type" there will be only jffs2_raw_inodes, and all other node
(jffs2_raw_dirent) will be stored in the "second type". It generates
summary at the end of all "fist type" eraseblock. (the "second type"
will be scanned as before, because all information is needed in
jffs_raw_dirent at mount time)
Ceratinly all of these things are optional (as you can see above you
have to select it from kernel config). The JFFS2 image produced by
sumtool is also usable with previous kernel because the summary node is
JFFS2_FEATURE_RWCOMPAT_DELETE.
I think it can be usefull not only for us. David, may I commit it to the
CVS?
Regards,
Ferenc
[-- Attachment #2: jffs2-summary.patch --]
[-- Type: text/x-patch, Size: 36792 bytes --]
diff --unified --recursive --new-file mtd2/fs/Kconfig mtd/fs/Kconfig
--- mtd2/fs/Kconfig 2004-07-16 17:20:59.000000000 +0200
+++ mtd/fs/Kconfig 2004-10-20 15:10:42.000000000 +0200
@@ -68,6 +68,19 @@
Say 'N' unless you have NAND flash and you are willing to test and
develop JFFS2 support for it.
+config JFFS2_FS_SUMMARY
+ bool "JFFS2 inode summary support (EXPERIMENTAL)"
+ depends on JFFS2_FS
+ default n
+ help
+ This feature makes it possible to use inode summary information
+ for faster filesystem mount - specially on NAND.
+
+ The summary information can be inserted into a filesystem image
+ by the utility 'sumtool'.
+
+ If unsure, say 'N'.
+
config JFFS2_COMPRESSION_OPTIONS
bool "Advanced compression options for JFFS2"
default n
diff --unified --recursive --new-file mtd2/fs/jffs2/scan.c mtd/fs/jffs2/scan.c
--- mtd2/fs/jffs2/scan.c 2004-09-12 11:56:13.000000000 +0200
+++ mtd/fs/jffs2/scan.c 2004-10-20 15:44:25.000000000 +0200
@@ -4,6 +4,8 @@
* Copyright (C) 2001-2003 Red Hat, Inc.
*
* Created by David Woodhouse <dwmw2@redhat.com>
+ * Inode summary support by Zoltan Sogor, Ferenc Havasi, Patrik Kluba
+ * University of Szeged, Hungary
*
* For licensing information, see the file 'LICENCE' in this directory.
*
@@ -58,6 +60,11 @@
static int jffs2_scan_dirent_node(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb,
struct jffs2_raw_dirent *rd, uint32_t ofs);
+
+#ifdef CONFIG_JFFS2_FS_SUMMARY
+static struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino);
+#endif
+
#define BLK_STATE_ALLFF 0
#define BLK_STATE_CLEAN 1
#define BLK_STATE_PARTDIRTY 2
@@ -292,6 +299,25 @@
#ifdef CONFIG_JFFS2_FS_NAND
int cleanmarkerfound = 0;
#endif
+#ifdef CONFIG_JFFS2_FS_SUMMARY
+ struct jffs2_raw_node_ref *raw;
+ struct jffs2_raw_node_ref *cache_ref;
+ struct jffs2_inode_cache *ic;
+
+ typedef struct sum_marker {
+ jint32_t offset;
+ jint32_t magic;
+ } sum_marker;
+
+ sum_marker *sm;
+ int i;
+ int sumsize;
+ uint32_t ino;
+ uint32_t crc;
+ struct jffs2_inode_sum_node *summary;
+ struct jffs2_inode_sum_record *sum_rec;
+ int bad_sum = 0;
+#endif
ofs = jeb->offset;
prevofs = jeb->offset - 1;
@@ -314,10 +340,217 @@
}
}
#endif
+
+#ifdef CONFIG_JFFS2_FS_SUMMARY
+ /* Looking for summary marker */
+ sm = (sum_marker *)kmalloc(sizeof(*sm), GFP_KERNEL);
+ if (!sm) {
+ return -ENOMEM;
+ }
+
+ err = jffs2_fill_scan_buf(c, (unsigned char *) sm, jeb->offset + c->sector_size - 8, 8);
+
+ if (err) {
+ return err;
+ }
+
+ if (je32_to_cpu(sm->magic) == JFFS2_SUM_MAGIC) {
+ ofs = je32_to_cpu(sm->offset);
+ sumsize = c->sector_size - ofs;
+ ofs += jeb->offset;
+
+ D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Inode summary information found at 0x%x (%d bytes)\n", ofs, sumsize));
+
+ summary = (struct jffs2_inode_sum_node *) kmalloc(sumsize, GFP_KERNEL);
+
+ if (!summary) {
+ kfree(sm);
+ return -ENOMEM;
+ }
+
+ err = jffs2_fill_scan_buf(c, (unsigned char *)summary, ofs, sumsize);
+
+ if (err) {
+ kfree(sm);
+ kfree(summary);
+ return err;
+ }
+
+ /* OK, now check for node validity and CRC */
+ crcnode.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
+ crcnode.nodetype = cpu_to_je16(JFFS2_NODETYPE_INODE_SUM);
+ crcnode.totlen = summary->totlen;
+ hdr_crc = crc32(0, &crcnode, sizeof(crcnode)-4);
+
+ if (je32_to_cpu(summary->hdr_crc) != hdr_crc) {
+ D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node header is corrupt (bad CRC or no summary at all)\n"));
+ bad_sum = 1;
+ }
+
+ if ((!bad_sum) && (je32_to_cpu(summary->totlen) != sumsize)) {
+ D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (wrong erasesize?)\n"));
+ bad_sum = 1;
+ }
+
+ crc = crc32(0, summary, sizeof(struct jffs2_inode_sum_node)-8);
+
+ if ((!bad_sum) && (je32_to_cpu(summary->node_crc) != crc)) {
+ D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (bad CRC)\n"));
+ bad_sum = 1;
+ }
+
+ sum_rec = (struct jffs2_inode_sum_record *) &(summary->sum[0]);
+ crc = crc32(0, sum_rec, sumsize - sizeof(struct jffs2_inode_sum_node));
+
+ if ((!bad_sum) && (je32_to_cpu(summary->sum_crc) != crc)) {
+ D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node data is corrupt (bad CRC)\n"));
+ bad_sum = 1;
+ }
+
+ if (!bad_sum) {
+
+ if ( je32_to_cpu(summary->cln_mkr) ){
+
+ D1(printk(KERN_DEBUG "Summary : CLEANMARKER node \n"));
+
+ if (je32_to_cpu(summary->cln_mkr) != c->cleanmarker_size) {
+ printk(KERN_DEBUG "CLEANMARKER node has totlen 0x%x != normal 0x%x\n",
+ je32_to_cpu(summary->cln_mkr), c->cleanmarker_size);
+ UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) );
+ }
+ else if (jeb->first_node) {
+ printk(KERN_DEBUG "CLEANMARKER node not first node in block (0x%08x)\n", jeb->offset);
+ UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) );
+ }
+ else {
+ struct jffs2_raw_node_ref *marker_ref = jffs2_alloc_raw_node_ref();
+
+ if (!marker_ref) {
+ printk(KERN_NOTICE "Failed to allocate node ref for clean marker\n");
+ return -ENOMEM;
+ }
+
+ marker_ref->next_in_ino = NULL;
+ marker_ref->next_phys = NULL;
+ marker_ref->flash_offset = jeb->offset | REF_NORMAL;
+ marker_ref->__totlen = je32_to_cpu(summary->cln_mkr);
+ jeb->first_node = jeb->last_node = marker_ref;
+
+ USED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) );
+
+ }
+ }
+
+ for(i = 0; i < je16_to_cpu(summary->sum_num); i++) {
+
+ D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Processing summary information %d\n", i));
+
+ //JFFS2_NODETYPE_INODE:
+ ino = je32_to_cpu(sum_rec->inode);
+ D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Inode at 0x%08x\n", jeb->offset + je32_to_cpu(sum_rec->offset)));
+ raw = jffs2_alloc_raw_node_ref();
+ if (!raw) {
+ printk(KERN_NOTICE "jffs2_scan_eraseblock(): allocation of node reference failed\n");
+ kfree(sm);
+ kfree(summary);
+ return -ENOMEM;
+ }
+
+ ic = jffs2_get_ino_cache(c, ino);
+ if (!ic) {
+ ic = jffs2_scan_make_ino_cache(c, ino);
+ if (!ic) {
+ printk(KERN_NOTICE "jffs2_scan_eraseblock(): scan_make_ino_cache failed\n");
+ jffs2_free_raw_node_ref(raw);
+ kfree(sm);
+ kfree(summary);
+ return -ENOMEM;
+ }
+ }
+
+ raw->flash_offset = (jeb->offset + je32_to_cpu(sum_rec->offset)) | REF_UNCHECKED;
+ raw->__totlen = PAD(je32_to_cpu(sum_rec->totlen));
+ raw->next_phys = NULL;
+ raw->next_in_ino = ic->nodes;
+
+ ic->nodes = raw;
+ if (!jeb->first_node)
+ jeb->first_node = raw;
+ if (jeb->last_node)
+ jeb->last_node->next_phys = raw;
+ jeb->last_node = raw;
+
+ /* do we need this? this requires storing another 4 bytes per record in the cache or an expensive reading */
+ pseudo_random += je32_to_cpu(sum_rec->version);
+
+ UNCHECKED_SPACE(PAD(je32_to_cpu(sum_rec->totlen)));
+
+ sum_rec++;
+ }
+
+ kfree(sm);
+ kfree(summary);
+
+ /* for ACCT_PARANOIA_CHECK */
+ cache_ref = jffs2_alloc_raw_node_ref();
+
+ if (!cache_ref) {
+ printk(KERN_NOTICE "Failed to allocate node ref for cache\n");
+ return -ENOMEM;
+ }
+
+ cache_ref->next_in_ino = NULL;
+ cache_ref->next_phys = NULL;
+ cache_ref->flash_offset = ofs | REF_NORMAL;
+ cache_ref->__totlen = sumsize;
+
+ if (!jeb->first_node)
+ jeb->first_node = cache_ref;
+ if (jeb->last_node)
+ jeb->last_node->next_phys = cache_ref;
+ jeb->last_node = cache_ref;
+
+ USED_SPACE(sumsize);
+
+ /* somebody check this and all of space accounting in summary support */
+
+ if ((jeb->used_size + jeb->unchecked_size) == PAD(c->cleanmarker_size) && !jeb->dirty_size
+ && (!jeb->first_node || !jeb->first_node->next_in_ino) ) {
+ return BLK_STATE_CLEANMARKER;
+ }
+ /* move blocks with max 4 byte dirty space to cleanlist */
+ else if (!ISDIRTY(c->sector_size - (jeb->used_size + jeb->unchecked_size))) {
+ c->dirty_size -= jeb->dirty_size;
+ c->wasted_size += jeb->dirty_size;
+ jeb->wasted_size += jeb->dirty_size;
+ jeb->dirty_size = 0;
+ return BLK_STATE_CLEAN;
+ }
+ else if (jeb->used_size || jeb->unchecked_size) {
+ return BLK_STATE_PARTDIRTY;
+ }
+ else {
+ return BLK_STATE_ALLDIRTY;
+ }
+ }
+ }
+ D1(printk(KERN_DEBUG "Summary end\n"));
+
+ ofs = jeb->offset;
+ prevofs = jeb->offset - 1;
+
+#endif
+
buf_ofs = jeb->offset;
if (!buf_size) {
buf_len = c->sector_size;
+#ifdef CONFIG_JFFS2_FS_SUMMARY
+ /* must reread because of summary test */
+ err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len);
+ if (err)
+ return err;
+#endif
} else {
buf_len = EMPTY_SCAN_SIZE;
err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len);
diff --unified --recursive --new-file mtd2/include/linux/jffs2.h mtd/include/linux/jffs2.h
--- mtd2/include/linux/jffs2.h 2004-05-25 13:31:55.000000000 +0200
+++ mtd/include/linux/jffs2.h 2004-10-20 14:53:52.000000000 +0200
@@ -28,6 +28,9 @@
#define JFFS2_EMPTY_BITMASK 0xffff
#define JFFS2_DIRTY_BITMASK 0x0000
+/* Summary node MAGIC marker */
+#define JFFS2_SUM_MAGIC 0x02851885
+
/* We only allow a single char for length, and 0xFF is empty flash so
we don't want it confused with a real length. Hence max 254.
*/
@@ -61,6 +64,7 @@
#define JFFS2_NODETYPE_INODE (JFFS2_FEATURE_INCOMPAT | JFFS2_NODE_ACCURATE | 2)
#define JFFS2_NODETYPE_CLEANMARKER (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3)
#define JFFS2_NODETYPE_PADDING (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 4)
+#define JFFS2_NODETYPE_INODE_SUM (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 6)
// Maybe later...
//#define JFFS2_NODETYPE_CHECKPOINT (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3)
@@ -148,10 +152,31 @@
uint8_t data[0];
} __attribute__((packed));
+struct jffs2_inode_sum_node{
+ jint16_t magic;
+ jint16_t nodetype; /* = JFFS2_NODETYPE_INODE_SUM */
+ jint32_t totlen;
+ jint32_t hdr_crc;
+ jint16_t sum_num; /* number of sum entries*/
+ jint32_t cln_mkr; /* clean marker size, 0 = no cleanmarker */
+ jint32_t sum_crc; /* summary information crc */
+ jint32_t node_crc; /* node crc */
+ jint32_t sum[0]; /* inode summary info */
+} __attribute__((packed));
+
+struct jffs2_inode_sum_record{
+ jint32_t inode;
+ jint32_t version;
+ jint32_t offset;
+ jint32_t totlen;
+} __attribute__((packed));
+
+
union jffs2_node_union {
struct jffs2_raw_inode i;
struct jffs2_raw_dirent d;
struct jffs2_unknown_node u;
+ struct jffs2_inode_sum_node s;
};
#endif /* __LINUX_JFFS2_H__ */
diff --unified --recursive --new-file mtd2/util/Makefile mtd/util/Makefile
--- mtd2/util/Makefile 2004-07-13 19:49:43.000000000 +0200
+++ mtd/util/Makefile 2004-10-19 15:11:02.000000000 +0200
@@ -13,7 +13,7 @@
TARGETS = ftl_format flash_erase flash_eraseall nanddump doc_loadbios \
mkfs.jffs ftl_check mkfs.jffs2 flash_lock flash_unlock \
flash_info mtd_debug flashcp nandwrite jffs2dump \
- nftldump nftl_format docfdisk #jffs2reader
+ nftldump nftl_format docfdisk sumtool #jffs2reader
SYMLINKS = compr_lzari.c compr_lzo.c
@@ -48,6 +48,9 @@
jffs2dump: jffs2dump.o crc32.o
$(CC) $(LDFLAGS) -o $@ $^
+sumtool: sumtool.o crc32.o
+ $(CC) $(LDFLAGS) -o $@ $^
+
install: ${TARGETS}
mkdir -p ${DESTDIR}/${SBINDIR}
install -m0755 -oroot -groot ${TARGETS} ${DESTDIR}/${SBINDIR}/
diff --unified --recursive --new-file mtd2/util/jffs2dump.c mtd/util/jffs2dump.c
--- mtd2/util/jffs2dump.c 2004-06-19 00:11:48.000000000 +0200
+++ mtd/util/jffs2dump.c 2004-10-20 14:55:28.000000000 +0200
@@ -3,7 +3,7 @@
*
* Copyright (C) 2003 Thomas Gleixner (tglx@linutronix.de)
*
- * $Id: jffs2dump.c,v 1.6 2004/06/18 22:11:48 gleixner Exp $
+ * $Id: jffs2dump.c,v 1.1 2004/10/19 07:19:55 weth Exp $
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
@@ -277,7 +277,63 @@
p += PAD(je32_to_cpu (node->d.totlen));
break;
-
+
+ case JFFS2_NODETYPE_INODE_SUM:{
+
+ int i;
+ jint32_t *offset,*magic;
+
+ printf ("%8s Inode Sum node at 0x%08x, totlen 0x%08x, sum_num %5d, cleanmarker size %5d\n",
+ obsolete ? "Obsolete" : "",
+ p - data,
+ je32_to_cpu (node->s.totlen),
+ je16_to_cpu (node->s.sum_num),
+ je32_to_cpu (node->s.cln_mkr));
+
+ crc = crc32 (0, node, sizeof (struct jffs2_inode_sum_node) - 8);
+ if (crc != je32_to_cpu (node->s.node_crc)) {
+ printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.node_crc), crc);
+ p += PAD(je32_to_cpu (node->s.totlen));
+ dirty += PAD(je32_to_cpu (node->s.totlen));;
+ continue;
+ }
+
+ crc = crc32(0, p + sizeof (struct jffs2_inode_sum_node), je32_to_cpu (node->s.totlen) - sizeof(struct jffs2_inode_sum_node));
+ if (crc != je32_to_cpu(node->s.sum_crc)) {
+ printf ("Wrong data_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.sum_crc), crc);
+ p += PAD(je32_to_cpu (node->s.totlen));
+ dirty += PAD(je32_to_cpu (node->s.totlen));;
+ continue;
+ }
+
+ if(verbose){
+ for(i = 0; i < je16_to_cpu (node->s.sum_num); i++){
+ struct jffs2_inode_sum_record *sp;
+ sp = (struct jffs2_inode_sum_record *) (p + sizeof (struct jffs2_inode_sum_node));
+
+ printf ("%14s #ino %5d, version %5d, offset %8d, totlen 0x%08x\n",
+ "",
+ je32_to_cpu (sp[i].inode),
+ je32_to_cpu (sp[i].version),
+ je32_to_cpu (sp[i].offset),
+ je32_to_cpu (sp[i].totlen));
+
+ }
+
+ offset = (jint32_t *)((char *)p + je32_to_cpu(node->s.totlen) - 8);
+ magic = (jint32_t *)((char *)p + je32_to_cpu(node->s.totlen) - 4);
+
+ printf("%14s Sum Node Offset 0x%08x, Magic 0x%08x\n",
+ "",
+ je32_to_cpu(*offset),
+ je32_to_cpu(*magic));
+ }
+
+ p += PAD(je32_to_cpu (node->s.totlen));
+ break;
+
+ }
+
case JFFS2_NODETYPE_CLEANMARKER:
if (verbose) {
printf ("%8s Cleanmarker at 0x%08x, totlen 0x%08x\n",
@@ -418,9 +474,9 @@
write (fd, &newnode, sizeof (struct jffs2_raw_dirent));
write (fd, p + sizeof (struct jffs2_raw_dirent), PAD (je32_to_cpu (node->d.totlen) - sizeof (struct jffs2_raw_dirent)));
- p += PAD(je32_to_cpu (node->d.totlen));
+ p += PAD(je32_to_cpu (node->d.totlen));
break;
-
+
case JFFS2_NODETYPE_CLEANMARKER:
case JFFS2_NODETYPE_PADDING:
newnode.u.magic = cnv_e16 (node->u.magic);
diff --unified --recursive --new-file mtd2/util/sumtool.c mtd/util/sumtool.c
--- mtd2/util/sumtool.c 1970-01-01 01:00:00.000000000 +0100
+++ mtd/util/sumtool.c 2004-10-20 11:56:08.000000000 +0200
@@ -0,0 +1,800 @@
+/*
+ * sumtool.c
+ *
+ * Copyright (C) 2004 Zoltan Sogor <weth@inf.u-szeged.hu>,
+ * Ferenc Havasi <havasi@inf.u-szeged.hu>,
+ * Patrik Kluba <pajko@halom.u-szeged.hu>,
+ * University of Szeged, Hungary
+ *
+ * $Id: sumtool.c,v 1.2 2004/10/20 09:56:08 hafy Exp $
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+ *
+ * Overview:
+ * This is a utility to reorder nodes and insert inode summary information
+ * into JFFS2 image for faster mount time - specially on NAND.
+ *
+ */
+
+#include <errno.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <string.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <time.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/param.h>
+#include <asm/types.h>
+#include <dirent.h>
+#include <mtd/jffs2-user.h>
+#include <endian.h>
+#include <byteswap.h>
+#include <getopt.h>
+#include "crc32.h"
+
+#define PAD(x) (((x)+3)&~3)
+
+#define SINODE 0 /* Inode Type*/
+#define SONODE 1 /* Other type*/
+
+static const char *const app_name = "sumtool";
+
+typedef struct sum_storage {
+ jint32_t inode;
+ jint32_t version;
+ jint32_t offset;
+ jint32_t totlen;
+ struct sum_storage *next;
+} sum_storage;
+
+static sum_storage *sum_collected = NULL; /* summary info list */
+static int sum_records = 0; /* number of sumary records */
+
+
+static int verbose = 0;
+static int add_cleanmarkers = 1; /* add cleanmarker to output */
+static int use_input_cleanmarker_size = 1; /* use input file's cleanmarker size (default) */
+static int found_cleanmarkers = 0; /* cleanmarker found in input file */
+static struct jffs2_unknown_node cleanmarker;
+static int cleanmarker_size = sizeof(cleanmarker);
+static const char *short_options = "o:i:e:hvVblnc:";
+static int erase_block_size = 65536;
+static int target_endian = __BYTE_ORDER;
+static int out_fd = -1;
+static int in_fd = -1;
+
+static uint8_t *inode_buffer = NULL; /* buffer for inodes */
+static unsigned int ino_ofs = 0; /* inode buffer offset */
+
+static uint8_t *dirent_buffer = NULL; /* buffer for directory entries and other (symlink, spec. device files, etc.)*/
+static unsigned int dent_ofs = 0; /* directory enrty buffer offset*/
+
+static uint8_t *file_buffer = NULL; /* file buffer contains the actual erase block*/
+static unsigned int file_ofs = 0; /* position in the buffer */
+
+static struct option long_options[] = {
+ {"output", 1, NULL, 'o'},
+ {"input", 1, NULL, 'i'},
+ {"eraseblock", 1, NULL, 'e'},
+ {"help", 0, NULL, 'h'},
+ {"verbose", 0, NULL, 'v'},
+ {"version", 0, NULL, 'V'},
+ {"bigendian", 0, NULL, 'b'},
+ {"littleendian", 0, NULL, 'l'},
+ {"no-cleanmarkers", 0, NULL, 'n'},
+ {"cleanmarker", 1, NULL, 'c'},
+ {NULL, 0, NULL, 0}
+};
+
+static char *helptext =
+ "Usage: sumtool [OPTIONS] -i inputfile -o outputfile\n"
+ "Convert the input JFFS2 file to a SUM-ed JFFS2 file\n\n"
+ "Options:\n"
+ " -e, --eraseblock=SIZE Use erase block size SIZE (default: 64KiB)\n"
+ " (usually 16KiB on NAND)\n"
+ " -c, --cleanmarker=SIZE Size of cleanmarker (default 12).\n"
+ " (usually 16 bytes on NAND, and will be set to\n"
+ " this value if left at the default 12). Will be\n"
+ " stored in OOB after each physical page composing\n"
+ " a physical erase block.\n"
+ " -n, --no-cleanmarkers Don't add a cleanmarker to every eraseblock\n"
+ " -o, --output=FILE Output to FILE \n"
+ " -i, --input=FILE Input from FILE \n"
+ " -b, --bigendian Image is big endian\n"
+ " -l --littleendian Image is little endian\n"
+ " -h, --help Display this help text\n"
+ " -v, --verbose Verbose operation\n"
+ " -V, --version Display version information\n\n";
+
+
+static char *revtext = "$Revision: 1.2 $";
+
+static unsigned char ffbuf[16] = {
+ 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
+ 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff
+};
+
+static void verror_msg(const char *s, va_list p) {
+ fflush(stdout);
+ fprintf(stderr, "%s: ", app_name);
+ vfprintf(stderr, s, p);
+}
+
+static void error_msg_and_die(const char *s, ...) {
+ va_list p;
+
+ va_start(p, s);
+ verror_msg(s, p);
+ va_end(p);
+ putc('\n', stderr);
+ exit(EXIT_FAILURE);
+}
+
+static void vperror_msg(const char *s, va_list p) {
+ int err = errno;
+
+ if (s == 0)
+ s = "";
+ verror_msg(s, p);
+ if (*s)
+ s = ": ";
+ fprintf(stderr, "%s%s\n", s, strerror(err));
+}
+
+static void perror_msg_and_die(const char *s, ...) {
+ va_list p;
+
+ va_start(p, s);
+ vperror_msg(s, p);
+ va_end(p);
+ exit(EXIT_FAILURE);
+}
+
+
+
+static void full_write(void *target_buff, const void *buf, int len, int nd);
+
+void setup_cleanmarker() {
+
+ cleanmarker.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
+ cleanmarker.nodetype = cpu_to_je16(JFFS2_NODETYPE_CLEANMARKER);
+ cleanmarker.totlen = cpu_to_je32(cleanmarker_size);
+ cleanmarker.hdr_crc = cpu_to_je32(crc32(0, &cleanmarker, sizeof(struct jffs2_unknown_node)-4));
+}
+
+void process_options (int argc, char **argv){
+ int opt,c;
+
+ while ((opt = getopt_long(argc, argv, short_options, long_options, &c)) >= 0)
+ {
+ switch (opt)
+ {
+ case 'o':
+ if (out_fd != -1) {
+ error_msg_and_die("output filename specified more than once");
+ }
+ out_fd = open(optarg, O_CREAT | O_TRUNC | O_RDWR, 0644);
+ if (out_fd == -1) {
+ perror_msg_and_die("open output file");
+ }
+ break;
+
+ case 'i':
+ if (in_fd != -1) {
+ error_msg_and_die("input filename specified more than once");
+ }
+ in_fd = open(optarg, O_RDONLY);
+ if (in_fd == -1) {
+ perror_msg_and_die("open input file");
+ }
+ break;
+ case 'b':
+ target_endian = __BIG_ENDIAN;
+ break;
+ case 'l':
+ target_endian = __LITTLE_ENDIAN;
+ break;
+ case 'h':
+ case '?':
+ error_msg_and_die(helptext);
+
+ case 'v':
+ verbose = 1;
+ break;
+
+ case 'V':
+ error_msg_and_die("revision %.*s\n",
+ (int) strlen(revtext) - 13, revtext + 11);
+
+ case 'e': {
+ char *next;
+ unsigned units = 0;
+ erase_block_size = strtol(optarg, &next, 0);
+ if (!erase_block_size)
+ error_msg_and_die("Unrecognisable erase size\n");
+
+ if (*next) {
+ if (!strcmp(next, "KiB")) {
+ units = 1024;
+ } else if (!strcmp(next, "MiB")) {
+ units = 1024 * 1024;
+ } else {
+ error_msg_and_die("Unknown units in erasesize\n");
+ }
+ } else {
+ if (erase_block_size < 0x1000)
+ units = 1024;
+ else
+ units = 1;
+ }
+ erase_block_size *= units;
+
+ /* If it's less than 8KiB, they're not allowed */
+ if (erase_block_size < 0x2000) {
+ fprintf(stderr, "Erase size 0x%x too small. Increasing to 8KiB minimum\n",
+ erase_block_size);
+ erase_block_size = 0x2000;
+ }
+ break;
+ }
+
+ case 'n':
+ add_cleanmarkers = 0;
+ break;
+ case 'c':
+ cleanmarker_size = strtol(optarg, NULL, 0);
+
+ if (cleanmarker_size < sizeof(cleanmarker)) {
+ error_msg_and_die("cleanmarker size must be >= 12");
+ }
+ if (cleanmarker_size >= erase_block_size) {
+ error_msg_and_die("cleanmarker size must be < eraseblock size");
+ }
+
+ use_input_cleanmarker_size = 0;
+ found_cleanmarkers = 1;
+ setup_cleanmarker();
+
+ break;
+
+ }
+ }
+}
+
+
+void init_buffers() {
+
+ inode_buffer = malloc(erase_block_size);
+
+ if (!inode_buffer) {
+ perror("out of memory");
+ close (in_fd);
+ close (out_fd);
+ exit(1);
+ }
+
+ dirent_buffer = malloc(erase_block_size);
+
+ if (!dirent_buffer) {
+ perror("out of memory");
+ close (in_fd);
+ close (out_fd);
+ exit(1);
+ }
+
+ file_buffer = malloc(erase_block_size);
+
+ if (!file_buffer) {
+ perror("out of memory");
+ close (in_fd);
+ close (out_fd);
+ exit(1);
+ }
+}
+
+void clean_buffers() {
+
+ if (inode_buffer)
+ free(inode_buffer);
+ if (dirent_buffer)
+ free(dirent_buffer);
+ if (file_buffer)
+ free(file_buffer);
+}
+
+int load_next_block() {
+
+ int ret;
+ ret = read(in_fd, file_buffer, erase_block_size);
+ file_ofs = 0;
+
+ if(verbose)
+ printf("Load next block : %d bytes read\n",ret);
+
+ return ret;
+}
+
+void write_buff_to_file(int nd) {
+
+ int ret;
+ int len = erase_block_size;
+ uint8_t *buf = NULL;
+
+ if (!nd) {
+ buf = inode_buffer;
+ while (len > 0) {
+ ret = write(out_fd, buf, len);
+
+ if (ret < 0)
+ perror_msg_and_die("write");
+
+ if (ret == 0)
+ perror_msg_and_die("write returned zero");
+
+ len -= ret;
+ buf += ret;
+ }
+ ino_ofs = 0;
+ }
+ else {
+ buf = dirent_buffer;
+ while (len > 0) {
+ ret = write(out_fd, buf, len);
+
+ if (ret < 0)
+ perror_msg_and_die("write");
+
+ if (ret == 0)
+ perror_msg_and_die("write returned zero");
+
+ len -= ret;
+ buf += ret;
+ }
+ dent_ofs = 0;
+ }
+}
+
+void dump_sum_records() {
+
+ struct jffs2_inode_sum_node isum;
+ struct sum_storage *temp;
+ jint32_t offset;
+ jint32_t *wpage;
+ int datasize;
+ int infosize;
+ int padsize;
+ jint32_t magic = cpu_to_je32(JFFS2_SUM_MAGIC);
+
+ if (!sum_records)
+ return;
+
+ datasize = sum_records * sizeof(struct jffs2_inode_sum_record) + 8;
+ infosize = sizeof(struct jffs2_inode_sum_node) + datasize;
+ padsize = erase_block_size - ino_ofs - infosize;
+ infosize += padsize; datasize += padsize;
+ offset = cpu_to_je32(ino_ofs);
+ jint32_t *tpage = (jint32_t *) malloc(datasize);
+
+ if(!tpage)
+ error_msg_and_die("Can't allocate memory to dump summary information!\n");
+
+ memset(tpage, 0xff, datasize);
+ memset(&isum, 0, sizeof(isum));
+
+ isum.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
+ isum.nodetype = cpu_to_je16(JFFS2_NODETYPE_INODE_SUM);
+ isum.totlen = cpu_to_je32(infosize);
+ isum.hdr_crc = cpu_to_je32(crc32(0, &isum, sizeof(struct jffs2_unknown_node) - 4));
+
+ if (add_cleanmarkers && found_cleanmarkers) {
+ isum.cln_mkr = cpu_to_je32(cleanmarker_size);
+ }
+ else{
+ isum.cln_mkr = cpu_to_je32(0);
+ }
+
+ isum.sum_num = cpu_to_je16(sum_records);
+ wpage = tpage;
+
+ while (sum_records) {
+ *(wpage++) = sum_collected->inode;
+ *(wpage++) = sum_collected->version;
+ *(wpage++) = sum_collected->offset;
+ *(wpage++) = sum_collected->totlen;
+ temp = sum_collected;
+ sum_collected = sum_collected->next;
+ free(temp);
+ sum_records--;
+ }
+
+ ((char *)wpage) += padsize;
+ *(wpage++) = offset;
+ *(wpage++) = magic;
+ isum.sum_crc = cpu_to_je32(crc32(0, tpage, datasize));
+ isum.node_crc = cpu_to_je32(crc32(0, &isum, sizeof(isum) - 8));
+
+ full_write(inode_buffer + ino_ofs, &isum, sizeof(isum), SINODE);
+ full_write(inode_buffer + ino_ofs, tpage, datasize, SINODE);
+
+ free(tpage);
+}
+
+static void full_write(void *target_buff, const void *buf, int len, int nd) {
+ memcpy(target_buff, buf, len);
+
+ if (!nd)
+ ino_ofs += len;
+ else
+ dent_ofs += len;
+}
+
+static void pad(int req, int nd) {
+ if (!nd) {
+
+ while (req) {
+ if (req > sizeof(ffbuf)) {
+ full_write(inode_buffer + ino_ofs, ffbuf, sizeof(ffbuf), nd);
+ req -= sizeof(ffbuf);
+ } else {
+ full_write(inode_buffer + ino_ofs, ffbuf, req, nd);
+ req = 0;
+ }
+ }
+ }
+ else {
+ while (req) {
+ if (req > sizeof(ffbuf)) {
+ full_write(dirent_buffer + dent_ofs, ffbuf, sizeof(ffbuf), nd);
+ req -= sizeof(ffbuf);
+ }
+ else {
+ full_write(dirent_buffer + dent_ofs, ffbuf, req, nd);
+ req = 0;
+ }
+ }
+ }
+}
+
+static inline void padword(int nd) {
+
+ if (!nd){
+ if (ino_ofs % 4) {
+ full_write(inode_buffer + ino_ofs, ffbuf, 4 - (ino_ofs % 4), nd);
+ }
+ }
+ else {
+ if (dent_ofs % 4) {
+ full_write(dirent_buffer + dent_ofs, ffbuf, 4 - (dent_ofs % 4), nd);
+ }
+ }
+}
+
+static inline void pad_block_if_less_than(int req, int nd) {
+ if (!nd) {
+ int datasize = ((sum_records + 1) * sizeof(struct jffs2_inode_sum_record)) + sizeof(struct jffs2_inode_sum_node) + 8;
+ datasize += (4 - (datasize % 4)) % 4;
+ if (ino_ofs + req > erase_block_size - datasize) {
+ dump_sum_records();
+ write_buff_to_file(nd);
+ }
+
+ if (add_cleanmarkers && found_cleanmarkers) {
+ if (!ino_ofs) {
+ full_write(inode_buffer, &cleanmarker, sizeof(cleanmarker), nd);
+ pad(cleanmarker_size - sizeof(cleanmarker), nd);
+ padword(nd);
+ }
+ }
+
+ }
+ else {
+ if (dent_ofs + req > erase_block_size) {
+ pad(erase_block_size - dent_ofs, nd);
+ write_buff_to_file(nd);
+ }
+
+ if (add_cleanmarkers && found_cleanmarkers) {
+ if (!dent_ofs) {
+ full_write(dirent_buffer, &cleanmarker, sizeof(cleanmarker), nd);
+ pad(cleanmarker_size - sizeof(cleanmarker), nd);
+ padword(nd);
+ }
+ }
+ }
+}
+
+void flush_buffers() {
+
+ if ((add_cleanmarkers == 1) && (found_cleanmarkers == 1)) { /* CLEANMARKER */
+ if (ino_ofs != cleanmarker_size) { /* INODE BUFFER */
+
+ int datasize = ((sum_records + 1) * sizeof(struct jffs2_inode_sum_record)) + sizeof(struct jffs2_inode_sum_node) + 8;
+ datasize += (4 - (datasize % 4)) % 4;
+
+ /* If we have a full inode buffer, then write out inode and summary data */
+ if (ino_ofs + sizeof(struct jffs2_raw_inode) + JFFS2_MIN_DATA_LEN > erase_block_size - datasize) {
+ dump_sum_records();
+ write_buff_to_file(SINODE);
+ }
+ /* else just write out inode data */
+ else{
+ pad(erase_block_size - ino_ofs, SINODE);
+ write_buff_to_file(SINODE);
+ }
+ }
+
+ if (dent_ofs != cleanmarker_size) { /* DIRENT AND OTHERS BUFFER */
+ pad(erase_block_size - dent_ofs, SONODE);
+ write_buff_to_file(SONODE);
+ }
+
+ }
+ else { /* NO CLEANMARKER */
+ if (ino_ofs != 0) { /* INODE BUFFER */
+
+ int datasize = ((sum_records + 1) * sizeof(struct jffs2_inode_sum_record)) + sizeof(struct jffs2_inode_sum_node) + 8;
+ datasize += (4 - (datasize % 4)) % 4;
+
+ /* If we have a full inode buffer, then write out inode and summary data */
+ if (ino_ofs + sizeof(struct jffs2_raw_inode) + JFFS2_MIN_DATA_LEN > erase_block_size - datasize) {
+ dump_sum_records();
+ write_buff_to_file(SINODE);
+ }
+ /* Else just write out inode data */
+ else{
+ pad(erase_block_size - ino_ofs, SINODE);
+ write_buff_to_file(SINODE);
+ }
+ }
+
+ if (dent_ofs != 0) { /* DIRENT AND OTHER BUFFER */
+ pad(erase_block_size - dent_ofs, SONODE);
+ write_buff_to_file(SONODE);
+ }
+ }
+}
+
+
+void write_dirent_to_buff(union jffs2_node_union *node) {
+
+ pad_block_if_less_than(je32_to_cpu (node->d.totlen), SONODE);
+ full_write(dirent_buffer + dent_ofs, &(node->d), je32_to_cpu (node->d.totlen), SONODE);
+ padword(SONODE);
+}
+
+void add_sum_entry(union jffs2_node_union *node) {
+
+ sum_storage *walk;
+ sum_storage *temp = (sum_storage *) malloc(sizeof(sum_storage));
+
+ if (!temp)
+ error_msg_and_die("Can't allocate memory for summary information!\n");
+
+ temp->inode = node->i.ino;
+ temp->version = node->i.version;
+ temp->offset = cpu_to_je32(ino_ofs);
+ temp->totlen = node->i.totlen;
+ temp->next = NULL;
+
+ if (!sum_collected) {
+ sum_collected = temp;
+ }
+ else {
+ walk = sum_collected;
+
+ while (walk->next) {
+ walk = walk->next;
+ }
+ walk->next = temp;
+ }
+ sum_records++;
+}
+
+void write_inode_to_buff(union jffs2_node_union *node) {
+
+ pad_block_if_less_than(je32_to_cpu (node->i.totlen), SINODE);
+ add_sum_entry(node); /* Add inode summary entry to summary list */
+ full_write(inode_buffer + ino_ofs, &(node->i), je32_to_cpu (node->i.totlen), SINODE); /* Write out the inode to inode_buffer */
+ padword(SINODE);
+
+}
+
+
+void create_summed_image(int inp_size) {
+ uint8_t *p = file_buffer;
+ union jffs2_node_union *node;
+ uint32_t crc;
+ uint16_t type;
+ int bitchbitmask = 0;
+ int obsolete;
+
+ char name[256];
+
+ while ( p < (file_buffer + inp_size)) {
+
+ node = (union jffs2_node_union*) p;
+
+ /* Skip empty space */
+ if (je16_to_cpu (node->u.magic) == 0xFFFF && je16_to_cpu (node->u.nodetype) == 0xFFFF) {
+ p += 4;
+ continue;
+ }
+
+ if (je16_to_cpu (node->u.magic) != JFFS2_MAGIC_BITMASK) {
+ if (!bitchbitmask++)
+ printf ("Wrong bitmask at 0x%08x, 0x%04x\n", p - file_buffer, je16_to_cpu (node->u.magic));
+ p += 4;
+ continue;
+ }
+
+ bitchbitmask = 0;
+
+ type = je16_to_cpu(node->u.nodetype);
+ if ((type & JFFS2_NODE_ACCURATE) != JFFS2_NODE_ACCURATE) {
+ obsolete = 1;
+ type |= JFFS2_NODE_ACCURATE;
+ } else
+ obsolete = 0;
+
+ node->u.nodetype = cpu_to_je16(type);
+
+ crc = crc32 (0, node, sizeof (struct jffs2_unknown_node) - 4);
+ if (crc != je32_to_cpu (node->u.hdr_crc)) {
+ printf ("Wrong hdr_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->u.hdr_crc), crc);
+ p += 4;
+ continue;
+ }
+
+ switch(je16_to_cpu(node->u.nodetype)) {
+
+ case JFFS2_NODETYPE_INODE:
+ if(verbose)
+ printf ("%8s Inode node at 0x%08x, totlen 0x%08x, #ino %5d, version %5d, isize %8d, csize %8d, dsize %8d, offset %8d\n",
+ obsolete ? "Obsolete" : "",
+ p - file_buffer, je32_to_cpu (node->i.totlen), je32_to_cpu (node->i.ino),
+ je32_to_cpu ( node->i.version), je32_to_cpu (node->i.isize),
+ je32_to_cpu (node->i.csize), je32_to_cpu (node->i.dsize), je32_to_cpu (node->i.offset));
+
+ crc = crc32 (0, node, sizeof (struct jffs2_raw_inode) - 8);
+ if (crc != je32_to_cpu (node->i.node_crc)) {
+ printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.node_crc), crc);
+ p += PAD(je32_to_cpu (node->i.totlen));
+ continue;
+ }
+
+ crc = crc32(0, p + sizeof (struct jffs2_raw_inode), je32_to_cpu(node->i.csize));
+ if (crc != je32_to_cpu(node->i.data_crc)) {
+ printf ("Wrong data_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.data_crc), crc);
+ p += PAD(je32_to_cpu (node->i.totlen));
+ continue;
+ }
+
+ write_inode_to_buff(node);
+
+ p += PAD(je32_to_cpu (node->i.totlen));
+ break;
+
+ case JFFS2_NODETYPE_DIRENT:
+ memcpy (name, node->d.name, node->d.nsize);
+ name [node->d.nsize] = 0x0;
+
+ if(verbose)
+ printf ("%8s Dirent node at 0x%08x, totlen 0x%08x, #pino %5d, version %5d, #ino %8d, nsize %8d, name %s\n",
+ obsolete ? "Obsolete" : "",
+ p - file_buffer, je32_to_cpu (node->d.totlen), je32_to_cpu (node->d.pino),
+ je32_to_cpu ( node->d.version), je32_to_cpu (node->d.ino),
+ node->d.nsize, name);
+
+ crc = crc32 (0, node, sizeof (struct jffs2_raw_dirent) - 8);
+ if (crc != je32_to_cpu (node->d.node_crc)) {
+ printf ("Wrong node_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.node_crc), crc);
+ p += PAD(je32_to_cpu (node->d.totlen));
+ continue;
+ }
+
+ crc = crc32(0, p + sizeof (struct jffs2_raw_dirent), node->d.nsize);
+ if (crc != je32_to_cpu(node->d.name_crc)) {
+ printf ("Wrong name_crc at 0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.name_crc), crc);
+ p += PAD(je32_to_cpu (node->d.totlen));
+ continue;
+ }
+
+ write_dirent_to_buff(node);
+
+ p += PAD(je32_to_cpu (node->d.totlen));
+ break;
+
+ case JFFS2_NODETYPE_CLEANMARKER:
+ if (verbose) {
+ printf ("%8s Cleanmarker at 0x%08x, totlen 0x%08x\n",
+ obsolete ? "Obsolete" : "",
+ p - file_buffer, je32_to_cpu (node->u.totlen));
+ }
+
+ if(!found_cleanmarkers){
+ found_cleanmarkers = 1;
+
+ if(add_cleanmarkers == 1 && use_input_cleanmarker_size == 1){
+ cleanmarker_size = je32_to_cpu (node->u.totlen);
+ setup_cleanmarker();
+ }
+ }
+
+ p += PAD(je32_to_cpu (node->u.totlen));
+ break;
+
+ case JFFS2_NODETYPE_PADDING:
+ if (verbose) {
+ printf ("%8s Padding node at 0x%08x, totlen 0x%08x\n",
+ obsolete ? "Obsolete" : "",
+ p - file_buffer, je32_to_cpu (node->u.totlen));
+ }
+ p += PAD(je32_to_cpu (node->u.totlen));
+ break;
+
+ case 0xffff:
+ p += 4;
+ break;
+
+ default:
+ if (verbose) {
+ printf ("%8s Unknown node at 0x%08x, totlen 0x%08x\n",
+ obsolete ? "Obsolete" : "",
+ p - file_buffer, je32_to_cpu (node->u.totlen));
+ }
+
+ write_dirent_to_buff(node);
+
+ p += PAD(je32_to_cpu (node->u.totlen));
+ }
+ }
+}
+
+int main(int argc, char **argv) {
+
+ int ret;
+
+ process_options(argc,argv);
+
+ if ((in_fd == -1) || (out_fd == -1)) {
+
+ if(in_fd != -1)
+ close(in_fd);
+ if(out_fd != -1)
+ close(out_fd);
+
+ error_msg_and_die("You must specify input and output files!\n");
+ }
+
+ init_buffers();
+
+ while ((ret = load_next_block())) {
+ create_summed_image(ret);
+ }
+
+ flush_buffers();
+ clean_buffers();
+
+ if (in_fd != -1)
+ close(in_fd);
+ if (out_fd != -1)
+ close(out_fd);
+
+ return 0;
+}
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-01-13 17:40 ` Kenneth Johansson
@ 2004-01-13 18:43 ` Jörn Engel
0 siblings, 0 replies; 113+ messages in thread
From: Jörn Engel @ 2004-01-13 18:43 UTC (permalink / raw)
To: Kenneth Johansson
Cc: David Woodhouse, Jarkko Lavinen (NMP/Helsinki), MTD List
On Tue, 13 January 2004 18:40:09 +0100, Kenneth Johansson wrote:
> >
> > But on nand it is not only possible, but also likely that flash size
> > is greater than ram size. Does your approach still work then?
>
> Obviously no. But maybe it's still cheaper to do the work in big chunks
> I don't know you need to actually do a test to see what happens. It's
> not so easy to see in advance what will happen. I thought for sure that
> it would be faster to only read what was needed but it was not, at least
> not on a 200 MHz ppc if you have an P4 3GHz the picture is quite
> different.
Could be, at least. As long as we don't know where the time is lost,
everything else remains guesswork.
Anyway, David's approach sounds quite sane to me.
Jörn
--
A victorious army first wins and then seeks battle.
-- Sun Tzu
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-01-13 17:29 ` Jörn Engel
@ 2004-01-13 17:40 ` Kenneth Johansson
2004-01-13 18:43 ` Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Kenneth Johansson @ 2004-01-13 17:40 UTC (permalink / raw)
To: Jörn Engel; +Cc: Jarkko Lavinen (NMP/Helsinki), David Woodhouse, MTD List
On Tue, 2004-01-13 at 18:29, Jörn Engel wrote:
> On Tue, 13 January 2004 17:21:15 +0100, Kenneth Johansson wrote:
> >
> > hmm I reread the firt post why is read so slow?? When I thought about
> > using nand I read that reading a byte took something like 50 ns. That is
> > like 20MB per second for an 8 bit device. This is in the region where it
> > did not pay to be smart,the accesses overhead to do read only what's
> > needed is greater than the time wasted reading uneccessary data.
>
> But on nand it is not only possible, but also likely that flash size
> is greater than ram size. Does your approach still work then?
>
> Jörn
Obviously no. But maybe it's still cheaper to do the work in big chunks
I don't know you need to actually do a test to see what happens. It's
not so easy to see in advance what will happen. I thought for sure that
it would be faster to only read what was needed but it was not, at least
not on a 200 MHz ppc if you have an P4 3GHz the picture is quite
different.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-01-13 16:21 ` Kenneth Johansson
@ 2004-01-13 17:29 ` Jörn Engel
2004-01-13 17:40 ` Kenneth Johansson
0 siblings, 1 reply; 113+ messages in thread
From: Jörn Engel @ 2004-01-13 17:29 UTC (permalink / raw)
To: Kenneth Johansson
Cc: Jarkko Lavinen (NMP/Helsinki), David Woodhouse, MTD List
On Tue, 13 January 2004 17:21:15 +0100, Kenneth Johansson wrote:
>
> hmm I reread the firt post why is read so slow?? When I thought about
> using nand I read that reading a byte took something like 50 ns. That is
> like 20MB per second for an 8 bit device. This is in the region where it
> did not pay to be smart,the accesses overhead to do read only what's
> needed is greater than the time wasted reading uneccessary data.
But on nand it is not only possible, but also likely that flash size
is greater than ram size. Does your approach still work then?
Jörn
--
To announce that there must be no criticism of the President, or that we
are to stand by the President, right or wrong, is not only unpatriotic
and servile, but is morally treasonable to the American public.
-- Theodore Roosevelt, Kansas City Star, 1918
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-01-13 15:30 ` David Woodhouse
@ 2004-01-13 16:21 ` Kenneth Johansson
2004-01-13 17:29 ` Jörn Engel
0 siblings, 1 reply; 113+ messages in thread
From: Kenneth Johansson @ 2004-01-13 16:21 UTC (permalink / raw)
To: David Woodhouse; +Cc: Jarkko Lavinen (NMP/Helsinki), MTD List
On Tue, 2004-01-13 at 16:30, David Woodhouse wrote:
> > When I tried to be smart and skip the copy and only read in what needed
> > to be read that was significantly slower.
>
> Bear in mind that the question was about NAND flash though -- it's not
> directly deferenceable, and you're not going to get the same results.
hmm I reread the firt post why is read so slow?? When I thought about
using nand I read that reading a byte took something like 50 ns. That is
like 20MB per second for an 8 bit device. This is in the region where it
did not pay to be smart,the accesses overhead to do read only what's
needed is greater than the time wasted reading uneccessary data.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-01-13 15:25 ` Kenneth Johansson
@ 2004-01-13 15:30 ` David Woodhouse
2004-01-13 16:21 ` Kenneth Johansson
0 siblings, 1 reply; 113+ messages in thread
From: David Woodhouse @ 2004-01-13 15:30 UTC (permalink / raw)
To: Kenneth Johansson; +Cc: MTD List, Jarkko Lavinen (NMP/Helsinki)
On Tue, 2004-01-13 at 16:25 +0100, Kenneth Johansson wrote:
> I have done some work on the JFFS2 code in u-boot and found that the
> fastest way was to load the entire FLASH into a continuous disk image in
> DRAM and then have code that directly dereference pointers into this
> image without any access functions or read on demand.
What we do in the current JFFS2 code is similar. We directly dereference
pointers into the _flash_ without access functions, rather than copying
it into DRAM first.
> When I tried to be smart and skip the copy and only read in what needed
> to be read that was significantly slower.
Bear in mind that the question was about NAND flash though -- it's not
directly deferenceable, and you're not going to get the same results.
--
dwmw2
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-01-13 13:39 ` David Woodhouse
@ 2004-01-13 15:25 ` Kenneth Johansson
2004-01-13 15:30 ` David Woodhouse
0 siblings, 1 reply; 113+ messages in thread
From: Kenneth Johansson @ 2004-01-13 15:25 UTC (permalink / raw)
To: David Woodhouse; +Cc: MTD List, Jarkko Lavinen (NMP/Helsinki)
On Tue, 2004-01-13 at 14:39, David Woodhouse wrote:
> On Tue, 2004-01-13 at 14:50 +0200, Jarkko Lavinen (NMP/Helsinki) wrote:
> > Is there anything that could be done to reduce the mount time of large
> > JFFS2 partitions? Lazy mounting with prioritized scanning?
>
> We improved it a lot by not checking CRC32 on all nodes until later --
> but we still need to read the whole medium at mount time.
>
> We _can_ fix that. There's a limited amount of knowledge we're gaining
> from our scan at mount time. For each inode on the medium, we need:
> Its nlink
> A list of the physical addresses of all its nodes
>
> We currently read the whole of every eraseblock to work this out.
> However, we could write a block 'tailer' at the _end_ of each eraseblock
> as we fill it, containing enough information in compressed form about
> the contents of the eraseblock.
>
> That would mean that we only have to read a small proportion of each
> properly-finished eraseblock, and should speed up the mount
> significantly.
I have done some work on the JFFS2 code in u-boot and found that the
fastest way was to load the entire FLASH into a continuous disk image in
DRAM and then have code that directly dereference pointers into this
image without any access functions or read on demand.
When I tried to be smart and skip the copy and only read in what needed
to be read that was significantly slower.
Never underestimate brute force :)
If you have very very poor bandwidth from the flash the the reverse is
true but I had to slow down my flash 10 times to break even.
^ permalink raw reply [flat|nested] 113+ messages in thread
* Re: JFFS2 mount time
2004-01-13 12:50 Jarkko Lavinen (NMP/Helsinki)
2004-01-13 13:19 ` Joakim Tjernlund
@ 2004-01-13 13:39 ` David Woodhouse
2004-01-13 15:25 ` Kenneth Johansson
2005-01-04 15:00 ` Steven Scholz
2 siblings, 1 reply; 113+ messages in thread
From: David Woodhouse @ 2004-01-13 13:39 UTC (permalink / raw)
To: Jarkko Lavinen (NMP/Helsinki); +Cc: MTD List
On Tue, 2004-01-13 at 14:50 +0200, Jarkko Lavinen (NMP/Helsinki) wrote:
> Is there anything that could be done to reduce the mount time of large
> JFFS2 partitions? Lazy mounting with prioritized scanning?
We improved it a lot by not checking CRC32 on all nodes until later --
but we still need to read the whole medium at mount time.
We _can_ fix that. There's a limited amount of knowledge we're gaining
from our scan at mount time. For each inode on the medium, we need:
Its nlink
A list of the physical addresses of all its nodes
We currently read the whole of every eraseblock to work this out.
However, we could write a block 'tailer' at the _end_ of each eraseblock
as we fill it, containing enough information in compressed form about
the contents of the eraseblock.
That would mean that we only have to read a small proportion of each
properly-finished eraseblock, and should speed up the mount
significantly.
--
dwmw2
^ permalink raw reply [flat|nested] 113+ messages in thread
* RE: JFFS2 mount time
2004-01-13 12:50 Jarkko Lavinen (NMP/Helsinki)
@ 2004-01-13 13:19 ` Joakim Tjernlund
2004-01-13 13:39 ` David Woodhouse
2005-01-04 15:00 ` Steven Scholz
2 siblings, 0 replies; 113+ messages in thread
From: Joakim Tjernlund @ 2004-01-13 13:19 UTC (permalink / raw)
To: 'Jarkko Lavinen (NMP/Helsinki)', 'MTD List'
> I am working with an embedded device with 64MB nand chip. I am able to
> read from chip at 6MB/s and able to read the whole chip in 10 seconds
> using character mtd device. The device has Omap1610 running at 190MHz,
> kernel 2.4.21.
>
> With 80% file system usage the mount time is 8 seconds which is
> unacceptable for root file system on a handheld device. I assume the
> scan time is linear with respect to the size of the flash partition
> and the file system usage.
>
> Is there anything that could be done to reduce the mount time of large
> JFFS2 partitions? Lazy mounting with prioritized scanning?
Yes, it is already done in JFFS2 CVS. Mount time for me is about a second on
a 64MB JFFS2 NOR flash.
Jocke
^ permalink raw reply [flat|nested] 113+ messages in thread
* JFFS2 mount time
@ 2004-01-13 12:50 Jarkko Lavinen (NMP/Helsinki)
2004-01-13 13:19 ` Joakim Tjernlund
` (2 more replies)
0 siblings, 3 replies; 113+ messages in thread
From: Jarkko Lavinen (NMP/Helsinki) @ 2004-01-13 12:50 UTC (permalink / raw)
To: MTD List
I am working with an embedded device with 64MB nand chip. I am able to
read from chip at 6MB/s and able to read the whole chip in 10 seconds
using character mtd device. The device has Omap1610 running at 190MHz,
kernel 2.4.21.
With 80% file system usage the mount time is 8 seconds which is
unacceptable for root file system on a handheld device. I assume the
scan time is linear with respect to the size of the flash partition
and the file system usage.
Is there anything that could be done to reduce the mount time of large
JFFS2 partitions? Lazy mounting with prioritized scanning?
Jarkko Lavinen
^ permalink raw reply [flat|nested] 113+ messages in thread
end of thread, other threads:[~2005-01-11 8:34 UTC | newest]
Thread overview: 113+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2004-12-15 23:19 JFFS2 mount time Gareth Bult (Encryptec)
2004-12-16 0:15 ` Josh Boyer
2004-12-16 1:02 ` Gareth Bult (Encryptec)
2004-12-16 12:53 ` Josh Boyer
2004-12-16 21:22 ` Gareth Bult (Encryptec)
2004-12-16 21:28 ` Josh Boyer
2004-12-16 21:47 ` Gareth Bult (Encryptec)
2004-12-17 12:54 ` Josh Boyer
2004-12-17 15:33 ` Gareth Bult (Encryptec)
2004-12-17 16:02 ` Josh Boyer
2004-12-17 16:46 ` Gareth Bult (Encryptec)
2004-12-17 17:08 ` Artem B. Bityuckiy
2004-12-17 17:10 ` Josh Boyer
2004-12-17 17:26 ` Gareth Bult (Encryptec)
2004-12-17 17:35 ` Josh Boyer
2004-12-17 18:09 ` Gareth Bult (Encryptec)
2004-12-17 19:14 ` jasmine
2004-12-17 20:55 ` Gareth Bult (Encryptec)
2004-12-18 16:02 ` Jörn Engel
2004-12-20 16:34 ` Josh Boyer
2004-12-20 17:12 ` Gareth Bult (Encryptec)
2004-12-21 13:09 ` Jörn Engel
2004-12-21 13:24 ` Gareth Bult (Encryptec)
2004-12-21 13:34 ` Jörn Engel
2004-12-18 16:19 ` Jörn Engel
2004-12-18 17:32 ` Gareth Bult (Encryptec)
2004-12-18 17:52 ` Jörn Engel
2004-12-18 18:11 ` Jörn Engel
2004-12-18 20:48 ` Gareth Bult (Encryptec)
2004-12-19 2:44 ` Jörn Engel
2004-12-21 13:30 ` Jörn Engel
2004-12-21 13:39 ` [PATCH 1/22] Add drivers/mtd/devices/blockmtd.c Jörn Engel
2004-12-21 13:41 ` [PATCH 2/22] Add copyrights Jörn Engel
2004-12-21 13:42 ` [PATCH 3/22] Remove read-only option Jörn Engel
2004-12-21 13:44 ` [PATCH 4/22] Change init/exit functions Jörn Engel
2004-12-21 13:45 ` [PATCH 5/22] Remove gcc warnings Jörn Engel
2004-12-21 13:47 ` [PATCH 6/22] Remove debug macros Jörn Engel
2004-12-21 13:48 ` [PATCH 7/22] Lindent Jörn Engel
2004-12-21 13:49 ` [PATCH 8/22] Remove sync interface Jörn Engel
2004-12-21 13:51 ` [PATCH 9/22] Change parameter interface to phram-style Jörn Engel
2004-12-21 13:53 ` [PATCH 10/22] Cleanup macro usage Jörn Engel
2004-12-21 13:54 ` [PATCH 11/22] kfree simplifications Jörn Engel
2004-12-21 13:55 ` [PATCH 12/22] change blockmtd_sync Jörn Engel
2004-12-21 13:57 ` [PATCH 13/22] remove erase regions Jörn Engel
2004-12-21 14:01 ` [PATCH 14/22] Change add_device Jörn Engel
2004-12-21 14:02 ` [PATCH 15/22] Rename unreadable mutex Jörn Engel
2004-12-21 14:03 ` [PATCH 16/22] list changes Jörn Engel
2004-12-21 14:04 ` [PATCH 17/22] Rename central struct Jörn Engel
2004-12-21 14:06 ` [PATCH 18/22] Function renaming Jörn Engel
2004-12-21 14:09 ` [PATCH 19/22] Fold various erase functions Jörn Engel
2004-12-21 14:10 ` [PATCH 20/22] Fold various write functions Jörn Engel
2004-12-21 14:11 ` [PATCH 21/22] Default erase size Jörn Engel
2004-12-21 14:13 ` [PATCH 22/22] Readahead Jörn Engel
2004-12-21 18:42 ` [PATCH 13/22] remove erase regions Christopher Hoover
2004-12-21 18:49 ` Jörn Engel
2004-12-21 21:09 ` Christopher Hoover
2004-12-22 2:47 ` Eric W. Biederman
2004-12-22 8:59 ` Jörn Engel
2004-12-22 10:05 ` Eric W. Biederman
2004-12-22 10:41 ` Jörn Engel
2004-12-21 13:42 ` [PATCH 1/22] Add drivers/mtd/devices/blockmtd.c Gareth Bult (Encryptec)
2004-12-21 14:15 ` Jörn Engel
2004-12-21 13:40 ` JFFS2 mount time Gareth Bult (Encryptec)
2004-12-21 15:00 ` David Woodhouse
[not found] ` <1103644945.10792.175.camel@squizzey.bult.co.uk>
2004-12-21 16:04 ` Jörn Engel
2004-12-16 13:43 ` Ferenc Havasi
2004-12-20 16:01 ` Gareth Bult (Encryptec)
2004-12-20 16:09 ` Ferenc Havasi
2004-12-20 16:39 ` Gareth Bult (Encryptec)
2004-12-20 17:48 ` Gareth Bult (Encryptec)
-- strict thread matches above, loose matches on Subject: below --
2004-10-20 14:26 Ferenc Havasi
2004-10-20 15:26 ` [OBORONA-SPAM] " Artem B. Bityuckiy
2004-10-20 15:49 ` Ferenc Havasi
2004-10-20 15:53 ` Artem B. Bityuckiy
2004-10-21 6:29 ` Artem B. Bityuckiy
2004-10-21 6:54 ` Ferenc Havasi
2004-10-21 7:16 ` Artem B. Bityuckiy
2004-10-21 19:50 ` Ferenc Havasi
2004-10-21 12:49 ` Jarkko Lavinen
2004-10-21 19:11 ` Ferenc Havasi
2004-10-22 9:58 ` Ferenc Havasi
2004-10-21 13:24 ` David Woodhouse
2004-10-21 20:05 ` Ferenc Havasi
2004-10-22 12:44 ` Artem Bityuckiy
2004-10-25 9:36 ` Ferenc Havasi
2004-10-25 10:56 ` Artem Bityuckiy
2004-10-25 15:30 ` Ferenc Havasi
2004-10-26 9:59 ` Artem Bityuckiy
2004-10-26 10:21 ` Ferenc Havasi
2004-10-26 11:05 ` Artem Bityuckiy
2004-10-26 13:52 ` Ferenc Havasi
2004-10-25 11:21 ` Artem Bityuckiy
2004-10-26 9:29 ` Jarkko Lavinen
2004-10-26 10:24 ` Ferenc Havasi
2004-10-26 10:34 ` Artem Bityuckiy
2004-01-13 12:50 Jarkko Lavinen (NMP/Helsinki)
2004-01-13 13:19 ` Joakim Tjernlund
2004-01-13 13:39 ` David Woodhouse
2004-01-13 15:25 ` Kenneth Johansson
2004-01-13 15:30 ` David Woodhouse
2004-01-13 16:21 ` Kenneth Johansson
2004-01-13 17:29 ` Jörn Engel
2004-01-13 17:40 ` Kenneth Johansson
2004-01-13 18:43 ` Jörn Engel
2005-01-04 15:00 ` Steven Scholz
2005-01-05 10:45 ` Artem B. Bityuckiy
2005-01-05 11:10 ` Ferenc Havasi
2005-01-05 11:24 ` Steven Scholz
2005-01-05 11:45 ` Artem B. Bityuckiy
2005-01-06 10:50 ` Ferenc Havasi
2005-01-10 15:14 ` Steven Scholz
2005-01-10 15:25 ` Steven Scholz
2005-01-11 8:26 ` Ferenc Havasi
2005-01-11 8:34 ` Steven Scholz
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.