JFFS2 mount time

JFFS2 mount time

[Ferenc Havasi]

[-- Attachment #1: Type: text/plain, Size: 1792 bytes --]

Dear All,

Here is the latest version of our mount time improvement.

Using of it:
- apply this patch on the latest version of MTD
- compile sumtool (make command in mtd/util)
- make your JFFS2 image as before (or you can use already created images 
as well)
- run sumtool to insert summary information, for example:
   ./sumtool -i original.jffs2 -o new.jffs2 -e128KiB
- recompile your kernel with "JFFS2 inode summary support"

Jarkko made a measurement on a real NAND device: his JFFS2 image was 
120819928 (115M), after running sumtool the new image was 123338752 (117M).

Using the original mount time was 55 sec, with the new image it is only 
8.5 sec.

It works very similar as our previous improvement: stores special 
information at the end of the erase blocks, and at mount time if there 
is this kind of information the scaning of the erase block is unneccessary.

New things compared to our previous improvement:
- it was fully rewritten
- we separated the user space tool from mkfs. (sumtool)
- sumtool now not only inserts the summary information but also make 
some node-reordering. There will be two kind of erase blocks: in the 
"first type" there will be only jffs2_raw_inodes, and all other node 
(jffs2_raw_dirent) will be stored in the "second type". It generates 
summary at the end of all "fist type" eraseblock. (the "second type" 
will be scanned as before, because all information is needed in 
jffs_raw_dirent at mount time)

Ceratinly all of these things are optional (as you can see above you 
have to select it from kernel config). The JFFS2 image produced by 
sumtool is also usable with previous kernel because the summary node is 
JFFS2_FEATURE_RWCOMPAT_DELETE.

I think it can be usefull not only for us. David, may I commit it to the 
CVS?

Regards,
Ferenc

[-- Attachment #2: jffs2-summary.patch --]
[-- Type: text/x-patch, Size: 36792 bytes --]

diff --unified --recursive --new-file mtd2/fs/Kconfig mtd/fs/Kconfig
--- mtd2/fs/Kconfig	2004-07-16 17:20:59.000000000 +0200
+++ mtd/fs/Kconfig	2004-10-20 15:10:42.000000000 +0200
@@ -68,6 +68,19 @@
 	  Say 'N' unless you have NAND flash and you are willing to test and
 	  develop JFFS2 support for it.
 
+config JFFS2_FS_SUMMARY
+	bool "JFFS2 inode summary support (EXPERIMENTAL)" 
+	depends on JFFS2_FS
+	default n
+        help
+          This feature makes it possible to use inode summary information
+          for faster filesystem mount - specially on NAND.
+
+          The summary information can be inserted into a filesystem image 
+          by the utility 'sumtool'.
+
+	  If unsure, say 'N'.
+          
 config JFFS2_COMPRESSION_OPTIONS
 	bool "Advanced compression options for JFFS2"
 	default n
diff --unified --recursive --new-file mtd2/fs/jffs2/scan.c mtd/fs/jffs2/scan.c
--- mtd2/fs/jffs2/scan.c	2004-09-12 11:56:13.000000000 +0200
+++ mtd/fs/jffs2/scan.c	2004-10-20 15:44:25.000000000 +0200
@@ -4,6 +4,8 @@
  * Copyright (C) 2001-2003 Red Hat, Inc.
  *
  * Created by David Woodhouse <dwmw2@redhat.com>
+ * Inode summary support by Zoltan Sogor, Ferenc Havasi, Patrik Kluba
+ *                          University of Szeged, Hungary
  *
  * For licensing information, see the file 'LICENCE' in this directory.
  *
@@ -58,6 +60,11 @@
 static int jffs2_scan_dirent_node(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb,
 				 struct jffs2_raw_dirent *rd, uint32_t ofs);
 
+
+#ifdef CONFIG_JFFS2_FS_SUMMARY
+static struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino);
+#endif
+
 #define BLK_STATE_ALLFF		0
 #define BLK_STATE_CLEAN		1
 #define BLK_STATE_PARTDIRTY	2
@@ -292,6 +299,25 @@
 #ifdef CONFIG_JFFS2_FS_NAND
 	int cleanmarkerfound = 0;
 #endif
+#ifdef CONFIG_JFFS2_FS_SUMMARY
+	struct jffs2_raw_node_ref *raw;
+	struct jffs2_raw_node_ref *cache_ref;
+	struct jffs2_inode_cache *ic;
+		
+	typedef struct sum_marker {
+		jint32_t offset;
+		jint32_t magic;
+	} sum_marker;
+	
+	sum_marker *sm;	
+	int i;
+	int sumsize;
+	uint32_t ino;
+	uint32_t crc;
+	struct jffs2_inode_sum_node *summary;
+	struct jffs2_inode_sum_record *sum_rec;
+	int bad_sum = 0;
+#endif
 
 	ofs = jeb->offset;
 	prevofs = jeb->offset - 1;
@@ -314,10 +340,217 @@
 		}
 	}
 #endif
+
+#ifdef CONFIG_JFFS2_FS_SUMMARY
+	/* Looking for summary marker */
+	sm = (sum_marker *)kmalloc(sizeof(*sm), GFP_KERNEL);
+	if (!sm) {
+	        return -ENOMEM;
+	}
+	
+	err = jffs2_fill_scan_buf(c, (unsigned char *) sm, jeb->offset + c->sector_size - 8, 8);
+	
+	if (err) {
+	        return err;
+	}
+	
+	if (je32_to_cpu(sm->magic) == JFFS2_SUM_MAGIC) {
+		ofs = je32_to_cpu(sm->offset);
+		sumsize = c->sector_size - ofs;
+		ofs += jeb->offset;
+		
+		D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Inode summary information found at 0x%x (%d bytes)\n", ofs, sumsize));
+		
+		summary = (struct jffs2_inode_sum_node *) kmalloc(sumsize, GFP_KERNEL);
+			
+		if (!summary) {
+				kfree(sm);
+				return -ENOMEM;
+		}
+		
+		err = jffs2_fill_scan_buf(c, (unsigned char *)summary, ofs, sumsize);
+		
+		if (err) {
+				kfree(sm);
+				kfree(summary);
+				return err;
+		}
+  
+		/* OK, now check for node validity and CRC */
+		crcnode.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
+		crcnode.nodetype = cpu_to_je16(JFFS2_NODETYPE_INODE_SUM);
+		crcnode.totlen = summary->totlen;
+		hdr_crc = crc32(0, &crcnode, sizeof(crcnode)-4);
+		
+		if (je32_to_cpu(summary->hdr_crc) != hdr_crc) {
+				D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node header is corrupt (bad CRC or no summary at all)\n"));
+				bad_sum = 1;
+		}
+		
+		if ((!bad_sum) && (je32_to_cpu(summary->totlen) != sumsize)) {
+				D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (wrong erasesize?)\n"));
+				bad_sum = 1;
+		}
+		
+		crc = crc32(0, summary, sizeof(struct jffs2_inode_sum_node)-8);
+			
+		if ((!bad_sum) && (je32_to_cpu(summary->node_crc) != crc)) {
+				D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (bad CRC)\n"));
+				bad_sum = 1;
+		}
+		
+		sum_rec = (struct jffs2_inode_sum_record *) &(summary->sum[0]);
+		crc = crc32(0, sum_rec, sumsize - sizeof(struct jffs2_inode_sum_node));
+  
+		if ((!bad_sum) && (je32_to_cpu(summary->sum_crc) != crc)) {
+				D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node data is corrupt (bad CRC)\n"));
+				bad_sum = 1;
+		}
+		
+		if (!bad_sum) {
+			
+			if ( je32_to_cpu(summary->cln_mkr) ){
+				
+				D1(printk(KERN_DEBUG "Summary : CLEANMARKER node \n"));
+				
+				if (je32_to_cpu(summary->cln_mkr) != c->cleanmarker_size) {
+					printk(KERN_DEBUG "CLEANMARKER node has totlen 0x%x != normal 0x%x\n", 
+					   je32_to_cpu(summary->cln_mkr), c->cleanmarker_size);
+					UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) );
+				} 
+				else if (jeb->first_node) {
+					printk(KERN_DEBUG "CLEANMARKER node not first node in block (0x%08x)\n", jeb->offset);
+					UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) );
+				} 
+				else {
+					struct jffs2_raw_node_ref *marker_ref = jffs2_alloc_raw_node_ref();
+						
+					if (!marker_ref) {
+						printk(KERN_NOTICE "Failed to allocate node ref for clean marker\n");
+						return -ENOMEM;
+					}
+					
+					marker_ref->next_in_ino = NULL;
+					marker_ref->next_phys = NULL;
+					marker_ref->flash_offset = jeb->offset | REF_NORMAL;
+					marker_ref->__totlen = je32_to_cpu(summary->cln_mkr);
+					jeb->first_node = jeb->last_node = marker_ref;
+				
+					USED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) );
+									
+				}
+			}
+			
+			for(i = 0; i < je16_to_cpu(summary->sum_num); i++) {
+				
+				D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Processing summary information %d\n", i));
+				
+				//JFFS2_NODETYPE_INODE:
+				ino = je32_to_cpu(sum_rec->inode);
+				D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Inode at 0x%08x\n", jeb->offset + je32_to_cpu(sum_rec->offset)));
+				raw = jffs2_alloc_raw_node_ref();
+				if (!raw) {
+						printk(KERN_NOTICE "jffs2_scan_eraseblock(): allocation of node reference failed\n");
+						kfree(sm);
+						kfree(summary);
+						return -ENOMEM;
+				}
+
+				ic = jffs2_get_ino_cache(c, ino);
+				if (!ic) {
+						ic = jffs2_scan_make_ino_cache(c, ino);
+						if (!ic) {
+								printk(KERN_NOTICE "jffs2_scan_eraseblock(): scan_make_ino_cache failed\n");
+								jffs2_free_raw_node_ref(raw);
+							kfree(sm);
+							kfree(summary);
+							return -ENOMEM;
+						}
+				}
+
+				raw->flash_offset = (jeb->offset + je32_to_cpu(sum_rec->offset)) | REF_UNCHECKED;
+				raw->__totlen = PAD(je32_to_cpu(sum_rec->totlen));
+				raw->next_phys = NULL;
+				raw->next_in_ino = ic->nodes;
+
+				ic->nodes = raw;
+				if (!jeb->first_node)
+						jeb->first_node = raw;
+				if (jeb->last_node)
+						jeb->last_node->next_phys = raw;
+				jeb->last_node = raw;
+
+				/* do we need this? this requires storing another 4 bytes per record in the cache or an expensive reading */
+				pseudo_random += je32_to_cpu(sum_rec->version);
+
+				UNCHECKED_SPACE(PAD(je32_to_cpu(sum_rec->totlen)));
+				
+				sum_rec++;
+			}
+			
+			kfree(sm);
+			kfree(summary);
+
+			/* for ACCT_PARANOIA_CHECK */
+			cache_ref = jffs2_alloc_raw_node_ref();
+			
+			if (!cache_ref) {
+				printk(KERN_NOTICE "Failed to allocate node ref for cache\n");
+				return -ENOMEM;
+			}
+			
+			cache_ref->next_in_ino = NULL;
+			cache_ref->next_phys = NULL;
+			cache_ref->flash_offset = ofs | REF_NORMAL;
+			cache_ref->__totlen = sumsize;
+			
+			if (!jeb->first_node)
+				jeb->first_node = cache_ref;
+			if (jeb->last_node)
+				jeb->last_node->next_phys = cache_ref;
+			jeb->last_node = cache_ref;
+			
+			USED_SPACE(sumsize);
+			
+			/* somebody check this and all of space accounting in summary support */
+	
+			if ((jeb->used_size + jeb->unchecked_size) == PAD(c->cleanmarker_size) && !jeb->dirty_size 
+				&& (!jeb->first_node || !jeb->first_node->next_in_ino) ) { 
+					return BLK_STATE_CLEANMARKER; 
+				}		
+			/* move blocks with max 4 byte dirty space to cleanlist */	
+			else if (!ISDIRTY(c->sector_size - (jeb->used_size + jeb->unchecked_size))) {
+				c->dirty_size -= jeb->dirty_size;
+				c->wasted_size += jeb->dirty_size; 
+				jeb->wasted_size += jeb->dirty_size;
+				jeb->dirty_size = 0;
+				return BLK_STATE_CLEAN;
+			}
+			else if (jeb->used_size || jeb->unchecked_size) { 
+					return BLK_STATE_PARTDIRTY; 
+			}
+			else { 
+					return BLK_STATE_ALLDIRTY; 
+			}
+		}   
+	}
+	D1(printk(KERN_DEBUG "Summary end\n"));
+	
+	ofs = jeb->offset;
+	prevofs = jeb->offset - 1;
+
+#endif
+
 	buf_ofs = jeb->offset;
 
 	if (!buf_size) {
 		buf_len = c->sector_size;
+#ifdef CONFIG_JFFS2_FS_SUMMARY
+		/* must reread because of summary test */
+		err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len);
+		if (err)
+			return err;
+#endif
 	} else {
 		buf_len = EMPTY_SCAN_SIZE;
 		err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len);
diff --unified --recursive --new-file mtd2/include/linux/jffs2.h mtd/include/linux/jffs2.h
--- mtd2/include/linux/jffs2.h	2004-05-25 13:31:55.000000000 +0200
+++ mtd/include/linux/jffs2.h	2004-10-20 14:53:52.000000000 +0200
@@ -28,6 +28,9 @@
 #define JFFS2_EMPTY_BITMASK 0xffff
 #define JFFS2_DIRTY_BITMASK 0x0000
 
+/* Summary node MAGIC marker */
+#define JFFS2_SUM_MAGIC	0x02851885 
+
 /* We only allow a single char for length, and 0xFF is empty flash so
    we don't want it confused with a real length. Hence max 254.
 */
@@ -61,6 +64,7 @@
 #define JFFS2_NODETYPE_INODE (JFFS2_FEATURE_INCOMPAT | JFFS2_NODE_ACCURATE | 2)
 #define JFFS2_NODETYPE_CLEANMARKER (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3)
 #define JFFS2_NODETYPE_PADDING (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 4)
+#define JFFS2_NODETYPE_INODE_SUM (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 6)
 
 // Maybe later...
 //#define JFFS2_NODETYPE_CHECKPOINT (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3)
@@ -148,10 +152,31 @@
 	uint8_t data[0];
 } __attribute__((packed));
 
+struct jffs2_inode_sum_node{
+    jint16_t magic;
+	jint16_t nodetype; /* = JFFS2_NODETYPE_INODE_SUM */
+	jint32_t totlen;
+	jint32_t hdr_crc;
+	jint16_t sum_num;	/* number of sum entries*/
+	jint32_t cln_mkr;	/* clean marker size, 0 = no cleanmarker */
+	jint32_t sum_crc;	/* summary information crc */
+	jint32_t node_crc; 	/* node crc */
+	jint32_t sum[0]; 	/* inode summary info */
+} __attribute__((packed));
+
+struct jffs2_inode_sum_record{
+	jint32_t inode;
+	jint32_t version;
+	jint32_t offset;
+	jint32_t totlen; 
+} __attribute__((packed));
+
+
 union jffs2_node_union {
 	struct jffs2_raw_inode i;
 	struct jffs2_raw_dirent d;
 	struct jffs2_unknown_node u;
+	struct jffs2_inode_sum_node s;
 };
 
 #endif /* __LINUX_JFFS2_H__ */
diff --unified --recursive --new-file mtd2/util/Makefile mtd/util/Makefile
--- mtd2/util/Makefile	2004-07-13 19:49:43.000000000 +0200
+++ mtd/util/Makefile	2004-10-19 15:11:02.000000000 +0200
@@ -13,7 +13,7 @@
 TARGETS = ftl_format flash_erase flash_eraseall nanddump doc_loadbios \
 	mkfs.jffs ftl_check mkfs.jffs2 flash_lock flash_unlock \
 	flash_info mtd_debug flashcp nandwrite jffs2dump \
-	nftldump nftl_format docfdisk #jffs2reader
+	nftldump nftl_format docfdisk sumtool #jffs2reader
 
 SYMLINKS = compr_lzari.c compr_lzo.c
 
@@ -48,6 +48,9 @@
 jffs2dump: jffs2dump.o crc32.o
 	$(CC) $(LDFLAGS) -o $@ $^
 
+sumtool: sumtool.o crc32.o
+	$(CC) $(LDFLAGS) -o $@ $^
+
 install: ${TARGETS}
 	mkdir -p ${DESTDIR}/${SBINDIR}
 	install -m0755 -oroot -groot ${TARGETS} ${DESTDIR}/${SBINDIR}/
diff --unified --recursive --new-file mtd2/util/jffs2dump.c mtd/util/jffs2dump.c
--- mtd2/util/jffs2dump.c	2004-06-19 00:11:48.000000000 +0200
+++ mtd/util/jffs2dump.c	2004-10-20 14:55:28.000000000 +0200
@@ -3,7 +3,7 @@
  *
  *  Copyright (C) 2003 Thomas Gleixner (tglx@linutronix.de)
  *
- * $Id: jffs2dump.c,v 1.6 2004/06/18 22:11:48 gleixner Exp $
+ * $Id: jffs2dump.c,v 1.1 2004/10/19 07:19:55 weth Exp $
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License version 2 as
@@ -277,7 +277,63 @@
 
 			p += PAD(je32_to_cpu (node->d.totlen));						
 			break;
-	
+
+		case JFFS2_NODETYPE_INODE_SUM:{
+			
+			int i;
+			jint32_t *offset,*magic;
+			
+			printf ("%8s Inode Sum  node at 0x%08x, totlen 0x%08x, sum_num  %5d, cleanmarker size %5d\n",
+					obsolete ? "Obsolete" : "",
+					p - data,
+					je32_to_cpu (node->s.totlen),
+					je16_to_cpu (node->s.sum_num),
+					je32_to_cpu (node->s.cln_mkr));
+
+			crc = crc32 (0, node, sizeof (struct jffs2_inode_sum_node) - 8);
+			if (crc != je32_to_cpu (node->s.node_crc)) {
+				printf ("Wrong node_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.node_crc), crc);
+				p += PAD(je32_to_cpu (node->s.totlen));
+				dirty += PAD(je32_to_cpu (node->s.totlen));;
+				continue;
+			}
+			
+			crc = crc32(0, p + sizeof (struct jffs2_inode_sum_node),  je32_to_cpu (node->s.totlen) - sizeof(struct jffs2_inode_sum_node));
+			if (crc != je32_to_cpu(node->s.sum_crc)) {
+				printf ("Wrong data_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.sum_crc), crc);
+				p += PAD(je32_to_cpu (node->s.totlen));
+				dirty += PAD(je32_to_cpu (node->s.totlen));;
+				continue;
+			}
+
+			if(verbose){
+				for(i = 0; i < je16_to_cpu (node->s.sum_num); i++){
+					struct jffs2_inode_sum_record *sp;
+					sp = (struct jffs2_inode_sum_record *) (p + sizeof (struct jffs2_inode_sum_node));
+						
+					printf ("%14s #ino  %5d,  version %5d, offset %8d, totlen 0x%08x\n",
+						"",
+						je32_to_cpu (sp[i].inode),
+						je32_to_cpu (sp[i].version),
+						je32_to_cpu (sp[i].offset), 
+						je32_to_cpu (sp[i].totlen));
+					
+				}
+				
+				offset = (jint32_t *)((char *)p + je32_to_cpu(node->s.totlen) - 8);
+				magic = (jint32_t *)((char *)p + je32_to_cpu(node->s.totlen) - 4);
+				
+				printf("%14s Sum Node Offset  0x%08x,  Magic 0x%08x\n",
+					"",
+					je32_to_cpu(*offset),
+					je32_to_cpu(*magic));
+			}
+			
+			p += PAD(je32_to_cpu (node->s.totlen));
+			break;
+			
+		}
+			
 		case JFFS2_NODETYPE_CLEANMARKER:
 			if (verbose) {
 				printf ("%8s Cleanmarker     at 0x%08x, totlen 0x%08x\n", 
@@ -418,9 +474,9 @@
 
 			write (fd, &newnode, sizeof (struct jffs2_raw_dirent));
 			write (fd, p + sizeof (struct jffs2_raw_dirent), PAD (je32_to_cpu (node->d.totlen) -  sizeof (struct jffs2_raw_dirent)));
-			p += PAD(je32_to_cpu (node->d.totlen));						
+			p += PAD(je32_to_cpu (node->d.totlen));
 			break;
-	
+	                        
 		case JFFS2_NODETYPE_CLEANMARKER:
 		case JFFS2_NODETYPE_PADDING:
 			newnode.u.magic = cnv_e16 (node->u.magic);
diff --unified --recursive --new-file mtd2/util/sumtool.c mtd/util/sumtool.c
--- mtd2/util/sumtool.c	1970-01-01 01:00:00.000000000 +0100
+++ mtd/util/sumtool.c	2004-10-20 11:56:08.000000000 +0200
@@ -0,0 +1,800 @@
+/*
+ *  sumtool.c
+ *
+ *  Copyright (C) 2004 Zoltan Sogor <weth@inf.u-szeged.hu>,
+ *                     Ferenc Havasi <havasi@inf.u-szeged.hu>,
+ *                     Patrik Kluba <pajko@halom.u-szeged.hu>,
+ *                     University of Szeged, Hungary
+ *
+ * $Id: sumtool.c,v 1.2 2004/10/20 09:56:08 hafy Exp $
+ * 
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+ *
+ * Overview:
+ *   This is a utility to reorder nodes and insert inode summary information
+ *   into JFFS2 image for faster mount time - specially on NAND.
+ *
+ */
+
+#include <errno.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <string.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <time.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/param.h>
+#include <asm/types.h>
+#include <dirent.h>
+#include <mtd/jffs2-user.h>
+#include <endian.h>
+#include <byteswap.h>
+#include <getopt.h>
+#include "crc32.h"
+
+#define PAD(x) (((x)+3)&~3)
+
+#define SINODE 0	/* Inode Type*/
+#define SONODE 1  	/* Other type*/
+
+static const char *const app_name = "sumtool";
+
+typedef struct sum_storage {
+	jint32_t inode;
+	jint32_t version;
+	jint32_t offset;
+	jint32_t totlen;
+	struct sum_storage *next;
+} sum_storage;
+
+static sum_storage *sum_collected = NULL;	/* summary info list */
+static int sum_records = 0;					/* number of sumary records */
+
+
+static int verbose = 0;
+static int add_cleanmarkers = 1;			/* add cleanmarker to output */
+static int use_input_cleanmarker_size = 1;	/* use input file's cleanmarker size (default) */
+static int found_cleanmarkers = 0;			/* cleanmarker found in input file */
+static struct jffs2_unknown_node cleanmarker;
+static int cleanmarker_size = sizeof(cleanmarker);
+static const char *short_options = "o:i:e:hvVblnc:";
+static int erase_block_size = 65536;
+static int target_endian = __BYTE_ORDER;
+static int out_fd = -1;
+static int in_fd = -1;
+
+static uint8_t *inode_buffer = NULL; 	/* buffer for inodes */
+static unsigned int ino_ofs = 0;	 	/* inode buffer offset */
+
+static uint8_t *dirent_buffer = NULL;	/* buffer for directory entries and other (symlink, spec. device files, etc.)*/ 
+static unsigned int dent_ofs = 0;		/* directory enrty buffer offset*/	
+
+static uint8_t *file_buffer = NULL;		/* file buffer contains the actual erase block*/
+static unsigned int file_ofs = 0;		/* position in the buffer */
+
+static struct option long_options[] = {
+	{"output", 1, NULL, 'o'},
+	{"input", 1, NULL, 'i'},
+	{"eraseblock", 1, NULL, 'e'},
+	{"help", 0, NULL, 'h'},
+	{"verbose", 0, NULL, 'v'},
+	{"version", 0, NULL, 'V'},
+	{"bigendian", 0, NULL, 'b'},
+	{"littleendian", 0, NULL, 'l'},	
+	{"no-cleanmarkers", 0, NULL, 'n'},
+	{"cleanmarker", 1, NULL, 'c'},
+	{NULL, 0, NULL, 0}
+};
+
+static char *helptext =
+	"Usage: sumtool [OPTIONS] -i inputfile -o outputfile\n"
+	"Convert the input JFFS2 file to a SUM-ed JFFS2 file\n\n"
+	"Options:\n"
+	"  -e, --eraseblock=SIZE     Use erase block size SIZE (default: 64KiB)\n"
+	"                            (usually 16KiB on NAND)\n"
+	"  -c, --cleanmarker=SIZE    Size of cleanmarker (default 12).\n"
+	"                            (usually 16 bytes on NAND, and will be set to\n"
+	"                            this value if left at the default 12). Will be\n"
+	"                            stored in OOB after each physical page composing\n"
+	"                            a physical erase block.\n"
+	"  -n, --no-cleanmarkers     Don't add a cleanmarker to every eraseblock\n"
+	"  -o, --output=FILE         Output to FILE \n"
+	"  -i, --input=FILE          Input from FILE \n"
+	"  -b, --bigendian	     Image is big endian\n"
+	"  -l  --littleendian        Image is little endian\n"
+	"  -h, --help                Display this help text\n"
+	"  -v, --verbose             Verbose operation\n"
+	"  -V, --version             Display version information\n\n";
+
+
+static char *revtext = "$Revision: 1.2 $";
+
+static unsigned char ffbuf[16] = {
+	0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
+	0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff
+};
+
+static void verror_msg(const char *s, va_list p) {
+	fflush(stdout);
+	fprintf(stderr, "%s: ", app_name);
+	vfprintf(stderr, s, p);
+}
+
+static void error_msg_and_die(const char *s, ...) {
+	va_list p;
+
+	va_start(p, s);
+	verror_msg(s, p);
+	va_end(p);
+	putc('\n', stderr);
+	exit(EXIT_FAILURE);
+}
+
+static void vperror_msg(const char *s, va_list p) {
+	int err = errno;
+
+	if (s == 0)
+		s = "";
+	verror_msg(s, p);
+	if (*s)
+		s = ": ";
+	fprintf(stderr, "%s%s\n", s, strerror(err));
+}
+
+static void perror_msg_and_die(const char *s, ...) {
+	va_list p;
+
+	va_start(p, s);
+	vperror_msg(s, p);
+	va_end(p);
+	exit(EXIT_FAILURE);
+}
+
+
+
+static void full_write(void *target_buff, const void *buf, int len, int nd);
+
+void setup_cleanmarker() {
+
+	cleanmarker.magic    = cpu_to_je16(JFFS2_MAGIC_BITMASK);
+	cleanmarker.nodetype = cpu_to_je16(JFFS2_NODETYPE_CLEANMARKER);
+	cleanmarker.totlen   = cpu_to_je32(cleanmarker_size);
+	cleanmarker.hdr_crc  = cpu_to_je32(crc32(0, &cleanmarker, sizeof(struct jffs2_unknown_node)-4));
+}
+
+void process_options (int argc, char **argv){
+	int opt,c;
+	
+	while ((opt = getopt_long(argc, argv, short_options, long_options, &c)) >= 0) 
+	{
+		switch (opt) 
+		{
+			case 'o':
+				if (out_fd != -1) {
+					error_msg_and_die("output filename specified more than once");
+				}
+				out_fd = open(optarg, O_CREAT | O_TRUNC | O_RDWR, 0644);
+				if (out_fd == -1) {
+					perror_msg_and_die("open output file");
+				}
+				break;
+				
+			case 'i':
+				if (in_fd != -1) {
+					error_msg_and_die("input filename specified more than once");
+				}
+				in_fd = open(optarg, O_RDONLY);
+				if (in_fd == -1) {
+					perror_msg_and_die("open input file");
+				}
+				break;
+			case 'b':
+				target_endian = __BIG_ENDIAN;
+				break;
+			case 'l':
+				target_endian = __LITTLE_ENDIAN;
+				break;	
+			case 'h':
+			case '?':
+				error_msg_and_die(helptext);
+	
+			case 'v':
+				verbose = 1;
+				break;
+	
+			case 'V':
+				error_msg_and_die("revision %.*s\n",
+						(int) strlen(revtext) - 13, revtext + 11);
+	
+			case 'e': {
+				char *next;
+				unsigned units = 0;
+				erase_block_size = strtol(optarg, &next, 0);
+				if (!erase_block_size)
+					error_msg_and_die("Unrecognisable erase size\n");
+	
+				if (*next) {
+					if (!strcmp(next, "KiB")) {
+						units = 1024;
+					} else if (!strcmp(next, "MiB")) {
+						units = 1024 * 1024;
+					} else {
+						error_msg_and_die("Unknown units in erasesize\n");
+					}
+				} else {
+					if (erase_block_size < 0x1000)
+						units = 1024;
+					else
+						units = 1;
+				}
+				erase_block_size *= units;
+	
+				/* If it's less than 8KiB, they're not allowed */
+				if (erase_block_size < 0x2000) {
+					fprintf(stderr, "Erase size 0x%x too small. Increasing to 8KiB minimum\n",
+						erase_block_size);
+					erase_block_size = 0x2000;
+				}
+				break;
+			}
+
+			case 'n':
+				add_cleanmarkers = 0;
+				break;
+			case 'c':
+				cleanmarker_size = strtol(optarg, NULL, 0);
+			
+				if (cleanmarker_size < sizeof(cleanmarker)) {
+					error_msg_and_die("cleanmarker size must be >= 12");
+				}
+				if (cleanmarker_size >= erase_block_size) {
+					error_msg_and_die("cleanmarker size must be < eraseblock size");
+				}
+				
+				use_input_cleanmarker_size = 0;
+				found_cleanmarkers = 1;
+				setup_cleanmarker();
+				
+				break;
+			
+		}
+	}
+}
+
+
+void init_buffers() {
+	
+	inode_buffer = malloc(erase_block_size);
+	
+	if (!inode_buffer) {
+		perror("out of memory");
+		close (in_fd);
+		close (out_fd);
+		exit(1);
+	}
+		
+	dirent_buffer = malloc(erase_block_size);
+	
+	if (!dirent_buffer) {
+		perror("out of memory");
+		close (in_fd);
+		close (out_fd);
+		exit(1);
+	}
+		
+	file_buffer = malloc(erase_block_size);
+	
+	if (!file_buffer) {
+		perror("out of memory");
+		close (in_fd);
+		close (out_fd);
+		exit(1);
+	}
+}
+
+void clean_buffers() {
+	
+	if (inode_buffer) 
+		free(inode_buffer);
+	if (dirent_buffer)
+		free(dirent_buffer);
+	if (file_buffer)
+		free(file_buffer);
+}
+
+int load_next_block() {
+	
+	int ret;
+	ret = read(in_fd, file_buffer, erase_block_size);
+	file_ofs = 0;
+	
+	if(verbose)
+		printf("Load next block : %d bytes read\n",ret);
+	
+	return ret;
+}
+
+void write_buff_to_file(int nd) {
+	
+	int ret;
+	int len = erase_block_size;
+	uint8_t *buf = NULL;
+	
+	if (!nd) {
+		buf = inode_buffer;
+		while (len > 0) {
+			ret = write(out_fd, buf, len);
+	
+			if (ret < 0)
+				perror_msg_and_die("write");
+	
+			if (ret == 0)
+				perror_msg_and_die("write returned zero");
+	
+			len -= ret;
+			buf += ret;
+		}
+		ino_ofs = 0;
+	}
+	else {
+		buf = dirent_buffer;
+		while (len > 0) {
+			ret = write(out_fd, buf, len);
+	
+			if (ret < 0)
+				perror_msg_and_die("write");
+	
+			if (ret == 0)
+				perror_msg_and_die("write returned zero");
+	
+			len -= ret;
+			buf += ret;
+		}
+		dent_ofs = 0;
+	}
+}
+
+void dump_sum_records() {
+	
+    struct jffs2_inode_sum_node isum;
+    struct sum_storage *temp;
+	jint32_t offset;
+	jint32_t *wpage;
+	int datasize;
+	int infosize;
+	int padsize;
+	jint32_t magic = cpu_to_je32(JFFS2_SUM_MAGIC);
+	
+	if (!sum_records) 
+		return; 
+	
+	datasize = sum_records * sizeof(struct jffs2_inode_sum_record) + 8;
+	infosize = sizeof(struct jffs2_inode_sum_node) + datasize;
+	padsize = erase_block_size - ino_ofs - infosize;
+	infosize += padsize; datasize += padsize;
+	offset = cpu_to_je32(ino_ofs);
+	jint32_t *tpage = (jint32_t *) malloc(datasize);
+	
+	if(!tpage)
+		error_msg_and_die("Can't allocate memory to dump summary information!\n");
+	
+	memset(tpage, 0xff, datasize);
+	memset(&isum, 0, sizeof(isum));
+	
+	isum.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
+	isum.nodetype = cpu_to_je16(JFFS2_NODETYPE_INODE_SUM);
+	isum.totlen = cpu_to_je32(infosize);
+	isum.hdr_crc = cpu_to_je32(crc32(0, &isum, sizeof(struct jffs2_unknown_node) - 4));
+		
+	if (add_cleanmarkers && found_cleanmarkers) {
+		isum.cln_mkr = cpu_to_je32(cleanmarker_size);	
+	}
+	else{
+		isum.cln_mkr = cpu_to_je32(0);
+	}
+	
+	isum.sum_num = cpu_to_je16(sum_records);
+	wpage = tpage;
+	
+	while (sum_records) {
+		*(wpage++) = sum_collected->inode;
+		*(wpage++) = sum_collected->version;
+		*(wpage++) = sum_collected->offset;
+		*(wpage++) = sum_collected->totlen;
+		temp = sum_collected;
+		sum_collected = sum_collected->next;
+		free(temp);
+		sum_records--;
+	}
+	
+	((char *)wpage) += padsize;
+	*(wpage++) = offset;
+	*(wpage++) = magic;
+	isum.sum_crc = cpu_to_je32(crc32(0, tpage, datasize));
+	isum.node_crc = cpu_to_je32(crc32(0, &isum, sizeof(isum) - 8));
+	
+	full_write(inode_buffer + ino_ofs, &isum, sizeof(isum), SINODE);
+	full_write(inode_buffer + ino_ofs, tpage, datasize, SINODE);
+	
+	free(tpage);
+}
+
+static void full_write(void *target_buff, const void *buf, int len, int nd) {
+	memcpy(target_buff, buf, len);
+	
+	if (!nd)
+		ino_ofs += len;
+	else 
+		dent_ofs += len;
+}
+
+static void pad(int req, int nd) {
+	if (!nd) {
+		
+		while (req) {
+			if (req > sizeof(ffbuf)) {
+				full_write(inode_buffer + ino_ofs, ffbuf, sizeof(ffbuf), nd);
+				req -= sizeof(ffbuf);
+			} else {
+				full_write(inode_buffer + ino_ofs, ffbuf, req, nd);
+				req = 0;
+			}
+		}
+	} 
+	else {
+		while (req) {
+			if (req > sizeof(ffbuf)) {
+				full_write(dirent_buffer + dent_ofs, ffbuf, sizeof(ffbuf), nd);
+				req -= sizeof(ffbuf);
+			} 
+			else {
+				full_write(dirent_buffer + dent_ofs, ffbuf, req, nd);
+				req = 0;
+			}
+		}
+	}
+}
+
+static inline void padword(int nd) {
+	
+	if (!nd){
+		if (ino_ofs % 4) {
+			full_write(inode_buffer + ino_ofs, ffbuf, 4 - (ino_ofs % 4), nd);
+		}
+	} 
+	else {
+		if (dent_ofs % 4) {
+			full_write(dirent_buffer + dent_ofs, ffbuf, 4 - (dent_ofs % 4), nd);
+		}
+	}
+}
+
+static inline void pad_block_if_less_than(int req, int nd) {
+    if (!nd) {
+	    int datasize = ((sum_records + 1) * sizeof(struct jffs2_inode_sum_record)) + sizeof(struct jffs2_inode_sum_node) + 8;
+	    datasize += (4 - (datasize % 4)) % 4;
+	    if (ino_ofs + req > erase_block_size - datasize) {
+	        dump_sum_records();
+			write_buff_to_file(nd);
+	    }
+		
+		if (add_cleanmarkers && found_cleanmarkers) {
+			if (!ino_ofs) {
+				full_write(inode_buffer, &cleanmarker, sizeof(cleanmarker), nd);
+				pad(cleanmarker_size - sizeof(cleanmarker), nd);
+				padword(nd);
+			}
+		}
+			
+    }
+	else {
+	    if (dent_ofs + req > erase_block_size)  {
+	        pad(erase_block_size - dent_ofs, nd);
+			write_buff_to_file(nd);
+	    }
+		
+		if (add_cleanmarkers && found_cleanmarkers) {
+			if (!dent_ofs) {
+				full_write(dirent_buffer, &cleanmarker, sizeof(cleanmarker), nd);
+				pad(cleanmarker_size - sizeof(cleanmarker), nd);
+				padword(nd);
+			}
+    	}
+	}	
+}
+
+void flush_buffers() {
+	
+	if ((add_cleanmarkers == 1) && (found_cleanmarkers == 1)) { /* CLEANMARKER */
+		if (ino_ofs != cleanmarker_size) {	/* INODE BUFFER */
+			
+		    int datasize = ((sum_records + 1) * sizeof(struct jffs2_inode_sum_record)) + sizeof(struct jffs2_inode_sum_node) + 8;
+		    datasize += (4 - (datasize % 4)) % 4;
+			
+			/* If we have a full inode buffer, then write out inode and summary data  */
+		    if (ino_ofs + sizeof(struct jffs2_raw_inode) + JFFS2_MIN_DATA_LEN > erase_block_size - datasize) {
+		        dump_sum_records();
+				write_buff_to_file(SINODE);
+		    }
+			/* else just write out inode data */
+			else{
+				pad(erase_block_size - ino_ofs, SINODE);
+				write_buff_to_file(SINODE);
+			}
+		}
+		
+		if (dent_ofs != cleanmarker_size) { /* DIRENT AND OTHERS BUFFER */
+			pad(erase_block_size - dent_ofs, SONODE);
+			write_buff_to_file(SONODE);
+		}
+		
+	}
+	else { /* NO CLEANMARKER */
+		if (ino_ofs != 0) { /* INODE BUFFER */
+			
+		    int datasize = ((sum_records + 1) * sizeof(struct jffs2_inode_sum_record)) + sizeof(struct jffs2_inode_sum_node) + 8;
+		    datasize += (4 - (datasize % 4)) % 4;
+			
+			/* If we have a full inode buffer, then write out inode and summary data */
+		    if (ino_ofs + sizeof(struct jffs2_raw_inode) + JFFS2_MIN_DATA_LEN > erase_block_size - datasize) {
+		        dump_sum_records();
+				write_buff_to_file(SINODE);
+		    }
+			/* Else just write out inode data */
+			else{
+				pad(erase_block_size - ino_ofs, SINODE);
+				write_buff_to_file(SINODE);
+			}
+		}
+		
+		if (dent_ofs != 0) { /* DIRENT AND OTHER BUFFER */
+			pad(erase_block_size - dent_ofs, SONODE);
+			write_buff_to_file(SONODE);
+		}
+	}
+}
+
+
+void write_dirent_to_buff(union jffs2_node_union *node) {
+	
+	pad_block_if_less_than(je32_to_cpu (node->d.totlen), SONODE);
+	full_write(dirent_buffer + dent_ofs, &(node->d), je32_to_cpu (node->d.totlen), SONODE);
+	padword(SONODE);	
+}
+
+void add_sum_entry(union jffs2_node_union *node) {
+	
+	sum_storage *walk;
+	sum_storage *temp = (sum_storage *) malloc(sizeof(sum_storage));
+	
+	if (!temp)
+		error_msg_and_die("Can't allocate memory for summary information!\n");
+	
+	temp->inode = node->i.ino;
+	temp->version = node->i.version;
+	temp->offset = cpu_to_je32(ino_ofs); 
+	temp->totlen = node->i.totlen;
+	temp->next = NULL;
+	
+	if (!sum_collected) {
+	    sum_collected = temp;
+	} 
+	else {
+	    walk = sum_collected;
+		
+	    while (walk->next) {
+			walk = walk->next;
+	    }
+		walk->next = temp;
+	}
+	sum_records++;
+}
+
+void write_inode_to_buff(union jffs2_node_union *node) {
+	
+	pad_block_if_less_than(je32_to_cpu (node->i.totlen), SINODE);  
+	add_sum_entry(node);	/* Add inode summary entry to summary list */
+	full_write(inode_buffer + ino_ofs, &(node->i), je32_to_cpu (node->i.totlen), SINODE);	/* Write out the inode to inode_buffer */
+	padword(SINODE);
+	
+}
+
+
+void create_summed_image(int inp_size) {
+	uint8_t		*p = file_buffer;
+	union jffs2_node_union 	*node;
+	uint32_t	crc;
+	uint16_t	type;
+	int		bitchbitmask = 0;
+	int		obsolete;
+	
+	char	name[256];
+	
+	while ( p < (file_buffer + inp_size)) {
+		
+		node = (union jffs2_node_union*) p;
+		
+		/* Skip empty space */
+		if (je16_to_cpu (node->u.magic) == 0xFFFF && je16_to_cpu (node->u.nodetype) == 0xFFFF) {
+			p += 4;
+			continue;
+		}
+		
+		if (je16_to_cpu (node->u.magic) != JFFS2_MAGIC_BITMASK)	{
+			if (!bitchbitmask++)
+    			    printf ("Wrong bitmask  at  0x%08x, 0x%04x\n", p - file_buffer, je16_to_cpu (node->u.magic));
+			p += 4;
+			continue;
+		}
+		
+		bitchbitmask = 0;
+		
+		type = je16_to_cpu(node->u.nodetype);
+		if ((type & JFFS2_NODE_ACCURATE) != JFFS2_NODE_ACCURATE) {
+			obsolete = 1;
+			type |= JFFS2_NODE_ACCURATE;
+		} else
+			obsolete = 0;
+		
+		node->u.nodetype = cpu_to_je16(type);
+	    
+		crc = crc32 (0, node, sizeof (struct jffs2_unknown_node) - 4);
+		if (crc != je32_to_cpu (node->u.hdr_crc)) {
+			printf ("Wrong hdr_crc  at  0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->u.hdr_crc), crc);
+			p += 4;
+			continue;
+		}
+		
+		switch(je16_to_cpu(node->u.nodetype)) {
+		
+			case JFFS2_NODETYPE_INODE:
+				if(verbose)
+					printf ("%8s Inode      node at 0x%08x, totlen 0x%08x, #ino  %5d, version %5d, isize %8d, csize %8d, dsize %8d, offset %8d\n",
+						obsolete ? "Obsolete" : "",
+						p - file_buffer, je32_to_cpu (node->i.totlen), je32_to_cpu (node->i.ino),
+						je32_to_cpu ( node->i.version), je32_to_cpu (node->i.isize), 
+						je32_to_cpu (node->i.csize), je32_to_cpu (node->i.dsize), je32_to_cpu (node->i.offset));
+	
+				crc = crc32 (0, node, sizeof (struct jffs2_raw_inode) - 8);
+				if (crc != je32_to_cpu (node->i.node_crc)) {
+					printf ("Wrong node_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.node_crc), crc);
+					p += PAD(je32_to_cpu (node->i.totlen));
+					continue;
+				}
+				
+				crc = crc32(0, p + sizeof (struct jffs2_raw_inode), je32_to_cpu(node->i.csize));
+				if (crc != je32_to_cpu(node->i.data_crc)) {
+					printf ("Wrong data_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.data_crc), crc);
+					p += PAD(je32_to_cpu (node->i.totlen));
+					continue;
+				}
+				
+				write_inode_to_buff(node);
+				
+				p += PAD(je32_to_cpu (node->i.totlen));
+				break;
+				
+			case JFFS2_NODETYPE_DIRENT:
+				memcpy (name, node->d.name, node->d.nsize);
+				name [node->d.nsize] = 0x0;
+			
+				if(verbose)
+					printf ("%8s Dirent     node at 0x%08x, totlen 0x%08x, #pino %5d, version %5d, #ino  %8d, nsize %8d, name %s\n",
+						obsolete ? "Obsolete" : "",
+						p - file_buffer, je32_to_cpu (node->d.totlen), je32_to_cpu (node->d.pino),
+						je32_to_cpu ( node->d.version), je32_to_cpu (node->d.ino), 
+						node->d.nsize, name);
+	
+				crc = crc32 (0, node, sizeof (struct jffs2_raw_dirent) - 8);
+				if (crc != je32_to_cpu (node->d.node_crc)) {
+					printf ("Wrong node_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.node_crc), crc);
+					p += PAD(je32_to_cpu (node->d.totlen));
+					continue;
+				}
+				
+				crc = crc32(0, p + sizeof (struct jffs2_raw_dirent), node->d.nsize);
+				if (crc != je32_to_cpu(node->d.name_crc)) {
+					printf ("Wrong name_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.name_crc), crc);
+					p += PAD(je32_to_cpu (node->d.totlen));
+					continue;
+				}
+	
+				write_dirent_to_buff(node);
+				
+				p += PAD(je32_to_cpu (node->d.totlen));						
+				break;
+		
+			case JFFS2_NODETYPE_CLEANMARKER:
+				if (verbose) {
+					printf ("%8s Cleanmarker     at 0x%08x, totlen 0x%08x\n", 
+						obsolete ? "Obsolete" : "",
+						p - file_buffer, je32_to_cpu (node->u.totlen));
+				}
+				
+				if(!found_cleanmarkers){
+					found_cleanmarkers = 1;
+					
+					if(add_cleanmarkers == 1 && use_input_cleanmarker_size == 1){
+						cleanmarker_size = je32_to_cpu (node->u.totlen);
+						setup_cleanmarker();
+					}
+				}			
+				
+				p += PAD(je32_to_cpu (node->u.totlen));						
+				break;
+		
+			case JFFS2_NODETYPE_PADDING:
+				if (verbose) {
+					printf ("%8s Padding    node at 0x%08x, totlen 0x%08x\n", 
+						obsolete ? "Obsolete" : "",
+						p - file_buffer, je32_to_cpu (node->u.totlen));
+				}		
+				p += PAD(je32_to_cpu (node->u.totlen));						
+				break;
+				
+			case 0xffff:
+				p += 4;
+				break;
+				
+			default:	
+				if (verbose) {
+					printf ("%8s Unknown    node at 0x%08x, totlen 0x%08x\n", 
+						obsolete ? "Obsolete" : "",
+						p - file_buffer, je32_to_cpu (node->u.totlen));
+				}
+				
+				write_dirent_to_buff(node);
+				
+				p += PAD(je32_to_cpu (node->u.totlen));						
+		}	
+	}
+}
+
+int main(int argc, char **argv) {
+	
+	int ret;
+	
+	process_options(argc,argv);
+	
+	if ((in_fd == -1) || (out_fd == -1))	{
+		
+		if(in_fd != -1)
+			close(in_fd);
+		if(out_fd != -1)
+			close(out_fd);
+		
+		error_msg_and_die("You must specify input and output files!\n");
+	}
+	
+	init_buffers();
+	
+	while ((ret = load_next_block())) {
+		create_summed_image(ret);	
+	}
+
+	flush_buffers();
+	clean_buffers();
+	
+	if (in_fd != -1)
+		close(in_fd);
+	if (out_fd != -1)
+		close(out_fd);
+	
+	return 0;
+}

Re: [OBORONA-SPAM] JFFS2 mount time

[Artem B. Bityuckiy]

Ferenc,

I didn't investigate your patch well yet, but after 3 minuets of looking 
to it I have two questions:

1. Why did you introduce the JFFS2_SUM_MAGIC constant? As I understand, 
the node's magic field is needed to identify the *beginning of node*, 
*not the node type*. The type of node is defined by the next field, 
called 'nodetype'. You use it (JFFS2_NODETYPE_INODE_SUM). So, IMHO, the 
JFFS2_SUM_MAGIC constant doesn't fit into the common rules...

2. This is very minor of course, just a remark. IMHO, its better to 
avoid too many ifdefs, so, I think it is unnecessary to place the 
function prototype under ifdef. I mead:

+#ifdef CONFIG_JFFS2_FS_SUMMARY
+static struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct 
jffs2_sb_info *c, uint32_t ino);
+#endif



Ferenc Havasi wrote:
> Dear All,
> 
> Here is the latest version of our mount time improvement.
> 
> Using of it:
> - apply this patch on the latest version of MTD
> - compile sumtool (make command in mtd/util)
> - make your JFFS2 image as before (or you can use already created images 
> as well)
> - run sumtool to insert summary information, for example:
>   ./sumtool -i original.jffs2 -o new.jffs2 -e128KiB
> - recompile your kernel with "JFFS2 inode summary support"
> 
> Jarkko made a measurement on a real NAND device: his JFFS2 image was 
> 120819928 (115M), after running sumtool the new image was 123338752 (117M).
> 
> Using the original mount time was 55 sec, with the new image it is only 
> 8.5 sec.
> 
> It works very similar as our previous improvement: stores special 
> information at the end of the erase blocks, and at mount time if there 
> is this kind of information the scaning of the erase block is unneccessary.
> 
> New things compared to our previous improvement:
> - it was fully rewritten
> - we separated the user space tool from mkfs. (sumtool)
> - sumtool now not only inserts the summary information but also make 
> some node-reordering. There will be two kind of erase blocks: in the 
> "first type" there will be only jffs2_raw_inodes, and all other node 
> (jffs2_raw_dirent) will be stored in the "second type". It generates 
> summary at the end of all "fist type" eraseblock. (the "second type" 
> will be scanned as before, because all information is needed in 
> jffs_raw_dirent at mount time)
> 
> Ceratinly all of these things are optional (as you can see above you 
> have to select it from kernel config). The JFFS2 image produced by 
> sumtool is also usable with previous kernel because the summary node is 
> JFFS2_FEATURE_RWCOMPAT_DELETE.
> 
> I think it can be usefull not only for us. David, may I commit it to the 
> CVS?
> 
> Regards,
> Ferenc

-- 
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: JFFS2 mount time

[Ferenc Havasi]

Hi Artem,

> 1. Why did you introduce the JFFS2_SUM_MAGIC constant? As I understand, 
> the node's magic field is needed to identify the *beginning of node*, 
> *not the node type*. The type of node is defined by the next field, 
> called 'nodetype'. You use it (JFFS2_NODETYPE_INODE_SUM). So, IMHO, the 
> JFFS2_SUM_MAGIC constant doesn't fit into the common rules...

The reason is the following: the summary node is at the end of the erase 
block, and it has not fixed size (its size depends on the information it 
stores).

The main advantage of using summary node is to avoid the original 
scanning method. So we cannot use the original full-scanning method to 
determine  the begining of the summary node (using only 
JFFS2_NODETYPE_INODE_SUM).

Our method is the following:
- read some bytes at the end of the erase block
- if the last word is JFFS2_SUM_MAGIC than we will almost sure that it 
is an erase block which has summary
- the word before this magic is the length of the node
- using this length we can check that it is really a 
JFFS2_NODETYPE_INODE_SUM node, and process it

I can't image more effective method to determine the begining of the 
summary node. (if you have better suggestion...) And because the magic 
is inside of the summary node I think it is fit to the philosophy of 
JFFS2 - but a little bit tricky.

> 2. This is very minor of course, just a remark. IMHO, its better to 
> avoid too many ifdefs, so, I think it is unnecessary to place the 
> function prototype under ifdef. I mead:
> 
> +#ifdef CONFIG_JFFS2_FS_SUMMARY
> +static struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct 
> jffs2_sb_info *c, uint32_t ino);
> +#endif

Yes, I aggree. I will modify it.

Bye,
Ferenc

Re: JFFS2 mount time

[Artem B. Bityuckiy]

Ferenc Havasi wrote:
> The reason is the following: the summary node is at the end of the erase 
> block, and it has not fixed size (its size depends on the information it 
> stores).
> 
> The main advantage of using summary node is to avoid the original 
> scanning method. So we cannot use the original full-scanning method to 
> determine  the begining of the summary node (using only 
> JFFS2_NODETYPE_INODE_SUM).
> 
> Our method is the following:
> - read some bytes at the end of the erase block
> - if the last word is JFFS2_SUM_MAGIC than we will almost sure that it 
> is an erase block which has summary
> - the word before this magic is the length of the node
> - using this length we can check that it is really a 
> JFFS2_NODETYPE_INODE_SUM node, and process it
> 
> I can't image more effective method to determine the begining of the 
> summary node. (if you have better suggestion...) And because the magic 
> is inside of the summary node I think it is fit to the philosophy of 
> JFFS2 - but a little bit tricky.
Ok, I got it. I was wrong, sorry.

-- 
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: JFFS2 mount time

[Artem B. Bityuckiy]

Hello Ferenc,

As I understand, you only prepare JFFS2 image with summaries. This is 
great until we do not change anything. For read-only file-systems this 
is OK.

But what if files/direntries are changed/deleted ? Do you write summary 
information dynamically? How are you going to place nodes/direntries to 
different blocks dynamically?

Ferenc Havasi wrote:
> Dear All,
> 
> Here is the latest version of our mount time improvement.
> 
> Using of it:
> - apply this patch on the latest version of MTD
> - compile sumtool (make command in mtd/util)
> - make your JFFS2 image as before (or you can use already created images 
> as well)
> - run sumtool to insert summary information, for example:
>   ./sumtool -i original.jffs2 -o new.jffs2 -e128KiB
> - recompile your kernel with "JFFS2 inode summary support"
> 
> Jarkko made a measurement on a real NAND device: his JFFS2 image was 
> 120819928 (115M), after running sumtool the new image was 123338752 (117M).
> 
> Using the original mount time was 55 sec, with the new image it is only 
> 8.5 sec.
> 
> It works very similar as our previous improvement: stores special 
> information at the end of the erase blocks, and at mount time if there 
> is this kind of information the scaning of the erase block is unneccessary.
> 
> New things compared to our previous improvement:
> - it was fully rewritten
> - we separated the user space tool from mkfs. (sumtool)
> - sumtool now not only inserts the summary information but also make 
> some node-reordering. There will be two kind of erase blocks: in the 
> "first type" there will be only jffs2_raw_inodes, and all other node 
> (jffs2_raw_dirent) will be stored in the "second type". It generates 
> summary at the end of all "fist type" eraseblock. (the "second type" 
> will be scanned as before, because all information is needed in 
> jffs_raw_dirent at mount time)
> 
> Ceratinly all of these things are optional (as you can see above you 
> have to select it from kernel config). The JFFS2 image produced by 
> sumtool is also usable with previous kernel because the summary node is 
> JFFS2_FEATURE_RWCOMPAT_DELETE.
> 
> I think it can be usefull not only for us. David, may I commit it to the 
> CVS?
> 
> Regards,
> Ferenc

-- 
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: JFFS2 mount time

[Ferenc Havasi]

Hi Artem,

> As I understand, you only prepare JFFS2 image with summaries. This is 
> great until we do not change anything. For read-only file-systems this 
> is OK.
> 
> But what if files/direntries are changed/deleted ? Do you write summary 
> information dynamically? How are you going to place nodes/direntries to 
> different blocks dynamically?

You are right, there is a small change which is really important (and 
will be ready very soon) to extend jffs2_mark_node_obsolete() to mark 
not only the node but also its entry in the summary.

Any other improvement can be done later, because after it the filesystem 
will be always coherent, because we write summary only at the of the 
erasy blocks, when it is fully "finished" - so if there is a summary 
somewhere we will not need to extend it, only to mark the obscolated nodes.

We also plan in the near future to implement the ability of generating 
summary dinamically when the filesystem finishes an erase block - which 
keep this "fast mount time" permament.

Bye,
Ferenc

Re: JFFS2 mount time

[Artem B. Bityuckiy]

> You are right, there is a small change which is really important (and 
> will be ready very soon) to extend jffs2_mark_node_obsolete() to mark 
> not only the node but also its entry in the summary.
Unfortunately, you can not mark entries as obsoleted in your summary 
node in case of NAND.

If you write your summary only for *full* blocks, you will not need to 
mark entries obsoleted, even if you have NOR flash (but you can on NOR). 
The partially filled blocks must not have the summary node (you can 
introduce special marker and write it to OOB of the last page of 
NAND/last word of sector on NOR which tells if there is the summary node 
present).

So, fully filled block will have summary and will be scanned very 
quickly, partially filled ones will have no summary and will be fully 
scanned, free blocks will have cleanmarkers and will not be scanned, 
other blocks will be either erased or considered free.

> 
> Any other improvement can be done later, because after it the filesystem 
> will be always coherent, because we write summary only at the of the 
> erasy blocks, when it is fully "finished" - so if there is a summary 
> somewhere we will not need to extend it, only to mark the obscolated nodes.
Yes, nice, but why do you need to mark obsoleted nodes in summary ??? 
When you insert node to the fragtree or dirents to the list, JFFS2 code 
will detect obsoleted nodes automatically, no need to mark them physically.

> 
> We also plan in the near future to implement the ability of generating 
> summary dinamically when the filesystem finishes an erase block - which 
> keep this "fast mount time" permament.
This would be perfect.


-- 
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: JFFS2 mount time

[Ferenc Havasi]

Hi Artem,

> Unfortunately, you can not mark entries as obsoleted in your summary 
> node in case of NAND.
> 
> If you write your summary only for *full* blocks, you will not need to 
> mark entries obsoleted, even if you have NOR flash (but you can on NOR). 
> The partially filled blocks must not have the summary node (you can 
> introduce special marker and write it to OOB of the last page of 
> NAND/last word of sector on NOR which tells if there is the summary node 
> present).

Really, you are right.

So we only have to solve this problem on NOR. I think the easiest 
solution is to set jffs2_can_mark_obsolete() to false if the summary 
support is enabled.

Bye,
Ferenc

JFFS2 mount time - more

[Artem B. Bityuckiy]

Ferenc,

I have 3 more questions.

1. How large are your summary nodes (in average) for blocks full of 
dirents/nodes ?
2. Why do not you use compression for them?
3. Why did you introduce new tool instead of just adding new options to 
the mkfs.jffs2 ?

-- 
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: JFFS2 mount time - 3 more questions

[Ferenc Havasi]

Hi Artem,

> 1. How large are your summary nodes (in average) for blocks full of 
> dirents/nodes ?

It heavily depends on
- the size of the earase block
- the sizes of the nodes
It is 4 words for every jffs2_raw_inode. Dirents are stored separatedly 
  without summary.

> 2. Why do not you use compression for them?

To make boot time as fast as possible :) But not a bad idea. If someone 
needs it we can make a new option.

> 3. Why did you introduce new tool instead of just adding new options to 
> the mkfs.jffs2 ?

I think it is "nicer", cleaner design, and uing this separation the 
reordering of the nodes is much more easier.

Bye,
Ferenc

Re: JFFS2 mount time

[Jarkko Lavinen]

On Wed, Oct 20, 2004 at 04:26:27PM +0200, ext Ferenc Havasi wrote:
> Jarkko made a measurement on a real NAND device: his JFFS2 image was 
> 120819928 (115M), after running sumtool the new image was 123338752 (117M).
> 
> Using the original mount time was 55 sec, with the new image it is only 
> 8.5 sec.

My initial test was only about the mount time. I have now also tried
to exercise the patched file system and with very little testing I get
CRC or ECC errors.

  # mount /dev/mtdblock2 /mnt -t jffs2
  # mkdir /mnt/testdir
  # umount /mnt
  jffs2_flush_wbuf(): Write failed with -5
  # mount /dev/mtdblock2 /mnt -t jffs2
  mtd->read(0x1fbec bytes from 0x1fc0414) returned ECC error
  Empty flash at 0x01fc2f1c ends at 0x01fc3000
  #

With plain 2.6.9-rc4-omap1 with fresh CVS MTD code, I don't see anything
weird occuring.

Jarkko Lavinen

Re: JFFS2 mount time

[Ferenc Havasi]

Hi Jarkko,

> My initial test was only about the mount time. I have now also tried
> to exercise the patched file system and with very little testing I get
> CRC or ECC errors.
> 
>   # mount /dev/mtdblock2 /mnt -t jffs2
>   # mkdir /mnt/testdir
>   # umount /mnt
>   jffs2_flush_wbuf(): Write failed with -5
>   # mount /dev/mtdblock2 /mnt -t jffs2
>   mtd->read(0x1fbec bytes from 0x1fc0414) returned ECC error
>   Empty flash at 0x01fc2f1c ends at 0x01fc3000
>   #
> 
> With plain 2.6.9-rc4-omap1 with fresh CVS MTD code, I don't see anything
> weird occuring.

Can you send me the full kernel log file?

Another interesting test would be to test the new image (sumtool) with 
the CVS MTD code. Because the summary is RWCOMPAT_DELETE node it should 
works well. It would be nice to now if there is CRC/ECC errors in this case.

Thanks,
Ferenc

Re: JFFS2 mount time

[Ferenc Havasi]

Jarkko Lavinen wrote:

> My initial test was only about the mount time. I have now also tried
> to exercise the patched file system and with very little testing I get
> CRC or ECC errors.
> 
>   # mount /dev/mtdblock2 /mnt -t jffs2
>   # mkdir /mnt/testdir
>   # umount /mnt
>   jffs2_flush_wbuf(): Write failed with -5
>   # mount /dev/mtdblock2 /mnt -t jffs2
>   mtd->read(0x1fbec bytes from 0x1fc0414) returned ECC error
>   Empty flash at 0x01fc2f1c ends at 0x01fc3000
>   #
> 
> With plain 2.6.9-rc4-omap1 with fresh CVS MTD code, I don't see anything
> weird occuring.

We are tring to find out what happens here...

Jarkko previously sent me some more detail, the logs starts with:
> sh-2.05b# /rootfstest.sh
> Mounting file system: 			Ok
> Creating a test directory: 						Ok
> Creating a test file: mtd->read(0x44 bytes from 0x1fa344c) returned ECC error
> Data CRC 6d0b1da8 != calculated CRC 9c4f3838 for node at 01fa344c
> mtd->read(0x44 bytes from 0x1fa3e20) returned ECC error
> Data CRC 5127cb7f != calculated CRC 057e127c for node at 01fa3e20
> mtd->read(0x44 bytes from 0x1fa4388) returned ECC error
> mtd->read(0x44 bytes from 0x1fa5cb8) returned ECC error
> mtd->read(0x44 bytes from 0x1fa6748) returned ECC error
> mtd->read(0x44 bytes from 0x1fa7b30) returned ECC error
> Data CRC 72b41a04 != calculated CRC ebc121db for node at 01fa7b30
> mtd->read(0x44 bytes from 0x1fa866c) returned ECC error
> Data CRC 9ff1d419 != calculated CRC cb2cce56 for node at 01fa866c

It means the mounting is done successfully. The first problem is when 
the filesystem try to read jffs2_raw_inode nodes (if I am right the 0x44 
is the size of that). It is not successfull (I don't know why), and it 
cause CRC errors, too. The only one differences should be only the the 
original version already read this 0x44 before (during mount time), the 
summary version did not read yet, just know where it is from the summary.

Jarkko, one more interesing thing can be (if you have that image) to see 
what is at the place 0x1fa344c, 01fa3e20, ... with the tool jffs2dump.

If anyone have any idea that is welcome. Unfortunatelly we don't have 
real NAND device, and it works with our emulator.

Thanks,
Ferenc

Re: JFFS2 mount time

[David Woodhouse]

On Wed, 2004-10-20 at 16:26 +0200, Ferenc Havasi wrote:
> Dear All,
> 
> Here is the latest version of our mount time improvement.

It's looking good, but the kernel really needs to be able to write these
summaries for _itself_ in order to give a real improvement over the long
term. If the file system has to be read-only we might as well be using
cramfs, and if the summary becomes obsolete over time we might as well
not bother in a lot of cases.

-- 
dwmw2

Re: JFFS2 mount time

[Ferenc Havasi]

David Woodhouse wrote:
> It's looking good, but the kernel really needs to be able to write these
> summaries for _itself_ in order to give a real improvement over the long
> term. If the file system has to be read-only we might as well be using
> cramfs, and if the summary becomes obsolete over time we might as well
> not bother in a lot of cases.

Our plan for it:

We would like to store some additional information in jeb struct:
- a type information, where there this type can be INODE_ONLY and 
ANYTHING_OTHER. This information is easy to detect during mount time.
- a predicted summary size (calculated dinamically). It will be used to 
  decide when to generate the summary. Ceratinly only for INODE_ONLY 
erase blocks.

If I am right every node allocation is done by jffs2_reserve_space(). We 
would like to modify it, and introduce a new interface for it called 
jffs2_reserve_space_for_inode() function. Every inode storing function 
(there is no too much I think) should call 
jffs2_reserve_space_for_inode() with some extra information (inode 
number...).

jffs2_reserve_space() should use only ANYTHING_OTHER eraseblocks, as 
jffs2_reserve_space_for_inode() use only INODE_ONLY ones. If there is no 
free space in them it should use the usual technique to find a clean 
eraseblock and start to store the new node in it.

The generating of summary is also the task of 
jffs2_reserve_space_for_inode(), if the new inode (+summary) is not fit 
in the erase block, it will generates summary.

What do you think?

Regards,
Ferenc

Re: JFFS2 mount time

[Artem Bityuckiy]

Hello Ferenc,

At first, please, let me describe your design shortly to be sure I 
understand it and we both thinking the same way.

Essentially, your design is based on the fact that you do not want to 
refer directory entries in the summary nodes. Motivation that you will 
keep almost the copy of direntries in the summary, thus:
1. duplicating too many information.
2. you suppose there will not be the mount speed acceleration.

So, for this purpose you are going to distribute the inode nodes and 
other (including direntry nodes) by different blocks. Those blocks, who 
contain only the inode nodes, will have summaries, other blocks - will not.

I think this is not the best solution. Why? In general, because I do not 
like the following:
A. Your idea to distribute inode nodes and other nodes between different 
blocks.
B. Your assumption that the directory information in summaries will not 
affect the mount time.

The following are reasons concerning the item A.

1. Your change will affect JFFS2 very heavily. You will introduce 
restriction into JFFS2. Another improvements may not work with such 
restriction. Now all the blocks are equivalent. But you want to 
distinguish between two kins of blocks. Don't you think it is too 
complicated decision?

2. Think about the wear-leveling. In JFFS it was ideal. In JFFS2 it is 
good, but not so ideal. I average, the inode nodes are changed more 
often (just think about FIFOs, we told about them in this list 
recently). So, you will need to Garbage Collect the NODE_ONLY blocks 
more often. So, I afraid the wear-leveling will suffer from your 
improvement.

3. Imagine the file system with *lots* of very small files. I this case, 
  the direntries portion on the media will be large enough. And the 
mount time of such file system will not be improved very well.

4. It seems for me you will need to increase the number of blocks which 
are reserved for the garbage collection (double ?). This is also minor 
drawback.

The following are reasons concerning the item B.

I believe that if we have directory references in summaries, this will 
increase the mount speed.

1. At first, we will store fewer data! We don't need to keep the common 
headers, CRCs and mctimes.
2. At the second, we may compress summary (direntries aren't compressed)!
3. And the third, on NAND there is difference between reading lots of 
different pages or few pages.

I propose the another design.

1. Keep direntry references in summaries too and hence, do not 
distinguish between blocks with inode nodes and direntries.
2. Compress summaries.

So, you will avoid a lot of problems related to teaching the GC 
distinguish between different blocks. This will be more natural. I 
believe, summaries must refer *any* node in block. This is more simple 
and clean design.

Why you do not like this?

I see only one potential problem: direntries may have long names (up to 
255 symbols). this may lead to large summaries.

But in this case we may do:
1. Improve the JFFS2 itself. Keep, say, only 20, characters in the 
full_dirent structure. Most of direntries will fit. For other, we will 
just read the flash.
2. We may not touch JFFS2, and keep only 20 characters in summaries. For 
other direntries, we may read them from flash (keeping theirs flash 
offsets instead of names).

Comments?

-- 
Best regards, Artem B. Bityuckiy
Oktet Labs (St. Petersburg), Software Engineer.
+78124286709 (office) +79112449030 (mobile)
E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru

Re: JFFS2 mount time

[Ferenc Havasi]

Hi Artem,

> So, for this purpose you are going to distribute the inode nodes and 
> other (including direntry nodes) by different blocks. Those blocks, who 
> contain only the inode nodes, will have summaries, other blocks - will not.

Yes, I think there are three kinds of nodes:
- type A contains relevant amount of data which is not needed at mount 
time (jffs2_raw_inode)
- type B is (almost) fully needed at mount time (jffs2_raw_dirent)
- type C is any other (unkown, developements in the future...)

To achieve as much mount time speed up as possible I think we should 
distinguish them.

Using summary the really relevant speed up will be only at node type
A. We can also generate summary for type B, but that (as you wrote) 
relevant ratio of the information will be duplicated.

So we whould like to intorduce two kinds of erase blocks:
- erase blocks with summary: it will store (now only) type A nodes, 
maybe later some of type B
- erase block without summary: it will store all of type C and B nodes 
which is not stored before

> 1. Your change will affect JFFS2 very heavily. You will introduce 
> restriction into JFFS2. Another improvements may not work with such 
> restriction. Now all the blocks are equivalent. But you want to 
> distinguish between two kins of blocks. Don't you think it is too 
> complicated decision?

What kind of restriction do you mean? We don't introduce any 
restrictions. The "type C" kind of nodes are processed as before, using 
the usual scanning method. If you what to force for every node to make 
their represenation in the summary, that whould be a restriction.

I think for some kinds of node summary is meaningful, and for some kinds 
not.

If we mix them that can be a very big slow down, if you what to process 
them only with making a reference in the summary to its offset, because 
if you (for example) what to read only 50 bytes (size of the node) you 
will have to read 512/2048 bytes depening on the flash. (where mostly 
there will be inode nodes which is not neccesery to read because that is 
int he summary)

But if all of this "not summarized, small" nodes are stored in a
"seperated" erase block than the this 512/2048 byte reading will not be 
unnecessary (because on the remaining 462-1998 bytes will store also 
relevant information, which is not in the summary).

> 2. Think about the wear-leveling. In JFFS it was ideal. In JFFS2 it is 
> good, but not so ideal. I average, the inode nodes are changed more 
> often (just think about FIFOs, we told about them in this list 
> recently). So, you will need to Garbage Collect the NODE_ONLY blocks 
> more often. So, I afraid the wear-leveling will suffer from your 
> improvement.

I think the GC solves it "automaticly". This mark 
(SUMMARIZED/NOT_SUMMARIZED) is not a premament thing, it is done "pseudo 
randomly".

I aggree that it cause some different behavior in wear-leveling but I 
don't think it makes it relevantly worse.

> 4. It seems for me you will need to increase the number of blocks which 
> are reserved for the garbage collection (double ?). This is also minor 
> drawback.

I don't understand what do you mean here.

> I believe that if we have directory references in summaries, this will 
> increase the mount speed.
> 1. At first, we will store fewer data! We don't need to keep the common 
> headers, CRCs and mctimes.
> 2. At the second, we may compress summary (direntries aren't compressed)!
> 3. And the third, on NAND there is difference between reading lots of 
> different pages or few pages.

Yes, we should try it - to store dirents in SUMMARIZED erase blocks. But 
it can be a improvement later, for first we need a well working stable 
system - and this is urgent for us now.

> 2. Compress summaries.

It makes harder to determine the optimal time of summary generation (it 
is easy to see the summary size, but here the compressed size of it the 
relevant). It can cause smaller image but may cause some slow down, too. 
We may introduce it later as an option.

So now we have two open discussion:
- is the SUMMARIZED / NOT_SUMMARIZED distiguishment good or not
- in the first version do we need dirents in the summary or not

Fortunatelly the effects (and side effects) of this improvements will be 
active only if the new kernel option is enabled, and don't kill any 
other future improvements.

I curious about (at least) David's optinion about these topics.

Bye,
Ferenc

Re: JFFS2 mount time

[Artem Bityuckiy]

Hello Ferenc,

 > Yes, I think there are three kinds of nodes:
 > - type A contains relevant amount of data which is not needed at mount
 > time (jffs2_raw_inode)
 > - type B is (almost) fully needed at mount time (jffs2_raw_dirent)
 > - type C is any other (unkown, developements in the future...)
 >
 > To achieve as much mount time speed up as possible I think we should
 > distinguish them.
This is what I really do not like.

Ok, let us discuss now only this topic. Lt I explain why I believe it is 
vad and very *unnatural* to introduce two or more kinds of blocks.

The example of JFFS2 change that I consider natural is the introduction 
of new node type. It is natural, because of when JFFS2 was designed, 
this possibility was foreseen and taken into account. It is relatively 
easy to do this. It is possible to do this and do not affect other 
things in the JFFS2.

Conversely, the introducing several block types was not foreseen in the 
JFFS2 design. And all things in the JFFS2 are coded with the assumption 
all the blocks are equivalent.

This is my point view on the issue in general.

Now I will try to illustrate why I think so.

1. In JFFS2 there are several lists of blocks - clean_list, dirty_list, 
very_dirty_list?. Are you going to introduce clean_list_typeA, 
dirty_list_typeA, very_dirty_list_typeA, clean_list_typeB, 
dirty_list_typeB, very_dirty_list_typeB ?

2. Just do 'grep "_list" * | grep -e "\(dirty\)\|\(very\)"' and see how 
many places in JFFS2 where these lists are changed. Do you think it is 
natural to introduce 3 more lists? I believe not. What if somebody else 
will introduce one more type of block?

3. There is write buffer in the JFFS2 which is used in case of NAND. Are 
you going to have two wbufs? This is also significant change.

4. Now the GC just gives one block, and moves all the valid nodes to 
another one. In your case (if you have the JFFS2 image which was created 
  by older code, without your patch, where all node types are mixed), 
you will need to move one type of nodes to one block, another to the 
another block.

So, I think you will be needed to change many things in JFFS2. You have 
a risk to hit on a can of worms.

So, do you agree that this change is *unnatural* ?

===================================================================

 >> 4. It seems for me you will need to increase the number of blocks
 >> which are reserved for the garbage collection (double ?). This is also
 >> minor drawback.
 > I don't understand what do you mean here

I mean the sb->resv_blocks_gcmerge and related. You will need to 
increase it, which is not very good.


-- 
Best regards, Artem B. Bityuckiy
Oktet Labs (St. Petersburg), Software Engineer.
+78124286709 (office) +79112449030 (mobile)
E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru

Re: JFFS2 mount time

[Ferenc Havasi]

Hi Artem,

>  > To achieve as much mount time speed up as possible I think we should
>  > distinguish them.
> This is what I really do not like.
> 
> Ok, let us discuss now only this topic. Lt I explain why I believe it is 
> vad and very *unnatural* to introduce two or more kinds of blocks.

You are right, it can be unnatural in point of the original design of 
the JFFS2. But I think in point of the connection of this optimization 
and JFFS2 it is more natural than simple store offsets in the summary, 
or copy all the information into it.

Our plan was modify wbuf (make a second one) and modify 
jffs2_reserve_space to select the right wbuf and generate summary. Never 
planded to introduce new clean_*, dirty_*, ... lists, thats really too 
difficult.


> 3. There is write buffer in the JFFS2 which is used in case of NAND. Are 
> you going to have two wbufs? This is also significant change.

Yes, we started to implement it yesterday and now agree. It is really 
not easy, and we don't write to rewrite the NAND handling part of JFFS2 
whithout a real NAND device. Maybe at the design of JFFS3 :)

So you convinced me. We will change the design of summary. The inodes 
and dirents will be also in the summary. All other nodes will be copied 
as itself into the summary and cause a warning. The summary support will 
be a required thing for new node types, too.

In the kernel we will have to modify
1. jffs2_scan_eraseblock(), as it is already in our patch
2. jeb struct to store generated the summary dinamically (one plus field)
3. jffs2_reserve_space(), which will have a new parameter (summary 
size), which can be JFFS2_SUMMARY_INODE_SIZE or 
JFFS2_SUMMARY_DIRENT_SIZE(namelen). It can decide when to generate 
summary and it can do this generation.
4. jffs2_flash_writev(), which is used to write info to flash. It can 
parse the node (similar to sumtool) and store the summary of it in its jeb.

If it works we'll check the effect of compressing the summary. (size and 
speed)

Comments?

Bye,
Ferenc

P.s.: Thanks for this good conversation.

Re: JFFS2 mount time

[Artem Bityuckiy]

Hello Ferenc,

Ferenc Havasi wrote:
> In the kernel we will have to modify
> 1. jffs2_scan_eraseblock(), as it is already in our patch
> 2. jeb struct to store generated the summary dinamically (one plus field)
IMHO, since the summary relates only to one block, the current block, it 
is logical to refer the summary from the jffs2_sb_info, not from 
jffs2_erase_blocks. It is also not very nice to store it in the 
jffs2_erase_blocks since it will increase the size of array of JFFS2 
blocks (c->blocks[]).

> 3. jffs2_reserve_space(), which will have a new parameter (summary 
> size), which can be JFFS2_SUMMARY_INODE_SIZE or 
> JFFS2_SUMMARY_DIRENT_SIZE(namelen). It can decide when to generate 
> summary and it can do this generation.
Yes, I also think so.

Currently the jffs2_do_reserve_space() do (as I understand):
1. If the current block (c->nextblock) have space and it is sufficient 
for request, it reserves it.
2. If the c->nextblock has fewer size, than requested, the c->nextblock 
is wasted, put to the correspondent list (dirty_list, etc), free block 
is taken and reserved.

Thus, the jffs2_do_reserve_space() should be improved to be able to save 
some space for summary. And, some function like jffs2_write_summary() 
which will be called before jffs2_do_reserve_space() takes new block 
from the free_list.

> 4. jffs2_flash_writev(), which is used to write info to flash. It can 
> parse the node (similar to sumtool) and store the summary of it in its jeb.
May be write here... Didn't think a lot... May be as I wrote, in 
jffs2_do_reserve_space()...

I also offer you to include direntries in summaries and compress them. See:

sizeof(struct jffs2_raw_dirent) = 40 (without name)
you will need to store in your summary only:

totlen
pino
version
ino
nsize
type
name

which is 24 bytes. You don't store all data! Of course, in case of long 
names things are not so good...

If you also compress them, they will be smaller (minus 50-70%)!

So, if there are few direntries in block, why not to store them in summary?

Did you measured the time of summary uncompress on your system? I can't 
know for sure, but I suspect that if you have, say, 200MHz system, the 
time of uncompression = o(time of block read)!

There is one more issue: if there are too many direntries in block, 
summary may become too large (the compression helps here). In this case 
you may not write summary or don't mention direntries in summary.

-- 
Best regards, Artem B. Bityuckiy
Oktet Labs (St. Petersburg), Software Engineer.
+78124286709 (office) +79112449030 (mobile)
E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru

Re: JFFS2 mount time

[Ferenc Havasi]

Hi Artem,

> IMHO, since the summary relates only to one block, the current block, it 
> is logical to refer the summary from the jffs2_sb_info, not from 
> jffs2_erase_blocks. It is also not very nice to store it in the 
> jffs2_erase_blocks since it will increase the size of array of JFFS2 
> blocks (c->blocks[]).

Is it sure than only one non-full erase block is in the filesystem? 
Non-full means here that there is some nodes already in that, but also 
there is some free space at the end of it.

>> 4. jffs2_flash_writev(), which is used to write info to flash. It can 
>> parse the node (similar to sumtool) and store the summary of it in its 
>> jeb.
> 
> May be write here... Didn't think a lot... May be as I wrote, in 
> jffs2_do_reserve_space()...

As I see jffs2_do_reserve space is called before inode/... allocation in 
most cases. So at that time the summary information is not know - but at 
writing it have to be known certainly.

> So, if there are few direntries in block, why not to store them in summary?

You may misunderstood me. In the previous letter I wrote: "So you 
convinced me. We will change the design of summary. The inodes and 
dirents will be also in the summary."

So now we do plan to store dirents in the summary. :)

> Did you measured the time of summary uncompress on your system? I can't 
> know for sure, but I suspect that if you have, say, 200MHz system, the 
> time of uncompression = o(time of block read)!

It depends on the compressor.

We will test it with zlib/rtime. I whould like to implement as an 
optional feature.

> There is one more issue: if there are too many direntries in block, 
> summary may become too large (the compression helps here). In this case 
> you may not write summary or don't mention direntries in summary.

Let see how it work, and after we can make it more optimal :)

Bye,
Ferenc

Re: JFFS2 mount time

[Artem Bityuckiy]

Ferenc,

> Is it sure than only one non-full erase block is in the filesystem? 
> Non-full means here that there is some nodes already in that, but also 
> there is some free space at the end of it.
I didn't analyse this accurately, but my vision is that there is one 
current block (c->nextblock). Even GC moves nodes to it. This is because 
  the jffs2_do_reserve_space() is always used (even by GC), and the 
jffs2_do_reserve_space() always uses c->nextblock.

> As I see jffs2_do_reserve space is called before inode/... allocation in 
> most cases. So at that time the summary information is not know - but at 
> writing it have to be known certainly.
May be... From another hand you may write summary every time the 
jffs2_reserve_space() fetches new block from the free_list...
Anyway, this is not fundamental...

> You may misunderstood me. In the previous letter I wrote: "So you 
> convinced me. We will change the design of summary. The inodes and 
> dirents will be also in the summary."
> 
> So now we do plan to store dirents in the summary. :)
OK, sorry. :-)

> Let see how it work, and after we can make it more optimal :)
Agree :-)

Also, please, take into account that there may be checkpoint nodes (I'm 
implementing this). So, I think you need to have a generic mechanism to 
add new node types to your summary.

Also, I think it is good to have a generic mechanism to just refer some 
nodes from summaries (for example, direntries with long names or 
something else).

Thank you for conversation too.
:-)

-- 
Best regards, Artem B. Bityuckiy
Oktet Labs (St. Petersburg), Software Engineer.
+78124286709 (office) +79112449030 (mobile)
E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru

Re: JFFS2 mount time

[Ferenc Havasi]

Hi Artem,

> Also, please, take into account that there may be checkpoint nodes (I'm 
> implementing this). So, I think you need to have a generic mechanism to 
> add new node types to your summary.
 >
> Also, I think it is good to have a generic mechanism to just refer some 
> nodes from summaries (for example, direntries with long names or 
> something else).

Yes, it will be easy to extend.

We also need a this general support - because we will introduce a new 
node type, too, becauseof the model file support, which will start to 
commit when David finishes his patch for Linus.

Bye,
Ferenc

Re: JFFS2 mount time

[Artem Bityuckiy]

> I curious about (at least) David's optinion about these topics.
I also wonder why people are not very active :-)


-- 
Best regards, Artem B. Bityuckiy
Oktet Labs (St. Petersburg), Software Engineer.
+78124286709 (office) +79112449030 (mobile)
E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru

Re: JFFS2 mount time

[Jarkko Lavinen]

[-- Attachment #1: Type: text/plain, Size: 2655 bytes --]

I tried to see with jffs2dump how much Inodes and Dirents I have on
root filesystem on Arm testbed. Quick and dirty Perl script attached.
This isn't accurate as the calculated total image size misses at least
the final padding on the last erase block.

The size of the plain JFFS2 image is 31.1 MiB. The root fs consists of
all applications and libraries and no user data.

  $ jffs2dump -c rootfs.jffs2 | perl jffs2stats.pl
  Number of dirents:   6144.
   Total dirent node space:  304911 (0.9%)
   Average dirent len: 49.6
   Total dirent name space:  76671
   Average name len:   12.5

  Number of Inodes:    21197
   Total Inode space:  32254866 (99.1%)
   Average Inode size: 1521.7

  Padding:        37326 0.1%
  Total image size: 32559777
  $ ls -l rootfs.jffs2
  -rw-r--r--  1 root root 32597104 Oct 20 15:11 rootfs.jffs2

With sumtool the image size grows to 31.8 MiB

  $ jffs2dump -c rootfs-sum.jffs2 | perl jffs2stats.pl
  Number of dirents:   6144.
   Total dirent node space:  304911 (0.9%)
   Average dirent len: 49.6
   Total dirent name space:  76671
   Average name len:   12.5

  Number of Inodes:    21197
   Total Inode space:  32254866 (97.2%)
   Average Inode size: 1521.7

  Number of Inode Summary nodes:  251
   Total Inode Sum space: 631524, (1.9%)
   Average Sum node size: 2516.0

  Padding:        153063 0.5%
  Total image size: 33191301
  $ ls -l rootfs-sum.jffs2
  -rw-r--r--  1 root root 33423360 Oct 20 15:23 rootfs-sum.jffs2


If dentries were stored just as they are (unstripped and uncompressed)
in the summary, the summary size would grow by 50% to about 3% of the
whole image size.

On Fri, Oct 22, 2004 at 04:44:13PM +0400, ext Artem Bityuckiy wrote:
> I believe that if we have directory references in summaries, this will 
> increase the mount speed.
> 
> 1. At first, we will store fewer data! We don't need to keep the common 
> headers, CRCs and mctimes.
> 2. At the second, we may compress summary (direntries aren't compressed)!
> 3. And the third, on NAND there is difference between reading lots of 
> different pages or few pages.


I tried Ferenc's earlier mount time patch in August and the 52s mount
time dropped then to 14s. If I understand right, inodes and dentries
were then mixed in the erase block and the summary was for inodes
only.  This shows reading dentries from semirandom places is
expensive.

Ferenc's latest patch put dentries on their own erase block in
consecutive order.  Considering only the read efficiency from the
media, reading consecutive, uncompressed, and unstripped dentries from
a summary should cost no more than reading them from dedicated erase
block.

Jarkko Lavinen

[-- Attachment #2: jffs2stats.pl --]
[-- Type: text/x-perl, Size: 3230 bytes --]

#! /usr/bin/perl

$EBLOCKSIZE=131072;

$dirents = $totdirentlen = $totnamelen = 0;
$inodes = $totinodelen = 0;
$sumnodes = $totsumnodelen = 0;
$totpadlen = 0;
$gaps = $totgaplen = 0;
$nextaddr = 0;

sub checkpadding {
    my ($addr, $totlen) = @_;

    my $len = hex($addr) - $nextaddr;

    if ($len > 0) {
	if (hex($addr) % $EBLOCKSIZE == 0 || $len <= 3) {
	    $totalpadlen += $len;
	} else {
	    print sprintf "Gap seen at %08x .. $addr, length $len\n", $nextaddr;

	    $gaps++;
	    $totgaplen += hex($addr) - $nextaddr;
	}
    }

    $nextaddr = hex($addr) + hex($totlen);
}

while(<>) {
    chop;
    if (/^\s+Dirent/) {
	die "Cannot parse $_" if (! /^ \s+ 
				  Dirent     \s+ 
				  node \s at \s+ (\w+), \s+ 
				  totlen     \s+ (\w+), \s+ 
				  \#pino     \s+ (\w+), \s+ 
				  version    \s+ (\w+), \s+ 
				  \#ino      \s+ (\w+), \s+ 
				  nsize      \s+ (\w+), \s+
				  name       \s+ (.*) 
				  $/x);

	my ($addr, $totlen, $pino, $version, $ino, $nsize, $name) = ($1, $2, $3, $4, $5, $6, $7);
	&checkpadding($addr, $totlen);

	$dirents++;
	$totdirentlen += hex($totlen);
	$totnamelen += hex($nsize);
    } elsif (/^\s+Inode Sum/) { 
	die "Cannot parse $_" if (! /^ \s+ 
				  Inode \s Sum \s+
				  node \s at          \s+ (\w+), \s+
				  totlen              \s+ (\w+), \s+
				  sum_num             \s+ (\w+), \s+
				  cleanmarker \s size \s+ (\w+)  \s*
				  $/x);

	my ($addr, $totlen, $sum_num, $cleanmarksize) = ($1, $2, $3, $4);
	&checkpadding($addr, $totlen);
	
	$sumnodes++;
	$totsumnodelen += hex($totlen);
    } elsif (/^\s+Inode/) {
	die "Cannot parse $_" if (! /^ \s+ 
				  Inode \s+
				  node \s at \s+ (\w+), \s+
				  totlen     \s+ (\w+), \s+
				  \#ino      \s+ (\w+), \s+
				  version    \s+ (\w+), \s+
				  isize      \s+ (\w+), \s+
				  csize      \s+ (\w+), \s+ 
				  dsize      \s+ (\w+), \s+
				  offset     \s+ (\w+)  \s*
				  $/x);

	my ($addr, $totlen, $ino, $version, $isize, $csize, $dsize, $offset) = ($1, $2, $3, $4, $5, $6, $7, $8);
	&checkpadding($addr, $totlen);

	$inodes++;
	$totinodelen += hex($totlen);
    } else {
	die "Cannot parse $_";
    }
}

$totalsize = $totdirentlen + $totinodelen + $totsumnodelen + $totpadlen + $totgaplen;

print "Number of dirents:\t$dirents.\n";
print sprintf " Total dirent node space:\t$totdirentlen (%.1f%%)\n", 100.0*$totdirentlen/$totalsize;
print " Average dirent len:\t", sprintf("%.1f", $totdirentlen/$dirents), "\n" if ($dirents > 0);
print " Total dirent name space:\t$totnamelen\n";
print sprintf(" Average name len:\t%.1f\n", $totnamelen/$dirents) if ($dirents > 0);
print "\n";
print "Number of Inodes:\t$inodes\n";
print sprintf " Total Inode space:\t$totinodelen (%.1f%%)\n", 100.0*$totinodelen/$totalsize;
print " Average Inode size:\t", sprintf("%.1f", $totinodelen/$inodes), "\n" if ($inodes > 0);

if ($sumnodes) {
    print "\n";
    print "Number of Inode Summary nodes:\t$sumnodes\n";
    print sprintf " Total Inode Sum space:\t$totsumnodelen, (%.1f%%)\n", 100.0*$totsumnodelen/$totalsize;
    print " Average Sum node size:\t", sprintf("%.1f", $totsumnodelen/$sumnodes), "\n";
}

print sprintf "\nPadding:\t$totalpadlen %.1f%%\n", 100.0*$totalpadlen/$totalsize;

print "Total image size: $totalsize\n";

Re: JFFS2 mount time

[Ferenc Havasi]

Hi Jarkko,

 > If dentries were stored just as they are (unstripped and uncompressed)
 > in the summary, the summary size would grow by 50% to about 3% of the
 > whole image size.

Thanks, good to know it.

Did you got ECC/CRC errors? The most interest test for me whould be to 
test the new (sumtool) image with the original kernel (because the 
summary nodes are compatibles it should work), and see if there is 
ECC/CRC errors or not.

Bye,
Ferenc

Re: JFFS2 mount time

[Artem Bityuckiy]

Hello Jarkko,

> I tried Ferenc's earlier mount time patch in August and the 52s mount
> time dropped then to 14s. If I understand right, inodes and dentries
> were then mixed in the erase block and the summary was for inodes
> only.  This shows reading dentries from semirandom places is
> expensive.
This is very good that direntries are distributed more or less uniformly 
in average.

> 
> Ferenc's latest patch put dentries on their own erase block in
> consecutive order.  Considering only the read efficiency from the
> media, reading consecutive, uncompressed, and unstripped dentries from
> a summary should cost no more than reading them from dedicated erase
> block.
> 
Definitely true - the second patch must be better than the first one. But
unfortunately, it hard to do this dinamically :-( Ferenc tried...

But in my proposition, we will also refer direntries in the summary - 
this is not the same as to read direntries from where they are placed, 
this is another thing, especially in case of NAND! There is difference 
(if we have NAND) - whether to read one 512 NAND page containing 
compressed information about 20-25 direntries or to read 20-25 
*different* NAND pages.

So, I think, new design will also better than the early Ferenc's patch :-)

-- 
Best regards, Artem B. Bityuckiy
Oktet Labs (St. Petersburg), Software Engineer.
+78124286709 (office) +79112449030 (mobile)
E-mail: dedekind@oktetlabs.ru, web: http://www.oktetlabs.ru

JFFS2 mount time

[Gareth Bult (Encryptec)]

Hi,

I've been working on the MTD block driver for JFFS2 and now seem to have
a relatively stable environment with a JFFS2 root filesystem on a USB
flash key device. 
(mtd block device speed now approximates actual device speed on both
read and write)

It seems to be much quicker (generally) than the previous ext2
incarnation.

However, I'm looking at 30secs to mount the root filesystem.

I would be very grateful of some advice on the following before I start
digging through the JFFS2 code itself;

a. Is it possible (conceptually) to background the mount scan so the
system can boot while the scan happens ?
b. Is there an easy way to bypass the scan and does JFFS3 do this ? 

Any help/advice would be much appreciated.

Many thanks,
Gareth.

Re: JFFS2 mount time

[Josh Boyer]

Gareth Bult (Encryptec) wrote:
> Hi,
> 
> I've been working on the MTD block driver for JFFS2 and now seem to have
> a relatively stable environment with a JFFS2 root filesystem on a USB
> flash key device. 
> (mtd block device speed now approximates actual device speed on both
> read and write)

I assume you mean blkmtd here.  The mtdblock driver is used to present 
an IDE type interface to flash, whereas blkmtd is used to present a 
flash interface to an IDE type device.

> 
> It seems to be much quicker (generally) than the previous ext2
> incarnation.
> 
> However, I'm looking at 30secs to mount the root filesystem.

How big of a device?

> a. Is it possible (conceptually) to background the mount scan so the
> system can boot while the scan happens ?

Which version of the code are you using?  Mount time has been 
significantly improved in newer versions of the code.

> b. Is there an easy way to bypass the scan and does JFFS3 do this ? 

Not that I know of, because the code needs to read the node information 
in order to get the versioning of inodes, etc. correct.

JFFS3 is essentially the exact same code base as JFFS2 right now.  It 
was just forked about a week ago.

josh

Re: JFFS2 mount time

[Gareth Bult (Encryptec)]

Hi,

Sure, the file I've been changing is blkmtd.c.

The device I'm aiming for and testing on is 256Mb with a 128k erase
size.

Currently I'm using the code included in the 2.6.9 kernel source tree
with tools from cvs as of about a week ago.

Is this likely to contain the latest speedups or should I be looking
elsewhere?

tia
Gareth.

On Wed, 2004-12-15 at 18:15 -0600, Josh Boyer wrote:
> Gareth Bult (Encryptec) wrote:
> > Hi,
> > 
> > I've been working on the MTD block driver for JFFS2 and now seem to have
> > a relatively stable environment with a JFFS2 root filesystem on a USB
> > flash key device. 
> > (mtd block device speed now approximates actual device speed on both
> > read and write)
> 
> I assume you mean blkmtd here.  The mtdblock driver is used to present 
> an IDE type interface to flash, whereas blkmtd is used to present a 
> flash interface to an IDE type device.
> 
> > 
> > It seems to be much quicker (generally) than the previous ext2
> > incarnation.
> > 
> > However, I'm looking at 30secs to mount the root filesystem.
> 
> How big of a device?
> 
> > a. Is it possible (conceptually) to background the mount scan so the
> > system can boot while the scan happens ?
> 
> Which version of the code are you using?  Mount time has been 
> significantly improved in newer versions of the code.
> 
> > b. Is there an easy way to bypass the scan and does JFFS3 do this ? 
> 
> Not that I know of, because the code needs to read the node information 
> in order to get the versioning of inodes, etc. correct.
> 
> JFFS3 is essentially the exact same code base as JFFS2 right now.  It 
> was just forked about a week ago.
> 
> josh
> 

Re: JFFS2 mount time

[Josh Boyer]

On Wed, 2004-12-15 at 19:02, Gareth Bult (Encryptec) wrote:
> Hi,

Quick FYI, don't top post.  The email gremlin will chase you down and
make you read http://david.woodhou.se/email.html if you break too many
rules ;)

> 
> Sure, the file I've been changing is blkmtd.c.

Ok, cool.

> 
> The device I'm aiming for and testing on is 256Mb with a 128k erase
> size.
> 
> Currently I'm using the code included in the 2.6.9 kernel source tree
> with tools from cvs as of about a week ago.

Ok, good.


> Is this likely to contain the latest speedups or should I be looking
> elsewhere?

That should have the speedups I was referring to.  Basically checking
the node CRCs is deferred until after mount time so the mount succeeds
much more quickly.

There is one more feature pending that I know of.  Ferenc Havasi has
some eraseblock summary changes that should help.  I believe the
intention was to commit that to JFFS3, but I haven't seen it go in yet.

josh

Re: JFFS2 mount time

[Gareth Bult (Encryptec)]

Hi,
> There is one more feature pending that I know of.  Ferenc Havasi has
> some eraseblock summary changes that should help.  I believe the
> intention was to commit that to JFFS3, but I haven't seen it go in yet.

Ok, I have these and will be trying them presently .. 

just wondered if there was any way I could avoid running "patch" .. :)

Tvm
Gareth.



> josh
> 

Re: JFFS2 mount time

[Josh Boyer]

Gareth Bult (Encryptec) wrote:
> Hi,
> 
>>There is one more feature pending that I know of.  Ferenc Havasi has
>>some eraseblock summary changes that should help.  I believe the
>>intention was to commit that to JFFS3, but I haven't seen it go in yet.
> 
> 
> Ok, I have these and will be trying them presently .. 
> 
> just wondered if there was any way I could avoid running "patch" .. :)

Bug Ferenc to commit ;).  I'm not sure if he subscribes to this list or not.

Also, Artem Bityuckiy is working on an inode checkpoint feature for 
JFFS3.  Once he gets it working, it should increase overall performance.

If you have any ideas or patches, feel free to submit them to the list. 
  That's what JFFS3 was cloned for, and there isn't too much concern 
with backwards compatibility.

josh

Re: JFFS2 mount time

[Gareth Bult (Encryptec)]

If you have any ideas or patches, feel free to submit them to the list. 
>   That's what JFFS3 was cloned for, and there isn't too much concern 
> with backwards compatibility.

Mmm,

blkmtd.c is pretty much a rewrite, less than half the size and uses the
system buffer cache instead of reading / writing directly to the device.
This boosts performance x 3 on read x lots on write. [I get x 100
faster] (albeit you lose bad block detection on write)

If anyone's interested in the code, let me know and I'll clean it up and
post it .. however it'll be a new file and not a patch.

I've only tested it on USB key flash, but I'm assuming it'll be similar
for any block device / hard disk / whatever ..

Gareth.

On Thu, 2004-12-16 at 15:28 -0600, Josh Boyer wrote:
> Gareth Bult (Encryptec) wrote:
> > Hi,
> > 
> >>There is one more feature pending that I know of.  Ferenc Havasi has
> >>some eraseblock summary changes that should help.  I believe the
> >>intention was to commit that to JFFS3, but I haven't seen it go in yet.
> > 
> > 
> > Ok, I have these and will be trying them presently .. 
> > 
> > just wondered if there was any way I could avoid running "patch" .. :)
> 
> Bug Ferenc to commit ;).  I'm not sure if he subscribes to this list or not.
> 
> Also, Artem Bityuckiy is working on an inode checkpoint feature for 
> JFFS3.  Once he gets it working, it should increase overall performance.
> 
> If you have any ideas or patches, feel free to submit them to the list. 
>   That's what JFFS3 was cloned for, and there isn't too much concern 
> with backwards compatibility.
> 
> josh

Re: JFFS2 mount time

[Josh Boyer]

On Thu, 2004-12-16 at 15:47, Gareth Bult (Encryptec) wrote:
> 
> blkmtd.c is pretty much a rewrite, less than half the size and uses the
> system buffer cache instead of reading / writing directly to the device.
> This boosts performance x 3 on read x lots on write. [I get x 100
> faster] (albeit you lose bad block detection on write)

Hm, ok.  An alternative to using the system buffer would be to make
JFFS[23] use a write buffer for those devices.  It's already done on
NAND which has some special handling.  The support for ECC'd NOR
probably matches more closely what you'd want to do.  Take a look at
wbuf.c.

> 
> If anyone's interested in the code, let me know and I'll clean it up and
> post it .. however it'll be a new file and not a patch.

Well, if it's better than the original we can always take yours and
replace what's there.  Send it on, performance gains are always
welcome.  I'm sure there are others interested in what you've done that
will review the code.

josh

Re: JFFS2 mount time

[Gareth Bult (Encryptec)]

Hi,

On Fri, 2004-12-17 at 06:54 -0600, Josh Boyer wrote:

> Hm, ok.  An alternative to using the system buffer would be to make
> JFFS[23] use a write buffer for those devices.  

Forgive my ignorance .. but what's the difference between using a write
buffer and using the system buffer ?

>From my pov, the system buffer is there, already 100% reliable and
somewhat larger than any [potential] write buffer ???

(Given the system buffer uses the underlying block devices read / write
routines and the block device emulator will typically sit on a block
device such as a hard drive or USB flash key .. [?])

> It's already done on
> NAND which has some special handling.  The support for ECC'd NOR
> probably matches more closely what you'd want to do.  Take a look at
> wbuf.c.

Mmm, after looking at wbuf.c I'm none the wiser as to what it does or
how it relates ..

> Well, if it's better than the original we can always take yours and
> replace what's there.  Send it on, performance gains are always
> welcome.  I'm sure there are others interested in what you've done that
> will review the code.

I think once I get this FL release out the way I'll have a a more
in-depth look at JFFS*. It seems to be written almost exclusively for
MTD devices .. which is great .. for MTD devices .. but for block
devices and flash (USB for example) presented through a standard
interface (SCSI for example) it seems there is scope for a much
"thinner" filesystem generic devices with a subset of JFFS's feature
set.

Gareth.

Re: JFFS2 mount time

[Josh Boyer]

Gareth Bult (Encryptec) wrote:>
> 
> Forgive my ignorance .. but what's the difference between using a write
> buffer and using the system buffer ?
> 
>>From my pov, the system buffer is there, already 100% reliable and
> somewhat larger than any [potential] write buffer ???

Perhaps alternative was the wrong word to use on my part.  I'll just 
wait until I see your code.  I'm probably confusing myself by making 
assumptions on what it's doing :).

> 
> Mmm, after looking at wbuf.c I'm none the wiser as to what it does or
> how it relates ..

It might not relate.  IIRC there were some discussions on buffering the 
writes done by JFFS[23] to the size of a sector on the block device so 
that partial writes aren't done unnecessarily.  This is what the wbuf.c 
file does.  NAND and ECC'd NOR require this because you can only write 
to a page once (usually) before an erase is needed to write again.

For disk devices I don't think it's required, but it could help.  Or 
maybe your code does this at the MTD level.  I dunno.  I could be 
spouting nonsense again ;).


> 
> 
> I think once I get this FL release out the way I'll have a a more
> in-depth look at JFFS*. It seems to be written almost exclusively for
> MTD devices .. which is great .. for MTD devices .. but for block
> devices and flash (USB for example) presented through a standard
> interface (SCSI for example) it seems there is scope for a much
> "thinner" filesystem generic devices with a subset of JFFS's feature
> set.

Yes, it is a Journaled _Flash_ File System.  So of course it's written 
for MTD devices.  Not sure why blkmtd.c was originally written.  Test 
vehicle for JFFS2, "why not", "I'm bored", etc.  Who knows? :)

josh

Re: JFFS2 mount time

[Gareth Bult (Encryptec)]

On Fri, 2004-12-17 at 10:02 -0600, Josh Boyer wrote:
> For disk devices I don't think it's required, but it could help.  Or 
> maybe your code does this at the MTD level.  I dunno.  I could be 
> spouting nonsense again ;).

Mmm, as long as writes are aligned on 2k pages and are linear / 
unidirectional, I'm not sure anything else is really a problem 
for USB flash drives ..

> Yes, it is a Journaled _Flash_ File System.  So of course it's written 
> for MTD devices.  Not sure why blkmtd.c was originally written.  Test 
> vehicle for JFFS2, "why not", "I'm bored", etc.  Who knows? :)

Mm, my point exactly , Journaled _Flash_ File System - as opposed to
Journaled _MTD_ _Flash_ File System .. !!

USB flash is still flash! - just presented via a generic interface as
opposed to an embedded one .. (AFAIK)

I guess I was sort of hoping blkmtd.c was written to enable JFFS2 to be
more generic and embedded device independent. I suspect however it was
written purely for testing purposes .. :(

As I understand it, the Flash in USB keys is identical to the MTD type
embedded devices, except that the USB keys always come with
micro-controllers that handle and optimise read/write/erase operations
and present them via a generic PC/USB based disk interface. I'm not sure
how fast MTD can be driven, but key manufacturers seem to think that
keys should be able to run at 20Mb/sec read and 10Mb/sec write [for
large block read/writes] which is much faster than is required for a key
to replace a hard disk in a workstation.

I'm thinking that JFFS2 could draw in a huge additional user base if it
catered for (or at least supported) such devices .. (!)

Gareth.

Re: JFFS2 mount time

[Artem B. Bityuckiy]

On Fri, 17 Dec 2004, Gareth Bult (Encryptec) wrote:

> On Fri, 2004-12-17 at 10:02 -0600, Josh Boyer wrote:
> > For disk devices I don't think it's required, but it could help.  Or 
> > maybe your code does this at the MTD level.  I dunno.  I could be 
> > spouting nonsense again ;).
> 
> Mmm, as long as writes are aligned on 2k pages and are linear / 
> unidirectional, I'm not sure anything else is really a problem 
> for USB flash drives ..
> 
> > Yes, it is a Journaled _Flash_ File System.  So of course it's written 
> > for MTD devices.  Not sure why blkmtd.c was originally written.  Test 
> > vehicle for JFFS2, "why not", "I'm bored", etc.  Who knows? :)
> 
> Mm, my point exactly , Journaled _Flash_ File System - as opposed to
> Journaled _MTD_ _Flash_ File System .. !!
> 
> USB flash is still flash! - just presented via a generic interface as
> opposed to an embedded one .. (AFAIK)
> 
> I guess I was sort of hoping blkmtd.c was written to enable JFFS2 to be
> more generic and embedded device independent. I suspect however it was
> written purely for testing purposes .. :(
> 
> As I understand it, the Flash in USB keys is identical to the MTD type
> embedded devices, except that the USB keys always come with
> micro-controllers that handle and optimise read/write/erase operations
> and present them via a generic PC/USB based disk interface. I'm not sure
> how fast MTD can be driven, but key manufacturers seem to think that
> keys should be able to run at 20Mb/sec read and 10Mb/sec write [for
> large block read/writes] which is much faster than is required for a key
> to replace a hard disk in a workstation.
> 
> I'm thinking that JFFS2 could draw in a huge additional user base if it
> catered for (or at least supported) such devices .. (!

My understanding was that it is only seen as SCSI block device from 
outside. E.g. it is not possible, say, to erase sector, to write one 
page, etc. If so, just foget about JFFS2 on that, IMHO.

> 
> Gareth.
> 
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 

--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: JFFS2 mount time

[Josh Boyer]

Gareth Bult (Encryptec) wrote:
> 
> Mm, my point exactly , Journaled _Flash_ File System - as opposed to
> Journaled _MTD_ _Flash_ File System .. !!
> 
> USB flash is still flash! - just presented via a generic interface as
> opposed to an embedded one .. (AFAIK)

No, it's really not.  There is no access to the low level flash, so MTD 
cannot directly drive it.  Same principle as the Compact Flash devices, 
which MTD doesn't deal with either (other than via blkmtd).

> 
> I guess I was sort of hoping blkmtd.c was written to enable JFFS2 to be
> more generic and embedded device independent. I suspect however it was
> written purely for testing purposes .. :(

Yes, most likely.

> 
> As I understand it, the Flash in USB keys is identical to the MTD type
> embedded devices, except that the USB keys always come with
> micro-controllers that handle and optimise read/write/erase operations
> and present them via a generic PC/USB based disk interface. I'm not sure
> how fast MTD can be driven, but key manufacturers seem to think that
> keys should be able to run at 20Mb/sec read and 10Mb/sec write [for
> large block read/writes] which is much faster than is required for a key
> to replace a hard disk in a workstation.

The flash used in the devices is identical, but MTD has no access to it.

> 
> I'm thinking that JFFS2 could draw in a huge additional user base if it
> catered for (or at least supported) such devices .. (!)

That's the problem.  There is no filesystem for these "abominations" 
really.  They pretend to be IDE, SCSI, etc. but are really flash 
underneath.  Neither the MTD driver or the IDE drivers (for example) 
deal with them really well, but I'm not sure that they can.

Did you try ext3 on the device with a native driver (IDE, SCSI, USB 
specific) instead of via blkmtd.c?

josh

Re: JFFS2 mount time

[Gareth Bult (Encryptec)]

On Fri, 2004-12-17 at 11:10 -0600, Josh Boyer wrote:
> That's the problem.  There is no filesystem for these "abominations" 
> really.  They pretend to be IDE, SCSI, etc. but are really flash 
> underneath.  Neither the MTD driver or the IDE drivers (for example) 
> deal with them really well, but I'm not sure that they can.
> 
> Did you try ext3 on the device with a native driver (IDE, SCSI, USB 
> specific) instead of via blkmtd.c?

Sure, ext2 and ext3 work just fine .. however .. as far as I'm aware,
although ext3 is log structured, it's not optimised to run in a linear
fashion and doesn't worry about repeatedly writing to the same block ..
which is likely to cause the USB flash controller to execute an erase
prior to write .. which will quickly wear down the device [?!]

(Am I wrong? in which case why isn't everyone using ext3 instead of
JFFS?)

The other requirement of course is inline compression, which last time I
looked ext3 didn't do out of the box (?)

I'm going to deliberately wear down some blocks on a key a little later
(at block level) to see what the OS makes of it ... (and whether it
takes 100,000 or 1,000,000 cycles .. I see conflicting specs ..

I guess the basic spec for a USB flash key (you may call them
abominations, but you should see the sales stats!) would be;

a. purely log structured fs with linear / circular operation
b. write blocks aligned to 2k (or 4k might be smarter to fit with buffer
page sizes)
c. erase blocks @ 128k
d. inline compression

Strikes me JFFS2 does all this, possibly bar the generic device
support ..

Gareth.

Re: JFFS2 mount time

[Josh Boyer]

Gareth Bult (Encryptec) wrote:
> 
> 
> Sure, ext2 and ext3 work just fine .. however .. as far as I'm aware,
> although ext3 is log structured, it's not optimised to run in a linear
> fashion and doesn't worry about repeatedly writing to the same block ..
> which is likely to cause the USB flash controller to execute an erase
> prior to write .. which will quickly wear down the device [?!]
> 
> (Am I wrong? in which case why isn't everyone using ext3 instead of
> JFFS?)

Most of the controllers for these devices do the wear leveling 
themselves.  At least that was my understanding.

We don't deal much with these devices, so I'm not sure what everyone 
else uses.  Alot of the CF devices are FAT formatted to work with 
Windows, etc.

> 
> The other requirement of course is inline compression, which last time I
> looked ext3 didn't do out of the box (?)

No, it doesn't.  There are patches for ext2 to make it use compression 
(e2compr), but then you lose the journaling.

> 
> I'm going to deliberately wear down some blocks on a key a little later
> (at block level) to see what the OS makes of it ... (and whether it
> takes 100,000 or 1,000,000 cycles .. I see conflicting specs ..

That probably depends on the actual flash chips used within the device. 
  Not all of them use the same type of flash.

> 
> I guess the basic spec for a USB flash key (you may call them
> abominations, but you should see the sales stats!) would be;

Hey, they certainly are handy.  I'm not arguing with that :).

> 
> a. purely log structured fs with linear / circular operation
> b. write blocks aligned to 2k (or 4k might be smarter to fit with buffer
> page sizes)
> c. erase blocks @ 128k
> d. inline compression

Again, most controllers do all the wear leveling, erasing for you.  So 
you are left with compression, right?

And that is certainly a lacking thing.  A block-based, writeable, 
compressed filesystem that does journalling.  Sounds like a good 
opportunity to me :).

josh

Re: JFFS2 mount time

[Gareth Bult (Encryptec)]

On Fri, 2004-12-17 at 11:35 -0600, Josh Boyer wrote:
> Gareth Bult (Encryptec) wrote:
> Again, most controllers do all the wear leveling, erasing for you.  So 
> you are left with compression, right?
> 
> And that is certainly a lacking thing.  A block-based, writeable, 
> compressed filesystem that does journalling.  Sounds like a good 
> opportunity to me :).

Mmm, sounds a bit like JFFS2 on a block emulator , yeah ?

;-)

Gareth.

Re: JFFS2 mount time

[jasmine]

On Fri, 17 Dec 2004, Gareth Bult (Encryptec) wrote:

> Mmm, sounds a bit like JFFS2 on a block emulator , yeah ?

Only if you actively enjoy wasting processor cycles (and microjoules) and 
ROM footprint...

-J.

Re: JFFS2 mount time

[Gareth Bult (Encryptec)]

On Fri, 2004-12-17 at 19:14 +0000, jasmine@linuxgrrls.org wrote:
> On Fri, 17 Dec 2004, Gareth Bult (Encryptec) wrote:
> 
> > Mmm, sounds a bit like JFFS2 on a block emulator , yeah ?
> 
> Only if you actively enjoy wasting processor cycles (and microjoules) and 
> ROM footprint...

Processor cycles, got em to spare ..
Microjoules coming out of my ears ..
ROM footprint .. don't use 'em ..

What's the alternative then .. :)

Gareth.

Re: JFFS2 mount time

[Jörn Engel]

On Fri, 17 December 2004 11:35:04 -0600, Josh Boyer wrote:
> 
> Again, most controllers do all the wear leveling, erasing for you.  So 
> you are left with compression, right?

If you have enough trust in your El Cheapo hardware supplier.  There
is no standard that requires wear leveling and there are reports that
basically tell you to do your own wear leveling on some devices.  With
no good way to distinguish "some" devices from "others", you can read
that as "all". ;)

USB keys and all the various other consumer-type flash devices are
good enough to beat 3 1/2" floppies, that's about it.  But that's
quite good enough for most people.  Floppies sold like crazy after
all.

Jörn

-- 
Simplicity is prerequisite for reliability.
-- Edsger W. Dijkstra

Re: JFFS2 mount time

[Josh Boyer]

On Sat, 2004-12-18 at 10:02, Jörn Engel wrote:
> On Fri, 17 December 2004 11:35:04 -0600, Josh Boyer wrote:
> > 
> > Again, most controllers do all the wear leveling, erasing for you.  So 
> > you are left with compression, right?
> 
> If you have enough trust in your El Cheapo hardware supplier.  There
> is no standard that requires wear leveling and there are reports that
> basically tell you to do your own wear leveling on some devices.  With
> no good way to distinguish "some" devices from "others", you can read
> that as "all". ;)

Ok, that's why I said "most".  And if your El Cheapo hardware has built
in wear leveling, then doing wear leveling on top of that is always
questionable.  It might not make things worse, but it's probably not
very efficient.

Now if you have El Super Cheapo hardware that explicitly states you need
to do wear leveling, that's a different story ;).

> 
> USB keys and all the various other consumer-type flash devices are
> good enough to beat 3 1/2" floppies, that's about it.  But that's
> quite good enough for most people.  Floppies sold like crazy after
> all.

That's because it was the only removable media for PCs for a long time. 
At least to the non-geeks.  I can remember buying spare hard drives and
carrying those from machine to machine if I needed to do big transfers
;).

josh

Re: JFFS2 mount time

[Gareth Bult (Encryptec)]

On Mon, 2004-12-20 at 10:34 -0600, Josh Boyer wrote:
> Ok, that's why I said "most".  And if your El Cheapo hardware has built
> in wear leveling, then doing wear leveling on top of that is always
> questionable.  It might not make things worse, but it's probably not
> very efficient.

Mmm, if the USB flash micro-controllers have build-in wear levelling,
they'd be quite something .. (!) 

> Now if you have El Super Cheapo hardware that explicitly states you need
> to do wear leveling, that's a different story ;).

Here's a quote from a "Kingston" advert;

"Controllers automatically lock out bad memory cells and move the data
to avoid corruption. Controllers also automatically distribute write
cycles across the flash cells to extend the life of the flash memory
card"

What this means exactly in real terms .. [?]
(for example, do you lose some of your key to a bad block table, if so
how much ???)

Here's a bit from "Verbatim";

"For ultra-reliability, Verbatim’s new Store ‘n’ Go Pro drives feature
an on-board 32 bit ARM-7 microprocessor that manages I/O operations and
many of the drive’s technical features. An advanced wear-leveling
algorithm is used to distribute writes evenly among flash storage cells.
By evenly distributing the writing, Store ‘n’ Go Pro achieves an
unprecedented reliability rating of over one million write/erase cycles.
Advanced Error Detection Code/Error Correction Code furthers the Store
‘n’ Go Pro’s immunity to failure."

Interesting articles / links;
http://www.digitalmediadesigner.com/articles/viewarticle.jsp?id=27727
http://www.techworld.com/features/index.cfm?fuseaction=displayfeature&FeatureID=413

In summary: lots of manufacturers seem to be quoting virtual / physical
block mapping on the fly with integrated wear levelling ... (!)

So maybe wear levelling is out of date at filesystem level ?
The techworld link to the M-Systems chips looks interesting ..

Can anyone with detailed flash experience comment ?

Gareth.

> That's because it was the only removable media for PCs for a long time. 
> At least to the non-geeks.  I can remember buying spare hard drives and
> carrying those from machine to machine if I needed to do big transfers
> ;).
> 
> josh
> 

Re: JFFS2 mount time

[Jörn Engel]

On Mon, 20 December 2004 17:12:28 +0000, Gareth Bult (Encryptec) wrote:
> On Mon, 2004-12-20 at 10:34 -0600, Josh Boyer wrote:
> > Ok, that's why I said "most".  And if your El Cheapo hardware has built
> > in wear leveling, then doing wear leveling on top of that is always
> > questionable.  It might not make things worse, but it's probably not
> > very efficient.

Imo jffs2-style wear leveling is very efficient.  So efficient, it
doesn't really matter whether the hardware does the same or not.

Difference between no wear leveling and what jffs2 does is - at most -
1% additional writes on the jffs2 side.  That's not much.  Agreed, it
can actually cost latency, but from a life-time or overall performance
point of view, it doesn't really matter.

On the other hand, it gives you absolute condidence that your El
Cheapo hardware doesn't need to have anything it may or may not
promise to have.  No nasty surprised when you really don't need them.

> > Now if you have El Super Cheapo hardware that explicitly states you need
> > to do wear leveling, that's a different story ;).

Noone will state that.  It's not exactly good marketing.  You might
remember so interesting stories about what marketing promised and what
actually got delivered, no?

> In summary: lots of manufacturers seem to be quoting virtual / physical
> block mapping on the fly with integrated wear levelling ... (!)
> 
> So maybe wear levelling is out of date at filesystem level ?
> The techworld link to the M-Systems chips looks interesting ..
> 
> Can anyone with detailed flash experience comment ?

The endorsed features sound quite nice, agreed.  But I have still to
read a spec that convinces me.

Jörn

-- 
My second remark is that our intellectual powers are rather geared to
master static relations and that our powers to visualize processes
evolving in time are relatively poorly developed.
-- Edsger W. Dijkstra

Re: JFFS2 mount time

[Gareth Bult (Encryptec)]

[-- Attachment #1: Type: text/plain, Size: 1335 bytes --]

Hi,

On Tue, 2004-12-21 at 14:09 +0100, Jörn Engel wrote:
> Imo jffs2-style wear leveling is very efficient.  So efficient, it
> doesn't really matter whether the hardware does the same or not.

:) I was of the same opinion but didn't really have any evidence to back
it up .. 

> Difference between no wear leveling and what jffs2 does is - at most -
> 1% additional writes on the jffs2 side.  That's not much.  Agreed, it
> can actually cost latency, but from a life-time or overall performance
> point of view, it doesn't really matter.

Sure.

> On the other hand, it gives you absolute condidence that your El
> Cheapo hardware doesn't need to have anything it may or may not
> promise to have.  No nasty surprised when you really don't need them.

:)

> Noone will state that.  It's not exactly good marketing.  You might
> remember so interesting stories about what marketing promised and what
> actually got delivered, no?

Sure, although I have noticed that some people shout more loudly than
others with regards to the wear levelling technology in their devices ..

> The endorsed features sound quite nice, agreed.  But I have still to
> read a spec that convinces me.

Mmm .. anyone know a magazine editor who'd like to acquire a bunch of
keys and runs some tests for us .. ;-)

Gareth.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: JFFS2 mount time

[Jörn Engel]

On Tue, 21 December 2004 13:24:35 +0000, Gareth Bult (Encryptec) wrote:
> 
> Mmm .. anyone know a magazine editor who'd like to acquire a bunch of
> keys and runs some tests for us .. ;-)

Somewhere in my lkml-mailbox was a report of some device not doing
decent wear leveling.  It should be in the archives somewhere.  Ext3
killed the devices rather quickly, jffs2 didn't.

Jörn

-- 
Data dominates. If you've chosen the right data structures and organized
things well, the algorithms will almost always be self-evident. Data
structures, not algorithms, are central to programming.
-- Rob Pike

Re: JFFS2 mount time

[Jörn Engel]

On Thu, 16 December 2004 21:47:49 +0000, Gareth Bult (Encryptec) wrote:
> 
> blkmtd.c is pretty much a rewrite, less than half the size and uses the
> system buffer cache instead of reading / writing directly to the device.
> This boosts performance x 3 on read x lots on write. [I get x 100
> faster] (albeit you lose bad block detection on write)
> 
> If anyone's interested in the code, let me know and I'll clean it up and
> post it .. however it'll be a new file and not a patch.

Sure am!  If the code lives up to your promises I'd like to read it
(might learn something new) and possibly use it sometime soon.

Jörn

-- 
The competent programmer is fully aware of the strictly limited size of
his own skull; therefore he approaches the programming task in full
humility, and among other things he avoids clever tricks like the plague. 
-- Edsger W. Dijkstra

Re: JFFS2 mount time

[Gareth Bult (Encryptec)]

[-- Attachment #1: Type: text/plain, Size: 1098 bytes --]

Ok,

There's a public health warning attached to this code ..

I've not cleaned it up and I've not added any comments ..

(!)

fyi; I don't use mkfs.jffs2 .. I flash_eraseall the device then work
from there with "tar".. there seems to be no benefit [for me] to using
mkfs and it seems a little more reliant on the device size than the
erase program.

Gareth.

On Sat, 2004-12-18 at 17:19 +0100, Jörn Engel wrote:
> On Thu, 16 December 2004 21:47:49 +0000, Gareth Bult (Encryptec) wrote:
> > 
> > blkmtd.c is pretty much a rewrite, less than half the size and uses the
> > system buffer cache instead of reading / writing directly to the device.
> > This boosts performance x 3 on read x lots on write. [I get x 100
> > faster] (albeit you lose bad block detection on write)
> > 
> > If anyone's interested in the code, let me know and I'll clean it up and
> > post it .. however it'll be a new file and not a patch.
> 
> Sure am!  If the code lives up to your promises I'd like to read it
> (might learn something new) and possibly use it sometime soon.
> 
> Jörn
> 

[-- Attachment #2: blkmtd.c --]
[-- Type: text/x-csrc, Size: 16190 bytes --]

/*
 * $Id: blkmtd.c,v 1.23 2004/08/09 14:03:19 dwmw2 Exp $
 *
 * blkmtd.c - use a block device as a fake MTD
 *
 * Author: Simon Evans <spse@secret.org.uk>
 *
 * Copyright (C) 2001,2002 Simon Evans
 *
 * Licence: GPL
 *
 * How it works:
 *	The driver uses raw/io to read/write the device and the page
 *	cache to cache access. Writes update the page cache with the
 *	new data and mark it dirty and add the page into a BIO which
 *	is then written out.
 *
 *	It can be loaded Read-Only to prevent erases and writes to the
 *	medium.
 *
 */

#include <linux/config.h>
#include <linux/module.h>
#include <linux/fs.h>
#include <linux/blkdev.h>
#include <linux/bio.h>
#include <linux/pagemap.h>
#include <linux/list.h>
#include <linux/init.h>
#include <linux/mtd/mtd.h>
#include <linux/buffer_head.h>

#define err(format, arg...) printk(KERN_ERR "blkmtd: " format "\n" , ## arg)
#define info(format, arg...) printk(KERN_INFO "blkmtd: " format "\n" , ## arg)
#define warn(format, arg...) printk(KERN_WARNING "blkmtd: " format "\n" , ## arg)
#define crit(format, arg...) printk(KERN_CRIT "blkmtd: " format "\n" , ## arg)


/* Default erase size in K, always make it a multiple of PAGE_SIZE */
#define CONFIG_MTD_BLKDEV_ERASESIZE (128 << 10)	/* 128KiB */
#define VERSION "$Revision: 1.23 $"

/* Info for the block device */
struct blkmtd_dev {
	struct list_head list;
	struct block_device *blkdev;
	struct mtd_info mtd_info;
	struct semaphore wrbuf_mutex;
};


/* Static info about the MTD, used in cleanup_module */
static LIST_HEAD(blkmtd_device_list);

static void blkmtd_sync(struct mtd_info *mtd);

#define MAX_DEVICES 4

/* Module parameters passed by insmod/modprobe */
char *device[MAX_DEVICES];    /* the block device to use */
int erasesz[MAX_DEVICES];     /* optional default erase size */
int ro[MAX_DEVICES];          /* optional read only flag */
int sync;


MODULE_LICENSE("GPL");
MODULE_AUTHOR("Simon Evans <spse@secret.org.uk>");
MODULE_DESCRIPTION("Emulate an MTD using a block device");
MODULE_PARM(device, "1-4s");
MODULE_PARM_DESC(device, "block device to use");
MODULE_PARM(erasesz, "1-4i");
MODULE_PARM_DESC(erasesz, "optional erase size to use in KiB. eg 4=4KiB.");
MODULE_PARM(ro, "1-4i");
MODULE_PARM_DESC(ro, "1=Read only, writes and erases cause errors");
MODULE_PARM(sync, "i");
MODULE_PARM_DESC(sync, "1=Synchronous writes");

#define PAGE_READAHEAD 64

void cache_readahead(struct address_space *mapping, int index)
{
	filler_t *filler = (filler_t*)(mapping->a_ops->readpage);
	int i,pagei;
	unsigned ret = 0;
	unsigned long end_index;
	struct page *page;
	LIST_HEAD(page_pool);
	struct inode *inode = mapping->host;
	loff_t isize = i_size_read(inode);

	if(!isize) {
		printk(KERN_INFO "iSize=0 in cache_readahead\n");
		return;
	}

	end_index = ((isize - 1) >> PAGE_CACHE_SHIFT);

	spin_lock_irq(&mapping->tree_lock);
	for(i=0;i<PAGE_READAHEAD;i++) {
		pagei=index+i;
		if (pagei > end_index) {
			printk(KERN_INFO "Overrun end of disk in cache readahead\n");
			break;
		}
		page = radix_tree_lookup(&mapping->page_tree, pagei);
		if(page && (!i)) break;
		if(page) continue;
		spin_unlock_irq(&mapping->tree_lock);
		page = page_cache_alloc_cold(mapping);
		spin_lock_irq(&mapping->tree_lock);
		if(!page) break;
		page->index=pagei;
		list_add(&page->lru, &page_pool);
		ret++;
	}
	spin_unlock_irq(&mapping->tree_lock);
	if(ret) {
		read_cache_pages(mapping, &page_pool, filler, NULL);
		//printk(KERN_INFO "Readahead [%d/%d] pages from [%d]\n",ret,PAGE_READAHEAD,index);
	}
}

static struct page* page_readahead(struct address_space *mapping, int index)
{
	filler_t *filler = (filler_t*)(mapping->a_ops->readpage);
	cache_readahead(mapping,index);
	return read_cache_page(mapping, index, filler, NULL);
}
/**
 * write_pages - write block of data to device via the page cache
 * @dev: device to write to
 * @buf: data source or NULL if erase (output is set to 0xff)
 * @to: offset into output device
 * @len: amount to data to write
 * @retlen: amount of data written
 *
 * Grab pages from the page cache and fill them with the source data.
 * Non page aligned start and end result in a readin of the page and
 * part of the page being modified. Pages are added to the bio and then written
 * out.
 */

static int write_pages(struct blkmtd_dev *dev, const u_char *buf, loff_t to, size_t len, size_t *retlen)
{
	struct	page *page;
	struct	address_space *mapping = dev->blkdev->bd_inode->i_mapping;
	//
	int	err	= 0;			// return status
	int	index	= to >> PAGE_SHIFT;	// page index
	int	offset 	= to & ~PAGE_MASK;	// page offset
	//

	static int my_read()
	{
		err=0;
		page = page_readahead(mapping,index);
		if( unlikely(!page) ) {
			crit("unable to read page (%d) in write_pages\n",index);
			err=-ENOMEM;
		}
		//if(IS_ERR(page)) err=-EIO;
		if(err) printk("READ ERROR\n");
		return err;
	}
	//
	static int my_erase()
	{
		int pages = len >> PAGE_SHIFT;
		unsigned char *p;
		unsigned char *max;
		int format;

		while(pages) {
			//printk(KERN_INFO "readahead on index (%d)\n",index);
			if(my_read()) return err;

			max = ((char*)page_address(page)+PAGE_SIZE);
			format = 0;

			for(p=(unsigned char*)page_address(page); p<max; p++) 
				if(*p != 0xff) {
					format=1;
					break;
				}

			if(format) {
				//DEBUG(3, "write_pages: erasing\n");
				lock_page(page);
				memset(page_address(page), 0xff, PAGE_SIZE);
				set_page_dirty(page);
				unlock_page(page);
			}
			page_cache_release(page);
			pages--;
			index++;
		}
		*retlen = len;
		return 0;
	}

	static int my_write()
	{
		int cpylen;

		if(retlen) *retlen = 0;
		//printk(KERN_INFO "Write req: %d\n",len);
		while(len) {
			if((offset+len) > PAGE_SIZE) 
				cpylen = PAGE_SIZE - offset;	// multiple pages
			else	cpylen = len;			// this page
			len = len - cpylen;
			//
			//	Get page
			//
			if(my_read()) return err;
			//
			if(memcmp(page_address(page)+offset,buf,cpylen)) {
				lock_page(page);
				memcpy(page_address(page) + offset,buf, cpylen);
				set_page_dirty(page);
				unlock_page(page);
			}
			page_cache_release(page);

			if(retlen) *retlen += cpylen;
			//
			buf+=cpylen;
			offset = 0;
			index++;
			//
		}
		//printk(KERN_INFO "Write act: %d\n",*retlen);
		return 0;
	}

	err=0;
	down(&dev->wrbuf_mutex);
	err = buf ? my_write() : my_erase();
	up(&dev->wrbuf_mutex);
	return err;
}

/* erase a specified part of the device */
static int blkmtd_erase(struct mtd_info *mtd, struct erase_info *instr)
{
	struct blkmtd_dev *dev = mtd->priv;
	struct mtd_erase_region_info *einfo = mtd->eraseregions;
	int numregions = mtd->numeraseregions;
	size_t from;
	u_long len;
	int err = -EIO;
	size_t retlen;

	instr->state = MTD_ERASING;
	from = instr->addr;
	len = instr->len;

	/* check erase region has valid start and length */
	//DEBUG(2, "blkmtd: erase: dev = `%s' from = 0x%zx len = 0x%lx\n", mtd->name+9, from, len);
	while(numregions) {
		DEBUG(3, "blkmtd: checking erase region = 0x%08X size = 0x%X num = 0x%x\n",
		      einfo->offset, einfo->erasesize, einfo->numblocks);
		if(from >= einfo->offset
		   && from < einfo->offset + (einfo->erasesize * einfo->numblocks)) {
			if(len == einfo->erasesize
			   && ( (from - einfo->offset) % einfo->erasesize == 0))
				break;
		}
		numregions--;
		einfo++;
	}

	if(!numregions) {
		/* Not a valid erase block */
		err("erase: invalid erase request 0x%lX @ 0x%08zX", len, from);
		instr->state = MTD_ERASE_FAILED;
		err = -EIO;
	}

	if(instr->state != MTD_ERASE_FAILED) {
		/* do the erase */
		DEBUG(3, "Doing erase from = %zd len = %ld\n", from, len);
		err = write_pages(dev, NULL, from, len, &retlen);
		if(err || retlen != len) {
			err("erase failed err = %d", err);
			instr->state = MTD_ERASE_FAILED;
		} else {
			instr->state = MTD_ERASE_DONE;
		}
	}

	DEBUG(3, "blkmtd: erase: checking callback\n");
	mtd_erase_callback(instr);
	DEBUG(2, "blkmtd: erase: finished (err = %d)\n", err);
	return err;
}

static int blkmtd_read(struct mtd_info *mtd, loff_t from, size_t len, size_t *retlen, u_char *buf)
{
	struct blkmtd_dev *dev = mtd->priv;
	struct page *page;
	//
	int index	= from >> PAGE_SHIFT;
	int offset	= from - (index << PAGE_SHIFT);
	int cpylen;

	if(from > mtd->size) return -EINVAL;
	if(from + len > mtd->size) len = mtd->size - from;

	if(retlen) *retlen = 0;

	while(len) {
		if((offset+len) > PAGE_SIZE) 
			cpylen = PAGE_SIZE - offset;	// multiple pages
		else	cpylen = len;			// this page
		len = len - cpylen;
		//
		//	Get page
		//
		page = page_readahead(dev->blkdev->bd_inode->i_mapping,index);
		if(!page) return -ENOMEM;
		if(IS_ERR(page)) return -EIO;
		//
		memcpy(buf, page_address(page) + offset, cpylen);
		page_cache_release(page);
		//
		if(retlen) *retlen += cpylen;
		buf+=cpylen;
		offset = 0;
		index++;
		//
	}
	return 0;
}


/* write data to the underlying device */
static int blkmtd_write(struct mtd_info *mtd, loff_t to, size_t len,
			size_t *retlen, const u_char *buf)
{
	struct blkmtd_dev *dev = mtd->priv;
	int err;

	if(!len) return 0;
	if(to >= mtd->size) return -ENOSPC;
	if(to + len > mtd->size) len = mtd->size - to;

	err = write_pages(dev, buf, to, len, retlen);
	if(err > 0) err = 0;
	return err;
}


/* sync the device - wait until the write queue is empty */
static void blkmtd_sync(struct mtd_info *mtd)
{
	sync_blockdev(((struct blkmtd_dev*)mtd->priv)->blkdev);
	return;
}

static void free_device(struct blkmtd_dev *dev)
{
	DEBUG(2, "blkmtd: free_device() dev = %p\n", dev);
	if(dev) {
		if(dev->mtd_info.eraseregions)
			kfree(dev->mtd_info.eraseregions);
		if(dev->mtd_info.name)
			kfree(dev->mtd_info.name);

		if(dev->blkdev) {
			invalidate_inode_pages(dev->blkdev->bd_inode->i_mapping);
			close_bdev_excl(dev->blkdev);
		}
		kfree(dev);
	}
}


/* For a given size and initial erase size, calculate the number
 * and size of each erase region. Goes round the loop twice,
 * once to find out how many regions, then allocates space,
 * then round the loop again to fill it in.
 */
static struct mtd_erase_region_info *calc_erase_regions(
	size_t erase_size, size_t total_size, int *regions)
{
	struct mtd_erase_region_info *info = NULL;

	DEBUG(2, "calc_erase_regions, es = %zd size = %zd regions = %d\n",
	      erase_size, total_size, *regions);
	/* Make any user specified erasesize be a power of 2
	   and at least PAGE_SIZE */
	if(erase_size) {
		int es = erase_size;
		erase_size = 1;
		while(es != 1) {
			es >>= 1;
			erase_size <<= 1;
		}
		if(erase_size < PAGE_SIZE)
			erase_size = PAGE_SIZE;
	} else {
		erase_size = CONFIG_MTD_BLKDEV_ERASESIZE;
	}

	*regions = 0;

	do {
		int tot_size = total_size;
		int er_size = erase_size;
		int count = 0, offset = 0, regcnt = 0;

		while(tot_size) {
			count = tot_size / er_size;
			if(count) {
				tot_size = tot_size % er_size;
				if(info) {
					DEBUG(2, "adding to erase info off=%d er=%d cnt=%d\n",
					      offset, er_size, count);
					(info+regcnt)->offset = offset;
					(info+regcnt)->erasesize = er_size;
					(info+regcnt)->numblocks = count;
					(*regions)++;
				}
				regcnt++;
				offset += (count * er_size);
			}
			while(er_size > tot_size)
				er_size >>= 1;
		}
		if(info == NULL) {
			info = kmalloc(regcnt * sizeof(struct mtd_erase_region_info), GFP_KERNEL);
			if(!info)
				break;
		}
	} while(!(*regions));
	DEBUG(2, "calc_erase_regions done, es = %zd size = %zd regions = %d\n",
	      erase_size, total_size, *regions);
	return info;
}


extern dev_t __init name_to_dev_t(const char *line);

static struct blkmtd_dev *add_device(char *devname, int readonly, int erase_size)
{
	struct block_device *bdev;
	int mode;
	struct blkmtd_dev *dev;

	if(!devname)
		return NULL;

	/* Get a handle on the device */


#ifdef MODULE
	mode = (readonly) ? O_RDONLY : O_RDWR;
	bdev = open_bdev_excl(devname, mode, NULL);
#else
	mode = (readonly) ? FMODE_READ : FMODE_WRITE;
	bdev = open_by_devnum(name_to_dev_t(devname), mode);
#endif
	if(IS_ERR(bdev)) {
		err("error: cannot open device %s", devname);
		DEBUG(2, "blkmtd: opening bdev returned %ld\n", PTR_ERR(bdev));
		return NULL;
	}

	DEBUG(1, "blkmtd: found a block device major = %d, minor = %d\n",
	      MAJOR(bdev->bd_dev), MINOR(bdev->bd_dev));

	if(MAJOR(bdev->bd_dev) == MTD_BLOCK_MAJOR) {
		err("attempting to use an MTD device as a block device");
		blkdev_put(bdev);
		return NULL;
	}

	dev = kmalloc(sizeof(struct blkmtd_dev), GFP_KERNEL);
	if(dev == NULL) {
		blkdev_put(bdev);
		return NULL;
	}

	memset(dev, 0, sizeof(struct blkmtd_dev));
	dev->blkdev = bdev;
	atomic_set(&(dev->blkdev->bd_inode->i_mapping->truncate_count), 0);
	if(!readonly) {
		init_MUTEX(&dev->wrbuf_mutex);
	}

	dev->mtd_info.size = dev->blkdev->bd_inode->i_size & PAGE_MASK;

	/* Setup the MTD structure */
	/* make the name contain the block device in */
	dev->mtd_info.name = kmalloc(sizeof("blkmtd: ") + strlen(devname), GFP_KERNEL);
	if(dev->mtd_info.name == NULL)
		goto devinit_err;

	sprintf(dev->mtd_info.name, "blkmtd: %s", devname);
	dev->mtd_info.eraseregions = calc_erase_regions(erase_size, dev->mtd_info.size,
							&dev->mtd_info.numeraseregions);
	if(dev->mtd_info.eraseregions == NULL)
		goto devinit_err;

	dev->mtd_info.erasesize = dev->mtd_info.eraseregions->erasesize;
	DEBUG(1, "blkmtd: init: found %d erase regions\n",
	      dev->mtd_info.numeraseregions);

	if(readonly) {
		dev->mtd_info.type = MTD_ROM;
		dev->mtd_info.flags = MTD_CAP_ROM;
	} else {
		dev->mtd_info.type = MTD_RAM;
		dev->mtd_info.flags = MTD_CAP_RAM;
		dev->mtd_info.erase = blkmtd_erase;
		dev->mtd_info.write = blkmtd_write;
		dev->mtd_info.writev = default_mtd_writev;
		dev->mtd_info.sync = blkmtd_sync;
	}
	dev->mtd_info.read = blkmtd_read;
	dev->mtd_info.readv = default_mtd_readv;
	dev->mtd_info.priv = dev;
	dev->mtd_info.owner = THIS_MODULE;

	list_add(&dev->list, &blkmtd_device_list);
	if (add_mtd_device(&dev->mtd_info)) {
		/* Device didnt get added, so free the entry */
		list_del(&dev->list);
		goto devinit_err;
	} else {
		info("mtd%d: [%s] erase_size = %dKiB %s [%ld]",
		     dev->mtd_info.index, dev->mtd_info.name + strlen("blkmtd: "),
		     dev->mtd_info.erasesize >> 10,
		     readonly ? "(read-only)" : "", PAGE_SIZE);
	}
	return dev;

 devinit_err:
	free_device(dev);
	return NULL;
}


/* Cleanup and exit - sync the device and kill of the kernel thread */
static void __devexit cleanup_blkmtd(void)
{
	struct list_head *temp1, *temp2;

	/* Remove the MTD devices */
	list_for_each_safe(temp1, temp2, &blkmtd_device_list) {
		struct blkmtd_dev *dev = list_entry(temp1, struct blkmtd_dev,
						    list);
		blkmtd_sync(&dev->mtd_info);
		del_mtd_device(&dev->mtd_info);
		info("mtd%d: [%s] removed", dev->mtd_info.index,
		     dev->mtd_info.name + strlen("blkmtd: "));
		list_del(&dev->list);
		free_device(dev);
	}
}

#ifndef MODULE

/* Handle kernel boot params */


static int __init param_blkmtd_device(char *str)
{
	int i;

	for(i = 0; i < MAX_DEVICES; i++) {
		device[i] = str;
		DEBUG(2, "blkmtd: device setup: %d = %s\n", i, device[i]);
		strsep(&str, ",");
	}
	return 1;
}


static int __init param_blkmtd_erasesz(char *str)
{
	int i;
	for(i = 0; i < MAX_DEVICES; i++) {
		char *val = strsep(&str, ",");
		if(val)
			erasesz[i] = simple_strtoul(val, NULL, 0);
		DEBUG(2, "blkmtd: erasesz setup: %d = %d\n", i, erasesz[i]);
	}

	return 1;
}


static int __init param_blkmtd_ro(char *str)
{
	int i;
	for(i = 0; i < MAX_DEVICES; i++) {
		char *val = strsep(&str, ",");
		if(val)
			ro[i] = simple_strtoul(val, NULL, 0);
		DEBUG(2, "blkmtd: ro setup: %d = %d\n", i, ro[i]);
	}

	return 1;
}


static int __init param_blkmtd_sync(char *str)
{
	if(str[0] == '1')
		sync = 1;
	return 1;
}

__setup("blkmtd_device=", param_blkmtd_device);
__setup("blkmtd_erasesz=", param_blkmtd_erasesz);
__setup("blkmtd_ro=", param_blkmtd_ro);
__setup("blkmtd_sync=", param_blkmtd_sync);

#endif


/* Startup */
static int __init init_blkmtd(void)
{
	int i;

	info("version " VERSION);
	/* Check args - device[0] is the bare minimum*/
	if(!device[0]) {
		err("error: missing `device' name\n");
		return -EINVAL;
	}

	for(i = 0; i < MAX_DEVICES; i++)
		add_device(device[i], ro[i], erasesz[i] << 10);

	if(list_empty(&blkmtd_device_list))
		return -EINVAL;

	return 0;
}

module_init(init_blkmtd);
module_exit(cleanup_blkmtd);

Re: JFFS2 mount time

[Jörn Engel]

On Sat, 18 December 2004 17:32:35 +0000, Gareth Bult (Encryptec) wrote:
> 
> There's a public health warning attached to this code ..
> 
> I've not cleaned it up and I've not added any comments ..

I'll wear protective gloves then.  Thanks!

> fyi; I don't use mkfs.jffs2 .. I flash_eraseall the device then work
> from there with "tar".. there seems to be no benefit [for me] to using
> mkfs and it seems a little more reliant on the device size than the
> erase program.

Makes a lot of sense.  Both approaches have some advantages.

Jörn

-- 
The story so far:
In the beginning the Universe was created.  This has made a lot
of people very angry and been widely regarded as a bad move.
-- Douglas Adams?

Re: JFFS2 mount time

[Jörn Engel]

On Sat, 18 December 2004 18:52:22 +0100, Jörn Engel wrote:
> On Sat, 18 December 2004 17:32:35 +0000, Gareth Bult (Encryptec) wrote:
> > 
> > There's a public health warning attached to this code ..
> > 
> > I've not cleaned it up and I've not added any comments ..
> 
> I'll wear protective gloves then.  Thanks!

Not a pretty.  Looks like quite a few cleanups from phram.c can be
applied to this one as well.  From a first glance:
o Coding style (Lindent?)
o Function length
o #ifdef MODULE
o parameters have an insane interface
o ro-devices.  the same can be achieved with chmod.

But you started the work, which is a good thing.

Jörn

-- 
Public Domain  - Free as in Beer
General Public - Free as in Speech
BSD License    - Free as in Enterprise
Shared Source  - Free as in "Work will make you..."

Re: JFFS2 mount time

[Gareth Bult (Encryptec)]

Like I said, hacked to work [for me] and not to look pretty.

To be honest what's there will do me for now .. although I'd like to
clean it up a little when I get time.

fyi; don't confuse the old and new code.

> o Coding style (Lindent?)

Whatever was there already.

> o Function length

My functions are shorter than the ones that were there, lots can be
cleaned up here.

> o #ifdef MODULE

.. ? I only use it as a module, not of this code was touched.

> o parameters have an insane interface

.. again , parameters and interfaces are all from the original with the
exception of a couple of new routines which mirror existing parameter
lists.

> o ro-devices.  the same can be achieved with chmod.

Again, not touched this.

Gareth.

On Sat, 2004-12-18 at 19:11 +0100, Jörn Engel wrote:
> On Sat, 18 December 2004 18:52:22 +0100, Jörn Engel wrote:
> > On Sat, 18 December 2004 17:32:35 +0000, Gareth Bult (Encryptec) wrote:
> > > 
> > > There's a public health warning attached to this code ..
> > > 
> > > I've not cleaned it up and I've not added any comments ..
> > 
> > I'll wear protective gloves then.  Thanks!
> 
> Not a pretty.  Looks like quite a few cleanups from phram.c can be
> applied to this one as well.  From a first glance:
> o Coding style (Lindent?)
> o Function length
> o #ifdef MODULE
> o parameters have an insane interface
> o ro-devices.  the same can be achieved with chmod.
> 
> But you started the work, which is a good thing.
> 
> Jörn
> 

Re: JFFS2 mount time

[Jörn Engel]

On Sat, 18 December 2004 20:48:50 +0000, Gareth Bult (Encryptec) wrote:
> 
> Like I said, hacked to work [for me] and not to look pretty.
> 
> To be honest what's there will do me for now .. although I'd like to
> clean it up a little when I get time.
> 
> fyi; don't confuse the old and new code.

Sure.  I had a look at tkdiff and there was ~50% unchanged code.
"This code ain't pretty." didn't mean "You sh*thead are stupid and
cannot create decent code."  It was more like "It still ain't pretty,
although you already improved it."

> > o Coding style (Lindent?)
> 
> Whatever was there already.
> 
> > o Function length
> 
> My functions are shorter than the ones that were there, lots can be
> cleaned up here.
> 
> > o #ifdef MODULE
> 
> .. ? I only use it as a module, not of this code was touched.
> 
> > o parameters have an insane interface
> 
> .. again , parameters and interfaces are all from the original with the
> exception of a couple of new routines which mirror existing parameter
> lists.
> 
> > o ro-devices.  the same can be achieved with chmod.
> 
> Again, not touched this.

Yep.  Those five point were the obvious stuff I could find in five
minutes.  Fixing them takes a lot more than five minutes and changing
the parameter interface basically means copying the thing and fixing
the new one.

I did the same with slram/phram.  The slram interface was different
for module/non-module built and insane once you had more than a single
device.  Mtdblock looks quite similar and seems to need the same cure.
So whatever you can improve, please do.

Jörn

-- 
It's just what we asked for, but not what we want!
-- anonymous

Re: JFFS2 mount time

[Jörn Engel]

On Sat, 18 December 2004 20:48:50 +0000, Gareth Bult (Encryptec) wrote:
> 
> To be honest what's there will do me for now .. although I'd like to
> clean it up a little when I get time.

Guess you've never been patch-bombed before.  I took the liberty to do
some untested cleanups in ~20 patches.  Most likely the end result
will not work anymore, but the direction is where I'd like to go.

If you find the time, could you test the patches and see which work
and which don't?  The former will go into cvs then and I'll try to fix
the latter.

Jörn

-- 
It does not matter how slowly you go, so long as you do not stop.
-- Confucius

Re: JFFS2 mount time

[Gareth Bult (Encryptec)]

[-- Attachment #1: Type: text/plain, Size: 686 bytes --]

.. and I find the patches where ?

On Tue, 2004-12-21 at 14:30 +0100, Jörn Engel wrote:
> On Sat, 18 December 2004 20:48:50 +0000, Gareth Bult (Encryptec) wrote:
> > 
> > To be honest what's there will do me for now .. although I'd like to
> > clean it up a little when I get time.
> 
> Guess you've never been patch-bombed before.  I took the liberty to do
> some untested cleanups in ~20 patches.  Most likely the end result
> will not work anymore, but the direction is where I'd like to go.
> 
> If you find the time, could you test the patches and see which work
> and which don't?  The former will go into cvs then and I'll try to fix
> the latter.
> 
> Jörn
> 

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: JFFS2 mount time

[David Woodhouse]

On Tue, 2004-12-21 at 13:40 +0000, Gareth Bult (Encryptec) wrote:
> .. and I find the patches where ?

On the list after I approved all 22 of them; they were trapped for
moderation because they had attachments. After looking at one or two of
yours which were also trapped, I approved them all because I thought
they were all due to the GPG signature. Unfortunately I let through an
HTML mail. Please don't send those -- and please don't top-post either.

-- 
dwmw2

Re: JFFS2 mount time

[Jörn Engel]

On Tue, 21 December 2004 16:02:25 +0000, Gareth Bult (Encryptec) wrote:
> 
> Ahhhhhh!

I get that a lot. ;)

Jörn

-- 
Fantasy is more important than knowledge. Knowledge is limited,
while fantasy embraces the whole world.
-- Albert Einstein

Re: JFFS2 mount time

[Ferenc Havasi]

[-- Attachment #1: Type: text/plain, Size: 343 bytes --]

Hi Gareth,

> However, I'm looking at 30secs to mount the root filesystem.

Did you try our patch? I send it now again, we made it fresh for the 
current CVS. See its usage in the archive (16th November).

It will part of JFFS3.

This is designed for NAND devices - I don't know too much about your 
environment, but it may help.

Bye,
Ferenc

[-- Attachment #2: jffs2-summary-20041207.patch --]
[-- Type: text/plain, Size: 88751 bytes --]

diff --unified --new-file --recursive mtd/fs/Kconfig mtd-dec07/fs/Kconfig
--- mtd/fs/Kconfig	2004-11-04 00:00:11.000000000 +0100
+++ mtd-dec07/fs/Kconfig	2004-12-07 16:03:02.000000000 +0100
@@ -77,6 +77,19 @@
 		 ECC for JFFS2.  This type of flash chip is not common, however it is
 		 available from STMicro.
 
+config JFFS2_SUMMARY
+        bool "JFFS2 summary support (EXPERIMENTAL)" 
+        depends on JFFS2_FS
+        default n
+        help 
+          This feature makes it possible to use summary information
+          for faster filesystem mount - specially on NAND.
+
+          The summary information can be inserted into a filesystem image
+          by the utility 'sumtool'.
+
+          If unsure, say 'N'.
+
 config JFFS2_COMPRESSION_OPTIONS
 	bool "Advanced compression options for JFFS2"
 	default n
diff --unified --new-file --recursive mtd/fs/jffs2/Makefile.common mtd-dec07/fs/jffs2/Makefile.common
--- mtd/fs/jffs2/Makefile.common	2004-11-04 00:00:11.000000000 +0100
+++ mtd-dec07/fs/jffs2/Makefile.common	2004-12-07 16:03:02.000000000 +0100
@@ -16,3 +16,4 @@
 jffs2-$(CONFIG_JFFS2_RUBIN)	+= compr_rubin.o
 jffs2-$(CONFIG_JFFS2_RTIME)	+= compr_rtime.o
 jffs2-$(CONFIG_JFFS2_ZLIB)	+= compr_zlib.o
+jffs2-$(CONFIG_JFFS2_SUMMARY)   += summary.o
diff --unified --new-file --recursive mtd/fs/jffs2/build.c mtd-dec07/fs/jffs2/build.c
--- mtd/fs/jffs2/build.c	2004-11-28 00:00:13.000000000 +0100
+++ mtd-dec07/fs/jffs2/build.c	2004-12-07 16:08:31.000000000 +0100
@@ -331,6 +331,10 @@
 		c->blocks[i].first_node = NULL;
 		c->blocks[i].last_node = NULL;
 		c->blocks[i].bad_count = 0;
+ #ifdef CONFIG_JFFS2_SUMMARY	
+ 		c->blocks[i].sum_collected = NULL;
+ #endif
+
 	}
 
 	init_MUTEX(&c->alloc_sem);
diff --unified --new-file --recursive mtd/fs/jffs2/dir.c mtd-dec07/fs/jffs2/dir.c
--- mtd/fs/jffs2/dir.c	2004-11-17 00:00:14.000000000 +0100
+++ mtd-dec07/fs/jffs2/dir.c	2004-12-07 16:03:02.000000000 +0100
@@ -22,6 +22,7 @@
 #include <linux/time.h>
 #include "nodelist.h"
 
+
 /* Urgh. Please tell me there's a nicer way of doing these. */
 #include <linux/version.h>
 #if LINUX_VERSION_CODE < KERNEL_VERSION(2,5,48)
@@ -314,7 +315,7 @@
 	 * Just the node will do for now, though 
 	 */
 	namelen = dentry->d_name.len;
-	ret = jffs2_reserve_space(c, sizeof(*ri) + strlen(target), &phys_ofs, &alloclen, ALLOC_NORMAL);
+	ret = jffs2_reserve_space(c, sizeof(*ri) + strlen(target), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
 
 	if (ret) {
 		jffs2_free_raw_inode(ri);
@@ -360,7 +361,7 @@
 	up(&f->sem);
 
 	jffs2_complete_reservation(c);
-	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
+	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
 	if (ret) {
 		/* Eep. */
 		jffs2_clear_inode(inode);
@@ -445,7 +446,7 @@
 	 * Just the node will do for now, though 
 	 */
 	namelen = dentry->d_name.len;
-	ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL);
+	ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
 
 	if (ret) {
 		jffs2_free_raw_inode(ri);
@@ -488,7 +489,7 @@
 	up(&f->sem);
 
 	jffs2_complete_reservation(c);
-	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
+	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
 	if (ret) {
 		/* Eep. */
 		jffs2_clear_inode(inode);
@@ -597,7 +598,7 @@
 	 * Just the node will do for now, though 
 	 */
 	namelen = dentry->d_name.len;
-	ret = jffs2_reserve_space(c, sizeof(*ri) + devlen, &phys_ofs, &alloclen, ALLOC_NORMAL);
+	ret = jffs2_reserve_space(c, sizeof(*ri) + devlen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
 
 	if (ret) {
 		jffs2_free_raw_inode(ri);
@@ -642,7 +643,7 @@
 	up(&f->sem);
 
 	jffs2_complete_reservation(c);
-	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
+	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
 	if (ret) {
 		/* Eep. */
 		jffs2_clear_inode(inode);
@@ -796,4 +797,3 @@
 
 	return 0;
 }
-
diff --unified --new-file --recursive mtd/fs/jffs2/file.c mtd-dec07/fs/jffs2/file.c
--- mtd/fs/jffs2/file.c	2004-11-17 00:00:14.000000000 +0100
+++ mtd-dec07/fs/jffs2/file.c	2004-12-07 16:03:02.000000000 +0100
@@ -22,6 +22,7 @@
 #include <linux/jffs2.h>
 #include "nodelist.h"
 
+
 extern int generic_file_open(struct inode *, struct file *) __attribute__((weak));
 extern loff_t generic_file_llseek(struct file *file, loff_t offset, int origin) __attribute__((weak));
 
@@ -135,7 +136,7 @@
 		D1(printk(KERN_DEBUG "Writing new hole frag 0x%x-0x%x between current EOF and new page\n",
 			  (unsigned int)inode->i_size, pageofs));
 
-		ret = jffs2_reserve_space(c, sizeof(ri), &phys_ofs, &alloc_len, ALLOC_NORMAL);
+		ret = jffs2_reserve_space(c, sizeof(ri), &phys_ofs, &alloc_len, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
 		if (ret)
 			return ret;
 
diff --unified --new-file --recursive mtd/fs/jffs2/fs.c mtd-dec07/fs/jffs2/fs.c
--- mtd/fs/jffs2/fs.c	2004-11-29 00:00:09.000000000 +0100
+++ mtd-dec07/fs/jffs2/fs.c	2004-12-07 16:03:02.000000000 +0100
@@ -74,7 +74,7 @@
 		return -ENOMEM;
 	}
 		
-	ret = jffs2_reserve_space(c, sizeof(*ri) + mdatalen, &phys_ofs, &alloclen, ALLOC_NORMAL);
+	ret = jffs2_reserve_space(c, sizeof(*ri) + mdatalen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
 	if (ret) {
 		jffs2_free_raw_inode(ri);
 		if (S_ISLNK(inode->i_mode & S_IFMT))
diff --unified --new-file --recursive mtd/fs/jffs2/gc.c mtd-dec07/fs/jffs2/gc.c
--- mtd/fs/jffs2/gc.c	2004-11-17 00:00:14.000000000 +0100
+++ mtd-dec07/fs/jffs2/gc.c	2004-12-07 16:03:02.000000000 +0100
@@ -506,7 +506,7 @@
 	   don't want to force wastage of the end of a block if splitting would
 	   work. */
 	ret = jffs2_reserve_space_gc(c, min_t(uint32_t, sizeof(struct jffs2_raw_inode) + JFFS2_MIN_DATA_LEN, 
-					      rawlen), &phys_ofs, &alloclen);
+					      rawlen), &phys_ofs, &alloclen, rawlen); /* this is not optimal yet */
 	if (ret)
 		return ret;
 
@@ -614,7 +614,7 @@
 			ACCT_SANITY_CHECK(c,jeb);
 			D1(ACCT_PARANOIA_CHECK(jeb));
 
-			ret = jffs2_reserve_space_gc(c, rawlen, &phys_ofs, &dummy);
+			ret = jffs2_reserve_space_gc(c, rawlen, &phys_ofs, &dummy, rawlen); /* this is not optimal yet */
 
 			if (!ret) {
 				D1(printk(KERN_DEBUG "Allocated space at 0x%08x to retry failed write.\n", phys_ofs));
@@ -692,7 +692,7 @@
 
 	}
 	
-	ret = jffs2_reserve_space_gc(c, sizeof(ri) + mdatalen, &phys_ofs, &alloclen);
+	ret = jffs2_reserve_space_gc(c, sizeof(ri) + mdatalen, &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE);
 	if (ret) {
 		printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_metadata failed: %d\n",
 		       sizeof(ri)+ mdatalen, ret);
@@ -759,7 +759,7 @@
 	rd.node_crc = cpu_to_je32(crc32(0, &rd, sizeof(rd)-8));
 	rd.name_crc = cpu_to_je32(crc32(0, fd->name, rd.nsize));
 	
-	ret = jffs2_reserve_space_gc(c, sizeof(rd)+rd.nsize, &phys_ofs, &alloclen);
+	ret = jffs2_reserve_space_gc(c, sizeof(rd)+rd.nsize, &phys_ofs, &alloclen, JFFS2_SUMMARY_DIRENT_SIZE(rd.nsize));
 	if (ret) {
 		printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_dirent failed: %d\n",
 		       sizeof(rd)+rd.nsize, ret);
@@ -961,7 +961,7 @@
 	ri.data_crc = cpu_to_je32(0);
 	ri.node_crc = cpu_to_je32(crc32(0, &ri, sizeof(ri)-8));
 
-	ret = jffs2_reserve_space_gc(c, sizeof(ri), &phys_ofs, &alloclen);
+	ret = jffs2_reserve_space_gc(c, sizeof(ri), &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE);
 	if (ret) {
 		printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_hole failed: %d\n",
 		       sizeof(ri), ret);
@@ -1186,7 +1186,7 @@
 		uint32_t cdatalen;
 		uint16_t comprtype = JFFS2_COMPR_NONE;
 
-		ret = jffs2_reserve_space_gc(c, sizeof(ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen);
+		ret = jffs2_reserve_space_gc(c, sizeof(ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE);
 
 		if (ret) {
 			printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_dnode failed: %d\n",
@@ -1243,4 +1243,3 @@
 	jffs2_gc_release_page(c, pg_ptr, &pg);
 	return ret;
 }
-
diff --unified --new-file --recursive mtd/fs/jffs2/nodelist.h mtd-dec07/fs/jffs2/nodelist.h
--- mtd/fs/jffs2/nodelist.h	2004-11-26 15:08:31.000000000 +0100
+++ mtd-dec07/fs/jffs2/nodelist.h	2004-12-07 16:03:02.000000000 +0100
@@ -20,6 +20,7 @@
 #include <linux/jffs2.h>
 #include <linux/jffs2_fs_sb.h>
 #include <linux/jffs2_fs_i.h>
+#include "summary.h"
 
 #ifdef __ECOS
 #include "os-ecos.h"
@@ -194,6 +195,10 @@
 	int bad_count;
 	uint32_t offset;		/* of this block in the MTD */
 
+#ifdef CONFIG_JFFS2_SUMMARY	
+	struct jffs2_sum_info *sum_collected;
+#endif
+	
 	uint32_t unchecked_size;
 	uint32_t used_size;
 	uint32_t dirty_size;
@@ -392,8 +397,8 @@
 
 /* nodemgmt.c */
 int jffs2_thread_should_wake(struct jffs2_sb_info *c);
-int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio);
-int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len);
+int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio, uint32_t sumsize);
+int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize);
 int jffs2_add_physical_node_ref(struct jffs2_sb_info *c, struct jffs2_raw_node_ref *new);
 void jffs2_complete_reservation(struct jffs2_sb_info *c);
 void jffs2_mark_node_obsolete(struct jffs2_sb_info *c, struct jffs2_raw_node_ref *raw);
@@ -455,6 +460,10 @@
 /* scan.c */
 int jffs2_scan_medium(struct jffs2_sb_info *c);
 void jffs2_rotate_lists(struct jffs2_sb_info *c);
+int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf,
+				uint32_t ofs, uint32_t len);
+struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino);
+
 
 /* build.c */
 int jffs2_do_mount_fs(struct jffs2_sb_info *c);
diff --unified --new-file --recursive mtd/fs/jffs2/nodemgmt.c mtd-dec07/fs/jffs2/nodemgmt.c
--- mtd/fs/jffs2/nodemgmt.c	2004-11-26 15:08:31.000000000 +0100
+++ mtd-dec07/fs/jffs2/nodemgmt.c	2004-12-07 16:03:02.000000000 +0100
@@ -38,9 +38,9 @@
  *	for the requested allocation.
  */
 
-static int jffs2_do_reserve_space(struct jffs2_sb_info *c,  uint32_t minsize, uint32_t *ofs, uint32_t *len);
+static int jffs2_do_reserve_space(struct jffs2_sb_info *c,  uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize);
 
-int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio)
+int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio, uint32_t sumsize)
 {
 	int ret = -EAGAIN;
 	int blocksneeded = c->resv_blocks_write;
@@ -129,7 +129,7 @@
 			spin_lock(&c->erase_completion_lock);
 		}
 
-		ret = jffs2_do_reserve_space(c, minsize, ofs, len);
+		ret = jffs2_do_reserve_space(c, minsize, ofs, len, sumsize);
 		if (ret) {
 			D1(printk(KERN_DEBUG "jffs2_reserve_space: ret is %d\n", ret));
 		}
@@ -140,7 +140,7 @@
 	return ret;
 }
 
-int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len)
+int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize)
 {
 	int ret = -EAGAIN;
 	minsize = PAD(minsize);
@@ -149,7 +149,7 @@
 
 	spin_lock(&c->erase_completion_lock);
 	while(ret == -EAGAIN) {
-		ret = jffs2_do_reserve_space(c, minsize, ofs, len);
+		ret = jffs2_do_reserve_space(c, minsize, ofs, len, sumsize);
 		if (ret) {
 		        D1(printk(KERN_DEBUG "jffs2_reserve_space_gc: looping, ret is %d\n", ret));
 		}
@@ -159,50 +159,112 @@
 }
 
 /* Called with alloc sem _and_ erase_completion_lock */
-static int jffs2_do_reserve_space(struct jffs2_sb_info *c,  uint32_t minsize, uint32_t *ofs, uint32_t *len)
+static int jffs2_do_reserve_space(struct jffs2_sb_info *c,  uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize)
 {
 	struct jffs2_eraseblock *jeb = c->nextblock;
+	uint32_t nofree_size;
 	
  restart:
-	if (jeb && minsize > jeb->free_size) {
-		/* Skip the end of this block and file it as having some dirty space */
-		/* If there's a pending write to it, flush now */
-		if (jffs2_wbuf_dirty(c)) {
-			spin_unlock(&c->erase_completion_lock);
-			D1(printk(KERN_DEBUG "jffs2_do_reserve_space: Flushing write buffer\n"));			    
-			jffs2_flush_wbuf_pad(c);
-			spin_lock(&c->erase_completion_lock);
-			jeb = c->nextblock;
-			goto restart;
-		}
-		c->wasted_size += jeb->free_size;
-		c->free_size -= jeb->free_size;
-		jeb->wasted_size += jeb->free_size;
-		jeb->free_size = 0;
+	nofree_size = 0;
+	
+#ifdef CONFIG_JFFS2_SUMMARY
+
+	if (sumsize != JFFS2_SUMMARY_NOSUM_SIZE) {
+		int ret;
+                if (jeb) {
+                        if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret;
+                        nofree_size = sumsize + jeb->sum_collected->sum_size + JFFS2_SUMMARY_FRAME_SIZE;
+                }
 		
-		/* Check, if we have a dirty block now, or if it was dirty already */
-		if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) {
-			c->dirty_size += jeb->wasted_size;
-			c->wasted_size -= jeb->wasted_size;
-			jeb->dirty_size += jeb->wasted_size;
-			jeb->wasted_size = 0;
-			if (VERYDIRTY(c, jeb->dirty_size)) {
-				D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
+		D1(printk(KERN_DEBUG "JFFS2: minsize %d , jeb->free(%d) , sum_collected->size(%d) , sumsize(%d)\n",minsize,jeb->free_size,jeb->sum_collected->sum_size,sumsize));
+		
+		if (jeb && (minsize + jeb->sum_collected->sum_size + sumsize + JFFS2_SUMMARY_FRAME_SIZE > jeb->free_size)) {
+			D1(printk(KERN_DEBUG "JFFS2: generating summary for 0x%08x.\n", jeb->offset));
+			if (jeb->sum_collected->sum_size == JFFS2_SUMMARY_NOSUM_SIZE) {
+				sumsize = JFFS2_SUMMARY_NOSUM_SIZE;
+				jffs2_sum_clean_collected(jeb);
+				goto restart;
+			}
+			
+			ret = jffs2_sum_write_sumnode(c);
+			
+			if (ret)
+				return ret;
+			
+			if (jeb->sum_collected->sum_size == JFFS2_SUMMARY_NOSUM_SIZE) { //jffs2_write_sumnode can't write out the summary information
+				sumsize = JFFS2_SUMMARY_NOSUM_SIZE;
+				jffs2_sum_clean_collected(jeb);
+				goto restart;
+			}
+			
+			/* Check, if we have a dirty block now, or if it was dirty already */
+			if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) {
+				c->dirty_size += jeb->wasted_size;
+				c->wasted_size -= jeb->wasted_size;
+				jeb->dirty_size += jeb->wasted_size;
+				jeb->wasted_size = 0;
+				if (VERYDIRTY(c, jeb->dirty_size)) {
+					D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
+					  jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
+					list_add_tail(&jeb->list, &c->very_dirty_list);
+				} else {
+					D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
+					  jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
+					list_add_tail(&jeb->list, &c->dirty_list);
+				}
+			} else { 
+				D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
 				  jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
-				list_add_tail(&jeb->list, &c->very_dirty_list);
-			} else {
-				D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
+				list_add_tail(&jeb->list, &c->clean_list);
+			}
+			c->nextblock = jeb = NULL;
+		}
+	}
+	else {	
+#endif			
+		if (jeb && minsize > jeb->free_size) {
+			/* Skip the end of this block and file it as having some dirty space */
+			/* If there's a pending write to it, flush now */
+			
+			if (jffs2_wbuf_dirty(c)) {
+				spin_unlock(&c->erase_completion_lock);
+				D1(printk(KERN_DEBUG "jffs2_do_reserve_space: Flushing write buffer\n"));			    
+				jffs2_flush_wbuf_pad(c);
+				spin_lock(&c->erase_completion_lock);
+				jeb = c->nextblock;
+				goto restart;
+			}
+			
+			c->wasted_size += jeb->free_size;
+			c->free_size -= jeb->free_size;
+			jeb->wasted_size += jeb->free_size;
+			jeb->free_size = 0;
+			
+			/* Check, if we have a dirty block now, or if it was dirty already */
+			if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) {
+				c->dirty_size += jeb->wasted_size;
+				c->wasted_size -= jeb->wasted_size;
+				jeb->dirty_size += jeb->wasted_size;
+				jeb->wasted_size = 0;
+				if (VERYDIRTY(c, jeb->dirty_size)) {
+					D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
+					  jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
+					list_add_tail(&jeb->list, &c->very_dirty_list);
+				} else {
+					D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
+					  jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
+					list_add_tail(&jeb->list, &c->dirty_list);
+				}
+			} else { 
+				D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
 				  jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
-				list_add_tail(&jeb->list, &c->dirty_list);
+				list_add_tail(&jeb->list, &c->clean_list);
 			}
-		} else { 
-			D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
-			  jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
-			list_add_tail(&jeb->list, &c->clean_list);
+			c->nextblock = jeb = NULL;
 		}
-		c->nextblock = jeb = NULL;
+#ifdef CONFIG_JFFS2_SUMMARY
 	}
-	
+#endif	
 	if (!jeb) {
 		struct list_head *next;
 		/* Take the next block off the 'free' list */
@@ -266,7 +328,7 @@
 	/* OK, jeb (==c->nextblock) is now pointing at a block which definitely has
 	   enough space */
 	*ofs = jeb->offset + (c->sector_size - jeb->free_size);
-	*len = jeb->free_size;
+	*len = jeb->free_size - nofree_size;
 
 	if (c->cleanmarker_size && jeb->used_size == c->cleanmarker_size &&
 	    !jeb->first_node->next_in_ino) {
diff --unified --new-file --recursive mtd/fs/jffs2/os-linux.h mtd-dec07/fs/jffs2/os-linux.h
--- mtd/fs/jffs2/os-linux.h	2004-11-17 00:00:14.000000000 +0100
+++ mtd-dec07/fs/jffs2/os-linux.h	2004-12-07 16:03:02.000000000 +0100
@@ -100,7 +100,13 @@
 #define jffs2_is_readonly(c) (OFNI_BS_2SFFJ(c)->s_flags & MS_RDONLY)
 
 #if (!defined CONFIG_JFFS2_FS_NAND && !defined CONFIG_JFFS2_FS_NOR_ECC)
+
+#ifndef CONFIG_JFFS2_SUMMARY
 #define jffs2_can_mark_obsolete(c) (1)
+#else
+#define jffs2_can_mark_obsolete(c) (0)
+#endif
+
 #define jffs2_cleanmarker_oob(c) (0)
 #define jffs2_write_nand_cleanmarker(c,jeb) (-EIO)
 
@@ -121,7 +127,12 @@
 
 #else /* NAND and/or ECC'd NOR support present */
 
+#ifndef CONFIG_JFFS2_SUMMARY
 #define jffs2_can_mark_obsolete(c) ((c->mtd->type == MTD_NORFLASH && !(c->mtd->flags & MTD_ECC)) || c->mtd->type == MTD_RAM)
+#else
+#define jffs2_can_mark_obsolete(c) (0)
+#endif
+
 #define jffs2_cleanmarker_oob(c) (c->mtd->type == MTD_NANDFLASH)
 
 #define jffs2_flash_write_oob(c, ofs, len, retlen, buf) ((c)->mtd->write_oob((c)->mtd, ofs, len, retlen, buf))
@@ -218,5 +229,3 @@
 
 
 #endif /* __JFFS2_OS_LINUX_H__ */
-
-
diff --unified --new-file --recursive mtd/fs/jffs2/scan.c mtd-dec07/fs/jffs2/scan.c
--- mtd/fs/jffs2/scan.c	2004-11-26 15:08:31.000000000 +0100
+++ mtd-dec07/fs/jffs2/scan.c	2004-12-07 16:03:02.000000000 +0100
@@ -18,22 +18,10 @@
 #include <linux/crc32.h>
 #include <linux/compiler.h>
 #include "nodelist.h"
+#include "summary.h"
 
 #define EMPTY_SCAN_SIZE 1024
 
-#define DIRTY_SPACE(x) do { typeof(x) _x = (x); \
-		c->free_size -= _x; c->dirty_size += _x; \
-		jeb->free_size -= _x ; jeb->dirty_size += _x; \
-		}while(0)
-#define USED_SPACE(x) do { typeof(x) _x = (x); \
-		c->free_size -= _x; c->used_size += _x; \
-		jeb->free_size -= _x ; jeb->used_size += _x; \
-		}while(0)
-#define UNCHECKED_SPACE(x) do { typeof(x) _x = (x); \
-		c->free_size -= _x; c->unchecked_size += _x; \
-		jeb->free_size -= _x ; jeb->unchecked_size += _x; \
-		}while(0)
-
 #define noisy_printk(noise, args...) do { \
 	if (*(noise)) { \
 		printk(KERN_NOTICE args); \
@@ -58,13 +46,6 @@
 static int jffs2_scan_dirent_node(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb,
 				 struct jffs2_raw_dirent *rd, uint32_t ofs);
 
-#define BLK_STATE_ALLFF		0
-#define BLK_STATE_CLEAN		1
-#define BLK_STATE_PARTDIRTY	2
-#define BLK_STATE_CLEANMARKER	3
-#define BLK_STATE_ALLDIRTY	4
-#define BLK_STATE_BADBLOCK	5
-
 static inline int min_free(struct jffs2_sb_info *c)
 {
 	uint32_t min = 2 * sizeof(struct jffs2_raw_inode);
@@ -257,7 +238,7 @@
 	return ret;
 }
 
-static int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf,
+int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf,
 				uint32_t ofs, uint32_t len)
 {
 	int ret;
@@ -286,6 +267,11 @@
 	uint32_t hdr_crc, buf_ofs, buf_len;
 	int err;
 	int noise = 0;
+	
+#ifdef CONFIG_JFFS2_SUMMARY
+	struct jffs2_sum_marker *sm;
+#endif	
+	
 #ifdef CONFIG_JFFS2_FS_NAND
 	int cleanmarkerfound = 0;
 #endif
@@ -311,10 +297,54 @@
 		}
 	}
 #endif
+	
+#ifdef CONFIG_JFFS2_SUMMARY	
+	sm = (struct jffs2_sum_marker *)kmalloc(sizeof(struct jffs2_sum_marker), GFP_KERNEL);
+	if (!sm) {
+	    return -ENOMEM;
+	}
+	
+	err = jffs2_fill_scan_buf(c, (unsigned char *) sm, jeb->offset + c->sector_size - sizeof(struct jffs2_sum_marker), sizeof(struct jffs2_sum_marker));
+	
+	if (err) {
+		kfree(sm);
+	        return err;
+	}
+	
+	if (je32_to_cpu(sm->magic) == JFFS2_SUM_MAGIC ) {
+
+		if(je32_to_cpu(sm->erase_size) == c->sector_size) {
+			int ret = jffs2_sum_scan_sumnode(c,jeb,je32_to_cpu(sm->offset),&pseudo_random);
+			
+			if (ret) {
+				kfree(sm);
+				return ret;
+			}
+		}
+		
+		printk(KERN_WARNING "FS erase_block_size != JFFS2 erase_block_size => skipping summary information\n");
+		
+	}
+	
+	kfree(sm);
+	
+	ofs = jeb->offset;
+	prevofs = jeb->offset - 1;
+	
+#endif
+	
 	buf_ofs = jeb->offset;
 
 	if (!buf_size) {
 		buf_len = c->sector_size;
+		
+#ifdef CONFIG_JFFS2_SUMMARY
+		/* must reread because of summary test */
+		err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len);
+		if (err)
+			return err;
+#endif
+
 	} else {
 		buf_len = EMPTY_SCAN_SIZE;
 		err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len);
@@ -356,6 +386,8 @@
 
 	noise = 10;
 
+	D1(printk(KERN_DEBUG "JFFS2: no summary found in jeb 0x%08x. Apply original scan.\n",jeb->offset));
+
 scan_more:	
 	while(ofs < jeb->offset + c->sector_size) {
 
@@ -571,6 +603,9 @@
 			break;
 
 		case JFFS2_NODETYPE_PADDING:
+#ifdef CONFIG_JFFS2_SUMMARY
+	                jffs2_sum_add_padding_mem(jeb,je32_to_cpu(node->totlen));
+#endif
 			DIRTY_SPACE(PAD(je32_to_cpu(node->totlen)));
 			ofs += PAD(je32_to_cpu(node->totlen));
 			break;
@@ -634,7 +669,7 @@
 		return BLK_STATE_ALLDIRTY;
 }
 
-static struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino)
+struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino)
 {
 	struct jffs2_inode_cache *ic;
 
@@ -728,6 +763,11 @@
 	pseudo_random += je32_to_cpu(ri->version);
 
 	UNCHECKED_SPACE(PAD(je32_to_cpu(ri->totlen)));
+	
+#ifdef CONFIG_JFFS2_SUMMARY
+	jffs2_sum_add_inode_mem(jeb,ri,ofs);
+#endif
+	
 	return 0;
 }
 
@@ -806,6 +846,10 @@
 	USED_SPACE(PAD(je32_to_cpu(rd->totlen)));
 	jffs2_add_fd_to_list(c, fd, &ic->scan_dents);
 
+#ifdef CONFIG_JFFS2_SUMMARY
+	jffs2_sum_add_dirent_mem(jeb,rd,ofs);
+#endif
+	
 	return 0;
 }
 
diff --unified --new-file --recursive mtd/fs/jffs2/summary.c mtd-dec07/fs/jffs2/summary.c
--- mtd/fs/jffs2/summary.c	1970-01-01 01:00:00.000000000 +0100
+++ mtd-dec07/fs/jffs2/summary.c	2004-12-07 16:03:02.000000000 +0100
@@ -0,0 +1,720 @@
+/*
+ * JFFS2 -- Journalling Flash File System, Version 2.
+ *
+ * Copyright (C) 2004  Ferenc Havasi <havasi@inf.u-szeged.hu>,
+ *                     Zoltan Sogor <weth@inf.u-szeged.hu>,
+ *                     Patrik Kluba <pajko@halom.u-szeged.hu>,
+ *                     University of Szeged, Hungary
+ *
+ * For licensing information, see the file 'LICENCE' in this directory.
+ *
+ * $Id$
+ *
+ */
+
+#include <linux/kernel.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/mtd/mtd.h>
+#include <linux/pagemap.h>
+#include <linux/crc32.h>
+#include <linux/compiler.h>
+#include <linux/vmalloc.h>
+#include "nodelist.h"
+
+int jffs2_sum_init(struct jffs2_sb_info *c) 
+{
+        c->summary_buf = (jint32_t *) vmalloc(c->sector_size);
+        if (!c->summary_buf) {
+		printk(KERN_WARNING "JFFS2: can't allocate memory to dump summary information!\n");
+                return 1;
+        }
+        return 0;
+}
+
+void jffs2_sum_exit(struct jffs2_sb_info *c) 
+{
+        if (c->summary_buf) {
+                vfree(c->summary_buf);
+                c->summary_buf = NULL;
+        }
+}
+
+int jffs2_sum_care_sum_collected(struct jffs2_eraseblock *jeb)
+{
+	if (!jeb->sum_collected) {
+		jeb->sum_collected = (struct jffs2_sum_info *) kmalloc(sizeof(struct jffs2_sum_info), GFP_KERNEL);
+	
+		if (!jeb->sum_collected)
+			return -ENOMEM;
+			
+		jeb->sum_collected->sum_list = NULL;
+		jeb->sum_collected->sum_num = 0;
+		jeb->sum_collected->sum_size = 0; 
+		jeb->sum_collected->sum_padded = 0; 
+	}
+        return 0;
+}
+
+static int jffs2_sum_add_mem(struct jffs2_eraseblock *jeb, union jffs2_sum_mem *item) 
+{
+	
+	union jffs2_sum_mem *walk;
+        int ret;
+	
+        if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret;
+	
+	if (!jeb->sum_collected->sum_list) {
+		jeb->sum_collected->sum_list = (union jffs2_sum_mem *) item;
+	} 
+	else {
+		walk = jeb->sum_collected->sum_list;
+		
+		while (walk->u.next) {
+			walk = walk->u.next;
+		}
+		walk->u.next = (union jffs2_sum_mem *) item;
+	}
+	switch (je16_to_cpu(item->u.nodetype)) {
+    	    case JFFS2_NODETYPE_INODE:
+		jeb->sum_collected->sum_size += JFFS2_SUMMARY_INODE_SIZE;
+		jeb->sum_collected->sum_num++;
+		break;
+    	    case JFFS2_NODETYPE_DIRENT:
+		jeb->sum_collected->sum_size += JFFS2_SUMMARY_DIRENT_SIZE(item->d.nsize);
+		jeb->sum_collected->sum_num++;
+		break;
+	    default:
+		printk(KERN_WARNING "__jffs2_add_sum_mem(): UNKNOWN node type %d\n", je16_to_cpu(item->u.nodetype));
+		return 1;
+	}
+	return 0;
+}
+
+void jffs2_sum_clean_all_info(struct jffs2_sb_info *c)
+{
+	int i;
+	
+	for (i=0; i<c->nr_blocks; i++) {
+		struct jffs2_eraseblock *jeb = &c->blocks[i];
+		
+		jffs2_sum_clean_collected(jeb);
+		kfree(jeb->sum_collected);
+		jeb->sum_collected = NULL;
+	}
+}
+	
+/* These 3 functions are called from scan.c to collect summary info for not closed jeb */
+
+int jffs2_sum_add_padding_mem(struct jffs2_eraseblock *jeb, uint32_t size)
+{
+        int ret;
+	
+        if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret;
+        jeb->sum_collected->sum_padded += size;
+        return 0;
+}
+
+int jffs2_sum_add_inode_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_inode *ri, uint32_t ofs) 
+{
+	
+	struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *) kmalloc(sizeof(struct jffs2_sum_inode_mem), GFP_KERNEL);
+	
+	if (!temp)
+		return -ENOMEM;
+
+        ofs -= jeb->offset;
+	
+	temp->nodetype = ri->nodetype;
+	temp->inode = ri->ino;
+	temp->version = ri->version;
+	temp->offset = cpu_to_je32(ofs); 
+	temp->totlen = ri->totlen;
+	temp->next = NULL;
+	
+	return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp);
+}
+
+int jffs2_sum_add_dirent_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_dirent *rd, uint32_t ofs) 
+{
+	
+	struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *) 
+			kmalloc(sizeof(struct jffs2_sum_dirent_mem) + rd->nsize, GFP_KERNEL);
+	
+	if (!temp)
+		return -ENOMEM;
+	
+        ofs -= jeb->offset;
+
+	temp->nodetype = rd->nodetype;
+	temp->totlen = rd->totlen;
+	temp->offset = cpu_to_je32(ofs);
+	temp->pino = rd->pino;
+	temp->version = rd->version;
+	temp->ino = rd->ino;
+	temp->nsize = rd->nsize;
+	temp->type = rd->type;
+	temp->next = NULL;
+	
+	memcpy(temp->name, rd->name, rd->nsize);
+
+	return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp);
+}
+
+/* Cleanup every collected summary information */
+
+void jffs2_sum_clean_collected(struct jffs2_eraseblock *jeb) 
+{
+	
+	union jffs2_sum_mem *temp;
+	
+	if(jeb && jeb->sum_collected){
+		
+		while(jeb->sum_collected->sum_list){
+			temp = jeb->sum_collected->sum_list;
+			jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next;
+			kfree(temp);
+			jeb->sum_collected->sum_num--;
+		}
+		
+		if(jeb->sum_collected->sum_num != 0){
+			printk(KERN_WARNING "Ooops, something wrong happened! sum_num != 0, but sum_list = null ???");
+			jeb->sum_collected->sum_num = 0;
+		}
+	}	
+}
+
+/* Called from wbuf.c to collect writed node info */
+
+int jffs2_sum_add_kvec(struct jffs2_sb_info *c, const struct kvec *invecs, unsigned long count, uint32_t ofs)
+{
+	union jffs2_node_union *node;
+	struct jffs2_eraseblock *jeb;	
+        int ret;
+	
+	node = (union jffs2_node_union *) invecs[0].iov_base;
+	jeb = &c->blocks[ofs / c->sector_size];
+        ofs -= jeb->offset;
+       
+        if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret;        
+	
+	switch(je16_to_cpu(node->u.nodetype)){
+		case JFFS2_NODETYPE_INODE : {
+			struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *) 
+				kmalloc(sizeof(struct jffs2_sum_inode_mem), GFP_KERNEL);
+			
+			if (!temp)
+				return -ENOMEM;
+
+			temp->nodetype = node->i.nodetype;
+			temp->inode = node->i.ino;
+			temp->version = node->i.version;
+			temp->offset = cpu_to_je32(ofs); 
+			temp->totlen = node->i.totlen;
+			temp->next = NULL;
+						
+			return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp);
+			
+			break;
+		}
+		
+		case JFFS2_NODETYPE_DIRENT : {
+			struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *) 
+				kmalloc(sizeof(struct jffs2_sum_dirent_mem) + node->d.nsize, GFP_KERNEL);
+			
+			if (!temp)
+				return -ENOMEM;
+			
+			temp->nodetype = node->d.nodetype;
+			temp->totlen = node->d.totlen;
+			temp->offset = cpu_to_je32(ofs);
+			temp->pino = node->d.pino;
+			temp->version = node->d.version;
+			temp->ino = node->d.ino;
+			temp->nsize = node->d.nsize;
+			temp->type = node->d.type;
+			temp->next = NULL;
+			
+			memcpy(temp->name,invecs[1].iov_base,node->d.nsize);
+			
+			return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp);
+			
+			break;
+		}
+		
+		case JFFS2_NODETYPE_PADDING : {
+			D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node PADDING\n"));
+                        jeb->sum_collected->sum_padded += je32_to_cpu(node->u.totlen);
+			break;
+		}
+		
+		case JFFS2_NODETYPE_CLEANMARKER : {
+			D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node CLEANMARKER\n"));
+			break;
+		}
+		
+		case JFFS2_NODETYPE_SUMMARY : {
+			D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node SUMMARY\n"));
+			break;
+		}
+		
+		default : {
+			printk(KERN_WARNING "jffs2_sum_add_kvec(): Node not supported\n");
+			BUG();
+			break;
+		}
+	}
+	
+	return 0;
+}
+
+/* Process the summary information - called from jffs2_scan_eraseblock() */
+
+int jffs2_sum_scan_sumnode(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb, uint32_t ofs, uint32_t *pseudo_random)
+{
+	
+	struct jffs2_unknown_node crcnode;
+	struct jffs2_raw_node_ref *raw;
+	struct jffs2_raw_node_ref *cache_ref;
+	struct jffs2_inode_cache *ic;
+	struct jffs2_full_dirent *fd;
+		
+	int i, err;
+	int bad_sum = 0;
+	int sumsize;
+	uint32_t ino;
+	uint32_t crc;
+	struct jffs2_summary_node *summary;
+		
+	sumsize = c->sector_size - ofs;
+	ofs += jeb->offset;
+	
+	D1(printk(KERN_DEBUG "JFFS2: summary found for 0x%08x at 0x%08x (0x%x bytes)\n", jeb->offset, ofs, sumsize));
+	
+	summary = (struct jffs2_summary_node *) kmalloc(sumsize, GFP_KERNEL);
+		
+	if (!summary) {
+			return -ENOMEM;
+	}
+	
+	err = jffs2_fill_scan_buf(c, (unsigned char *)summary, ofs, sumsize);
+	
+	if (err) {
+			kfree(summary);
+			return err;
+	}
+
+	/* OK, now check for node validity and CRC */
+	crcnode.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
+	crcnode.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY);
+	crcnode.totlen = summary->totlen;
+	crc = crc32(0, &crcnode, sizeof(crcnode)-4);
+	
+	if (je32_to_cpu(summary->hdr_crc) != crc) {
+			D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node header is corrupt (bad CRC or no summary at all)\n"));
+			bad_sum = 1;
+	}
+	
+	if ((!bad_sum) && (je32_to_cpu(summary->totlen) != sumsize)) {
+			D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (wrong erasesize?)\n"));
+			bad_sum = 1;
+	}
+	
+	crc = crc32(0, summary, sizeof(struct jffs2_summary_node)-8);
+		
+	if ((!bad_sum) && (je32_to_cpu(summary->node_crc) != crc)) {
+			D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (bad CRC)\n"));
+			bad_sum = 1;
+	}
+	
+	crc = crc32(0, summary->sum, sumsize - sizeof(struct jffs2_summary_node));
+
+	if ((!bad_sum) && (je32_to_cpu(summary->sum_crc) != crc)) {
+			D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node data is corrupt (bad CRC)\n"));
+			bad_sum = 1;
+	}
+	
+	if (!bad_sum) {
+		
+		struct jffs2_sum_unknown_flash *sp;
+		sp = (struct jffs2_sum_unknown_flash *) summary->sum;
+
+		if ( je32_to_cpu(summary->cln_mkr) ) {
+			
+			D1(printk(KERN_DEBUG "Summary : CLEANMARKER node \n"));
+			
+			if (je32_to_cpu(summary->cln_mkr) != c->cleanmarker_size) {
+				D1(printk(KERN_DEBUG "CLEANMARKER node has totlen 0x%x != normal 0x%x\n", 
+				   je32_to_cpu(summary->cln_mkr), c->cleanmarker_size));
+				UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) );
+			} 
+			else if (jeb->first_node) {
+				D1(printk(KERN_DEBUG "CLEANMARKER node not first node in block (0x%08x)\n", jeb->offset);
+				UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr))));
+			} 
+			else {
+				struct jffs2_raw_node_ref *marker_ref = jffs2_alloc_raw_node_ref();
+					
+				if (!marker_ref) {
+					D1(printk(KERN_NOTICE "Failed to allocate node ref for clean marker\n"));
+					kfree(summary);
+					return -ENOMEM;
+				}
+				
+				marker_ref->next_in_ino = NULL;
+				marker_ref->next_phys = NULL;
+				marker_ref->flash_offset = jeb->offset | REF_NORMAL;
+				marker_ref->__totlen = je32_to_cpu(summary->cln_mkr);
+				jeb->first_node = jeb->last_node = marker_ref;
+			
+				USED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) );
+								
+			}
+		}
+
+		if ( je32_to_cpu(summary->padded) ) {
+                        DIRTY_SPACE(je32_to_cpu(summary->padded));
+                }
+		
+		for(i = 0; i < je16_to_cpu(summary->sum_num); i++) {
+			
+			D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Processing summary information %d\n", i));
+			uint8_t *temp8ptr = NULL;
+			
+			switch (je16_to_cpu(sp->nodetype)) {
+				
+				case JFFS2_NODETYPE_INODE : {
+					struct jffs2_sum_inode_flash *spi;
+					spi = (struct jffs2_sum_inode_flash *) sp;
+						
+					ino = je32_to_cpu(spi->inode);
+					D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Inode at 0x%08x\n", jeb->offset + je32_to_cpu(spi->offset)));
+					raw = jffs2_alloc_raw_node_ref();
+					if (!raw) {
+						printk(KERN_NOTICE "jffs2_scan_eraseblock(): allocation of node reference failed\n");
+						kfree(summary);
+						return -ENOMEM;
+					}
+		
+					ic = jffs2_get_ino_cache(c, ino);
+					if (!ic) {
+						ic = jffs2_scan_make_ino_cache(c, ino);
+						if (!ic) {
+							printk(KERN_NOTICE "jffs2_scan_eraseblock(): scan_make_ino_cache failed\n");
+							jffs2_free_raw_node_ref(raw);
+							kfree(summary);
+							return -ENOMEM;
+						}
+					}
+						
+					raw->flash_offset = (jeb->offset + je32_to_cpu(spi->offset)) | REF_UNCHECKED;
+					raw->__totlen = PAD(je32_to_cpu(spi->totlen));
+					raw->next_phys = NULL;
+					raw->next_in_ino = ic->nodes;
+						
+					ic->nodes = raw;
+					if (!jeb->first_node)
+							jeb->first_node = raw;
+					if (jeb->last_node)
+							jeb->last_node->next_phys = raw;
+					jeb->last_node = raw;
+						
+					*pseudo_random += je32_to_cpu(spi->version);
+					UNCHECKED_SPACE(PAD(je32_to_cpu(spi->totlen)));
+					
+					temp8ptr = (uint8_t *) sp;
+					temp8ptr += JFFS2_SUMMARY_INODE_SIZE;
+					sp = (struct jffs2_sum_unknown_flash *) temp8ptr;
+					
+					break;
+				}
+					
+				case JFFS2_NODETYPE_DIRENT : {
+					struct jffs2_sum_dirent_flash *spd;
+					spd = (struct jffs2_sum_dirent_flash *) sp;
+					
+					fd = jffs2_alloc_full_dirent(spd->nsize+1);
+					if (!fd) {
+						kfree(summary);
+						return -ENOMEM;
+					}
+					
+					memcpy(&fd->name, spd->name, spd->nsize);
+					fd->name[spd->nsize] = 0;
+					
+					raw = jffs2_alloc_raw_node_ref();
+					if (!raw) {
+						jffs2_free_full_dirent(fd);
+						printk(KERN_NOTICE "jffs2_scan_dirent_node(): allocation of node reference failed\n");
+						kfree(summary);
+						return -ENOMEM;
+					}
+					
+					ic = jffs2_scan_make_ino_cache(c, je32_to_cpu(spd->pino));
+					if (!ic) {
+						jffs2_free_full_dirent(fd);
+						jffs2_free_raw_node_ref(raw);
+						kfree(summary);
+						return -ENOMEM;
+					}
+					
+					raw->__totlen = PAD(je32_to_cpu(spd->totlen));
+					raw->flash_offset = (jeb->offset + je32_to_cpu(spd->offset)) | REF_PRISTINE;
+					raw->next_phys = NULL;
+					raw->next_in_ino = ic->nodes;
+					ic->nodes = raw;
+					if (!jeb->first_node)
+						jeb->first_node = raw;
+					if (jeb->last_node)
+						jeb->last_node->next_phys = raw;
+					jeb->last_node = raw;
+				
+					fd->raw = raw;
+					fd->next = NULL;
+					fd->version = je32_to_cpu(spd->version);
+					fd->ino = je32_to_cpu(spd->ino);
+					fd->nhash = full_name_hash(fd->name, spd->nsize);
+					fd->type = spd->type;
+					USED_SPACE(PAD(je32_to_cpu(spd->totlen)));
+					jffs2_add_fd_to_list(c, fd, &ic->scan_dents);
+					
+					*pseudo_random += je32_to_cpu(spd->version);
+					
+					temp8ptr = (uint8_t *) sp;
+					temp8ptr += JFFS2_SUMMARY_DIRENT_SIZE(spd->nsize); 
+					sp = (struct jffs2_sum_unknown_flash *) temp8ptr;
+					
+					break;
+				}
+				
+				default : {
+					printk(KERN_WARNING "Kernel doesn't support this type of node !!! Exiting");
+					return -EIO;
+					break;			
+				}
+			}
+		}
+		
+		kfree(summary);
+
+		/* for ACCT_PARANOIA_CHECK */
+		cache_ref = jffs2_alloc_raw_node_ref();
+		
+		if (!cache_ref) {
+			printk(KERN_NOTICE "Failed to allocate node ref for cache\n");
+			return -ENOMEM;
+		}
+		
+		cache_ref->next_in_ino = NULL;
+		cache_ref->next_phys = NULL;
+		cache_ref->flash_offset = ofs | REF_NORMAL;
+		cache_ref->__totlen = sumsize;
+		
+		if (!jeb->first_node)
+			jeb->first_node = cache_ref;
+		if (jeb->last_node)
+			jeb->last_node->next_phys = cache_ref;
+		jeb->last_node = cache_ref;
+		
+		USED_SPACE(sumsize);
+
+                jeb->wasted_size += jeb->free_size;
+                c->wasted_size += jeb->free_size;
+                c->free_size -= jeb->free_size;
+                jeb->free_size = 0;
+
+		/* somebody check this and all of space accounting in summary support */
+
+		if ((jeb->used_size + jeb->unchecked_size) == PAD(c->cleanmarker_size) && !jeb->dirty_size 
+			&& (!jeb->first_node || !jeb->first_node->next_in_ino) ) { 
+				return BLK_STATE_CLEANMARKER; 
+			}		
+		/* move blocks with max 4 byte dirty space to cleanlist */	
+		else if (!ISDIRTY(c->sector_size - (jeb->used_size + jeb->unchecked_size))) {
+			c->dirty_size -= jeb->dirty_size;
+			c->wasted_size += jeb->dirty_size; 
+			jeb->wasted_size += jeb->dirty_size;
+			jeb->dirty_size = 0;
+			return BLK_STATE_CLEAN;
+		}
+		else if (jeb->used_size || jeb->unchecked_size) { 
+				return BLK_STATE_PARTDIRTY; 
+		}
+		else { 
+				return BLK_STATE_ALLDIRTY; 
+		}
+	}
+	
+	return 0;
+}
+
+/* Write out summary information - called from jffs2_do_reserve_space */
+
+int jffs2_sum_write_sumnode(struct jffs2_sb_info *c)
+{
+	struct jffs2_summary_node isum;
+	union jffs2_sum_mem *temp;
+	jint32_t offset;
+	jint32_t *wpage;
+	uint8_t *tempptr;
+	int datasize;
+	int infosize;
+	int padsize;
+	size_t retlen;
+	int ret;
+	struct jffs2_eraseblock *jeb;
+	struct kvec vecs[2];	
+	jint32_t magic = cpu_to_je32(JFFS2_SUM_MAGIC);
+		
+	D2(printk(KERN_DEBUG "jffs2_sum_write_sumnode()\n"));
+	
+	jeb = c->nextblock;
+	
+	if (!jeb->sum_collected->sum_num || !jeb->sum_collected->sum_list) {
+		printk(KERN_WARNING "JFFS2: jffs2_sum_write_sumnode(): empty summary info!!!\n");
+		BUG(); 
+	}
+	
+	datasize = jeb->sum_collected->sum_size + sizeof(struct jffs2_sum_marker);
+	infosize = sizeof(struct jffs2_summary_node) + datasize;
+	padsize = jeb->free_size - infosize;
+	infosize += padsize; datasize += padsize;
+	offset = cpu_to_je32(c->sector_size - jeb->free_size);
+	
+	if (padsize < 0) { // if jeb hasn't got enought free space for summary
+		
+		union jffs2_sum_mem *temp;	
+		
+		while(jeb->sum_collected->sum_list){ //cleanup sum_list
+			temp = jeb->sum_collected->sum_list;
+			jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next;
+			kfree(temp);
+			jeb->sum_collected->sum_num--;
+		}
+		
+		jeb->sum_collected->sum_list = NULL;
+		jeb->sum_collected->sum_num = 0;
+		jeb->sum_collected->sum_size = JFFS2_SUMMARY_NOSUM_SIZE; // don't try to write out summary for this node
+		
+		printk(KERN_WARNING "JFFS2: not enough space for summary, padsize = %d\n",padsize);
+                return 0;
+	}
+			
+	memset(c->summary_buf, 0xff, datasize);
+	memset(&isum, 0, sizeof(isum));
+	
+	isum.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
+	isum.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY);
+	isum.totlen = cpu_to_je32(infosize);
+	isum.hdr_crc = cpu_to_je32(crc32(0, &isum, sizeof(struct jffs2_unknown_node) - 4));
+        isum.padded = cpu_to_je32(jeb->sum_collected->sum_padded);
+		
+	if (c->cleanmarker_size) {
+		isum.cln_mkr = cpu_to_je32(c->cleanmarker_size);	
+	}
+	else{
+		isum.cln_mkr = cpu_to_je32(0);
+	}
+	
+	isum.sum_num = cpu_to_je16(jeb->sum_collected->sum_num);
+	wpage = c->summary_buf;
+	
+		
+	while (jeb->sum_collected->sum_num) { // write sum_data 
+		
+
+		switch(je16_to_cpu(jeb->sum_collected->sum_list->u.nodetype)){
+			
+			case JFFS2_NODETYPE_INODE : {
+				jint16_t *temp16ptr = (jint16_t *)wpage;
+				
+				*(temp16ptr++) = jeb->sum_collected->sum_list->i.nodetype;
+				wpage = (jint32_t *) temp16ptr;
+				
+				*(wpage++) = jeb->sum_collected->sum_list->i.inode;
+				*(wpage++) = jeb->sum_collected->sum_list->i.version;
+				*(wpage++) = jeb->sum_collected->sum_list->i.offset;
+				*(wpage++) = jeb->sum_collected->sum_list->i.totlen;
+				break;
+			}
+			
+			case JFFS2_NODETYPE_DIRENT : {
+				jint16_t *temp16ptr = (jint16_t *) wpage;
+				uint8_t *temp8ptr = NULL;
+				
+				*(temp16ptr++) = jeb->sum_collected->sum_list->d.nodetype;
+				wpage = (jint32_t *) temp16ptr;
+				
+				*(wpage++) = jeb->sum_collected->sum_list->d.totlen;
+				*(wpage++) = jeb->sum_collected->sum_list->d.offset;
+				*(wpage++) = jeb->sum_collected->sum_list->d.pino;
+				*(wpage++) = jeb->sum_collected->sum_list->d.version;
+				*(wpage++) = jeb->sum_collected->sum_list->d.ino;
+				
+				temp8ptr = (uint8_t *) wpage;
+				*(temp8ptr++) = jeb->sum_collected->sum_list->d.nsize;
+				*(temp8ptr++) = jeb->sum_collected->sum_list->d.type;
+								
+				memcpy(temp8ptr,jeb->sum_collected->sum_list->d.name,jeb->sum_collected->sum_list->d.nsize);
+				temp8ptr += jeb->sum_collected->sum_list->d.nsize;
+				wpage = (jint32_t *) temp8ptr;
+				
+				break;
+			}
+			
+			default : {
+				printk(KERN_WARNING "Unknown node in summary information!!!\n");
+				BUG();
+			}
+		}
+		
+		temp = jeb->sum_collected->sum_list;
+		jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next;
+		kfree(temp);
+		
+		jeb->sum_collected->sum_num--;
+	}
+	
+	jeb->sum_collected->sum_size = 0;
+	
+	tempptr = (uint8_t *) wpage;
+	tempptr += padsize;
+	wpage = (jint32_t *) tempptr;
+	
+	*(wpage++) = offset;
+	*(wpage++) = cpu_to_je32(c->sector_size);
+	*(wpage++) = magic;
+	isum.sum_crc = cpu_to_je32(crc32(0, c->summary_buf, datasize));
+	isum.node_crc = cpu_to_je32(crc32(0, &isum, sizeof(isum) - 8));
+	
+	vecs[0].iov_base = &isum;
+	vecs[0].iov_len = sizeof(isum);
+	vecs[1].iov_base = c->summary_buf;
+	vecs[1].iov_len = datasize;
+			
+	D1(printk(KERN_DEBUG "JFFS2: writing out data to flash to pos : 0x%08x\n",jeb->offset + c->sector_size - jeb->free_size));
+	
+	spin_unlock(&c->erase_completion_lock);
+	ret = jffs2_flash_writev(c, vecs, 2, jeb->offset + c->sector_size - jeb->free_size, &retlen, 0);
+	spin_lock(&c->erase_completion_lock);
+
+	
+	if (ret || (retlen != infosize)) {
+		printk(KERN_WARNING "JFFS2: write of %zd bytes at 0x%08x failed. returned %d, retlen %zd\n", 
+		      infosize, jeb->offset + c->sector_size - jeb->free_size, ret, retlen);
+
+		jeb->sum_collected->sum_size = JFFS2_SUMMARY_NOSUM_SIZE;
+				
+		WASTED_SPACE(infosize);
+		
+		return 0;
+	}	
+	
+	/*spin_unlock(&c->erase_completion_lock);
+	jffs2_flush_wbuf_pad(c); // summary filled the wbuf
+	spin_lock(&c->erase_completion_lock);*/
+
+	WASTED_SPACE(infosize);
+	
+	return 0;	
+}
diff --unified --new-file --recursive mtd/fs/jffs2/summary.h mtd-dec07/fs/jffs2/summary.h
--- mtd/fs/jffs2/summary.h	1970-01-01 01:00:00.000000000 +0100
+++ mtd-dec07/fs/jffs2/summary.h	2004-12-07 16:03:02.000000000 +0100
@@ -0,0 +1,158 @@
+/*
+ * JFFS2 -- Journalling Flash File System, Version 2.
+ *
+ * Copyright (C) 2004  Ferenc Havasi <havasi@inf.u-szeged.hu>,
+ *                     Zoltan Sogor <weth@inf.u-szeged.hu>,
+ *                     Patrik Kluba <pajko@halom.u-szeged.hu>,
+ *                     University of Szeged, Hungary
+ *
+ * For licensing information, see the file 'LICENCE' in this directory.
+ *
+ * $Id$
+ *
+ */
+
+#ifndef JFFS2_SUMMARY_H
+#define JFFS2_SUMMARY_H
+
+#include <linux/uio.h>
+#include <linux/jffs2.h>
+
+#define DIRTY_SPACE(x) do { typeof(x) _x = (x); \
+		c->free_size -= _x; c->dirty_size += _x; \
+		jeb->free_size -= _x ; jeb->dirty_size += _x; \
+		}while(0)
+#define USED_SPACE(x) do { typeof(x) _x = (x); \
+		c->free_size -= _x; c->used_size += _x; \
+		jeb->free_size -= _x ; jeb->used_size += _x; \
+		}while(0)
+#define WASTED_SPACE(x) do { typeof(x) _x = (x); \
+		c->free_size -= _x; c->wasted_size += _x; \
+		jeb->free_size -= _x ; jeb->wasted_size += _x; \
+		}while(0)
+#define UNCHECKED_SPACE(x) do { typeof(x) _x = (x); \
+		c->free_size -= _x; c->unchecked_size += _x; \
+		jeb->free_size -= _x ; jeb->unchecked_size += _x; \
+		}while(0)
+
+#define BLK_STATE_ALLFF		0
+#define BLK_STATE_CLEAN		1
+#define BLK_STATE_PARTDIRTY	2
+#define BLK_STATE_CLEANMARKER	3
+#define BLK_STATE_ALLDIRTY	4
+#define BLK_STATE_BADBLOCK	5
+
+#define JFFS2_SUMMARY_NOSUM_SIZE 0xffffffff
+#define JFFS2_SUMMARY_INODE_SIZE (sizeof(struct jffs2_sum_inode_flash))
+#define JFFS2_SUMMARY_DIRENT_SIZE(x) (sizeof(struct jffs2_sum_dirent_flash) + (x))
+
+struct jffs2_sum_unknown_flash
+{
+	jint16_t nodetype;	/* node type	*/
+};
+
+struct jffs2_sum_inode_flash
+{
+	jint16_t nodetype;	/* node type	*/
+	jint32_t inode;		/* inode number */
+	jint32_t version;	/* inode version */
+	jint32_t offset;	/* offset on jeb */
+	jint32_t totlen; 	/* record length */
+} __attribute__((packed));
+
+struct jffs2_sum_dirent_flash
+{
+	jint16_t nodetype;	/* == JFFS_NODETYPE_DIRENT */
+	jint32_t totlen;	/* record length */
+	jint32_t offset;	/* ofset on jeb */
+	jint32_t pino;		/* parent inode */
+	jint32_t version;	/* dirent version */
+	jint32_t ino; 		/* == zero for unlink */
+	uint8_t nsize;		/* dirent name size */
+	uint8_t type;		/* dirent type */
+	uint8_t name[0];	/* dirent name */
+} __attribute__((packed));
+
+union jffs2_sum_flash{
+	struct jffs2_sum_unknown_flash u;
+	struct jffs2_sum_inode_flash i;
+	struct jffs2_sum_dirent_flash d; 
+};
+
+/* list version of jffs2_sum_*flash for kernel and sumtool */
+struct jffs2_sum_unknown_mem
+{
+	union jffs2_sum_mem *next;
+	jint16_t nodetype;	/* node type	*/
+	
+};
+
+struct jffs2_sum_inode_mem
+{
+	union jffs2_sum_mem *next;
+	jint16_t nodetype;	/* node type	*/
+	jint32_t inode;		/* inode number */
+	jint32_t version;	/* inode version */
+	jint32_t offset;	/* offset on jeb */
+	jint32_t totlen; 	/* record length */
+} __attribute__((packed));
+
+struct jffs2_sum_dirent_mem
+{
+	union jffs2_sum_mem *next;
+	jint16_t nodetype;	/* == JFFS_NODETYPE_DIRENT */
+	jint32_t totlen;	/* record length */
+	jint32_t offset;	/* ofset on jeb */
+	jint32_t pino;		/* parent inode */
+	jint32_t version;	/* dirent version */
+	jint32_t ino; 		/* == zero for unlink */
+	uint8_t nsize;		/* dirent name size */
+	uint8_t type;		/* dirent type */
+	uint8_t name[0];	/* dirent name */
+} __attribute__((packed));
+
+union jffs2_sum_mem 
+{
+	struct jffs2_sum_unknown_mem u;
+	struct jffs2_sum_inode_mem i;
+	struct jffs2_sum_dirent_mem d; 
+};
+
+struct jffs2_sum_info
+{
+	uint32_t sum_size;
+	uint32_t sum_num;
+        uint32_t sum_padded;
+	union jffs2_sum_mem *sum_list;
+};
+
+struct jffs2_sum_marker
+{
+	jint32_t offset;
+	jint32_t erase_size;
+	jint32_t magic;
+};
+
+#define JFFS2_SUMMARY_FRAME_SIZE (sizeof(struct jffs2_summary_node)+sizeof(struct jffs2_sum_marker)+3)
+	
+#if !(defined(SUM_TOOL) || defined(JFFS2DUMP))
+
+int jffs2_sum_init(struct jffs2_sb_info *c);
+void jffs2_sum_exit(struct jffs2_sb_info *c);
+
+int jffs2_sum_care_sum_collected(struct jffs2_eraseblock *jeb);
+
+void jffs2_sum_clean_collected(struct jffs2_eraseblock *jeb);
+void jffs2_sum_clean_all_info(struct jffs2_sb_info *c); /* clean up all summary information in all jeb (umount) */
+
+int jffs2_sum_add_padding_mem(struct jffs2_eraseblock *jeb, uint32_t size);
+int jffs2_sum_add_inode_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_inode *ri, uint32_t ofs);	
+int jffs2_sum_add_dirent_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_dirent *rd, uint32_t ofs);	
+int jffs2_sum_add_kvec(struct jffs2_sb_info *c, const struct kvec *invecs, unsigned long count,  uint32_t to);	
+
+int jffs2_sum_scan_sumnode(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb, uint32_t ofs, uint32_t *pseudo_random);
+int jffs2_sum_write_sumnode(struct jffs2_sb_info *c);	
+
+#endif
+
+#endif /* JFFS2_SUMMARY_H */
diff --unified --new-file --recursive mtd/fs/jffs2/super-v24.c mtd-dec07/fs/jffs2/super-v24.c
--- mtd/fs/jffs2/super-v24.c	2004-11-17 00:00:14.000000000 +0100
+++ mtd-dec07/fs/jffs2/super-v24.c	2004-12-07 16:03:02.000000000 +0100
@@ -24,6 +24,7 @@
 #include <linux/mtd/mtd.h>
 #include "compr.h"
 #include "nodelist.h"
+#include "summary.h"
 
 #ifndef MTD_BLOCK_MAJOR
 #define MTD_BLOCK_MAJOR 31
@@ -72,6 +73,9 @@
 		put_mtd_device(c->mtd);
 		return NULL;
 	}
+#ifdef CONFIG_JFFS2_SUMMARY
+        jffs2_sum_init(c);
+#endif
 
 	return sb;
 }
@@ -88,6 +92,10 @@
 	down(&c->alloc_sem);
 	jffs2_flush_wbuf_pad(c);
 	up(&c->alloc_sem);
+#ifdef CONFIG_JFFS2_SUMMARY
+        jffs2_sum_clean_all_info(c);
+        jffs2_sum_exit(c);
+#endif
 	jffs2_free_ino_caches(c);
 	jffs2_free_raw_node_refs(c);
 	kfree(c->blocks);
@@ -110,6 +118,9 @@
 #ifdef CONFIG_JFFS2_FS_NAND
 	       " (NAND)"
 #endif
+#ifdef CONFIG_JFFS2_SUMMARY
+	       " (SUMMARY)"
+#endif
 	       " (C) 2001-2003 Red Hat, Inc.\n");
 
 #ifdef JFFS2_OUT_OF_KERNEL
diff --unified --new-file --recursive mtd/fs/jffs2/super.c mtd-dec07/fs/jffs2/super.c
--- mtd/fs/jffs2/super.c	2004-11-26 15:08:31.000000000 +0100
+++ mtd-dec07/fs/jffs2/super.c	2004-12-07 16:03:02.000000000 +0100
@@ -152,6 +152,10 @@
 		return ERR_PTR(ret);
 	}
 
+#ifdef CONFIG_JFFS2_SUMMARY
+        jffs2_sum_init(c);
+#endif
+
 	sb->s_flags |= MS_ACTIVE;
 	return sb;
 
@@ -275,6 +279,10 @@
 	down(&c->alloc_sem);
 	jffs2_flush_wbuf_pad(c);
 	up(&c->alloc_sem);
+#ifdef CONFIG_JFFS2_SUMMARY
+        jffs2_sum_clean_all_info(c);
+        jffs2_sum_exit(c);
+#endif
 	jffs2_free_ino_caches(c);
 	jffs2_free_raw_node_refs(c);
 	if (c->mtd->flags & MTD_NO_VIRTBLOCKS)
@@ -312,6 +320,9 @@
 #ifdef CONFIG_JFFS2_FS_NAND
 	       " (NAND)"
 #endif
+#ifdef CONFIG_JFFS2_SUMMARY
+               " (SUMMARY) "
+#endif
 	       " (C) 2001-2003 Red Hat, Inc.\n");
 
 	jffs2_inode_cachep = kmem_cache_create("jffs2_i",
diff --unified --new-file --recursive mtd/fs/jffs2/wbuf.c mtd-dec07/fs/jffs2/wbuf.c
--- mtd/fs/jffs2/wbuf.c	2004-11-26 15:08:31.000000000 +0100
+++ mtd-dec07/fs/jffs2/wbuf.c	2004-12-07 16:17:11.000000000 +0100
@@ -261,7 +261,7 @@
 
 
 	/* ... and get an allocation of space from a shiny new block instead */
-	ret = jffs2_reserve_space_gc(c, end-start, &ofs, &len);
+	ret = jffs2_reserve_space_gc(c, end-start, &ofs, &len, JFFS2_SUMMARY_NOSUM_SIZE);
 	if (ret) {
 		printk(KERN_WARNING "Failed to allocate space for wbuf recovery. Data loss ensues.\n");
 		if (buf)
@@ -601,8 +601,15 @@
 	uint32_t outvec_to = to;
 
 	/* If not NAND flash, don't bother */
-	if (!c->wbuf)
+ 	if (!c->wbuf){
+#ifdef CONFIG_JFFS2_SUMMARY
+ 		D1(printk("JFFS2.summary: NOT NAND MEMORY\n"));
+ 		if(jffs2_sum_add_kvec(c,invecs,count,(uint32_t) to)){
+ 			printk("jffs2_sum_add_kvec(): MEMORY ALLOCATION ERROR!");
+ 		}	
+#endif		
 		return jffs2_flash_direct_writev(c, invecs, count, to, retlen);
+	}
 	
 	down_write(&c->wbuf_sem);
 
@@ -801,6 +808,12 @@
 alldone:
 	*retlen = donelen;
 
+#ifdef CONFIG_JFFS2_SUMMARY
+	if(jffs2_sum_add_kvec(c,invecs,count,(uint32_t) to)){
+		printk("jffs2_sum_add_kvec(): MEMORY ALLOCATION ERROR!");
+	}
+#endif
+	
 	if (c->wbuf_len && ino)
 		jffs2_wbuf_dirties_inode(c, ino);
 
diff --unified --new-file --recursive mtd/fs/jffs2/write.c mtd-dec07/fs/jffs2/write.c
--- mtd/fs/jffs2/write.c	2004-11-17 00:00:14.000000000 +0100
+++ mtd-dec07/fs/jffs2/write.c	2004-12-07 16:03:02.000000000 +0100
@@ -173,13 +173,13 @@
 			D1(ACCT_PARANOIA_CHECK(jeb));
 
 			if (alloc_mode == ALLOC_GC) {
-				ret = jffs2_reserve_space_gc(c, sizeof(*ri) + datalen, &flash_ofs, &dummy);
+				ret = jffs2_reserve_space_gc(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, JFFS2_SUMMARY_INODE_SIZE);
 			} else {
 				/* Locking pain */
 				up(&f->sem);
 				jffs2_complete_reservation(c);
 			
-				ret = jffs2_reserve_space(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, alloc_mode);
+				ret = jffs2_reserve_space(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, alloc_mode, JFFS2_SUMMARY_INODE_SIZE);
 				down(&f->sem);
 			}
 
@@ -308,13 +308,13 @@
 			D1(ACCT_PARANOIA_CHECK(jeb));
 
 			if (alloc_mode == ALLOC_GC) {
-				ret = jffs2_reserve_space_gc(c, sizeof(*rd) + namelen, &flash_ofs, &dummy);
+				ret = jffs2_reserve_space_gc(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
 			} else {
 				/* Locking pain */
 				up(&f->sem);
 				jffs2_complete_reservation(c);
 			
-				ret = jffs2_reserve_space(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, alloc_mode);
+				ret = jffs2_reserve_space(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, alloc_mode, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
 				down(&f->sem);
 			}
 
@@ -371,7 +371,7 @@
 	retry:
 		D2(printk(KERN_DEBUG "jffs2_commit_write() loop: 0x%x to write to 0x%x\n", writelen, offset));
 
-		ret = jffs2_reserve_space(c, sizeof(*ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, ALLOC_NORMAL);
+		ret = jffs2_reserve_space(c, sizeof(*ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
 		if (ret) {
 			D1(printk(KERN_DEBUG "jffs2_reserve_space returned %d\n", ret));
 			break;
@@ -458,7 +458,7 @@
 	/* Try to reserve enough space for both node and dirent. 
 	 * Just the node will do for now, though 
 	 */
-	ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL);
+	ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
 	D1(printk(KERN_DEBUG "jffs2_do_create(): reserved 0x%x bytes\n", alloclen));
 	if (ret) {
 		up(&f->sem);
@@ -487,7 +487,7 @@
 
 	up(&f->sem);
 	jffs2_complete_reservation(c);
-	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
+	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
 		
 	if (ret) {
 		/* Eep. */
@@ -557,7 +557,7 @@
 		if (!rd)
 			return -ENOMEM;
 
-		ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_DELETION);
+		ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_DELETION, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
 		if (ret) {
 			jffs2_free_raw_dirent(rd);
 			return ret;
@@ -663,7 +663,7 @@
 	if (!rd)
 		return -ENOMEM;
 
-	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
+	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
 	if (ret) {
 		jffs2_free_raw_dirent(rd);
 		return ret;
diff --unified --new-file --recursive mtd/include/linux/jffs2.h mtd-dec07/include/linux/jffs2.h
--- mtd/include/linux/jffs2.h	2004-11-17 00:00:15.000000000 +0100
+++ mtd-dec07/include/linux/jffs2.h	2004-12-07 16:03:02.000000000 +0100
@@ -28,6 +28,9 @@
 #define JFFS2_EMPTY_BITMASK 0xffff
 #define JFFS2_DIRTY_BITMASK 0x0000
 
+/* Summary node MAGIC marker */
+#define JFFS2_SUM_MAGIC	0x02851885 
+
 /* We only allow a single char for length, and 0xFF is empty flash so
    we don't want it confused with a real length. Hence max 254.
 */
@@ -62,6 +65,8 @@
 #define JFFS2_NODETYPE_CLEANMARKER (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3)
 #define JFFS2_NODETYPE_PADDING (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 4)
 
+#define JFFS2_NODETYPE_SUMMARY (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 6)
+
 // Maybe later...
 //#define JFFS2_NODETYPE_CHECKPOINT (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3)
 //#define JFFS2_NODETYPE_OPTIONS (JFFS2_FEATURE_RWCOMPAT_COPY | JFFS2_NODE_ACCURATE | 4)
@@ -148,10 +153,24 @@
 	uint8_t data[0];
 } __attribute__((packed));
 
+struct jffs2_summary_node{
+    jint16_t magic;
+	jint16_t nodetype; 	/* = JFFS2_NODETYPE_INODE_SUM */
+	jint32_t totlen;
+	jint32_t hdr_crc;
+	jint16_t sum_num;	/* number of sum entries*/
+	jint32_t cln_mkr;	/* clean marker size, 0 = no cleanmarker */
+        jint32_t padded;        /* sum of the size of padding nodes */
+	jint32_t sum_crc;	/* summary information crc */
+	jint32_t node_crc; 	/* node crc */
+	jint32_t sum[0]; 	/* inode summary info */
+} __attribute__((packed));
+
 union jffs2_node_union {
 	struct jffs2_raw_inode i;
 	struct jffs2_raw_dirent d;
 	struct jffs2_unknown_node u;
+	struct jffs2_summary_node s;
 };
 
 #endif /* __LINUX_JFFS2_H__ */
diff --unified --new-file --recursive mtd/include/linux/jffs2_fs_sb.h mtd-dec07/include/linux/jffs2_fs_sb.h
--- mtd/include/linux/jffs2_fs_sb.h	2004-11-26 15:08:31.000000000 +0100
+++ mtd-dec07/include/linux/jffs2_fs_sb.h	2004-12-07 16:03:02.000000000 +0100
@@ -110,6 +110,9 @@
 	uint32_t fsdata_pos;
 	uint32_t fsdata_len;
 #endif
+#ifdef CONFIG_JFFS2_SUMMARY
+        jint32_t *summary_buf;
+#endif
 
 	/* OS-private pointer for getting back to master superblock info */
 	void *os_priv;
diff --unified --new-file --recursive mtd/util/Makefile mtd-dec07/util/Makefile
--- mtd/util/Makefile	2004-07-14 00:00:18.000000000 +0200
+++ mtd-dec07/util/Makefile	2004-12-07 16:03:02.000000000 +0100
@@ -13,7 +13,7 @@
 TARGETS = ftl_format flash_erase flash_eraseall nanddump doc_loadbios \
 	mkfs.jffs ftl_check mkfs.jffs2 flash_lock flash_unlock \
 	flash_info mtd_debug flashcp nandwrite jffs2dump \
-	nftldump nftl_format docfdisk #jffs2reader
+	nftldump nftl_format docfdisk sumtool #jffs2reader
 
 SYMLINKS = compr_lzari.c compr_lzo.c
 
@@ -48,6 +48,10 @@
 jffs2dump: jffs2dump.o crc32.o
 	$(CC) $(LDFLAGS) -o $@ $^
 
+sumtool: sumtool.o crc32.o
+	$(CC) $(LDFLAGS) -o $@ $^
+	
+
 install: ${TARGETS}
 	mkdir -p ${DESTDIR}/${SBINDIR}
 	install -m0755 -oroot -groot ${TARGETS} ${DESTDIR}/${SBINDIR}/
diff --unified --new-file --recursive mtd/util/jffs2dump.c mtd-dec07/util/jffs2dump.c
--- mtd/util/jffs2dump.c	2004-12-03 00:00:10.000000000 +0100
+++ mtd-dec07/util/jffs2dump.c	2004-12-07 16:03:02.000000000 +0100
@@ -35,6 +35,10 @@
 #include <getopt.h>
 #include "crc32.h"
 
+#define CONFIG_JFFS2_SUMMARY
+#define JFFS2DUMP
+#include "../fs/jffs2/summary.h"
+
 #define PROGRAM "jffs2dump"
 #define VERSION "$Revision: 1.7 $"
 
@@ -177,7 +181,7 @@
  */
 void do_dumpcontent (void) 
 {
-	char			*p = data;
+	char			*p = data, *p_free_begin;
 	union jffs2_node_union 	*node;
 	int			empty = 0, dirty = 0;
 	char			name[256];
@@ -186,16 +190,21 @@
 	int			bitchbitmask = 0;
 	int			obsolete;
 
+	p_free_begin = NULL;
 	while ( p < (data + imglen)) {
 		node = (union jffs2_node_union*) p;
 		
 		/* Skip empty space */
+		if (!p_free_begin) p_free_begin = p;
 		if (je16_to_cpu (node->u.magic) == 0xFFFF && je16_to_cpu (node->u.nodetype) == 0xFFFF) {
 			p += 4;
 			empty += 4;
 			continue;
 		}
-		
+
+		if (p != p_free_begin) printf("Empty space found from 0x%08x to 0x%08x\n", p_free_begin-data, p-data);
+		p_free_begin = NULL;
+
 		if (je16_to_cpu (node->u.magic) != JFFS2_MAGIC_BITMASK)	{
 			if (!bitchbitmask++)
     			    printf ("Wrong bitmask  at  0x%08x, 0x%04x\n", p - data, je16_to_cpu (node->u.magic));
@@ -277,7 +286,110 @@
 
 			p += PAD(je32_to_cpu (node->d.totlen));						
 			break;
-	
+
+		case JFFS2_NODETYPE_SUMMARY:{
+			
+			int i;
+			struct jffs2_sum_marker * sm;
+			
+			printf ("%8s Inode Sum  node at 0x%08x, totlen 0x%08x, sum_num  %5d, cleanmarker size %5d\n",
+					obsolete ? "Obsolete" : "",
+					p - data,
+					je32_to_cpu (node->s.totlen),
+					je16_to_cpu (node->s.sum_num),
+					je32_to_cpu (node->s.cln_mkr));
+
+			crc = crc32 (0, node, sizeof (struct jffs2_summary_node) - 8);
+			if (crc != je32_to_cpu (node->s.node_crc)) {
+				printf ("Wrong node_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.node_crc), crc);
+				p += PAD(je32_to_cpu (node->s.totlen));
+				dirty += PAD(je32_to_cpu (node->s.totlen));;
+				continue;
+			}
+			
+			crc = crc32(0, p + sizeof (struct jffs2_summary_node),  je32_to_cpu (node->s.totlen) - sizeof(struct jffs2_summary_node));
+			if (crc != je32_to_cpu(node->s.sum_crc)) {
+				printf ("Wrong data_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.sum_crc), crc);
+				p += PAD(je32_to_cpu (node->s.totlen));
+				dirty += PAD(je32_to_cpu (node->s.totlen));;
+				continue;
+			}
+
+			if(verbose){
+				struct jffs2_sum_unknown_flash *sp;
+				uint8_t *temp8ptr = NULL;
+				sp = (struct jffs2_sum_unknown_flash *) (p + sizeof (struct jffs2_summary_node));
+					
+				for(i = 0; i < je16_to_cpu (node->s.sum_num); i++){
+					
+					switch(je16_to_cpu(sp->nodetype)){
+						case JFFS2_NODETYPE_INODE : {
+							
+							struct jffs2_sum_inode_flash *spi;
+							spi = (struct jffs2_sum_inode_flash *) sp;
+								
+							printf ("%14s #ino  %5d,  version %5d, offset 0x%08x, totlen 0x%08x\n",
+							"",
+							je32_to_cpu (spi->inode),
+							je32_to_cpu (spi->version),
+							je32_to_cpu (spi->offset), 
+							je32_to_cpu (spi->totlen));
+							
+							temp8ptr = (uint8_t *) sp;
+							temp8ptr += JFFS2_SUMMARY_INODE_SIZE;
+							sp = (struct jffs2_sum_unknown_flash *) temp8ptr;
+							
+							break;	
+						}
+						
+						case JFFS2_NODETYPE_DIRENT : {
+							
+							char name[255];
+							struct jffs2_sum_dirent_flash *spd;
+							spd = (struct jffs2_sum_dirent_flash *) sp;
+							
+							memcpy(name,spd->name,spd->nsize);
+							name [spd->nsize] = 0x0;
+							
+							printf ("%14s dirent offset 0x%08x, totlen 0x%08x, #pino  %5d,  version %5d, #ino  %8d, nsize %8d, name %s \n",
+							"",
+							je32_to_cpu (spd->offset), 
+							je32_to_cpu (spd->totlen),
+							je32_to_cpu (spd->pino),
+							je32_to_cpu (spd->version),
+							je32_to_cpu (spd->ino),
+							spd->nsize,
+							name);
+							
+							temp8ptr = (uint8_t *) sp;
+							temp8ptr += JFFS2_SUMMARY_DIRENT_SIZE(spd->nsize); 
+							sp = (struct jffs2_sum_unknown_flash *) temp8ptr;
+							
+							break;
+						}
+						
+						default : {
+							break;
+						}
+					}	
+				}
+				
+				
+				sm = (struct jffs2_sum_marker *) ((char *)p + je32_to_cpu(node->s.totlen) - sizeof(struct jffs2_sum_marker));
+				
+				printf("%14s Sum Node Offset  0x%08x, Erase_block_size 0x%08x, Magic 0x%08x, Padded size 0x%08x\n" ,
+					"",
+					je32_to_cpu(sm->offset),
+					je32_to_cpu(sm->erase_size),
+					je32_to_cpu(sm->magic),
+                                        je32_to_cpu(node->s.padded));
+			}
+			
+			p += PAD(je32_to_cpu (node->s.totlen));
+			break;
+			
+		}
+			
 		case JFFS2_NODETYPE_CLEANMARKER:
 			if (verbose) {
 				printf ("%8s Cleanmarker     at 0x%08x, totlen 0x%08x\n", 
diff --unified --new-file --recursive mtd/util/sumtool.c mtd-dec07/util/sumtool.c
--- mtd/util/sumtool.c	1970-01-01 01:00:00.000000000 +0100
+++ mtd-dec07/util/sumtool.c	2004-12-07 17:47:31.000000000 +0100
@@ -0,0 +1,857 @@
+/*
+ *  sumtool.c
+ *
+ *  Copyright (C) 2004 Zoltan Sogor <weth@inf.u-szeged.hu>,
+ *                     Ferenc Havasi <havasi@inf.u-szeged.hu>,
+ *                     Patrik Kluba <pajko@halom.u-szeged.hu>,
+ *                     University of Szeged, Hungary
+ *
+ * $Id: sumtool.c,v 1.2 2004/10/20 09:56:08 hafy Exp $
+ * 
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version 2
+ * of the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
+ *
+ * Overview:
+ *   This is a utility to reorder nodes and insert inode summary information
+ *   into JFFS2 image for faster mount time - specially on NAND.
+ *
+ */
+
+#include <errno.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <string.h>
+#include <unistd.h>
+#include <fcntl.h>
+#include <time.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/param.h>
+#include <asm/types.h>
+#include <dirent.h>
+#include <mtd/jffs2-user.h>
+#include <endian.h>
+#include <byteswap.h>
+#include <getopt.h>
+#include "crc32.h"
+
+#define CONFIG_JFFS2_SUMMARY
+#define SUM_TOOL
+#include "../fs/jffs2/summary.h"
+
+
+#define PAD(x) (((x)+3)&~3)
+
+static const char *const app_name = "sumtool";
+
+static struct jffs2_sum_info *sum_collected = NULL;/* summary info list */
+
+static int verbose = 0;
+static int padto = 0;						/* pad the output with 0xFF to the end of the final eraseblock */
+static int add_cleanmarkers = 1;			/* add cleanmarker to output */
+static int use_input_cleanmarker_size = 1;	/* use input file's cleanmarker size (default) */
+static int found_cleanmarkers = 0;			/* cleanmarker found in input file */
+static struct jffs2_unknown_node cleanmarker;
+static int cleanmarker_size = sizeof(cleanmarker);
+static const char *short_options = "o:i:e:hvVblnc:p";
+static int erase_block_size = 65536;
+static int target_endian = __BYTE_ORDER;
+static int out_fd = -1;
+static int in_fd = -1;
+
+static uint8_t *data_buffer = NULL; 	/* buffer for inodes */
+static unsigned int data_ofs = 0;	 	/* inode buffer offset */
+
+static uint8_t *file_buffer = NULL;		/* file buffer contains the actual erase block*/
+static unsigned int file_ofs = 0;		/* position in the buffer */
+
+static struct option long_options[] = {
+	{"output", 1, NULL, 'o'},
+	{"input", 1, NULL, 'i'},
+	{"eraseblock", 1, NULL, 'e'},
+	{"help", 0, NULL, 'h'},
+	{"verbose", 0, NULL, 'v'},
+	{"version", 0, NULL, 'V'},
+	{"bigendian", 0, NULL, 'b'},
+	{"littleendian", 0, NULL, 'l'},	
+	{"no-cleanmarkers", 0, NULL, 'n'},
+	{"cleanmarker", 1, NULL, 'c'},
+	{"pad", 0, NULL, 'p'},
+	{NULL, 0, NULL, 0}
+};
+
+static char *helptext =
+	"Usage: sumtool [OPTIONS] -i inputfile -o outputfile\n"
+	"Convert the input JFFS2 file to a SUM-ed JFFS2 file\n\n"
+	"Options:\n"
+	"  -e, --eraseblock=SIZE     Use erase block size SIZE (default: 64KiB)\n"
+	"                            (usually 16KiB on NAND)\n"
+	"  -c, --cleanmarker=SIZE    Size of cleanmarker (default 12).\n"
+	"                            (usually 16 bytes on NAND, and will be set to\n"
+	"                            this value if left at the default 12). Will be\n"
+	"                            stored in OOB after each physical page composing\n"
+	"                            a physical erase block.\n"
+	"  -n, --no-cleanmarkers     Don't add a cleanmarker to every eraseblock\n"
+	"  -o, --output=FILE         Output to FILE \n"
+	"  -i, --input=FILE          Input from FILE \n"
+	"  -b, --bigendian           Image is big endian\n"
+	"  -l  --littleendian        Image is little endian\n"
+	"  -h, --help                Display this help text\n"
+	"  -v, --verbose             Verbose operation\n"
+	"  -V, --version             Display version information\n"
+	"  -p, --pad                 Pad the OUTPUT with 0xFF to the end of the final\n"
+	"                            eraseblock\n\n";
+
+
+static char *revtext = "$Revision: 1.3 $";
+
+static unsigned char ffbuf[16] = {
+	0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
+	0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff
+};
+
+static void verror_msg(const char *s, va_list p) {
+	fflush(stdout);
+	fprintf(stderr, "%s: ", app_name);
+	vfprintf(stderr, s, p);
+}
+
+static void error_msg_and_die(const char *s, ...) {
+	va_list p;
+
+	va_start(p, s);
+	verror_msg(s, p);
+	va_end(p);
+	putc('\n', stderr);
+	exit(EXIT_FAILURE);
+}
+
+static void vperror_msg(const char *s, va_list p) {
+	int err = errno;
+
+	if (s == 0)
+		s = "";
+	verror_msg(s, p);
+	if (*s)
+		s = ": ";
+	fprintf(stderr, "%s%s\n", s, strerror(err));
+}
+
+static void perror_msg_and_die(const char *s, ...) {
+	va_list p;
+
+	va_start(p, s);
+	vperror_msg(s, p);
+	va_end(p);
+	exit(EXIT_FAILURE);
+}
+
+
+
+static void full_write(void *target_buff, const void *buf, int len);
+
+void setup_cleanmarker() {
+
+	cleanmarker.magic    = cpu_to_je16(JFFS2_MAGIC_BITMASK);
+	cleanmarker.nodetype = cpu_to_je16(JFFS2_NODETYPE_CLEANMARKER);
+	cleanmarker.totlen   = cpu_to_je32(cleanmarker_size);
+	cleanmarker.hdr_crc  = cpu_to_je32(crc32(0, &cleanmarker, sizeof(struct jffs2_unknown_node)-4));
+}
+
+void process_options (int argc, char **argv){
+	int opt,c;
+	
+	while ((opt = getopt_long(argc, argv, short_options, long_options, &c)) >= 0) 
+	{
+		switch (opt) 
+		{
+			case 'o':
+				if (out_fd != -1) {
+					error_msg_and_die("output filename specified more than once");
+				}
+				out_fd = open(optarg, O_CREAT | O_TRUNC | O_RDWR, 0644);
+				if (out_fd == -1) {
+					perror_msg_and_die("open output file");
+				}
+				break;
+				
+			case 'i':
+				if (in_fd != -1) {
+					error_msg_and_die("input filename specified more than once");
+				}
+				in_fd = open(optarg, O_RDONLY);
+				if (in_fd == -1) {
+					perror_msg_and_die("open input file");
+				}
+				break;
+			case 'b':
+				target_endian = __BIG_ENDIAN;
+				break;
+			case 'l':
+				target_endian = __LITTLE_ENDIAN;
+				break;	
+			case 'h':
+			case '?':
+				error_msg_and_die(helptext);
+	
+			case 'v':
+				verbose = 1;
+				break;
+	
+			case 'V':
+				error_msg_and_die("revision %.*s\n",
+						(int) strlen(revtext) - 13, revtext + 11);
+	
+			case 'e': {
+				char *next;
+				unsigned units = 0;
+				erase_block_size = strtol(optarg, &next, 0);
+				if (!erase_block_size)
+					error_msg_and_die("Unrecognisable erase size\n");
+	
+				if (*next) {
+					if (!strcmp(next, "KiB")) {
+						units = 1024;
+					} else if (!strcmp(next, "MiB")) {
+						units = 1024 * 1024;
+					} else {
+						error_msg_and_die("Unknown units in erasesize\n");
+					}
+				} else {
+					if (erase_block_size < 0x1000)
+						units = 1024;
+					else
+						units = 1;
+				}
+				erase_block_size *= units;
+	
+				/* If it's less than 8KiB, they're not allowed */
+				if (erase_block_size < 0x2000) {
+					fprintf(stderr, "Erase size 0x%x too small. Increasing to 8KiB minimum\n",
+						erase_block_size);
+					erase_block_size = 0x2000;
+				}
+				break;
+			}
+
+			case 'n':
+				add_cleanmarkers = 0;
+				break;
+			case 'c':
+				cleanmarker_size = strtol(optarg, NULL, 0);
+			
+				if (cleanmarker_size < sizeof(cleanmarker)) {
+					error_msg_and_die("cleanmarker size must be >= 12");
+				}
+				if (cleanmarker_size >= erase_block_size) {
+					error_msg_and_die("cleanmarker size must be < eraseblock size");
+				}
+				
+				use_input_cleanmarker_size = 0;
+				found_cleanmarkers = 1;
+				setup_cleanmarker();
+				
+				break;
+			case 'p':
+				padto = 1;
+				break;
+			
+		}
+	}
+}
+
+
+void init_buffers() {
+	
+	data_buffer = malloc(erase_block_size);
+	
+	if (!data_buffer) {
+		perror("out of memory");
+		close (in_fd);
+		close (out_fd);
+		exit(1);
+	}
+		
+	file_buffer = malloc(erase_block_size);
+	
+	if (!file_buffer) {
+		perror("out of memory");
+		close (in_fd);
+		close (out_fd);
+		exit(1);
+	}
+}
+
+void init_sumlist(){
+	
+	sum_collected = (struct jffs2_sum_info *) malloc (sizeof(struct jffs2_sum_info));
+	
+	if (!sum_collected)
+		error_msg_and_die("Can't allocate memory for jffs2_sum_info!\n");
+	
+	sum_collected->sum_list = NULL;
+	sum_collected->sum_num = 0;
+	sum_collected->sum_size = 0; 
+
+}
+
+void clean_buffers() {
+	
+	if (data_buffer) 
+		free(data_buffer);
+	if (file_buffer)
+		free(file_buffer);
+}
+
+void clean_sumlist() {
+	
+	union jffs2_sum_mem *temp;
+	
+	if(sum_collected){
+		
+		while(sum_collected->sum_list){
+			temp = sum_collected->sum_list;
+			sum_collected->sum_list = sum_collected->sum_list->u.next;
+			free(temp);
+			sum_collected->sum_num--;
+		}
+		
+		if(sum_collected->sum_num != 0)
+			printf("Ooops, something wrong happened! sum_num != 0, but sum_list = null ???");
+		
+		free(sum_collected);
+	}	
+}
+
+int load_next_block() {
+	
+	int ret;
+	ret = read(in_fd, file_buffer, erase_block_size);
+	file_ofs = 0;
+	
+	if(verbose)
+		printf("Load next block : %d bytes read\n",ret);
+	
+	return ret;
+}
+
+void write_buff_to_file() {
+	
+	int ret;
+	int len = data_ofs;
+	
+	uint8_t *buf = NULL;
+	
+	buf = data_buffer;
+	while (len > 0) {
+		ret = write(out_fd, buf, len);
+	
+		if (ret < 0)
+			perror_msg_and_die("write");
+
+		if (ret == 0)
+			perror_msg_and_die("write returned zero");
+	
+		len -= ret;
+		buf += ret;
+	}
+	
+	data_ofs = 0;
+}
+
+void dump_sum_records() {
+	
+    struct jffs2_summary_node isum;
+    union jffs2_sum_mem *temp;
+	jint32_t offset;
+	jint32_t *wpage;
+	uint8_t *tempptr;
+	int datasize;
+	int infosize;
+	int padsize;
+	jint32_t magic = cpu_to_je32(JFFS2_SUM_MAGIC);
+	
+	if (!sum_collected->sum_num || !sum_collected->sum_list) 
+		return; 
+	
+	datasize = sum_collected->sum_size + sizeof(struct jffs2_sum_marker);
+	infosize = sizeof(struct jffs2_summary_node) + datasize;
+	padsize = erase_block_size - data_ofs - infosize;
+	infosize += padsize; datasize += padsize;
+	offset = cpu_to_je32(data_ofs);
+	
+	jint32_t *tpage = (jint32_t *) malloc(datasize);
+	
+	if(!tpage)
+		error_msg_and_die("Can't allocate memory to dump summary information!\n");
+	
+	memset(tpage, 0xff, datasize);
+	memset(&isum, 0, sizeof(isum));
+	
+	isum.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
+	isum.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY);
+	isum.totlen = cpu_to_je32(infosize);
+	isum.hdr_crc = cpu_to_je32(crc32(0, &isum, sizeof(struct jffs2_unknown_node) - 4));
+        isum.padded = cpu_to_je32(0);
+		
+	if (add_cleanmarkers && found_cleanmarkers) {
+		isum.cln_mkr = cpu_to_je32(cleanmarker_size);	
+	}
+	else{
+		isum.cln_mkr = cpu_to_je32(0);
+	}
+	
+	isum.sum_num = cpu_to_je16(sum_collected->sum_num);
+	wpage = tpage;
+	
+		
+	while (sum_collected->sum_num) {
+		
+
+		switch(je16_to_cpu(sum_collected->sum_list->u.nodetype)){
+			
+			case JFFS2_NODETYPE_INODE : {
+				jint16_t *temp16ptr = (jint16_t *)wpage;
+				
+				*(temp16ptr++) = sum_collected->sum_list->i.nodetype;
+				wpage = (jint32_t *) temp16ptr;
+				
+				*(wpage++) = sum_collected->sum_list->i.inode;
+				*(wpage++) = sum_collected->sum_list->i.version;
+				*(wpage++) = sum_collected->sum_list->i.offset;
+				*(wpage++) = sum_collected->sum_list->i.totlen;
+				break;
+			}
+			
+			case JFFS2_NODETYPE_DIRENT : {
+				jint16_t *temp16ptr = (jint16_t *) wpage;
+				uint8_t *temp8ptr = NULL;
+				
+				*(temp16ptr++) = sum_collected->sum_list->d.nodetype;
+				wpage = (jint32_t *) temp16ptr;
+				
+				*(wpage++) = sum_collected->sum_list->d.totlen;
+				*(wpage++) = sum_collected->sum_list->d.offset;
+				*(wpage++) = sum_collected->sum_list->d.pino;
+				*(wpage++) = sum_collected->sum_list->d.version;
+				*(wpage++) = sum_collected->sum_list->d.ino;
+				
+				temp8ptr = (uint8_t *) wpage;
+				*(temp8ptr++) = sum_collected->sum_list->d.nsize;
+				*(temp8ptr++) = sum_collected->sum_list->d.type;
+				
+				memcpy(temp8ptr,sum_collected->sum_list->d.name,sum_collected->sum_list->d.nsize);
+				temp8ptr += sum_collected->sum_list->d.nsize;
+				wpage = (jint32_t *) temp8ptr;
+				
+				break;
+			}
+			
+			default : {
+				printf("This is not good for me!!!!\n");
+			}
+		}
+		
+		temp = sum_collected->sum_list;
+		sum_collected->sum_list = sum_collected->sum_list->u.next;
+		free(temp);
+		
+		sum_collected->sum_num--;
+	}
+	
+	sum_collected->sum_size = 0;
+	
+	tempptr = (uint8_t *) wpage;
+	tempptr += padsize;
+	wpage = (jint32_t *) tempptr;
+	
+	*(wpage++) = offset;
+	*(wpage++) = cpu_to_je32(erase_block_size);
+	*(wpage++) = magic;
+	isum.sum_crc = cpu_to_je32(crc32(0, tpage, datasize));
+	isum.node_crc = cpu_to_je32(crc32(0, &isum, sizeof(isum) - 8));
+	
+	full_write(data_buffer + data_ofs, &isum, sizeof(isum));
+	full_write(data_buffer + data_ofs, tpage, datasize);
+	
+	free(tpage);
+
+}
+
+static void full_write(void *target_buff, const void *buf, int len) {
+	
+	memcpy(target_buff, buf, len);
+	data_ofs += len;
+}
+
+static void pad(int req) {
+	while (req) {
+		if (req > sizeof(ffbuf)) {
+			full_write(data_buffer + data_ofs, ffbuf, sizeof(ffbuf));
+			req -= sizeof(ffbuf);
+		} else {
+			full_write(data_buffer + data_ofs, ffbuf, req);
+			req = 0;
+		}
+	}
+} 
+
+static inline void padword() {
+	
+	if (data_ofs % 4) {
+		full_write(data_buffer + data_ofs, ffbuf, 4 - (data_ofs % 4));
+	}
+} 
+
+
+static inline void pad_block_if_less_than(int req,int plus) {
+	
+    int datasize = req + plus + sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8;
+    datasize += (4 - (datasize % 4)) % 4;
+	
+    if (data_ofs + req > erase_block_size - datasize) {
+        dump_sum_records();
+		write_buff_to_file();
+    }
+		
+	if (add_cleanmarkers && found_cleanmarkers) {
+		if (!data_ofs) {
+			full_write(data_buffer, &cleanmarker, sizeof(cleanmarker));
+			pad(cleanmarker_size - sizeof(cleanmarker));
+			padword();
+		}
+	}
+}
+
+void flush_buffers() {
+	
+	if ((add_cleanmarkers == 1) && (found_cleanmarkers == 1)) { /* CLEANMARKER */
+		if (data_ofs != cleanmarker_size) {	/* INODE BUFFER */
+			
+		    int datasize = sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8;
+		    datasize += (4 - (datasize % 4)) % 4;
+			
+			/* If we have a full inode buffer, then write out inode and summary data  */
+		    if (data_ofs + sizeof(struct jffs2_raw_inode) + 2*JFFS2_MIN_DATA_LEN > erase_block_size - datasize) {
+		        dump_sum_records();
+				write_buff_to_file();
+		    }
+			/* else just write out inode data */
+			else{
+				if(padto)
+					pad(erase_block_size - data_ofs);
+				write_buff_to_file();
+			}
+		}
+	}
+	else { /* NO CLEANMARKER */
+		if (data_ofs != 0) { /* INODE BUFFER */
+			
+		    int datasize = sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8;
+		    datasize += (4 - (datasize % 4)) % 4;
+			
+			/* If we have a full inode buffer, then write out inode and summary data */
+		    if (data_ofs + sizeof(struct jffs2_raw_inode) + 2*JFFS2_MIN_DATA_LEN > erase_block_size - datasize) {
+		        dump_sum_records();
+				write_buff_to_file();
+		    }
+			/* Else just write out inode data */
+			else{
+				if(padto)
+					pad(erase_block_size - data_ofs);
+				write_buff_to_file();
+			}
+		}
+	}
+}
+
+int add_sum_mem(union jffs2_sum_mem *item) {
+	
+	union jffs2_sum_mem *walk;
+	
+	if (!sum_collected->sum_list) {
+		sum_collected->sum_list = (union jffs2_sum_mem *) item;
+	} 
+	else {
+		walk = sum_collected->sum_list;
+		
+		while (walk->u.next) {
+			walk = walk->u.next;
+		}
+		walk->u.next = (union jffs2_sum_mem *) item;
+	}
+	switch (je16_to_cpu(item->u.nodetype)) {
+    	case JFFS2_NODETYPE_INODE:
+			sum_collected->sum_size += JFFS2_SUMMARY_INODE_SIZE;
+			sum_collected->sum_num++;
+			break;
+		
+    	case JFFS2_NODETYPE_DIRENT:
+			sum_collected->sum_size += JFFS2_SUMMARY_DIRENT_SIZE(item->d.nsize);
+			sum_collected->sum_num++;
+			break;
+		
+	    default:
+			error_msg_and_die("__jffs2_add_sum_mem(): UNKNOWN node type %d\n", je16_to_cpu(item->u.nodetype));
+	}
+	return 0;
+}
+
+void add_sum_inode_mem(union jffs2_node_union *node) {
+	
+	struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *) malloc(sizeof(struct jffs2_sum_inode_mem));
+	
+	if (!temp)
+		error_msg_and_die("Can't allocate memory for summary information!\n");
+	
+	temp->nodetype = node->i.nodetype;
+	temp->inode = node->i.ino;
+	temp->version = node->i.version;
+	temp->offset = cpu_to_je32(data_ofs); 
+	temp->totlen = node->i.totlen;
+	temp->next = NULL;
+
+	add_sum_mem((union jffs2_sum_mem *) temp);
+	
+}
+
+void add_sum_dirent_mem(union jffs2_node_union *node) {
+	
+	struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *) 
+			malloc(sizeof(struct jffs2_sum_dirent_mem) + node->d.nsize);
+	
+	if (!temp)
+		error_msg_and_die("Can't allocate memory for summary information!\n");
+	
+	temp->nodetype = node->d.nodetype;
+	temp->totlen = node->d.totlen;
+	temp->offset = cpu_to_je32(data_ofs);
+	temp->pino = node->d.pino;
+	temp->version = node->d.version;
+	temp->ino = node->d.ino;
+	temp->nsize = node->d.nsize;
+	temp->type = node->d.type;
+	temp->next = NULL;
+	
+	memcpy(temp->name,node->d.name,node->d.nsize);
+
+	add_sum_mem((union jffs2_sum_mem *) temp);
+	
+}
+
+void write_dirent_to_buff(union jffs2_node_union *node) {
+	
+	pad_block_if_less_than(je32_to_cpu (node->d.totlen),JFFS2_SUMMARY_DIRENT_SIZE(node->d.nsize));
+	add_sum_dirent_mem(node);
+	full_write(data_buffer + data_ofs, &(node->d), je32_to_cpu (node->d.totlen));
+	padword();	
+}
+
+
+void write_inode_to_buff(union jffs2_node_union *node) {
+	
+	pad_block_if_less_than(je32_to_cpu (node->i.totlen),JFFS2_SUMMARY_INODE_SIZE);  
+	add_sum_inode_mem(node);	/* Add inode summary mem to summary list */
+	full_write(data_buffer + data_ofs, &(node->i), je32_to_cpu (node->i.totlen));	/* Write out the inode to inode_buffer */
+	padword();
+	
+}
+
+
+void create_summed_image(int inp_size) {
+	uint8_t		*p = file_buffer;
+	union jffs2_node_union 	*node;
+	uint32_t	crc;
+	uint16_t	type;
+	int		bitchbitmask = 0;
+	int		obsolete;
+	
+	char	name[256];
+	
+	while ( p < (file_buffer + inp_size)) {
+		
+		node = (union jffs2_node_union *) p;
+		
+		/* Skip empty space */
+		if (je16_to_cpu (node->u.magic) == 0xFFFF && je16_to_cpu (node->u.nodetype) == 0xFFFF) {
+			p += 4;
+			continue;
+		}
+		
+		if (je16_to_cpu (node->u.magic) != JFFS2_MAGIC_BITMASK)	{
+			if (!bitchbitmask++)
+    			    printf ("Wrong bitmask  at  0x%08x, 0x%04x\n", p - file_buffer, je16_to_cpu (node->u.magic));
+			p += 4;
+			continue;
+		}
+		
+		bitchbitmask = 0;
+		
+		type = je16_to_cpu(node->u.nodetype);
+		if ((type & JFFS2_NODE_ACCURATE) != JFFS2_NODE_ACCURATE) {
+			obsolete = 1;
+			type |= JFFS2_NODE_ACCURATE;
+		} else
+			obsolete = 0;
+		
+		node->u.nodetype = cpu_to_je16(type);
+	    
+		crc = crc32 (0, node, sizeof (struct jffs2_unknown_node) - 4);
+		if (crc != je32_to_cpu (node->u.hdr_crc)) {
+			printf ("Wrong hdr_crc  at  0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->u.hdr_crc), crc);
+			p += 4;
+			continue;
+		}
+		
+		switch(je16_to_cpu(node->u.nodetype)) {
+		
+			case JFFS2_NODETYPE_INODE:
+				if(verbose)
+					printf ("%8s Inode      node at 0x%08x, totlen 0x%08x, #ino  %5d, version %5d, isize %8d, csize %8d, dsize %8d, offset %8d\n",
+						obsolete ? "Obsolete" : "",
+						p - file_buffer, je32_to_cpu (node->i.totlen), je32_to_cpu (node->i.ino),
+						je32_to_cpu ( node->i.version), je32_to_cpu (node->i.isize), 
+						je32_to_cpu (node->i.csize), je32_to_cpu (node->i.dsize), je32_to_cpu (node->i.offset));
+	
+				crc = crc32 (0, node, sizeof (struct jffs2_raw_inode) - 8);
+				if (crc != je32_to_cpu (node->i.node_crc)) {
+					printf ("Wrong node_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.node_crc), crc);
+					p += PAD(je32_to_cpu (node->i.totlen));
+					continue;
+				}
+				
+				crc = crc32(0, p + sizeof (struct jffs2_raw_inode), je32_to_cpu(node->i.csize));
+				if (crc != je32_to_cpu(node->i.data_crc)) {
+					printf ("Wrong data_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.data_crc), crc);
+					p += PAD(je32_to_cpu (node->i.totlen));
+					continue;
+				}
+				
+				write_inode_to_buff(node);
+				
+				p += PAD(je32_to_cpu (node->i.totlen));
+				break;
+				
+			case JFFS2_NODETYPE_DIRENT:
+				memcpy (name, node->d.name, node->d.nsize);
+				name [node->d.nsize] = 0x0;
+			
+				if(verbose)
+					printf ("%8s Dirent     node at 0x%08x, totlen 0x%08x, #pino %5d, version %5d, #ino  %8d, nsize %8d, name %s\n",
+						obsolete ? "Obsolete" : "",
+						p - file_buffer, je32_to_cpu (node->d.totlen), je32_to_cpu (node->d.pino),
+						je32_to_cpu ( node->d.version), je32_to_cpu (node->d.ino), 
+						node->d.nsize, name);
+	
+				crc = crc32 (0, node, sizeof (struct jffs2_raw_dirent) - 8);
+				if (crc != je32_to_cpu (node->d.node_crc)) {
+					printf ("Wrong node_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.node_crc), crc);
+					p += PAD(je32_to_cpu (node->d.totlen));
+					continue;
+				}
+				
+				crc = crc32(0, p + sizeof (struct jffs2_raw_dirent), node->d.nsize);
+				if (crc != je32_to_cpu(node->d.name_crc)) {
+					printf ("Wrong name_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.name_crc), crc);
+					p += PAD(je32_to_cpu (node->d.totlen));
+					continue;
+				}
+	
+				write_dirent_to_buff(node);
+				
+				p += PAD(je32_to_cpu (node->d.totlen));						
+				break;
+		
+			case JFFS2_NODETYPE_CLEANMARKER:
+				if (verbose) {
+					printf ("%8s Cleanmarker     at 0x%08x, totlen 0x%08x\n", 
+						obsolete ? "Obsolete" : "",
+						p - file_buffer, je32_to_cpu (node->u.totlen));
+				}
+				
+				if(!found_cleanmarkers){
+					found_cleanmarkers = 1;
+					
+					if(add_cleanmarkers == 1 && use_input_cleanmarker_size == 1){
+						cleanmarker_size = je32_to_cpu (node->u.totlen);
+						setup_cleanmarker();
+					}
+				}			
+				
+				p += PAD(je32_to_cpu (node->u.totlen));						
+				break;
+		
+			case JFFS2_NODETYPE_PADDING:
+				if (verbose) {
+					printf ("%8s Padding    node at 0x%08x, totlen 0x%08x\n", 
+						obsolete ? "Obsolete" : "",
+						p - file_buffer, je32_to_cpu (node->u.totlen));
+				}		
+				p += PAD(je32_to_cpu (node->u.totlen));						
+				break;
+				
+			case 0xffff:
+				p += 4;
+				break;
+				
+			default:	
+				if (verbose) {
+					printf ("%8s Unknown    node at 0x%08x, totlen 0x%08x\n", 
+						obsolete ? "Obsolete" : "",
+						p - file_buffer, je32_to_cpu (node->u.totlen));
+				}
+				
+				//write_unknown_to_buff(node);
+				
+				p += PAD(je32_to_cpu (node->u.totlen));						
+		}	
+	}
+}
+
+int main(int argc, char **argv) {
+	
+	int ret;
+	
+	process_options(argc,argv);
+	
+	if ((in_fd == -1) || (out_fd == -1))	{
+		
+		if(in_fd != -1)
+			close(in_fd);
+		if(out_fd != -1)
+			close(out_fd);
+		
+		error_msg_and_die("You must specify input and output files!\n");
+	}
+	
+	init_buffers();
+	init_sumlist();
+	
+	while ((ret = load_next_block())) {
+		create_summed_image(ret);	
+	}
+
+	flush_buffers();
+	clean_buffers();
+	clean_sumlist();
+	
+	if (in_fd != -1)
+		close(in_fd);
+	if (out_fd != -1)
+		close(out_fd);
+	
+	return 0;
+}

Re: JFFS2 mount time

[Gareth Bult (Encryptec)]

Hi,

I'm afraid this patch seems to be quite a long way from applying cleanly
to my source. I'm using 2.6.9 .. at least half a dozen failures or
more ...

:(

Will look into it in more detail later ...

Gareth.


On Thu, 2004-12-16 at 14:43 +0100, Ferenc Havasi wrote:
> Hi Gareth,
> 
> > However, I'm looking at 30secs to mount the root filesystem.
> 
> Did you try our patch? I send it now again, we made it fresh for the 
> current CVS. See its usage in the archive (16th November).
> 
> It will part of JFFS3.
> 
> This is designed for NAND devices - I don't know too much about your 
> environment, but it may help.
> 
> Bye,
> Ferenc
> plain text document attachment (jffs2-summary-20041207.patch)
> diff --unified --new-file --recursive mtd/fs/Kconfig mtd-dec07/fs/Kconfig
> --- mtd/fs/Kconfig	2004-11-04 00:00:11.000000000 +0100
> +++ mtd-dec07/fs/Kconfig	2004-12-07 16:03:02.000000000 +0100
> @@ -77,6 +77,19 @@
>  		 ECC for JFFS2.  This type of flash chip is not common, however it is
>  		 available from STMicro.
>  
> +config JFFS2_SUMMARY
> +        bool "JFFS2 summary support (EXPERIMENTAL)" 
> +        depends on JFFS2_FS
> +        default n
> +        help 
> +          This feature makes it possible to use summary information
> +          for faster filesystem mount - specially on NAND.
> +
> +          The summary information can be inserted into a filesystem image
> +          by the utility 'sumtool'.
> +
> +          If unsure, say 'N'.
> +
>  config JFFS2_COMPRESSION_OPTIONS
>  	bool "Advanced compression options for JFFS2"
>  	default n
> diff --unified --new-file --recursive mtd/fs/jffs2/Makefile.common mtd-dec07/fs/jffs2/Makefile.common
> --- mtd/fs/jffs2/Makefile.common	2004-11-04 00:00:11.000000000 +0100
> +++ mtd-dec07/fs/jffs2/Makefile.common	2004-12-07 16:03:02.000000000 +0100
> @@ -16,3 +16,4 @@
>  jffs2-$(CONFIG_JFFS2_RUBIN)	+= compr_rubin.o
>  jffs2-$(CONFIG_JFFS2_RTIME)	+= compr_rtime.o
>  jffs2-$(CONFIG_JFFS2_ZLIB)	+= compr_zlib.o
> +jffs2-$(CONFIG_JFFS2_SUMMARY)   += summary.o
> diff --unified --new-file --recursive mtd/fs/jffs2/build.c mtd-dec07/fs/jffs2/build.c
> --- mtd/fs/jffs2/build.c	2004-11-28 00:00:13.000000000 +0100
> +++ mtd-dec07/fs/jffs2/build.c	2004-12-07 16:08:31.000000000 +0100
> @@ -331,6 +331,10 @@
>  		c->blocks[i].first_node = NULL;
>  		c->blocks[i].last_node = NULL;
>  		c->blocks[i].bad_count = 0;
> + #ifdef CONFIG_JFFS2_SUMMARY	
> + 		c->blocks[i].sum_collected = NULL;
> + #endif
> +
>  	}
>  
>  	init_MUTEX(&c->alloc_sem);
> diff --unified --new-file --recursive mtd/fs/jffs2/dir.c mtd-dec07/fs/jffs2/dir.c
> --- mtd/fs/jffs2/dir.c	2004-11-17 00:00:14.000000000 +0100
> +++ mtd-dec07/fs/jffs2/dir.c	2004-12-07 16:03:02.000000000 +0100
> @@ -22,6 +22,7 @@
>  #include <linux/time.h>
>  #include "nodelist.h"
>  
> +
>  /* Urgh. Please tell me there's a nicer way of doing these. */
>  #include <linux/version.h>
>  #if LINUX_VERSION_CODE < KERNEL_VERSION(2,5,48)
> @@ -314,7 +315,7 @@
>  	 * Just the node will do for now, though 
>  	 */
>  	namelen = dentry->d_name.len;
> -	ret = jffs2_reserve_space(c, sizeof(*ri) + strlen(target), &phys_ofs, &alloclen, ALLOC_NORMAL);
> +	ret = jffs2_reserve_space(c, sizeof(*ri) + strlen(target), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
>  
>  	if (ret) {
>  		jffs2_free_raw_inode(ri);
> @@ -360,7 +361,7 @@
>  	up(&f->sem);
>  
>  	jffs2_complete_reservation(c);
> -	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
> +	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
>  	if (ret) {
>  		/* Eep. */
>  		jffs2_clear_inode(inode);
> @@ -445,7 +446,7 @@
>  	 * Just the node will do for now, though 
>  	 */
>  	namelen = dentry->d_name.len;
> -	ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL);
> +	ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
>  
>  	if (ret) {
>  		jffs2_free_raw_inode(ri);
> @@ -488,7 +489,7 @@
>  	up(&f->sem);
>  
>  	jffs2_complete_reservation(c);
> -	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
> +	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
>  	if (ret) {
>  		/* Eep. */
>  		jffs2_clear_inode(inode);
> @@ -597,7 +598,7 @@
>  	 * Just the node will do for now, though 
>  	 */
>  	namelen = dentry->d_name.len;
> -	ret = jffs2_reserve_space(c, sizeof(*ri) + devlen, &phys_ofs, &alloclen, ALLOC_NORMAL);
> +	ret = jffs2_reserve_space(c, sizeof(*ri) + devlen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
>  
>  	if (ret) {
>  		jffs2_free_raw_inode(ri);
> @@ -642,7 +643,7 @@
>  	up(&f->sem);
>  
>  	jffs2_complete_reservation(c);
> -	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
> +	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
>  	if (ret) {
>  		/* Eep. */
>  		jffs2_clear_inode(inode);
> @@ -796,4 +797,3 @@
>  
>  	return 0;
>  }
> -
> diff --unified --new-file --recursive mtd/fs/jffs2/file.c mtd-dec07/fs/jffs2/file.c
> --- mtd/fs/jffs2/file.c	2004-11-17 00:00:14.000000000 +0100
> +++ mtd-dec07/fs/jffs2/file.c	2004-12-07 16:03:02.000000000 +0100
> @@ -22,6 +22,7 @@
>  #include <linux/jffs2.h>
>  #include "nodelist.h"
>  
> +
>  extern int generic_file_open(struct inode *, struct file *) __attribute__((weak));
>  extern loff_t generic_file_llseek(struct file *file, loff_t offset, int origin) __attribute__((weak));
>  
> @@ -135,7 +136,7 @@
>  		D1(printk(KERN_DEBUG "Writing new hole frag 0x%x-0x%x between current EOF and new page\n",
>  			  (unsigned int)inode->i_size, pageofs));
>  
> -		ret = jffs2_reserve_space(c, sizeof(ri), &phys_ofs, &alloc_len, ALLOC_NORMAL);
> +		ret = jffs2_reserve_space(c, sizeof(ri), &phys_ofs, &alloc_len, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
>  		if (ret)
>  			return ret;
>  
> diff --unified --new-file --recursive mtd/fs/jffs2/fs.c mtd-dec07/fs/jffs2/fs.c
> --- mtd/fs/jffs2/fs.c	2004-11-29 00:00:09.000000000 +0100
> +++ mtd-dec07/fs/jffs2/fs.c	2004-12-07 16:03:02.000000000 +0100
> @@ -74,7 +74,7 @@
>  		return -ENOMEM;
>  	}
>  		
> -	ret = jffs2_reserve_space(c, sizeof(*ri) + mdatalen, &phys_ofs, &alloclen, ALLOC_NORMAL);
> +	ret = jffs2_reserve_space(c, sizeof(*ri) + mdatalen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
>  	if (ret) {
>  		jffs2_free_raw_inode(ri);
>  		if (S_ISLNK(inode->i_mode & S_IFMT))
> diff --unified --new-file --recursive mtd/fs/jffs2/gc.c mtd-dec07/fs/jffs2/gc.c
> --- mtd/fs/jffs2/gc.c	2004-11-17 00:00:14.000000000 +0100
> +++ mtd-dec07/fs/jffs2/gc.c	2004-12-07 16:03:02.000000000 +0100
> @@ -506,7 +506,7 @@
>  	   don't want to force wastage of the end of a block if splitting would
>  	   work. */
>  	ret = jffs2_reserve_space_gc(c, min_t(uint32_t, sizeof(struct jffs2_raw_inode) + JFFS2_MIN_DATA_LEN, 
> -					      rawlen), &phys_ofs, &alloclen);
> +					      rawlen), &phys_ofs, &alloclen, rawlen); /* this is not optimal yet */
>  	if (ret)
>  		return ret;
>  
> @@ -614,7 +614,7 @@
>  			ACCT_SANITY_CHECK(c,jeb);
>  			D1(ACCT_PARANOIA_CHECK(jeb));
>  
> -			ret = jffs2_reserve_space_gc(c, rawlen, &phys_ofs, &dummy);
> +			ret = jffs2_reserve_space_gc(c, rawlen, &phys_ofs, &dummy, rawlen); /* this is not optimal yet */
>  
>  			if (!ret) {
>  				D1(printk(KERN_DEBUG "Allocated space at 0x%08x to retry failed write.\n", phys_ofs));
> @@ -692,7 +692,7 @@
>  
>  	}
>  	
> -	ret = jffs2_reserve_space_gc(c, sizeof(ri) + mdatalen, &phys_ofs, &alloclen);
> +	ret = jffs2_reserve_space_gc(c, sizeof(ri) + mdatalen, &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE);
>  	if (ret) {
>  		printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_metadata failed: %d\n",
>  		       sizeof(ri)+ mdatalen, ret);
> @@ -759,7 +759,7 @@
>  	rd.node_crc = cpu_to_je32(crc32(0, &rd, sizeof(rd)-8));
>  	rd.name_crc = cpu_to_je32(crc32(0, fd->name, rd.nsize));
>  	
> -	ret = jffs2_reserve_space_gc(c, sizeof(rd)+rd.nsize, &phys_ofs, &alloclen);
> +	ret = jffs2_reserve_space_gc(c, sizeof(rd)+rd.nsize, &phys_ofs, &alloclen, JFFS2_SUMMARY_DIRENT_SIZE(rd.nsize));
>  	if (ret) {
>  		printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_dirent failed: %d\n",
>  		       sizeof(rd)+rd.nsize, ret);
> @@ -961,7 +961,7 @@
>  	ri.data_crc = cpu_to_je32(0);
>  	ri.node_crc = cpu_to_je32(crc32(0, &ri, sizeof(ri)-8));
>  
> -	ret = jffs2_reserve_space_gc(c, sizeof(ri), &phys_ofs, &alloclen);
> +	ret = jffs2_reserve_space_gc(c, sizeof(ri), &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE);
>  	if (ret) {
>  		printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_hole failed: %d\n",
>  		       sizeof(ri), ret);
> @@ -1186,7 +1186,7 @@
>  		uint32_t cdatalen;
>  		uint16_t comprtype = JFFS2_COMPR_NONE;
>  
> -		ret = jffs2_reserve_space_gc(c, sizeof(ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen);
> +		ret = jffs2_reserve_space_gc(c, sizeof(ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, JFFS2_SUMMARY_INODE_SIZE);
>  
>  		if (ret) {
>  			printk(KERN_WARNING "jffs2_reserve_space_gc of %zd bytes for garbage_collect_dnode failed: %d\n",
> @@ -1243,4 +1243,3 @@
>  	jffs2_gc_release_page(c, pg_ptr, &pg);
>  	return ret;
>  }
> -
> diff --unified --new-file --recursive mtd/fs/jffs2/nodelist.h mtd-dec07/fs/jffs2/nodelist.h
> --- mtd/fs/jffs2/nodelist.h	2004-11-26 15:08:31.000000000 +0100
> +++ mtd-dec07/fs/jffs2/nodelist.h	2004-12-07 16:03:02.000000000 +0100
> @@ -20,6 +20,7 @@
>  #include <linux/jffs2.h>
>  #include <linux/jffs2_fs_sb.h>
>  #include <linux/jffs2_fs_i.h>
> +#include "summary.h"
>  
>  #ifdef __ECOS
>  #include "os-ecos.h"
> @@ -194,6 +195,10 @@
>  	int bad_count;
>  	uint32_t offset;		/* of this block in the MTD */
>  
> +#ifdef CONFIG_JFFS2_SUMMARY	
> +	struct jffs2_sum_info *sum_collected;
> +#endif
> +	
>  	uint32_t unchecked_size;
>  	uint32_t used_size;
>  	uint32_t dirty_size;
> @@ -392,8 +397,8 @@
>  
>  /* nodemgmt.c */
>  int jffs2_thread_should_wake(struct jffs2_sb_info *c);
> -int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio);
> -int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len);
> +int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio, uint32_t sumsize);
> +int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize);
>  int jffs2_add_physical_node_ref(struct jffs2_sb_info *c, struct jffs2_raw_node_ref *new);
>  void jffs2_complete_reservation(struct jffs2_sb_info *c);
>  void jffs2_mark_node_obsolete(struct jffs2_sb_info *c, struct jffs2_raw_node_ref *raw);
> @@ -455,6 +460,10 @@
>  /* scan.c */
>  int jffs2_scan_medium(struct jffs2_sb_info *c);
>  void jffs2_rotate_lists(struct jffs2_sb_info *c);
> +int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf,
> +				uint32_t ofs, uint32_t len);
> +struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino);
> +
>  
>  /* build.c */
>  int jffs2_do_mount_fs(struct jffs2_sb_info *c);
> diff --unified --new-file --recursive mtd/fs/jffs2/nodemgmt.c mtd-dec07/fs/jffs2/nodemgmt.c
> --- mtd/fs/jffs2/nodemgmt.c	2004-11-26 15:08:31.000000000 +0100
> +++ mtd-dec07/fs/jffs2/nodemgmt.c	2004-12-07 16:03:02.000000000 +0100
> @@ -38,9 +38,9 @@
>   *	for the requested allocation.
>   */
>  
> -static int jffs2_do_reserve_space(struct jffs2_sb_info *c,  uint32_t minsize, uint32_t *ofs, uint32_t *len);
> +static int jffs2_do_reserve_space(struct jffs2_sb_info *c,  uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize);
>  
> -int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio)
> +int jffs2_reserve_space(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, int prio, uint32_t sumsize)
>  {
>  	int ret = -EAGAIN;
>  	int blocksneeded = c->resv_blocks_write;
> @@ -129,7 +129,7 @@
>  			spin_lock(&c->erase_completion_lock);
>  		}
>  
> -		ret = jffs2_do_reserve_space(c, minsize, ofs, len);
> +		ret = jffs2_do_reserve_space(c, minsize, ofs, len, sumsize);
>  		if (ret) {
>  			D1(printk(KERN_DEBUG "jffs2_reserve_space: ret is %d\n", ret));
>  		}
> @@ -140,7 +140,7 @@
>  	return ret;
>  }
>  
> -int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len)
> +int jffs2_reserve_space_gc(struct jffs2_sb_info *c, uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize)
>  {
>  	int ret = -EAGAIN;
>  	minsize = PAD(minsize);
> @@ -149,7 +149,7 @@
>  
>  	spin_lock(&c->erase_completion_lock);
>  	while(ret == -EAGAIN) {
> -		ret = jffs2_do_reserve_space(c, minsize, ofs, len);
> +		ret = jffs2_do_reserve_space(c, minsize, ofs, len, sumsize);
>  		if (ret) {
>  		        D1(printk(KERN_DEBUG "jffs2_reserve_space_gc: looping, ret is %d\n", ret));
>  		}
> @@ -159,50 +159,112 @@
>  }
>  
>  /* Called with alloc sem _and_ erase_completion_lock */
> -static int jffs2_do_reserve_space(struct jffs2_sb_info *c,  uint32_t minsize, uint32_t *ofs, uint32_t *len)
> +static int jffs2_do_reserve_space(struct jffs2_sb_info *c,  uint32_t minsize, uint32_t *ofs, uint32_t *len, uint32_t sumsize)
>  {
>  	struct jffs2_eraseblock *jeb = c->nextblock;
> +	uint32_t nofree_size;
>  	
>   restart:
> -	if (jeb && minsize > jeb->free_size) {
> -		/* Skip the end of this block and file it as having some dirty space */
> -		/* If there's a pending write to it, flush now */
> -		if (jffs2_wbuf_dirty(c)) {
> -			spin_unlock(&c->erase_completion_lock);
> -			D1(printk(KERN_DEBUG "jffs2_do_reserve_space: Flushing write buffer\n"));			    
> -			jffs2_flush_wbuf_pad(c);
> -			spin_lock(&c->erase_completion_lock);
> -			jeb = c->nextblock;
> -			goto restart;
> -		}
> -		c->wasted_size += jeb->free_size;
> -		c->free_size -= jeb->free_size;
> -		jeb->wasted_size += jeb->free_size;
> -		jeb->free_size = 0;
> +	nofree_size = 0;
> +	
> +#ifdef CONFIG_JFFS2_SUMMARY
> +
> +	if (sumsize != JFFS2_SUMMARY_NOSUM_SIZE) {
> +		int ret;
> +                if (jeb) {
> +                        if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret;
> +                        nofree_size = sumsize + jeb->sum_collected->sum_size + JFFS2_SUMMARY_FRAME_SIZE;
> +                }
>  		
> -		/* Check, if we have a dirty block now, or if it was dirty already */
> -		if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) {
> -			c->dirty_size += jeb->wasted_size;
> -			c->wasted_size -= jeb->wasted_size;
> -			jeb->dirty_size += jeb->wasted_size;
> -			jeb->wasted_size = 0;
> -			if (VERYDIRTY(c, jeb->dirty_size)) {
> -				D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
> +		D1(printk(KERN_DEBUG "JFFS2: minsize %d , jeb->free(%d) , sum_collected->size(%d) , sumsize(%d)\n",minsize,jeb->free_size,jeb->sum_collected->sum_size,sumsize));
> +		
> +		if (jeb && (minsize + jeb->sum_collected->sum_size + sumsize + JFFS2_SUMMARY_FRAME_SIZE > jeb->free_size)) {
> +			D1(printk(KERN_DEBUG "JFFS2: generating summary for 0x%08x.\n", jeb->offset));
> +			if (jeb->sum_collected->sum_size == JFFS2_SUMMARY_NOSUM_SIZE) {
> +				sumsize = JFFS2_SUMMARY_NOSUM_SIZE;
> +				jffs2_sum_clean_collected(jeb);
> +				goto restart;
> +			}
> +			
> +			ret = jffs2_sum_write_sumnode(c);
> +			
> +			if (ret)
> +				return ret;
> +			
> +			if (jeb->sum_collected->sum_size == JFFS2_SUMMARY_NOSUM_SIZE) { //jffs2_write_sumnode can't write out the summary information
> +				sumsize = JFFS2_SUMMARY_NOSUM_SIZE;
> +				jffs2_sum_clean_collected(jeb);
> +				goto restart;
> +			}
> +			
> +			/* Check, if we have a dirty block now, or if it was dirty already */
> +			if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) {
> +				c->dirty_size += jeb->wasted_size;
> +				c->wasted_size -= jeb->wasted_size;
> +				jeb->dirty_size += jeb->wasted_size;
> +				jeb->wasted_size = 0;
> +				if (VERYDIRTY(c, jeb->dirty_size)) {
> +					D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
> +					  jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
> +					list_add_tail(&jeb->list, &c->very_dirty_list);
> +				} else {
> +					D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
> +					  jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
> +					list_add_tail(&jeb->list, &c->dirty_list);
> +				}
> +			} else { 
> +				D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
>  				  jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
> -				list_add_tail(&jeb->list, &c->very_dirty_list);
> -			} else {
> -				D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
> +				list_add_tail(&jeb->list, &c->clean_list);
> +			}
> +			c->nextblock = jeb = NULL;
> +		}
> +	}
> +	else {	
> +#endif			
> +		if (jeb && minsize > jeb->free_size) {
> +			/* Skip the end of this block and file it as having some dirty space */
> +			/* If there's a pending write to it, flush now */
> +			
> +			if (jffs2_wbuf_dirty(c)) {
> +				spin_unlock(&c->erase_completion_lock);
> +				D1(printk(KERN_DEBUG "jffs2_do_reserve_space: Flushing write buffer\n"));			    
> +				jffs2_flush_wbuf_pad(c);
> +				spin_lock(&c->erase_completion_lock);
> +				jeb = c->nextblock;
> +				goto restart;
> +			}
> +			
> +			c->wasted_size += jeb->free_size;
> +			c->free_size -= jeb->free_size;
> +			jeb->wasted_size += jeb->free_size;
> +			jeb->free_size = 0;
> +			
> +			/* Check, if we have a dirty block now, or if it was dirty already */
> +			if (ISDIRTY (jeb->wasted_size + jeb->dirty_size)) {
> +				c->dirty_size += jeb->wasted_size;
> +				c->wasted_size -= jeb->wasted_size;
> +				jeb->dirty_size += jeb->wasted_size;
> +				jeb->wasted_size = 0;
> +				if (VERYDIRTY(c, jeb->dirty_size)) {
> +					D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to very_dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
> +					  jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
> +					list_add_tail(&jeb->list, &c->very_dirty_list);
> +				} else {
> +					D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to dirty_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
> +					  jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
> +					list_add_tail(&jeb->list, &c->dirty_list);
> +				}
> +			} else { 
> +				D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
>  				  jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
> -				list_add_tail(&jeb->list, &c->dirty_list);
> +				list_add_tail(&jeb->list, &c->clean_list);
>  			}
> -		} else { 
> -			D1(printk(KERN_DEBUG "Adding full erase block at 0x%08x to clean_list (free 0x%08x, dirty 0x%08x, used 0x%08x\n",
> -			  jeb->offset, jeb->free_size, jeb->dirty_size, jeb->used_size));
> -			list_add_tail(&jeb->list, &c->clean_list);
> +			c->nextblock = jeb = NULL;
>  		}
> -		c->nextblock = jeb = NULL;
> +#ifdef CONFIG_JFFS2_SUMMARY
>  	}
> -	
> +#endif	
>  	if (!jeb) {
>  		struct list_head *next;
>  		/* Take the next block off the 'free' list */
> @@ -266,7 +328,7 @@
>  	/* OK, jeb (==c->nextblock) is now pointing at a block which definitely has
>  	   enough space */
>  	*ofs = jeb->offset + (c->sector_size - jeb->free_size);
> -	*len = jeb->free_size;
> +	*len = jeb->free_size - nofree_size;
>  
>  	if (c->cleanmarker_size && jeb->used_size == c->cleanmarker_size &&
>  	    !jeb->first_node->next_in_ino) {
> diff --unified --new-file --recursive mtd/fs/jffs2/os-linux.h mtd-dec07/fs/jffs2/os-linux.h
> --- mtd/fs/jffs2/os-linux.h	2004-11-17 00:00:14.000000000 +0100
> +++ mtd-dec07/fs/jffs2/os-linux.h	2004-12-07 16:03:02.000000000 +0100
> @@ -100,7 +100,13 @@
>  #define jffs2_is_readonly(c) (OFNI_BS_2SFFJ(c)->s_flags & MS_RDONLY)
>  
>  #if (!defined CONFIG_JFFS2_FS_NAND && !defined CONFIG_JFFS2_FS_NOR_ECC)
> +
> +#ifndef CONFIG_JFFS2_SUMMARY
>  #define jffs2_can_mark_obsolete(c) (1)
> +#else
> +#define jffs2_can_mark_obsolete(c) (0)
> +#endif
> +
>  #define jffs2_cleanmarker_oob(c) (0)
>  #define jffs2_write_nand_cleanmarker(c,jeb) (-EIO)
>  
> @@ -121,7 +127,12 @@
>  
>  #else /* NAND and/or ECC'd NOR support present */
>  
> +#ifndef CONFIG_JFFS2_SUMMARY
>  #define jffs2_can_mark_obsolete(c) ((c->mtd->type == MTD_NORFLASH && !(c->mtd->flags & MTD_ECC)) || c->mtd->type == MTD_RAM)
> +#else
> +#define jffs2_can_mark_obsolete(c) (0)
> +#endif
> +
>  #define jffs2_cleanmarker_oob(c) (c->mtd->type == MTD_NANDFLASH)
>  
>  #define jffs2_flash_write_oob(c, ofs, len, retlen, buf) ((c)->mtd->write_oob((c)->mtd, ofs, len, retlen, buf))
> @@ -218,5 +229,3 @@
>  
> 
>  #endif /* __JFFS2_OS_LINUX_H__ */
> -
> -
> diff --unified --new-file --recursive mtd/fs/jffs2/scan.c mtd-dec07/fs/jffs2/scan.c
> --- mtd/fs/jffs2/scan.c	2004-11-26 15:08:31.000000000 +0100
> +++ mtd-dec07/fs/jffs2/scan.c	2004-12-07 16:03:02.000000000 +0100
> @@ -18,22 +18,10 @@
>  #include <linux/crc32.h>
>  #include <linux/compiler.h>
>  #include "nodelist.h"
> +#include "summary.h"
>  
>  #define EMPTY_SCAN_SIZE 1024
>  
> -#define DIRTY_SPACE(x) do { typeof(x) _x = (x); \
> -		c->free_size -= _x; c->dirty_size += _x; \
> -		jeb->free_size -= _x ; jeb->dirty_size += _x; \
> -		}while(0)
> -#define USED_SPACE(x) do { typeof(x) _x = (x); \
> -		c->free_size -= _x; c->used_size += _x; \
> -		jeb->free_size -= _x ; jeb->used_size += _x; \
> -		}while(0)
> -#define UNCHECKED_SPACE(x) do { typeof(x) _x = (x); \
> -		c->free_size -= _x; c->unchecked_size += _x; \
> -		jeb->free_size -= _x ; jeb->unchecked_size += _x; \
> -		}while(0)
> -
>  #define noisy_printk(noise, args...) do { \
>  	if (*(noise)) { \
>  		printk(KERN_NOTICE args); \
> @@ -58,13 +46,6 @@
>  static int jffs2_scan_dirent_node(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb,
>  				 struct jffs2_raw_dirent *rd, uint32_t ofs);
>  
> -#define BLK_STATE_ALLFF		0
> -#define BLK_STATE_CLEAN		1
> -#define BLK_STATE_PARTDIRTY	2
> -#define BLK_STATE_CLEANMARKER	3
> -#define BLK_STATE_ALLDIRTY	4
> -#define BLK_STATE_BADBLOCK	5
> -
>  static inline int min_free(struct jffs2_sb_info *c)
>  {
>  	uint32_t min = 2 * sizeof(struct jffs2_raw_inode);
> @@ -257,7 +238,7 @@
>  	return ret;
>  }
>  
> -static int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf,
> +int jffs2_fill_scan_buf (struct jffs2_sb_info *c, unsigned char *buf,
>  				uint32_t ofs, uint32_t len)
>  {
>  	int ret;
> @@ -286,6 +267,11 @@
>  	uint32_t hdr_crc, buf_ofs, buf_len;
>  	int err;
>  	int noise = 0;
> +	
> +#ifdef CONFIG_JFFS2_SUMMARY
> +	struct jffs2_sum_marker *sm;
> +#endif	
> +	
>  #ifdef CONFIG_JFFS2_FS_NAND
>  	int cleanmarkerfound = 0;
>  #endif
> @@ -311,10 +297,54 @@
>  		}
>  	}
>  #endif
> +	
> +#ifdef CONFIG_JFFS2_SUMMARY	
> +	sm = (struct jffs2_sum_marker *)kmalloc(sizeof(struct jffs2_sum_marker), GFP_KERNEL);
> +	if (!sm) {
> +	    return -ENOMEM;
> +	}
> +	
> +	err = jffs2_fill_scan_buf(c, (unsigned char *) sm, jeb->offset + c->sector_size - sizeof(struct jffs2_sum_marker), sizeof(struct jffs2_sum_marker));
> +	
> +	if (err) {
> +		kfree(sm);
> +	        return err;
> +	}
> +	
> +	if (je32_to_cpu(sm->magic) == JFFS2_SUM_MAGIC ) {
> +
> +		if(je32_to_cpu(sm->erase_size) == c->sector_size) {
> +			int ret = jffs2_sum_scan_sumnode(c,jeb,je32_to_cpu(sm->offset),&pseudo_random);
> +			
> +			if (ret) {
> +				kfree(sm);
> +				return ret;
> +			}
> +		}
> +		
> +		printk(KERN_WARNING "FS erase_block_size != JFFS2 erase_block_size => skipping summary information\n");
> +		
> +	}
> +	
> +	kfree(sm);
> +	
> +	ofs = jeb->offset;
> +	prevofs = jeb->offset - 1;
> +	
> +#endif
> +	
>  	buf_ofs = jeb->offset;
>  
>  	if (!buf_size) {
>  		buf_len = c->sector_size;
> +		
> +#ifdef CONFIG_JFFS2_SUMMARY
> +		/* must reread because of summary test */
> +		err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len);
> +		if (err)
> +			return err;
> +#endif
> +
>  	} else {
>  		buf_len = EMPTY_SCAN_SIZE;
>  		err = jffs2_fill_scan_buf(c, buf, buf_ofs, buf_len);
> @@ -356,6 +386,8 @@
>  
>  	noise = 10;
>  
> +	D1(printk(KERN_DEBUG "JFFS2: no summary found in jeb 0x%08x. Apply original scan.\n",jeb->offset));
> +
>  scan_more:	
>  	while(ofs < jeb->offset + c->sector_size) {
>  
> @@ -571,6 +603,9 @@
>  			break;
>  
>  		case JFFS2_NODETYPE_PADDING:
> +#ifdef CONFIG_JFFS2_SUMMARY
> +	                jffs2_sum_add_padding_mem(jeb,je32_to_cpu(node->totlen));
> +#endif
>  			DIRTY_SPACE(PAD(je32_to_cpu(node->totlen)));
>  			ofs += PAD(je32_to_cpu(node->totlen));
>  			break;
> @@ -634,7 +669,7 @@
>  		return BLK_STATE_ALLDIRTY;
>  }
>  
> -static struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino)
> +struct jffs2_inode_cache *jffs2_scan_make_ino_cache(struct jffs2_sb_info *c, uint32_t ino)
>  {
>  	struct jffs2_inode_cache *ic;
>  
> @@ -728,6 +763,11 @@
>  	pseudo_random += je32_to_cpu(ri->version);
>  
>  	UNCHECKED_SPACE(PAD(je32_to_cpu(ri->totlen)));
> +	
> +#ifdef CONFIG_JFFS2_SUMMARY
> +	jffs2_sum_add_inode_mem(jeb,ri,ofs);
> +#endif
> +	
>  	return 0;
>  }
>  
> @@ -806,6 +846,10 @@
>  	USED_SPACE(PAD(je32_to_cpu(rd->totlen)));
>  	jffs2_add_fd_to_list(c, fd, &ic->scan_dents);
>  
> +#ifdef CONFIG_JFFS2_SUMMARY
> +	jffs2_sum_add_dirent_mem(jeb,rd,ofs);
> +#endif
> +	
>  	return 0;
>  }
>  
> diff --unified --new-file --recursive mtd/fs/jffs2/summary.c mtd-dec07/fs/jffs2/summary.c
> --- mtd/fs/jffs2/summary.c	1970-01-01 01:00:00.000000000 +0100
> +++ mtd-dec07/fs/jffs2/summary.c	2004-12-07 16:03:02.000000000 +0100
> @@ -0,0 +1,720 @@
> +/*
> + * JFFS2 -- Journalling Flash File System, Version 2.
> + *
> + * Copyright (C) 2004  Ferenc Havasi <havasi@inf.u-szeged.hu>,
> + *                     Zoltan Sogor <weth@inf.u-szeged.hu>,
> + *                     Patrik Kluba <pajko@halom.u-szeged.hu>,
> + *                     University of Szeged, Hungary
> + *
> + * For licensing information, see the file 'LICENCE' in this directory.
> + *
> + * $Id$
> + *
> + */
> +
> +#include <linux/kernel.h>
> +#include <linux/sched.h>
> +#include <linux/slab.h>
> +#include <linux/mtd/mtd.h>
> +#include <linux/pagemap.h>
> +#include <linux/crc32.h>
> +#include <linux/compiler.h>
> +#include <linux/vmalloc.h>
> +#include "nodelist.h"
> +
> +int jffs2_sum_init(struct jffs2_sb_info *c) 
> +{
> +        c->summary_buf = (jint32_t *) vmalloc(c->sector_size);
> +        if (!c->summary_buf) {
> +		printk(KERN_WARNING "JFFS2: can't allocate memory to dump summary information!\n");
> +                return 1;
> +        }
> +        return 0;
> +}
> +
> +void jffs2_sum_exit(struct jffs2_sb_info *c) 
> +{
> +        if (c->summary_buf) {
> +                vfree(c->summary_buf);
> +                c->summary_buf = NULL;
> +        }
> +}
> +
> +int jffs2_sum_care_sum_collected(struct jffs2_eraseblock *jeb)
> +{
> +	if (!jeb->sum_collected) {
> +		jeb->sum_collected = (struct jffs2_sum_info *) kmalloc(sizeof(struct jffs2_sum_info), GFP_KERNEL);
> +	
> +		if (!jeb->sum_collected)
> +			return -ENOMEM;
> +			
> +		jeb->sum_collected->sum_list = NULL;
> +		jeb->sum_collected->sum_num = 0;
> +		jeb->sum_collected->sum_size = 0; 
> +		jeb->sum_collected->sum_padded = 0; 
> +	}
> +        return 0;
> +}
> +
> +static int jffs2_sum_add_mem(struct jffs2_eraseblock *jeb, union jffs2_sum_mem *item) 
> +{
> +	
> +	union jffs2_sum_mem *walk;
> +        int ret;
> +	
> +        if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret;
> +	
> +	if (!jeb->sum_collected->sum_list) {
> +		jeb->sum_collected->sum_list = (union jffs2_sum_mem *) item;
> +	} 
> +	else {
> +		walk = jeb->sum_collected->sum_list;
> +		
> +		while (walk->u.next) {
> +			walk = walk->u.next;
> +		}
> +		walk->u.next = (union jffs2_sum_mem *) item;
> +	}
> +	switch (je16_to_cpu(item->u.nodetype)) {
> +    	    case JFFS2_NODETYPE_INODE:
> +		jeb->sum_collected->sum_size += JFFS2_SUMMARY_INODE_SIZE;
> +		jeb->sum_collected->sum_num++;
> +		break;
> +    	    case JFFS2_NODETYPE_DIRENT:
> +		jeb->sum_collected->sum_size += JFFS2_SUMMARY_DIRENT_SIZE(item->d.nsize);
> +		jeb->sum_collected->sum_num++;
> +		break;
> +	    default:
> +		printk(KERN_WARNING "__jffs2_add_sum_mem(): UNKNOWN node type %d\n", je16_to_cpu(item->u.nodetype));
> +		return 1;
> +	}
> +	return 0;
> +}
> +
> +void jffs2_sum_clean_all_info(struct jffs2_sb_info *c)
> +{
> +	int i;
> +	
> +	for (i=0; i<c->nr_blocks; i++) {
> +		struct jffs2_eraseblock *jeb = &c->blocks[i];
> +		
> +		jffs2_sum_clean_collected(jeb);
> +		kfree(jeb->sum_collected);
> +		jeb->sum_collected = NULL;
> +	}
> +}
> +	
> +/* These 3 functions are called from scan.c to collect summary info for not closed jeb */
> +
> +int jffs2_sum_add_padding_mem(struct jffs2_eraseblock *jeb, uint32_t size)
> +{
> +        int ret;
> +	
> +        if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret;
> +        jeb->sum_collected->sum_padded += size;
> +        return 0;
> +}
> +
> +int jffs2_sum_add_inode_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_inode *ri, uint32_t ofs) 
> +{
> +	
> +	struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *) kmalloc(sizeof(struct jffs2_sum_inode_mem), GFP_KERNEL);
> +	
> +	if (!temp)
> +		return -ENOMEM;
> +
> +        ofs -= jeb->offset;
> +	
> +	temp->nodetype = ri->nodetype;
> +	temp->inode = ri->ino;
> +	temp->version = ri->version;
> +	temp->offset = cpu_to_je32(ofs); 
> +	temp->totlen = ri->totlen;
> +	temp->next = NULL;
> +	
> +	return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp);
> +}
> +
> +int jffs2_sum_add_dirent_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_dirent *rd, uint32_t ofs) 
> +{
> +	
> +	struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *) 
> +			kmalloc(sizeof(struct jffs2_sum_dirent_mem) + rd->nsize, GFP_KERNEL);
> +	
> +	if (!temp)
> +		return -ENOMEM;
> +	
> +        ofs -= jeb->offset;
> +
> +	temp->nodetype = rd->nodetype;
> +	temp->totlen = rd->totlen;
> +	temp->offset = cpu_to_je32(ofs);
> +	temp->pino = rd->pino;
> +	temp->version = rd->version;
> +	temp->ino = rd->ino;
> +	temp->nsize = rd->nsize;
> +	temp->type = rd->type;
> +	temp->next = NULL;
> +	
> +	memcpy(temp->name, rd->name, rd->nsize);
> +
> +	return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp);
> +}
> +
> +/* Cleanup every collected summary information */
> +
> +void jffs2_sum_clean_collected(struct jffs2_eraseblock *jeb) 
> +{
> +	
> +	union jffs2_sum_mem *temp;
> +	
> +	if(jeb && jeb->sum_collected){
> +		
> +		while(jeb->sum_collected->sum_list){
> +			temp = jeb->sum_collected->sum_list;
> +			jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next;
> +			kfree(temp);
> +			jeb->sum_collected->sum_num--;
> +		}
> +		
> +		if(jeb->sum_collected->sum_num != 0){
> +			printk(KERN_WARNING "Ooops, something wrong happened! sum_num != 0, but sum_list = null ???");
> +			jeb->sum_collected->sum_num = 0;
> +		}
> +	}	
> +}
> +
> +/* Called from wbuf.c to collect writed node info */
> +
> +int jffs2_sum_add_kvec(struct jffs2_sb_info *c, const struct kvec *invecs, unsigned long count, uint32_t ofs)
> +{
> +	union jffs2_node_union *node;
> +	struct jffs2_eraseblock *jeb;	
> +        int ret;
> +	
> +	node = (union jffs2_node_union *) invecs[0].iov_base;
> +	jeb = &c->blocks[ofs / c->sector_size];
> +        ofs -= jeb->offset;
> +       
> +        if ((ret=jffs2_sum_care_sum_collected(jeb))) return ret;        
> +	
> +	switch(je16_to_cpu(node->u.nodetype)){
> +		case JFFS2_NODETYPE_INODE : {
> +			struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *) 
> +				kmalloc(sizeof(struct jffs2_sum_inode_mem), GFP_KERNEL);
> +			
> +			if (!temp)
> +				return -ENOMEM;
> +
> +			temp->nodetype = node->i.nodetype;
> +			temp->inode = node->i.ino;
> +			temp->version = node->i.version;
> +			temp->offset = cpu_to_je32(ofs); 
> +			temp->totlen = node->i.totlen;
> +			temp->next = NULL;
> +						
> +			return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp);
> +			
> +			break;
> +		}
> +		
> +		case JFFS2_NODETYPE_DIRENT : {
> +			struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *) 
> +				kmalloc(sizeof(struct jffs2_sum_dirent_mem) + node->d.nsize, GFP_KERNEL);
> +			
> +			if (!temp)
> +				return -ENOMEM;
> +			
> +			temp->nodetype = node->d.nodetype;
> +			temp->totlen = node->d.totlen;
> +			temp->offset = cpu_to_je32(ofs);
> +			temp->pino = node->d.pino;
> +			temp->version = node->d.version;
> +			temp->ino = node->d.ino;
> +			temp->nsize = node->d.nsize;
> +			temp->type = node->d.type;
> +			temp->next = NULL;
> +			
> +			memcpy(temp->name,invecs[1].iov_base,node->d.nsize);
> +			
> +			return jffs2_sum_add_mem(jeb, (union jffs2_sum_mem *)temp);
> +			
> +			break;
> +		}
> +		
> +		case JFFS2_NODETYPE_PADDING : {
> +			D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node PADDING\n"));
> +                        jeb->sum_collected->sum_padded += je32_to_cpu(node->u.totlen);
> +			break;
> +		}
> +		
> +		case JFFS2_NODETYPE_CLEANMARKER : {
> +			D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node CLEANMARKER\n"));
> +			break;
> +		}
> +		
> +		case JFFS2_NODETYPE_SUMMARY : {
> +			D1(printk(KERN_DEBUG "jffs2_sum_add_kvec(): Node SUMMARY\n"));
> +			break;
> +		}
> +		
> +		default : {
> +			printk(KERN_WARNING "jffs2_sum_add_kvec(): Node not supported\n");
> +			BUG();
> +			break;
> +		}
> +	}
> +	
> +	return 0;
> +}
> +
> +/* Process the summary information - called from jffs2_scan_eraseblock() */
> +
> +int jffs2_sum_scan_sumnode(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb, uint32_t ofs, uint32_t *pseudo_random)
> +{
> +	
> +	struct jffs2_unknown_node crcnode;
> +	struct jffs2_raw_node_ref *raw;
> +	struct jffs2_raw_node_ref *cache_ref;
> +	struct jffs2_inode_cache *ic;
> +	struct jffs2_full_dirent *fd;
> +		
> +	int i, err;
> +	int bad_sum = 0;
> +	int sumsize;
> +	uint32_t ino;
> +	uint32_t crc;
> +	struct jffs2_summary_node *summary;
> +		
> +	sumsize = c->sector_size - ofs;
> +	ofs += jeb->offset;
> +	
> +	D1(printk(KERN_DEBUG "JFFS2: summary found for 0x%08x at 0x%08x (0x%x bytes)\n", jeb->offset, ofs, sumsize));
> +	
> +	summary = (struct jffs2_summary_node *) kmalloc(sumsize, GFP_KERNEL);
> +		
> +	if (!summary) {
> +			return -ENOMEM;
> +	}
> +	
> +	err = jffs2_fill_scan_buf(c, (unsigned char *)summary, ofs, sumsize);
> +	
> +	if (err) {
> +			kfree(summary);
> +			return err;
> +	}
> +
> +	/* OK, now check for node validity and CRC */
> +	crcnode.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
> +	crcnode.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY);
> +	crcnode.totlen = summary->totlen;
> +	crc = crc32(0, &crcnode, sizeof(crcnode)-4);
> +	
> +	if (je32_to_cpu(summary->hdr_crc) != crc) {
> +			D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node header is corrupt (bad CRC or no summary at all)\n"));
> +			bad_sum = 1;
> +	}
> +	
> +	if ((!bad_sum) && (je32_to_cpu(summary->totlen) != sumsize)) {
> +			D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (wrong erasesize?)\n"));
> +			bad_sum = 1;
> +	}
> +	
> +	crc = crc32(0, summary, sizeof(struct jffs2_summary_node)-8);
> +		
> +	if ((!bad_sum) && (je32_to_cpu(summary->node_crc) != crc)) {
> +			D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node is corrupt (bad CRC)\n"));
> +			bad_sum = 1;
> +	}
> +	
> +	crc = crc32(0, summary->sum, sumsize - sizeof(struct jffs2_summary_node));
> +
> +	if ((!bad_sum) && (je32_to_cpu(summary->sum_crc) != crc)) {
> +			D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Summary node data is corrupt (bad CRC)\n"));
> +			bad_sum = 1;
> +	}
> +	
> +	if (!bad_sum) {
> +		
> +		struct jffs2_sum_unknown_flash *sp;
> +		sp = (struct jffs2_sum_unknown_flash *) summary->sum;
> +
> +		if ( je32_to_cpu(summary->cln_mkr) ) {
> +			
> +			D1(printk(KERN_DEBUG "Summary : CLEANMARKER node \n"));
> +			
> +			if (je32_to_cpu(summary->cln_mkr) != c->cleanmarker_size) {
> +				D1(printk(KERN_DEBUG "CLEANMARKER node has totlen 0x%x != normal 0x%x\n", 
> +				   je32_to_cpu(summary->cln_mkr), c->cleanmarker_size));
> +				UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) );
> +			} 
> +			else if (jeb->first_node) {
> +				D1(printk(KERN_DEBUG "CLEANMARKER node not first node in block (0x%08x)\n", jeb->offset);
> +				UNCHECKED_SPACE( PAD(je32_to_cpu(summary->cln_mkr))));
> +			} 
> +			else {
> +				struct jffs2_raw_node_ref *marker_ref = jffs2_alloc_raw_node_ref();
> +					
> +				if (!marker_ref) {
> +					D1(printk(KERN_NOTICE "Failed to allocate node ref for clean marker\n"));
> +					kfree(summary);
> +					return -ENOMEM;
> +				}
> +				
> +				marker_ref->next_in_ino = NULL;
> +				marker_ref->next_phys = NULL;
> +				marker_ref->flash_offset = jeb->offset | REF_NORMAL;
> +				marker_ref->__totlen = je32_to_cpu(summary->cln_mkr);
> +				jeb->first_node = jeb->last_node = marker_ref;
> +			
> +				USED_SPACE( PAD(je32_to_cpu(summary->cln_mkr)) );
> +								
> +			}
> +		}
> +
> +		if ( je32_to_cpu(summary->padded) ) {
> +                        DIRTY_SPACE(je32_to_cpu(summary->padded));
> +                }
> +		
> +		for(i = 0; i < je16_to_cpu(summary->sum_num); i++) {
> +			
> +			D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Processing summary information %d\n", i));
> +			uint8_t *temp8ptr = NULL;
> +			
> +			switch (je16_to_cpu(sp->nodetype)) {
> +				
> +				case JFFS2_NODETYPE_INODE : {
> +					struct jffs2_sum_inode_flash *spi;
> +					spi = (struct jffs2_sum_inode_flash *) sp;
> +						
> +					ino = je32_to_cpu(spi->inode);
> +					D1(printk(KERN_DEBUG "jffs2_scan_eraseblock(): Inode at 0x%08x\n", jeb->offset + je32_to_cpu(spi->offset)));
> +					raw = jffs2_alloc_raw_node_ref();
> +					if (!raw) {
> +						printk(KERN_NOTICE "jffs2_scan_eraseblock(): allocation of node reference failed\n");
> +						kfree(summary);
> +						return -ENOMEM;
> +					}
> +		
> +					ic = jffs2_get_ino_cache(c, ino);
> +					if (!ic) {
> +						ic = jffs2_scan_make_ino_cache(c, ino);
> +						if (!ic) {
> +							printk(KERN_NOTICE "jffs2_scan_eraseblock(): scan_make_ino_cache failed\n");
> +							jffs2_free_raw_node_ref(raw);
> +							kfree(summary);
> +							return -ENOMEM;
> +						}
> +					}
> +						
> +					raw->flash_offset = (jeb->offset + je32_to_cpu(spi->offset)) | REF_UNCHECKED;
> +					raw->__totlen = PAD(je32_to_cpu(spi->totlen));
> +					raw->next_phys = NULL;
> +					raw->next_in_ino = ic->nodes;
> +						
> +					ic->nodes = raw;
> +					if (!jeb->first_node)
> +							jeb->first_node = raw;
> +					if (jeb->last_node)
> +							jeb->last_node->next_phys = raw;
> +					jeb->last_node = raw;
> +						
> +					*pseudo_random += je32_to_cpu(spi->version);
> +					UNCHECKED_SPACE(PAD(je32_to_cpu(spi->totlen)));
> +					
> +					temp8ptr = (uint8_t *) sp;
> +					temp8ptr += JFFS2_SUMMARY_INODE_SIZE;
> +					sp = (struct jffs2_sum_unknown_flash *) temp8ptr;
> +					
> +					break;
> +				}
> +					
> +				case JFFS2_NODETYPE_DIRENT : {
> +					struct jffs2_sum_dirent_flash *spd;
> +					spd = (struct jffs2_sum_dirent_flash *) sp;
> +					
> +					fd = jffs2_alloc_full_dirent(spd->nsize+1);
> +					if (!fd) {
> +						kfree(summary);
> +						return -ENOMEM;
> +					}
> +					
> +					memcpy(&fd->name, spd->name, spd->nsize);
> +					fd->name[spd->nsize] = 0;
> +					
> +					raw = jffs2_alloc_raw_node_ref();
> +					if (!raw) {
> +						jffs2_free_full_dirent(fd);
> +						printk(KERN_NOTICE "jffs2_scan_dirent_node(): allocation of node reference failed\n");
> +						kfree(summary);
> +						return -ENOMEM;
> +					}
> +					
> +					ic = jffs2_scan_make_ino_cache(c, je32_to_cpu(spd->pino));
> +					if (!ic) {
> +						jffs2_free_full_dirent(fd);
> +						jffs2_free_raw_node_ref(raw);
> +						kfree(summary);
> +						return -ENOMEM;
> +					}
> +					
> +					raw->__totlen = PAD(je32_to_cpu(spd->totlen));
> +					raw->flash_offset = (jeb->offset + je32_to_cpu(spd->offset)) | REF_PRISTINE;
> +					raw->next_phys = NULL;
> +					raw->next_in_ino = ic->nodes;
> +					ic->nodes = raw;
> +					if (!jeb->first_node)
> +						jeb->first_node = raw;
> +					if (jeb->last_node)
> +						jeb->last_node->next_phys = raw;
> +					jeb->last_node = raw;
> +				
> +					fd->raw = raw;
> +					fd->next = NULL;
> +					fd->version = je32_to_cpu(spd->version);
> +					fd->ino = je32_to_cpu(spd->ino);
> +					fd->nhash = full_name_hash(fd->name, spd->nsize);
> +					fd->type = spd->type;
> +					USED_SPACE(PAD(je32_to_cpu(spd->totlen)));
> +					jffs2_add_fd_to_list(c, fd, &ic->scan_dents);
> +					
> +					*pseudo_random += je32_to_cpu(spd->version);
> +					
> +					temp8ptr = (uint8_t *) sp;
> +					temp8ptr += JFFS2_SUMMARY_DIRENT_SIZE(spd->nsize); 
> +					sp = (struct jffs2_sum_unknown_flash *) temp8ptr;
> +					
> +					break;
> +				}
> +				
> +				default : {
> +					printk(KERN_WARNING "Kernel doesn't support this type of node !!! Exiting");
> +					return -EIO;
> +					break;			
> +				}
> +			}
> +		}
> +		
> +		kfree(summary);
> +
> +		/* for ACCT_PARANOIA_CHECK */
> +		cache_ref = jffs2_alloc_raw_node_ref();
> +		
> +		if (!cache_ref) {
> +			printk(KERN_NOTICE "Failed to allocate node ref for cache\n");
> +			return -ENOMEM;
> +		}
> +		
> +		cache_ref->next_in_ino = NULL;
> +		cache_ref->next_phys = NULL;
> +		cache_ref->flash_offset = ofs | REF_NORMAL;
> +		cache_ref->__totlen = sumsize;
> +		
> +		if (!jeb->first_node)
> +			jeb->first_node = cache_ref;
> +		if (jeb->last_node)
> +			jeb->last_node->next_phys = cache_ref;
> +		jeb->last_node = cache_ref;
> +		
> +		USED_SPACE(sumsize);
> +
> +                jeb->wasted_size += jeb->free_size;
> +                c->wasted_size += jeb->free_size;
> +                c->free_size -= jeb->free_size;
> +                jeb->free_size = 0;
> +
> +		/* somebody check this and all of space accounting in summary support */
> +
> +		if ((jeb->used_size + jeb->unchecked_size) == PAD(c->cleanmarker_size) && !jeb->dirty_size 
> +			&& (!jeb->first_node || !jeb->first_node->next_in_ino) ) { 
> +				return BLK_STATE_CLEANMARKER; 
> +			}		
> +		/* move blocks with max 4 byte dirty space to cleanlist */	
> +		else if (!ISDIRTY(c->sector_size - (jeb->used_size + jeb->unchecked_size))) {
> +			c->dirty_size -= jeb->dirty_size;
> +			c->wasted_size += jeb->dirty_size; 
> +			jeb->wasted_size += jeb->dirty_size;
> +			jeb->dirty_size = 0;
> +			return BLK_STATE_CLEAN;
> +		}
> +		else if (jeb->used_size || jeb->unchecked_size) { 
> +				return BLK_STATE_PARTDIRTY; 
> +		}
> +		else { 
> +				return BLK_STATE_ALLDIRTY; 
> +		}
> +	}
> +	
> +	return 0;
> +}
> +
> +/* Write out summary information - called from jffs2_do_reserve_space */
> +
> +int jffs2_sum_write_sumnode(struct jffs2_sb_info *c)
> +{
> +	struct jffs2_summary_node isum;
> +	union jffs2_sum_mem *temp;
> +	jint32_t offset;
> +	jint32_t *wpage;
> +	uint8_t *tempptr;
> +	int datasize;
> +	int infosize;
> +	int padsize;
> +	size_t retlen;
> +	int ret;
> +	struct jffs2_eraseblock *jeb;
> +	struct kvec vecs[2];	
> +	jint32_t magic = cpu_to_je32(JFFS2_SUM_MAGIC);
> +		
> +	D2(printk(KERN_DEBUG "jffs2_sum_write_sumnode()\n"));
> +	
> +	jeb = c->nextblock;
> +	
> +	if (!jeb->sum_collected->sum_num || !jeb->sum_collected->sum_list) {
> +		printk(KERN_WARNING "JFFS2: jffs2_sum_write_sumnode(): empty summary info!!!\n");
> +		BUG(); 
> +	}
> +	
> +	datasize = jeb->sum_collected->sum_size + sizeof(struct jffs2_sum_marker);
> +	infosize = sizeof(struct jffs2_summary_node) + datasize;
> +	padsize = jeb->free_size - infosize;
> +	infosize += padsize; datasize += padsize;
> +	offset = cpu_to_je32(c->sector_size - jeb->free_size);
> +	
> +	if (padsize < 0) { // if jeb hasn't got enought free space for summary
> +		
> +		union jffs2_sum_mem *temp;	
> +		
> +		while(jeb->sum_collected->sum_list){ //cleanup sum_list
> +			temp = jeb->sum_collected->sum_list;
> +			jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next;
> +			kfree(temp);
> +			jeb->sum_collected->sum_num--;
> +		}
> +		
> +		jeb->sum_collected->sum_list = NULL;
> +		jeb->sum_collected->sum_num = 0;
> +		jeb->sum_collected->sum_size = JFFS2_SUMMARY_NOSUM_SIZE; // don't try to write out summary for this node
> +		
> +		printk(KERN_WARNING "JFFS2: not enough space for summary, padsize = %d\n",padsize);
> +                return 0;
> +	}
> +			
> +	memset(c->summary_buf, 0xff, datasize);
> +	memset(&isum, 0, sizeof(isum));
> +	
> +	isum.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
> +	isum.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY);
> +	isum.totlen = cpu_to_je32(infosize);
> +	isum.hdr_crc = cpu_to_je32(crc32(0, &isum, sizeof(struct jffs2_unknown_node) - 4));
> +        isum.padded = cpu_to_je32(jeb->sum_collected->sum_padded);
> +		
> +	if (c->cleanmarker_size) {
> +		isum.cln_mkr = cpu_to_je32(c->cleanmarker_size);	
> +	}
> +	else{
> +		isum.cln_mkr = cpu_to_je32(0);
> +	}
> +	
> +	isum.sum_num = cpu_to_je16(jeb->sum_collected->sum_num);
> +	wpage = c->summary_buf;
> +	
> +		
> +	while (jeb->sum_collected->sum_num) { // write sum_data 
> +		
> +
> +		switch(je16_to_cpu(jeb->sum_collected->sum_list->u.nodetype)){
> +			
> +			case JFFS2_NODETYPE_INODE : {
> +				jint16_t *temp16ptr = (jint16_t *)wpage;
> +				
> +				*(temp16ptr++) = jeb->sum_collected->sum_list->i.nodetype;
> +				wpage = (jint32_t *) temp16ptr;
> +				
> +				*(wpage++) = jeb->sum_collected->sum_list->i.inode;
> +				*(wpage++) = jeb->sum_collected->sum_list->i.version;
> +				*(wpage++) = jeb->sum_collected->sum_list->i.offset;
> +				*(wpage++) = jeb->sum_collected->sum_list->i.totlen;
> +				break;
> +			}
> +			
> +			case JFFS2_NODETYPE_DIRENT : {
> +				jint16_t *temp16ptr = (jint16_t *) wpage;
> +				uint8_t *temp8ptr = NULL;
> +				
> +				*(temp16ptr++) = jeb->sum_collected->sum_list->d.nodetype;
> +				wpage = (jint32_t *) temp16ptr;
> +				
> +				*(wpage++) = jeb->sum_collected->sum_list->d.totlen;
> +				*(wpage++) = jeb->sum_collected->sum_list->d.offset;
> +				*(wpage++) = jeb->sum_collected->sum_list->d.pino;
> +				*(wpage++) = jeb->sum_collected->sum_list->d.version;
> +				*(wpage++) = jeb->sum_collected->sum_list->d.ino;
> +				
> +				temp8ptr = (uint8_t *) wpage;
> +				*(temp8ptr++) = jeb->sum_collected->sum_list->d.nsize;
> +				*(temp8ptr++) = jeb->sum_collected->sum_list->d.type;
> +								
> +				memcpy(temp8ptr,jeb->sum_collected->sum_list->d.name,jeb->sum_collected->sum_list->d.nsize);
> +				temp8ptr += jeb->sum_collected->sum_list->d.nsize;
> +				wpage = (jint32_t *) temp8ptr;
> +				
> +				break;
> +			}
> +			
> +			default : {
> +				printk(KERN_WARNING "Unknown node in summary information!!!\n");
> +				BUG();
> +			}
> +		}
> +		
> +		temp = jeb->sum_collected->sum_list;
> +		jeb->sum_collected->sum_list = jeb->sum_collected->sum_list->u.next;
> +		kfree(temp);
> +		
> +		jeb->sum_collected->sum_num--;
> +	}
> +	
> +	jeb->sum_collected->sum_size = 0;
> +	
> +	tempptr = (uint8_t *) wpage;
> +	tempptr += padsize;
> +	wpage = (jint32_t *) tempptr;
> +	
> +	*(wpage++) = offset;
> +	*(wpage++) = cpu_to_je32(c->sector_size);
> +	*(wpage++) = magic;
> +	isum.sum_crc = cpu_to_je32(crc32(0, c->summary_buf, datasize));
> +	isum.node_crc = cpu_to_je32(crc32(0, &isum, sizeof(isum) - 8));
> +	
> +	vecs[0].iov_base = &isum;
> +	vecs[0].iov_len = sizeof(isum);
> +	vecs[1].iov_base = c->summary_buf;
> +	vecs[1].iov_len = datasize;
> +			
> +	D1(printk(KERN_DEBUG "JFFS2: writing out data to flash to pos : 0x%08x\n",jeb->offset + c->sector_size - jeb->free_size));
> +	
> +	spin_unlock(&c->erase_completion_lock);
> +	ret = jffs2_flash_writev(c, vecs, 2, jeb->offset + c->sector_size - jeb->free_size, &retlen, 0);
> +	spin_lock(&c->erase_completion_lock);
> +
> +	
> +	if (ret || (retlen != infosize)) {
> +		printk(KERN_WARNING "JFFS2: write of %zd bytes at 0x%08x failed. returned %d, retlen %zd\n", 
> +		      infosize, jeb->offset + c->sector_size - jeb->free_size, ret, retlen);
> +
> +		jeb->sum_collected->sum_size = JFFS2_SUMMARY_NOSUM_SIZE;
> +				
> +		WASTED_SPACE(infosize);
> +		
> +		return 0;
> +	}	
> +	
> +	/*spin_unlock(&c->erase_completion_lock);
> +	jffs2_flush_wbuf_pad(c); // summary filled the wbuf
> +	spin_lock(&c->erase_completion_lock);*/
> +
> +	WASTED_SPACE(infosize);
> +	
> +	return 0;	
> +}
> diff --unified --new-file --recursive mtd/fs/jffs2/summary.h mtd-dec07/fs/jffs2/summary.h
> --- mtd/fs/jffs2/summary.h	1970-01-01 01:00:00.000000000 +0100
> +++ mtd-dec07/fs/jffs2/summary.h	2004-12-07 16:03:02.000000000 +0100
> @@ -0,0 +1,158 @@
> +/*
> + * JFFS2 -- Journalling Flash File System, Version 2.
> + *
> + * Copyright (C) 2004  Ferenc Havasi <havasi@inf.u-szeged.hu>,
> + *                     Zoltan Sogor <weth@inf.u-szeged.hu>,
> + *                     Patrik Kluba <pajko@halom.u-szeged.hu>,
> + *                     University of Szeged, Hungary
> + *
> + * For licensing information, see the file 'LICENCE' in this directory.
> + *
> + * $Id$
> + *
> + */
> +
> +#ifndef JFFS2_SUMMARY_H
> +#define JFFS2_SUMMARY_H
> +
> +#include <linux/uio.h>
> +#include <linux/jffs2.h>
> +
> +#define DIRTY_SPACE(x) do { typeof(x) _x = (x); \
> +		c->free_size -= _x; c->dirty_size += _x; \
> +		jeb->free_size -= _x ; jeb->dirty_size += _x; \
> +		}while(0)
> +#define USED_SPACE(x) do { typeof(x) _x = (x); \
> +		c->free_size -= _x; c->used_size += _x; \
> +		jeb->free_size -= _x ; jeb->used_size += _x; \
> +		}while(0)
> +#define WASTED_SPACE(x) do { typeof(x) _x = (x); \
> +		c->free_size -= _x; c->wasted_size += _x; \
> +		jeb->free_size -= _x ; jeb->wasted_size += _x; \
> +		}while(0)
> +#define UNCHECKED_SPACE(x) do { typeof(x) _x = (x); \
> +		c->free_size -= _x; c->unchecked_size += _x; \
> +		jeb->free_size -= _x ; jeb->unchecked_size += _x; \
> +		}while(0)
> +
> +#define BLK_STATE_ALLFF		0
> +#define BLK_STATE_CLEAN		1
> +#define BLK_STATE_PARTDIRTY	2
> +#define BLK_STATE_CLEANMARKER	3
> +#define BLK_STATE_ALLDIRTY	4
> +#define BLK_STATE_BADBLOCK	5
> +
> +#define JFFS2_SUMMARY_NOSUM_SIZE 0xffffffff
> +#define JFFS2_SUMMARY_INODE_SIZE (sizeof(struct jffs2_sum_inode_flash))
> +#define JFFS2_SUMMARY_DIRENT_SIZE(x) (sizeof(struct jffs2_sum_dirent_flash) + (x))
> +
> +struct jffs2_sum_unknown_flash
> +{
> +	jint16_t nodetype;	/* node type	*/
> +};
> +
> +struct jffs2_sum_inode_flash
> +{
> +	jint16_t nodetype;	/* node type	*/
> +	jint32_t inode;		/* inode number */
> +	jint32_t version;	/* inode version */
> +	jint32_t offset;	/* offset on jeb */
> +	jint32_t totlen; 	/* record length */
> +} __attribute__((packed));
> +
> +struct jffs2_sum_dirent_flash
> +{
> +	jint16_t nodetype;	/* == JFFS_NODETYPE_DIRENT */
> +	jint32_t totlen;	/* record length */
> +	jint32_t offset;	/* ofset on jeb */
> +	jint32_t pino;		/* parent inode */
> +	jint32_t version;	/* dirent version */
> +	jint32_t ino; 		/* == zero for unlink */
> +	uint8_t nsize;		/* dirent name size */
> +	uint8_t type;		/* dirent type */
> +	uint8_t name[0];	/* dirent name */
> +} __attribute__((packed));
> +
> +union jffs2_sum_flash{
> +	struct jffs2_sum_unknown_flash u;
> +	struct jffs2_sum_inode_flash i;
> +	struct jffs2_sum_dirent_flash d; 
> +};
> +
> +/* list version of jffs2_sum_*flash for kernel and sumtool */
> +struct jffs2_sum_unknown_mem
> +{
> +	union jffs2_sum_mem *next;
> +	jint16_t nodetype;	/* node type	*/
> +	
> +};
> +
> +struct jffs2_sum_inode_mem
> +{
> +	union jffs2_sum_mem *next;
> +	jint16_t nodetype;	/* node type	*/
> +	jint32_t inode;		/* inode number */
> +	jint32_t version;	/* inode version */
> +	jint32_t offset;	/* offset on jeb */
> +	jint32_t totlen; 	/* record length */
> +} __attribute__((packed));
> +
> +struct jffs2_sum_dirent_mem
> +{
> +	union jffs2_sum_mem *next;
> +	jint16_t nodetype;	/* == JFFS_NODETYPE_DIRENT */
> +	jint32_t totlen;	/* record length */
> +	jint32_t offset;	/* ofset on jeb */
> +	jint32_t pino;		/* parent inode */
> +	jint32_t version;	/* dirent version */
> +	jint32_t ino; 		/* == zero for unlink */
> +	uint8_t nsize;		/* dirent name size */
> +	uint8_t type;		/* dirent type */
> +	uint8_t name[0];	/* dirent name */
> +} __attribute__((packed));
> +
> +union jffs2_sum_mem 
> +{
> +	struct jffs2_sum_unknown_mem u;
> +	struct jffs2_sum_inode_mem i;
> +	struct jffs2_sum_dirent_mem d; 
> +};
> +
> +struct jffs2_sum_info
> +{
> +	uint32_t sum_size;
> +	uint32_t sum_num;
> +        uint32_t sum_padded;
> +	union jffs2_sum_mem *sum_list;
> +};
> +
> +struct jffs2_sum_marker
> +{
> +	jint32_t offset;
> +	jint32_t erase_size;
> +	jint32_t magic;
> +};
> +
> +#define JFFS2_SUMMARY_FRAME_SIZE (sizeof(struct jffs2_summary_node)+sizeof(struct jffs2_sum_marker)+3)
> +	
> +#if !(defined(SUM_TOOL) || defined(JFFS2DUMP))
> +
> +int jffs2_sum_init(struct jffs2_sb_info *c);
> +void jffs2_sum_exit(struct jffs2_sb_info *c);
> +
> +int jffs2_sum_care_sum_collected(struct jffs2_eraseblock *jeb);
> +
> +void jffs2_sum_clean_collected(struct jffs2_eraseblock *jeb);
> +void jffs2_sum_clean_all_info(struct jffs2_sb_info *c); /* clean up all summary information in all jeb (umount) */
> +
> +int jffs2_sum_add_padding_mem(struct jffs2_eraseblock *jeb, uint32_t size);
> +int jffs2_sum_add_inode_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_inode *ri, uint32_t ofs);	
> +int jffs2_sum_add_dirent_mem(struct jffs2_eraseblock *jeb, struct jffs2_raw_dirent *rd, uint32_t ofs);	
> +int jffs2_sum_add_kvec(struct jffs2_sb_info *c, const struct kvec *invecs, unsigned long count,  uint32_t to);	
> +
> +int jffs2_sum_scan_sumnode(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb, uint32_t ofs, uint32_t *pseudo_random);
> +int jffs2_sum_write_sumnode(struct jffs2_sb_info *c);	
> +
> +#endif
> +
> +#endif /* JFFS2_SUMMARY_H */
> diff --unified --new-file --recursive mtd/fs/jffs2/super-v24.c mtd-dec07/fs/jffs2/super-v24.c
> --- mtd/fs/jffs2/super-v24.c	2004-11-17 00:00:14.000000000 +0100
> +++ mtd-dec07/fs/jffs2/super-v24.c	2004-12-07 16:03:02.000000000 +0100
> @@ -24,6 +24,7 @@
>  #include <linux/mtd/mtd.h>
>  #include "compr.h"
>  #include "nodelist.h"
> +#include "summary.h"
>  
>  #ifndef MTD_BLOCK_MAJOR
>  #define MTD_BLOCK_MAJOR 31
> @@ -72,6 +73,9 @@
>  		put_mtd_device(c->mtd);
>  		return NULL;
>  	}
> +#ifdef CONFIG_JFFS2_SUMMARY
> +        jffs2_sum_init(c);
> +#endif
>  
>  	return sb;
>  }
> @@ -88,6 +92,10 @@
>  	down(&c->alloc_sem);
>  	jffs2_flush_wbuf_pad(c);
>  	up(&c->alloc_sem);
> +#ifdef CONFIG_JFFS2_SUMMARY
> +        jffs2_sum_clean_all_info(c);
> +        jffs2_sum_exit(c);
> +#endif
>  	jffs2_free_ino_caches(c);
>  	jffs2_free_raw_node_refs(c);
>  	kfree(c->blocks);
> @@ -110,6 +118,9 @@
>  #ifdef CONFIG_JFFS2_FS_NAND
>  	       " (NAND)"
>  #endif
> +#ifdef CONFIG_JFFS2_SUMMARY
> +	       " (SUMMARY)"
> +#endif
>  	       " (C) 2001-2003 Red Hat, Inc.\n");
>  
>  #ifdef JFFS2_OUT_OF_KERNEL
> diff --unified --new-file --recursive mtd/fs/jffs2/super.c mtd-dec07/fs/jffs2/super.c
> --- mtd/fs/jffs2/super.c	2004-11-26 15:08:31.000000000 +0100
> +++ mtd-dec07/fs/jffs2/super.c	2004-12-07 16:03:02.000000000 +0100
> @@ -152,6 +152,10 @@
>  		return ERR_PTR(ret);
>  	}
>  
> +#ifdef CONFIG_JFFS2_SUMMARY
> +        jffs2_sum_init(c);
> +#endif
> +
>  	sb->s_flags |= MS_ACTIVE;
>  	return sb;
>  
> @@ -275,6 +279,10 @@
>  	down(&c->alloc_sem);
>  	jffs2_flush_wbuf_pad(c);
>  	up(&c->alloc_sem);
> +#ifdef CONFIG_JFFS2_SUMMARY
> +        jffs2_sum_clean_all_info(c);
> +        jffs2_sum_exit(c);
> +#endif
>  	jffs2_free_ino_caches(c);
>  	jffs2_free_raw_node_refs(c);
>  	if (c->mtd->flags & MTD_NO_VIRTBLOCKS)
> @@ -312,6 +320,9 @@
>  #ifdef CONFIG_JFFS2_FS_NAND
>  	       " (NAND)"
>  #endif
> +#ifdef CONFIG_JFFS2_SUMMARY
> +               " (SUMMARY) "
> +#endif
>  	       " (C) 2001-2003 Red Hat, Inc.\n");
>  
>  	jffs2_inode_cachep = kmem_cache_create("jffs2_i",
> diff --unified --new-file --recursive mtd/fs/jffs2/wbuf.c mtd-dec07/fs/jffs2/wbuf.c
> --- mtd/fs/jffs2/wbuf.c	2004-11-26 15:08:31.000000000 +0100
> +++ mtd-dec07/fs/jffs2/wbuf.c	2004-12-07 16:17:11.000000000 +0100
> @@ -261,7 +261,7 @@
>  
> 
>  	/* ... and get an allocation of space from a shiny new block instead */
> -	ret = jffs2_reserve_space_gc(c, end-start, &ofs, &len);
> +	ret = jffs2_reserve_space_gc(c, end-start, &ofs, &len, JFFS2_SUMMARY_NOSUM_SIZE);
>  	if (ret) {
>  		printk(KERN_WARNING "Failed to allocate space for wbuf recovery. Data loss ensues.\n");
>  		if (buf)
> @@ -601,8 +601,15 @@
>  	uint32_t outvec_to = to;
>  
>  	/* If not NAND flash, don't bother */
> -	if (!c->wbuf)
> + 	if (!c->wbuf){
> +#ifdef CONFIG_JFFS2_SUMMARY
> + 		D1(printk("JFFS2.summary: NOT NAND MEMORY\n"));
> + 		if(jffs2_sum_add_kvec(c,invecs,count,(uint32_t) to)){
> + 			printk("jffs2_sum_add_kvec(): MEMORY ALLOCATION ERROR!");
> + 		}	
> +#endif		
>  		return jffs2_flash_direct_writev(c, invecs, count, to, retlen);
> +	}
>  	
>  	down_write(&c->wbuf_sem);
>  
> @@ -801,6 +808,12 @@
>  alldone:
>  	*retlen = donelen;
>  
> +#ifdef CONFIG_JFFS2_SUMMARY
> +	if(jffs2_sum_add_kvec(c,invecs,count,(uint32_t) to)){
> +		printk("jffs2_sum_add_kvec(): MEMORY ALLOCATION ERROR!");
> +	}
> +#endif
> +	
>  	if (c->wbuf_len && ino)
>  		jffs2_wbuf_dirties_inode(c, ino);
>  
> diff --unified --new-file --recursive mtd/fs/jffs2/write.c mtd-dec07/fs/jffs2/write.c
> --- mtd/fs/jffs2/write.c	2004-11-17 00:00:14.000000000 +0100
> +++ mtd-dec07/fs/jffs2/write.c	2004-12-07 16:03:02.000000000 +0100
> @@ -173,13 +173,13 @@
>  			D1(ACCT_PARANOIA_CHECK(jeb));
>  
>  			if (alloc_mode == ALLOC_GC) {
> -				ret = jffs2_reserve_space_gc(c, sizeof(*ri) + datalen, &flash_ofs, &dummy);
> +				ret = jffs2_reserve_space_gc(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, JFFS2_SUMMARY_INODE_SIZE);
>  			} else {
>  				/* Locking pain */
>  				up(&f->sem);
>  				jffs2_complete_reservation(c);
>  			
> -				ret = jffs2_reserve_space(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, alloc_mode);
> +				ret = jffs2_reserve_space(c, sizeof(*ri) + datalen, &flash_ofs, &dummy, alloc_mode, JFFS2_SUMMARY_INODE_SIZE);
>  				down(&f->sem);
>  			}
>  
> @@ -308,13 +308,13 @@
>  			D1(ACCT_PARANOIA_CHECK(jeb));
>  
>  			if (alloc_mode == ALLOC_GC) {
> -				ret = jffs2_reserve_space_gc(c, sizeof(*rd) + namelen, &flash_ofs, &dummy);
> +				ret = jffs2_reserve_space_gc(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
>  			} else {
>  				/* Locking pain */
>  				up(&f->sem);
>  				jffs2_complete_reservation(c);
>  			
> -				ret = jffs2_reserve_space(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, alloc_mode);
> +				ret = jffs2_reserve_space(c, sizeof(*rd) + namelen, &flash_ofs, &dummy, alloc_mode, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
>  				down(&f->sem);
>  			}
>  
> @@ -371,7 +371,7 @@
>  	retry:
>  		D2(printk(KERN_DEBUG "jffs2_commit_write() loop: 0x%x to write to 0x%x\n", writelen, offset));
>  
> -		ret = jffs2_reserve_space(c, sizeof(*ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, ALLOC_NORMAL);
> +		ret = jffs2_reserve_space(c, sizeof(*ri) + JFFS2_MIN_DATA_LEN, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
>  		if (ret) {
>  			D1(printk(KERN_DEBUG "jffs2_reserve_space returned %d\n", ret));
>  			break;
> @@ -458,7 +458,7 @@
>  	/* Try to reserve enough space for both node and dirent. 
>  	 * Just the node will do for now, though 
>  	 */
> -	ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL);
> +	ret = jffs2_reserve_space(c, sizeof(*ri), &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_INODE_SIZE);
>  	D1(printk(KERN_DEBUG "jffs2_do_create(): reserved 0x%x bytes\n", alloclen));
>  	if (ret) {
>  		up(&f->sem);
> @@ -487,7 +487,7 @@
>  
>  	up(&f->sem);
>  	jffs2_complete_reservation(c);
> -	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
> +	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
>  		
>  	if (ret) {
>  		/* Eep. */
> @@ -557,7 +557,7 @@
>  		if (!rd)
>  			return -ENOMEM;
>  
> -		ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_DELETION);
> +		ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_DELETION, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
>  		if (ret) {
>  			jffs2_free_raw_dirent(rd);
>  			return ret;
> @@ -663,7 +663,7 @@
>  	if (!rd)
>  		return -ENOMEM;
>  
> -	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL);
> +	ret = jffs2_reserve_space(c, sizeof(*rd)+namelen, &phys_ofs, &alloclen, ALLOC_NORMAL, JFFS2_SUMMARY_DIRENT_SIZE(namelen));
>  	if (ret) {
>  		jffs2_free_raw_dirent(rd);
>  		return ret;
> diff --unified --new-file --recursive mtd/include/linux/jffs2.h mtd-dec07/include/linux/jffs2.h
> --- mtd/include/linux/jffs2.h	2004-11-17 00:00:15.000000000 +0100
> +++ mtd-dec07/include/linux/jffs2.h	2004-12-07 16:03:02.000000000 +0100
> @@ -28,6 +28,9 @@
>  #define JFFS2_EMPTY_BITMASK 0xffff
>  #define JFFS2_DIRTY_BITMASK 0x0000
>  
> +/* Summary node MAGIC marker */
> +#define JFFS2_SUM_MAGIC	0x02851885 
> +
>  /* We only allow a single char for length, and 0xFF is empty flash so
>     we don't want it confused with a real length. Hence max 254.
>  */
> @@ -62,6 +65,8 @@
>  #define JFFS2_NODETYPE_CLEANMARKER (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3)
>  #define JFFS2_NODETYPE_PADDING (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 4)
>  
> +#define JFFS2_NODETYPE_SUMMARY (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 6)
> +
>  // Maybe later...
>  //#define JFFS2_NODETYPE_CHECKPOINT (JFFS2_FEATURE_RWCOMPAT_DELETE | JFFS2_NODE_ACCURATE | 3)
>  //#define JFFS2_NODETYPE_OPTIONS (JFFS2_FEATURE_RWCOMPAT_COPY | JFFS2_NODE_ACCURATE | 4)
> @@ -148,10 +153,24 @@
>  	uint8_t data[0];
>  } __attribute__((packed));
>  
> +struct jffs2_summary_node{
> +    jint16_t magic;
> +	jint16_t nodetype; 	/* = JFFS2_NODETYPE_INODE_SUM */
> +	jint32_t totlen;
> +	jint32_t hdr_crc;
> +	jint16_t sum_num;	/* number of sum entries*/
> +	jint32_t cln_mkr;	/* clean marker size, 0 = no cleanmarker */
> +        jint32_t padded;        /* sum of the size of padding nodes */
> +	jint32_t sum_crc;	/* summary information crc */
> +	jint32_t node_crc; 	/* node crc */
> +	jint32_t sum[0]; 	/* inode summary info */
> +} __attribute__((packed));
> +
>  union jffs2_node_union {
>  	struct jffs2_raw_inode i;
>  	struct jffs2_raw_dirent d;
>  	struct jffs2_unknown_node u;
> +	struct jffs2_summary_node s;
>  };
>  
>  #endif /* __LINUX_JFFS2_H__ */
> diff --unified --new-file --recursive mtd/include/linux/jffs2_fs_sb.h mtd-dec07/include/linux/jffs2_fs_sb.h
> --- mtd/include/linux/jffs2_fs_sb.h	2004-11-26 15:08:31.000000000 +0100
> +++ mtd-dec07/include/linux/jffs2_fs_sb.h	2004-12-07 16:03:02.000000000 +0100
> @@ -110,6 +110,9 @@
>  	uint32_t fsdata_pos;
>  	uint32_t fsdata_len;
>  #endif
> +#ifdef CONFIG_JFFS2_SUMMARY
> +        jint32_t *summary_buf;
> +#endif
>  
>  	/* OS-private pointer for getting back to master superblock info */
>  	void *os_priv;
> diff --unified --new-file --recursive mtd/util/Makefile mtd-dec07/util/Makefile
> --- mtd/util/Makefile	2004-07-14 00:00:18.000000000 +0200
> +++ mtd-dec07/util/Makefile	2004-12-07 16:03:02.000000000 +0100
> @@ -13,7 +13,7 @@
>  TARGETS = ftl_format flash_erase flash_eraseall nanddump doc_loadbios \
>  	mkfs.jffs ftl_check mkfs.jffs2 flash_lock flash_unlock \
>  	flash_info mtd_debug flashcp nandwrite jffs2dump \
> -	nftldump nftl_format docfdisk #jffs2reader
> +	nftldump nftl_format docfdisk sumtool #jffs2reader
>  
>  SYMLINKS = compr_lzari.c compr_lzo.c
>  
> @@ -48,6 +48,10 @@
>  jffs2dump: jffs2dump.o crc32.o
>  	$(CC) $(LDFLAGS) -o $@ $^
>  
> +sumtool: sumtool.o crc32.o
> +	$(CC) $(LDFLAGS) -o $@ $^
> +	
> +
>  install: ${TARGETS}
>  	mkdir -p ${DESTDIR}/${SBINDIR}
>  	install -m0755 -oroot -groot ${TARGETS} ${DESTDIR}/${SBINDIR}/
> diff --unified --new-file --recursive mtd/util/jffs2dump.c mtd-dec07/util/jffs2dump.c
> --- mtd/util/jffs2dump.c	2004-12-03 00:00:10.000000000 +0100
> +++ mtd-dec07/util/jffs2dump.c	2004-12-07 16:03:02.000000000 +0100
> @@ -35,6 +35,10 @@
>  #include <getopt.h>
>  #include "crc32.h"
>  
> +#define CONFIG_JFFS2_SUMMARY
> +#define JFFS2DUMP
> +#include "../fs/jffs2/summary.h"
> +
>  #define PROGRAM "jffs2dump"
>  #define VERSION "$Revision: 1.7 $"
>  
> @@ -177,7 +181,7 @@
>   */
>  void do_dumpcontent (void) 
>  {
> -	char			*p = data;
> +	char			*p = data, *p_free_begin;
>  	union jffs2_node_union 	*node;
>  	int			empty = 0, dirty = 0;
>  	char			name[256];
> @@ -186,16 +190,21 @@
>  	int			bitchbitmask = 0;
>  	int			obsolete;
>  
> +	p_free_begin = NULL;
>  	while ( p < (data + imglen)) {
>  		node = (union jffs2_node_union*) p;
>  		
>  		/* Skip empty space */
> +		if (!p_free_begin) p_free_begin = p;
>  		if (je16_to_cpu (node->u.magic) == 0xFFFF && je16_to_cpu (node->u.nodetype) == 0xFFFF) {
>  			p += 4;
>  			empty += 4;
>  			continue;
>  		}
> -		
> +
> +		if (p != p_free_begin) printf("Empty space found from 0x%08x to 0x%08x\n", p_free_begin-data, p-data);
> +		p_free_begin = NULL;
> +
>  		if (je16_to_cpu (node->u.magic) != JFFS2_MAGIC_BITMASK)	{
>  			if (!bitchbitmask++)
>      			    printf ("Wrong bitmask  at  0x%08x, 0x%04x\n", p - data, je16_to_cpu (node->u.magic));
> @@ -277,7 +286,110 @@
>  
>  			p += PAD(je32_to_cpu (node->d.totlen));						
>  			break;
> -	
> +
> +		case JFFS2_NODETYPE_SUMMARY:{
> +			
> +			int i;
> +			struct jffs2_sum_marker * sm;
> +			
> +			printf ("%8s Inode Sum  node at 0x%08x, totlen 0x%08x, sum_num  %5d, cleanmarker size %5d\n",
> +					obsolete ? "Obsolete" : "",
> +					p - data,
> +					je32_to_cpu (node->s.totlen),
> +					je16_to_cpu (node->s.sum_num),
> +					je32_to_cpu (node->s.cln_mkr));
> +
> +			crc = crc32 (0, node, sizeof (struct jffs2_summary_node) - 8);
> +			if (crc != je32_to_cpu (node->s.node_crc)) {
> +				printf ("Wrong node_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.node_crc), crc);
> +				p += PAD(je32_to_cpu (node->s.totlen));
> +				dirty += PAD(je32_to_cpu (node->s.totlen));;
> +				continue;
> +			}
> +			
> +			crc = crc32(0, p + sizeof (struct jffs2_summary_node),  je32_to_cpu (node->s.totlen) - sizeof(struct jffs2_summary_node));
> +			if (crc != je32_to_cpu(node->s.sum_crc)) {
> +				printf ("Wrong data_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - data, je32_to_cpu (node->s.sum_crc), crc);
> +				p += PAD(je32_to_cpu (node->s.totlen));
> +				dirty += PAD(je32_to_cpu (node->s.totlen));;
> +				continue;
> +			}
> +
> +			if(verbose){
> +				struct jffs2_sum_unknown_flash *sp;
> +				uint8_t *temp8ptr = NULL;
> +				sp = (struct jffs2_sum_unknown_flash *) (p + sizeof (struct jffs2_summary_node));
> +					
> +				for(i = 0; i < je16_to_cpu (node->s.sum_num); i++){
> +					
> +					switch(je16_to_cpu(sp->nodetype)){
> +						case JFFS2_NODETYPE_INODE : {
> +							
> +							struct jffs2_sum_inode_flash *spi;
> +							spi = (struct jffs2_sum_inode_flash *) sp;
> +								
> +							printf ("%14s #ino  %5d,  version %5d, offset 0x%08x, totlen 0x%08x\n",
> +							"",
> +							je32_to_cpu (spi->inode),
> +							je32_to_cpu (spi->version),
> +							je32_to_cpu (spi->offset), 
> +							je32_to_cpu (spi->totlen));
> +							
> +							temp8ptr = (uint8_t *) sp;
> +							temp8ptr += JFFS2_SUMMARY_INODE_SIZE;
> +							sp = (struct jffs2_sum_unknown_flash *) temp8ptr;
> +							
> +							break;	
> +						}
> +						
> +						case JFFS2_NODETYPE_DIRENT : {
> +							
> +							char name[255];
> +							struct jffs2_sum_dirent_flash *spd;
> +							spd = (struct jffs2_sum_dirent_flash *) sp;
> +							
> +							memcpy(name,spd->name,spd->nsize);
> +							name [spd->nsize] = 0x0;
> +							
> +							printf ("%14s dirent offset 0x%08x, totlen 0x%08x, #pino  %5d,  version %5d, #ino  %8d, nsize %8d, name %s \n",
> +							"",
> +							je32_to_cpu (spd->offset), 
> +							je32_to_cpu (spd->totlen),
> +							je32_to_cpu (spd->pino),
> +							je32_to_cpu (spd->version),
> +							je32_to_cpu (spd->ino),
> +							spd->nsize,
> +							name);
> +							
> +							temp8ptr = (uint8_t *) sp;
> +							temp8ptr += JFFS2_SUMMARY_DIRENT_SIZE(spd->nsize); 
> +							sp = (struct jffs2_sum_unknown_flash *) temp8ptr;
> +							
> +							break;
> +						}
> +						
> +						default : {
> +							break;
> +						}
> +					}	
> +				}
> +				
> +				
> +				sm = (struct jffs2_sum_marker *) ((char *)p + je32_to_cpu(node->s.totlen) - sizeof(struct jffs2_sum_marker));
> +				
> +				printf("%14s Sum Node Offset  0x%08x, Erase_block_size 0x%08x, Magic 0x%08x, Padded size 0x%08x\n" ,
> +					"",
> +					je32_to_cpu(sm->offset),
> +					je32_to_cpu(sm->erase_size),
> +					je32_to_cpu(sm->magic),
> +                                        je32_to_cpu(node->s.padded));
> +			}
> +			
> +			p += PAD(je32_to_cpu (node->s.totlen));
> +			break;
> +			
> +		}
> +			
>  		case JFFS2_NODETYPE_CLEANMARKER:
>  			if (verbose) {
>  				printf ("%8s Cleanmarker     at 0x%08x, totlen 0x%08x\n", 
> diff --unified --new-file --recursive mtd/util/sumtool.c mtd-dec07/util/sumtool.c
> --- mtd/util/sumtool.c	1970-01-01 01:00:00.000000000 +0100
> +++ mtd-dec07/util/sumtool.c	2004-12-07 17:47:31.000000000 +0100
> @@ -0,0 +1,857 @@
> +/*
> + *  sumtool.c
> + *
> + *  Copyright (C) 2004 Zoltan Sogor <weth@inf.u-szeged.hu>,
> + *                     Ferenc Havasi <havasi@inf.u-szeged.hu>,
> + *                     Patrik Kluba <pajko@halom.u-szeged.hu>,
> + *                     University of Szeged, Hungary
> + *
> + * $Id: sumtool.c,v 1.2 2004/10/20 09:56:08 hafy Exp $
> + * 
> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License
> + * as published by the Free Software Foundation; either version 2
> + * of the License, or (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program; if not, write to the Free Software
> + * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.
> + *
> + * Overview:
> + *   This is a utility to reorder nodes and insert inode summary information
> + *   into JFFS2 image for faster mount time - specially on NAND.
> + *
> + */
> +
> +#include <errno.h>
> +#include <stdint.h>
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <stdarg.h>
> +#include <string.h>
> +#include <unistd.h>
> +#include <fcntl.h>
> +#include <time.h>
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <sys/param.h>
> +#include <asm/types.h>
> +#include <dirent.h>
> +#include <mtd/jffs2-user.h>
> +#include <endian.h>
> +#include <byteswap.h>
> +#include <getopt.h>
> +#include "crc32.h"
> +
> +#define CONFIG_JFFS2_SUMMARY
> +#define SUM_TOOL
> +#include "../fs/jffs2/summary.h"
> +
> +
> +#define PAD(x) (((x)+3)&~3)
> +
> +static const char *const app_name = "sumtool";
> +
> +static struct jffs2_sum_info *sum_collected = NULL;/* summary info list */
> +
> +static int verbose = 0;
> +static int padto = 0;						/* pad the output with 0xFF to the end of the final eraseblock */
> +static int add_cleanmarkers = 1;			/* add cleanmarker to output */
> +static int use_input_cleanmarker_size = 1;	/* use input file's cleanmarker size (default) */
> +static int found_cleanmarkers = 0;			/* cleanmarker found in input file */
> +static struct jffs2_unknown_node cleanmarker;
> +static int cleanmarker_size = sizeof(cleanmarker);
> +static const char *short_options = "o:i:e:hvVblnc:p";
> +static int erase_block_size = 65536;
> +static int target_endian = __BYTE_ORDER;
> +static int out_fd = -1;
> +static int in_fd = -1;
> +
> +static uint8_t *data_buffer = NULL; 	/* buffer for inodes */
> +static unsigned int data_ofs = 0;	 	/* inode buffer offset */
> +
> +static uint8_t *file_buffer = NULL;		/* file buffer contains the actual erase block*/
> +static unsigned int file_ofs = 0;		/* position in the buffer */
> +
> +static struct option long_options[] = {
> +	{"output", 1, NULL, 'o'},
> +	{"input", 1, NULL, 'i'},
> +	{"eraseblock", 1, NULL, 'e'},
> +	{"help", 0, NULL, 'h'},
> +	{"verbose", 0, NULL, 'v'},
> +	{"version", 0, NULL, 'V'},
> +	{"bigendian", 0, NULL, 'b'},
> +	{"littleendian", 0, NULL, 'l'},	
> +	{"no-cleanmarkers", 0, NULL, 'n'},
> +	{"cleanmarker", 1, NULL, 'c'},
> +	{"pad", 0, NULL, 'p'},
> +	{NULL, 0, NULL, 0}
> +};
> +
> +static char *helptext =
> +	"Usage: sumtool [OPTIONS] -i inputfile -o outputfile\n"
> +	"Convert the input JFFS2 file to a SUM-ed JFFS2 file\n\n"
> +	"Options:\n"
> +	"  -e, --eraseblock=SIZE     Use erase block size SIZE (default: 64KiB)\n"
> +	"                            (usually 16KiB on NAND)\n"
> +	"  -c, --cleanmarker=SIZE    Size of cleanmarker (default 12).\n"
> +	"                            (usually 16 bytes on NAND, and will be set to\n"
> +	"                            this value if left at the default 12). Will be\n"
> +	"                            stored in OOB after each physical page composing\n"
> +	"                            a physical erase block.\n"
> +	"  -n, --no-cleanmarkers     Don't add a cleanmarker to every eraseblock\n"
> +	"  -o, --output=FILE         Output to FILE \n"
> +	"  -i, --input=FILE          Input from FILE \n"
> +	"  -b, --bigendian           Image is big endian\n"
> +	"  -l  --littleendian        Image is little endian\n"
> +	"  -h, --help                Display this help text\n"
> +	"  -v, --verbose             Verbose operation\n"
> +	"  -V, --version             Display version information\n"
> +	"  -p, --pad                 Pad the OUTPUT with 0xFF to the end of the final\n"
> +	"                            eraseblock\n\n";
> +
> +
> +static char *revtext = "$Revision: 1.3 $";
> +
> +static unsigned char ffbuf[16] = {
> +	0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
> +	0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff
> +};
> +
> +static void verror_msg(const char *s, va_list p) {
> +	fflush(stdout);
> +	fprintf(stderr, "%s: ", app_name);
> +	vfprintf(stderr, s, p);
> +}
> +
> +static void error_msg_and_die(const char *s, ...) {
> +	va_list p;
> +
> +	va_start(p, s);
> +	verror_msg(s, p);
> +	va_end(p);
> +	putc('\n', stderr);
> +	exit(EXIT_FAILURE);
> +}
> +
> +static void vperror_msg(const char *s, va_list p) {
> +	int err = errno;
> +
> +	if (s == 0)
> +		s = "";
> +	verror_msg(s, p);
> +	if (*s)
> +		s = ": ";
> +	fprintf(stderr, "%s%s\n", s, strerror(err));
> +}
> +
> +static void perror_msg_and_die(const char *s, ...) {
> +	va_list p;
> +
> +	va_start(p, s);
> +	vperror_msg(s, p);
> +	va_end(p);
> +	exit(EXIT_FAILURE);
> +}
> +
> +
> +
> +static void full_write(void *target_buff, const void *buf, int len);
> +
> +void setup_cleanmarker() {
> +
> +	cleanmarker.magic    = cpu_to_je16(JFFS2_MAGIC_BITMASK);
> +	cleanmarker.nodetype = cpu_to_je16(JFFS2_NODETYPE_CLEANMARKER);
> +	cleanmarker.totlen   = cpu_to_je32(cleanmarker_size);
> +	cleanmarker.hdr_crc  = cpu_to_je32(crc32(0, &cleanmarker, sizeof(struct jffs2_unknown_node)-4));
> +}
> +
> +void process_options (int argc, char **argv){
> +	int opt,c;
> +	
> +	while ((opt = getopt_long(argc, argv, short_options, long_options, &c)) >= 0) 
> +	{
> +		switch (opt) 
> +		{
> +			case 'o':
> +				if (out_fd != -1) {
> +					error_msg_and_die("output filename specified more than once");
> +				}
> +				out_fd = open(optarg, O_CREAT | O_TRUNC | O_RDWR, 0644);
> +				if (out_fd == -1) {
> +					perror_msg_and_die("open output file");
> +				}
> +				break;
> +				
> +			case 'i':
> +				if (in_fd != -1) {
> +					error_msg_and_die("input filename specified more than once");
> +				}
> +				in_fd = open(optarg, O_RDONLY);
> +				if (in_fd == -1) {
> +					perror_msg_and_die("open input file");
> +				}
> +				break;
> +			case 'b':
> +				target_endian = __BIG_ENDIAN;
> +				break;
> +			case 'l':
> +				target_endian = __LITTLE_ENDIAN;
> +				break;	
> +			case 'h':
> +			case '?':
> +				error_msg_and_die(helptext);
> +	
> +			case 'v':
> +				verbose = 1;
> +				break;
> +	
> +			case 'V':
> +				error_msg_and_die("revision %.*s\n",
> +						(int) strlen(revtext) - 13, revtext + 11);
> +	
> +			case 'e': {
> +				char *next;
> +				unsigned units = 0;
> +				erase_block_size = strtol(optarg, &next, 0);
> +				if (!erase_block_size)
> +					error_msg_and_die("Unrecognisable erase size\n");
> +	
> +				if (*next) {
> +					if (!strcmp(next, "KiB")) {
> +						units = 1024;
> +					} else if (!strcmp(next, "MiB")) {
> +						units = 1024 * 1024;
> +					} else {
> +						error_msg_and_die("Unknown units in erasesize\n");
> +					}
> +				} else {
> +					if (erase_block_size < 0x1000)
> +						units = 1024;
> +					else
> +						units = 1;
> +				}
> +				erase_block_size *= units;
> +	
> +				/* If it's less than 8KiB, they're not allowed */
> +				if (erase_block_size < 0x2000) {
> +					fprintf(stderr, "Erase size 0x%x too small. Increasing to 8KiB minimum\n",
> +						erase_block_size);
> +					erase_block_size = 0x2000;
> +				}
> +				break;
> +			}
> +
> +			case 'n':
> +				add_cleanmarkers = 0;
> +				break;
> +			case 'c':
> +				cleanmarker_size = strtol(optarg, NULL, 0);
> +			
> +				if (cleanmarker_size < sizeof(cleanmarker)) {
> +					error_msg_and_die("cleanmarker size must be >= 12");
> +				}
> +				if (cleanmarker_size >= erase_block_size) {
> +					error_msg_and_die("cleanmarker size must be < eraseblock size");
> +				}
> +				
> +				use_input_cleanmarker_size = 0;
> +				found_cleanmarkers = 1;
> +				setup_cleanmarker();
> +				
> +				break;
> +			case 'p':
> +				padto = 1;
> +				break;
> +			
> +		}
> +	}
> +}
> +
> +
> +void init_buffers() {
> +	
> +	data_buffer = malloc(erase_block_size);
> +	
> +	if (!data_buffer) {
> +		perror("out of memory");
> +		close (in_fd);
> +		close (out_fd);
> +		exit(1);
> +	}
> +		
> +	file_buffer = malloc(erase_block_size);
> +	
> +	if (!file_buffer) {
> +		perror("out of memory");
> +		close (in_fd);
> +		close (out_fd);
> +		exit(1);
> +	}
> +}
> +
> +void init_sumlist(){
> +	
> +	sum_collected = (struct jffs2_sum_info *) malloc (sizeof(struct jffs2_sum_info));
> +	
> +	if (!sum_collected)
> +		error_msg_and_die("Can't allocate memory for jffs2_sum_info!\n");
> +	
> +	sum_collected->sum_list = NULL;
> +	sum_collected->sum_num = 0;
> +	sum_collected->sum_size = 0; 
> +
> +}
> +
> +void clean_buffers() {
> +	
> +	if (data_buffer) 
> +		free(data_buffer);
> +	if (file_buffer)
> +		free(file_buffer);
> +}
> +
> +void clean_sumlist() {
> +	
> +	union jffs2_sum_mem *temp;
> +	
> +	if(sum_collected){
> +		
> +		while(sum_collected->sum_list){
> +			temp = sum_collected->sum_list;
> +			sum_collected->sum_list = sum_collected->sum_list->u.next;
> +			free(temp);
> +			sum_collected->sum_num--;
> +		}
> +		
> +		if(sum_collected->sum_num != 0)
> +			printf("Ooops, something wrong happened! sum_num != 0, but sum_list = null ???");
> +		
> +		free(sum_collected);
> +	}	
> +}
> +
> +int load_next_block() {
> +	
> +	int ret;
> +	ret = read(in_fd, file_buffer, erase_block_size);
> +	file_ofs = 0;
> +	
> +	if(verbose)
> +		printf("Load next block : %d bytes read\n",ret);
> +	
> +	return ret;
> +}
> +
> +void write_buff_to_file() {
> +	
> +	int ret;
> +	int len = data_ofs;
> +	
> +	uint8_t *buf = NULL;
> +	
> +	buf = data_buffer;
> +	while (len > 0) {
> +		ret = write(out_fd, buf, len);
> +	
> +		if (ret < 0)
> +			perror_msg_and_die("write");
> +
> +		if (ret == 0)
> +			perror_msg_and_die("write returned zero");
> +	
> +		len -= ret;
> +		buf += ret;
> +	}
> +	
> +	data_ofs = 0;
> +}
> +
> +void dump_sum_records() {
> +	
> +    struct jffs2_summary_node isum;
> +    union jffs2_sum_mem *temp;
> +	jint32_t offset;
> +	jint32_t *wpage;
> +	uint8_t *tempptr;
> +	int datasize;
> +	int infosize;
> +	int padsize;
> +	jint32_t magic = cpu_to_je32(JFFS2_SUM_MAGIC);
> +	
> +	if (!sum_collected->sum_num || !sum_collected->sum_list) 
> +		return; 
> +	
> +	datasize = sum_collected->sum_size + sizeof(struct jffs2_sum_marker);
> +	infosize = sizeof(struct jffs2_summary_node) + datasize;
> +	padsize = erase_block_size - data_ofs - infosize;
> +	infosize += padsize; datasize += padsize;
> +	offset = cpu_to_je32(data_ofs);
> +	
> +	jint32_t *tpage = (jint32_t *) malloc(datasize);
> +	
> +	if(!tpage)
> +		error_msg_and_die("Can't allocate memory to dump summary information!\n");
> +	
> +	memset(tpage, 0xff, datasize);
> +	memset(&isum, 0, sizeof(isum));
> +	
> +	isum.magic = cpu_to_je16(JFFS2_MAGIC_BITMASK);
> +	isum.nodetype = cpu_to_je16(JFFS2_NODETYPE_SUMMARY);
> +	isum.totlen = cpu_to_je32(infosize);
> +	isum.hdr_crc = cpu_to_je32(crc32(0, &isum, sizeof(struct jffs2_unknown_node) - 4));
> +        isum.padded = cpu_to_je32(0);
> +		
> +	if (add_cleanmarkers && found_cleanmarkers) {
> +		isum.cln_mkr = cpu_to_je32(cleanmarker_size);	
> +	}
> +	else{
> +		isum.cln_mkr = cpu_to_je32(0);
> +	}
> +	
> +	isum.sum_num = cpu_to_je16(sum_collected->sum_num);
> +	wpage = tpage;
> +	
> +		
> +	while (sum_collected->sum_num) {
> +		
> +
> +		switch(je16_to_cpu(sum_collected->sum_list->u.nodetype)){
> +			
> +			case JFFS2_NODETYPE_INODE : {
> +				jint16_t *temp16ptr = (jint16_t *)wpage;
> +				
> +				*(temp16ptr++) = sum_collected->sum_list->i.nodetype;
> +				wpage = (jint32_t *) temp16ptr;
> +				
> +				*(wpage++) = sum_collected->sum_list->i.inode;
> +				*(wpage++) = sum_collected->sum_list->i.version;
> +				*(wpage++) = sum_collected->sum_list->i.offset;
> +				*(wpage++) = sum_collected->sum_list->i.totlen;
> +				break;
> +			}
> +			
> +			case JFFS2_NODETYPE_DIRENT : {
> +				jint16_t *temp16ptr = (jint16_t *) wpage;
> +				uint8_t *temp8ptr = NULL;
> +				
> +				*(temp16ptr++) = sum_collected->sum_list->d.nodetype;
> +				wpage = (jint32_t *) temp16ptr;
> +				
> +				*(wpage++) = sum_collected->sum_list->d.totlen;
> +				*(wpage++) = sum_collected->sum_list->d.offset;
> +				*(wpage++) = sum_collected->sum_list->d.pino;
> +				*(wpage++) = sum_collected->sum_list->d.version;
> +				*(wpage++) = sum_collected->sum_list->d.ino;
> +				
> +				temp8ptr = (uint8_t *) wpage;
> +				*(temp8ptr++) = sum_collected->sum_list->d.nsize;
> +				*(temp8ptr++) = sum_collected->sum_list->d.type;
> +				
> +				memcpy(temp8ptr,sum_collected->sum_list->d.name,sum_collected->sum_list->d.nsize);
> +				temp8ptr += sum_collected->sum_list->d.nsize;
> +				wpage = (jint32_t *) temp8ptr;
> +				
> +				break;
> +			}
> +			
> +			default : {
> +				printf("This is not good for me!!!!\n");
> +			}
> +		}
> +		
> +		temp = sum_collected->sum_list;
> +		sum_collected->sum_list = sum_collected->sum_list->u.next;
> +		free(temp);
> +		
> +		sum_collected->sum_num--;
> +	}
> +	
> +	sum_collected->sum_size = 0;
> +	
> +	tempptr = (uint8_t *) wpage;
> +	tempptr += padsize;
> +	wpage = (jint32_t *) tempptr;
> +	
> +	*(wpage++) = offset;
> +	*(wpage++) = cpu_to_je32(erase_block_size);
> +	*(wpage++) = magic;
> +	isum.sum_crc = cpu_to_je32(crc32(0, tpage, datasize));
> +	isum.node_crc = cpu_to_je32(crc32(0, &isum, sizeof(isum) - 8));
> +	
> +	full_write(data_buffer + data_ofs, &isum, sizeof(isum));
> +	full_write(data_buffer + data_ofs, tpage, datasize);
> +	
> +	free(tpage);
> +
> +}
> +
> +static void full_write(void *target_buff, const void *buf, int len) {
> +	
> +	memcpy(target_buff, buf, len);
> +	data_ofs += len;
> +}
> +
> +static void pad(int req) {
> +	while (req) {
> +		if (req > sizeof(ffbuf)) {
> +			full_write(data_buffer + data_ofs, ffbuf, sizeof(ffbuf));
> +			req -= sizeof(ffbuf);
> +		} else {
> +			full_write(data_buffer + data_ofs, ffbuf, req);
> +			req = 0;
> +		}
> +	}
> +} 
> +
> +static inline void padword() {
> +	
> +	if (data_ofs % 4) {
> +		full_write(data_buffer + data_ofs, ffbuf, 4 - (data_ofs % 4));
> +	}
> +} 
> +
> +
> +static inline void pad_block_if_less_than(int req,int plus) {
> +	
> +    int datasize = req + plus + sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8;
> +    datasize += (4 - (datasize % 4)) % 4;
> +	
> +    if (data_ofs + req > erase_block_size - datasize) {
> +        dump_sum_records();
> +		write_buff_to_file();
> +    }
> +		
> +	if (add_cleanmarkers && found_cleanmarkers) {
> +		if (!data_ofs) {
> +			full_write(data_buffer, &cleanmarker, sizeof(cleanmarker));
> +			pad(cleanmarker_size - sizeof(cleanmarker));
> +			padword();
> +		}
> +	}
> +}
> +
> +void flush_buffers() {
> +	
> +	if ((add_cleanmarkers == 1) && (found_cleanmarkers == 1)) { /* CLEANMARKER */
> +		if (data_ofs != cleanmarker_size) {	/* INODE BUFFER */
> +			
> +		    int datasize = sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8;
> +		    datasize += (4 - (datasize % 4)) % 4;
> +			
> +			/* If we have a full inode buffer, then write out inode and summary data  */
> +		    if (data_ofs + sizeof(struct jffs2_raw_inode) + 2*JFFS2_MIN_DATA_LEN > erase_block_size - datasize) {
> +		        dump_sum_records();
> +				write_buff_to_file();
> +		    }
> +			/* else just write out inode data */
> +			else{
> +				if(padto)
> +					pad(erase_block_size - data_ofs);
> +				write_buff_to_file();
> +			}
> +		}
> +	}
> +	else { /* NO CLEANMARKER */
> +		if (data_ofs != 0) { /* INODE BUFFER */
> +			
> +		    int datasize = sum_collected->sum_size + sizeof(struct jffs2_summary_node) + 8;
> +		    datasize += (4 - (datasize % 4)) % 4;
> +			
> +			/* If we have a full inode buffer, then write out inode and summary data */
> +		    if (data_ofs + sizeof(struct jffs2_raw_inode) + 2*JFFS2_MIN_DATA_LEN > erase_block_size - datasize) {
> +		        dump_sum_records();
> +				write_buff_to_file();
> +		    }
> +			/* Else just write out inode data */
> +			else{
> +				if(padto)
> +					pad(erase_block_size - data_ofs);
> +				write_buff_to_file();
> +			}
> +		}
> +	}
> +}
> +
> +int add_sum_mem(union jffs2_sum_mem *item) {
> +	
> +	union jffs2_sum_mem *walk;
> +	
> +	if (!sum_collected->sum_list) {
> +		sum_collected->sum_list = (union jffs2_sum_mem *) item;
> +	} 
> +	else {
> +		walk = sum_collected->sum_list;
> +		
> +		while (walk->u.next) {
> +			walk = walk->u.next;
> +		}
> +		walk->u.next = (union jffs2_sum_mem *) item;
> +	}
> +	switch (je16_to_cpu(item->u.nodetype)) {
> +    	case JFFS2_NODETYPE_INODE:
> +			sum_collected->sum_size += JFFS2_SUMMARY_INODE_SIZE;
> +			sum_collected->sum_num++;
> +			break;
> +		
> +    	case JFFS2_NODETYPE_DIRENT:
> +			sum_collected->sum_size += JFFS2_SUMMARY_DIRENT_SIZE(item->d.nsize);
> +			sum_collected->sum_num++;
> +			break;
> +		
> +	    default:
> +			error_msg_and_die("__jffs2_add_sum_mem(): UNKNOWN node type %d\n", je16_to_cpu(item->u.nodetype));
> +	}
> +	return 0;
> +}
> +
> +void add_sum_inode_mem(union jffs2_node_union *node) {
> +	
> +	struct jffs2_sum_inode_mem *temp = (struct jffs2_sum_inode_mem *) malloc(sizeof(struct jffs2_sum_inode_mem));
> +	
> +	if (!temp)
> +		error_msg_and_die("Can't allocate memory for summary information!\n");
> +	
> +	temp->nodetype = node->i.nodetype;
> +	temp->inode = node->i.ino;
> +	temp->version = node->i.version;
> +	temp->offset = cpu_to_je32(data_ofs); 
> +	temp->totlen = node->i.totlen;
> +	temp->next = NULL;
> +
> +	add_sum_mem((union jffs2_sum_mem *) temp);
> +	
> +}
> +
> +void add_sum_dirent_mem(union jffs2_node_union *node) {
> +	
> +	struct jffs2_sum_dirent_mem *temp = (struct jffs2_sum_dirent_mem *) 
> +			malloc(sizeof(struct jffs2_sum_dirent_mem) + node->d.nsize);
> +	
> +	if (!temp)
> +		error_msg_and_die("Can't allocate memory for summary information!\n");
> +	
> +	temp->nodetype = node->d.nodetype;
> +	temp->totlen = node->d.totlen;
> +	temp->offset = cpu_to_je32(data_ofs);
> +	temp->pino = node->d.pino;
> +	temp->version = node->d.version;
> +	temp->ino = node->d.ino;
> +	temp->nsize = node->d.nsize;
> +	temp->type = node->d.type;
> +	temp->next = NULL;
> +	
> +	memcpy(temp->name,node->d.name,node->d.nsize);
> +
> +	add_sum_mem((union jffs2_sum_mem *) temp);
> +	
> +}
> +
> +void write_dirent_to_buff(union jffs2_node_union *node) {
> +	
> +	pad_block_if_less_than(je32_to_cpu (node->d.totlen),JFFS2_SUMMARY_DIRENT_SIZE(node->d.nsize));
> +	add_sum_dirent_mem(node);
> +	full_write(data_buffer + data_ofs, &(node->d), je32_to_cpu (node->d.totlen));
> +	padword();	
> +}
> +
> +
> +void write_inode_to_buff(union jffs2_node_union *node) {
> +	
> +	pad_block_if_less_than(je32_to_cpu (node->i.totlen),JFFS2_SUMMARY_INODE_SIZE);  
> +	add_sum_inode_mem(node);	/* Add inode summary mem to summary list */
> +	full_write(data_buffer + data_ofs, &(node->i), je32_to_cpu (node->i.totlen));	/* Write out the inode to inode_buffer */
> +	padword();
> +	
> +}
> +
> +
> +void create_summed_image(int inp_size) {
> +	uint8_t		*p = file_buffer;
> +	union jffs2_node_union 	*node;
> +	uint32_t	crc;
> +	uint16_t	type;
> +	int		bitchbitmask = 0;
> +	int		obsolete;
> +	
> +	char	name[256];
> +	
> +	while ( p < (file_buffer + inp_size)) {
> +		
> +		node = (union jffs2_node_union *) p;
> +		
> +		/* Skip empty space */
> +		if (je16_to_cpu (node->u.magic) == 0xFFFF && je16_to_cpu (node->u.nodetype) == 0xFFFF) {
> +			p += 4;
> +			continue;
> +		}
> +		
> +		if (je16_to_cpu (node->u.magic) != JFFS2_MAGIC_BITMASK)	{
> +			if (!bitchbitmask++)
> +    			    printf ("Wrong bitmask  at  0x%08x, 0x%04x\n", p - file_buffer, je16_to_cpu (node->u.magic));
> +			p += 4;
> +			continue;
> +		}
> +		
> +		bitchbitmask = 0;
> +		
> +		type = je16_to_cpu(node->u.nodetype);
> +		if ((type & JFFS2_NODE_ACCURATE) != JFFS2_NODE_ACCURATE) {
> +			obsolete = 1;
> +			type |= JFFS2_NODE_ACCURATE;
> +		} else
> +			obsolete = 0;
> +		
> +		node->u.nodetype = cpu_to_je16(type);
> +	    
> +		crc = crc32 (0, node, sizeof (struct jffs2_unknown_node) - 4);
> +		if (crc != je32_to_cpu (node->u.hdr_crc)) {
> +			printf ("Wrong hdr_crc  at  0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->u.hdr_crc), crc);
> +			p += 4;
> +			continue;
> +		}
> +		
> +		switch(je16_to_cpu(node->u.nodetype)) {
> +		
> +			case JFFS2_NODETYPE_INODE:
> +				if(verbose)
> +					printf ("%8s Inode      node at 0x%08x, totlen 0x%08x, #ino  %5d, version %5d, isize %8d, csize %8d, dsize %8d, offset %8d\n",
> +						obsolete ? "Obsolete" : "",
> +						p - file_buffer, je32_to_cpu (node->i.totlen), je32_to_cpu (node->i.ino),
> +						je32_to_cpu ( node->i.version), je32_to_cpu (node->i.isize), 
> +						je32_to_cpu (node->i.csize), je32_to_cpu (node->i.dsize), je32_to_cpu (node->i.offset));
> +	
> +				crc = crc32 (0, node, sizeof (struct jffs2_raw_inode) - 8);
> +				if (crc != je32_to_cpu (node->i.node_crc)) {
> +					printf ("Wrong node_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.node_crc), crc);
> +					p += PAD(je32_to_cpu (node->i.totlen));
> +					continue;
> +				}
> +				
> +				crc = crc32(0, p + sizeof (struct jffs2_raw_inode), je32_to_cpu(node->i.csize));
> +				if (crc != je32_to_cpu(node->i.data_crc)) {
> +					printf ("Wrong data_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->i.data_crc), crc);
> +					p += PAD(je32_to_cpu (node->i.totlen));
> +					continue;
> +				}
> +				
> +				write_inode_to_buff(node);
> +				
> +				p += PAD(je32_to_cpu (node->i.totlen));
> +				break;
> +				
> +			case JFFS2_NODETYPE_DIRENT:
> +				memcpy (name, node->d.name, node->d.nsize);
> +				name [node->d.nsize] = 0x0;
> +			
> +				if(verbose)
> +					printf ("%8s Dirent     node at 0x%08x, totlen 0x%08x, #pino %5d, version %5d, #ino  %8d, nsize %8d, name %s\n",
> +						obsolete ? "Obsolete" : "",
> +						p - file_buffer, je32_to_cpu (node->d.totlen), je32_to_cpu (node->d.pino),
> +						je32_to_cpu ( node->d.version), je32_to_cpu (node->d.ino), 
> +						node->d.nsize, name);
> +	
> +				crc = crc32 (0, node, sizeof (struct jffs2_raw_dirent) - 8);
> +				if (crc != je32_to_cpu (node->d.node_crc)) {
> +					printf ("Wrong node_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.node_crc), crc);
> +					p += PAD(je32_to_cpu (node->d.totlen));
> +					continue;
> +				}
> +				
> +				crc = crc32(0, p + sizeof (struct jffs2_raw_dirent), node->d.nsize);
> +				if (crc != je32_to_cpu(node->d.name_crc)) {
> +					printf ("Wrong name_crc at  0x%08x, 0x%08x instead of 0x%08x\n", p - file_buffer, je32_to_cpu (node->d.name_crc), crc);
> +					p += PAD(je32_to_cpu (node->d.totlen));
> +					continue;
> +				}
> +	
> +				write_dirent_to_buff(node);
> +				
> +				p += PAD(je32_to_cpu (node->d.totlen));						
> +				break;
> +		
> +			case JFFS2_NODETYPE_CLEANMARKER:
> +				if (verbose) {
> +					printf ("%8s Cleanmarker     at 0x%08x, totlen 0x%08x\n", 
> +						obsolete ? "Obsolete" : "",
> +						p - file_buffer, je32_to_cpu (node->u.totlen));
> +				}
> +				
> +				if(!found_cleanmarkers){
> +					found_cleanmarkers = 1;
> +					
> +					if(add_cleanmarkers == 1 && use_input_cleanmarker_size == 1){
> +						cleanmarker_size = je32_to_cpu (node->u.totlen);
> +						setup_cleanmarker();
> +					}
> +				}			
> +				
> +				p += PAD(je32_to_cpu (node->u.totlen));						
> +				break;
> +		
> +			case JFFS2_NODETYPE_PADDING:
> +				if (verbose) {
> +					printf ("%8s Padding    node at 0x%08x, totlen 0x%08x\n", 
> +						obsolete ? "Obsolete" : "",
> +						p - file_buffer, je32_to_cpu (node->u.totlen));
> +				}		
> +				p += PAD(je32_to_cpu (node->u.totlen));						
> +				break;
> +				
> +			case 0xffff:
> +				p += 4;
> +				break;
> +				
> +			default:	
> +				if (verbose) {
> +					printf ("%8s Unknown    node at 0x%08x, totlen 0x%08x\n", 
> +						obsolete ? "Obsolete" : "",
> +						p - file_buffer, je32_to_cpu (node->u.totlen));
> +				}
> +				
> +				//write_unknown_to_buff(node);
> +				
> +				p += PAD(je32_to_cpu (node->u.totlen));						
> +		}	
> +	}
> +}
> +
> +int main(int argc, char **argv) {
> +	
> +	int ret;
> +	
> +	process_options(argc,argv);
> +	
> +	if ((in_fd == -1) || (out_fd == -1))	{
> +		
> +		if(in_fd != -1)
> +			close(in_fd);
> +		if(out_fd != -1)
> +			close(out_fd);
> +		
> +		error_msg_and_die("You must specify input and output files!\n");
> +	}
> +	
> +	init_buffers();
> +	init_sumlist();
> +	
> +	while ((ret = load_next_block())) {
> +		create_summed_image(ret);	
> +	}
> +
> +	flush_buffers();
> +	clean_buffers();
> +	clean_sumlist();
> +	
> +	if (in_fd != -1)
> +		close(in_fd);
> +	if (out_fd != -1)
> +		close(out_fd);
> +	
> +	return 0;
> +}
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: JFFS2 mount time

[Ferenc Havasi]

Hi Gareth,

> I'm afraid this patch seems to be quite a long way from applying cleanly
> to my source. I'm using 2.6.9 .. at least half a dozen failures or
> more ...

It works with JFFS2 in the latest MTD CVS.

JFFS3 will contains this feature soon.

Regards,
Ferenc

Re: JFFS2 mount time

[Gareth Bult (Encryptec)]

[-- Attachment #1: Type: text/plain, Size: 656 bytes --]

Yup,

I've cvs'd the latest source and patched ... kernel seems to build Ok ..
Just trying a full build from clean to make sure, then I'll give it a
go ..

Is there an example of how to use sumtool anywhere ?

If I eraseall the flash then mount JFFS2 and copy files on, do I need to
use sumtool ?

Gareth.

On Mon, 2004-12-20 at 17:09 +0100, Ferenc Havasi wrote:

> Hi Gareth,
> 
> > I'm afraid this patch seems to be quite a long way from applying cleanly
> > to my source. I'm using 2.6.9 .. at least half a dozen failures or
> > more ...
> 
> It works with JFFS2 in the latest MTD CVS.
> 
> JFFS3 will contains this feature soon.
> 
> Regards,
> Ferenc

[-- Attachment #2: Type: text/html, Size: 1194 bytes --]

Re: JFFS2 mount time

[Gareth Bult (Encryptec)]

Hi,

Patches applied and it seems to be working ..

After dd'ing a live FS off the key, sumtool'ing it and dd'ing it back
on, it seems to have cut mount time from 31 seconds to 11 seconds ...

This is certainly much better .. any way of making it quicker .. ;-)

Gareth.


On Mon, 2004-12-20 at 17:09 +0100, Ferenc Havasi wrote:
> Hi Gareth,
> 
> > I'm afraid this patch seems to be quite a long way from applying cleanly
> > to my source. I'm using 2.6.9 .. at least half a dozen failures or
> > more ...
> 
> It works with JFFS2 in the latest MTD CVS.
> 
> JFFS3 will contains this feature soon.
> 
> Regards,
> Ferenc

JFFS2 mount time

[Jarkko Lavinen (NMP/Helsinki)]

I am working with an embedded device with 64MB nand chip. I am able to
read from chip at 6MB/s and able to read the whole chip in 10 seconds
using character mtd device. The device has Omap1610 running at 190MHz,
kernel 2.4.21.

With 80% file system usage the mount time is 8 seconds which is
unacceptable for root file system on a handheld device. I assume the
scan time is linear with respect to the size of the flash partition
and the file system usage.

Is there anything that could be done to reduce the mount time of large
JFFS2 partitions?  Lazy mounting with prioritized scanning?

Jarkko Lavinen

RE: JFFS2 mount time

[Joakim Tjernlund]

> I am working with an embedded device with 64MB nand chip. I am able to
> read from chip at 6MB/s and able to read the whole chip in 10 seconds
> using character mtd device. The device has Omap1610 running at 190MHz,
> kernel 2.4.21.
> 
> With 80% file system usage the mount time is 8 seconds which is
> unacceptable for root file system on a handheld device. I assume the
> scan time is linear with respect to the size of the flash partition
> and the file system usage.
> 
> Is there anything that could be done to reduce the mount time of large
> JFFS2 partitions?  Lazy mounting with prioritized scanning?

Yes, it is already done in JFFS2 CVS. Mount time for me is about a second on
a 64MB JFFS2 NOR flash.

      Jocke

Re: JFFS2 mount time

[David Woodhouse]

On Tue, 2004-01-13 at 14:50 +0200, Jarkko Lavinen (NMP/Helsinki) wrote:
> Is there anything that could be done to reduce the mount time of large
> JFFS2 partitions?  Lazy mounting with prioritized scanning?

We improved it a lot by not checking CRC32 on all nodes until later --
but we still need to read the whole medium at mount time.

We _can_ fix that. There's a limited amount of knowledge we're gaining
from our scan at mount time. For each inode on the medium, we need:
		Its nlink
		A list of the physical addresses of all its nodes

We currently read the whole of every eraseblock to work this out.
However, we could write a block 'tailer' at the _end_ of each eraseblock
as we fill it, containing enough information in compressed form about
the contents of the eraseblock.

That would mean that we only have to read a small proportion of each
properly-finished eraseblock, and should speed up the mount
significantly.

-- 
dwmw2

Re: JFFS2 mount time

[Kenneth Johansson]

On Tue, 2004-01-13 at 14:39, David Woodhouse wrote:
> On Tue, 2004-01-13 at 14:50 +0200, Jarkko Lavinen (NMP/Helsinki) wrote:
> > Is there anything that could be done to reduce the mount time of large
> > JFFS2 partitions?  Lazy mounting with prioritized scanning?
> 
> We improved it a lot by not checking CRC32 on all nodes until later --
> but we still need to read the whole medium at mount time.
> 
> We _can_ fix that. There's a limited amount of knowledge we're gaining
> from our scan at mount time. For each inode on the medium, we need:
> 		Its nlink
> 		A list of the physical addresses of all its nodes
> 
> We currently read the whole of every eraseblock to work this out.
> However, we could write a block 'tailer' at the _end_ of each eraseblock
> as we fill it, containing enough information in compressed form about
> the contents of the eraseblock.
> 
> That would mean that we only have to read a small proportion of each
> properly-finished eraseblock, and should speed up the mount
> significantly.

I have done some work on the JFFS2 code in u-boot and found that the
fastest way was to load the entire FLASH into a continuous disk image in
DRAM and then have code that directly dereference pointers into this
image without any access functions or read on demand.

When I tried to be smart and skip the copy and only read in what needed
to be read that was significantly slower.

Never underestimate brute force :)

If you have very very poor bandwidth from the flash the the reverse is
true but I had to slow down my flash 10 times to break even. 

Re: JFFS2 mount time

[David Woodhouse]

On Tue, 2004-01-13 at 16:25 +0100, Kenneth Johansson wrote:
> I have done some work on the JFFS2 code in u-boot and found that the
> fastest way was to load the entire FLASH into a continuous disk image in
> DRAM and then have code that directly dereference pointers into this
> image without any access functions or read on demand.

What we do in the current JFFS2 code is similar. We directly dereference
pointers into the _flash_ without access functions, rather than copying
it into DRAM first.

> When I tried to be smart and skip the copy and only read in what needed
> to be read that was significantly slower.

Bear in mind that the question was about NAND flash though -- it's not
directly deferenceable, and you're not going to get the same results.

-- 
dwmw2

Re: JFFS2 mount time

[Kenneth Johansson]

On Tue, 2004-01-13 at 16:30, David Woodhouse wrote:

> > When I tried to be smart and skip the copy and only read in what needed
> > to be read that was significantly slower.
> 
> Bear in mind that the question was about NAND flash though -- it's not
> directly deferenceable, and you're not going to get the same results.

hmm I reread the firt post why is read so slow?? When I thought about
using nand I read that reading a byte took something like 50 ns. That is
like 20MB per second for an 8 bit device. This is in the region where it
did not pay to be smart,the accesses overhead to do read only what's
needed is greater than the time wasted reading uneccessary data. 

Re: JFFS2 mount time

[Jörn Engel]

On Tue, 13 January 2004 17:21:15 +0100, Kenneth Johansson wrote:
> 
> hmm I reread the firt post why is read so slow?? When I thought about
> using nand I read that reading a byte took something like 50 ns. That is
> like 20MB per second for an 8 bit device. This is in the region where it
> did not pay to be smart,the accesses overhead to do read only what's
> needed is greater than the time wasted reading uneccessary data. 

But on nand it is not only possible, but also likely that flash size
is greater than ram size.  Does your approach still work then?

Jörn

-- 
To announce that there must be no criticism of the President, or that we
are to stand by the President, right or wrong, is not only unpatriotic
and servile, but is morally treasonable to the American public.
-- Theodore Roosevelt, Kansas City Star, 1918

Re: JFFS2 mount time

[Kenneth Johansson]

On Tue, 2004-01-13 at 18:29, Jörn Engel wrote:
> On Tue, 13 January 2004 17:21:15 +0100, Kenneth Johansson wrote:
> > 
> > hmm I reread the firt post why is read so slow?? When I thought about
> > using nand I read that reading a byte took something like 50 ns. That is
> > like 20MB per second for an 8 bit device. This is in the region where it
> > did not pay to be smart,the accesses overhead to do read only what's
> > needed is greater than the time wasted reading uneccessary data. 
> 
> But on nand it is not only possible, but also likely that flash size
> is greater than ram size.  Does your approach still work then?
> 
> Jörn

Obviously no. But maybe it's still cheaper to do the work in big chunks
I don't know you need to actually do a test to see what happens. It's
not so easy to see in advance what will happen. I thought for sure that
it would be faster to only read what was needed but it was not, at least
not on a 200 MHz ppc if you have an P4 3GHz the picture is quite
different.

Re: JFFS2 mount time

[Jörn Engel]

On Tue, 13 January 2004 18:40:09 +0100, Kenneth Johansson wrote:
> > 
> > But on nand it is not only possible, but also likely that flash size
> > is greater than ram size.  Does your approach still work then?
> 
> Obviously no. But maybe it's still cheaper to do the work in big chunks
> I don't know you need to actually do a test to see what happens. It's
> not so easy to see in advance what will happen. I thought for sure that
> it would be faster to only read what was needed but it was not, at least
> not on a 200 MHz ppc if you have an P4 3GHz the picture is quite
> different.

Could be, at least.  As long as we don't know where the time is lost,
everything else remains guesswork.

Anyway, David's approach sounds quite sane to me.

Jörn

-- 
A victorious army first wins and then seeks battle.
-- Sun Tzu

Re: JFFS2 mount time

[Steven Scholz]

Hi,

> Is there anything that could be done to reduce the mount time of large
> JFFS2 partitions?  Lazy mounting with prioritized scanning?

I wonder if the current CVS head is much better (i.e. faster) in mounting a 
JFFS2 root fs then linux-2.6.10?

Or in other words: does it make sense to use the current CVS instead of the code 
in linux-2.6.10?

--
Steven

Re: JFFS2 mount time

[Artem B. Bityuckiy]

Hi

Try JFFS3 with "summaries" enabled or Ferenc Havasi's JFFS2 "summary" 
patch. The mount speed should be increased with it.

Note: JFFS3 development is in progress so be ready if it has bugs.

Steven Scholz wrote:
> Hi,
> 
>> Is there anything that could be done to reduce the mount time of large
>> JFFS2 partitions?  Lazy mounting with prioritized scanning?
> 
> 
> I wonder if the current CVS head is much better (i.e. faster) in 
> mounting a JFFS2 root fs then linux-2.6.10?
> 
> Or in other words: does it make sense to use the current CVS instead of 
> the code in linux-2.6.10?
> 
> -- 
> Steven
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 

-- 
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: JFFS2 mount time

[Ferenc Havasi]

Hi Steven,

> Try JFFS3 with "summaries" enabled or Ferenc Havasi's JFFS2 "summary" 
> patch. The mount speed should be increased with it.
> 
> Note: JFFS3 development is in progress so be ready if it has bugs.

Becauseof it, we maintain a summary patch for JFFS2. You can download 
the latest patch from http://www.inf.u-szeged.hu/jffs2/mount.php
(I'm updating the page now, so you may wait a little.)

Summary designed specially to speed up the mounttime, so it should help you.

Regards,
Ferenc

Re: JFFS2 mount time

[Steven Scholz]

Hi Ferenc,

>> Note: JFFS3 development is in progress so be ready if it has bugs.
> 
> Becauseof it, we maintain a summary patch for JFFS2. You can download 
> the latest patch from http://www.inf.u-szeged.hu/jffs2/mount.php
> (I'm updating the page now, so you may wait a little.)
> 
> Summary designed specially to speed up the mounttime, so it should help 
> you.

Thanks a million. I'll try that. I guess I can just use your Patch on a vanilla 
linux-2.6.10!?

BTW: On the link mentioned above I read "Speed up the mount time of JFFS2 - 
specially when it uses NAND flash. " How about NOR flash?

--
Steven

Re: JFFS2 mount time

[Artem B. Bityuckiy]

> Thanks a million. I'll try that. I guess I can just use your Patch on a 
> vanilla linux-2.6.10!?
> 
> BTW: On the link mentioned above I read "Speed up the mount time of 
> JFFS2 - specially when it uses NAND flash. " How about NOR flash?
> 
My understanding of the problem is that summary may help in case of NOR, 
but in
general it shouldn't increase the mount speed a lot (am I wrong?). If we 
change
the patch a bit (asking not to check any CRC in the block has summary) 
we may
have better mount speed (and iget()). And possibly in case of nor we 
even doesn't
need to store any data in summary - only header. Header will show us 
that the block
was successfully written (no unclean reboots) and we my trust it and do 
not check CRC.

Moreover, looking through the patch I suspect it wasn't tested with NOR, 
is it?


-- 
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: JFFS2 mount time

[Ferenc Havasi]

Hi Steven,

>> Thanks a million. I'll try that. I guess I can just use your Patch on 
>> a vanilla linux-2.6.10!?
>>
>> BTW: On the link mentioned above I read "Speed up the mount time of 
>> JFFS2 - specially when it uses NAND flash. " How about NOR flash?
>>
> My understanding of the problem is that summary may help in case of NOR, 
> but in
> general it shouldn't increase the mount speed a lot (am I wrong?). 

Artem is right.

> Moreover, looking through the patch I suspect it wasn't tested with NOR, 
> is it?

It should work with NOR, too. Now we tested it again, and noticed some 
problems. One of them are "general", and an other is NOR specific. We 
will fix it soon, so you may wait a little before testing it.

 > If we change
 > the patch a bit (asking not to check any CRC in the block has summary)
 > we may
 > have better mount speed (and iget()). And possibly in case of nor we
 > even doesn't
 > need to store any data in summary - only header. Header will show us
 > that the block
 > was successfully written (no unclean reboots) and we my trust it and
 > do not check CRC.

Yes, if everything will be stable we should start to make it more optimal.

Bye,
Ferenc

Re: JFFS2 mount time

[Steven Scholz]

Hi,


>> Thanks a million. I'll try that. I guess I can just use your Patch on 
>> a vanilla linux-2.6.10!?
>>
>> BTW: On the link mentioned above I read "Speed up the mount time of 
>> JFFS2 - specially when it uses NAND flash. " How about NOR flash?
>>
> My understanding of the problem is that summary may help in case of NOR, 
> but in
> general it shouldn't increase the mount speed a lot (am I wrong?).

As I wrote in another thread the summary patch reduced the mount time from 6 
seconds to less than 1 second on my embedded ARM9 system with 8MB NAND flash.
But now GC still slows down the start up of user applications as it checks all 
CRCs right after booting.

> If we change
> the patch a bit (asking not to check any CRC in the block has summary) 
> we may
> have better mount speed (and iget()). And possibly in case of nor we 
> even doesn't
> need to store any data in summary - only header. Header will show us 
> that the block
> was successfully written (no unclean reboots) and we my trust it and do 
> not check CRC.

Makes sense to me.
Ferenc, what do you think about that?
No need to check CRC if we have a summary, right?
This could fix my GC problem mentioned above and in the thread
http://lists.infradead.org/pipermail/linux-mtd/2005-January/011411.html

--
Steven

Re: JFFS2 mount time

[Steven Scholz]

Steven Scholz wrote:
> As I wrote in another thread the summary patch reduced the mount time 
> from 6 seconds to less than 1 second on my embedded ARM9 system with 8MB 
> NAND flash.
Of course I meant the good old NOR flash.

--
Steven

Re: JFFS2 mount time

[Ferenc Havasi]

Hi Steven,

> As I wrote in another thread the summary patch reduced the mount time 
> from 6 seconds to less than 1 second on my embedded ARM9 system with 8MB 
> NAND flash.
> But now GC still slows down the start up of user applications as it 
> checks all CRCs right after booting.

>> If we change
>> the patch a bit (asking not to check any CRC in the block has summary) 
>> we may
>> have better mount speed (and iget()). And possibly in case of nor we 
>> even doesn't
>> need to store any data in summary - only header. Header will show us 
>> that the block
>> was successfully written (no unclean reboots) and we my trust it and 
>> do not check CRC.

If I am right CRCs are not only against the effect of unclean reboots 
but also to handle flash errors. On NAND flashes the ECC handles this 
problem but NOR doesn't have any error detection system.

> Ferenc, what do you think about that?
> No need to check CRC if we have a summary, right?

Give us some days to invetigate it more deeply. I think we will be able 
to imrove the summary patch in some way.

Bye,
Ferenc

Re: JFFS2 mount time

[Steven Scholz]

Hi Ferenc,

> If I am right CRCs are not only against the effect of unclean reboots 
> but also to handle flash errors. On NAND flashes the ECC handles this 
> problem but NOR doesn't have any error detection system.
> 
>> Ferenc, what do you think about that?
>> No need to check CRC if we have a summary, right?
> 
> Give us some days to invetigate it more deeply. I think we will be able 
> to imrove the summary patch in some way.

I now get

Eep. Child "libutil.so" (ino #230) of dir ino #5 doesn't exist!
Eep. Child "libthread_db.so" (ino #227) of dir ino #5 doesn't exist!
Eep. Child "libutil.so.1" (ino #231) of dir ino #5 doesn't exist!
Eep. Child "libthread_db-1.0.so" (ino #226) of dir ino #5 doesn't exist!
Eep. Child "libutil-2.3.3.so" (ino #229) of dir ino #5 doesn't exist!
Eep. Child "libthread_db.so.1" (ino #228) of dir ino #5 doesn't exist!
Eep. Child "adjtimex" (ino #237) of dir ino #9 doesn't exist!
Eep. Child "backlight" (ino #238) of dir ino #9 doesn't exist!
Eep. Child "log" (ino #299) of dir ino #13 doesn't exist!
Eep. Child "demo_320_240_16.xml" (ino #234) of dir ino #232 doesn't exist!
Eep. Child "demo.xml" (ino #233) of dir ino #232 doesn't exist!
Eep. Child "fonttest" (ino #235) of dir ino #232 doesn't exist!
VFS: Mounted root (jffs2 filesystem).
Freeing init memory: 72K

while booting! :-(

I am not sure if this has to do with the summary patch. How can I find out?

--
Steven