kmalloc jffs2_do_mount_fs

kmalloc jffs2_do_mount_fs

[Ingo Flaschberger]

Hi

after 3 nights... debugging why my sharp zaurus do not boot from my
initrd.. i found:

kmalloc does not give you more than 128k (only i you "hack" it).
jffs2 with a filesystem about 40Mb need more when mounting..
to fix this.. i have luckily found a patch at the list:
http://lists.infradead.org/pipermail/linux-mtd/2002-December/006577.html

but why doe you not apply it to the source or warn because of this
"jffs2" problem???

thnx & bye, Ingo

Re: kmalloc jffs2_do_mount_fs

[David Woodhouse]

On Fri, 2003-10-10 at 05:13 +0200, Ingo Flaschberger wrote:
> but why doe you not apply it to the source or warn because of this
> "jffs2" problem???

Partly because I'm unconvinced that we want to actually _use_ such a
small erasesize. It means you split far more page writes into two nodes
to avoid crossing block boundaries, and hence waste space.

I suspect the main reason Thomas objected to my original 'min 64KiB' was
because of the 5-erase-block threshold. Since I've just wound that
threshold _up_ on NAND because and it's also based in part on nr_blocks
now, we could perhaps rethink the minimum erasesize. I didn't manage to
tie him down on #mtd yesterday though.

-- 
dwmw2

Re: kmalloc jffs2_do_mount_fs

[Ingo Flaschberger]

Hi

> On Fri, 2003-10-10 at 05:13 +0200, Ingo Flaschberger wrote:
> > but why doe you not apply it to the source or warn because of this
> > "jffs2" problem???
>
> Partly because I'm unconvinced that we want to actually _use_ such a
> small erasesize. It means you split far more page writes into two nodes
> to avoid crossing block boundaries, and hence waste space.
>
> I suspect the main reason Thomas objected to my original 'min 64KiB' was
> because of the 5-erase-block threshold. Since I've just wound that
> threshold _up_ on NAND because and it's also based in part on nr_blocks
> now, we could perhaps rethink the minimum erasesize. I didn't manage to
> tie him down on #mtd yesterday though.

i use 16k erase block size but the jffs2 partition is

Re: kmalloc jffs2_do_mount_fs

[Jörn Engel]

On Fri, 10 October 2003 06:46:21 +0100, David Woodhouse wrote:
> 
> Partly because I'm unconvinced that we want to actually _use_ such a
> small erasesize. It means you split far more page writes into two nodes
> to avoid crossing block boundaries, and hence waste space.
> 
> I suspect the main reason Thomas objected to my original 'min 64KiB' was
> because of the 5-erase-block threshold. Since I've just wound that
> threshold _up_ on NAND because and it's also based in part on nr_blocks
> now, we could perhaps rethink the minimum erasesize. I didn't manage to
> tie him down on #mtd yesterday though.

But this is wrong (TM).  You missed at least one variable in your
equation.  Five erase blocks is quite enough for NAND, as long as you
don't have more blocks total than on a similar NOR device.  The
problems start when you have too many blocks - you need more free
blocks.

This means that you need what I tried to start with one - dynamic 1)
numbers of erase blocks and possibly dynamic block sizes as well.
Anything else may be a decent assumption for one device or another,
but not for all.

1) dynamic for compile time - it should stay static for any given
device.

Jörn

-- 
Fancy algorithms are buggier than simple ones, and they're much harder
to implement. Use simple algorithms as well as simple data structures.
-- Rob Pike

RE: kmalloc jffs2_do_mount_fs

[Dave Ellis]

On Fri, 2003-10-10 at 05:13 +0200, Ingo Flaschberger wrote:
> kmalloc does not give you more than 128k (only i you "hack" it).
> jffs2 with a filesystem about 40Mb need more when mounting..
> to fix this.. i have luckily found a patch at the list:
>
http://lists.infradead.org/pipermail/linux-mtd/2002-December/006577.html

As David pointed out in his reply to my original post, the corresponding
kfree() needs to change to vfree(). I am including an updated patch
that does that. It is clean against current CVS, but I have only tested
it with my code, which is nowhere near current.

I am still using this patch since I have questions about possible
problems
with the virtual blocks. The virtual block size depends on the size of a
kernel
structure that can change and may differ for some architectures. If the
flash
is removable, it may be unusable when moved to a different system. Even
if it is not removable (mine isn't for now) a newer kernel could break
it.

It also complicates building images with mkfs.jffs2, since the block
size depends on the size of the partition. With vmalloc() I can use the
same image for any flash with the same erase size.

Also, with NAND flash, I don't see how to determine the virtual block
size when loading an image. I use U-Boot to erase the flash and load 
the image. It is not clear how the clean markers interact with virtual
blocks or what should it do when one of erase blocks in a virtual block
is bad.

Dave Ellis

diff -Naur kmalloc/build.c vmalloc/build.c
--- kmalloc/build.c	Wed Oct 15 10:03:20 2003
+++ vmalloc/build.c	Wed Oct 15 10:48:34 2003
@@ -286,7 +286,7 @@
 
 	c->free_size = c->flash_size;
 	c->nr_blocks = c->flash_size / c->sector_size;
-	c->blocks = kmalloc(sizeof(struct jffs2_eraseblock) *
c->nr_blocks, GFP_KERNEL);
+	c->blocks = vmalloc(sizeof(struct jffs2_eraseblock) *
c->nr_blocks);
 	if (!c->blocks)
 		return -ENOMEM;
 	for (i=0; i<c->nr_blocks; i++) {
@@ -325,7 +325,7 @@
 		D1(printk(KERN_DEBUG "build_fs failed\n"));
 		jffs2_free_ino_caches(c);
 		jffs2_free_raw_node_refs(c);
-		kfree(c->blocks);
+		vfree(c->blocks);
 		return -EIO;
 	}
 
diff -Naur kmalloc/fs.c vmalloc/fs.c
--- kmalloc/fs.c	Wed Oct 15 10:03:20 2003
+++ vmalloc/fs.c	Wed Oct 15 10:48:34 2003
@@ -444,8 +444,6 @@
 	 * to reduce the memorysize for c->blocks. (kmalloc allows max.
128K allocation)
 	 */
 	blocks = c->flash_size / c->mtd->erasesize;
-	while ((blocks * sizeof (struct jffs2_eraseblock)) > (128 *
1024))
-		blocks >>= 1;
 	
 	c->sector_size = c->flash_size / blocks;
 	if (c->sector_size != c->mtd->erasesize)
@@ -506,7 +504,7 @@
  out_nodes:
 	jffs2_free_ino_caches(c);
 	jffs2_free_raw_node_refs(c);
-	kfree(c->blocks);
+	vfree(c->blocks);
  out_inohash:
 	kfree(c->inocache_list);
  out_wbuf:
diff -Naur kmalloc/super-v24.c vmalloc/super-v24.c
--- kmalloc/super-v24.c	Wed Oct 15 10:03:21 2003
+++ vmalloc/super-v24.c	Wed Oct 15 10:48:34 2003
@@ -89,7 +89,7 @@
 	up(&c->alloc_sem);
 	jffs2_free_ino_caches(c);
 	jffs2_free_raw_node_refs(c);
-	kfree(c->blocks);
+	vfree(c->blocks);
 	jffs2_nand_flash_cleanup(c);
 	kfree(c->inocache_list);
 	if (c->mtd->sync)
diff -Naur kmalloc/super.c vmalloc/super.c
--- kmalloc/super.c	Wed Oct 15 10:03:21 2003
+++ vmalloc/super.c	Wed Oct 15 10:48:34 2003
@@ -264,7 +264,7 @@
 	up(&c->alloc_sem);
 	jffs2_free_ino_caches(c);
 	jffs2_free_raw_node_refs(c);
-	kfree(c->blocks);
+	vfree(c->blocks);
 	jffs2_nand_flash_cleanup(c);
 	kfree(c->inocache_list);
 	if (c->mtd->sync)