U-Boot: using nand write to flash an ubi image, ubi part fails

U-Boot: using nand write to flash an ubi image, ubi part fails

[Lars Michael]

Hello,

Using mkfs.ubifs and ubinize I have created an image with 5 volumes. In Linux I could flash it using "ubiformat /dev/mtd1 -f /root/ubi.img" and attach it using "ubiattach /dev/ubi_ctrl -m 1". It worked.

In production I would prefer to write the image from U-Boot. So in U-Boot I used "nand erase" then "nand write" to flash the image. Then I want to attach by "ubi part" command - but this fails: "UBI error: ubi_init: UBI error: cannot initialize UBI, error -17"

Then I booted Linux (via nfs) and tried to re-format+attach, but now it fails. Format complains about eraseblock contents and missing erase counters: 

ubiformat: warning!: 1099 of 2036 eraseblocks contain non-ubifs data
ubiformat: warning!: only 937 of 2036 eraseblocks have valid erase counter

The ubiattach now reports lots of:
...
UBI error: check_corruption: PEB 0 contains corrupted VID header, and the data does not contain all 0xFF, this may be a non-UBI PEB or a severe VID header corruption which requires manual inspection

UBI warning: process_eb: valid VID header but corrupted EC header at PEB 4
...

I realize that the U-Boot 'nand erase' does not preserve the erase counters. So what would be the correct way to erase and write in U-Boot? And why is the ubiformat not able to correct the problems?

My kernel is 2.6.29 patched with 165 patched from the ubi back port tree.
U-Boot is 2009.08 with ubifs support, but I am not sure if there are relevant patches. The CPU is a v4 ColdFire and 256MB Micron NAND.

Any help is appreciated, thanks. Below are the detailed listing:


-> mtd

device nand0 <NAND>, # parts = 2
 #: name		size		offset		mask_flags
 0: u-boot              0x00100000	0x00000000	0
 1: ubi                 0x0ff00000	0x00100000	0

active partition: nand0,0 - (u-boot) 0x00100000 @ 0x00000000

defaults:
mtdids  : nand0=NAND
mtdparts: mtdparts=NAND:1m(u-boot),-(ubi)

-> nand erase ubi

NAND erase: device 0 offset 0x100000, size 0xff00000
Erasing at 0x100000 --   0% complete.
<cut>
Erasing at 0xfd60000 --  99% complete.
Skipping bad block at  0x0ff80000                                          

Skipping bad block at  0x0ffa0000                                          

Skipping bad block at  0x0ffc0000                                          

Skipping bad block at  0x0ffe0000                                          

OK

Using FEC0 device
TFTP from server 10.28.151.113; our IP address is 10.28.151.112
Filename '/twr5441x/ubi.img'.
Load address: 0x40010000
Loading: *\b#################################################################
	 #################################################################
<cut>
	 #################################################################
	 ########################################################
done
Bytes transferred = 58064896 (3760000 hex)
-> md 40010000 100

40010000: 55424923 01000000 00000000 00000000    UBI#............
40010010: 00000800 00001000 27162ae3 00000000    ........'.*.....
40010020: 00000000 00000000 00000000 00000000    ................
40010030: 00000000 00000000 00000000 7d780a89    ............}x..
40010040: ffffffff ffffffff ffffffff ffffffff    ................
<cut>
400113f0: 00000000 00000000 00000000 00000000    ................
-> nand write ${loadaddr} 100000 ${filesize}

NAND write: device 0 offset 0x100000, size 0x3760000
 58064896 bytes written: OK
-> ubi part ubi

Creating 1 MTD partitions on "nand0":
0x000000100000-0x000010000000 : "mtd=1"
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size:   131072 bytes (128 KiB)
UBI: logical eraseblock size:    126976 bytes
UBI: smallest flash I/O unit:    2048
UBI: VID header offset:          2048 (aligned 2048)
UBI: data offset:                4096
UBI warning: process_lvol: volume table copy #1 is corrupted
UBI: create volume table (copy #1)
UBI: volume table was restored
UBI: volume 5 ("datafs") re-sized from 133 to 1316 LEBs
UBI: attached mtd1 to ubi0
UBI: MTD device name:            "mtd=1"
UBI: MTD device size:            255 MiB
UBI: number of good PEBs:        2036
UBI: number of bad PEBs:         4
UBI: max. allowed volumes:       128
UBI: wear-leveling threshold:    4096
UBI: number of internal volumes: 1
UBI: number of user volumes:     5
UBI: available PEBs:             0
UBI: total number of reserved PEBs: 2036
UBI: number of PEBs reserved for bad PEB handling: 20
UBI: max/mean erase counter: 5/0
UBI error: ubi_init: cannot attach mtd1
UBI: mtd1 is detached from ubi0
UBI error: ubi_init: UBI error: cannot initialize UBI, error -17
UBI init error -17
exit not allowed from main input shell.

------------------

[root@M54418TWR root]# ubiformat /dev/mtd1 -f /root/ubi.img
ubiformat: mtd1 (NAND), size 267386880 bytes (255.0 MiB), 131072 eraseblocks of 131072 bytes (128.0 KiB), min. I/O size 2048 bytes

libscan: scanning eraseblock 0 --  0 % complete  
...
libscan: scanning eraseblock 2039 -- 100 % complete  
ubiformat: 303 eraseblocks have valid erase counter, mean value is 0
ubiformat: 643 eraseblocks are supposedly empty
ubiformat: bad eraseblocks: 2036, 2037, 2038, 2039
ubiformat: warning!: 1090 of 2036 eraseblocks contain non-ubifs data
ubiformat: continue? (yes/no)  yes
ubiformat: warning!: only 303 of 2036 eraseblocks have valid erase counter
ubiformat: erase counter 0 will be used for all eraseblocks
ubiformat: note, arbitrary erase counter value may be specified using -e option
ubiformat: continue? (yes/no)  yes
ubiformat: use erase counter 0 for all eraseblocks

ubiformat: flashing eraseblock 0 --  0 % complete  
...
ubiformat: flashing eraseblock 442 -- 100 % complete  

ubiformat: formatting eraseblock 443 --  0 % complete  
...
ubiformat: formatting eraseblock 2039 -- 100 % complete  

[root@M54418TWR root]# ubiattach /dev/ubi_ctrl -m 1
UBI: attaching mtd1 to ubi0

UBI: physical eraseblock size:   131072 bytes (128 KiB)

UBI: logical eraseblock size:    126976 bytes

UBI: smallest flash I/O unit:    2048

UBI: VID header offset:          2048 (aligned 2048)

UBI: data offset:                4096

UBI error: check_corruption: PEB 0 contains corrupted VID header, and the data does not contain all 0xFF, this may be a non-UBI PEB or a severe VID header corruption which requires manual inspection

UBI warning: process_eb: valid VID header but corrupted EC header at PEB 4

UBI warning: process_eb: valid VID header but corrupted EC header at PEB 7

UBI warning: process_eb: valid VID header but corrupted EC header at PEB 8

UBI warning: process_eb: valid VID header but corrupted EC header at PEB 12

UBI error: check_corruption: PEB 16 contains corrupted VID header, and the data does not contain all 0xFF, this may be a non-UBI PEB or a severe VID header corruption which requires manual inspection

UBI warning: process_eb: valid VID header but corrupted EC header at PEB 22

...


Thanks,

- Lars

Re: U-Boot: using nand write to flash an ubi image, ubi part fails

[Artem Bityutskiy]

On Wed, 2011-06-01 at 00:57 -0700, Lars Michael wrote:
> Hello,
> 
> Using mkfs.ubifs and ubinize I have created an image with 5 volumes.
> In Linux I could flash it using "ubiformat /dev/mtd1 -f /root/ubi.img"
> and attach it using "ubiattach /dev/ubi_ctrl -m 1". It worked.

OK.

> In production I would prefer to write the image from U-Boot. So in
> U-Boot I used "nand erase" then "nand write" to flash the image. Then
> I want to attach by "ubi part" command - but this fails: "UBI error:
> ubi_init: UBI error: cannot initialize UBI, error -17"

Sorry, I'm not familiar with U-boot, so probably cc-ing U-boot mailing
list would be a good idea.

> Then I booted Linux (via nfs) and tried to re-format+attach, but now
> it fails. Format complains about eraseblock contents and missing erase
> counters: 
> 
> ubiformat: warning!: 1099 of 2036 eraseblocks contain non-ubifs data
> ubiformat: warning!: only 937 of 2036 eraseblocks have valid erase
> counter

OK, this means that many eraseblocks in the flash contain some data
which does not look like UBI headers.

> 
> The ubiattach now reports lots of:
> ...
> UBI error: check_corruption: PEB 0 contains corrupted VID header, and
> the data does not contain all 0xFF, this may be a non-UBI PEB or a
> severe VID header corruption which requires manual inspection
> 
> UBI warning: process_eb: valid VID header but corrupted EC header at
> PEB 4
> ...

OK.

> I realize that the U-Boot 'nand erase' does not preserve the erase
> counters. So what would be the correct way to erase and write in
> U-Boot?

I do not know, you need to ask U-boot-aware people. I've CCed some.

>  And why is the ubiformat not able to correct the problems?

It should be able to format any flash. Probably this is a bug. Can you
reproduce this with nandsim please? Just insmod nandsim on your PC (see
http://www.linux-mtd.infradead.org/faq/nand.html#L_nand_nandsim or
modinfo nandsim), write some garbage there or your image, and try to
format it.

If you can describe how I can reproduce this in my PC, I might look at
this. Or you can just debug ubiformat yourself.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

[U-Boot] U-Boot: using nand write to flash an ubi image, ubi part fails

[Artem Bityutskiy]

On Wed, 2011-06-01 at 00:57 -0700, Lars Michael wrote:
> Hello,
> 
> Using mkfs.ubifs and ubinize I have created an image with 5 volumes.
> In Linux I could flash it using "ubiformat /dev/mtd1 -f /root/ubi.img"
> and attach it using "ubiattach /dev/ubi_ctrl -m 1". It worked.

OK.

> In production I would prefer to write the image from U-Boot. So in
> U-Boot I used "nand erase" then "nand write" to flash the image. Then
> I want to attach by "ubi part" command - but this fails: "UBI error:
> ubi_init: UBI error: cannot initialize UBI, error -17"

Sorry, I'm not familiar with U-boot, so probably cc-ing U-boot mailing
list would be a good idea.

> Then I booted Linux (via nfs) and tried to re-format+attach, but now
> it fails. Format complains about eraseblock contents and missing erase
> counters: 
> 
> ubiformat: warning!: 1099 of 2036 eraseblocks contain non-ubifs data
> ubiformat: warning!: only 937 of 2036 eraseblocks have valid erase
> counter

OK, this means that many eraseblocks in the flash contain some data
which does not look like UBI headers.

> 
> The ubiattach now reports lots of:
> ...
> UBI error: check_corruption: PEB 0 contains corrupted VID header, and
> the data does not contain all 0xFF, this may be a non-UBI PEB or a
> severe VID header corruption which requires manual inspection
> 
> UBI warning: process_eb: valid VID header but corrupted EC header at
> PEB 4
> ...

OK.

> I realize that the U-Boot 'nand erase' does not preserve the erase
> counters. So what would be the correct way to erase and write in
> U-Boot?

I do not know, you need to ask U-boot-aware people. I've CCed some.

>  And why is the ubiformat not able to correct the problems?

It should be able to format any flash. Probably this is a bug. Can you
reproduce this with nandsim please? Just insmod nandsim on your PC (see
http://www.linux-mtd.infradead.org/faq/nand.html#L_nand_nandsim or
modinfo nandsim), write some garbage there or your image, and try to
format it.

If you can describe how I can reproduce this in my PC, I might look at
this. Or you can just debug ubiformat yourself.

-- 
Best Regards,
Artem Bityutskiy (????? ????????)

Re: U-Boot: using nand write to flash an ubi image, ubi part fails

[Ben Gardiner]

Hi Lars

On Wed, Jun 1, 2011 at 3:57 AM, Lars Michael <lh_post@yahoo.com> wrote:
> Hello,
>
> Using mkfs.ubifs and ubinize I have created an image with 5 volumes. In Linux I could flash it using "ubiformat /dev/mtd1 -f /root/ubi.img" and attach it using "ubiattach /dev/ubi_ctrl -m 1". It worked.
>
> In production I would prefer to write the image from U-Boot. So in U-Boot I used "nand erase" then "nand write" to flash the image. Then I want to attach by "ubi part" command - but this fails: "UBI error: ubi_init: UBI error: cannot initialize UBI, error -17"
> [...]
> I realize that the U-Boot 'nand erase' does not preserve the erase counters. So what would be the correct way to erase and write in U-Boot? And why is the ubiformat not able to correct the problems?

It's true that the ECs are not preserved. But it's also true that
trailing pages of 0xff are not trimmed either. You definitely need to
drop trailing 0xff pages when writing ubinized images containing UBIFS
volumes. To do this you can use either the new 'nand write' variant,
.trimffs [1] which is currently patches that have been Acked. Or you
can use the free-space-fixup feature which is available in the
ubifs-2.6 tree and requires that you use a mkfs.ubifs built from the
the mtd-utils tree.

I've never seen the particular error "ubi_init: UBI error: cannot
initialize UBI, error -17" when I have erroneously written a ubinized
image with nand write. But maybe that is due to the paritculars of my
platform drivers or it may be due to the version of u-boot you are
running there -- see below.

I reccomend giving both the 'nand write.trimffs' patches and the
free-space-fixup feature a try (indivually) and reporting back with
your success. After you update u-boot, see below:

> My kernel is 2.6.29 patched with 165 patched from the ubi back port tree.
> U-Boot is 2009.08 with ubifs support, but I am not sure if there are relevant patches. The CPU is a v4 ColdFire and 256MB Micron NAND.

2009.08 is maybe too old. There have been several fixes to both ubi
and ubis in u-boot since then:

$git log --oneline v2009.08..u-boot/master  -- fs/ubifs/ drivers/mtd/ubi/
c7b7d45 UBIFS: Fix dereferencing type-punned pointer compiler warning
64b6817 ubifs.c: BUG: Error following links
b1a14f8 UBIFS: Change ubifsload to not read beyond the requested size
cb9c09d UBIFS: Add ubifsumount command to unmount an active volume
6d8962e Switch from archive libraries to partial linking
2e5167c Replace CONFIG_RELOC_FIXUP_WORKS by CONFIG_NEEDS_MANUAL_RELOC
86af10c Fix "ubi part" cmd re-entrancy
ceeba00 UBI: initialise update marker
1b1f9a9 UBI: Ensure that "background thread" operations are really executed
6865168 ubifsmount fails due to not initialized list
87d93a1 move prototypes for gunzip() and zunzip() to common.h
4b142fe common: delete CONFIG_SYS_64BIT_VSPRINTF and CONFIG_SYS_64BIT_STRTOUL
521af04 Conditionally perform common relocation fixups
ef37c68 ubifs: Correct dereferencing of files-after-symlinks
9d7952e ubifs: Add support for looking up directory and relative symlinks
02f9990 Move __set/clear_bit from ubifs.h to bitops.h


Best Regards,
Ben Gardiner

---
Nanometrics Inc.
http://www.nanometrics.ca

[U-Boot] U-Boot: using nand write to flash an ubi image, ubi part fails

[Ben Gardiner]

Hi Lars

On Wed, Jun 1, 2011 at 3:57 AM, Lars Michael <lh_post@yahoo.com> wrote:
> Hello,
>
> Using mkfs.ubifs and ubinize I have created an image with 5 volumes. In Linux I could flash it using "ubiformat /dev/mtd1 -f /root/ubi.img" and attach it using "ubiattach /dev/ubi_ctrl -m 1". It worked.
>
> In production I would prefer to write the image from U-Boot. So in U-Boot I used "nand erase" then "nand write" to flash the image. Then I want to attach by "ubi part" command - but this fails: "UBI error: ubi_init: UBI error: cannot initialize UBI, error -17"
> [...]
> I realize that the U-Boot 'nand erase' does not preserve the erase counters. So what would be the correct way to erase and write in U-Boot? And why is the ubiformat not able to correct the problems?

It's true that the ECs are not preserved. But it's also true that
trailing pages of 0xff are not trimmed either. You definitely need to
drop trailing 0xff pages when writing ubinized images containing UBIFS
volumes. To do this you can use either the new 'nand write' variant,
.trimffs [1] which is currently patches that have been Acked. Or you
can use the free-space-fixup feature which is available in the
ubifs-2.6 tree and requires that you use a mkfs.ubifs built from the
the mtd-utils tree.

I've never seen the particular error "ubi_init: UBI error: cannot
initialize UBI, error -17" when I have erroneously written a ubinized
image with nand write. But maybe that is due to the paritculars of my
platform drivers or it may be due to the version of u-boot you are
running there -- see below.

I reccomend giving both the 'nand write.trimffs' patches and the
free-space-fixup feature a try (indivually) and reporting back with
your success. After you update u-boot, see below:

> My kernel is 2.6.29 patched with 165 patched from the ubi back port tree.
> U-Boot is 2009.08 with ubifs support, but I am not sure if there are relevant patches. The CPU is a v4 ColdFire and 256MB Micron NAND.

2009.08 is maybe too old. There have been several fixes to both ubi
and ubis in u-boot since then:

$git log --oneline v2009.08..u-boot/master  -- fs/ubifs/ drivers/mtd/ubi/
c7b7d45 UBIFS: Fix dereferencing type-punned pointer compiler warning
64b6817 ubifs.c: BUG: Error following links
b1a14f8 UBIFS: Change ubifsload to not read beyond the requested size
cb9c09d UBIFS: Add ubifsumount command to unmount an active volume
6d8962e Switch from archive libraries to partial linking
2e5167c Replace CONFIG_RELOC_FIXUP_WORKS by CONFIG_NEEDS_MANUAL_RELOC
86af10c Fix "ubi part" cmd re-entrancy
ceeba00 UBI: initialise update marker
1b1f9a9 UBI: Ensure that "background thread" operations are really executed
6865168 ubifsmount fails due to not initialized list
87d93a1 move prototypes for gunzip() and zunzip() to common.h
4b142fe common: delete CONFIG_SYS_64BIT_VSPRINTF and CONFIG_SYS_64BIT_STRTOUL
521af04 Conditionally perform common relocation fixups
ef37c68 ubifs: Correct dereferencing of files-after-symlinks
9d7952e ubifs: Add support for looking up directory and relative symlinks
02f9990 Move __set/clear_bit from ubifs.h to bitops.h


Best Regards,
Ben Gardiner

---
Nanometrics Inc.
http://www.nanometrics.ca

Re: U-Boot: using nand write to flash an ubi image, ubi part fails

[Lars Michael]

Hello Ben,

--- On Wed, 1/6/11, Ben Gardiner <bengardiner@nanometrics.ca> wrote:
> > Using mkfs.ubifs and ubinize I have created an image
> with 5 volumes. In Linux I could flash it using "ubiformat
> /dev/mtd1 -f /root/ubi.img" and attach it using "ubiattach
> /dev/ubi_ctrl -m 1". It worked.
> >
> > In production I would prefer to write the image from
> U-Boot. So in U-Boot I used "nand erase" then "nand write"
> to flash the image. Then I want to attach by "ubi part"
> command - but this fails: "UBI error: ubi_init: UBI error:
> cannot initialize UBI, error -17"
> > [...]
> > I realize that the U-Boot 'nand erase' does not
> preserve the erase counters. So what would be the correct
> way to erase and write in U-Boot? And why is the ubiformat
> not able to correct the problems?
> 
> It's true that the ECs are not preserved. But it's also
> true that
> trailing pages of 0xff are not trimmed either. You
> definitely need to
> drop trailing 0xff pages when writing ubinized images
> containing UBIFS
> volumes. To do this you can use either the new 'nand write'
> variant,
> .trimffs [1] which is currently patches that have been
> Acked. Or you
> can use the free-space-fixup feature which is available in
> the
> ubifs-2.6 tree and requires that you use a mkfs.ubifs built
> from the
> the mtd-utils tree.

Ok, I have updated my mtd-utils and recompiled the mkfs.ubifs to get
support for the free-space-fixup feature. Next I have pulled
all ubi updates from the 2.6.29 backport tree which gave med 154 patches
including the fixup.

Now I can build the image, ubiformat, ubiattach and mount with success.
After a reboot I can ubiattach and mount with success.

Next step is to get it to work in U-Boot too.

Thanks and regards,
- Lars

[U-Boot] U-Boot: using nand write to flash an ubi image, ubi part fails

[Lars Michael]

Hello Ben,

--- On Wed, 1/6/11, Ben Gardiner <bengardiner@nanometrics.ca> wrote:
> > Using mkfs.ubifs and ubinize I have created an image
> with 5 volumes. In Linux I could flash it using "ubiformat
> /dev/mtd1 -f /root/ubi.img" and attach it using "ubiattach
> /dev/ubi_ctrl -m 1". It worked.
> >
> > In production I would prefer to write the image from
> U-Boot. So in U-Boot I used "nand erase" then "nand write"
> to flash the image. Then I want to attach by "ubi part"
> command - but this fails: "UBI error: ubi_init: UBI error:
> cannot initialize UBI, error -17"
> > [...]
> > I realize that the U-Boot 'nand erase' does not
> preserve the erase counters. So what would be the correct
> way to erase and write in U-Boot? And why is the ubiformat
> not able to correct the problems?
> 
> It's true that the ECs are not preserved. But it's also
> true that
> trailing pages of 0xff are not trimmed either. You
> definitely need to
> drop trailing 0xff pages when writing ubinized images
> containing UBIFS
> volumes. To do this you can use either the new 'nand write'
> variant,
> .trimffs [1] which is currently patches that have been
> Acked. Or you
> can use the free-space-fixup feature which is available in
> the
> ubifs-2.6 tree and requires that you use a mkfs.ubifs built
> from the
> the mtd-utils tree.

Ok, I have updated my mtd-utils and recompiled the mkfs.ubifs to get
support for the free-space-fixup feature. Next I have pulled
all ubi updates from the 2.6.29 backport tree which gave med 154 patches
including the fixup.

Now I can build the image, ubiformat, ubiattach and mount with success.
After a reboot I can ubiattach and mount with success.

Next step is to get it to work in U-Boot too.

Thanks and regards,
- Lars

[U-Boot] U-Boot: using nand write to flash an ubi image, ubi part fails

[Aaron Williams]

Hi Lars,

Were you ever successful in getting UBI to work in U-Boot? I am in a similar 
boat. I have UBI working beautifully in Linux (2.6.32.27) but so far I've had 
no luck in U-Boot.

-Aaron

On Monday, July 18, 2011 02:29:08 AM Lars Michael wrote:
> Hello Ben,
> 
> --- On Wed, 1/6/11, Ben Gardiner <bengardiner@nanometrics.ca> wrote:
> > > Using mkfs.ubifs and ubinize I have created an image
> > 
> > with 5 volumes. In Linux I could flash it using "ubiformat
> > /dev/mtd1 -f /root/ubi.img" and attach it using "ubiattach
> > /dev/ubi_ctrl -m 1". It worked.
> > 
> > > In production I would prefer to write the image from
> > 
> > U-Boot. So in U-Boot I used "nand erase" then "nand write"
> > to flash the image. Then I want to attach by "ubi part"
> > command - but this fails: "UBI error: ubi_init: UBI error:
> > cannot initialize UBI, error -17"
> > 
> > > [...]
> > > I realize that the U-Boot 'nand erase' does not
> > 
> > preserve the erase counters. So what would be the correct
> > way to erase and write in U-Boot? And why is the ubiformat
> > not able to correct the problems?
> > 
> > It's true that the ECs are not preserved. But it's also
> > true that
> > trailing pages of 0xff are not trimmed either. You
> > definitely need to
> > drop trailing 0xff pages when writing ubinized images
> > containing UBIFS
> > volumes. To do this you can use either the new 'nand write'
> > variant,
> > .trimffs [1] which is currently patches that have been
> > Acked. Or you
> > can use the free-space-fixup feature which is available in
> > the
> > ubifs-2.6 tree and requires that you use a mkfs.ubifs built
> > from the
> > the mtd-utils tree.
> 
> Ok, I have updated my mtd-utils and recompiled the mkfs.ubifs to get
> support for the free-space-fixup feature. Next I have pulled
> all ubi updates from the 2.6.29 backport tree which gave med 154 patches
> including the fixup.
> 
> Now I can build the image, ubiformat, ubiattach and mount with success.
> After a reboot I can ubiattach and mount with success.
> 
> Next step is to get it to work in U-Boot too.
> 
> Thanks and regards,
> - Lars
> 
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> http://lists.denx.de/mailman/listinfo/u-boot

-- 
Aaron Williams <Aaron.Williams@cavium.com>
 (408) 943-7198

Re: U-Boot: using nand write to flash an ubi image, ubi part fails

[Matthew L. Creech]

On Wed, Jun 1, 2011 at 3:57 AM, Lars Michael <lh_post@yahoo.com> wrote:
> Hello,
>
> Using mkfs.ubifs and ubinize I have created an image with 5 volumes. In Linux I could flash it using "ubiformat /dev/mtd1 -f /root/ubi.img" and attach it using "ubiattach /dev/ubi_ctrl -m 1". It worked.
>
> In production I would prefer to write the image from U-Boot. So in U-Boot I used "nand erase" then "nand write" to flash the image. Then I want to attach by "ubi part" command - but this fails: "UBI error: ubi_init: UBI error: cannot initialize UBI, error -17"
>

Basically, this isn't a valid combination: "nand write" doesn't know
anything about UBI, and a UBIFS image isn't suitable for flashing
directly to NAND - it expects to sit on top of UBI.

There are 2 ways to get it working:

1. Don't use "nand write".  If U-Boot is built with UBI support, you
can use "ubi write" to write a UBI image to flash.  I _think_ you can
use this to overwrite an existing volume with a new image, preserving
ECs (haven't tried it myself).  It certainly works fine to write a new
image to blank flash.

2. Use the "ubinize" tool to create an image suitable for writing
directly to NAND.  This takes a UBIFS image and "wraps" it to turn it
into a UBI image, which you can then use with "nand write".

I've used both these methods to write UBIFS images to flash within
U-Boot, and the 2nd method also works with industrial NAND
programmers.

Hope this helps

-- 
Matthew L. Creech

[U-Boot] U-Boot: using nand write to flash an ubi image, ubi part fails

[Matthew L. Creech]

On Wed, Jun 1, 2011 at 3:57 AM, Lars Michael <lh_post@yahoo.com> wrote:
> Hello,
>
> Using mkfs.ubifs and ubinize I have created an image with 5 volumes. In Linux I could flash it using "ubiformat /dev/mtd1 -f /root/ubi.img" and attach it using "ubiattach /dev/ubi_ctrl -m 1". It worked.
>
> In production I would prefer to write the image from U-Boot. So in U-Boot I used "nand erase" then "nand write" to flash the image. Then I want to attach by "ubi part" command - but this fails: "UBI error: ubi_init: UBI error: cannot initialize UBI, error -17"
>

Basically, this isn't a valid combination: "nand write" doesn't know
anything about UBI, and a UBIFS image isn't suitable for flashing
directly to NAND - it expects to sit on top of UBI.

There are 2 ways to get it working:

1. Don't use "nand write".  If U-Boot is built with UBI support, you
can use "ubi write" to write a UBI image to flash.  I _think_ you can
use this to overwrite an existing volume with a new image, preserving
ECs (haven't tried it myself).  It certainly works fine to write a new
image to blank flash.

2. Use the "ubinize" tool to create an image suitable for writing
directly to NAND.  This takes a UBIFS image and "wraps" it to turn it
into a UBI image, which you can then use with "nand write".

I've used both these methods to write UBIFS images to flash within
U-Boot, and the 2nd method also works with industrial NAND
programmers.

Hope this helps

-- 
Matthew L. Creech

[U-Boot] U-Boot: using nand write to flash an ubi image, ubi part fails

[Lars Michael]

--- On Thu, 29/9/11, Aaron Williams <Aaron.Williams@cavium.com> wrote:
> From: Aaron Williams <Aaron.Williams@cavium.com>
> Subject: Re: [U-Boot] U-Boot: using nand write to flash an ubi image, ubi part fails
> To: u-boot at lists.denx.de
> Cc: "Lars Michael" <lh_post@yahoo.com>
> Date: Thursday, 29 September, 2011, 4:11
> Hi Lars,
> 
> Were you ever successful in getting UBI to work in U-Boot?
> I am in a similar 
> boat. I have UBI working beautifully in Linux (2.6.32.27)
> but so far I've had 
> no luck in U-Boot.
> 
> -Aaron

Hi Aaron,

Good question, I am not able to write a ubi image (created by ubinize) 
and then attatch to it from U-Boot. I found that I need the free space
fixup patch. My U-Boot is rather old (2009.08) and I can not easily
apply that patch due to too many other changes. So for now, I must
write the image from Linux, do the fixup and then U-Boot is able to
attach. 

Regards,
- Lars 

> On Monday, July 18, 2011 02:29:08 AM Lars Michael wrote:
> > Hello Ben,
> > 
> > --- On Wed, 1/6/11, Ben Gardiner <bengardiner@nanometrics.ca>
> wrote:
> > > > Using mkfs.ubifs and ubinize I have created
> an image
> > > 
> > > with 5 volumes. In Linux I could flash it using
> "ubiformat
> > > /dev/mtd1 -f /root/ubi.img" and attach it using
> "ubiattach
> > > /dev/ubi_ctrl -m 1". It worked.
> > > 
> > > > In production I would prefer to write the
> image from
> > > 
> > > U-Boot. So in U-Boot I used "nand erase" then
> "nand write"
> > > to flash the image. Then I want to attach by "ubi
> part"
> > > command - but this fails: "UBI error: ubi_init:
> UBI error:
> > > cannot initialize UBI, error -17"
> > > 
> > > > [...]
> > > > I realize that the U-Boot 'nand erase' does
> not
> > > 
> > > preserve the erase counters. So what would be the
> correct
> > > way to erase and write in U-Boot? And why is the
> ubiformat
> > > not able to correct the problems?
> > > 
> > > It's true that the ECs are not preserved. But
> it's also
> > > true that
> > > trailing pages of 0xff are not trimmed either.
> You
> > > definitely need to
> > > drop trailing 0xff pages when writing ubinized
> images
> > > containing UBIFS
> > > volumes. To do this you can use either the new
> 'nand write'
> > > variant,
> > > .trimffs [1] which is currently patches that have
> been
> > > Acked. Or you
> > > can use the free-space-fixup feature which is
> available in
> > > the
> > > ubifs-2.6 tree and requires that you use a
> mkfs.ubifs built
> > > from the
> > > the mtd-utils tree.
> > 
> > Ok, I have updated my mtd-utils and recompiled the
> mkfs.ubifs to get
> > support for the free-space-fixup feature. Next I have
> pulled
> > all ubi updates from the 2.6.29 backport tree which
> gave med 154 patches
> > including the fixup.
> > 
> > Now I can build the image, ubiformat, ubiattach and
> mount with success.
> > After a reboot I can ubiattach and mount with
> success.
> > 
> > Next step is to get it to work in U-Boot too.
> > 
> > Thanks and regards,
> > - Lars
> > 
> > _______________________________________________
> > U-Boot mailing list
> > U-Boot at lists.denx.de
> > http://lists.denx.de/mailman/listinfo/u-boot
> 
> -- 
> Aaron Williams <Aaron.Williams@cavium.com>
>  (408) 943-7198
> 

U-Boot: using nand write to flash an ubi image, ubi part fails

[Lars Michael]

--- On Thu, 29/9/11, Matthew L. Creech <mlcreech@gmail.com> wrote:
> On Wed, Jun 1, 2011 at 3:57 AM, Lars
> Michael <lh_post@yahoo.com> wrote:
> > Using mkfs.ubifs and ubinize I have created an image
> with 5 volumes. In Linux I could flash it using "ubiformat
> /dev/mtd1 -f /root/ubi.img" and attach it using "ubiattach
> /dev/ubi_ctrl -m 1". It worked.
> >
> > In production I would prefer to write the image from
> U-Boot. So in U-Boot I used "nand erase" then "nand write"
> to flash the image. Then I want to attach by "ubi part"
> command - but this fails: "UBI error: ubi_init: UBI error:
> cannot initialize UBI, error -17"
> >
> 
> Basically, this isn't a valid combination: "nand write"
> doesn't know
> anything about UBI, and a UBIFS image isn't suitable for
> flashing
> directly to NAND - it expects to sit on top of UBI.

Hi Matthew,

I agree, which is why I used ubinize to create a UBI image (with 5 UBIFS images) and then tried to write that image.

> There are 2 ways to get it working:
> 
> 1. Don't use "nand write".  If U-Boot is built with
> UBI support, you
> can use "ubi write" to write a UBI image to flash.  I
> _think_ you can
> use this to overwrite an existing volume with a new image,
> preserving
> ECs (haven't tried it myself).  It certainly works
> fine to write a new
> image to blank flash.
>
> 2. Use the "ubinize" tool to create an image suitable for
> writing
> directly to NAND.  This takes a UBIFS image and
> "wraps" it to turn it
> into a UBI image, which you can then use with "nand
> write".

This is what I am doing (or plan to do). But to get it to work, I need
the space fixup feature for the nand write in U-Boot. I got that feature in Linux, so in Linux it works. Unfortunately it is not easy to patch 
that one in my 2009.08 U-Boot.

Thanks for your comments.

Regards
Lars

[U-Boot] U-Boot: using nand write to flash an ubi image, ubi part fails

[Lars Michael]

--- On Thu, 29/9/11, Matthew L. Creech <mlcreech@gmail.com> wrote:
> On Wed, Jun 1, 2011 at 3:57 AM, Lars
> Michael <lh_post@yahoo.com> wrote:
> > Using mkfs.ubifs and ubinize I have created an image
> with 5 volumes. In Linux I could flash it using "ubiformat
> /dev/mtd1 -f /root/ubi.img" and attach it using "ubiattach
> /dev/ubi_ctrl -m 1". It worked.
> >
> > In production I would prefer to write the image from
> U-Boot. So in U-Boot I used "nand erase" then "nand write"
> to flash the image. Then I want to attach by "ubi part"
> command - but this fails: "UBI error: ubi_init: UBI error:
> cannot initialize UBI, error -17"
> >
> 
> Basically, this isn't a valid combination: "nand write"
> doesn't know
> anything about UBI, and a UBIFS image isn't suitable for
> flashing
> directly to NAND - it expects to sit on top of UBI.

Hi Matthew,

I agree, which is why I used ubinize to create a UBI image (with 5 UBIFS images) and then tried to write that image.

> There are 2 ways to get it working:
> 
> 1. Don't use "nand write".? If U-Boot is built with
> UBI support, you
> can use "ubi write" to write a UBI image to flash.? I
> _think_ you can
> use this to overwrite an existing volume with a new image,
> preserving
> ECs (haven't tried it myself).? It certainly works
> fine to write a new
> image to blank flash.
>
> 2. Use the "ubinize" tool to create an image suitable for
> writing
> directly to NAND.? This takes a UBIFS image and
> "wraps" it to turn it
> into a UBI image, which you can then use with "nand
> write".

This is what I am doing (or plan to do). But to get it to work, I need
the space fixup feature for the nand write in U-Boot. I got that feature in Linux, so in Linux it works. Unfortunately it is not easy to patch 
that one in my 2009.08 U-Boot.

Thanks for your comments.

Regards
Lars

[U-Boot] U-Boot: using nand write to flash an ubi image, ubi part fails

[Aaron Williams]

Thanks.

I finally found the problem on our end and now I have ubifs working
beautifully.

-Aaron

On 09/29/2011 08:33 PM, Lars Michael wrote:
> --- On Thu, 29/9/11, Aaron Williams <Aaron.Williams@cavium.com> wrote:
>> From: Aaron Williams <Aaron.Williams@cavium.com>
>> Subject: Re: [U-Boot] U-Boot: using nand write to flash an ubi image, ubi part fails
>> To: u-boot at lists.denx.de
>> Cc: "Lars Michael" <lh_post@yahoo.com>
>> Date: Thursday, 29 September, 2011, 4:11
>> Hi Lars,
>>
>> Were you ever successful in getting UBI to work in U-Boot?
>> I am in a similar 
>> boat. I have UBI working beautifully in Linux (2.6.32.27)
>> but so far I've had 
>> no luck in U-Boot.
>>
>> -Aaron
> Hi Aaron,
>
> Good question, I am not able to write a ubi image (created by ubinize) 
> and then attatch to it from U-Boot. I found that I need the free space
> fixup patch. My U-Boot is rather old (2009.08) and I can not easily
> apply that patch due to too many other changes. So for now, I must
> write the image from Linux, do the fixup and then U-Boot is able to
> attach. 
>
> Regards,
> - Lars 
>
>> On Monday, July 18, 2011 02:29:08 AM Lars Michael wrote:
>>> Hello Ben,
>>>
>>> --- On Wed, 1/6/11, Ben Gardiner <bengardiner@nanometrics.ca>
>> wrote:
>>>>> Using mkfs.ubifs and ubinize I have created
>> an image
>>>> with 5 volumes. In Linux I could flash it using
>> "ubiformat
>>>> /dev/mtd1 -f /root/ubi.img" and attach it using
>> "ubiattach
>>>> /dev/ubi_ctrl -m 1". It worked.
>>>>
>>>>> In production I would prefer to write the
>> image from
>>>> U-Boot. So in U-Boot I used "nand erase" then
>> "nand write"
>>>> to flash the image. Then I want to attach by "ubi
>> part"
>>>> command - but this fails: "UBI error: ubi_init:
>> UBI error:
>>>> cannot initialize UBI, error -17"
>>>>
>>>>> [...]
>>>>> I realize that the U-Boot 'nand erase' does
>> not
>>>> preserve the erase counters. So what would be the
>> correct
>>>> way to erase and write in U-Boot? And why is the
>> ubiformat
>>>> not able to correct the problems?
>>>>
>>>> It's true that the ECs are not preserved. But
>> it's also
>>>> true that
>>>> trailing pages of 0xff are not trimmed either.
>> You
>>>> definitely need to
>>>> drop trailing 0xff pages when writing ubinized
>> images
>>>> containing UBIFS
>>>> volumes. To do this you can use either the new
>> 'nand write'
>>>> variant,
>>>> .trimffs [1] which is currently patches that have
>> been
>>>> Acked. Or you
>>>> can use the free-space-fixup feature which is
>> available in
>>>> the
>>>> ubifs-2.6 tree and requires that you use a
>> mkfs.ubifs built
>>>> from the
>>>> the mtd-utils tree.
>>> Ok, I have updated my mtd-utils and recompiled the
>> mkfs.ubifs to get
>>> support for the free-space-fixup feature. Next I have
>> pulled
>>> all ubi updates from the 2.6.29 backport tree which
>> gave med 154 patches
>>> including the fixup.
>>>
>>> Now I can build the image, ubiformat, ubiattach and
>> mount with success.
>>> After a reboot I can ubiattach and mount with
>> success.
>>> Next step is to get it to work in U-Boot too.
>>>
>>> Thanks and regards,
>>> - Lars
>>>
>>> _______________________________________________
>>> U-Boot mailing list
>>> U-Boot at lists.denx.de
>>> http://lists.denx.de/mailman/listinfo/u-boot
>> -- 
>> Aaron Williams <Aaron.Williams@cavium.com>
>>  (408) 943-7198
>>

[U-Boot] U-Boot: using nand write to flash an ubi image, ubi part fails

[Lars Michael]

--- On Fri, 30/9/11, Aaron Williams <Aaron.Williams@cavium.com> wrote:
> I finally found the problem on our end and now I have ubifs
> working
> beautifully.
> 
> -Aaron

Good, do you mind sharing what the solution to your problem is and what version of U-Boot you are running?

Thanks,
- Lars 

Re: [U-Boot] U-Boot: using nand write to flash an ubi image, ubi part fails

[Scott Wood]

On 09/29/2011 10:44 PM, Lars Michael wrote:
> --- On Thu, 29/9/11, Matthew L. Creech <mlcreech@gmail.com> wrote:
>> 2. Use the "ubinize" tool to create an image suitable for
>> writing
>> directly to NAND.  This takes a UBIFS image and
>> "wraps" it to turn it
>> into a UBI image, which you can then use with "nand
>> write".
> 
> This is what I am doing (or plan to do). But to get it to work, I need
> the space fixup feature for the nand write in U-Boot. I got that feature in Linux, so in Linux it works. Unfortunately it is not easy to patch 
> that one in my 2009.08 U-Boot.

Try using a current U-Boot, and use "nand write.trimffs".

-Scott

[U-Boot] U-Boot: using nand write to flash an ubi image, ubi part fails

[Scott Wood]

On 09/29/2011 10:44 PM, Lars Michael wrote:
> --- On Thu, 29/9/11, Matthew L. Creech <mlcreech@gmail.com> wrote:
>> 2. Use the "ubinize" tool to create an image suitable for
>> writing
>> directly to NAND.  This takes a UBIFS image and
>> "wraps" it to turn it
>> into a UBI image, which you can then use with "nand
>> write".
> 
> This is what I am doing (or plan to do). But to get it to work, I need
> the space fixup feature for the nand write in U-Boot. I got that feature in Linux, so in Linux it works. Unfortunately it is not easy to patch 
> that one in my 2009.08 U-Boot.

Try using a current U-Boot, and use "nand write.trimffs".

-Scott