[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Lukasz Majewski]

Dear All,

Since DFU usage at u-boot is spreading to different device types (MMC,
NAND), file systems, raw partitions, ubi, etc, I think that it is a
good moment to unify and structure the form of dfu_alt_info environment
variable.


Proposed new format for single dfu entity:

NAME     | Type | Device | Dev_num | Dev_part | FS   | start | size  | 

where:

Name     - name of the alt setting (as seen by dfu-util -l)
Type     - description of the image (raw, file, img, command [*])
Device   - physical device on which data are stored (mmc, nand, "-")
Dev_num  - number of the device - it is possible to store data via DFU
           on several devices of the same type.
Dev_part - number of partitions on the device.
FS       - information about file system on the device (fat,
	   ext2/ext4, ubi).
start    - start offset from the beginning of the Device (byte or LBA
           number addressed)
size     - maximal number of blocks to be stored on the Device
 	   (expressed with Bytes of LBA number) (protection from
 	   overwriting the whole device)


Example:

NAME     | Type | Device | Dev_num | Dev_part | FS   | start | size  |
----------------------------------------------------------------------
u-boot   | raw  | mmc    | 0       | "-" [**] | "-"  | 0x80  | 0x100 |
uImage   | file | mmc    | 0       | 2        | ext4 | "-"   | "-"   |
root.img | img  | mmc    | 0       | 5        | "-"  | "-"   | "-"   |
root.img | raw  | mmc    | 0       | "-"      | "-"  | 0x1000| 0x4000|

u-boot   | raw  | nand   | 0       | "-"      | "-"  | 0x100 | 0x100 |
uImage   | file | nand   | 0       | 3        | ubi  | "-"   | "-"   | 

command  | cmd  | "-"    | "-"     | "-"      | "-"  | "-"   | "-"   |

[*]  - support for sending command via DFU (see below)
[**] - "-" - indicates that no value is provided for this field.
	Despite of that it is mandatory to facilitate parsing. 


Please share any possible use cases - for now and for the future. My
goal is to devise dfu_alt_info structure which would work for us for a
long time.

After setting up new format, it would be possible to replace different
dfu command variations (i.e. dfu mmc 0, dfu nand) with only one command
"dfu".


------------------------------------------------------------------------
DFU extension - Command execution. 
------------------------------------------------------------------------

As a follow up I plan to add support for sending file filled with
command[s]. This would provide interface for automated tests and
force board to reset itself after emulated reset command from dfu-util
program.

dfu_alt_info = "command cmd - - - - - -;"

and then on host:
dfu-util -aN -D file_with_commands

The end step would be to extend dfu-util to support -C switch, which
would allow to send u-boot command quoted as text:

dfu-util -aN -C "reset"


-- 
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Tormod Volden]

On Tue, Jul 16, 2013 at 5:35 PM, Lukasz Majewski wrote:
> The end step would be to extend dfu-util to support -C switch, which
> would allow to send u-boot command quoted as text:
>
> dfu-util -aN -C "reset"

Instead of adding a new, rather u-boot specific option to dfu-util,
would implementation of the commonly used "-" filename for stdin work
for you?

echo "reset" | dfu-util -aN -

Cheers,
Tormod

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Lukasz Majewski]

On Tue, 16 Jul 2013 23:27:44 +0200
Tormod Volden <lists.tormod@gmail.com> wrote:

Hi Tormod,

> On Tue, Jul 16, 2013 at 5:35 PM, Lukasz Majewski wrote:
> > The end step would be to extend dfu-util to support -C switch, which
> > would allow to send u-boot command quoted as text:
> >
> > dfu-util -aN -C "reset"
> 
> Instead of adding a new, rather u-boot specific option to dfu-util,
> would implementation of the commonly used "-" filename for stdin work
> for you?

Definitely YES :-).

> 
> echo "reset" | dfu-util -aN -

I think, that above is more versatile solution.

> 
> Cheers,
> Tormod

Regards,
Lukasz Majewski

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20130716/f416b3e9/attachment.pgp>

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Heiko Schocher]

Hello Lukasz,

Am 16.07.2013 17:35, schrieb Lukasz Majewski:
> Dear All,
>
> Since DFU usage at u-boot is spreading to different device types (MMC,
> NAND), file systems, raw partitions, ubi, etc, I think that it is a
> good moment to unify and structure the form of dfu_alt_info environment
> variable.

Full Ack!

> Proposed new format for single dfu entity:
>
> NAME     | Type | Device | Dev_num | Dev_part | FS   | start | size  |
>
> where:
>
> Name     - name of the alt setting (as seen by dfu-util -l)
> Type     - description of the image (raw, file, img, command [*])
> Device   - physical device on which data are stored (mmc, nand, "-")
> Dev_num  - number of the device - it is possible to store data via DFU
>             on several devices of the same type.
> Dev_part - number of partitions on the device.

Should this be "number of the partition on the device"

You mean here the mtd partition for storing, right?

> FS       - information about file system on the device (fat,
> 	   ext2/ext4, ubi).
> start    - start offset from the beginning of the Device (byte or LBA
>             number addressed)
> size     - maximal number of blocks to be stored on the Device
>   	   (expressed with Bytes of LBA number) (protection from
>   	   overwriting the whole device)
>
>
> Example:

Maybe dummy questions ...

> NAME     | Type | Device | Dev_num | Dev_part | FS   | start | size  |
> ----------------------------------------------------------------------
> u-boot   | raw  | mmc    | 0       | "-" [**] | "-"  | 0x80  | 0x100 |
> uImage   | file | mmc    | 0       | 2        | ext4 | "-"   | "-"   |

Is this enough information? Where to store the uImage file on the ext4
partition?

> root.img | img  | mmc    | 0       | 5        | "-"  | "-"   | "-"   |

img means here: getting an image and storing it on the mtd partition
"Dev_part" if start and size are marked with "-", beginning
on start of the partition?

Wouldn't it be better to use here mtd partition names instead
numbers for "Dev_part"?

What if "start" and "size" are filled with values for the "Type" "img"?
Or is this forbidden for the "Type" "img"?

> root.img | raw  | mmc    | 0       | "-"      | "-"  | 0x1000| 0x4000|
>
> u-boot   | raw  | nand   | 0       | "-"      | "-"  | 0x100 | 0x100 |
> uImage   | file | nand   | 0       | 3        | ubi  | "-"   | "-"   |

s/uImage/rootfs.img ? s/file/img ?

For the FS "ubi" we need to specify, how to burn this into nand.
I think we have no "ubi format" command. With "ubi write ..."
we can only write ubi volumes ... and we want here to burn an ubi image,
which was created with ubinize and contains one or more ubi volumes.

Maybe usecase for updating ubi volumes:
ubivolumename   | img | nand   | 0       | 3        | ubivol | "-"   | "-"   |

If you want to update an ubivolume ...

results in this steps:
- get image per dfu @ img_addr
   We need to get here the complete image, as ubi write
   cannot write incremental ...
- ubi part get_mtd_partition_name("Dev_part=3") vid_header_offset
                                                 ^
                                                 From where get we this parameter ...
                                                 value in "start"?
- ubi write img_addr ubivolumename ${filesize}

> command  | cmd  | "-"    | "-"     | "-"      | "-"  | "-"   | "-"   |
>
> [*]  - support for sending command via DFU (see below)
> [**] - "-" - indicates that no value is provided for this field.
> 	Despite of that it is mandatory to facilitate parsing.
>
>
> Please share any possible use cases - for now and for the future. My
> goal is to devise dfu_alt_info structure which would work for us for a
> long time.
>
> After setting up new format, it would be possible to replace different
> dfu command variations (i.e. dfu mmc 0, dfu nand) with only one command
> "dfu".
>
>
> ------------------------------------------------------------------------
> DFU extension - Command execution.
> ------------------------------------------------------------------------
>
> As a follow up I plan to add support for sending file filled with
> command[s]. This would provide interface for automated tests and
> force board to reset itself after emulated reset command from dfu-util
> program.
>
> dfu_alt_info = "command cmd - - - - - -;"
>
> and then on host:
> dfu-util -aN -D file_with_commands
>
> The end step would be to extend dfu-util to support -C switch, which
> would allow to send u-boot command quoted as text:
>
> dfu-util -aN -C "reset"

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Lukasz Majewski]

On Wed, 17 Jul 2013 12:26:35 +0200 Heiko Schocher hs at denx.de wrote,

Hi Heiko,

> Hello Lukasz,
> 
> Am 16.07.2013 17:35, schrieb Lukasz Majewski:
> > Dear All,
> >
> > Since DFU usage at u-boot is spreading to different device types
> > (MMC, NAND), file systems, raw partitions, ubi, etc, I think that
> > it is a good moment to unify and structure the form of dfu_alt_info
> > environment variable.
> 
> Full Ack!
> 
> > Proposed new format for single dfu entity:
> >
> > NAME     | Type | Device | Dev_num | Dev_part | FS   | start |
> > size  |
> >
> > where:
> >
> > Name     - name of the alt setting (as seen by dfu-util -l)
> > Type     - description of the image (raw, file, img, command [*])
> > Device   - physical device on which data are stored (mmc, nand, "-")
> > Dev_num  - number of the device - it is possible to store data via
> > DFU on several devices of the same type.
> > Dev_part - number of partitions on the device.
> 
> Should this be "number of the partition on the device"

No. I made a mistake. It should be "partition to which we want to write"

> 
> You mean here the mtd partition for storing, right?

Yes, number of the partition to store data. The exact address will be
extracted from mtdparts or partitions env variable.

> 
> > FS       - information about file system on the device (fat,
> > 	   ext2/ext4, ubi).
> > start    - start offset from the beginning of the Device (byte or
> > LBA number addressed)
> > size     - maximal number of blocks to be stored on the Device
> >   	   (expressed with Bytes of LBA number) (protection from
> >   	   overwriting the whole device)
> >
> >
> > Example:
> 
> Maybe dummy questions ...
> 
> > NAME     | Type | Device | Dev_num | Dev_part | FS   | start |
> > size  |
> > ----------------------------------------------------------------------
> > u-boot   | raw  | mmc    | 0       | "-" [**] | "-"  | 0x80  |
> > 0x100 | uImage   | file | mmc    | 0       | 2        | ext4 |
> > "-"   | "-"   |
> 
> Is this enough information? Where to store the uImage file on the ext4
> partition?

To store uImage file on ext4/fat/ext2 partition it is enough to only
give:

uImage mmc 0 2, 

which maps to the following command:

ext4write mmc 0:2 0x44000000 /uImage [size]

The file size is taken from number of sent bytes via DFU/USB.

With writing to file system, one need to first store the whole
transmitted data to a buffer and then store the (big) buffer on the
SD/eMMC device.

Since, we aren't supporting "seek" kind of write to current ext4
implementation at u-boot, the "big" buffer must be used.

> 
> > root.img | img  | mmc    | 0       | 5        | "-"  | "-"   |
> > "-"   |
> 
> img means here: getting an image and storing it on the mtd partition
> "Dev_part" if start and size are marked with "-", beginning
> on start of the partition?

No, here img is for example a rootfs image. When storing it on the
eMMC, it would be better to specify destination partition.

> 
> Wouldn't it be better to use here mtd partition names instead
> numbers for "Dev_part"?

I'd prefer to create a new entrance here (Part_name):

NAME | Type | Device | Dev_num | Dev_part | Part_name | FS | start |
size |

I would like to avoid situation with ambiguous fields. One field
shall only serve for one (clear) purpose. When not needed/used - field
shall be marked as "-".

> 
> What if "start" and "size" are filled with values for the "Type"
> "img"? Or is this forbidden for the "Type" "img"?

I think, that each "Type" shall have predefined set of allowed
attributes:

1. img: "Device" + "Dev num" + "Dev parts"
2. raw: "Device" + "Dev num" + "start" + "size"
3. file: "Device" + "Dev num" + "Dev parts" + "FS"

and so on.


> 
> > root.img | raw  | mmc    | 0       | "-"      | "-"  | 0x1000|
> > 0x4000|
> >
> > u-boot   | raw  | nand   | 0       | "-"      | "-"  | 0x100 |
> > 0x100 | uImage   | file | nand   | 0       | 3        | ubi  |
> > "-"   | "-"   |
> 
> s/uImage/rootfs.img ? s/file/img ?

NAME: uImage and rootfs.img maps to files sent to board.

The "NAME" is used thereafter to provide exact name for file system
write (ext4write/ fatwrite). 

With DFU the distinction between dfu entities (files, raw images, etc)
is performed via alt settings (-aX switch at dfu-util).

> 
> For the FS "ubi" we need to specify, how to burn this into nand.
> I think we have no "ubi format" command. 

Is already ubi format command available at u-boot? 

_As a side note:_

I'm now developing proof-of-concept idea of executing set of commands
sent to u-boot by dfu-util [***].

Rationale for this is the lack of ability to reset u-boot after sending
data to u-boot via DFU. dfu-util has -R option, but this seems to reset
usb interface, not the target device.


I will set an env:

setenv dfu "dfu mmc 0;${dfu_commands}"

Then at dfu itself, I will read commands to be performed. Then I will
set $dfu_commands env and exit from dfu command. 



> With "ubi write ..."
> we can only write ubi volumes ... and we want here to burn an ubi
> image, which was created with ubinize and contains one or more ubi
> volumes
> 
> Maybe usecase for updating ubi volumes:
> ubivolumename   | img | nand   | 0       | 3        | ubivol | "-"
> | "-"   |
> 
> If you want to update an ubivolume ...
> 
> results in this steps:
> - get image per dfu @ img_addr
>    We need to get here the complete image, as ubi write
>    cannot write incremental ...

So each volume shall have different dfu entity. Then those volumes are
stored at ram with different addresses [*]?

> - ubi part get_mtd_partition_name("Dev_part=3") vid_header_offset
>                                                  ^
>                                                  From where get we
> this parameter ... value in "start"?

So here you get information about the ("Dev_part=3") partition start
offset [**]?

> - ubi write img_addr ubivolumename ${filesize}

Here you combine volumes sent at [*] and store it at correct offset
[**]?

I'm not an expert about ubi, so please be patient with my
questions :-). 

For me it looks like a mixture of ordinary u-boot commands with dfu
transfer[s] needed to perform correct ubi image write. Maybe [***] will
help you?

> 
> > command  | cmd  | "-"    | "-"     | "-"      | "-"  | "-"   |
> > "-"   |
> >
> > [*]  - support for sending command via DFU (see below)
> > [**] - "-" - indicates that no value is provided for this field.
> > 	Despite of that it is mandatory to facilitate parsing.
> >
> >
> > Please share any possible use cases - for now and for the future. My
> > goal is to devise dfu_alt_info structure which would work for us
> > for a long time.
> >
> > After setting up new format, it would be possible to replace
> > different dfu command variations (i.e. dfu mmc 0, dfu nand) with
> > only one command "dfu".
> >
> >
> > ------------------------------------------------------------------------
> > DFU extension - Command execution.
> > ------------------------------------------------------------------------
> >
> > As a follow up I plan to add support for sending file filled with
> > command[s]. This would provide interface for automated tests and
> > force board to reset itself after emulated reset command from
> > dfu-util program.
> >
> > dfu_alt_info = "command cmd - - - - - -;"
> >
> > and then on host:
> > dfu-util -aN -D file_with_commands
> >
> > The end step would be to extend dfu-util to support -C switch, which
> > would allow to send u-boot command quoted as text:
> >
> > dfu-util -aN -C "reset"
> 
> bye,
> Heiko



-- 
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Tormod Volden]

On Wed, Jul 17, 2013 at 4:34 PM, Lukasz Majewski wrote:
> _As a side note:_
>
> I'm now developing proof-of-concept idea of executing set of commands
> sent to u-boot by dfu-util [***].
>
> Rationale for this is the lack of ability to reset u-boot after sending
> data to u-boot via DFU. dfu-util has -R option, but this seems to reset
> usb interface, not the target device.

Actually, dfu-util -R  sends a DFU_DETACH request before performing
the USB reset. The reason for this was long unclear to me, until I
looked at the uboot dfu code (inherited from openmoko):

http://git.denx.de/?p=u-boot.git;a=blob;f=drivers/usb/gadget/f_dfu.c;h=d7ae0c0c6a9f6a95ca3a702d92636eb25e21253c;hb=HEAD#l314

"Proprietary extension: 'detach' from idle mode and get back to
runtime mode in case of USB Reset. As much as I dislike this, we just
can't use every USB bus reset to switch back to runtime mode, since at
least the Linux USB stack likes to send a number of resets in a row :(
"

The hack/abuse of sending a DFU_DETACH when in DFU mode is not part of
the DFU standard, but dfu-util supports it since openmoko was for long
the major target for dfu-util.

If you don't need to switch back from DFU mode to runtime mode, maybe
you can use this to reset u-boot?

Regards,
Tormod

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Heiko Schocher]

Hello Lukasz,

Am 17.07.2013 16:34, schrieb Lukasz Majewski:
> On Wed, 17 Jul 2013 12:26:35 +0200 Heiko Schocher hs at denx.de wrote,
>
> Hi Heiko,
>
>> Hello Lukasz,
>>
>> Am 16.07.2013 17:35, schrieb Lukasz Majewski:
>>> Dear All,
>>>
>>> Since DFU usage at u-boot is spreading to different device types
>>> (MMC, NAND), file systems, raw partitions, ubi, etc, I think that
>>> it is a good moment to unify and structure the form of dfu_alt_info
>>> environment variable.
>>
>> Full Ack!
>>
>>> Proposed new format for single dfu entity:
>>>
>>> NAME     | Type | Device | Dev_num | Dev_part | FS   | start |
>>> size  |
>>>
>>> where:
>>>
>>> Name     - name of the alt setting (as seen by dfu-util -l)
>>> Type     - description of the image (raw, file, img, command [*])
>>> Device   - physical device on which data are stored (mmc, nand, "-")
>>> Dev_num  - number of the device - it is possible to store data via
>>> DFU on several devices of the same type.
>>> Dev_part - number of partitions on the device.
>>
>> Should this be "number of the partition on the device"
>
> No. I made a mistake. It should be "partition to which we want to write"
>
>>
>> You mean here the mtd partition for storing, right?
>
> Yes, number of the partition to store data. The exact address will be
> extracted from mtdparts or partitions env variable.

Ok.

>>
>>> FS       - information about file system on the device (fat,
>>> 	   ext2/ext4, ubi).
>>> start    - start offset from the beginning of the Device (byte or
>>> LBA number addressed)
>>> size     - maximal number of blocks to be stored on the Device
>>>    	   (expressed with Bytes of LBA number) (protection from
>>>    	   overwriting the whole device)
>>>
>>>
>>> Example:
>>
>> Maybe dummy questions ...
>>
>>> NAME     | Type | Device | Dev_num | Dev_part | FS   | start |
>>> size  |
>>> ----------------------------------------------------------------------
>>> u-boot   | raw  | mmc    | 0       | "-" [**] | "-"  | 0x80  |
>>> 0x100 | uImage   | file | mmc    | 0       | 2        | ext4 |
>>> "-"   | "-"   |
>>
>> Is this enough information? Where to store the uImage file on the ext4
>> partition?
>
> To store uImage file on ext4/fat/ext2 partition it is enough to only
> give:
>
> uImage mmc 0 2,
>
> which maps to the following command:
>
> ext4write mmc 0:2 0x44000000 /uImage [size]

Hmm... and what if I have my uImage in /boot/ ?

> The file size is taken from number of sent bytes via DFU/USB.

Yes.

> With writing to file system, one need to first store the whole
> transmitted data to a buffer and then store the (big) buffer on the
> SD/eMMC device.

Ok, same for an ubi image.

> Since, we aren't supporting "seek" kind of write to current ext4
> implementation at u-boot, the "big" buffer must be used.

Ok, there are places to optimize ;-)

>>> root.img | img  | mmc    | 0       | 5        | "-"  | "-"   |
>>> "-"   |
>>
>> img means here: getting an image and storing it on the mtd partition
>> "Dev_part" if start and size are marked with "-", beginning
>> on start of the partition?
>
> No, here img is for example a rootfs image. When storing it on the
> eMMC, it would be better to specify destination partition.
>
>>
>> Wouldn't it be better to use here mtd partition names instead
>> numbers for "Dev_part"?
>
> I'd prefer to create a new entrance here (Part_name):
>
> NAME | Type | Device | Dev_num | Dev_part | Part_name | FS | start |
> size |
>
> I would like to avoid situation with ambiguous fields. One field
> shall only serve for one (clear) purpose. When not needed/used - field
> shall be marked as "-".

Ok. but this gives long constructs ...

>> What if "start" and "size" are filled with values for the "Type"
>> "img"? Or is this forbidden for the "Type" "img"?
>
> I think, that each "Type" shall have predefined set of allowed
> attributes:
>
> 1. img: "Device" + "Dev num" + "Dev parts"
> 2. raw: "Device" + "Dev num" + "start" + "size"
> 3. file: "Device" + "Dev num" + "Dev parts" + "FS"
>
> and so on.

If we introduce a "Part_name" this should be also possible:

4. img: "Device" + "Dev num" + "Part_name"
5. file: "Device" + "Dev num" + "Part_name" + "FS"

or?

>>> root.img | raw  | mmc    | 0       | "-"      | "-"  | 0x1000|
>>> 0x4000|
>>>
>>> u-boot   | raw  | nand   | 0       | "-"      | "-"  | 0x100 |
>>> 0x100 | uImage   | file | nand   | 0       | 3        | ubi  |
>>> "-"   | "-"   |
>>
>> s/uImage/rootfs.img ? s/file/img ?
>
> NAME: uImage and rootfs.img maps to files sent to board.
>
> The "NAME" is used thereafter to provide exact name for file system
> write (ext4write/ fatwrite).

Ah! If we have to write in a subdirectory, "NAME" is the fullpath + filename?

> With DFU the distinction between dfu entities (files, raw images, etc)
> is performed via alt settings (-aX switch at dfu-util).
>
>>
>> For the FS "ubi" we need to specify, how to burn this into nand.
>> I think we have no "ubi format" command.
>
> Is already ubi format command available at u-boot?

No.

> _As a side note:_
>
> I'm now developing proof-of-concept idea of executing set of commands
> sent to u-boot by dfu-util [***].

Ok.

> Rationale for this is the lack of ability to reset u-boot after sending
> data to u-boot via DFU. dfu-util has -R option, but this seems to reset
> usb interface, not the target device.

That answers my next question which comed in my mind ...

> I will set an env:
>
> setenv dfu "dfu mmc 0;${dfu_commands}"
>
> Then at dfu itself, I will read commands to be performed. Then I will
> set $dfu_commands env and exit from dfu command.

Hmm... is this not oversized for the "reset board" use case?

>> With "ubi write ..."
>> we can only write ubi volumes ... and we want here to burn an ubi
>> image, which was created with ubinize and contains one or more ubi
>> volumes
>>
>> Maybe usecase for updating ubi volumes:
>> ubivolumename   | img | nand   | 0       | 3        | ubivol | "-"
>> | "-"   |
>>
>> If you want to update an ubivolume ...
>>
>> results in this steps:
>> - get image per dfu @ img_addr
>>     We need to get here the complete image, as ubi write
>>     cannot write incremental ...
>
> So each volume shall have different dfu entity. Then those volumes are
> stored at ram with different addresses [*]?

Yes for every volume have different dfu entity.
You update only one volume with one dfu-util command, or? So
no different addresses needed ...

>> - ubi part get_mtd_partition_name("Dev_part=3") vid_header_offset
>>                                                   ^
>>                                                   From where get we
>> this parameter ... value in "start"?
>
> So here you get information about the ("Dev_part=3") partition start
> offset [**]?

Yes ... what is with the "vid_header_offset" parameter?

>> - ubi write img_addr ubivolumename ${filesize}
>
> Here you combine volumes sent at [*] and store it at correct offset
> [**]?

Yes! But just one volume at once ...

> I'm not an expert about ubi, so please be patient with my
> questions :-).

No problem! I also just learning ;-)

The above steps are just to get the idea, what must be done for
updating an ubi image, as this were the steps if you type in commands.

> For me it looks like a mixture of ordinary u-boot commands with dfu
> transfer[s] needed to perform correct ubi image write. Maybe [***] will
> help you?

If we have the dfu_buf address availiable ... maybe it is worth a
try ... thinking of sending per dfu a "run upd_ubi" and the
"upd_ubi" command do the complete update ... but, if we update
more than one image ... this will not work.

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Lukasz Majewski]

On Thu, 18 Jul 2013 07:36:40 +0200 Heiko Schocher hs at denx.de wrote,

Hi Heiko,

> Hello Lukasz,
> 
> Am 17.07.2013 16:34, schrieb Lukasz Majewski:
> > On Wed, 17 Jul 2013 12:26:35 +0200 Heiko Schocher hs at denx.de wrote,
> >
> > Hi Heiko,
> >
> >> Hello Lukasz,
> >>
> >> Am 16.07.2013 17:35, schrieb Lukasz Majewski:
> >>> Dear All,
> >>>
> >>> Since DFU usage at u-boot is spreading to different device types
> >>> (MMC, NAND), file systems, raw partitions, ubi, etc, I think that
> >>> it is a good moment to unify and structure the form of
> >>> dfu_alt_info environment variable.
> >>
> >> Full Ack!
> >>
> >>> Proposed new format for single dfu entity:
> >>>
> >>> NAME     | Type | Device | Dev_num | Dev_part | FS   | start |
> >>> size  |
> >>>
> >>> where:
> >>>
> >>> Name     - name of the alt setting (as seen by dfu-util -l)
> >>> Type     - description of the image (raw, file, img, command [*])
> >>> Device   - physical device on which data are stored (mmc, nand,
> >>> "-") Dev_num  - number of the device - it is possible to store
> >>> data via DFU on several devices of the same type.
> >>> Dev_part - number of partitions on the device.
> >>
> >> Should this be "number of the partition on the device"
> >
> > No. I made a mistake. It should be "partition to which we want to
> > write"
> >
> >>
> >> You mean here the mtd partition for storing, right?
> >
> > Yes, number of the partition to store data. The exact address will
> > be extracted from mtdparts or partitions env variable.
> 
> Ok.
> 
> >>
> >>> FS       - information about file system on the device (fat,
> >>> 	   ext2/ext4, ubi).
> >>> start    - start offset from the beginning of the Device (byte or
> >>> LBA number addressed)
> >>> size     - maximal number of blocks to be stored on the Device
> >>>    	   (expressed with Bytes of LBA number) (protection
> >>> from overwriting the whole device)
> >>>
> >>>
> >>> Example:
> >>
> >> Maybe dummy questions ...
> >>
> >>> NAME     | Type | Device | Dev_num | Dev_part | FS   | start |
> >>> size  |
> >>> ----------------------------------------------------------------------
> >>> u-boot   | raw  | mmc    | 0       | "-" [**] | "-"  | 0x80  |
> >>> 0x100 | uImage   | file | mmc    | 0       | 2        | ext4 |
> >>> "-"   | "-"   |
> >>
> >> Is this enough information? Where to store the uImage file on the
> >> ext4 partition?
> >
> > To store uImage file on ext4/fat/ext2 partition it is enough to only
> > give:
> >
> > uImage mmc 0 2,
> >
> > which maps to the following command:
> >
> > ext4write mmc 0:2 0x44000000 /uImage [size]
> 
> Hmm... and what if I have my uImage in /boot/ ?

Then you need to define NAME in a following way:
/boot/uImage.

I'm also aware, that this is NOT an elegant solution.

> 
> > The file size is taken from number of sent bytes via DFU/USB.
> 
> Yes.
> 
> > With writing to file system, one need to first store the whole
> > transmitted data to a buffer and then store the (big) buffer on the
> > SD/eMMC device.
> 
> Ok, same for an ubi image.
> 
> > Since, we aren't supporting "seek" kind of write to current ext4
> > implementation at u-boot, the "big" buffer must be used.
> 
> Ok, there are places to optimize ;-)

Frankly, handling those large buffers is cumbersome. I've observed that
our current sdhci implementation has some issues with handling such
large transfers.

> 
> >>> root.img | img  | mmc    | 0       | 5        | "-"  | "-"   |
> >>> "-"   |
> >>
> >> img means here: getting an image and storing it on the mtd
> >> partition "Dev_part" if start and size are marked with "-",
> >> beginning on start of the partition?
> >
> > No, here img is for example a rootfs image. When storing it on the
> > eMMC, it would be better to specify destination partition.
> >
> >>
> >> Wouldn't it be better to use here mtd partition names instead
> >> numbers for "Dev_part"?
> >
> > I'd prefer to create a new entrance here (Part_name):
> >
> > NAME | Type | Device | Dev_num | Dev_part | Part_name | FS | start |
> > size |
> >
> > I would like to avoid situation with ambiguous fields. One field
> > shall only serve for one (clear) purpose. When not needed/used -
> > field shall be marked as "-".
> 
> Ok. but this gives long constructs ...

I know, but we also want to clean up it once for a long time. I'm very
open for other ideas :-).

> 
> >> What if "start" and "size" are filled with values for the "Type"
> >> "img"? Or is this forbidden for the "Type" "img"?
> >
> > I think, that each "Type" shall have predefined set of allowed
> > attributes:
> >
> > 1. img: "Device" + "Dev num" + "Dev parts"
> > 2. raw: "Device" + "Dev num" + "start" + "size"
> > 3. file: "Device" + "Dev num" + "Dev parts" + "FS"
> >
> > and so on.
> 
> If we introduce a "Part_name" this should be also possible:
> 
> 4. img: "Device" + "Dev num" + "Part_name"
> 5. file: "Device" + "Dev num" + "Part_name" + "FS"
> 
> or?

Yes. This also looks valid.

> 
> >>> root.img | raw  | mmc    | 0       | "-"      | "-"  | 0x1000|
> >>> 0x4000|
> >>>
> >>> u-boot   | raw  | nand   | 0       | "-"      | "-"  | 0x100 |
> >>> 0x100 | uImage   | file | nand   | 0       | 3        | ubi  |
> >>> "-"   | "-"   |
> >>
> >> s/uImage/rootfs.img ? s/file/img ?
> >
> > NAME: uImage and rootfs.img maps to files sent to board.
> >
> > The "NAME" is used thereafter to provide exact name for file system
> > write (ext4write/ fatwrite).
> 
> Ah! If we have to write in a subdirectory, "NAME" is the fullpath +
> filename?

As stated above, NAME needs to be extended by fullpath.

> 
> > With DFU the distinction between dfu entities (files, raw images,
> > etc) is performed via alt settings (-aX switch at dfu-util).
> >
> >>
> >> For the FS "ubi" we need to specify, how to burn this into nand.
> >> I think we have no "ubi format" command.
> >
> > Is already ubi format command available at u-boot?
> 
> No.
> 
> > _As a side note:_
> >
> > I'm now developing proof-of-concept idea of executing set of
> > commands sent to u-boot by dfu-util [***].
> 
> Ok.
> 
> > Rationale for this is the lack of ability to reset u-boot after
> > sending data to u-boot via DFU. dfu-util has -R option, but this
> > seems to reset usb interface, not the target device.
> 
> That answers my next question which comed in my mind ...

However, as Tormod pointed out we could perform reset at DFU_DEATACH
internal DFU state.

> 
> > I will set an env:
> >
> > setenv dfu "dfu mmc 0;${dfu_commands}"
> >
> > Then at dfu itself, I will read commands to be performed. Then I
> > will set $dfu_commands env and exit from dfu command.
> 
> Hmm... is this not oversized for the "reset board" use case?

This is not only for reset :-). For me it seems possible to send a
bunch of commands via DFU to be executed on by one.

$dfu_commanfs=mmc read 0x44000000 0:2;crc32 0x44000000 0x80;.....
other...commands;reset

But having only reset implemented when dfu-util -R is executed is also
a good idea.

> 
> >> With "ubi write ..."
> >> we can only write ubi volumes ... and we want here to burn an ubi
> >> image, which was created with ubinize and contains one or more ubi
> >> volumes
> >>
> >> Maybe usecase for updating ubi volumes:
> >> ubivolumename   | img | nand   | 0       | 3        | ubivol | "-"
> >> | "-"   |
> >>
> >> If you want to update an ubivolume ...
> >>
> >> results in this steps:
> >> - get image per dfu @ img_addr
> >>     We need to get here the complete image, as ubi write
> >>     cannot write incremental ...
> >
> > So each volume shall have different dfu entity. Then those volumes
> > are stored at ram with different addresses [*]?
> 
> Yes for every volume have different dfu entity.
> You update only one volume with one dfu-util command, or? So
> no different addresses needed ...
> 
> >> - ubi part get_mtd_partition_name("Dev_part=3") vid_header_offset
> >>                                                   ^
> >>                                                   From where get we
> >> this parameter ... value in "start"?
> >
> > So here you get information about the ("Dev_part=3") partition start
> > offset [**]?
> 
> Yes ... what is with the "vid_header_offset" parameter?
> 
> >> - ubi write img_addr ubivolumename ${filesize}
> >
> > Here you combine volumes sent at [*] and store it at correct offset
> > [**]?
> 
> Yes! But just one volume at once ...
> 
> > I'm not an expert about ubi, so please be patient with my
> > questions :-).
> 
> No problem! I also just learning ;-)
> 
> The above steps are just to get the idea, what must be done for
> updating an ubi image, as this were the steps if you type in commands.
> 
> > For me it looks like a mixture of ordinary u-boot commands with dfu
> > transfer[s] needed to perform correct ubi image write. Maybe [***]
> > will help you?
> 
> If we have the dfu_buf address availiable ... maybe it is worth a
> try ... thinking of sending per dfu a "run upd_ubi" and the
> "upd_ubi" command do the complete update ... but, if we update
> more than one image ... this will not work.

Ok.

> 
> bye,
> Heiko



-- 
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Marek Vasut]

Dear Heiko Schocher,

> Hello Lukasz,
> 
> Am 16.07.2013 17:35, schrieb Lukasz Majewski:
> > Dear All,
> > 
> > Since DFU usage at u-boot is spreading to different device types (MMC,
> > NAND), file systems, raw partitions, ubi, etc, I think that it is a
> > good moment to unify and structure the form of dfu_alt_info environment
> > variable.
> 
> Full Ack!
> 
> > Proposed new format for single dfu entity:
> > 
> > NAME     | Type | Device | Dev_num | Dev_part | FS   | start | size  |
> > 
> > where:
> > 
> > Name     - name of the alt setting (as seen by dfu-util -l)
> > Type     - description of the image (raw, file, img, command [*])
> > Device   - physical device on which data are stored (mmc, nand, "-")
> > Dev_num  - number of the device - it is possible to store data via DFU
> > 
> >             on several devices of the same type.
> > 
> > Dev_part - number of partitions on the device.
> 
> Should this be "number of the partition on the device"
> 
> You mean here the mtd partition for storing, right?
> 
> > FS       - information about file system on the device (fat,
> > 
> > 	   ext2/ext4, ubi).
> > 
> > start    - start offset from the beginning of the Device (byte or LBA
> > 
> >             number addressed)
> > 
> > size     - maximal number of blocks to be stored on the Device
> > 
> >   	   (expressed with Bytes of LBA number) (protection from
> >   	   overwriting the whole device)
> > 
> > Example:
> Maybe dummy questions ...
> 
> > NAME     | Type | Device | Dev_num | Dev_part | FS   | start | size  |
> > ----------------------------------------------------------------------
> > u-boot   | raw  | mmc    | 0       | "-" [**] | "-"  | 0x80  | 0x100 |
> > uImage   | file | mmc    | 0       | 2        | ext4 | "-"   | "-"   |
> 
> Is this enough information? Where to store the uImage file on the ext4
> partition?
> 
> > root.img | img  | mmc    | 0       | 5        | "-"  | "-"   | "-"   |
> 
> img means here: getting an image and storing it on the mtd partition
> "Dev_part" if start and size are marked with "-", beginning
> on start of the partition?
> 
> Wouldn't it be better to use here mtd partition names instead
> numbers for "Dev_part"?
> 
> What if "start" and "size" are filled with values for the "Type" "img"?
> Or is this forbidden for the "Type" "img"?
> 
> > root.img | raw  | mmc    | 0       | "-"      | "-"  | 0x1000| 0x4000|
> > 
> > u-boot   | raw  | nand   | 0       | "-"      | "-"  | 0x100 | 0x100 |
> > uImage   | file | nand   | 0       | 3        | ubi  | "-"   | "-"   |
> 
> s/uImage/rootfs.img ? s/file/img ?
> 
> For the FS "ubi" we need to specify, how to burn this into nand.
> I think we have no "ubi format" command.

Try "nand write.trimffs" to write UBI images produced with ubinize .

Best regards,
Marek Vasut

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Heiko Schocher]

Hello Marek,

Am 18.07.2013 06:17, schrieb Marek Vasut:
> Dear Heiko Schocher,
>
>> Hello Lukasz,
>>
>> Am 16.07.2013 17:35, schrieb Lukasz Majewski:
>>> Dear All,
>>>
>>> Since DFU usage at u-boot is spreading to different device types (MMC,
>>> NAND), file systems, raw partitions, ubi, etc, I think that it is a
>>> good moment to unify and structure the form of dfu_alt_info environment
>>> variable.
>>
>> Full Ack!
>>
>>> Proposed new format for single dfu entity:
>>>
>>> NAME     | Type | Device | Dev_num | Dev_part | FS   | start | size  |
>>>
>>> where:
>>>
>>> Name     - name of the alt setting (as seen by dfu-util -l)
>>> Type     - description of the image (raw, file, img, command [*])
>>> Device   - physical device on which data are stored (mmc, nand, "-")
>>> Dev_num  - number of the device - it is possible to store data via DFU
>>>
>>>              on several devices of the same type.
>>>
>>> Dev_part - number of partitions on the device.
>>
>> Should this be "number of the partition on the device"
>>
>> You mean here the mtd partition for storing, right?
>>
>>> FS       - information about file system on the device (fat,
>>>
>>> 	   ext2/ext4, ubi).
>>>
>>> start    - start offset from the beginning of the Device (byte or LBA
>>>
>>>              number addressed)
>>>
>>> size     - maximal number of blocks to be stored on the Device
>>>
>>>    	   (expressed with Bytes of LBA number) (protection from
>>>    	   overwriting the whole device)
>>>
>>> Example:
>> Maybe dummy questions ...
>>
>>> NAME     | Type | Device | Dev_num | Dev_part | FS   | start | size  |
>>> ----------------------------------------------------------------------
>>> u-boot   | raw  | mmc    | 0       | "-" [**] | "-"  | 0x80  | 0x100 |
>>> uImage   | file | mmc    | 0       | 2        | ext4 | "-"   | "-"   |
>>
>> Is this enough information? Where to store the uImage file on the ext4
>> partition?
>>
>>> root.img | img  | mmc    | 0       | 5        | "-"  | "-"   | "-"   |
>>
>> img means here: getting an image and storing it on the mtd partition
>> "Dev_part" if start and size are marked with "-", beginning
>> on start of the partition?
>>
>> Wouldn't it be better to use here mtd partition names instead
>> numbers for "Dev_part"?
>>
>> What if "start" and "size" are filled with values for the "Type" "img"?
>> Or is this forbidden for the "Type" "img"?
>>
>>> root.img | raw  | mmc    | 0       | "-"      | "-"  | 0x1000| 0x4000|
>>>
>>> u-boot   | raw  | nand   | 0       | "-"      | "-"  | 0x100 | 0x100 |
>>> uImage   | file | nand   | 0       | 3        | ubi  | "-"   | "-"   |
>>
>> s/uImage/rootfs.img ? s/file/img ?
>>
>> For the FS "ubi" we need to specify, how to burn this into nand.
>> I think we have no "ubi format" command.
>
> Try "nand write.trimffs" to write UBI images produced with ubinize .

This solves not the erasecounter problem, or?

For UBI we need something like this:
http://www.linux-mtd.infradead.org/doc/ubi.html#L_flasher_algo

But I am not an UBI expert. It is possible I overlook something
obvious ...

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Wolfgang Denk]

Dear Heiko Schocher,

In message <51E77A1D.90403@denx.de> you wrote:
> 
> > Try "nand write.trimffs" to write UBI images produced with ubinize .
> 
> This solves not the erasecounter problem, or?
> 
> For UBI we need something like this:
> http://www.linux-mtd.infradead.org/doc/ubi.html#L_flasher_algo
> 
> But I am not an UBI expert. It is possible I overlook something
> obvious ...

No, you don't.  Devices managed by UBI should never be erased by
other, non-UBI-aware tools.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
I will also, for an appropriate fee, certify that  your  keyboard  is
object-oriented,  and  that  the bits on your hard disk are template-
compatible.            - Jeffrey S. Haemer in <411akr$3ga@cygnus.com>

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Marek Vasut]

Hi,

> Dear Heiko Schocher,
> 
> In message <51E77A1D.90403@denx.de> you wrote:
> > > Try "nand write.trimffs" to write UBI images produced with ubinize .
> > 
> > This solves not the erasecounter problem, or?
> > 
> > For UBI we need something like this:
> > http://www.linux-mtd.infradead.org/doc/ubi.html#L_flasher_algo
> > 
> > But I am not an UBI expert. It is possible I overlook something
> > obvious ...
> 
> No, you don't.  Devices managed by UBI should never be erased by
> other, non-UBI-aware tools.

I based my reply on the following commit in U-Boot and the fact that 
write.trimffs is used to flash UBI images. Maybe I was wrong?

commit c9494866df835bcee68e17339aec1090faa704da
Author: Ben Gardiner <bengardiner@nanometrics.ca>
Date:   Tue Jun 14 16:35:07 2011 -0400

    cmd_nand: add nand write.trimffs command
    
    Add another nand write. variant, trimffs. This command will request of
    nand_write_skip_bad() that all trailing all-0xff pages will be
    dropped from eraseblocks when they are written to flash as-per the
    reccommended behaviour of the UBI FAQ [1].
    
    The function that implements this timming is the drop_ffs() function
    by Artem Bityutskiy, ported from the mtd-utils tree.
    
    [1] http://www.linux-mtd.infradead.org/doc/ubi.html#L_flasher_algo

Best regards,
Marek Vasut

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Heiko Schocher]

Hello Marek,

Am 18.07.2013 17:10, schrieb Marek Vasut:
> Hi,
>
>> Dear Heiko Schocher,
>>
>> In message<51E77A1D.90403@denx.de>  you wrote:
>>>> Try "nand write.trimffs" to write UBI images produced with ubinize .
>>>
>>> This solves not the erasecounter problem, or?
>>>
>>> For UBI we need something like this:
>>> http://www.linux-mtd.infradead.org/doc/ubi.html#L_flasher_algo
>>>
>>> But I am not an UBI expert. It is possible I overlook something
>>> obvious ...
>>
>> No, you don't.  Devices managed by UBI should never be erased by
>> other, non-UBI-aware tools.
>
> I based my reply on the following commit in U-Boot and the fact that
> write.trimffs is used to flash UBI images. Maybe I was wrong?
>
> commit c9494866df835bcee68e17339aec1090faa704da
> Author: Ben Gardiner<bengardiner@nanometrics.ca>
> Date:   Tue Jun 14 16:35:07 2011 -0400
>
>      cmd_nand: add nand write.trimffs command
>
>      Add another nand write. variant, trimffs. This command will request of
>      nand_write_skip_bad() that all trailing all-0xff pages will be
>      dropped from eraseblocks when they are written to flash as-per the
>      reccommended behaviour of the UBI FAQ [1].
>
>      The function that implements this timming is the drop_ffs() function
>      by Artem Bityutskiy, ported from the mtd-utils tree.
>
>      [1] http://www.linux-mtd.infradead.org/doc/ubi.html#L_flasher_algo

Yes, that sounds as a step in the right direction, but where are
the erasecounters handled, as described in [1] ?

And as this is a "ubi function" and not nand specific, the command
should start with "ubi ..." ... as we have a "ubi write ...", but
ubi write is only for ubi volumes ... i tend to say, we need a
"ubi format ..." similiar to ubiformat in the mtd utils [2] ...

[2] http://git.infradead.org/mtd-utils.git
     ubiformat found in /ubi-utils/ubiformat.c

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Marek Vasut]

Dear Heiko Schocher,

> Hello Marek,
> 
> Am 18.07.2013 17:10, schrieb Marek Vasut:
> > Hi,
> > 
> >> Dear Heiko Schocher,
> >> 
> >> In message<51E77A1D.90403@denx.de>  you wrote:
> >>>> Try "nand write.trimffs" to write UBI images produced with ubinize .
> >>> 
> >>> This solves not the erasecounter problem, or?
> >>> 
> >>> For UBI we need something like this:
> >>> http://www.linux-mtd.infradead.org/doc/ubi.html#L_flasher_algo
> >>> 
> >>> But I am not an UBI expert. It is possible I overlook something
> >>> obvious ...
> >> 
> >> No, you don't.  Devices managed by UBI should never be erased by
> >> other, non-UBI-aware tools.
> > 
> > I based my reply on the following commit in U-Boot and the fact that
> > write.trimffs is used to flash UBI images. Maybe I was wrong?
> > 
> > commit c9494866df835bcee68e17339aec1090faa704da
> > Author: Ben Gardiner<bengardiner@nanometrics.ca>
> > Date:   Tue Jun 14 16:35:07 2011 -0400
> > 
> >      cmd_nand: add nand write.trimffs command
> >      
> >      Add another nand write. variant, trimffs. This command will request
> >      of nand_write_skip_bad() that all trailing all-0xff pages will be
> >      dropped from eraseblocks when they are written to flash as-per the
> >      reccommended behaviour of the UBI FAQ [1].
> >      
> >      The function that implements this timming is the drop_ffs() function
> >      by Artem Bityutskiy, ported from the mtd-utils tree.
> >      
> >      [1] http://www.linux-mtd.infradead.org/doc/ubi.html#L_flasher_algo
> 
> Yes, that sounds as a step in the right direction, but where are
> the erasecounters handled, as described in [1] ?

I don't think they're handled anywhere.

> And as this is a "ubi function" and not nand specific, the command
> should start with "ubi ..." ... as we have a "ubi write ...", but
> ubi write is only for ubi volumes ... i tend to say, we need a
> "ubi format ..." similiar to ubiformat in the mtd utils [2] ...
> 
> [2] http://git.infradead.org/mtd-utils.git
>      ubiformat found in /ubi-utils/ubiformat.c

Full agreement.

Best regards,
Marek Vasut

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Tom Rini]

On Tue, Jul 16, 2013 at 05:35:21PM +0200, Lukasz Majewski wrote:

> Dear All,
> 
> Since DFU usage at u-boot is spreading to different device types (MMC,
> NAND), file systems, raw partitions, ubi, etc, I think that it is a
> good moment to unify and structure the form of dfu_alt_info environment
> variable.
[snip]
> NAME     | Type | Device | Dev_num | Dev_part | FS   | start | size  | 
[snip]
> u-boot   | raw  | nand   | 0       | "-"      | "-"  | 0x100 | 0x100 |
> uImage   | file | nand   | 0       | 3        | ubi  | "-"   | "-"   | 

We also need to cover:

uImage     | raw  | nand   | 0       | "kernel" | "-"  | "-"   | "-"   |

Since partitions provide start/size.  And I'm not totally sure we're
providing enough information for 'destination resides in this part of
UBI container'.  It could be "/boot/uImage in rootfs in ubi0" or "raw to
kernel0 in ubi0".

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20130718/4f91d63a/attachment.pgp>

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Michael Cashwell]

On Jul 18, 2013, at 12:39 PM, Tom Rini <trini@ti.com> wrote:

> uImage     | raw  | nand   | 0       | "kernel" | "-"  | "-"   | "-"   |
> 
> Since partitions provide start/size.

I've got some WIP that pulls the alt info from a GPT partition map on mmc. That alt settings in DFU can be strings or numbers was handy to allow the name to identify the partition.

-Mike

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Lukasz Majewski]

On Thu, 18 Jul 2013 13:30:55 -0400
Michael Cashwell <mboards@prograde.net> wrote:

> On Jul 18, 2013, at 12:39 PM, Tom Rini <trini@ti.com> wrote:
> 
> > uImage     | raw  | nand   | 0       | "kernel" | "-"  | "-"   |
> > "-"   |
> > 
> > Since partitions provide start/size.
> 
> I've got some WIP that pulls the alt info from a GPT partition map on
> mmc. 

Extraction the part of dfu_alt_info from GPT or $partitions env variable
(as defined at e.g. Trats) is also a good idea.

The best solution would be to produce alt settings information as a mix
of GPT layout information (extraction of part num, start offset,
size... any more?) [*] and some predefined dfu_alt_info env variable. In
this way we can reuse already available code (since GPT layout is
already in place for most of targets).

But what about mtd parts (Nand/ UBI)? It shall be also doable to
extract information [*] from it. Am I correct?

> That alt settings in DFU can be strings or numbers was handy to
> allow the name to identify the partition.
> 
> -Mike
> 
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> http://lists.denx.de/mailman/listinfo/u-boot

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20130718/3b7a9e9c/attachment.pgp>

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Tom Rini]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/18/2013 04:17 PM, Lukasz Majewski wrote:
> On Thu, 18 Jul 2013 13:30:55 -0400 Michael Cashwell
> <mboards@prograde.net> wrote:
> 
>> On Jul 18, 2013, at 12:39 PM, Tom Rini <trini@ti.com> wrote:
>> 
>>> uImage     | raw  | nand   | 0       | "kernel" | "-"  | "-"
>>> | "-"   |
>>> 
>>> Since partitions provide start/size.
>> 
>> I've got some WIP that pulls the alt info from a GPT partition
>> map on mmc.
> 
> Extraction the part of dfu_alt_info from GPT or $partitions env
> variable (as defined at e.g. Trats) is also a good idea.
> 
> The best solution would be to produce alt settings information as a
> mix of GPT layout information (extraction of part num, start
> offset, size... any more?) [*] and some predefined dfu_alt_info env
> variable. In this way we can reuse already available code (since
> GPT layout is already in place for most of targets).
> 
> But what about mtd parts (Nand/ UBI)? It shall be also doable to 
> extract information [*] from it. Am I correct?

Right, we support that today even in DFU.

- -- 
Tom
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJR6G00AAoJENk4IS6UOR1WA9cQAKRC0fOrdoVcQyrqeAzf+ahC
y18UHlFsmS+0fVzJzc0z3m7RiUYNCFEsNMm2JFedss7xgpP7GCwMDLRcLhy4CkEW
m5hjISL/C3pJKoAbCocKcUe4TD53Vmukr+sBPwKUR2N62iRcVYCO0Ved8P/4QXUz
g+Ylc8F6C6pPXxTEi8cn8oigmZzckJqn02AjfhY474uQZvKNezM1eA6uBOAbCugM
RJYjM+AxjOeL4dNYJTQEcmLAqcQPF0KW9yxfbMYi79h09qVJTqZIyvIHVIX4XfUp
E64yh21LhuhEyKxUnMcZbvcn5RQ2CvROY2a121D+hFWCxrnLeumIA49Hz8ilVYBt
1P6qzNotUbciOr9M33pedFOziYua/0Y6Wy4f5sMqYOaCy+qhO9KnaiJe0SkOUYah
XNMZ5tVMOAhCwDhmnmLSDWfZZMl9LBzibzIsw7qPzb/ZYtwA7CEa9WSSwZVw7Zsj
jj0+/j4uis4a4UOwj1E7ui3CgiRLu7ck5vBBTJ1VJlRvdCvBknio2aJ3/ZcJaXjc
U76Aly890n4EfwlclwRpeXVGg2eObsherz6gOuWqmVmjBYuiwQAKiT7X4VnqtWmM
V6yB6VmOKO5Am9GmpaWplep8l1vJgkmWd++3kwR75NgZIgmizwIpRxG56hd+oav8
0MH2gBeO6rZo/WdnJwJS
=IYIY
-----END PGP SIGNATURE-----

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Lukasz Majewski]

On Thu, 18 Jul 2013 13:30:55 -0400 Michael Cashwell
mboards at prograde.net wrote,
> On Jul 18, 2013, at 12:39 PM, Tom Rini <trini@ti.com> wrote:
> 
> > uImage     | raw  | nand   | 0       | "kernel" | "-"  | "-"   |
> > "-"   |
> > 
> > Since partitions provide start/size.
> 

Hi Michael,

> I've got some WIP that pulls the alt info from a GPT partition map on
> mmc. That alt settings in DFU can be strings or numbers was handy to
> allow the name to identify the partition.

Will you be ready to show the mentioned patch in a near future?

> 
> -Mike



-- 
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Lukasz Majewski]

Dear All,

As a follow up from u-boot's mini-summit at ELCE 2013, I would like to
once again share the new format of dfu_alt_info env variable (for those
who couldn't attend).

dfu_alt_info_cleanup:

    |---+-------------+-------------+---------+----------------------|
    |   | <dev_name>: | <alt_name>, | <type>  | extra params         |
    |---+-------------+-------------+---------+----------------------|
    | 1 | mmc0:       | rootfs,     | part;   | -                    |
    | 2 | mmc0:       | uImage,     | file,   | part, dir;           |
    | 3 | nand0:      | u-boot.bin, | raw,    | start, size;         |
    | 4 | mmc0:       | u-boot.bin, | raw,    | start_lba, size_lba; |
    | 5 | ram0:       | uImage,     | ram,    | start, size;         |
    | 6 | ubi0:       | kernel0,    | raw;    | -                    |
    |---+-------------+-------------+---------+----------------------|
    | 7 | cmd:        | cmd,        | cmd;    | -                    |


Some useful info:

+ Each <type> is responsible for providing information
  about additional parameters (in conjunction with <dev_name>)

+ Ad 1. Extra parameters are extracted from GPT/MBR/MTD

+ Part can be given as text or number (text -> number conversion will
  be performed anyway)

+ Part: Ext2/4, FAT, UBI?

I'm especially interested in feedback from AM335x owners. I would
like to know how UBI partitions are handled there.

-- 
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Wolfgang Denk]

Dear Lukasz,

In message <20131031182541.3a3020c6@amdc308.digital.local> you wrote:
> 
> As a follow up from u-boot's mini-summit at ELCE 2013, I would like to
> once again share the new format of dfu_alt_info env variable (for those
> who couldn't attend).
> 
> dfu_alt_info_cleanup:
> 
>     |---+-------------+-------------+---------+----------------------|
>     |   | <dev_name>: | <alt_name>, | <type>  | extra params         |
>     |---+-------------+-------------+---------+----------------------|
>     | 1 | mmc0:       | rootfs,     | part;   | -                    |
>     | 2 | mmc0:       | uImage,     | file,   | part, dir;           |
>     | 3 | nand0:      | u-boot.bin, | raw,    | start, size;         |
>     | 4 | mmc0:       | u-boot.bin, | raw,    | start_lba, size_lba; |
>     | 5 | ram0:       | uImage,     | ram,    | start, size;         |
>     | 6 | ubi0:       | kernel0,    | raw;    | -                    |
>     |---+-------------+-------------+---------+----------------------|
>     | 7 | cmd:        | cmd,        | cmd;    | -                    |

Why can we not have a common format for all media supporting the "raw"
type?  I mean, why can we not also use "start, size" for mmc?  It
should be trivial to convert into a LBA number.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"Life sucks, but it's better than the alternative."
- Peter da Silva

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Lukasz Majewski]

Hi Wolfgang,

> Dear Lukasz,
> 
> In message <20131031182541.3a3020c6@amdc308.digital.local> you wrote:
> > 
> > As a follow up from u-boot's mini-summit at ELCE 2013, I would like
> > to once again share the new format of dfu_alt_info env variable
> > (for those who couldn't attend).
> > 
> > dfu_alt_info_cleanup:
> > 
> >     |---+-------------+-------------+---------+----------------------|
> >     |   | <dev_name>: | <alt_name>, | <type>  | extra
> > params         |
> > |---+-------------+-------------+---------+----------------------|
> > | 1 | mmc0:       | rootfs,     | part;   | -                    |
> > | 2 | mmc0:       | uImage,     | file,   | part, dir;           |
> > | 3 | nand0:      | u-boot.bin, | raw,    | start, size;         |
> > | 4 | mmc0:       | u-boot.bin, | raw,    | start_lba, size_lba; |
> > | 5 | ram0:       | uImage,     | ram,    | start, size;         |
> > | 6 | ubi0:       | kernel0,    | raw;    | -                    |
> > |---+-------------+-------------+---------+----------------------|
> > | 7 | cmd:        | cmd,        | cmd;    | -                    |
> 
> Why can we not have a common format for all media supporting the "raw"
> type?  I mean, why can we not also use "start, size" for mmc?  It
> should be trivial to convert into a LBA number.

I think, that I could do better with this table.

What I have meant, is that the "raw" extra parameters are the same in
both cases. The interpretation of start and size (if they are LBA or
"normal" addresses) depends on the <dev_name>.


> 
> Best regards,
> 
> Wolfgang Denk
> 

Best regards,
Lukasz Majewski
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20131031/3abb1002/attachment.pgp>

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Wolfgang Denk]

Dear Lukasz,

In message <20131031222010.082d453f@jawa> you wrote:
>
> > Why can we not have a common format for all media supporting the "raw"
> > type?  I mean, why can we not also use "start, size" for mmc?  It
> > should be trivial to convert into a LBA number.
> 
> I think, that I could do better with this table.
> 
> What I have meant, is that the "raw" extra parameters are the same in
> both cases. The interpretation of start and size (if they are LBA or
> "normal" addresses) depends on the <dev_name>.

I think they should _not_ be interpreted differently. I would expect
that "start" is an offset in bytes from the beginning of the device,
for all of the cases.  Similar, "size" would be the number of bytes,
for all cases - bot bytes here and sectors there.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
In the bathtub of history the truth is harder to hold than the  soap,
and much more difficult to find ...     - Terry Pratchett, _Sourcery_

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Lukasz Majewski]

Hi Wolfgang,

> Dear Lukasz,
> 
> In message <20131031222010.082d453f@jawa> you wrote:
> >
> > > Why can we not have a common format for all media supporting the
> > > "raw" type?  I mean, why can we not also use "start, size" for
> > > mmc?  It should be trivial to convert into a LBA number.
> > 
> > I think, that I could do better with this table.
> > 
> > What I have meant, is that the "raw" extra parameters are the same
> > in both cases. The interpretation of start and size (if they are
> > LBA or "normal" addresses) depends on the <dev_name>.
> 
> I think they should _not_ be interpreted differently. I would expect
> that "start" is an offset in bytes from the beginning of the device,
> for all of the cases.  Similar, "size" would be the number of bytes,
> for all cases - bot bytes here and sectors there.

Ok, lets do it in this way. LBA conversion will be done just before
eMMC access.

> 
> Best regards,
> 
> Wolfgang Denk
> 



-- 
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group

[U-Boot] [RFC][DFU] Unification of dfu_alt_info alt settings description + command execution

[Heiko Schocher]

Hello Lukasz,

Am 31.10.2013 18:25, schrieb Lukasz Majewski:
> Dear All,
>
> As a follow up from u-boot's mini-summit at ELCE 2013, I would like to
> once again share the new format of dfu_alt_info env variable (for those
> who couldn't attend).
>
> dfu_alt_info_cleanup:
>
>      |---+-------------+-------------+---------+----------------------|
>      |   |<dev_name>: |<alt_name>, |<type>   | extra params         |
>      |---+-------------+-------------+---------+----------------------|
>      | 1 | mmc0:       | rootfs,     | part;   | -                    |
>      | 2 | mmc0:       | uImage,     | file,   | part, dir;           |
>      | 3 | nand0:      | u-boot.bin, | raw,    | start, size;         |

One more usecase for nand (just for documentation):

        | 4 | nand0:      | u-boot      | part,   | -                    |

Here is "alt_name" == "mtd partition name" -> start, size from mtd info

>      | 4 | mmc0:       | u-boot.bin, | raw,    | start_lba, size_lba; |
>      | 5 | ram0:       | uImage,     | ram,    | start, size;         |
>      | 6 | ubi0:       | kernel0,    | raw;    | -                    |

There are 2 usecases with ubi (Please correct me, if I am wrong):

a) update a complete ubi device:

        | 6 | ubi0:      | kernel,     | part    | -                    |
        | 7 | ubi0:      | kernel,     | raw     | start, size          |

with them, we can update a complete "ubi device" with n ubi volumes
in it (This is currently used on the am335x based siemens boards).

I am not sure if ubi is more a "device" or more a "type". But as ubi can
be used also on nor flash, I think in the meantime, it is more a device
than a type in the sight of dfu ... but I vary here in my opinion...

This usecase is currently implemented for nand, and same code is
used as for "raw" or "part" nand usecases, just in the ubi case a
flag is set, which does:

- erase nand sectors with WITH_DROP_FFS
- erase spare sectors at the end of the area

b) update single ubi volume.

We need this infos for updating a ubi volume

- ubi device name = mtd partition name
- vid header offset
- ubi volume name
- ubi volume size
- ubi volume type

- steps for updating a ubi volume
   - attach to an ubi device
     "ubi part device_name vid_header_offset"
   - if volume_name !exist, create it
     "ubi create volume_name size type"
   - write ubi volume
     "ubi write img_addr volume_name size"


        | 8 | ubi0:      | data,        | volume  | vid_off, vol_nam, vol_sz, vol_type |
                           ^
                           device name
   You see, the list gets long ... or maybe we can cover this usecase
   with dfu command?

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany