[Buildroot] [PATCH] Introduce support for using either e2fsimage of genext2fs

[Buildroot] [PATCH] Introduce support for using either e2fsimage of genext2fs

[Dan Christensen]

Hello,

I recently started using buildroot-2009.08 for a project using an AT91
series ARM CPU and wanted to boot off of an SD card. ?This was going
quite well until I ran into a few problems with U-Boot booting the
image I'd written to the SD card. ?It would boot fine off of FAT, and
an ext2 filesystem I'd created with mkfs.ext2, but not the filesystem
I copied to the card with dd.

So I started investigating the differences between the two images I
had with tune2fs, and it seems as though genext2fs makes a much
"simpler" ext2 filesystem than mkfs.ext2 does today. ?I'm not sure
about the history or if there is particular significance behind the
way it generates the filesystem, if there is any at all. ?This led me
to another tool called e2fsimage. ?e2fsimage seems to generate a
filesystem more like today's mkfs.ext2 generates.

The task of incorporating this additional tool into buildroot seemed
straight forward, but ultimately led to reorganizing the target/ext2/
directory quite a bit. ?I tried to follow the conventions that I'd
seen elsewhere, but wasn't sure what guidelines I should be following.

That being said, I would appreciate it if the following patch could be
reviewed for correctness. ?It worked for me (I could toggle between
genext2fs and e2fsimage) but I reworked quite a bit of the logic in
the makefiles and wouldn't be surprised if there were some bugs I
hadn't noticed.

Thanks, and I hope this is helpful.
Dan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: buildroot-2009.08-e2fsimage.patch
Type: application/octet-stream
Size: 20451 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20091021/80321df8/attachment-0001.obj>

[Buildroot] [PATCH] Introduce support for using either e2fsimage of genext2fs

[Dan Christensen]

Hello.

On Wed, Oct 21, 2009 at 7:16 PM, Dan Christensen <opello@opello.org> wrote:
> I recently started using buildroot-2009.08 for a project using an AT91
> series ARM CPU and wanted to boot off of an SD card. ?This was going
> quite well until I ran into a few problems with U-Boot booting the
> image I'd written to the SD card. ?It would boot fine off of FAT, and
> an ext2 filesystem I'd created with mkfs.ext2, but not the filesystem
> I copied to the card with dd.
>
> So I started investigating the differences between the two images I
> had with tune2fs, and it seems as though genext2fs makes a much
> "simpler" ext2 filesystem than mkfs.ext2 does today. ?I'm not sure
> about the history or if there is particular significance behind the
> way it generates the filesystem, if there is any at all. ?This led me
> to another tool called e2fsimage. ?e2fsimage seems to generate a
> filesystem more like today's mkfs.ext2 generates.
>
> The task of incorporating this additional tool into buildroot seemed
> straight forward, but ultimately led to reorganizing the target/ext2/
> directory quite a bit. ?I tried to follow the conventions that I'd
> seen elsewhere, but wasn't sure what guidelines I should be following.
>
> That being said, I would appreciate it if the following patch could be
> reviewed for correctness. ?It worked for me (I could toggle between
> genext2fs and e2fsimage) but I reworked quite a bit of the logic in
> the makefiles and wouldn't be surprised if there were some bugs I
> hadn't noticed.

I was hoping to get some feedback on this, and had some further ideas.
 But I didn't see anyone reply.

I reworked this patch to be from the latest git as of a few minutes
ago.  I hope this allows someone to examine it more easily, and
comment on it.

I did end up with a few questions about how e2fsimage's dependencies
should be met.  It depends on e2fslibs-dev (debian package name), and
I see that e2fsprogs is already a package built for the target.  I was
wondering if this could be build for the host as well, similar to how
fakeroot has both a 'target' and a 'host' configuration.  I wasn't
sure how this would work with the new Makefile.autotools.in changes in
e2fsprogs.mk.
The way I solved this for myself was adding support to build it to my
own e2fsprogs.mk (not included in this patch as it doesn't seem the
correct way to manage the dependency).

The other significant question I had was the CFLAGS that should be
used for e2fsimage.  I simply copied the ones used for genext2fs
(-Wall -O2 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
-D_FILE_OFFSET_BITS=64) and it seemed to work.

Thanks for your time.
Dan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ext2-reorganize-and-add-support-for-e2fsimage.patch
Type: application/octet-stream
Size: 17062 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20091102/9f3b66a1/attachment-0001.obj>

[Buildroot] [PATCH] Introduce support for using either e2fsimage of genext2fs

[Will Newton]

On Tue, Nov 3, 2009 at 2:08 AM, Dan Christensen <opello@opello.org> wrote:

Hi Dan,

I don't personally use ext2 filesystems myself, so I can't really
comment on one tool versus the other, but on the whole your patch
looks like it makes sense.

> I was hoping to get some feedback on this, and had some further ideas.
> ?But I didn't see anyone reply.
>
> I reworked this patch to be from the latest git as of a few minutes
> ago. ?I hope this allows someone to examine it more easily, and
> comment on it.
>
> I did end up with a few questions about how e2fsimage's dependencies
> should be met. ?It depends on e2fslibs-dev (debian package name), and
> I see that e2fsprogs is already a package built for the target. ?I was
> wondering if this could be build for the host as well, similar to how
> fakeroot has both a 'target' and a 'host' configuration. ?I wasn't
> sure how this would work with the new Makefile.autotools.in changes in
> e2fsprogs.mk.
> The way I solved this for myself was adding support to build it to my
> own e2fsprogs.mk (not included in this patch as it doesn't seem the
> correct way to manage the dependency).

I think the buildroot way is to try and minimize the dependencies
required on the host system, so we build host dependencies where
possible. The autotools infrastructure is not currently capable of
building host packages, although I think Thomas's upcoming changes
will likely fix that.

> The other significant question I had was the CFLAGS that should be
> used for e2fsimage. ?I simply copied the ones used for genext2fs
> (-Wall -O2 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
> -D_FILE_OFFSET_BITS=64) and it seemed to work.

Do we need to set CFLAGS at all? Won't configure take care of setting
some appropriate flags for us?

[Buildroot] [PATCH] Introduce support for using either e2fsimage of genext2fs

[Dan Christensen]

Thanks for the reply, Will.

On Tue, Nov 3, 2009 at 4:20 AM, Will Newton <will.newton@gmail.com> wrote:
> On Tue, Nov 3, 2009 at 2:08 AM, Dan Christensen <opello@opello.org> wrote:

> I don't personally use ext2 filesystems myself, so I can't really
> comment on one tool versus the other, but on the whole your patch
> looks like it makes sense.

That's good news.

>> I did end up with a few questions about how e2fsimage's dependencies
>> should be met. ?It depends on e2fslibs-dev (debian package name), and
>> I see that e2fsprogs is already a package built for the target. ?I was
>> wondering if this could be build for the host as well, similar to how
>> fakeroot has both a 'target' and a 'host' configuration. ?I wasn't
>> sure how this would work with the new Makefile.autotools.in changes in
>> e2fsprogs.mk.
>> The way I solved this for myself was adding support to build it to my
>> own e2fsprogs.mk (not included in this patch as it doesn't seem the
>> correct way to manage the dependency).
>
> I think the buildroot way is to try and minimize the dependencies
> required on the host system, so we build host dependencies where
> possible. The autotools infrastructure is not currently capable of
> building host packages, although I think Thomas's upcoming changes
> will likely fix that.

I'll keep a look out for those changes.

>> The other significant question I had was the CFLAGS that should be
>> used for e2fsimage. ?I simply copied the ones used for genext2fs
>> (-Wall -O2 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
>> -D_FILE_OFFSET_BITS=64) and it seemed to work.
>
> Do we need to set CFLAGS at all? Won't configure take care of setting
> some appropriate flags for us?

The configure is pretty minimal (not autotools based) but it did work
fine in my tests, so probably not necessary.
I updated the patch to only pass -Wall -O2, since those still seem sensible.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ext2-reorganize-and-add-support-for-e2fsimage.patch
Type: application/octet-stream
Size: 16992 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20091103/067887e8/attachment-0001.obj>

[Buildroot] [PATCH] Introduce support for using either e2fsimage of genext2fs

[Peter Korsgaard]

>>>>> "Dan" == Dan Christensen <opello@opello.org> writes:

Hi,

 >> That being said, I would appreciate it if the following patch could be
 >> reviewed for correctness. ?It worked for me (I could toggle between
 >> genext2fs and e2fsimage) but I reworked quite a bit of the logic in
 >> the makefiles and wouldn't be surprised if there were some bugs I
 >> hadn't noticed.

 Dan> I was hoping to get some feedback on this, and had some further ideas.
 Dan>  But I didn't see anyone reply.

 Dan> I reworked this patch to be from the latest git as of a few minutes
 Dan> ago.  I hope this allows someone to examine it more easily, and
 Dan> comment on it.

First of all, sorry for the slow response.

I would prefer to have a single tool for ext2 instead of having to
maintain both genext2fs and e2fsimage, but I'm afraid e2fsimage isn't a
good option. According to it's website it hasn't seen a release since
2004, and it isn't in Debian (genext2fs is), and until now we haven't
had any issues with genext2fs.

The ext2 read issue you're seing in u-boot is because u-boot is
buggy. A patch has recently been posted which fixes it:

http://thread.gmane.org/gmane.comp.boot-loaders.u-boot/71890

Also see the thread leading up to that patch:

http://thread.gmane.org/gmane.comp.boot-loaders.u-boot/71863/focus=71867

FYI, it was afaik broken last year when revision 1 support was added:

http://thread.gmane.org/gmane.comp.boot-loaders.u-boot/44630/focus=44727

Could you check if a genext2fs-generated image works for you after you
apply that patch to u-boot?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH] Introduce support for using either e2fsimage of genext2fs

[Dan Christensen]

Hello,

On Sun, Nov 22, 2009 at 1:17 PM, Peter Korsgaard <jacmet@uclibc.org> wrote:
> I would prefer to have a single tool for ext2 instead of having to
> maintain both genext2fs and e2fsimage, but I'm afraid e2fsimage isn't a
> good option. According to it's website it hasn't seen a release since
> 2004, and it isn't in Debian (genext2fs is), and until now we haven't
> had any issues with genext2fs.

That's perfectly understandable, I just wasn't sure how to resolve the
problem and that was the avenue I took.

> The ext2 read issue you're seing in u-boot is because u-boot is
> buggy. A patch has recently been posted which fixes it:
>
> http://thread.gmane.org/gmane.comp.boot-loaders.u-boot/71890
>
> Also see the thread leading up to that patch:
>
> http://thread.gmane.org/gmane.comp.boot-loaders.u-boot/71863/focus=71867
>
> FYI, it was afaik broken last year when revision 1 support was added:
>
> http://thread.gmane.org/gmane.comp.boot-loaders.u-boot/44630/focus=44727
>
> Could you check if a genext2fs-generated image works for you after you
> apply that patch to u-boot?

It did indeed fix the problems I was seeing, thank you very much for
pointing it out to me.  I also replied to the patch saying I'd tested
it.

Thanks again!
Dan