All of lore.kernel.org
 help / color / mirror / Atom feed
* RFC: Adding a new global MACHINE_ENDIAN variable
@ 2007-01-22  0:12 Rod Whitby
  2007-01-22 17:01 ` Koen Kooi
  0 siblings, 1 reply; 19+ messages in thread
From: Rod Whitby @ 2007-01-22  0:12 UTC (permalink / raw)
  To: openembedded-devel

There exist machines (like the ixp4xx range of processors) which can be
run in either little-endian or big-endian mode, and both modes are
supported by OE.

I propose a new "MACHINE_ENDIAN" (naming courtesy of RP) variable to do
this selection (i.e. for machines that support both, the build can be
selected in local.conf).

NSLU2-Linux has previously had an "ARCH_BYTE_SEX" variable for this
purpose - I would move all instances of that to the new variable name.
I would also update Angstrom to use the new global variable name instead
of reusing the nslu2-linux-specific variable (which will be deprecated).

The supported values of the variable would be "be" and "le".

-- Rod



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: RFC: Adding a new global MACHINE_ENDIAN variable
  2007-01-22  0:12 RFC: Adding a new global MACHINE_ENDIAN variable Rod Whitby
@ 2007-01-22 17:01 ` Koen Kooi
  2007-01-22 19:14   ` Rod Whitby
  0 siblings, 1 reply; 19+ messages in thread
From: Koen Kooi @ 2007-01-22 17:01 UTC (permalink / raw)
  To: openembedded-devel

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Rod Whitby schreef:
> There exist machines (like the ixp4xx range of processors) which can be
> run in either little-endian or big-endian mode, and both modes are
> supported by OE.
> 
> I propose a new "MACHINE_ENDIAN" (naming courtesy of RP) variable to do
> this selection (i.e. for machines that support both, the build can be
> selected in local.conf).

No, that's just needles confusion, just make a .inc file (like ixp4xx.inc) that has the
common bits and 2 machine files that set up the appropriate endiannes stuff. That way you
can use machine overrides to configure stuff instead of putting more anonymous python
functions in the metadata.

regards,

Koen "stop adding MAGIC_VARs" Kooi
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFFtO3NMkyGM64RGpERAnveAKCB2/xOEcBcMmF4CDPF7kAqAGQS+ACfetns
soDxhcws50KBPKr71LbmEqk=
=Hsop
-----END PGP SIGNATURE-----



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: RFC: Adding a new global MACHINE_ENDIAN variable
  2007-01-22 17:01 ` Koen Kooi
@ 2007-01-22 19:14   ` Rod Whitby
  2007-01-22 20:03     ` Koen Kooi
  0 siblings, 1 reply; 19+ messages in thread
From: Rod Whitby @ 2007-01-22 19:14 UTC (permalink / raw)
  To: openembedded-devel

Koen Kooi wrote:
> Rod Whitby schreef:
>>> There exist machines (like the ixp4xx range of processors) which can be
>>> run in either little-endian or big-endian mode, and both modes are
>>> supported by OE.
>>>
>>> I propose a new "MACHINE_ENDIAN" (naming courtesy of RP) variable to do
>>> this selection (i.e. for machines that support both, the build can be
>>> selected in local.conf).
> 
> No, that's just needles confusion, just make a .inc file (like ixp4xx.inc) that has the
> common bits and 2 machine files that set up the appropriate endiannes stuff. That way you
> can use machine overrides to configure stuff instead of putting more anonymous python
> functions in the metadata.

There currently is an ARCH_BYTE_SEX "magic" variable (which nslu2-linux
added), which is used 77 times in OE as it stands today (mostly in
SlugOS files).  First thing I will do is replace that with
MACHINE_ENDIAN (as it's a better name).

As for MACHINE_ENDIAN as a magic variable, to do as you suggest requires
that there be an easy way to have a "machine architecture" override
(i.e. ixp4xx) as well as the usual "actual machine/board" override (i.e.
nslu2le).  Currently there is only one such override in place, which
means that in *every* place that there currently is an _ixp4xx override
in the metadata, that needs to have 2 other duplicates of that line (one
for _nslu2le and one for _nslu2be) added in each of those .bb files.

I'm happy to go down that path instead of the MACHINE_ENDIAN path (if
that's the consensus of all the OE core team members - RP and koen
currently have diametrically opposed opinions on this - what do others
think?).  But I will need a way of inserting "ixp4xxle/be" and "ixp4xx"
in the set of overrides for both nslu2le and nslu2be machines.  I expect
this will need to be done in the ixp4xx.conf file by some hacky python
which looks for ${MACHINE} in the override list and adds "ixp4xxle/be"
and "ixp4xx" after it.

Koen, with that extra information and context, is your input to the
consensus still the same?

RP, does Koen's reply change your recommendation (to use MACHINE_ENDIAN)?

[Note that I'll be changing ARCH_BYTE_SEX to MACHINE_ENDIAN first
everywhere it exists, for clarity sake, as a first step.]

-- Rod (just looking for consensus before acting ...)



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: RFC: Adding a new global MACHINE_ENDIAN variable
  2007-01-22 19:14   ` Rod Whitby
@ 2007-01-22 20:03     ` Koen Kooi
  2007-01-23  0:10       ` Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN) Rod Whitby
  0 siblings, 1 reply; 19+ messages in thread
From: Koen Kooi @ 2007-01-22 20:03 UTC (permalink / raw)
  To: openembedded-devel

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Rod Whitby schreef:
> Koen Kooi wrote:
>> Rod Whitby schreef:
>>>> There exist machines (like the ixp4xx range of processors) which can be
>>>> run in either little-endian or big-endian mode, and both modes are
>>>> supported by OE.
>>>>
>>>> I propose a new "MACHINE_ENDIAN" (naming courtesy of RP) variable to do
>>>> this selection (i.e. for machines that support both, the build can be
>>>> selected in local.conf).
>> No, that's just needles confusion, just make a .inc file (like ixp4xx.inc) that has the
>> common bits and 2 machine files that set up the appropriate endiannes stuff. That way you
>> can use machine overrides to configure stuff instead of putting more anonymous python
>> functions in the metadata.
> 
> There currently is an ARCH_BYTE_SEX "magic" variable (which nslu2-linux
> added), which is used 77 times in OE as it stands today (mostly in
> SlugOS files).  First thing I will do is replace that with
> MACHINE_ENDIAN (as it's a better name).
> 
> As for MACHINE_ENDIAN as a magic variable, to do as you suggest requires
> that there be an easy way to have a "machine architecture" override
> (i.e. ixp4xx) as well as the usual "actual machine/board" override (i.e.
> nslu2le).  Currently there is only one such override in place, which
> means that in *every* place that there currently is an _ixp4xx override
> in the metadata, that needs to have 2 other duplicates of that line (one
> for _nslu2le and one for _nslu2be) added in each of those .bb files.
> 
> I'm happy to go down that path instead of the MACHINE_ENDIAN path (if
> that's the consensus of all the OE core team members - RP and koen
> currently have diametrically opposed opinions on this - what do others
> think?).  But I will need a way of inserting "ixp4xxle/be" and "ixp4xx"
> in the set of overrides for both nslu2le and nslu2be machines.  I expect
> this will need to be done in the ixp4xx.conf file by some hacky python
> which looks for ${MACHINE} in the override list and adds "ixp4xxle/be"
> and "ixp4xx" after it.
> 
> Koen, with that extra information and context, is your input to the
> consensus still the same?

I think adding two machines adheres to the principle of least surprise and makes is
possible to build for the 2 machines in parallel (/me hugs bitbake trunk + machine-via-env).
Also take a look at the amount of python in ixp4xx.conf to translate endianness into
various settings. With 2 machines, you can do without the python, which makes it for new
OE users look less arcane, and easier for python n00bs like me to modify.

regards,

Koen


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFFtRilMkyGM64RGpERAu07AKC2oMsHFQ3rD9MXFMhb2Yd3k7pHmgCeO6nf
SBy9UW8sKfS04A6NMS+IMjU=
=jUws
-----END PGP SIGNATURE-----



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN)
  2007-01-22 20:03     ` Koen Kooi
@ 2007-01-23  0:10       ` Rod Whitby
  2007-01-23  5:40         ` Removal of the proprietary Intel IXP ethernet driver (ixp4{00, 25}-eth, ixp-osal, ixp4xx-csr) from OE Rod Whitby
  2007-01-23 11:13         ` Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN) Richard Purdie
  0 siblings, 2 replies; 19+ messages in thread
From: Rod Whitby @ 2007-01-23  0:10 UTC (permalink / raw)
  To: openembedded-devel

Koen Kooi wrote:
> Rod Whitby schreef:
>>> Koen, with that extra information and context, is your input to the
>>> consensus still the same?
> 
> I think adding two machines adheres to the principle of least surprise and makes is
> possible to build for the 2 machines in parallel (/me hugs bitbake trunk + machine-via-env).
> Also take a look at the amount of python in ixp4xx.conf to translate endianness into
> various settings. With 2 machines, you can do without the python, which makes it for new
> OE users look less arcane, and easier for python n00bs like me to modify.

OK, based on a good IRC conversation with Koen this morning, we're going
to do the following:

1) Replace ARCH_BYTE_SEX with IXP4XX_MACHINE_ENDIAN throughout OE

2) Set IXP4XX_MACHINE_ENDIAN in conf/machine/{ixp4xx,nslu2}{be,le}.conf
and deprecate conf/machine/{ixp4xx,nslu2}.conf in favour of the
endian-specific machines.

2) Make sure that any ixp4xx-specific file that uses
IXP4XX_MACHINE_ENDIAN also has a COMPATIBLE_MACHINE setting to match.

3) Replace any usage of IXP4XX_MACHINE_ENDIAN in a non-ixp4xx-specific
file with autofoo stuff to determine the endianness.

4) Clean up conf/machine/include/ixp4xx to use tune-xscale and
tune-thumb, and remove other variables which already have global OE
equivalents.

5) Replace _ixp4xx overrides with _ixp4xxle and _ixp4xxbe overrides
instead.  Similarly for nslu2.  This will be done as a second step (in
the meantime, I'll just add ixp4xx directly to the overrides as well as
${MACHINE}).

*** Note that this means that "ixp4xx" or "nslu2" will no longer be
valid MACHINE settings - you must use "ixp4xxle", "ixp4xxbe", "nslu2le",
or "nslu2be" instead.


Last chance for objections .... commits will start in 6 hours.

-- Rod



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Removal of the proprietary Intel IXP ethernet driver (ixp4{00, 25}-eth, ixp-osal, ixp4xx-csr) from OE
  2007-01-23  0:10       ` Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN) Rod Whitby
@ 2007-01-23  5:40         ` Rod Whitby
  2007-01-23 19:23           ` Robert Wörle
  2009-04-20  5:52           ` Removal of the proprietary Intel IXP ethernet driver (ixp4{00, 25}-eth, ixp-osal, ixp4xx-csr) from OE Rod Whitby
  2007-01-23 11:13         ` Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN) Richard Purdie
  1 sibling, 2 replies; 19+ messages in thread
From: Rod Whitby @ 2007-01-23  5:40 UTC (permalink / raw)
  To: openembedded-devel; +Cc: John Bowler, nslu2-developers

For a long time, the only way to use the internal ethernet port for
ixp4xx devices was to use the proprietary intel driver.  The driver
could not be linked with the kernel, and tainted the kernel.

Thankfully, that has now changed with the development of the open source
ixp npe kernel driver, which is GPL and can be freely linked with the
kernel.  Note that the microcode firmware blob (ixp4xx-npe) which needs
to be loaded by the driver is still Intel Proprietary and requires a
click-through license on any firmware image that includes it.

The ixp4xx-kernel package (as of 2.6.19) now supports this new driver.
It has been tested on the NSLU2, NAS100d, and Loft devices, and as of
the 0.3.1 version is considered stable.

Accordingly, I propose to remove the Intel Proprietary driver from the
OpenEmbedded metadata.  I also propose to remove versions of the
ixp4xx-kernel earlier than 2.6.19.

The following packages will be obsoleted, and then removed:

ixp425-eth/ixp400-eth_1.4
ixp425-eth/ixp400-eth_1.5
ixp425-eth/ixp400-eth_1.5.1
ixp425-eth/ixp425-eth_1.1
ixp425-eth/ixp425-eth_1.2
ixp4xx/ixp-osal_1.5
ixp4xx/ixp-osal_2.0
ixp4xx/ixp-osal_2.1
ixp4xx/ixp-osal_2.1.1
ixp4xx/ixp4xx-csr_1.4
ixp4xx/ixp4xx-csr_1.5
ixp4xx/ixp4xx-csr_2.0
ixp4xx/ixp4xx-csr_2.1
ixp4xx/ixp4xx-csr_2.1.1
linux/ixp4xx-kernel_2.6.15.3
linux/ixp4xx-kernel_2.6.16
linux/ixp4xx-kernel_2.6.17
linux/ixp4xx-kernel_2.6.18

Any objections?

[Note that this will not affect anyone's ability to build from the
SlugOS binary release source code archives - they are in SVN.]

-- Rod





^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN)
  2007-01-23  0:10       ` Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN) Rod Whitby
  2007-01-23  5:40         ` Removal of the proprietary Intel IXP ethernet driver (ixp4{00, 25}-eth, ixp-osal, ixp4xx-csr) from OE Rod Whitby
@ 2007-01-23 11:13         ` Richard Purdie
  2007-01-23 19:37           ` Rod Whitby
  2007-01-28 12:41           ` Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN) Rod Whitby
  1 sibling, 2 replies; 19+ messages in thread
From: Richard Purdie @ 2007-01-23 11:13 UTC (permalink / raw)
  To: openembedded-devel

On Tue, 2007-01-23 at 10:40 +1030, Rod Whitby wrote:
> Last chance for objections .... commits will start in 6 hours.

For the record, no objections from me. After consideration, I think Koen
is right and two machines make sense in this case. Koen has had the
experience of trying this in practise which often helps! :)

Where possible I would like to see it using the siteinfo check so the
metadata is generic.

Cheers,

Richard




^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Removal of the proprietary Intel IXP ethernet driver (ixp4{00, 25}-eth, ixp-osal, ixp4xx-csr) from OE
  2007-01-23  5:40         ` Removal of the proprietary Intel IXP ethernet driver (ixp4{00, 25}-eth, ixp-osal, ixp4xx-csr) from OE Rod Whitby
@ 2007-01-23 19:23           ` Robert Wörle
  2007-01-23 19:56             ` Rod Whitby
  2009-04-20  5:52           ` Removal of the proprietary Intel IXP ethernet driver (ixp4{00, 25}-eth, ixp-osal, ixp4xx-csr) from OE Rod Whitby
  1 sibling, 1 reply; 19+ messages in thread
From: Robert Wörle @ 2007-01-23 19:23 UTC (permalink / raw)
  To: openembedded-devel

Rod Whitby schrieb:
> For a long time, the only way to use the internal ethernet port for
> ixp4xx devices was to use the proprietary intel driver.  The driver
> could not be linked with the kernel, and tainted the kernel.
>
> Thankfully, that has now changed with the development of the open source
> ixp npe kernel driver, which is GPL and can be freely linked with the
> kernel.  Note that the microcode firmware blob (ixp4xx-npe) which needs
> to be loaded by the driver is still Intel Proprietary and requires a
> click-through license on any firmware image that includes it.
>   
Dear Rod

Im not having such hardware but i am  interested in this drivers setup.
Do you think this code can be of any example ? I am in a similar
situation where i am confronted with a propriatory hardware driver.
Maybe that intel code as such is able to show me an good example of a
"firmware loading gpl module" and i could alter my code in such similar
way . Maybe the manufacturer looks at this and can go free with it also .
Could you explain to me the "click-through" license ?  Does one need to
agree on that on every module load or when will this come to play ?

kind regards
rob_w



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN)
  2007-01-23 11:13         ` Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN) Richard Purdie
@ 2007-01-23 19:37           ` Rod Whitby
  2007-01-23 20:34             ` Richard Purdie
  2007-01-28 12:41           ` Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN) Rod Whitby
  1 sibling, 1 reply; 19+ messages in thread
From: Rod Whitby @ 2007-01-23 19:37 UTC (permalink / raw)
  To: openembedded-devel

Richard Purdie wrote:
> On Tue, 2007-01-23 at 10:40 +1030, Rod Whitby wrote:
>> Last chance for objections .... commits will start in 6 hours.
> 
> For the record, no objections from me. After consideration, I think Koen
> is right and two machines make sense in this case. Koen has had the
> experience of trying this in practise which often helps! :)
> 
> Where possible I would like to see it using the siteinfo check so the
> metadata is generic.

Are you referring to the existing SITEINFO_ENDIANESS (sic) variable
(which I only just found ...) ?

Sure, if that is a guaranteed supported variable I can easily use that
one instead throughout.

-- Rod



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Removal of the proprietary Intel IXP ethernet driver (ixp4{00, 25}-eth, ixp-osal, ixp4xx-csr) from OE
  2007-01-23 19:23           ` Robert Wörle
@ 2007-01-23 19:56             ` Rod Whitby
  2007-01-25  3:15               ` RFC: customisable pivot-root functionality Rod Whitby
  0 siblings, 1 reply; 19+ messages in thread
From: Rod Whitby @ 2007-01-23 19:56 UTC (permalink / raw)
  To: openembedded-devel

Robert Wörle wrote:
> Rod Whitby schrieb:
>> For a long time, the only way to use the internal ethernet port for
>> ixp4xx devices was to use the proprietary intel driver.  The driver
>> could not be linked with the kernel, and tainted the kernel.
>>
>> Thankfully, that has now changed with the development of the open source
>> ixp npe kernel driver, which is GPL and can be freely linked with the
>> kernel.  Note that the microcode firmware blob (ixp4xx-npe) which needs
>> to be loaded by the driver is still Intel Proprietary and requires a
>> click-through license on any firmware image that includes it.
>>   
> Dear Rod
> 
> Im not having such hardware but i am  interested in this drivers setup.
> Do you think this code can be of any example ?

The Intel code?  No.  It's horrible code.  Subscribe to
linux-arm-kernel, and follow the patch review of the ixp4xx ethernet
driver that's happening right now if you want to see a good example of a
GPL rewrite of proprietary code and how to load firmware.

> I am in a similar
> situation where i am confronted with a propriatory hardware driver.
> Maybe that intel code as such is able to show me an good example of a
> "firmware loading gpl module" and i could alter my code in such similar
> way.

The intel code does not load the firmware, it has it compiled into the
driver (which is what makes it unable to be linked to the kernel).  The
open source driver does load the firmware from a separate file or MTD
partition.  The open source driver is built by the ixp4xx-kernel
package, which remains in OE for your viewing pleasure.

> Maybe the manufacturer looks at this and can go free with it also .
> Could you explain to me the "click-through" license ?  Does one need to
> agree on that on every module load or when will this come to play ?

Go to the intel site.  Try to download the ixp4xx software, and then
read the license which is presented to you.  I am not a lawyer, so I
cannot explain it to you - you really need to accept it on your own
behalf, or have your lawyer explain it to you.

-- Rod



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN)
  2007-01-23 19:37           ` Rod Whitby
@ 2007-01-23 20:34             ` Richard Purdie
  2007-01-24  6:41               ` SITEINFO_ENDIAN(N)ESS (NN, not N) Rod Whitby
  0 siblings, 1 reply; 19+ messages in thread
From: Richard Purdie @ 2007-01-23 20:34 UTC (permalink / raw)
  To: openembedded-devel

On Wed, 2007-01-24 at 06:07 +1030, Rod Whitby wrote:
> Are you referring to the existing SITEINFO_ENDIANESS (sic) variable
> (which I only just found ...) ?

Yes.

> Sure, if that is a guaranteed supported variable I can easily use that
> one instead throughout.

Its a recent addition and was a spinoff from the siteinfo changes which
tried to merge the site files for big/little endian etc. (it came
basically for free). Its certainly supported and I'm encouraging its
use. The big benefit is any new arch would need site files and if you
add site files this all just then works :)

Cheers,

Richard




^ permalink raw reply	[flat|nested] 19+ messages in thread

* SITEINFO_ENDIAN(N)ESS (NN, not N)
  2007-01-23 20:34             ` Richard Purdie
@ 2007-01-24  6:41               ` Rod Whitby
  2007-01-24  6:44                 ` Rod Whitby
  0 siblings, 1 reply; 19+ messages in thread
From: Rod Whitby @ 2007-01-24  6:41 UTC (permalink / raw)
  To: openembedded-devel

Richard Purdie wrote:
> On Wed, 2007-01-24 at 06:07 +1030, Rod Whitby wrote:
>> Are you referring to the existing SITEINFO_ENDIANESS (sic) variable
>> (which I only just found ...) ?
> 
> Yes.

Shouldn't it be spelled SITEINFO_ENDIANNESS? (i.e. double N)

(Both Google and Wikipedia say that it is endianness, not endianess)

Anyone object to me changing the 4 places it's currently used, before I
use it in 73 new places?

-- Rod



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: SITEINFO_ENDIAN(N)ESS (NN, not N)
  2007-01-24  6:41               ` SITEINFO_ENDIAN(N)ESS (NN, not N) Rod Whitby
@ 2007-01-24  6:44                 ` Rod Whitby
  0 siblings, 0 replies; 19+ messages in thread
From: Rod Whitby @ 2007-01-24  6:44 UTC (permalink / raw)
  To: openembedded-devel

Rod Whitby wrote:
> Richard Purdie wrote:
>> On Wed, 2007-01-24 at 06:07 +1030, Rod Whitby wrote:
>>> Are you referring to the existing SITEINFO_ENDIANESS (sic) variable
>>> (which I only just found ...) ?
>> Yes.
> 
> Shouldn't it be spelled SITEINFO_ENDIANNESS? (i.e. double N)
> 
> (Both Google and Wikipedia say that it is endianness, not endianess)
> 
> Anyone object to me changing the 4 places it's currently used, before I
> use it in 73 new places?

Scratch that, I just found the 71 other existing places where endianess
is used in lower-case :-(

I'll remain consistent with the existing speling (sic).

-- Rod



^ permalink raw reply	[flat|nested] 19+ messages in thread

* RFC: customisable pivot-root functionality
  2007-01-23 19:56             ` Rod Whitby
@ 2007-01-25  3:15               ` Rod Whitby
  2007-01-25  3:53                 ` Justin Patrin
                                   ` (2 more replies)
  0 siblings, 3 replies; 19+ messages in thread
From: Rod Whitby @ 2007-01-25  3:15 UTC (permalink / raw)
  To: openembedded-devel

As part of the "remove all slugos-specific stuff from OE, and make it
generally useful and available to multiple distributions" effort, I need
to replicate the current slugos-init pivot-root functionality, and I
want to do it in a way which might be useful for other OE platforms (or
at least is not tied to one specific NSLU2 distribution).  At a minimum,
I want to use it for a derivative of Angstrom, which I'm calling
"MokoSlug" and possible retro-fit it into the Unslung firmware too.

First I'll describe how it currently works, and then I'll ask for input
on how the same functionality might be useful on other OE platforms.

When an NSLU2 boots the OpenEmbedded-based SlugOS firmware, the kernel
and rootfs are loaded from internal flash.  The JFFS2 rootfs is
functional as-is on first boot, and has enough functionality
out-of-the-box to run ipkg update and ipkg install.  The trouble is that
there is not much room in an 8MB flash for a kernel, functional rootfs,
and additional packages.  Also, you don't want to be continually writing
to the internal flash (yes, you can use ramdisks for some things, but
you often want persistant regular changes to your rootfs that don't
require a copy-on-shutdown to preserve the information).

So we have a script called "turnup" which allows you to modify the jffs2
rootfs's /linuxrc file to load the rootfs from an external disk (or over
nfs for that matter).  I've included the help text from turnup at the
end of this message.

Then on the next boot, the new /linuxrc file is executed, and the rootfs
is loaded from external disk - giving you all the room in the world to
install new packages (without having to go through ipkg-link or bind
mount contortions).

The other advantage of this scheme is that if your external disk fails
(or is unplugged), then the boot falls back to the internal flash, which
has enough functionality to debug and fix the reasons why the boot from
external disk failed (assuming it wasn't your cat unplugging the usb cable).

So the questions are:

1) Is there similar functionality elsewhere in OE that I haven't seen,
and could just use as-is?

2) Would this functionality be of use to any other devices that use OE?

3) Who else is interested in giving input to, and reviewing the output
of, this effort?

Thanks,

-- Rod

[turnup script help follows]

root@nslu2:~$ turnup
usage: turnup command [options]
 commands:
  help
    output this help
  init
    correct errors in network information
    initialise network information when DHCP is not available
    change network information
  disk [-i] [-s<seconds>] <device>|<uuid> [mount options]
    With -i make <device> a bootable file system then (with or
    without -i) arrange for the next reboot to use that device.
    The device must already be formatted as a file system, with
    -i it must be completely empty, without it must contain an
    apparently bootable file system. -s (for example -s5)
    specifies a delay in seconds to wait at boot time before
    mounting the device.
  memstick [-i] <device>|<uuid> [mount options]
    Behaves as disk however options appropriate to a flash memory
    stick are automatically added
  nfs [-i] <nfs mount path> [mount options]
    <nfs mount path> must be a mountable NFS file system.  With
    -i the partition must be empty and is initialised with a
    bootable file system.  Without -i the partition must already
    contain a bootable file system.  In either case the NFS
    partition must be available to be mounted without root id
    sqashing (i.e. root must be root) and it will be selected
    as the root file system for subsequent reboots.
    A default set of -o options are provided, additional options
    may be given on the command line (multiple -o options will
    be combined into a single -o).
  flash
    Revert to booting from the flash disk on next reboot.
  ram
    Boot (once) into a ramdisk, subsequent boots will be to
    the flash file system.
  preserve
    Save the system configuration to the SysConf partition, you
    will need to create the SysConf partition from the boot loader
    before using this if SysConf does not already exist.  This
    just runs 'sysconf save'.
  restore
    Restore a previously saved system configuration.  This just
    runs 'sysconf restore'.
 disk formatting:
  The argument to 'nfs' or 'disk' must be an empty partition
  of sufficient size to hold the root file system (at least
  16MByte but more is recommended to allow package installation).
  An appropriate ext3 partition can be made using the command:

    mke2fs -j <device>  # for example: /dev/sda1

  An appropriate NFS partition can be emptied using 'rm', but
  must be set up (exported) on the NFS server.
root@nslu2:~$



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: RFC: customisable pivot-root functionality
  2007-01-25  3:15               ` RFC: customisable pivot-root functionality Rod Whitby
@ 2007-01-25  3:53                 ` Justin Patrin
  2007-01-25 12:45                 ` Cliff Brake
  2007-01-25 16:54                 ` Hans Henry von Tresckow
  2 siblings, 0 replies; 19+ messages in thread
From: Justin Patrin @ 2007-01-25  3:53 UTC (permalink / raw)
  To: openembedded-devel

On 1/24/07, Rod Whitby <rod@whitby.id.au> wrote:
> As part of the "remove all slugos-specific stuff from OE, and make it
> generally useful and available to multiple distributions" effort, I need
> to replicate the current slugos-init pivot-root functionality, and I
> want to do it in a way which might be useful for other OE platforms (or
> at least is not tied to one specific NSLU2 distribution).  At a minimum,
> I want to use it for a derivative of Angstrom, which I'm calling
> "MokoSlug" and possible retro-fit it into the Unslung firmware too.
>
> First I'll describe how it currently works, and then I'll ask for input
> on how the same functionality might be useful on other OE platforms.
>
> When an NSLU2 boots the OpenEmbedded-based SlugOS firmware, the kernel
> and rootfs are loaded from internal flash.  The JFFS2 rootfs is
> functional as-is on first boot, and has enough functionality
> out-of-the-box to run ipkg update and ipkg install.  The trouble is that
> there is not much room in an 8MB flash for a kernel, functional rootfs,
> and additional packages.  Also, you don't want to be continually writing
> to the internal flash (yes, you can use ramdisks for some things, but
> you often want persistant regular changes to your rootfs that don't
> require a copy-on-shutdown to preserve the information).
>
> So we have a script called "turnup" which allows you to modify the jffs2
> rootfs's /linuxrc file to load the rootfs from an external disk (or over
> nfs for that matter).  I've included the help text from turnup at the
> end of this message.
>
> Then on the next boot, the new /linuxrc file is executed, and the rootfs
> is loaded from external disk - giving you all the room in the world to
> install new packages (without having to go through ipkg-link or bind
> mount contortions).
>
> The other advantage of this scheme is that if your external disk fails
> (or is unplugged), then the boot falls back to the internal flash, which
> has enough functionality to debug and fix the reasons why the boot from
> external disk failed (assuming it wasn't your cat unplugging the usb cable).
>
> So the questions are:
>
> 1) Is there similar functionality elsewhere in OE that I haven't seen,
> and could just use as-is?
>

There was a pivot-root image in OE at one time which was used for the
C3000 to pivot from the flash to the hard drive. It may still be in OE
(and if it isn't it's in the history). You can look there for some of
what has been previously donw.

> 2) Would this functionality be of use to any other devices that use OE?

Not for me, but I can imagine other uses like with your slugs.

>
> 3) Who else is interested in giving input to, and reviewing the output
> of, this effort?
>
> Thanks,
>
> -- Rod
>
> [turnup script help follows]
>
> root@nslu2:~$ turnup
> usage: turnup command [options]
>  commands:
>   help
>     output this help
>   init
>     correct errors in network information
>     initialise network information when DHCP is not available
>     change network information
>   disk [-i] [-s<seconds>] <device>|<uuid> [mount options]
>     With -i make <device> a bootable file system then (with or
>     without -i) arrange for the next reboot to use that device.
>     The device must already be formatted as a file system, with
>     -i it must be completely empty, without it must contain an
>     apparently bootable file system. -s (for example -s5)
>     specifies a delay in seconds to wait at boot time before
>     mounting the device.
>   memstick [-i] <device>|<uuid> [mount options]
>     Behaves as disk however options appropriate to a flash memory
>     stick are automatically added
>   nfs [-i] <nfs mount path> [mount options]
>     <nfs mount path> must be a mountable NFS file system.  With
>     -i the partition must be empty and is initialised with a
>     bootable file system.  Without -i the partition must already
>     contain a bootable file system.  In either case the NFS
>     partition must be available to be mounted without root id
>     sqashing (i.e. root must be root) and it will be selected
>     as the root file system for subsequent reboots.
>     A default set of -o options are provided, additional options
>     may be given on the command line (multiple -o options will
>     be combined into a single -o).
>   flash
>     Revert to booting from the flash disk on next reboot.
>   ram
>     Boot (once) into a ramdisk, subsequent boots will be to
>     the flash file system.
>   preserve
>     Save the system configuration to the SysConf partition, you
>     will need to create the SysConf partition from the boot loader
>     before using this if SysConf does not already exist.  This
>     just runs 'sysconf save'.
>   restore
>     Restore a previously saved system configuration.  This just
>     runs 'sysconf restore'.
>  disk formatting:
>   The argument to 'nfs' or 'disk' must be an empty partition
>   of sufficient size to hold the root file system (at least
>   16MByte but more is recommended to allow package installation).
>   An appropriate ext3 partition can be made using the command:
>
>     mke2fs -j <device>  # for example: /dev/sda1
>
>   An appropriate NFS partition can be emptied using 'rm', but
>   must be set up (exported) on the NFS server.
> root@nslu2:~$
>
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-devel
>


-- 
Justin Patrin



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: RFC: customisable pivot-root functionality
  2007-01-25  3:15               ` RFC: customisable pivot-root functionality Rod Whitby
  2007-01-25  3:53                 ` Justin Patrin
@ 2007-01-25 12:45                 ` Cliff Brake
  2007-01-25 16:54                 ` Hans Henry von Tresckow
  2 siblings, 0 replies; 19+ messages in thread
From: Cliff Brake @ 2007-01-25 12:45 UTC (permalink / raw)
  To: openembedded-devel

On 1/24/07, Rod Whitby <rod@whitby.id.au> wrote:
> As part of the "remove all slugos-specific stuff from OE, and make it
> generally useful and available to multiple distributions" effort, I need
> to replicate the current slugos-init pivot-root functionality, and I
> want to do it in a way which might be useful for other OE platforms (or
> at least is not tied to one specific NSLU2 distribution).  At a minimum,
> I want to use it for a derivative of Angstrom, which I'm calling
> "MokoSlug" and possible retro-fit it into the Unslung firmware too.
...

> 2) Would this functionality be of use to any other devices that use OE?

Yes.

> 3) Who else is interested in giving input to, and reviewing the output
> of, this effort?

Yes, I'm currently working on two projects that will required the
ability to do field updates -- (stick a USB stick in the device and it
updates itself).  I'm currently planning to have a small RO update
image (which never gets updated) that is smart enough to update the
main rootfs and then pivot-root to it.  It would be nice if this could
be made somewhat general as it is probably a fairly common
requirement.  Another reason for the RO update image is to be able to
easily recover if the main image gets corrupted.  I'm not sure if
there is anything in common with your slug work, but It would be nice
to have a general framework available to do the pivot-root stuff.

Cliff

-- 
=======================
Cliff Brake
http://bec-systems.com



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: RFC: customisable pivot-root functionality
  2007-01-25  3:15               ` RFC: customisable pivot-root functionality Rod Whitby
  2007-01-25  3:53                 ` Justin Patrin
  2007-01-25 12:45                 ` Cliff Brake
@ 2007-01-25 16:54                 ` Hans Henry von Tresckow
  2 siblings, 0 replies; 19+ messages in thread
From: Hans Henry von Tresckow @ 2007-01-25 16:54 UTC (permalink / raw)
  To: openembedded-devel

>
> 1) Is there similar functionality elsewhere in OE that I haven't seen,
> and could just use as-is?
>
>
> Have you looked at altboot? it uses custom init-scripts and can be
configured to do the kind of things you were looking at. Originally it was
written to allow Zaurus machines to boot from loop-images on SD or even over
NFS. I have it set up to boot a kenel and rootfs on my SD card.


I hope this helps,

Henry


^ permalink raw reply	[flat|nested] 19+ messages in thread

* Re: Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN)
  2007-01-23 11:13         ` Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN) Richard Purdie
  2007-01-23 19:37           ` Rod Whitby
@ 2007-01-28 12:41           ` Rod Whitby
  1 sibling, 0 replies; 19+ messages in thread
From: Rod Whitby @ 2007-01-28 12:41 UTC (permalink / raw)
  To: openembedded-devel

Richard Purdie wrote:
> On Tue, 2007-01-23 at 10:40 +1030, Rod Whitby wrote:
>> Last chance for objections .... commits will start in 6 hours.
> 
> For the record, no objections from me. After consideration, I think Koen
> is right and two machines make sense in this case. Koen has had the
> experience of trying this in practise which often helps! :)
> 
> Where possible I would like to see it using the siteinfo check so the
> metadata is generic.

This has now been done (using SITEINFO_ENDIANESS).

Neither ARCH_BYTE_SEX or IXP4XX_MACHINE_ENDIAN exist anywhere in OE any
more.

-- Rod



^ permalink raw reply	[flat|nested] 19+ messages in thread

* Removal of the proprietary Intel IXP ethernet driver (ixp4{00, 25}-eth, ixp-osal, ixp4xx-csr) from OE
  2007-01-23  5:40         ` Removal of the proprietary Intel IXP ethernet driver (ixp4{00, 25}-eth, ixp-osal, ixp4xx-csr) from OE Rod Whitby
  2007-01-23 19:23           ` Robert Wörle
@ 2009-04-20  5:52           ` Rod Whitby
  1 sibling, 0 replies; 19+ messages in thread
From: Rod Whitby @ 2009-04-20  5:52 UTC (permalink / raw)
  To: openembedded-devel; +Cc: nslu2-developers

It seems that I intended to do this, but never got around to it.

<http://lists.linuxtogo.org/pipermail/openembedded-devel/2007-January/001279.html>

There were no objections at the time (over 2 years ago), so unless there
are any objections in the next 48 hours it will be enacted.

-- Rod

Rod Whitby wrote (on 23/01/2007):
> For a long time, the only way to use the internal ethernet port for
> ixp4xx devices was to use the proprietary intel driver.  The driver
> could not be linked with the kernel, and tainted the kernel.
> 
> Thankfully, that has now changed with the development of the open source
> ixp npe kernel driver, which is GPL and can be freely linked with the
> kernel.  Note that the microcode firmware blob (ixp4xx-npe) which needs
> to be loaded by the driver is still Intel Proprietary and requires a
> click-through license on any firmware image that includes it.
> 
> The ixp4xx-kernel package (as of 2.6.19) now supports this new driver.
> It has been tested on the NSLU2, NAS100d, and Loft devices, and as of
> the 0.3.1 version is considered stable.
> 
> Accordingly, I propose to remove the Intel Proprietary driver from the
> OpenEmbedded metadata.  I also propose to remove versions of the
> ixp4xx-kernel earlier than 2.6.19.
> 
> The following packages will be obsoleted, and then removed:
> 
> ixp425-eth/ixp400-eth_1.4
> ixp425-eth/ixp400-eth_1.5
> ixp425-eth/ixp400-eth_1.5.1
> ixp425-eth/ixp425-eth_1.1
> ixp425-eth/ixp425-eth_1.2
> ixp4xx/ixp-osal_1.5
> ixp4xx/ixp-osal_2.0
> ixp4xx/ixp-osal_2.1
> ixp4xx/ixp-osal_2.1.1
> ixp4xx/ixp4xx-csr_1.4
> ixp4xx/ixp4xx-csr_1.5
> ixp4xx/ixp4xx-csr_2.0
> ixp4xx/ixp4xx-csr_2.1
> ixp4xx/ixp4xx-csr_2.1.1
> linux/ixp4xx-kernel_2.6.15.3
> linux/ixp4xx-kernel_2.6.16
> linux/ixp4xx-kernel_2.6.17
> linux/ixp4xx-kernel_2.6.18
> 
> Any objections?
> 
> [Note that this will not affect anyone's ability to build from the
> SlugOS binary release source code archives - they are in SVN.]
> 
> -- Rod




^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2009-04-20  5:56 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-01-22  0:12 RFC: Adding a new global MACHINE_ENDIAN variable Rod Whitby
2007-01-22 17:01 ` Koen Kooi
2007-01-22 19:14   ` Rod Whitby
2007-01-22 20:03     ` Koen Kooi
2007-01-23  0:10       ` Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN) Rod Whitby
2007-01-23  5:40         ` Removal of the proprietary Intel IXP ethernet driver (ixp4{00, 25}-eth, ixp-osal, ixp4xx-csr) from OE Rod Whitby
2007-01-23 19:23           ` Robert Wörle
2007-01-23 19:56             ` Rod Whitby
2007-01-25  3:15               ` RFC: customisable pivot-root functionality Rod Whitby
2007-01-25  3:53                 ` Justin Patrin
2007-01-25 12:45                 ` Cliff Brake
2007-01-25 16:54                 ` Hans Henry von Tresckow
2009-04-20  5:52           ` Removal of the proprietary Intel IXP ethernet driver (ixp4{00, 25}-eth, ixp-osal, ixp4xx-csr) from OE Rod Whitby
2007-01-23 11:13         ` Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN) Richard Purdie
2007-01-23 19:37           ` Rod Whitby
2007-01-23 20:34             ` Richard Purdie
2007-01-24  6:41               ` SITEINFO_ENDIAN(N)ESS (NN, not N) Rod Whitby
2007-01-24  6:44                 ` Rod Whitby
2007-01-28 12:41           ` Deprecating ixp4xx, nslu2 in favour of endian-specific machine settings (Was: RFC on MACHINE_ENDIAN) Rod Whitby

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.