All of lore.kernel.org
 help / color / mirror / Atom feed
* RFC: splitting deploy/ipk into subarchs
@ 2007-02-19 11:18 Koen Kooi
  2007-02-19 11:59 ` Marcin Juszkiewicz
  0 siblings, 1 reply; 9+ messages in thread
From: Koen Kooi @ 2007-02-19 11:18 UTC (permalink / raw)
  To: Using the OpenEmbedded metadata to build Linux Distributions

[-- Attachment #1: Type: text/plain, Size: 497 bytes --]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

When using multimachine and ipkg deploy/ipk gets awfully crowded after a while. I want to
propose sorting it by subarch. Attached is a patch that adds support for that to
rootfs_ipk.bbclass.

What do you think about that?

regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFF2YeZMkyGM64RGpERAjYCAJ9nBjdpcvTlvijaQ7o/UlZGWg68tgCdHw4m
IzGLM3yx+X9k/ZgMo+WqT6E=
=N6sc
-----END PGP SIGNATURE-----

[-- Attachment #2: rootfs-ipk-with-sorted-deploy.patch --]
[-- Type: text/x-patch, Size: 1579 bytes --]

# 
# old_revision [bc83c51286e87a2b8aa3f79f074cadc05dedb7de]
# 
# patch "classes/rootfs_ipk.bbclass"
#  from [7824b4ee6ac7524e12d7411575e8bbf4b8a9d321]
#    to [016f168e95ecc8150984cc48a14ecbd73a5fd9ed]
# 
============================================================
--- classes/rootfs_ipk.bbclass	7824b4ee6ac7524e12d7411575e8bbf4b8a9d321
+++ classes/rootfs_ipk.bbclass	016f168e95ecc8150984cc48a14ecbd73a5fd9ed
@@ -18,17 +18,26 @@ fakeroot rootfs_ipk_do_rootfs () {
 		
 	mkdir -p ${IMAGE_ROOTFS}/dev
 
-	if [ -z "${DEPLOY_KEEP_PACKAGES}" ]; then
+	ipkgarchs="${PACKAGE_ARCHS}"
+
+        if [ -z "${DEPLOY_KEEP_PACKAGES}" ]; then
 		touch ${DEPLOY_DIR_IPK}/Packages
 		ipkg-make-index -r ${DEPLOY_DIR_IPK}/Packages -p ${DEPLOY_DIR_IPK}/Packages -l ${DEPLOY_DIR_IPK}/Packages.filelist -m ${DEPLOY_DIR_IPK}
-	fi
+		if [-e ${DEPLOY_DIR_IPK}/$arch/] ; then 
+			touch ${DEPLOY_DIR_IPK}/$arch/Packages
+			ipkg-make-index -r ${DEPLOY_DIR_IPK}/$arch/Packages -p ${DEPLOY_DIR_IPK}/$arch/Packages -l ${DEPLOY_DIR_IPK}/$arch/Packages.filelist -m ${DEPLOY_DIR_IPK}/$arch
+                fi
+        fi
+	
 	mkdir -p ${T}
 	echo "src oe file:${DEPLOY_DIR_IPK}" > ${T}/ipkg.conf
-	ipkgarchs="${PACKAGE_ARCHS}"
 	priority=1
 	for arch in $ipkgarchs; do
 		echo "arch $arch $priority" >> ${T}/ipkg.conf
 		priority=$(expr $priority + 5)
+		if [-e ${DEPLOY_DIR_IPK}/$arch/Packages] ; then
+		        echo "src oe file:${DEPLOY_DIR_IPK}/$arch" >> ${T}/ipkg.conf
+	        fi
 	done
 	ipkg-cl ${IPKG_ARGS} update
 	if [ ! -z "${LINGUAS_INSTALL}" ]; then

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: RFC: splitting deploy/ipk into subarchs
  2007-02-19 11:18 RFC: splitting deploy/ipk into subarchs Koen Kooi
@ 2007-02-19 11:59 ` Marcin Juszkiewicz
  2007-02-19 12:30   ` Koen Kooi
  0 siblings, 1 reply; 9+ messages in thread
From: Marcin Juszkiewicz @ 2007-02-19 11:59 UTC (permalink / raw)
  To: openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 677 bytes --]

Dnia poniedziałek, 19 lutego 2007, Koen Kooi napisał:

> When using multimachine and ipkg deploy/ipk gets awfully crowded after
> a while. I want to propose sorting it by subarch. Attached is a patch
> that adds support for that to rootfs_ipk.bbclass.
>
> What do you think about that?

I like this idea. ipkg-make-index is slow enough already.

Attached fixed, tested version of your rootfs_ipk.bbclass change. Also 
added patch to package_ipk.bbclass which will store package in proper 
dir.

-- 
JID: hrw-jabber.org
OpenEmbedded developer/consultant

         Great minds discuss ideas; average minds discuss events;
         small minds discuss people.

[-- Attachment #2: build-rootfs-from-split-ipkdir.diff --]
[-- Type: text/x-diff, Size: 1601 bytes --]

# 
# old_revision [bc83c51286e87a2b8aa3f79f074cadc05dedb7de]
# 
# patch "classes/rootfs_ipk.bbclass"
#  from [7824b4ee6ac7524e12d7411575e8bbf4b8a9d321]
#    to [a30d56a031d7b85d13bb62d4956e77252a4b79fa]
# 
============================================================
--- classes/rootfs_ipk.bbclass	7824b4ee6ac7524e12d7411575e8bbf4b8a9d321
+++ classes/rootfs_ipk.bbclass	a30d56a031d7b85d13bb62d4956e77252a4b79fa
@@ -18,17 +18,25 @@ fakeroot rootfs_ipk_do_rootfs () {
 		
 	mkdir -p ${IMAGE_ROOTFS}/dev
 
-	if [ -z "${DEPLOY_KEEP_PACKAGES}" ]; then
-		touch ${DEPLOY_DIR_IPK}/Packages
-		ipkg-make-index -r ${DEPLOY_DIR_IPK}/Packages -p ${DEPLOY_DIR_IPK}/Packages -l ${DEPLOY_DIR_IPK}/Packages.filelist -m ${DEPLOY_DIR_IPK}
-	fi
+	ipkgarchs="${PACKAGE_ARCHS}"
+
+	for arch in $ipkgarchs; do
+		if [ -z "${DEPLOY_KEEP_PACKAGES}" ]; then
+			if [ -e ${DEPLOY_DIR_IPK}/$arch/ ] ; then 
+				touch ${DEPLOY_DIR_IPK}/$arch/Packages
+				ipkg-make-index -r ${DEPLOY_DIR_IPK}/$arch/Packages -p ${DEPLOY_DIR_IPK}/$arch/Packages -l ${DEPLOY_DIR_IPK}/$arch/Packages.filelist -m ${DEPLOY_DIR_IPK}/$arch/
+			fi
+		fi
+	done
+
 	mkdir -p ${T}
-	echo "src oe file:${DEPLOY_DIR_IPK}" > ${T}/ipkg.conf
-	ipkgarchs="${PACKAGE_ARCHS}"
 	priority=1
 	for arch in $ipkgarchs; do
 		echo "arch $arch $priority" >> ${T}/ipkg.conf
 		priority=$(expr $priority + 5)
+		if [ -e ${DEPLOY_DIR_IPK}/$arch/Packages ] ; then
+		        echo "src oe-$arch file:${DEPLOY_DIR_IPK}/$arch" >> ${T}/ipkg.conf
+	    fi
 	done
 	ipkg-cl ${IPKG_ARGS} update
 	if [ ! -z "${LINGUAS_INSTALL}" ]; then

[-- Attachment #3: store-package-in-proper-ipkdir.diff --]
[-- Type: text/x-diff, Size: 711 bytes --]

# 
# old_revision [bc83c51286e87a2b8aa3f79f074cadc05dedb7de]
# 
# patch "classes/package_ipk.bbclass"
#  from [6f5d44fb7cf065ce7b680009e5c9bcb554f760c4]
#    to [57535d540caec17311adeb1e15f49d290d6cc8ac]
# 
============================================================
--- classes/package_ipk.bbclass	6f5d44fb7cf065ce7b680009e5c9bcb554f760c4
+++ classes/package_ipk.bbclass	57535d540caec17311adeb1e15f49d290d6cc8ac
@@ -75,6 +75,9 @@ python do_package_ipk () {
 	if not outdir:
 		bb.error("DEPLOY_DIR_IPK not defined, unable to package")
 		return
+
+	arch = bb.data.getVar('PACKAGE_ARCH', d, 1)
+	outdir = "%s/%s" % (outdir, arch)
 	bb.mkdirhier(outdir)
 
 	dvar = bb.data.getVar('D', d, 1)

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: RFC: splitting deploy/ipk into subarchs
  2007-02-19 11:59 ` Marcin Juszkiewicz
@ 2007-02-19 12:30   ` Koen Kooi
  2007-02-20  8:19     ` Koen Kooi
  0 siblings, 1 reply; 9+ messages in thread
From: Koen Kooi @ 2007-02-19 12:30 UTC (permalink / raw)
  To: openembedded-devel

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Marcin Juszkiewicz schreef:
> Dnia poniedziałek, 19 lutego 2007, Koen Kooi napisał:
> 
>> When using multimachine and ipkg deploy/ipk gets awfully crowded after
>> a while. I want to propose sorting it by subarch. Attached is a patch
>> that adds support for that to rootfs_ipk.bbclass.
>>
>> What do you think about that?
> 
> I like this idea. ipkg-make-index is slow enough already.
> 
> Attached fixed, tested version of your rootfs_ipk.bbclass change. Also 
> added patch to package_ipk.bbclass which will store package in proper 
> dir.

Nice

> -	echo "src oe file:${DEPLOY_DIR_IPK}" > ${T}/ipkg.conf

This breaks backward compat, could you add that back in?

regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFF2ZhMMkyGM64RGpERAtwOAJ0d4gxAt3a/XFjsJP+5xFvsapJdswCePBQN
QCZEkVp954PrWVq1QERxVtQ=
=Qtcf
-----END PGP SIGNATURE-----



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: RFC: splitting deploy/ipk into subarchs
  2007-02-19 12:30   ` Koen Kooi
@ 2007-02-20  8:19     ` Koen Kooi
  2007-02-20 13:48       ` Koen Kooi
  0 siblings, 1 reply; 9+ messages in thread
From: Koen Kooi @ 2007-02-20  8:19 UTC (permalink / raw)
  To: openembedded-devel

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Koen Kooi schreef:
> Marcin Juszkiewicz schreef:
>> Dnia poniedziaBek, 19 lutego 2007, Koen Kooi napisaB:
> 
>>> When using multimachine and ipkg deploy/ipk gets awfully crowded after
>>> a while. I want to propose sorting it by subarch. Attached is a patch
>>> that adds support for that to rootfs_ipk.bbclass.
>>>
>>> What do you think about that?
>> I like this idea. ipkg-make-index is slow enough already.
> 
>> Attached fixed, tested version of your rootfs_ipk.bbclass change. Also 
>> added patch to package_ipk.bbclass which will store package in proper 
>> dir.

Marcin and I merged a slightly different version of this today.

regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFF2q70MkyGM64RGpERAjXNAJ9B2sh7Ly/KvcFkSfKNmAgXw+7VwACfbFfn
/sKicIMGzfx8EYkT/B/YR78=
=s1Iz
-----END PGP SIGNATURE-----



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: RFC: splitting deploy/ipk into subarchs
  2007-02-20  8:19     ` Koen Kooi
@ 2007-02-20 13:48       ` Koen Kooi
  2007-02-23  2:26         ` Matthias Hentges
  0 siblings, 1 reply; 9+ messages in thread
From: Koen Kooi @ 2007-02-20 13:48 UTC (permalink / raw)
  To: openembedded-devel

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Koen Kooi schreef:

> Marcin and I merged a slightly different version of this today.

As some people found out, their upload scripts broke with this change. I'm sorry that
happened, but those scripts are more easily fixed than ipkg-utils :(
Due to bug in ipkg-utils being I haven't been using a straight rsync for a while now, so
for the curious:

rsync -az . user@remote.host:/feed/location/unsorted ; ssh user@remote.host 'cd
feed/location/unsorted ; sh sort.sh'

where sort.sh is http://www.angstrom-distribution.org/unstable/feed/unsorted/sort.sh.txt

And while you're at it, check wether your rsync uses --delete, and if not, check if you
remove Packages* before running ipkg-make-index. If that wasn't the case you have been
unknowingly suffering for a buggy Packages* for a while now. Combined with ipkg .163 on
the device I suspect 'ipkg upgrade' doesn't behave as deterministically as you might think.

regards,

Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFF2vw3MkyGM64RGpERAv9OAJ4ij31yxqCC8M855mQ8hUvhdj56UQCfVk4z
zfpTSMD0IajJc9VRF2dyuQA=
=rvcF
-----END PGP SIGNATURE-----



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: RFC: splitting deploy/ipk into subarchs
  2007-02-20 13:48       ` Koen Kooi
@ 2007-02-23  2:26         ` Matthias Hentges
  2007-02-23  8:05           ` Koen Kooi
  0 siblings, 1 reply; 9+ messages in thread
From: Matthias Hentges @ 2007-02-23  2:26 UTC (permalink / raw)
  To: openembedded-devel

Am Dienstag, den 20.02.2007, 14:48 +0100 schrieb Koen Kooi:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Koen Kooi schreef:
> 
> > Marcin and I merged a slightly different version of this today.
> 
> As some people found out, their upload scripts broke with this change. I'm sorry that
> happened, but those scripts are more easily fixed than ipkg-utils :(
> Due to bug in ipkg-utils being I haven't been using a straight rsync for a while now, so
> for the curious:

Why not make this change optional via a variable 
(ie: SPLIT_DEPLOYDIR_IPK = -1 to turn it off) ?

The current version breaks local OE feeds and requires needless
post-processing.

A slugos deploy dir looks like this now:

all
armv5te
i686
ixp4xxle

I honestly don't see any advantage to the previous mode of operation. If
it is needed for unique situations like the rare case of a multi-machine
build, it should be off by default and - by all means - optional.

My 2 cents
--
Matthias Hentges




^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: RFC: splitting deploy/ipk into subarchs
  2007-02-23  2:26         ` Matthias Hentges
@ 2007-02-23  8:05           ` Koen Kooi
  2007-02-24  0:03             ` Matthias Hentges
  2007-02-24 11:18             ` Michael Rozdoba
  0 siblings, 2 replies; 9+ messages in thread
From: Koen Kooi @ 2007-02-23  8:05 UTC (permalink / raw)
  To: openembedded-devel

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Matthias Hentges schreef:
> Am Dienstag, den 20.02.2007, 14:48 +0100 schrieb Koen Kooi:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Koen Kooi schreef:
>>
>>> Marcin and I merged a slightly different version of this today.
>> As some people found out, their upload scripts broke with this change. I'm sorry that
>> happened, but those scripts are more easily fixed than ipkg-utils :(
>> Due to bug in ipkg-utils being I haven't been using a straight rsync for a while now, so
>> for the curious:
> 
> Why not make this change optional via a variable 
> (ie: SPLIT_DEPLOYDIR_IPK = -1 to turn it off) ?
> 
> The current version breaks local OE feeds

People's feeds are outside the scope of OE. Repeat after me: deploy never was, isn't and
never will be a feed.

> and requires needless post-processing.

As I pointed out before, you needed postprocessing anyway due to bugs in ipkg and
ipkg-utils, so your point is moot.


> A slugos deploy dir looks like this now:
> 
> all
> armv5te
> i686
> ixp4xxle
> 
> I honestly don't see any advantage to the previous mode of operation.

Go read the openembedded-devel list some time.


> If
> it is needed for unique situations 

No

> like the rare case of a multi-machine build

Apart from that now being the point, it actually isn't rare, the majority of the
(maintained) distros in OE use it (celinux. generic, angstrom, openmoko, openzaurus,
slugos and jline).

> it should be off by default and - by all means - optional.

No it should not. Next time respond to the RFCs instead of starting some uninformed rant
afterward.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)

iD8DBQFF3qA+MkyGM64RGpERAsq1AJ9jkjFqiolnJWv/Mul6S59sCG2gZwCcDWRi
WEqSA0lYqeEuhz/9vBImS00=
=EmZ6
-----END PGP SIGNATURE-----



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: RFC: splitting deploy/ipk into subarchs
  2007-02-23  8:05           ` Koen Kooi
@ 2007-02-24  0:03             ` Matthias Hentges
  2007-02-24 11:18             ` Michael Rozdoba
  1 sibling, 0 replies; 9+ messages in thread
From: Matthias Hentges @ 2007-02-24  0:03 UTC (permalink / raw)
  To: openembedded-devel

Am Freitag, den 23.02.2007, 09:05 +0100 schrieb Koen Kooi:

> Matthias Hentges schreef:
> > Am Dienstag, den 20.02.2007, 14:48 +0100 schrieb Koen Kooi:
> >> -----BEGIN PGP SIGNED MESSAGE-----
> >> Hash: SHA1
> >>
> >> Koen Kooi schreef:
> >> 
> > Why not make this change optional via a variable 
> > (ie: SPLIT_DEPLOYDIR_IPK = -1 to turn it off) ?
> > 
> > The current version breaks local OE feeds
> 
> People's feeds are outside the scope of OE. Repeat after me: deploy never was, isn't and
> never will be a feed.

It worked just fine as a feed before the change and people were using
it as such, deal with it.

> > and requires needless post-processing.
> 
> As I pointed out before, you needed postprocessing anyway due to bugs in ipkg and
> ipkg-utils, so your point is moot.

You may have needed post-processing, I personally never did. Not even
when I was building for 4 machines.... As I said, the old deploy
directory worked just fine as a feed out of the box. If people weren't
using it as a feed, please explain the existence of
meta/package-index.bb.

> 
> > A slugos deploy dir looks like this now:
> > 
> > all
> > armv5te
> > i686
> > ixp4xxle
> > 
> > I honestly don't see any advantage to the previous mode of operation.
> 
> Go read the openembedded-devel list some time.

Well, If I wouldn't read it from time to time, I wouldn't have
responded to this thread in the first place.
Sounds logical, now, doesn't it?

> 
> > If
> > it is needed for unique situations 
> 
> No

Oh yes.

> > like the rare case of a multi-machine build
> 
> Apart from that now being the point, 

Yes, that's exactly the point. A change benefiting very few people
affecting basically every one in a mostly negative way.

> it actually isn't rare, the majority of the
> (maintained) distros in OE use it (celinux. generic, angstrom, openmoko, openzaurus,
> slugos and jline).

Oh right, because the majority of OE users are distribution maintainers,
right!
And every maintainer always builds every machine in existance. Sorry, I
forgot!
</sarcasm>

Your SlugOS exsample is pretty braindead as I had already shown you in
my previous email that slugos builds _suffer_ from the change, they
don't benefit from it. You do actually _read_ the emails you are
"responding" to, right?

SlugOS doesn't even use multi-machine magic to build images for
multiple supported machines (it may or may not use it for LE/BE
builds). You might wanna check with reality from time to time before
spouting nonsense.

> > it should be off by default and - by all means - optional.
> 
> No it should not. Next time respond to the RFCs instead of starting some uninformed rant
> afterward.

a) Not even 24 hours passed between the first email and the resulting
commit. What kind of RFC is that? How are people supposed to actually
comment on it (you know, the C part of RFC? ) if the commit is out
before most subscribers even noticed what was going on?

b) I do have other things on my mind than to read this mailinglist on
a daily basis. I only do so from time to time and respond to matters
of my concern and / or interest.

c) It wasn't a "rant", I was stating facts, even if you are too
close-minded to see it.

d) It was in no way uninformed. You may remember me maintaining my own
bloody distribution for multiple machines in the past? How much more
informed about this matter do you want me to get? I even gave a very
informed short example ( in the form of SlugOS ) of why I think the
change should be optional. If anyone is posting uninformed "rants", it
is you.

e) If you do not want any comments, then by all means, don't request
them! But if you actually _do_ care what other people think about a
certain matter, please stop responding like a know-all-be-all impolite
smart-ass that puts down other people for not thinking 100% alike.





^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: RFC: splitting deploy/ipk into subarchs
  2007-02-23  8:05           ` Koen Kooi
  2007-02-24  0:03             ` Matthias Hentges
@ 2007-02-24 11:18             ` Michael Rozdoba
  1 sibling, 0 replies; 9+ messages in thread
From: Michael Rozdoba @ 2007-02-24 11:18 UTC (permalink / raw)
  To: openembedded-devel

Koen Kooi wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Matthias Hentges schreef:
>> Am Dienstag, den 20.02.2007, 14:48 +0100 schrieb Koen Kooi:
>>> -----BEGIN PGP SIGNED MESSAGE-----

>> it should be off by default and - by all means - optional.
> 
> No it should not. Next time respond to the RFCs instead of starting some uninformed rant
> afterward.

Matthias's post did not read as a rant.

A response earlier certainly would have been more helpful, however as he 
points out, if the preceding procedure was meant to be a RFC the period 
allowed for this was very short.

How long should be allowed for feedback is an issue I'm not qualified to 
comment on, however given the time allowed it is certainly unreasonable 
at the very least to complain when constructive feedback is received 
after the event.

-- 
Michael
m r o z a t u k g a t e w a y d o t n e t



^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2007-02-24 11:18 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-02-19 11:18 RFC: splitting deploy/ipk into subarchs Koen Kooi
2007-02-19 11:59 ` Marcin Juszkiewicz
2007-02-19 12:30   ` Koen Kooi
2007-02-20  8:19     ` Koen Kooi
2007-02-20 13:48       ` Koen Kooi
2007-02-23  2:26         ` Matthias Hentges
2007-02-23  8:05           ` Koen Kooi
2007-02-24  0:03             ` Matthias Hentges
2007-02-24 11:18             ` Michael Rozdoba

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.