* RFC: splitting deploy/ipk into subarchs
@ 2007-02-19 11:18 Koen Kooi
2007-02-19 11:59 ` Marcin Juszkiewicz
0 siblings, 1 reply; 9+ messages in thread
From: Koen Kooi @ 2007-02-19 11:18 UTC (permalink / raw)
To: Using the OpenEmbedded metadata to build Linux Distributions
[-- Attachment #1: Type: text/plain, Size: 497 bytes --]
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
When using multimachine and ipkg deploy/ipk gets awfully crowded after a while. I want to
propose sorting it by subarch. Attached is a patch that adds support for that to
rootfs_ipk.bbclass.
What do you think about that?
regards,
Koen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)
iD8DBQFF2YeZMkyGM64RGpERAjYCAJ9nBjdpcvTlvijaQ7o/UlZGWg68tgCdHw4m
IzGLM3yx+X9k/ZgMo+WqT6E=
=N6sc
-----END PGP SIGNATURE-----
[-- Attachment #2: rootfs-ipk-with-sorted-deploy.patch --]
[-- Type: text/x-patch, Size: 1579 bytes --]
#
# old_revision [bc83c51286e87a2b8aa3f79f074cadc05dedb7de]
#
# patch "classes/rootfs_ipk.bbclass"
# from [7824b4ee6ac7524e12d7411575e8bbf4b8a9d321]
# to [016f168e95ecc8150984cc48a14ecbd73a5fd9ed]
#
============================================================
--- classes/rootfs_ipk.bbclass 7824b4ee6ac7524e12d7411575e8bbf4b8a9d321
+++ classes/rootfs_ipk.bbclass 016f168e95ecc8150984cc48a14ecbd73a5fd9ed
@@ -18,17 +18,26 @@ fakeroot rootfs_ipk_do_rootfs () {
mkdir -p ${IMAGE_ROOTFS}/dev
- if [ -z "${DEPLOY_KEEP_PACKAGES}" ]; then
+ ipkgarchs="${PACKAGE_ARCHS}"
+
+ if [ -z "${DEPLOY_KEEP_PACKAGES}" ]; then
touch ${DEPLOY_DIR_IPK}/Packages
ipkg-make-index -r ${DEPLOY_DIR_IPK}/Packages -p ${DEPLOY_DIR_IPK}/Packages -l ${DEPLOY_DIR_IPK}/Packages.filelist -m ${DEPLOY_DIR_IPK}
- fi
+ if [-e ${DEPLOY_DIR_IPK}/$arch/] ; then
+ touch ${DEPLOY_DIR_IPK}/$arch/Packages
+ ipkg-make-index -r ${DEPLOY_DIR_IPK}/$arch/Packages -p ${DEPLOY_DIR_IPK}/$arch/Packages -l ${DEPLOY_DIR_IPK}/$arch/Packages.filelist -m ${DEPLOY_DIR_IPK}/$arch
+ fi
+ fi
+
mkdir -p ${T}
echo "src oe file:${DEPLOY_DIR_IPK}" > ${T}/ipkg.conf
- ipkgarchs="${PACKAGE_ARCHS}"
priority=1
for arch in $ipkgarchs; do
echo "arch $arch $priority" >> ${T}/ipkg.conf
priority=$(expr $priority + 5)
+ if [-e ${DEPLOY_DIR_IPK}/$arch/Packages] ; then
+ echo "src oe file:${DEPLOY_DIR_IPK}/$arch" >> ${T}/ipkg.conf
+ fi
done
ipkg-cl ${IPKG_ARGS} update
if [ ! -z "${LINGUAS_INSTALL}" ]; then
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: RFC: splitting deploy/ipk into subarchs 2007-02-19 11:18 RFC: splitting deploy/ipk into subarchs Koen Kooi @ 2007-02-19 11:59 ` Marcin Juszkiewicz 2007-02-19 12:30 ` Koen Kooi 0 siblings, 1 reply; 9+ messages in thread From: Marcin Juszkiewicz @ 2007-02-19 11:59 UTC (permalink / raw) To: openembedded-devel [-- Attachment #1: Type: text/plain, Size: 677 bytes --] Dnia poniedziałek, 19 lutego 2007, Koen Kooi napisał: > When using multimachine and ipkg deploy/ipk gets awfully crowded after > a while. I want to propose sorting it by subarch. Attached is a patch > that adds support for that to rootfs_ipk.bbclass. > > What do you think about that? I like this idea. ipkg-make-index is slow enough already. Attached fixed, tested version of your rootfs_ipk.bbclass change. Also added patch to package_ipk.bbclass which will store package in proper dir. -- JID: hrw-jabber.org OpenEmbedded developer/consultant Great minds discuss ideas; average minds discuss events; small minds discuss people. [-- Attachment #2: build-rootfs-from-split-ipkdir.diff --] [-- Type: text/x-diff, Size: 1601 bytes --] # # old_revision [bc83c51286e87a2b8aa3f79f074cadc05dedb7de] # # patch "classes/rootfs_ipk.bbclass" # from [7824b4ee6ac7524e12d7411575e8bbf4b8a9d321] # to [a30d56a031d7b85d13bb62d4956e77252a4b79fa] # ============================================================ --- classes/rootfs_ipk.bbclass 7824b4ee6ac7524e12d7411575e8bbf4b8a9d321 +++ classes/rootfs_ipk.bbclass a30d56a031d7b85d13bb62d4956e77252a4b79fa @@ -18,17 +18,25 @@ fakeroot rootfs_ipk_do_rootfs () { mkdir -p ${IMAGE_ROOTFS}/dev - if [ -z "${DEPLOY_KEEP_PACKAGES}" ]; then - touch ${DEPLOY_DIR_IPK}/Packages - ipkg-make-index -r ${DEPLOY_DIR_IPK}/Packages -p ${DEPLOY_DIR_IPK}/Packages -l ${DEPLOY_DIR_IPK}/Packages.filelist -m ${DEPLOY_DIR_IPK} - fi + ipkgarchs="${PACKAGE_ARCHS}" + + for arch in $ipkgarchs; do + if [ -z "${DEPLOY_KEEP_PACKAGES}" ]; then + if [ -e ${DEPLOY_DIR_IPK}/$arch/ ] ; then + touch ${DEPLOY_DIR_IPK}/$arch/Packages + ipkg-make-index -r ${DEPLOY_DIR_IPK}/$arch/Packages -p ${DEPLOY_DIR_IPK}/$arch/Packages -l ${DEPLOY_DIR_IPK}/$arch/Packages.filelist -m ${DEPLOY_DIR_IPK}/$arch/ + fi + fi + done + mkdir -p ${T} - echo "src oe file:${DEPLOY_DIR_IPK}" > ${T}/ipkg.conf - ipkgarchs="${PACKAGE_ARCHS}" priority=1 for arch in $ipkgarchs; do echo "arch $arch $priority" >> ${T}/ipkg.conf priority=$(expr $priority + 5) + if [ -e ${DEPLOY_DIR_IPK}/$arch/Packages ] ; then + echo "src oe-$arch file:${DEPLOY_DIR_IPK}/$arch" >> ${T}/ipkg.conf + fi done ipkg-cl ${IPKG_ARGS} update if [ ! -z "${LINGUAS_INSTALL}" ]; then [-- Attachment #3: store-package-in-proper-ipkdir.diff --] [-- Type: text/x-diff, Size: 711 bytes --] # # old_revision [bc83c51286e87a2b8aa3f79f074cadc05dedb7de] # # patch "classes/package_ipk.bbclass" # from [6f5d44fb7cf065ce7b680009e5c9bcb554f760c4] # to [57535d540caec17311adeb1e15f49d290d6cc8ac] # ============================================================ --- classes/package_ipk.bbclass 6f5d44fb7cf065ce7b680009e5c9bcb554f760c4 +++ classes/package_ipk.bbclass 57535d540caec17311adeb1e15f49d290d6cc8ac @@ -75,6 +75,9 @@ python do_package_ipk () { if not outdir: bb.error("DEPLOY_DIR_IPK not defined, unable to package") return + + arch = bb.data.getVar('PACKAGE_ARCH', d, 1) + outdir = "%s/%s" % (outdir, arch) bb.mkdirhier(outdir) dvar = bb.data.getVar('D', d, 1) ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: RFC: splitting deploy/ipk into subarchs 2007-02-19 11:59 ` Marcin Juszkiewicz @ 2007-02-19 12:30 ` Koen Kooi 2007-02-20 8:19 ` Koen Kooi 0 siblings, 1 reply; 9+ messages in thread From: Koen Kooi @ 2007-02-19 12:30 UTC (permalink / raw) To: openembedded-devel -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Marcin Juszkiewicz schreef: > Dnia poniedziałek, 19 lutego 2007, Koen Kooi napisał: > >> When using multimachine and ipkg deploy/ipk gets awfully crowded after >> a while. I want to propose sorting it by subarch. Attached is a patch >> that adds support for that to rootfs_ipk.bbclass. >> >> What do you think about that? > > I like this idea. ipkg-make-index is slow enough already. > > Attached fixed, tested version of your rootfs_ipk.bbclass change. Also > added patch to package_ipk.bbclass which will store package in proper > dir. Nice > - echo "src oe file:${DEPLOY_DIR_IPK}" > ${T}/ipkg.conf This breaks backward compat, could you add that back in? regards, Koen -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFF2ZhMMkyGM64RGpERAtwOAJ0d4gxAt3a/XFjsJP+5xFvsapJdswCePBQN QCZEkVp954PrWVq1QERxVtQ= =Qtcf -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: RFC: splitting deploy/ipk into subarchs 2007-02-19 12:30 ` Koen Kooi @ 2007-02-20 8:19 ` Koen Kooi 2007-02-20 13:48 ` Koen Kooi 0 siblings, 1 reply; 9+ messages in thread From: Koen Kooi @ 2007-02-20 8:19 UTC (permalink / raw) To: openembedded-devel -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Koen Kooi schreef: > Marcin Juszkiewicz schreef: >> Dnia poniedziaBek, 19 lutego 2007, Koen Kooi napisaB: > >>> When using multimachine and ipkg deploy/ipk gets awfully crowded after >>> a while. I want to propose sorting it by subarch. Attached is a patch >>> that adds support for that to rootfs_ipk.bbclass. >>> >>> What do you think about that? >> I like this idea. ipkg-make-index is slow enough already. > >> Attached fixed, tested version of your rootfs_ipk.bbclass change. Also >> added patch to package_ipk.bbclass which will store package in proper >> dir. Marcin and I merged a slightly different version of this today. regards, Koen -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFF2q70MkyGM64RGpERAjXNAJ9B2sh7Ly/KvcFkSfKNmAgXw+7VwACfbFfn /sKicIMGzfx8EYkT/B/YR78= =s1Iz -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: RFC: splitting deploy/ipk into subarchs 2007-02-20 8:19 ` Koen Kooi @ 2007-02-20 13:48 ` Koen Kooi 2007-02-23 2:26 ` Matthias Hentges 0 siblings, 1 reply; 9+ messages in thread From: Koen Kooi @ 2007-02-20 13:48 UTC (permalink / raw) To: openembedded-devel -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Koen Kooi schreef: > Marcin and I merged a slightly different version of this today. As some people found out, their upload scripts broke with this change. I'm sorry that happened, but those scripts are more easily fixed than ipkg-utils :( Due to bug in ipkg-utils being I haven't been using a straight rsync for a while now, so for the curious: rsync -az . user@remote.host:/feed/location/unsorted ; ssh user@remote.host 'cd feed/location/unsorted ; sh sort.sh' where sort.sh is http://www.angstrom-distribution.org/unstable/feed/unsorted/sort.sh.txt And while you're at it, check wether your rsync uses --delete, and if not, check if you remove Packages* before running ipkg-make-index. If that wasn't the case you have been unknowingly suffering for a buggy Packages* for a while now. Combined with ipkg .163 on the device I suspect 'ipkg upgrade' doesn't behave as deterministically as you might think. regards, Koen -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFF2vw3MkyGM64RGpERAv9OAJ4ij31yxqCC8M855mQ8hUvhdj56UQCfVk4z zfpTSMD0IajJc9VRF2dyuQA= =rvcF -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: RFC: splitting deploy/ipk into subarchs 2007-02-20 13:48 ` Koen Kooi @ 2007-02-23 2:26 ` Matthias Hentges 2007-02-23 8:05 ` Koen Kooi 0 siblings, 1 reply; 9+ messages in thread From: Matthias Hentges @ 2007-02-23 2:26 UTC (permalink / raw) To: openembedded-devel Am Dienstag, den 20.02.2007, 14:48 +0100 schrieb Koen Kooi: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Koen Kooi schreef: > > > Marcin and I merged a slightly different version of this today. > > As some people found out, their upload scripts broke with this change. I'm sorry that > happened, but those scripts are more easily fixed than ipkg-utils :( > Due to bug in ipkg-utils being I haven't been using a straight rsync for a while now, so > for the curious: Why not make this change optional via a variable (ie: SPLIT_DEPLOYDIR_IPK = -1 to turn it off) ? The current version breaks local OE feeds and requires needless post-processing. A slugos deploy dir looks like this now: all armv5te i686 ixp4xxle I honestly don't see any advantage to the previous mode of operation. If it is needed for unique situations like the rare case of a multi-machine build, it should be off by default and - by all means - optional. My 2 cents -- Matthias Hentges ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: RFC: splitting deploy/ipk into subarchs 2007-02-23 2:26 ` Matthias Hentges @ 2007-02-23 8:05 ` Koen Kooi 2007-02-24 0:03 ` Matthias Hentges 2007-02-24 11:18 ` Michael Rozdoba 0 siblings, 2 replies; 9+ messages in thread From: Koen Kooi @ 2007-02-23 8:05 UTC (permalink / raw) To: openembedded-devel -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Matthias Hentges schreef: > Am Dienstag, den 20.02.2007, 14:48 +0100 schrieb Koen Kooi: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Koen Kooi schreef: >> >>> Marcin and I merged a slightly different version of this today. >> As some people found out, their upload scripts broke with this change. I'm sorry that >> happened, but those scripts are more easily fixed than ipkg-utils :( >> Due to bug in ipkg-utils being I haven't been using a straight rsync for a while now, so >> for the curious: > > Why not make this change optional via a variable > (ie: SPLIT_DEPLOYDIR_IPK = -1 to turn it off) ? > > The current version breaks local OE feeds People's feeds are outside the scope of OE. Repeat after me: deploy never was, isn't and never will be a feed. > and requires needless post-processing. As I pointed out before, you needed postprocessing anyway due to bugs in ipkg and ipkg-utils, so your point is moot. > A slugos deploy dir looks like this now: > > all > armv5te > i686 > ixp4xxle > > I honestly don't see any advantage to the previous mode of operation. Go read the openembedded-devel list some time. > If > it is needed for unique situations No > like the rare case of a multi-machine build Apart from that now being the point, it actually isn't rare, the majority of the (maintained) distros in OE use it (celinux. generic, angstrom, openmoko, openzaurus, slugos and jline). > it should be off by default and - by all means - optional. No it should not. Next time respond to the RFCs instead of starting some uninformed rant afterward. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFF3qA+MkyGM64RGpERAsq1AJ9jkjFqiolnJWv/Mul6S59sCG2gZwCcDWRi WEqSA0lYqeEuhz/9vBImS00= =EmZ6 -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: RFC: splitting deploy/ipk into subarchs 2007-02-23 8:05 ` Koen Kooi @ 2007-02-24 0:03 ` Matthias Hentges 2007-02-24 11:18 ` Michael Rozdoba 1 sibling, 0 replies; 9+ messages in thread From: Matthias Hentges @ 2007-02-24 0:03 UTC (permalink / raw) To: openembedded-devel Am Freitag, den 23.02.2007, 09:05 +0100 schrieb Koen Kooi: > Matthias Hentges schreef: > > Am Dienstag, den 20.02.2007, 14:48 +0100 schrieb Koen Kooi: > >> -----BEGIN PGP SIGNED MESSAGE----- > >> Hash: SHA1 > >> > >> Koen Kooi schreef: > >> > > Why not make this change optional via a variable > > (ie: SPLIT_DEPLOYDIR_IPK = -1 to turn it off) ? > > > > The current version breaks local OE feeds > > People's feeds are outside the scope of OE. Repeat after me: deploy never was, isn't and > never will be a feed. It worked just fine as a feed before the change and people were using it as such, deal with it. > > and requires needless post-processing. > > As I pointed out before, you needed postprocessing anyway due to bugs in ipkg and > ipkg-utils, so your point is moot. You may have needed post-processing, I personally never did. Not even when I was building for 4 machines.... As I said, the old deploy directory worked just fine as a feed out of the box. If people weren't using it as a feed, please explain the existence of meta/package-index.bb. > > > A slugos deploy dir looks like this now: > > > > all > > armv5te > > i686 > > ixp4xxle > > > > I honestly don't see any advantage to the previous mode of operation. > > Go read the openembedded-devel list some time. Well, If I wouldn't read it from time to time, I wouldn't have responded to this thread in the first place. Sounds logical, now, doesn't it? > > > If > > it is needed for unique situations > > No Oh yes. > > like the rare case of a multi-machine build > > Apart from that now being the point, Yes, that's exactly the point. A change benefiting very few people affecting basically every one in a mostly negative way. > it actually isn't rare, the majority of the > (maintained) distros in OE use it (celinux. generic, angstrom, openmoko, openzaurus, > slugos and jline). Oh right, because the majority of OE users are distribution maintainers, right! And every maintainer always builds every machine in existance. Sorry, I forgot! </sarcasm> Your SlugOS exsample is pretty braindead as I had already shown you in my previous email that slugos builds _suffer_ from the change, they don't benefit from it. You do actually _read_ the emails you are "responding" to, right? SlugOS doesn't even use multi-machine magic to build images for multiple supported machines (it may or may not use it for LE/BE builds). You might wanna check with reality from time to time before spouting nonsense. > > it should be off by default and - by all means - optional. > > No it should not. Next time respond to the RFCs instead of starting some uninformed rant > afterward. a) Not even 24 hours passed between the first email and the resulting commit. What kind of RFC is that? How are people supposed to actually comment on it (you know, the C part of RFC? ) if the commit is out before most subscribers even noticed what was going on? b) I do have other things on my mind than to read this mailinglist on a daily basis. I only do so from time to time and respond to matters of my concern and / or interest. c) It wasn't a "rant", I was stating facts, even if you are too close-minded to see it. d) It was in no way uninformed. You may remember me maintaining my own bloody distribution for multiple machines in the past? How much more informed about this matter do you want me to get? I even gave a very informed short example ( in the form of SlugOS ) of why I think the change should be optional. If anyone is posting uninformed "rants", it is you. e) If you do not want any comments, then by all means, don't request them! But if you actually _do_ care what other people think about a certain matter, please stop responding like a know-all-be-all impolite smart-ass that puts down other people for not thinking 100% alike. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: RFC: splitting deploy/ipk into subarchs 2007-02-23 8:05 ` Koen Kooi 2007-02-24 0:03 ` Matthias Hentges @ 2007-02-24 11:18 ` Michael Rozdoba 1 sibling, 0 replies; 9+ messages in thread From: Michael Rozdoba @ 2007-02-24 11:18 UTC (permalink / raw) To: openembedded-devel Koen Kooi wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Matthias Hentges schreef: >> Am Dienstag, den 20.02.2007, 14:48 +0100 schrieb Koen Kooi: >>> -----BEGIN PGP SIGNED MESSAGE----- >> it should be off by default and - by all means - optional. > > No it should not. Next time respond to the RFCs instead of starting some uninformed rant > afterward. Matthias's post did not read as a rant. A response earlier certainly would have been more helpful, however as he points out, if the preceding procedure was meant to be a RFC the period allowed for this was very short. How long should be allowed for feedback is an issue I'm not qualified to comment on, however given the time allowed it is certainly unreasonable at the very least to complain when constructive feedback is received after the event. -- Michael m r o z a t u k g a t e w a y d o t n e t ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2007-02-24 11:18 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2007-02-19 11:18 RFC: splitting deploy/ipk into subarchs Koen Kooi 2007-02-19 11:59 ` Marcin Juszkiewicz 2007-02-19 12:30 ` Koen Kooi 2007-02-20 8:19 ` Koen Kooi 2007-02-20 13:48 ` Koen Kooi 2007-02-23 2:26 ` Matthias Hentges 2007-02-23 8:05 ` Koen Kooi 2007-02-24 0:03 ` Matthias Hentges 2007-02-24 11:18 ` Michael Rozdoba
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.