From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail5.wrs.com (mail5.windriver.com [192.103.53.11]) by mail.openembedded.org (Postfix) with ESMTP id AA088774B5 for ; Thu, 2 Jun 2016 19:43:33 +0000 (UTC) Received: from ALA-HCA.corp.ad.wrs.com (ala-hca.corp.ad.wrs.com [147.11.189.40]) by mail5.wrs.com (8.15.2/8.15.2) with ESMTPS id u52JhGvr012992 (version=TLSv1 cipher=AES128-SHA bits=128 verify=OK); Thu, 2 Jun 2016 12:43:28 -0700 Received: from server.local (147.11.116.160) by ALA-HCA.corp.ad.wrs.com (147.11.189.40) with Microsoft SMTP Server id 14.3.248.2; Thu, 2 Jun 2016 12:43:18 -0700 To: Paul Eggleton References: <16461829.3PMlXGZ7eP@peggleto-mobl.ger.corp.intel.com> <0d8a5a78-304b-5018-556d-d586149d0fde@windriver.com> <1570741.InmcqNLx36@peggleto-mobl.ger.corp.intel.com> From: Bruce Ashfield Message-ID: <0b2ff27d-57b5-1c29-ab05-a6fd6cb2b321@windriver.com> Date: Thu, 2 Jun 2016 15:43:17 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.1.0 MIME-Version: 1.0 In-Reply-To: <1570741.InmcqNLx36@peggleto-mobl.ger.corp.intel.com> Cc: Chris Larson , openembedded-core@lists.openembedded.org Subject: Re: [PATCH 01/42] gcc: Add gcc6 recipes X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Jun 2016 19:43:33 -0000 Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit On 2016-06-02 3:38 PM, Paul Eggleton wrote: > On Thu, 02 Jun 2016 15:30:14 Bruce Ashfield wrote: >> On 2016-06-02 3:24 PM, Paul Eggleton wrote: >>> On Thu, 02 Jun 2016 15:19:37 Bruce Ashfield wrote: >>>> On 2016-06-02 3:15 PM, Paul Eggleton wrote: >>>>> On Thu, 02 Jun 2016 18:05:35 Martin Jansa wrote: >>>>>> On Wed, Jun 01, 2016 at 10:23:35AM +1200, Paul Eggleton wrote: >>>>>>> On Wed, 01 Jun 2016 10:20:23 Paul Eggleton wrote: >>>>>>>> On Tue, 31 May 2016 11:12:21 Jussi Kukkonen wrote: >>>>>>>>> On 11 May 2016 at 20:35, Khem Raj wrote: >>>>>>>>>> +#BASEURI ?= "${GNU_MIRROR}/gcc/gcc-${PV}/gcc-${PV}.tar.bz2" >>>>>>>>>> +BASEURI ?= "git:// >>>>>>>>>> github.com/gcc-mirror/gcc;branch=gcc-6-branch;protocol=git" >>>>>>>>> >>>>>>>>> I guess this is where git2_github.com.gcc-mirror.gcc.tar.gz download >>>>>>>>> comes from? It increased the size of my downloads-directory by >30% >>>>>>>>> -- >>>>>>>>> and I must have quite a bit of old junk in that directory already. >>>>>>>>> >>>>>>>>> Is there no other solution to this than a 2.5G git copy (honest >>>>>>>>> question, I don't know gcc)? >>>>>>>> >>>>>>>> We went down this road before and it wasn't great for users. There is >>>>>>>> at >>>>>>>> least a tarball for 6.1, we'd presumably need some patches on top of >>>>>>>> that (~43 if we were to simply take what's in the gcc-6-branch >>>>>>>> between >>>>>>>> 6.1 and bd9a826, minus the "daily bumps"). >>>>>>> >>>>>>> Perhaps that was a little unclear - when I say "we went down this road >>>>>>> before" I'm agreeing with Jussi - we pointed to a git branch in a >>>>>>> previous release with the same resulting huge download for everyone, >>>>>>> something I think we should avoid if at all possible. >>>>>> >>>>>> Maybe it's biggest but at least there is only one copy of it: >>>>>> >>>>>> top 10 in my downloads (archives and then git clones). >>>>>> >>>>>> 823M >>>>>> /OE/downloads/git2_git.yoctoproject.org.linux-yocto-3.14.git.tar.gz >>>>>> 831M >>>>>> /OE/downloads/git2_git.yoctoproject.org.linux-yocto-3.10.git.tar.gz >>>>>> 893M >>>>>> /OE/downloads/git2_git.yoctoproject.org.linux-yocto-3.17.git.tar.gz >>>>>> 934M >>>>>> /OE/downloads/git2_git.yoctoproject.org.linux-yocto-3.19.git.tar.gz >>>>>> 969M >>>>>> /OE/downloads/git2_git.yoctoproject.org.linux-yocto-4.1.git.tar.gz >>>>>> 1.2G >>>>>> /OE/downloads/git2_git.yoctoproject.org.linux-yocto-dev.git.tar.gz >>>>>> >>>>>> 2.4G /OE/downloads/git2_github.com.gcc-mirror.gcc.tar.gz >>>>>> >>>>>> 854M >>>>>> /OE/downloads/git2/github.com.qtproject.qtwebengine-chromium.git >>>>>> >>>>>> 877M /OE/downloads/git2/git.yoctoproject.org.linux-yocto-3.10.git >>>>>> 900M /OE/downloads/git2/git.yoctoproject.org.linux-yocto-3.14.git >>>>>> 921M /OE/downloads/git2/git.yoctoproject.org.linux-yocto-3.17.git >>>>>> 964M /OE/downloads/git2/git.yoctoproject.org.linux-yocto-3.19.git >>>>>> 999M /OE/downloads/git2/git.yoctoproject.org.linux-yocto-4.1.git >>>>>> 1.3G /OE/downloads/git2/git.yoctoproject.org.linux-yocto-dev.git >>>>>> 2.3G /OE/downloads/git2/git.yoctoproject.org.linux-yocto-4.4.git >>>>> >>>>> No argument from me there - it does seem a little wasteful given that >>>>> there must be a huge overlap between those repos. >>>> >>>> In other build systems, I've simply been able to reference a common >>>> repository for the majority of the blobs. That isn't an option with >>>> oe/bitbake (as far as I know). >>> >>> It's possible we could add functionality along those lines to at least >>> save download time; disk usage would probably be unaffected though. >> >> Nope. Disk usage goes down as well. With alternate objects, each >> repo is only the size of what is unique to the repo. > > I guess it depends on how you do it. Reducing both would be ideal, of course. > Both can absolutely be reduced. In the past, I've had a korg pristine clone available. That's your big fetch (download and disk), after that all kernel clones refer to it. They are only 10's of Megabytes on disk and on the wire after that. >>>> We could also do shallow clones, but I'm also unsure of how well that >>>> is supported. >>> >>> I know Chris looked into adding this, I don't recall how far he got. >>> >>>> With the amount of series that are completely rebased and dynamic in >>>> content, the trees do need to be separate per version. It becomes >>>> completely un-bisectable and maintainable any other way. >>> >>> Bisectability is function of how the branch is managed, not the repository >>> - surely? Perhaps I'm missing something. >> >> Terminology perhaps? branch, repository .. whatever. They are all >> the same. >> >> Granularity of kernel commits. Over time with features like lttng, >> aufs, preempt-rt, backports, etc, etc, It becomes impossible to >> integrate new features and kernel updates without conflicts. >> >> You end up with a tree riddled with merge commits, reverts and other >> garbage that obscure the real changes. >> >> I've lived that life, and its a not fun. > > I don't think anyone would want to move to that model, I agree it's ugly. It > seems to me that one repo doesn't mandate that though - all it would require > would be unique branch names, so you'd end up multiplying the number of > standard/xxx branches by the number of versions. We should definitely try > going down the alternate objects / shallow clones routes first before > potentially breaking people's workflow however. Yep. I've also tried that, in fact, that's where we started. You can have a jungle of branches .. and it can definitely work. But I keep getting pressed to reduce branches, not add more. But if push came to shove .. it definitely is viable (and less nasty than the alternatives). Bruce > > Cheers, > Paul >