Integrating Golang

Integrating Golang

[Leo Schwab]

We have gotten it into our minds to write some of the apps on our
device in Go, a shiny new compiled language from Google (
http://golang.org/ ).  However, Go has very -- shall we say --
independent ideas about how to organize, fetch, build, and deploy code
which would, at first glance, appear to conflict with Yocto's
mechanisms for same.  Go's support for cross-compilation is also
fairly new and undergoing refinement.  We've kluged together a couple
of recipes which are target-specific, but a general solution for
Go-built targets in Yocto has not revealed itself.

Has anyone else done any work here?  Is there anything that I can
steal^H^H^H^H^Htake inspiration from?  Or am I in completely
unexplored territory?

Thanks in advance,
Schwab

Re: Integrating Golang

[Hermanus Botha]

So I don't really know much about Yocto. But I would also like to see some 
more Go-lang action on Yocto.

This section of their manual seems promising. 
http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#cross-
development-toolchain-generation

Re: Integrating Golang

[Hermanus Botha]

Hermanus Botha <wkj.id10t@...> writes:

> 
> So I don't really know much about Yocto. But I would also like to see some 
> more Go-lang action on Yocto.
> 
> This section of their manual seems promising. 
> http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#cross-
> development-toolchain-generation
> 


I guess this link is broken. In that manual, it's section: 
"4.2. Cross-Development Toolchain Generation"

Manual: 
http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html

Re: Integrating Golang

[Khem Raj]

On Wed, Apr 16, 2014 at 12:10 PM, Hermanus Botha <wkj.id10t@gmail.com> wrote:
> So I don't really know much about Yocto. But I would also like to see some
> more Go-lang action on Yocto.

Start by generating the gcc runtime  and language support in gcc
recipes. add it to RUNTIMETARGET and adjust PACKAGES
you also need to add it to LANGUAGES that should get it to some point.

>
> This section of their manual seems promising.
> http://www.yoctoproject.org/docs/current/ref-manual/ref-manual.html#cross-
> development-toolchain-generation
>
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

Re: Integrating Golang

[Martin Donnelly]

On 05/03/2014 21:55, Leo Schwab wrote:
> Has anyone else done any work here?  Is there anything that I can 
> steal^H^H^H^H^Htake inspiration from?  Or am I in completely 
> unexplored territory?

I posted an RFC patch set last year but it didn't get any traction and
we subsequently ruled out use of go so I never followed up on it but it
might be a useful starting point.

http://lists.openembedded.org/pipermail/openembedded-core/2013-May/078606.html

Cheers

- Martin

Re: Integrating Golang

[Khem Raj]

On Thu, Apr 17, 2014 at 1:49 AM, Martin Donnelly <martin.donnelly@ge.com> wrote:
> On 05/03/2014 21:55, Leo Schwab wrote:
>> Has anyone else done any work here?  Is there anything that I can
>> steal^H^H^H^H^Htake inspiration from?  Or am I in completely
>> unexplored territory?
>
> I posted an RFC patch set last year but it didn't get any traction and
> we subsequently ruled out use of go so I never followed up on it but it
> might be a useful starting point.
>
> http://lists.openembedded.org/pipermail/openembedded-core/2013-May/078606.html
>

Thats looks reasonable patch to me. Provided it can be forward ported
to gcc 4.8+
you should resubmit it for master

> Cheers
>
> - Martin
> --
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

Re: Integrating Golang

[Leo Schwab]

On Thu, Apr 17, 2014 at 1:49 AM, Martin Donnelly <martin.donnelly@ge.com> wrote:
> On 05/03/2014 21:55, Leo Schwab wrote:
>> Has anyone else done any work here?  Is there anything that I can
>> steal^H^H^H^H^Htake inspiration from?  Or am I in completely
>> unexplored territory?
>
> I posted an RFC patch set last year but it didn't get any traction and
> we subsequently ruled out use of go so I never followed up on it but it
> might be a useful starting point.
>
> http://lists.openembedded.org/pipermail/openembedded-core/2013-May/078606.html
>
We cobbled together our own recipes to build the Go compiler
(available upon request), which seem to be working well enough.  We
didn't need to make any changes to the recipes building the GCC
toolchain.

What I was more interested in learning was how people were integrating
the building of Go applications into Yocto.  Specifically: 'go get
foo' will build 'foo' and, in the process, fetch and build all its
dependencies.  This conflicts directly with Yocto's duties -- fetching
and building dependencies.

My current not-solution is rather ugly:  Create a repository that is a
standard Go build environment pre-populated with the app source code
and all of its dependencies.  The idea is that 'go get foo' will find
everything it needs already there and won't try and fetch anything
else.  Then create a simple Yocto recipe that fetches this repository
and builds it using 'go get ...'.  The repository itself is maintained
via rather delicate use of git-subtree.

This gets the job done, and prevents Go and Yocto having a fight about
who fetches what and how, but it's damned ugly...

Schwab

Re: Integrating Golang

[Ilya Dmitrichenko]

Hi Leo and Martin,

On 17 April 2014 20:54, Leo Schwab <lschwab@sensity.com> wrote:
> On Thu, Apr 17, 2014 at 1:49 AM, Martin Donnelly <martin.donnelly@ge.com> wrote:
>> On 05/03/2014 21:55, Leo Schwab wrote:
>>> Has anyone else done any work here?  Is there anything that I can
>>> steal^H^H^H^H^Htake inspiration from?  Or am I in completely
>>> unexplored territory?
>>
>> I posted an RFC patch set last year but it didn't get any traction and
>> we subsequently ruled out use of go so I never followed up on it but it
>> might be a useful starting point.
>>
>> http://lists.openembedded.org/pipermail/openembedded-core/2013-May/078606.html
>>
> We cobbled together our own recipes to build the Go compiler
> (available upon request), which seem to be working well enough.  We
> didn't need to make any changes to the recipes building the GCC
> toolchain.

I would be very much interested to take a look at what you did. My
use-case right now is to build Docker with Yocto.

Are using gogcc? From what understood it doen't build static binary
like gc does... which is, to be honest, my favourite feature of Go.

> What I was more interested in learning was how people were integrating
> the building of Go applications into Yocto.  Specifically: 'go get
> foo' will build 'foo' and, in the process, fetch and build all its
> dependencies.  This conflicts directly with Yocto's duties -- fetching
> and building dependencies.

I understand your concern with `go get` fetching dependencies that
bitbake should rather fetch. However, I would rather avoid using
global $GOPATH and set it to be relative to the build directory, i.e.
have per-project $GOPATH. That way `go get` is just a step in the
build process and doesn't intervine with bitbake in any way.

I have previously written a few recipes for some Python packages, and
do know that Yocto doesn't use any of the convinient tools (i.e.
easy_install or pip), but I would debate that it should and, perhaps,
in combination with virtualenv as well. There is currently no support
for Python 3 in Yocto, which is something that needs looking into and
I would also look into using pip then also.

> My current not-solution is rather ugly:  Create a repository that is a
> standard Go build environment pre-populated with the app source code
> and all of its dependencies.  The idea is that 'go get foo' will find
> everything it needs already there and won't try and fetch anything
> else.  Then create a simple Yocto recipe that fetches this repository
> and builds it using 'go get ...'.  The repository itself is maintained
> via rather delicate use of git-subtree.
> This gets the job done, and prevents Go and Yocto having a fight about
> who fetches what and how, but it's damned ugly...

This is kind of an okay solution, as Go community is not interested to
have `go get` to handle versions of dependcy, which is somewhat fair.
So if you needs to pin specific revisions, you will have to do either
subtree or submodule or script that just clones those deps or or
anything else of this kind.

I don't think it's any useful if you have to write bitbake recipe for
each little Go library you might be using, this is not C and `go get`
just works for most part. And when it doesn't, I would either fork and
import from my own fork or checkout a working revision of give
dependency and do git add on it, checking it into my project's main
tree.

I think at this stage it's improtant to provide a working Go compiler
for the Yocto community to use on their own projects. There are
currently only very few things that one might treat as system-level
tools, well, I can only recall of one actually, and it's the one I'm
attempting to integrate - Docker.

Cheers,
-- 
Ilya

Re: Integrating Golang

[Ilya Dmitrichenko]

Just a quick update, thought I'd let you know that I have a basic
working layer here:

https://github.com/errordeveloper/oe-meta-go

Re: Integrating Golang

[Burton, Ross]

On 10 September 2014 13:53, Ilya Dmitrichenko <errordeveloper@gmail.com> wrote:
> Just a quick update, thought I'd let you know that I have a basic
> working layer here:
>
> https://github.com/errordeveloper/oe-meta-go

Could you consider submitting this to the layer index, layers.openembedded.org?

Cheers,
Ross

Re: Integrating Golang

[Paul Eggleton]

On Wednesday 10 September 2014 14:36:37 Burton, Ross wrote:
> On 10 September 2014 13:53, Ilya Dmitrichenko <errordeveloper@gmail.com> 
wrote:
> > Just a quick update, thought I'd let you know that I have a basic
> > working layer here:
> > 
> > https://github.com/errordeveloper/oe-meta-go
> 
> Could you consider submitting this to the layer index,
> layers.openembedded.org?

He has and it's been published. Thanks for the reminder though ;)

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: Integrating Golang

[Maciej Borzecki]

On Wednesday 10 of September 2014 13:53:14 Ilya Dmitrichenko wrote:
> Just a quick update, thought I'd let you know that I have a basic
> working layer here:
> 
> https://github.com/errordeveloper/oe-meta-go

There's also https://github.com/digitallumens/meta-golang

Based on that I've prepped some patches not long ago that integrate go into 
OE-core base on what Chris did in meta-golang. The patches are for both meta--
golang and OE-core. OE-core piece enables Go GCC frontend and libgo same way 
as Fortran or Java is handled. Also, Chris has already defined a usable 
golang.bbclass that handles gccgo build for you. So far it seems to work with 
some simple use cases on BeagleBone Black and Raspberry.

If there's interest I can update the patches to current master and post them 
to the list.

-- 

Maciej Borzęcki 
Senior Software Engineer Open-RnD Sp. z o.o. 
www.open-rnd.pl, Facebook, Twitter 
mobile: +48 telefon, fax: +48 42 657 9079 

Niniejsza wiadomość wraz z załącznikami może zawierać chronione prawem lub 
poufne informacje i została wysłana wyłącznie do wiadomości i użytku osób, do 
których została zaadresowana. Jeśli wiadomość została otrzymana przypadkowo 
zabrania się jej kopiowania lub rozsyłania do osób trzecich. W takim przypadku 
uprasza się o natychmiastowe zniszczenie wiadomości oraz poinformowanie 
nadawcy o zaistniałej sytuacji za pomocą wiadomości zwrotnej. Dziękujemy. 

This message, including any attachments hereto, may contain privileged or 
confidential information and is sent solely for the attention and use of the 
intended addressee(s). If you are not an intended addressee, you may neither 
use this message nor copy or deliver it to anyone. In such case, you should 
immediately destroy this message and kindly notify the sender by reply email. 
Thank you.

Re: Integrating Golang

[Chris Elledge]

I'm actively using meta-golang for development and continuing to add features as I find them necessary. If anyone has suggestions on what would be useful as a developer of embedded go applications, let me know. 

Specifically, I am most interested on input for what would be useful configuration variables to have available for golang based recipes:
* Ways to request how to breakdown a golang project into combinations of shared libraries, static libraries, and executables.
* Being able to define which golang packages are to be placed into libraries vs statically linked into the executables.

If someone manages to get this properly integrated into yocto, I would be happy to move my development directly to the project.

-Chris

On Oct 30, 2014, at 5:47 AM, Maciej Borzecki <maciej.borzecki@open-rnd.pl> wrote:
> There's also https://github.com/digitallumens/meta-golang
> 
> Based on that I've prepped some patches not long ago that integrate go into 
> OE-core base on what Chris did in meta-golang. The patches are for both meta--
> golang and OE-core. OE-core piece enables Go GCC frontend and libgo same way 
> as Fortran or Java is handled. Also, Chris has already defined a usable 
> golang.bbclass that handles gccgo build for you. So far it seems to work with 
> some simple use cases on BeagleBone Black and Raspberry.
> 
> If there's interest I can update the patches to current master and post them 
> to the list.