[Buildroot] Chromium v7 test

[Buildroot] Chromium v7 test

[Fame Castle]

Hi Joseph,

I finally found the time to test your chromium-v7-wip branch. I cloned the
repo (only the chromium-v7-wip branch) into a new buildroot folder. I then
set up a Linux 4.18 kernel, GlibC, Mesa and XOrg. The build fails at a step
I think I have seen before. (DeJa Vu?)

This is the output:

.../ChromeTest/buildroot/output/build/chromium-68.0.3440.106/build/config/linux/pkg-config.py
nss -v -lssl3
Returned 1.
stderr:

Package nss was not found in the pkg-config search path.
Perhaps you should add the directory containing `nss.pc'
to the PKG_CONFIG_PATH environment variable
No package 'nss' found
Could not run pkg-config.

See //third_party/nss/BUILD.gn:15:3: whence it was called.
  pkg_config("system_nss_no_ssl_config") {
  ^---------------------------------------
See //crypto/BUILD.gn:221:25: which caused the file to be included.
    public_configs += [ "//third_party/nss:system_nss_no_ssl_config" ]
                        ^-------------------------------------------

I compiled for i586 architecture.

Is this the problem you have to deal with too?

Cheers,
Famecastle
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20180905/4daa3eb2/attachment.html>

[Buildroot] Chromium v7 test

[Thomas Preston]

Hi Fame, Joseph, buildroot,

Sorry to resurrect this old thread! We're in the process of creating a
buildroot package for CEF [0] and the work you've done on this Chromium package
is interesting!

What's the status of this work and is there a more up to date branch? Perhaps
we could collaborate?


On 05/09/2018 09:11, Fame Castle wrote:> Hi Joseph,
> I finally found the time to test your chromium-v7-wip branch. I cloned the
> repo (only the chromium-v7-wip branch) into a new buildroot folder. I then set
> up a Linux 4.18 kernel, GlibC, Mesa and XOrg. The build fails at a step I
> think I have seen before. (DeJa Vu?)
> [snip]
> Package nss was not found in the pkg-config search path.
> Perhaps you should add the directory containing `nss.pc'
> to the PKG_CONFIG_PATH environment variable
> No package 'nss' found
> Could not run pkg-config.
> [snip]

I fixed this by adding `host-nss` to the dependencies. I also had to patch
`crypto/nss_util.cc` to `#include <nss/nss.h>` directly because of a
conflicting nss.h header in the include path (from glibc I think). My WIP
changes are here [1], it starts to build but I'm still fighting components.
This isn't really a priority for us atm, the CEF package is.

@Joseph, I noticed that you removed the `bootstrap.py` stage and referenced
`gn` directly (rather than `out/Release/gn`) in your patch to build Chromium
v75 [2]. Is gn on your path somehow? I'm not sure if this WIP patch is supposed
to work yet :)

We're actively working on this, so happy to chat about it.

Many thanks,
Thomas

[0] https://bitbucket.org/chromiumembedded/cef/src/master/
[1] https://github.com/tompreston/buildroot/commits/tpreston/ct-chromium-v7-wip
[2] https://github.com/tompreston/buildroot/commit/e43ead54965b161459a10172c152f4df7ac0721a#diff-26dd0e5b06da6c4c821993a00b55177fL165

[Buildroot] Chromium v7 test

[Joseph Kogut]

Hi Thomas,

On Wed, Sep 11, 2019 at 3:24 AM Thomas Preston
<thomas.preston@codethink.co.uk> wrote:
>
> Hi Fame, Joseph, buildroot,
>
> Sorry to resurrect this old thread! We're in the process of creating a
> buildroot package for CEF [0] and the work you've done on this Chromium package
> is interesting!
>

No problem. This is an interesting area, and I'd be glad to see some
Chromium based packages and applications added. Part of my motivation
for creating a Chromium package was actually to add CEF at some point.

> What's the status of this work and is there a more up to date branch? Perhaps
> we could collaborate?
>
>

The last build I tried was Chromium 75.0.3770.100. You can find that
branch here [1].

> On 05/09/2018 09:11, Fame Castle wrote:> Hi Joseph,
> > I finally found the time to test your chromium-v7-wip branch. I cloned the
> > repo (only the chromium-v7-wip branch) into a new buildroot folder. I then set
> > up a Linux 4.18 kernel, GlibC, Mesa and XOrg. The build fails at a step I
> > think I have seen before. (DeJa Vu?)
> > [snip]
> > Package nss was not found in the pkg-config search path.
> > Perhaps you should add the directory containing `nss.pc'
> > to the PKG_CONFIG_PATH environment variable
> > No package 'nss' found
> > Could not run pkg-config.
> > [snip]
>
> I fixed this by adding `host-nss` to the dependencies. I also had to patch
> `crypto/nss_util.cc` to `#include <nss/nss.h>` directly because of a
> conflicting nss.h header in the include path (from glibc I think). My WIP
> changes are here [1], it starts to build but I'm still fighting components.
> This isn't really a priority for us atm, the CEF package is.
>
> @Joseph, I noticed that you removed the `bootstrap.py` stage and referenced
> `gn` directly (rather than `out/Release/gn`) in your patch to build Chromium
> v75 [2]. Is gn on your path somehow? I'm not sure if this WIP patch is supposed
> to work yet :)
>

Right. It's been a while since I've worked on this branch, but I would
assume that gn is being found on my system path here. I think my
intention was to add it as a package at some point. Anyway, the WIP
branch isn't supposed to work yet, which is why I haven't submitted
it.

> We're actively working on this, so happy to chat about it.
>

Thanks for starting the work on this, and I'm happy to contribute
however I can. If you push your work to a public repository, I'd be
happy to look it over, and you can CC me on any patches you send to
the mailing list.

Best,
Joseph

[1] https://github.com/jakogut/buildroot/tree/chromium-v7-wip

[Buildroot] Chromium v7 test

[Thomas Preston]

Hi,
Thanks for getting back to us so quickly.

On 11/09/2019 17:29, Joseph Kogut wrote:
> On Wed, Sep 11, 2019 at 3:24 AM Thomas Preston
> <thomas.preston@codethink.co.uk> wrote:
>> Sorry to resurrect this old thread! We're in the process of creating a
>> buildroot package for CEF [0] and the work you've done on this Chromium package
>> is interesting!
> 
> No problem. This is an interesting area, and I'd be glad to see some
> Chromium based packages and applications added. Part of my motivation
> for creating a Chromium package was actually to add CEF at some point.

Okay great. Our original strategy was to have the CEF package depend on the
Chromium package but as it turns out, CEF modifies the Chromium source, then
relies heavily on the Chromium build system. This isn't something which
buildroot really supports, so we're just downloading the Chromium tarball in
the CEF package, using the _EXTRA_DOWNLOADS makefile variable.

This might change as we discover more though...

>> @Joseph, I noticed that you removed the `bootstrap.py` stage and referenced
>> `gn` directly (rather than `out/Release/gn`) in your patch to build Chromium
>> v75 [2]. Is gn on your path somehow? I'm not sure if this WIP patch is supposed
>> to work yet :)
> 
> Right. It's been a while since I've worked on this branch, but I would
> assume that gn is being found on my system path here. I think my
> intention was to add it as a package at some point. Anyway, the WIP
> branch isn't supposed to work yet, which is why I haven't submitted
> it.

Okay got it. There seems to be a copy of gn bundled in with the Chromium tarball
at buildtools/linux64/gn, which is what the depot_tools/gn script searches for
anyway - so we're just using that for now. We /think/ we can get away without
using depot_tools.

> Thanks for starting the work on this, and I'm happy to contribute
> however I can. If you push your work to a public repository, I'd be
> happy to look it over, and you can CC me on any patches you send to
> the mailing list.

Will do, much appreciated! There's nothing stopping us from working in the open
so we'll get patches out as soon as we can.

Tom

[Buildroot] Chromium v7 test

[Fame Castle]

Hi Thomas,
it's great to see your interest in this. If I can help you in
any way, please let me know. I'm still interested in this as a long term
goal, while I'm working on different projects now. I would however
appreciate any progress with this.

I think the following is interesting:
I am using CEF with Python (called cefpython). I install it using the
Python Package Manager pip and then on my host machine I got a folder with
cef browser binaries somewhere. I just took those binaries and the Python
bindings and copied them into my Buildroot-OS. Then, I was actually able to
get my Python CEF application working. It went fluently! So at its core,
this should be really possible. However, this doesn't feel like a nice
implementation. And, some other window management libraries which theirself
take CEF and enable you to control the application window, failed with
Segmentation faults. I'm confident this may be due to the usage of my
host's binaries. But I think it's interesting that it worked. So we should
find our way and make successfull Chromium builds.

I hope you have good success with this. All the best,
~ Fame(castle)

On Wed, Sep 11, 2019 at 6:50 PM Thomas Preston <
thomas.preston@codethink.co.uk> wrote:

> Hi,
> Thanks for getting back to us so quickly.
>
> On 11/09/2019 17:29, Joseph Kogut wrote:
> > On Wed, Sep 11, 2019 at 3:24 AM Thomas Preston
> > <thomas.preston@codethink.co.uk> wrote:
> >> Sorry to resurrect this old thread! We're in the process of creating a
> >> buildroot package for CEF [0] and the work you've done on this Chromium
> package
> >> is interesting!
> >
> > No problem. This is an interesting area, and I'd be glad to see some
> > Chromium based packages and applications added. Part of my motivation
> > for creating a Chromium package was actually to add CEF at some point.
>
> Okay great. Our original strategy was to have the CEF package depend on the
> Chromium package but as it turns out, CEF modifies the Chromium source,
> then
> relies heavily on the Chromium build system. This isn't something which
> buildroot really supports, so we're just downloading the Chromium tarball
> in
> the CEF package, using the _EXTRA_DOWNLOADS makefile variable.
>
> This might change as we discover more though...
>
> >> @Joseph, I noticed that you removed the `bootstrap.py` stage and
> referenced
> >> `gn` directly (rather than `out/Release/gn`) in your patch to build
> Chromium
> >> v75 [2]. Is gn on your path somehow? I'm not sure if this WIP patch is
> supposed
> >> to work yet :)
> >
> > Right. It's been a while since I've worked on this branch, but I would
> > assume that gn is being found on my system path here. I think my
> > intention was to add it as a package at some point. Anyway, the WIP
> > branch isn't supposed to work yet, which is why I haven't submitted
> > it.
>
> Okay got it. There seems to be a copy of gn bundled in with the Chromium
> tarball
> at buildtools/linux64/gn, which is what the depot_tools/gn script searches
> for
> anyway - so we're just using that for now. We /think/ we can get away
> without
> using depot_tools.
>
> > Thanks for starting the work on this, and I'm happy to contribute
> > however I can. If you push your work to a public repository, I'd be
> > happy to look it over, and you can CC me on any patches you send to
> > the mailing list.
>
> Will do, much appreciated! There's nothing stopping us from working in the
> open
> so we'll get patches out as soon as we can.
>
> Tom
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20190911/4da0f6c7/attachment.html>

[Buildroot] Chromium v7 test

[Thomas Preston]

Hi Fame,

On 11/09/2019 17:57, Fame Castle wrote:
> Hi Thomas,
> it's great to see your interest in this. If I can help you in any
> way, please let me know. I'm still interested in this as a long term
> goal, while I'm working on different projects now. I would however
> appreciate any progress with this.

No problem, we'll keep you updated.

> I think the following is interesting:
> I am using CEF with Python (called cefpython). I install it using the
> Python Package Manager pip and then on my host machine I got a folder
> with cef browser binaries somewhere. I just took those binaries and the
> Python bindings and copied them into my Buildroot-OS. Then, I was
> actually able to get my Python CEF application working. It went
> fluently! So at its core, this should be really possible. However, this
> doesn't feel like a nice implementation. And, some other window
> management libraries which theirself take CEF and enable you to control
> the application window, failed with Segmentation faults. I'm confident
> this may be due to the usage of my host's binaries. But I think it's
> interesting that it worked. So we should find our way and make
> successfull Chromium builds.

Our client also has a work-around too, but maintaining this out-of-tree is a
pain.

Additionally we want to handle cross-compiling. Without trying it, Joseph's
method (using Configs) looks sensible, but I wonder if there's a tighter way to
integrate this with buildroot toolchain arch?

Tom

[Buildroot] Chromium v7 test

[Thomas Preston]

Hi Joseph, all,

On 11/09/2019 17:50, Thomas Preston wrote:
> On 11/09/2019 17:29, Joseph Kogut wrote:
>> Thanks for starting the work on this, and I'm happy to contribute
>> however I can. If you push your work to a public repository, I'd be
>> happy to look it over, and you can CC me on any patches you send to
>> the mailing list.
>
> Will do, much appreciated! There's nothing stopping us from working in the
> open so we'll get patches out as soon as we can.

We're still working through our initial CEF build and we're hoping to use the
chromium-v6 branch as a "working" reference. However we can't get it to build
for x86_64 because the generated ninja files have some crazy cflag includes
(from pkg-config):

    STAGING_DIR=/mnt/output/host/x86_64-buildroot-linux-gnu/sysroot

    $ grep "cflags =" output/build/chromium-67.0.3396.79/out/Release/obj/dbus/dbus.ninja
    cflags =
    [snip]
    -isystem../../../../host/x86_64-buildroot-linux-gnu/sysroot/mnt/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/include/dbus-1.0
    -isystem../../../../host/x86_64-buildroot-linux-gnu/sysroot/mnt/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/dbus-1.0/include
    -isystem../../../../host/x86_64-buildroot-linux-gnu/sysroot/mnt/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/include/glib-2.0
    -isystem../../../../host/x86_64-buildroot-linux-gnu/sysroot/mnt/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/glib-2.0/include
    -isystem../../../../host/x86_64-buildroot-linux-gnu/sysroot/mnt/output/host/x86_64-buildroot-linux-gnu/sysroot/usr/include
    [snip]

See how there is a relative path to the sysroot, then the full path. I have a
branch with a single fixup here [0]. There's a Debian 9 build container there
too because this branch is quite old now.

Joseph, I don't know if you saw a similar error, or could advise on the above?
It looks like pkg-config isn't using `target_sysroot` correctly.
Thanks


# CEF
Regarding the CEF package we're actually working on, we're having issues
deciding which values to use for `use_sysroot` and `use_custom_libcxx`.

We've gone with `use_sysroot=false` and we're trying to get pkg-config to give
us the right buildroot host/target includes, which we're having some success
with.

For `use_custom_libcxx=true`, we have an issue with both true and false cases:
With true, gn uses pkg-config to generate the following include paths in the
ninja files:

    -I/mnt/output/host/bin/../x86_64-buildroot-linux-gnu/sysroot/usr/include/glib-2.0
    -I/mnt/output/host/bin/../x86_64-buildroot-linux-gnu/sysroot/usr/lib/glib-2.0/include
    -I/mnt/output/host/bin/../x86_64-buildroot-linux-gnu/sysroot/usr/include
    [snip]
    -isystem../../buildtools/third_party/libc++/trunk/include
    -isystem../../buildtools/third_party/libc++abi/trunk/include

There is a GNU C math.h inside sysroot/usr/include which overrides (I think!)
the bundled LLVM libc++ cmath, which causes lots of global namespace errors to
do with math symbols.  Our client has a working build and their pkg-config does
not generate this `sysroot/usr/include` include path, so one solution might be
to fix the includes returned from pkg-config.

For `use_custom_libcxx=false`, the build gets a lot further but we see some
incorrect linking. For example:

    $ ldd output/build/libcef/out/Release/host/generate_colors_info
    [snip]
    libjpeg.so.8 => not found

    $ find output/host/ -name "*jpeg*"
    [snip]
    output/host/lib/pkgconfig/libjpeg.pc
    output/host/lib/libjpeg.so.9
    output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/pkgconfig/libjpeg.pc
    output/host/x86_64-buildroot-linux-gnu/sysroot/usr/lib/libjpeg.so.8

See how the host utility was incorrectly linked against the sysroot libjpeg. We
intend to get around this with the `pkg_config` and `host_pkgconfig` gn
arguments.

If anyone can advise on how the above is looking that would be useful. Once we
get a binary built, we'll send an RFC for people to take a look at.

Many thanks!
Thomas

[0] https://github.com/tompreston/buildroot/commits/tpreston/chromium-v6