All of lore.kernel.org
 help / color / mirror / Atom feed
* Trying to resolve build-time cyclic dependency between packages
@ 2015-07-29 13:26 Boszormenyi Zoltan
  2015-08-08 22:14 ` Khem Raj
  0 siblings, 1 reply; 4+ messages in thread
From: Boszormenyi Zoltan @ 2015-07-29 13:26 UTC (permalink / raw)
  To: openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 1039 bytes --]

Hi,

I have a customized Mesa 10.6.3 package recipe which depends on libva
to be able to build its mesa-va-driver subpackage, but libva also depends
on Mesa to support GLX.

Previously, I simply created a libva-noglx package that builds libva
with --disable-glx that Mesa depend on and the "real" libva package can
depend on Mesa.

But since I also have to produce an SDK image with all the *-dev packages,
creating such an initial image with bitbake fails because of a conflict
in the dependency chain: libva-dev -> mesa-dev -> libva-noglx-dev.

I have come up with the attached bbclass (based on Yocto 1.6) that
now Mesa can inherit and that patches the RDEPENDS / RRECOMMENDS
variables according to externally set variables. The comment at
the beginning of the file explains how to use it.

This pushes the cyclic dependency to opkg which happily breaks it and
installs both libva* and *mesa* packages properly. Creating the initial
image via bitbake works now.

Comments?

Best regards,
Zoltán Böszörményi


[-- Attachment #2: cyclic-deps.bbclass --]
[-- Type: text/plain, Size: 1766 bytes --]

# Resolve cyclic dependencies in final packages
#
# Example: Mesa vs libva cyclic dependency
#
# Resolving the cyclic dependency can be:
# 1. build libva without GLX support (libva-noglx package)
# 2. build mesa
# 3. build libva with GLX support
#
# Mesa's DEPENDS contains libva-noglx
# libva's DEPENDS contains mesa
# The subpackages of mesa will reference libva-noglx
#
# This class attempts to patch the final package dependencies
# with the following:
# CYCLIC_DEPENDS = "pkg"
# CYCLIC_DEPENDS_pkg = "basepkg"
#
# With the example of Mesa:
# CYCLIC_DEPENDS_PKGS = "libva"
# CYCLIC_DEPENDS_libva = "libva-noglx"

python package_depchains_append () {
    deps = d.getVar('CYCLIC_DEPENDS_PKGS', True).split()
    packages = d.getVar('PACKAGES', True).split()
    for pkg in packages:
        for dep in deps:
            depvar = 'CYCLIC_DEPENDS_' + dep
            basedep = d.getVar(depvar, True)
            bb.note("before cyclic dependency dep %s depvar %s basedep %s" % (dep, depvar, basedep))

            rdep = 'RDEPENDS_' + pkg
            rdepends = d.getVar(rdep, True)
            if rdepends:
                bb.note("before cyclic dependency fix %s = '%s'" % (rdep, rdepends))
                d.setVar(rdep, rdepends.replace(basedep, dep))
                bb.note("after cyclic dependency fix %s = '%s'" % (rdep, d.getVar(rdep, True)))

            rrec = 'RRECOMMENDS_' + pkg
            rrecommends = d.getVar(rrec, True)
            if rrecommends:
                bb.note("before cyclic dependency fix %s = '%s'" % (rrec, rrecommends))
                d.setVar(rrec, rrecommends.replace(basedep, dep))
                bb.note("after cyclic dependency fix %s = '%s'" % (rrec, d.getVar(rrec, True)))
}

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Trying to resolve build-time cyclic dependency between packages
  2015-07-29 13:26 Trying to resolve build-time cyclic dependency between packages Boszormenyi Zoltan
@ 2015-08-08 22:14 ` Khem Raj
  2020-02-26  8:49   ` Böszörményi Zoltán
  0 siblings, 1 reply; 4+ messages in thread
From: Khem Raj @ 2015-08-08 22:14 UTC (permalink / raw)
  To: openembedded-devel

[-- Attachment #1: Type: text/plain, Size: 1716 bytes --]


> On Jul 29, 2015, at 6:26 AM, Boszormenyi Zoltan <zboszor@pr.hu> wrote:
> 
> Hi,
> 
> I have a customized Mesa 10.6.3 package recipe which depends on libva
> to be able to build its mesa-va-driver subpackage, but libva also depends
> on Mesa to support GLX.
> 
> Previously, I simply created a libva-noglx package that builds libva
> with --disable-glx that Mesa depend on and the "real" libva package can
> depend on Mesa.
> 
> But since I also have to produce an SDK image with all the *-dev packages,
> creating such an initial image with bitbake fails because of a conflict
> in the dependency chain: libva-dev -> mesa-dev -> libva-noglx-dev.
> 

since libva-noglx is sort of a bootstrap package, you could empty/delete out the content
its puts into -dev packages and conflict will be gone. Other option is that you share
the content of -dev package in exclusive manner between libva and libva-noglx such
that they don’t package duplicate files

> I have come up with the attached bbclass (based on Yocto 1.6) that
> now Mesa can inherit and that patches the RDEPENDS / RRECOMMENDS
> variables according to externally set variables. The comment at
> the beginning of the file explains how to use it.
> 
> This pushes the cyclic dependency to opkg which happily breaks it and
> installs both libva* and *mesa* packages properly. Creating the initial
> image via bitbake works now.
> 
> Comments?
> 
> Best regards,
> Zoltán Böszörményi
> 
> <cyclic-deps.bbclass>--
> _______________________________________________
> Openembedded-devel mailing list
> Openembedded-devel@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-devel


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 211 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Trying to resolve build-time cyclic dependency between packages
  2015-08-08 22:14 ` Khem Raj
@ 2020-02-26  8:49   ` Böszörményi Zoltán
  2020-02-26 11:43     ` Böszörményi Zoltán
  0 siblings, 1 reply; 4+ messages in thread
From: Böszörményi Zoltán @ 2020-02-26  8:49 UTC (permalink / raw)
  To: openembedded-devel, Khem Raj

Hi,

I am reviving this old idea with Zeus.

I have implemented the idea in an ALMOST satisfactory way, tested with
Mesa 20.0.0.

Side note: Mesa 19.3 and 20.0 needs TOOLCHAIN="gcc" in Zeus,
otherwise linker errors occur. See:
https://gitlab.freedesktop.org/mesa/mesa/issues/2533

Mesa doesn't link to libva even with meson -Dgallium-va=true. It only
needs the libva.pc pkgconfig file and the headers from libva to build
its VA state tracker so the circular dependency is not that hard to solve.

I have this new recipe called libva-headers now that is a fork of the
libva recipe and just deletes the libraries in do_install_append.

My Mesa recipe addendum has this line plus PACKAGECONFIG has "va":

PACKAGECONFIG[va] = "-Dgallium-va=true,-Dgallium-va=false,libva-headers"

Now, building libva is happy without the warning about libva.so is
shipped by libva-headers (or libva-noglx in the previous incarnation)
but now marked as shipped by libva because it was built later.

There is another problem now: anything that uses libva as dependency,
like intel-vaapi-driver, implicitly pulls in mesa and then libva-headers
so there is a clash in prepare-sysroot between libva and libva-headers
because they still ship identical headers and pkgconfig files.

My current solution to the problem is to patch staging.bbclass to
handle a new variable called EXCLUDE_DEPS but a line like this below
needs to be added to all recipes that depends on libva, so it is
quite intrusive for now.

EXCLUDE_DEPS = "libva-headers"

I am looking into storing data in "sstate", so libva-headers would add
itself into a new class of packages that are not added to recipe-sysroot
unless it is in DEPENDS explicitly.

My question is: is there a generic framework to do this already?

This would allow me to not modify other recipes and may alsoserve other
recipes with circular dependencies.

The end goal is to allow Chromium to be built with use_vaapi=true now that
meta-browser has version 79 that includes the VAAPI patches. The current
solution with setting EXCLUDE_DEPS already does this but as I said, it's
not clean enough.

Best regards,
Zoltán Böszörményi

2015. 08. 09. 0:14 keltezéssel, Khem Raj írta:
> 
>> On Jul 29, 2015, at 6:26 AM, Boszormenyi Zoltan <zboszor@pr.hu> wrote:
>>
>> Hi,
>>
>> I have a customized Mesa 10.6.3 package recipe which depends on libva
>> to be able to build its mesa-va-driver subpackage, but libva also depends
>> on Mesa to support GLX.
>>
>> Previously, I simply created a libva-noglx package that builds libva
>> with --disable-glx that Mesa depend on and the "real" libva package can
>> depend on Mesa.
>>
>> But since I also have to produce an SDK image with all the *-dev packages,
>> creating such an initial image with bitbake fails because of a conflict
>> in the dependency chain: libva-dev -> mesa-dev -> libva-noglx-dev.
>>
> 
> since libva-noglx is sort of a bootstrap package, you could empty/delete out the content
> its puts into -dev packages and conflict will be gone. Other option is that you share
> the content of -dev package in exclusive manner between libva and libva-noglx such
> that they don’t package duplicate files
> 
>> I have come up with the attached bbclass (based on Yocto 1.6) that
>> now Mesa can inherit and that patches the RDEPENDS / RRECOMMENDS
>> variables according to externally set variables. The comment at
>> the beginning of the file explains how to use it.
>>
>> This pushes the cyclic dependency to opkg which happily breaks it and
>> installs both libva* and *mesa* packages properly. Creating the initial
>> image via bitbake works now.
>>
>> Comments?
>>
>> Best regards,
>> Zoltán Böszörményi
>>
>> <cyclic-deps.bbclass>--
>> _______________________________________________
>> Openembedded-devel mailing list
>> Openembedded-devel@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-devel
> 
> 



^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Trying to resolve build-time cyclic dependency between packages
  2020-02-26  8:49   ` Böszörményi Zoltán
@ 2020-02-26 11:43     ` Böszörményi Zoltán
  0 siblings, 0 replies; 4+ messages in thread
From: Böszörményi Zoltán @ 2020-02-26 11:43 UTC (permalink / raw)
  To: openembedded-devel, Khem Raj

2020. 02. 26. 9:49 keltezéssel, Böszörményi Zoltán via Openembedded-devel írta:
> Hi,
> 
> I am reviving this old idea with Zeus.
> 
> I have implemented the idea in an ALMOST satisfactory way, tested with
> Mesa 20.0.0.
> 
> Side note: Mesa 19.3 and 20.0 needs TOOLCHAIN="gcc" in Zeus,
> otherwise linker errors occur. See:
> https://gitlab.freedesktop.org/mesa/mesa/issues/2533
> 
> Mesa doesn't link to libva even with meson -Dgallium-va=true. It only
> needs the libva.pc pkgconfig file and the headers from libva to build
> its VA state tracker so the circular dependency is not that hard to solve.
> 
> I have this new recipe called libva-headers now that is a fork of the
> libva recipe and just deletes the libraries in do_install_append.
> 
> My Mesa recipe addendum has this line plus PACKAGECONFIG has "va":
> 
> PACKAGECONFIG[va] = "-Dgallium-va=true,-Dgallium-va=false,libva-headers"
> 
> Now, building libva is happy without the warning about libva.so is
> shipped by libva-headers (or libva-noglx in the previous incarnation)
> but now marked as shipped by libva because it was built later.
> 
> There is another problem now: anything that uses libva as dependency,
> like intel-vaapi-driver, implicitly pulls in mesa and then libva-headers
> so there is a clash in prepare-sysroot between libva and libva-headers
> because they still ship identical headers and pkgconfig files.
> 
> My current solution to the problem is to patch staging.bbclass to
> handle a new variable called EXCLUDE_DEPS but a line like this below
> needs to be added to all recipes that depends on libva, so it is
> quite intrusive for now.
> 
> EXCLUDE_DEPS = "libva-headers"
> 
> I am looking into storing data in "sstate", so libva-headers would add
> itself into a new class of packages that are not added to recipe-sysroot
> unless it is in DEPENDS explicitly.
> 
> My question is: is there a generic framework to do this already?

Talking to myself... YES THERE IS.
I just had to name the extra package as "libva-initial" and it works
the way I expected with no extra fuss.

> 
> This would allow me to not modify other recipes and may alsoserve other
> recipes with circular dependencies.
> 
> The end goal is to allow Chromium to be built with use_vaapi=true now that
> meta-browser has version 79 that includes the VAAPI patches. The current
> solution with setting EXCLUDE_DEPS already does this but as I said, it's
> not clean enough.
> 
> Best regards,
> Zoltán Böszörményi
> 
> 2015. 08. 09. 0:14 keltezéssel, Khem Raj írta:
>>
>>> On Jul 29, 2015, at 6:26 AM, Boszormenyi Zoltan <zboszor@pr.hu> wrote:
>>>
>>> Hi,
>>>
>>> I have a customized Mesa 10.6.3 package recipe which depends on libva
>>> to be able to build its mesa-va-driver subpackage, but libva also depends
>>> on Mesa to support GLX.
>>>
>>> Previously, I simply created a libva-noglx package that builds libva
>>> with --disable-glx that Mesa depend on and the "real" libva package can
>>> depend on Mesa.
>>>
>>> But since I also have to produce an SDK image with all the *-dev packages,
>>> creating such an initial image with bitbake fails because of a conflict
>>> in the dependency chain: libva-dev -> mesa-dev -> libva-noglx-dev.
>>>
>>
>> since libva-noglx is sort of a bootstrap package, you could empty/delete out the content
>> its puts into -dev packages and conflict will be gone. Other option is that you share
>> the content of -dev package in exclusive manner between libva and libva-noglx such
>> that they don’t package duplicate files
>>
>>> I have come up with the attached bbclass (based on Yocto 1.6) that
>>> now Mesa can inherit and that patches the RDEPENDS / RRECOMMENDS
>>> variables according to externally set variables. The comment at
>>> the beginning of the file explains how to use it.
>>>
>>> This pushes the cyclic dependency to opkg which happily breaks it and
>>> installs both libva* and *mesa* packages properly. Creating the initial
>>> image via bitbake works now.
>>>
>>> Comments?
>>>
>>> Best regards,
>>> Zoltán Böszörményi
>>>
>>> <cyclic-deps.bbclass>--
>>> _______________________________________________
>>> Openembedded-devel mailing list
>>> Openembedded-devel@lists.openembedded.org
>>> http://lists.openembedded.org/mailman/listinfo/openembedded-devel
>>
>>
> 



^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2020-02-26 11:43 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-07-29 13:26 Trying to resolve build-time cyclic dependency between packages Boszormenyi Zoltan
2015-08-08 22:14 ` Khem Raj
2020-02-26  8:49   ` Böszörményi Zoltán
2020-02-26 11:43     ` Böszörményi Zoltán

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.