Question about the behavior and intent of recrdeptask

* Question about the behavior and intent of recrdeptask
@ 2020-01-08  3:14 Masahiro Yamada
  2020-01-08  8:45 ` Richard Purdie
  0 siblings, 1 reply; 3+ messages in thread
From: Masahiro Yamada @ 2020-01-08  3:14 UTC (permalink / raw)
  To: bitbake-devel, openembedded-core

Hi.

I am trying to understand 'recrdeptask', but
the bitbake manual did not help me
fully understand it.

I hope this question/analysis will be helpful
for other new learners, too.

When I do "bitbake -g -u taskexp core-image-minimal",
I notice core-image-minimal:do_build
depends on the do_build task of a lot of other packages.
I was wondering where such dependencies come from.

I finally found out the code in
openembedded-core/meta/classes/meta.bbclass

  do_build[recrdeptask] = "do_build"

is the one.

core-image.bbclass (eventually) inherits meta.bbclass

I read the "3.10.4. Recursive Dependencies" section of bitbake manual,
but I could not understand how it works.

So, I wrote test code which consists of the following three recipes,
and figured out how the dependency graph is constructed.

------------------(foo.bb  begin)----------------
PN = 'foo'

do_foo1[nostamp] = "1"
do_foo2[nostamp] = "1"
do_build[nostamp] = "1"

python do_foo1() {
    bb.plain("foo.do_foo1");
}

python do_foo2() {
    bb.plain("foo.do_foo2");
}

python do_build() {
    bb.plain("foo.do_build");
}

addtask foo1 before do_foo2
addtask foo2 before do_build
addtask build

# inter-recipe dependency
do_foo1[depends] = "bar:do_bar1"

# recursive dependency
do_build[recrdeptask] = "do_build"
------------------(foo.bb  end)----------------

------------------(bar.bb  begin)--------------
PN = 'bar'

do_bar1[nostamp] = "1"
do_bar2[nostamp] = "1"
do_build[nostamp] = "1"

python do_bar1() {
    bb.plain("bar.do_bar1")
}

python do_bar2() {
    bb.plain("bar.do_bar2")
}

python do_build() {
    bb.plain("bar.do_build")
}

addtask bar1
addtask bar2 before do_build
addtask build

# inter-recipe dependency
do_bar2[depends] = "baz:do_baz2"
--------------------(bar.bb  end)----------------

--------------------(baz.bb  begin)--------------
PN = 'baz'

do_baz1[nostamp] = "1"
do_baz2[nostamp] = "1"
do_build[nostamp] = "1"

python do_baz1() {
    bb.plain("baz.do_baz1")
}

python do_baz2() {
    bb.plain("baz.do_baz2")
}

python do_build() {
    bb.plain("baz.do_build")
}

addtask baz1
addtask baz2
addtask build
-------------------(baz.bb  end)-------------------

The recipe-internal task orders and
the two "depends" flags construct
the following dependency graph.
(please see the figure with a fixed width font)

[Graph1]
baz:do_baz1       baz:do_baz2       baz:do_build
                      |
                      \/
bar:do_bar1       bar:do_bar2  -->  bar:do_build
   |
   \/
foo:do_foo1  -->  foo:do_foo2  -->  foo:do_build

Now, we will add the effect of:
do_build[recrdeptask] = "do_build"

As far as I understand, this works like this:

  If any arbitrary task of a recipe is reachable from
  foo:do_build, bitbake will add the do_build task
  of that recipe to the dependency graph.

bar:do_bar1 is reachable from foo:do_build,
bitbake adds a new dependency:
 bar:do_build -->  foo:do_build

Then we get the following graph:

[Graph2]
baz:do_baz1       baz:do_baz2       baz:do_build
                      |
                      \/
bar:do_bar1       bar:do_bar2  -->  bar:do_build
   |                                    |
   \/                                   \/
foo:do_foo1  -->  foo:do_foo2  -->  foo:do_build

This process continues recursively until we no longer
find a new dependency.

Now baz:do_baz2 is reachable from foo:do_build

So, bitbake adds a new dependency:
 baz:do_build  -->  foo:do_build

The final dependency graph is as follows:

[Graph3]
baz:do_baz1       baz:do_baz2           baz:do_build
                      |                          |
                      \/                         |
bar:do_bar1       bar:do_bar2  --> bar:do_build  |
   |                                    |        |
   \/                                   \/       \/
foo:do_foo1  -->  foo:do_foo2  -->    foo:do_build

Is my understanding correct?

I still do not understand why we need this flag.

[Q1] What is the intent of meta/classes/meta.bbclass ?

   do_build[recrdeptask] = "do_build"

I do not understand why do_build must have completed
before cure-image-*:do_build.

I think do_build is just no-op in most packages.

If someone does important operations in the do_build task,
that seems to be a pretty special case.
We can add explicit dependency where necessary, can't we?

[Q2] What is the background of necessity of 'recrdeptask'?

The behavior of 'recrdepends' looks so complicated to me.
If we perfectly describe the dependency for each pair of tasks
by the usual dependencies,
we can build up the correct dependency graph as a whole, can't we?

For example, assume the situation where we want the
following dependencies.

 [1] core-image-foo  depends on packagegroup-xxx
 [2] packagegroup-xxx depends on app1 and app2

The recursive dependency makes
core-image-foo directly depend on app1 and app2.

However, even without having recrdeptask,
core-image-foo eventually depends on app1 and app2,
via packagegroup-xxx.

I do not see such a feature in Make
because the dependency is recursive by nature.

So, the concept of  "recursive dependency" sounds
somewhat odd to me.

I guess I am missing important use-cases.
I'd like to know the reason why we need recrdeptask.

Thanks.

--
Best Regards
Masahiro Yamada

^ permalink raw reply	[flat|nested] 3+ messages in thread