All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] devtool: Fix source extraction for gcc shared source
@ 2021-01-19 15:53 Tomasz Dziendzielski
  2021-01-19 15:54 ` [OE-core] " Paul Barker
  2021-01-19 17:49 ` Richard Purdie
  0 siblings, 2 replies; 7+ messages in thread
From: Tomasz Dziendzielski @ 2021-01-19 15:53 UTC (permalink / raw)
  To: openembedded-core; +Cc: Tomasz Dziendzielski

If do_patch task is disabled then prepare do_configure dependencies to
fetch external sources and create symlink to ${S} in devtool workspace.

[YOCTO #13036]

Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
---
 .../gcc/gcc-shared-source.inc                 |  1 +
 scripts/lib/devtool/standard.py               | 30 ++++++++++++++++---
 2 files changed, 27 insertions(+), 4 deletions(-)

diff --git a/meta/recipes-devtools/gcc/gcc-shared-source.inc b/meta/recipes-devtools/gcc/gcc-shared-source.inc
index aac4b49313..9ef80f2074 100644
--- a/meta/recipes-devtools/gcc/gcc-shared-source.inc
+++ b/meta/recipes-devtools/gcc/gcc-shared-source.inc
@@ -2,6 +2,7 @@ do_fetch() {
 	:
 }
 do_fetch[noexec] = "1"
+do_patch[noexec] = "1"
 deltask do_unpack
 deltask do_patch
 
diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 261d642d4a..8b80fb8bfa 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -589,6 +589,16 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works
             else:
                 task = 'do_patch'
 
+                if 'noexec' in (d.getVarFlags(task, False) or []):
+                    logger.info('The %s recipe has %s disabled. Running only '
+                                       'do_configure task dependencies' % (pn, task))
+
+                    if 'depends' in d.getVarFlags('do_configure', False):
+                        pn = d.getVarFlags('do_configure', False)['depends']
+                        pn = pn.replace('${PV}', d.getVar('PV'))
+                        pn = pn.replace('${COMPILERDEP}', d.getVar('COMPILERDEP'))
+                        task = None
+
             # Run the fetch + unpack tasks
             res = tinfoil.build_targets(pn,
                                         task,
@@ -600,6 +610,17 @@ def _extract_source(srctree, keep_temp, devbranch, sync, config, basepath, works
         if not res:
             raise DevtoolError('Extracting source for %s failed' % pn)
 
+        if 'noexec' in (d.getVarFlags('do_patch', False) or []):
+            workshareddir = d.getVar('S')
+            if os.path.islink(srctree):
+                os.unlink(srctree)
+
+            os.symlink(workshareddir, srctree)
+
+            # The initial_rev file is created in devtool_post_unpack function that will not be executed if
+            # do_unpack/do_patch tasks are disabled so we have to directly say that source extraction was successful
+            return True, True
+
         try:
             with open(os.path.join(tempdir, 'initial_rev'), 'r') as f:
                 initial_rev = f.read()
@@ -847,10 +868,11 @@ def modify(args, config, basepath, workspace):
             if not initial_rev:
                 return 1
             logger.info('Source tree extracted to %s' % srctree)
-            # Get list of commits since this revision
-            (stdout, _) = bb.process.run('git rev-list --reverse %s..HEAD' % initial_rev, cwd=srctree)
-            commits = stdout.split()
-            check_commits = True
+            if os.path.exists(os.path.join(srctree, '.git')):
+                # Get list of commits since this revision
+                (stdout, _) = bb.process.run('git rev-list --reverse %s..HEAD' % initial_rev, cwd=srctree)
+                commits = stdout.split()
+                check_commits = True
         else:
             if os.path.exists(os.path.join(srctree, '.git')):
                 # Check if it's a tree previously extracted by us. This is done
-- 
2.29.2


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH] devtool: Fix source extraction for gcc shared source
  2021-01-19 15:53 [PATCH] devtool: Fix source extraction for gcc shared source Tomasz Dziendzielski
@ 2021-01-19 15:54 ` Paul Barker
  2021-01-19 16:44   ` Tomasz Dziendzielski
  2021-01-19 17:49 ` Richard Purdie
  1 sibling, 1 reply; 7+ messages in thread
From: Paul Barker @ 2021-01-19 15:54 UTC (permalink / raw)
  To: Tomasz Dziendzielski; +Cc: openembedded-core

On Tue, 19 Jan 2021 at 15:51, Tomasz Dziendzielski
<tomasz.dziendzielski@gmail.com> wrote:
>
> If do_patch task is disabled then prepare do_configure dependencies to
> fetch external sources and create symlink to ${S} in devtool workspace.
>
> [YOCTO #13036]
>
> Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
> ---
>  .../gcc/gcc-shared-source.inc                 |  1 +
>  scripts/lib/devtool/standard.py               | 30 ++++++++++++++++---
>  2 files changed, 27 insertions(+), 4 deletions(-)
>
> diff --git a/meta/recipes-devtools/gcc/gcc-shared-source.inc b/meta/recipes-devtools/gcc/gcc-shared-source.inc
> index aac4b49313..9ef80f2074 100644
> --- a/meta/recipes-devtools/gcc/gcc-shared-source.inc
> +++ b/meta/recipes-devtools/gcc/gcc-shared-source.inc
> @@ -2,6 +2,7 @@ do_fetch() {
>         :
>  }
>  do_fetch[noexec] = "1"
> +do_patch[noexec] = "1"

Have you tested whether the archiver class is still working after this
change? If not it'd be helpful to run `oe-selftest -r archiver` to
check.

-- 
Paul Barker
Konsulko Group

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH] devtool: Fix source extraction for gcc shared source
  2021-01-19 15:54 ` [OE-core] " Paul Barker
@ 2021-01-19 16:44   ` Tomasz Dziendzielski
  0 siblings, 0 replies; 7+ messages in thread
From: Tomasz Dziendzielski @ 2021-01-19 16:44 UTC (permalink / raw)
  To: Paul Barker; +Cc: openembedded-core

[-- Attachment #1: Type: text/plain, Size: 356 bytes --]

>Have you tested whether the archiver class is still working after this
>change? If not it'd be helpful to run `oe-selftest -r archiver` to
>check.

Hi Paul,
I have just run that script and it passed with no errors:
oe-selftest - OK - All required tests passed (successes=13, skipped=0,
failures=0, errors=0)

Best regards,
Tomasz Dziendzielski

[-- Attachment #2: Type: text/html, Size: 561 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH] devtool: Fix source extraction for gcc shared source
  2021-01-19 15:53 [PATCH] devtool: Fix source extraction for gcc shared source Tomasz Dziendzielski
  2021-01-19 15:54 ` [OE-core] " Paul Barker
@ 2021-01-19 17:49 ` Richard Purdie
  2021-01-19 18:05   ` Tomasz Dziendzielski
  1 sibling, 1 reply; 7+ messages in thread
From: Richard Purdie @ 2021-01-19 17:49 UTC (permalink / raw)
  To: Tomasz Dziendzielski, openembedded-core

On Tue, 2021-01-19 at 16:53 +0100, Tomasz Dziendzielski wrote:
> If do_patch task is disabled then prepare do_configure dependencies to
> fetch external sources and create symlink to ${S} in devtool workspace.
> 
> [YOCTO #13036]
> 
> Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
> ---
>  .../gcc/gcc-shared-source.inc                 |  1 +
>  scripts/lib/devtool/standard.py               | 30 ++++++++++++++++---
>  2 files changed, 27 insertions(+), 4 deletions(-)
> 
> diff --git a/meta/recipes-devtools/gcc/gcc-shared-source.inc b/meta/recipes-devtools/gcc/gcc-shared-source.inc
> index aac4b49313..9ef80f2074 100644
> --- a/meta/recipes-devtools/gcc/gcc-shared-source.inc
> +++ b/meta/recipes-devtools/gcc/gcc-shared-source.inc
> @@ -2,6 +2,7 @@ do_fetch() {
>  	:
>  }
>  do_fetch[noexec] = "1"
> +do_patch[noexec] = "1"
>  deltask do_unpack
>  deltask do_patch

This is in itself a little odd. The "deltask do_patch" line deletes the
task, so there is no task to set as noexec?

Cheers,

Richard



^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH] devtool: Fix source extraction for gcc shared source
  2021-01-19 17:49 ` Richard Purdie
@ 2021-01-19 18:05   ` Tomasz Dziendzielski
  2021-01-19 18:11     ` Richard Purdie
  0 siblings, 1 reply; 7+ messages in thread
From: Tomasz Dziendzielski @ 2021-01-19 18:05 UTC (permalink / raw)
  To: Richard Purdie; +Cc: openembedded-core

[-- Attachment #1: Type: text/plain, Size: 944 bytes --]

>> diff --git a/meta/recipes-devtools/gcc/gcc-shared-source.inc
b/meta/recipes-devtools/gcc/gcc-shared-source.inc
>> index aac4b49313..9ef80f2074 100644
>> --- a/meta/recipes-devtools/gcc/gcc-shared-source.inc
>> +++ b/meta/recipes-devtools/gcc/gcc-shared-source.inc
>> @@ -2,6 +2,7 @@ do_fetch() {
>>       :
>>  }
>>  do_fetch[noexec] = "1"
>> +do_patch[noexec] = "1"
>>  deltask do_unpack
>>  deltask do_patch

>This is in itself a little odd. The "deltask do_patch" line deletes the
>task, so there is no task to set as noexec?

This was added because from the devtool script I couldn't find another way
to determine if the task is run or not. If I add "do_patch[noexec]" (which
logically doesn't change anything) even if task is deleted, I can check "if
'noexec' in d.getVarFlags(task, False)" and tell devtool to execute
do_configure task dependencies (to prepare gcc sources).

Best regards,
Tomasz Dziendzielski

[-- Attachment #2: Type: text/html, Size: 1267 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH] devtool: Fix source extraction for gcc shared source
  2021-01-19 18:05   ` Tomasz Dziendzielski
@ 2021-01-19 18:11     ` Richard Purdie
  2021-01-19 18:57       ` Tomasz Dziendzielski
  0 siblings, 1 reply; 7+ messages in thread
From: Richard Purdie @ 2021-01-19 18:11 UTC (permalink / raw)
  To: Tomasz Dziendzielski; +Cc: openembedded-core

On Tue, 2021-01-19 at 19:05 +0100, Tomasz Dziendzielski wrote:
> > > diff --git a/meta/recipes-devtools/gcc/gcc-shared-source.inc
> > > b/meta/recipes-devtools/gcc/gcc-shared-source.inc
> > > index aac4b49313..9ef80f2074 100644
> > > --- a/meta/recipes-devtools/gcc/gcc-shared-source.inc
> > > +++ b/meta/recipes-devtools/gcc/gcc-shared-source.inc
> > > @@ -2,6 +2,7 @@ do_fetch() {
> > >       :
> > >  }
> > >  do_fetch[noexec] = "1"
> > > +do_patch[noexec] = "1"
> > >  deltask do_unpack
> > >  deltask do_patch
> 
> > This is in itself a little odd. The "deltask do_patch" line deletes
> > the
> > task, so there is no task to set as noexec?
> 
> This was added because from the devtool script I couldn't find
> another way to determine if the task is run or not. If I add
> "do_patch[noexec]" (which logically doesn't change anything) even if
> task is deleted, I can check "if 'noexec' in d.getVarFlags(task,
> False)" and tell devtool to execute do_configure task dependencies
> (to prepare gcc sources).

Try checking if the "task" flag is set for "do_patch"?

Cheers,

Richard



^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] [PATCH] devtool: Fix source extraction for gcc shared source
  2021-01-19 18:11     ` Richard Purdie
@ 2021-01-19 18:57       ` Tomasz Dziendzielski
  0 siblings, 0 replies; 7+ messages in thread
From: Tomasz Dziendzielski @ 2021-01-19 18:57 UTC (permalink / raw)
  To: Richard Purdie; +Cc: openembedded-core

[-- Attachment #1: Type: text/plain, Size: 1077 bytes --]

>Try checking if the "task" flag is set for "do_patch"
Good idea, thanks! It's working correctly. I can see both 'noexec' and
'task' flags can be set separately, so to make it more general (and make it
work for similar recipes without sources) I will check both 'noexec' and
'task' flags.
I will submit another patchset with suggested change  (see below) and with
another condition check to not affect kernel build.:

diff --git a/scripts/lib/devtool/standard.py
b/scripts/lib/devtool/standard.py
index 8b80fb8bfa..6f4cbbe219 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py

-                if 'noexec' in (d.getVarFlags(task, False) or []):
+                if 'noexec' in (d.getVarFlags(task, False) or []) or
'task' not in (d.getVarFlags(task, False) or []):
...
-        if 'noexec' in (d.getVarFlags('do_patch', False) or []):
+        if not is_kernel_yocto and ('noexec' in (d.getVarFlags('do_patch',
False) or []) or 'task' not in (d.getVarFlags('do_patch', False) or []):

Best regards,
Tomasz Dziendzielski

[-- Attachment #2: Type: text/html, Size: 1371 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2021-01-19 18:54 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-19 15:53 [PATCH] devtool: Fix source extraction for gcc shared source Tomasz Dziendzielski
2021-01-19 15:54 ` [OE-core] " Paul Barker
2021-01-19 16:44   ` Tomasz Dziendzielski
2021-01-19 17:49 ` Richard Purdie
2021-01-19 18:05   ` Tomasz Dziendzielski
2021-01-19 18:11     ` Richard Purdie
2021-01-19 18:57       ` Tomasz Dziendzielski

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.