All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH 0/3 v2] core/pkg-generic: store package<->files for staging and host
@ 2017-04-05 21:20 Yann E. MORIN
  2017-04-05 21:20 ` [Buildroot] [PATCH 1/3 v2] core/pkg-generic: rmove intermediate file-list files Yann E. MORIN
                   ` (3 more replies)
  0 siblings, 4 replies; 12+ messages in thread
From: Yann E. MORIN @ 2017-04-05 21:20 UTC (permalink / raw)
  To: buildroot

Hello All!

This small series does for staging and host as we do for target:
store the list of installed files and relate them to the package
that installed each.

---
Changes v1 -> v2:
  - split in three patches
  - in addition to staging, also store for host


Regards,
Yann E. MORIN.


The following changes since commit 4f50f4928ca3855d9d1b1b9699d2cb37e4ca65da

  Add out-of-source U-Boot device tree support (2017-04-05 22:29:23 +0200)


are available in the git repository at:

  git://git.buildroot.org/~ymorin/git/buildroot.git

for you to fetch changes up to 44910f2975b83c91148d08f3df11ea048eea0609

  core/pkg-generic: store file->package list for staging and host too (2017-04-05 23:15:27 +0200)


----------------------------------------------------------------
Yann E. MORIN (3):
      core/pkg-generic: rmove intermediate file-list files
      core/pk-generic: redirect only wsa when listing package's isntalled files
      core/pkg-generic: store file->package list for staging and host too

 package/pkg-generic.mk | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Buildroot] [PATCH 1/3 v2] core/pkg-generic: rmove intermediate file-list files
  2017-04-05 21:20 [Buildroot] [PATCH 0/3 v2] core/pkg-generic: store package<->files for staging and host Yann E. MORIN
@ 2017-04-05 21:20 ` Yann E. MORIN
  2017-04-06  6:01   ` Thomas Petazzoni
  2017-04-10 14:21   ` Arnout Vandecappelle
  2017-04-05 21:20 ` [Buildroot] [PATCH 2/3 v2] core/pk-generic: redirect only wsa when listing package's isntalled files Yann E. MORIN
                   ` (2 subsequent siblings)
  3 siblings, 2 replies; 12+ messages in thread
From: Yann E. MORIN @ 2017-04-05 21:20 UTC (permalink / raw)
  To: buildroot

To compute the list of files added by a package, we first store the list
of files before the install, do the install, list the files after the
install, and finally compare the two lists. The two lists are stored in
dot-files, hidden in the package's build dir.

We currently keep those two files, and only list the files installed in
target/

In followup patches, we'll also list files installed in staging/ as well
as files installed in host/.

Rather than add even more internal, hidden files in the package build
dir, we'll just re-use the same two temporary files to store the bfore
and after lists.

So, remove them after the comparison is done.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/pkg-generic.mk | 1 +
 1 file changed, 1 insertion(+)

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 31dbc54..6b3c69c 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -78,6 +78,7 @@ define step_pkg_size_end
 		while read hash file ; do \
 			echo "$(1),$${file}" >> $(BUILD_DIR)/packages-file-list.txt ; \
 		done
+	rm -f $($(PKG)_DIR)/.br_filelist_before $($(PKG)_DIR)/.br_filelist_after
 endef
 
 define step_pkg_size
-- 
2.9.3

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [Buildroot] [PATCH 2/3 v2] core/pk-generic: redirect only wsa when listing package's isntalled files
  2017-04-05 21:20 [Buildroot] [PATCH 0/3 v2] core/pkg-generic: store package<->files for staging and host Yann E. MORIN
  2017-04-05 21:20 ` [Buildroot] [PATCH 1/3 v2] core/pkg-generic: rmove intermediate file-list files Yann E. MORIN
@ 2017-04-05 21:20 ` Yann E. MORIN
  2017-04-06  6:02   ` Thomas Petazzoni
  2017-04-10 14:24   ` Arnout Vandecappelle
  2017-04-05 21:20 ` [Buildroot] [PATCH 3/3 v2] core/pkg-generic: store file->package list for staging and host too Yann E. MORIN
  2017-04-10 14:14 ` [Buildroot] [PATCH 0/3 v2] core/pkg-generic: store package<->files for staging and host Arnout Vandecappelle
  3 siblings, 2 replies; 12+ messages in thread
From: Yann E. MORIN @ 2017-04-05 21:20 UTC (permalink / raw)
  To: buildroot

There is no need to redirect again and again for each new file added to
the list; we can just redirect once and for all.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/pkg-generic.mk | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 6b3c69c..166efd6 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -76,8 +76,8 @@ define step_pkg_size_end
 		$($(PKG)_DIR)/.br_filelist_after
 	comm -13 $($(PKG)_DIR)/.br_filelist_before $($(PKG)_DIR)/.br_filelist_after | \
 		while read hash file ; do \
-			echo "$(1),$${file}" >> $(BUILD_DIR)/packages-file-list.txt ; \
-		done
+			echo "$(1),$${file}" ; \
+		done >> $(BUILD_DIR)/packages-file-list.txt
 	rm -f $($(PKG)_DIR)/.br_filelist_before $($(PKG)_DIR)/.br_filelist_after
 endef
 
-- 
2.9.3

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [Buildroot] [PATCH 3/3 v2] core/pkg-generic: store file->package list for staging and host too
  2017-04-05 21:20 [Buildroot] [PATCH 0/3 v2] core/pkg-generic: store package<->files for staging and host Yann E. MORIN
  2017-04-05 21:20 ` [Buildroot] [PATCH 1/3 v2] core/pkg-generic: rmove intermediate file-list files Yann E. MORIN
  2017-04-05 21:20 ` [Buildroot] [PATCH 2/3 v2] core/pk-generic: redirect only wsa when listing package's isntalled files Yann E. MORIN
@ 2017-04-05 21:20 ` Yann E. MORIN
  2017-04-10 14:49   ` Arnout Vandecappelle
  2017-04-10 14:14 ` [Buildroot] [PATCH 0/3 v2] core/pkg-generic: store package<->files for staging and host Arnout Vandecappelle
  3 siblings, 1 reply; 12+ messages in thread
From: Yann E. MORIN @ 2017-04-05 21:20 UTC (permalink / raw)
  To: buildroot

Currently, we store the list of files installed in target/ and associate
each of them to the package that installed it.

However, we sometime may need to know what package installed which file
in staging/, for example to debug header collision, or in host/, to
debug what package installed what host tool.

Enhance the step instrumentation to also generate the list for staging/
and host/.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 package/pkg-generic.mk | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 166efd6..725a543 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -62,7 +62,7 @@ GLOBAL_INSTRUMENTATION_HOOKS += step_time
 # files currently installed in the target. Note that the MD5 is also
 # stored, in order to identify if the files are overwritten.
 define step_pkg_size_start
-	(cd $(TARGET_DIR) ; find . -type f -print0 | xargs -0 md5sum) | sort > \
+	(cd $(2) ; find . -type f -print0 | xargs -0 md5sum) | sort > \
 		$($(PKG)_DIR)/.br_filelist_before
 endef
 
@@ -72,19 +72,25 @@ endef
 # a diff with the .br_filelist_before to compute the list of files
 # installed by this package.
 define step_pkg_size_end
-	(cd $(TARGET_DIR); find . -type f -print0 | xargs -0 md5sum) | sort > \
+	(cd $(2); find . -type f -print0 | xargs -0 md5sum) | sort > \
 		$($(PKG)_DIR)/.br_filelist_after
 	comm -13 $($(PKG)_DIR)/.br_filelist_before $($(PKG)_DIR)/.br_filelist_after | \
 		while read hash file ; do \
 			echo "$(1),$${file}" ; \
-		done >> $(BUILD_DIR)/packages-file-list.txt
+		done >> $(BUILD_DIR)/packages-file-list$(3).txt
 	rm -f $($(PKG)_DIR)/.br_filelist_before $($(PKG)_DIR)/.br_filelist_after
 endef
 
 define step_pkg_size
 	$(if $(filter install-target,$(2)),\
-		$(if $(filter start,$(1)),$(call step_pkg_size_start,$(3))) \
-		$(if $(filter end,$(1)),$(call step_pkg_size_end,$(3))))
+		$(if $(filter start,$(1)),$(call step_pkg_size_start,$(3),$(TARGET_DIR))) \
+		$(if $(filter end,$(1)),$(call step_pkg_size_end,$(3),$(TARGET_DIR))))
+	$(if $(filter install-staging,$(2)),\
+		$(if $(filter start,$(1)),$(call step_pkg_size_start,$(3),$(STAGING_DIR),-staging)) \
+		$(if $(filter end,$(1)),$(call step_pkg_size_end,$(3),$(STAGING_DIR),-staging)))
+	$(if $(filter install-host,$(2)),\
+		$(if $(filter start,$(1)),$(call step_pkg_size_start,$(3),$(HOST_DIR),-host)) \
+		$(if $(filter end,$(1)),$(call step_pkg_size_end,$(3),$(HOST_DIR),-host)))
 endef
 GLOBAL_INSTRUMENTATION_HOOKS += step_pkg_size
 
-- 
2.9.3

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [Buildroot] [PATCH 1/3 v2] core/pkg-generic: rmove intermediate file-list files
  2017-04-05 21:20 ` [Buildroot] [PATCH 1/3 v2] core/pkg-generic: rmove intermediate file-list files Yann E. MORIN
@ 2017-04-06  6:01   ` Thomas Petazzoni
  2017-04-10 14:21   ` Arnout Vandecappelle
  1 sibling, 0 replies; 12+ messages in thread
From: Thomas Petazzoni @ 2017-04-06  6:01 UTC (permalink / raw)
  To: buildroot

Hello,

On Wed,  5 Apr 2017 23:20:57 +0200, Yann E. MORIN wrote:
> To compute the list of files added by a package, we first store the list
> of files before the install, do the install, list the files after the
> install, and finally compare the two lists. The two lists are stored in
> dot-files, hidden in the package's build dir.

Typo in commit title: rmove -> remove

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Buildroot] [PATCH 2/3 v2] core/pk-generic: redirect only wsa when listing package's isntalled files
  2017-04-05 21:20 ` [Buildroot] [PATCH 2/3 v2] core/pk-generic: redirect only wsa when listing package's isntalled files Yann E. MORIN
@ 2017-04-06  6:02   ` Thomas Petazzoni
  2017-04-06 15:44     ` Yann E. MORIN
  2017-04-10 14:24   ` Arnout Vandecappelle
  1 sibling, 1 reply; 12+ messages in thread
From: Thomas Petazzoni @ 2017-04-06  6:02 UTC (permalink / raw)
  To: buildroot

Hello,

On Wed,  5 Apr 2017 23:20:58 +0200, Yann E. MORIN wrote:
> There is no need to redirect again and again for each new file added to
> the list; we can just redirect once and for all.

Typoes in the commit title:

 pk-generic -> pkg-generic

then I'm not even sure was you meant with "redirect only wsa when"

And isntalled -> installed.

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Buildroot] [PATCH 2/3 v2] core/pk-generic: redirect only wsa when listing package's isntalled files
  2017-04-06  6:02   ` Thomas Petazzoni
@ 2017-04-06 15:44     ` Yann E. MORIN
  0 siblings, 0 replies; 12+ messages in thread
From: Yann E. MORIN @ 2017-04-06 15:44 UTC (permalink / raw)
  To: buildroot

Thomas, All,

On 2017-04-06 08:02 +0200, Thomas Petazzoni spake thusly:
> On Wed,  5 Apr 2017 23:20:58 +0200, Yann E. MORIN wrote:
> > There is no need to redirect again and again for each new file added to
> > the list; we can just redirect once and for all.
> 
> Typoes in the commit title:
> 
>  pk-generic -> pkg-generic
> 
> then I'm not even sure was you meant with "redirect only wsa when"

Neither do I. Finger-fart...

Will fix and resend. Thanks! :-)

Regards,
Yann E. MORIN.

> And isntalled -> installed.
> 
> Thanks,
> 
> Thomas
> -- 
> Thomas Petazzoni, CTO, Free Electrons
> Embedded Linux and Kernel engineering
> http://free-electrons.com

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Buildroot] [PATCH 0/3 v2] core/pkg-generic: store package<->files for staging and host
  2017-04-05 21:20 [Buildroot] [PATCH 0/3 v2] core/pkg-generic: store package<->files for staging and host Yann E. MORIN
                   ` (2 preceding siblings ...)
  2017-04-05 21:20 ` [Buildroot] [PATCH 3/3 v2] core/pkg-generic: store file->package list for staging and host too Yann E. MORIN
@ 2017-04-10 14:14 ` Arnout Vandecappelle
  3 siblings, 0 replies; 12+ messages in thread
From: Arnout Vandecappelle @ 2017-04-10 14:14 UTC (permalink / raw)
  To: buildroot



On 05-04-17 23:20, Yann E. MORIN wrote:
> Hello All!
> 
> This small series does for staging and host as we do for target:
> store the list of installed files and relate them to the package
> that installed each.

 This series collides with ThomasDS'es extension with symlinks and directories
(http://patchwork.ozlabs.org/patch/724235/ and
http://patchwork.ozlabs.org/patch/724236/). Could you perhaps merge the series?


 Regards,
 Arnout

> 
> ---
> Changes v1 -> v2:
>   - split in three patches
>   - in addition to staging, also store for host
> 
> 
> Regards,
> Yann E. MORIN.
> 
> 
> The following changes since commit 4f50f4928ca3855d9d1b1b9699d2cb37e4ca65da
> 
>   Add out-of-source U-Boot device tree support (2017-04-05 22:29:23 +0200)
> 
> 
> are available in the git repository at:
> 
>   git://git.buildroot.org/~ymorin/git/buildroot.git
> 
> for you to fetch changes up to 44910f2975b83c91148d08f3df11ea048eea0609
> 
>   core/pkg-generic: store file->package list for staging and host too (2017-04-05 23:15:27 +0200)
> 
> 
> ----------------------------------------------------------------
> Yann E. MORIN (3):
>       core/pkg-generic: rmove intermediate file-list files
>       core/pk-generic: redirect only wsa when listing package's isntalled files
>       core/pkg-generic: store file->package list for staging and host too
> 
>  package/pkg-generic.mk | 19 +++++++++++++------
>  1 file changed, 13 insertions(+), 6 deletions(-)
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Buildroot] [PATCH 1/3 v2] core/pkg-generic: rmove intermediate file-list files
  2017-04-05 21:20 ` [Buildroot] [PATCH 1/3 v2] core/pkg-generic: rmove intermediate file-list files Yann E. MORIN
  2017-04-06  6:01   ` Thomas Petazzoni
@ 2017-04-10 14:21   ` Arnout Vandecappelle
  2017-04-10 14:23     ` Arnout Vandecappelle
  1 sibling, 1 reply; 12+ messages in thread
From: Arnout Vandecappelle @ 2017-04-10 14:21 UTC (permalink / raw)
  To: buildroot

 s/rmove/remove/

On 05-04-17 23:20, Yann E. MORIN wrote:
> To compute the list of files added by a package, we first store the list
> of files before the install, do the install, list the files after the
> install, and finally compare the two lists. The two lists are stored in
> dot-files, hidden in the package's build dir.
> 
> We currently keep those two files, and only list the files installed in
> target/
> 
> In followup patches, we'll also list files installed in staging/ as well
> as files installed in host/.
> 
> Rather than add even more internal, hidden files in the package build
> dir, we'll just re-use the same two temporary files to store the bfore
                                                                    ^e
> and after lists.

 Actually, I like to have as much as possible build info lingering around, so
I'd tend not to do this removal and instead rename to something with 'target' in
it. For example, I think we have no other way at the moment to detect when two
packages write the same file. IMO having internal hidden files in the package
build dir doesn't hurt anyone. I certainly do like the applied patches list!

 That said, I don't see any other reason to keep those files, and we probably
want explicit support for detecting collisions, so I'm OK with removing them
after all.

 Regards,
 Arnout

> 
> So, remove them after the comparison is done.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  package/pkg-generic.mk | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index 31dbc54..6b3c69c 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -78,6 +78,7 @@ define step_pkg_size_end
>  		while read hash file ; do \
>  			echo "$(1),$${file}" >> $(BUILD_DIR)/packages-file-list.txt ; \
>  		done
> +	rm -f $($(PKG)_DIR)/.br_filelist_before $($(PKG)_DIR)/.br_filelist_after
>  endef
>  
>  define step_pkg_size
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Buildroot] [PATCH 1/3 v2] core/pkg-generic: rmove intermediate file-list files
  2017-04-10 14:21   ` Arnout Vandecappelle
@ 2017-04-10 14:23     ` Arnout Vandecappelle
  0 siblings, 0 replies; 12+ messages in thread
From: Arnout Vandecappelle @ 2017-04-10 14:23 UTC (permalink / raw)
  To: buildroot



On 10-04-17 16:21, Arnout Vandecappelle wrote:
>  s/rmove/remove/
> 
> On 05-04-17 23:20, Yann E. MORIN wrote:
>> To compute the list of files added by a package, we first store the list
>> of files before the install, do the install, list the files after the
>> install, and finally compare the two lists. The two lists are stored in
>> dot-files, hidden in the package's build dir.
>>
>> We currently keep those two files, and only list the files installed in
>> target/
>>
>> In followup patches, we'll also list files installed in staging/ as well
>> as files installed in host/.
>>
>> Rather than add even more internal, hidden files in the package build
>> dir, we'll just re-use the same two temporary files to store the bfore
>                                                                     ^e
>> and after lists.
> 
>  Actually, I like to have as much as possible build info lingering around, so
> I'd tend not to do this removal and instead rename to something with 'target' in
> it. For example, I think we have no other way at the moment to detect when two
> packages write the same file. 

 D'oh, the file will of course simply appear twice in packages-file-list if two
packages write it... Silly me.

 Regards,
 Arnout


> IMO having internal hidden files in the package
> build dir doesn't hurt anyone. I certainly do like the applied patches list!
> 
>  That said, I don't see any other reason to keep those files, and we probably
> want explicit support for detecting collisions, so I'm OK with removing them
> after all.
[snip]

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Buildroot] [PATCH 2/3 v2] core/pk-generic: redirect only wsa when listing package's isntalled files
  2017-04-05 21:20 ` [Buildroot] [PATCH 2/3 v2] core/pk-generic: redirect only wsa when listing package's isntalled files Yann E. MORIN
  2017-04-06  6:02   ` Thomas Petazzoni
@ 2017-04-10 14:24   ` Arnout Vandecappelle
  1 sibling, 0 replies; 12+ messages in thread
From: Arnout Vandecappelle @ 2017-04-10 14:24 UTC (permalink / raw)
  To: buildroot



On 05-04-17 23:20, Yann E. MORIN wrote:
> There is no need to redirect again and again for each new file added to
> the list; we can just redirect once and for all.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

 Regards,
 Arnout

> ---
>  package/pkg-generic.mk | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index 6b3c69c..166efd6 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -76,8 +76,8 @@ define step_pkg_size_end
>  		$($(PKG)_DIR)/.br_filelist_after
>  	comm -13 $($(PKG)_DIR)/.br_filelist_before $($(PKG)_DIR)/.br_filelist_after | \
>  		while read hash file ; do \
> -			echo "$(1),$${file}" >> $(BUILD_DIR)/packages-file-list.txt ; \
> -		done
> +			echo "$(1),$${file}" ; \
> +		done >> $(BUILD_DIR)/packages-file-list.txt
>  	rm -f $($(PKG)_DIR)/.br_filelist_before $($(PKG)_DIR)/.br_filelist_after
>  endef
>  
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [Buildroot] [PATCH 3/3 v2] core/pkg-generic: store file->package list for staging and host too
  2017-04-05 21:20 ` [Buildroot] [PATCH 3/3 v2] core/pkg-generic: store file->package list for staging and host too Yann E. MORIN
@ 2017-04-10 14:49   ` Arnout Vandecappelle
  0 siblings, 0 replies; 12+ messages in thread
From: Arnout Vandecappelle @ 2017-04-10 14:49 UTC (permalink / raw)
  To: buildroot

 +1 to this feature.

On 05-04-17 23:20, Yann E. MORIN wrote:
> Currently, we store the list of files installed in target/ and associate
> each of them to the package that installed it.
> 
> However, we sometime may need to know what package installed which file
                      ^s

> in staging/, for example to debug header collision, or in host/, to
> debug what package installed what host tool.
> 
> Enhance the step instrumentation to also generate the list for staging/
> and host/.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  package/pkg-generic.mk | 16 +++++++++++-----
>  1 file changed, 11 insertions(+), 5 deletions(-)
> 
> diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
> index 166efd6..725a543 100644
> --- a/package/pkg-generic.mk
> +++ b/package/pkg-generic.mk
> @@ -62,7 +62,7 @@ GLOBAL_INSTRUMENTATION_HOOKS += step_time
>  # files currently installed in the target. Note that the MD5 is also
>  # stored, in order to identify if the files are overwritten.
>  define step_pkg_size_start
> -	(cd $(TARGET_DIR) ; find . -type f -print0 | xargs -0 md5sum) | sort > \
> +	(cd $(2) ; find . -type f -print0 | xargs -0 md5sum) | sort > \
>  		$($(PKG)_DIR)/.br_filelist_before
>  endef
>  
> @@ -72,19 +72,25 @@ endef
>  # a diff with the .br_filelist_before to compute the list of files
>  # installed by this package.
>  define step_pkg_size_end
> -	(cd $(TARGET_DIR); find . -type f -print0 | xargs -0 md5sum) | sort > \
> +	(cd $(2); find . -type f -print0 | xargs -0 md5sum) | sort > \
>  		$($(PKG)_DIR)/.br_filelist_after
>  	comm -13 $($(PKG)_DIR)/.br_filelist_before $($(PKG)_DIR)/.br_filelist_after | \
>  		while read hash file ; do \
>  			echo "$(1),$${file}" ; \
> -		done >> $(BUILD_DIR)/packages-file-list.txt
> +		done >> $(BUILD_DIR)/packages-file-list$(3).txt
>  	rm -f $($(PKG)_DIR)/.br_filelist_before $($(PKG)_DIR)/.br_filelist_after
>  endef
>  
>  define step_pkg_size
>  	$(if $(filter install-target,$(2)),\
> -		$(if $(filter start,$(1)),$(call step_pkg_size_start,$(3))) \
> -		$(if $(filter end,$(1)),$(call step_pkg_size_end,$(3))))
> +		$(if $(filter start,$(1)),$(call step_pkg_size_start,$(3),$(TARGET_DIR))) \
> +		$(if $(filter end,$(1)),$(call step_pkg_size_end,$(3),$(TARGET_DIR))))
> +	$(if $(filter install-staging,$(2)),\
> +		$(if $(filter start,$(1)),$(call step_pkg_size_start,$(3),$(STAGING_DIR),-staging)) \
> +		$(if $(filter end,$(1)),$(call step_pkg_size_end,$(3),$(STAGING_DIR),-staging)))
> +	$(if $(filter install-host,$(2)),\
> +		$(if $(filter start,$(1)),$(call step_pkg_size_start,$(3),$(HOST_DIR),-host)) \
> +		$(if $(filter end,$(1)),$(call step_pkg_size_end,$(3),$(HOST_DIR),-host)))


 How about the following alternative:

---
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 166efd630f..c27a76194c 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -62,8 +62,9 @@ GLOBAL_INSTRUMENTATION_HOOKS += step_time
 # files currently installed in the target. Note that the MD5 is also
 # stored, in order to identify if the files are overwritten.
 define step_pkg_size_start
-       (cd $(TARGET_DIR) ; find . -type f -print0 | xargs -0 md5sum) | sort > \
-               $($(PKG)_DIR)/.br_filelist_before
+       (cd $($(call UPPERCASE,$(2))_DIR); \
+               find . -type f -print0 | xargs -0 md5sum) | sort > \
+                       $($(PKG)_DIR)/.br_filelist_before
 endef

 # This hook will be called after the target installation of a
@@ -72,19 +73,19 @@ endef
 # a diff with the .br_filelist_before to compute the list of files
 # installed by this package.
 define step_pkg_size_end
-       (cd $(TARGET_DIR); find . -type f -print0 | xargs -0 md5sum) | sort > \
-               $($(PKG)_DIR)/.br_filelist_after
+       (cd $($(call UPPERCASE,$(2))_DIR); \
+               find . -type f -print0 | xargs -0 md5sum) | sort > \
+                       $($(PKG)_DIR)/.br_filelist_after
        comm -13 $($(PKG)_DIR)/.br_filelist_before
$($(PKG)_DIR)/.br_filelist_after | \
                while read hash file ; do \
                        echo "$(1),$${file}" ; \
-               done >> $(BUILD_DIR)/packages-file-list.txt
+               done >> $(BUILD_DIR)/packages-file-list-$(2).txt
        rm -f $($(PKG)_DIR)/.br_filelist_before $($(PKG)_DIR)/.br_filelist_after
 endef

 define step_pkg_size
-       $(if $(filter install-target,$(2)),\
-               $(if $(filter start,$(1)),$(call step_pkg_size_start,$(3))) \
-               $(if $(filter end,$(1)),$(call step_pkg_size_end,$(3))))
+       $(if $(filter install-%,$(2)),\
+               $(call step_pkg_size_$(1),$(3),$(subst install-,,$(2))))
 endef
 GLOBAL_INSTRUMENTATION_HOOKS += step_pkg_size
---

 Combine that with the refactoring by ThomasDS and you get something nice and
compact that is still very readable IMO. Oh, and if before/after is renamed to
start/end it becomes even better.

 Perhaps the refactoring of passing target as an argument should be a separate
patch, but I don't think so (it's kind of useless on its own and hard to
understand without the host and staging options).


 Regards,
 Arnout

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

^ permalink raw reply related	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2017-04-10 14:49 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-04-05 21:20 [Buildroot] [PATCH 0/3 v2] core/pkg-generic: store package<->files for staging and host Yann E. MORIN
2017-04-05 21:20 ` [Buildroot] [PATCH 1/3 v2] core/pkg-generic: rmove intermediate file-list files Yann E. MORIN
2017-04-06  6:01   ` Thomas Petazzoni
2017-04-10 14:21   ` Arnout Vandecappelle
2017-04-10 14:23     ` Arnout Vandecappelle
2017-04-05 21:20 ` [Buildroot] [PATCH 2/3 v2] core/pk-generic: redirect only wsa when listing package's isntalled files Yann E. MORIN
2017-04-06  6:02   ` Thomas Petazzoni
2017-04-06 15:44     ` Yann E. MORIN
2017-04-10 14:24   ` Arnout Vandecappelle
2017-04-05 21:20 ` [Buildroot] [PATCH 3/3 v2] core/pkg-generic: store file->package list for staging and host too Yann E. MORIN
2017-04-10 14:49   ` Arnout Vandecappelle
2017-04-10 14:14 ` [Buildroot] [PATCH 0/3 v2] core/pkg-generic: store package<->files for staging and host Arnout Vandecappelle

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.