All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH] system: add optional rsync with preserved permissions
@ 2021-10-04  6:41 José Pekkarinen
  2021-10-04  7:00 ` Thomas Petazzoni
  0 siblings, 1 reply; 6+ messages in thread
From: José Pekkarinen @ 2021-10-04  6:41 UTC (permalink / raw)
  To: buildroot; +Cc: José Pekkarinen

The following patch adds a config flag to let the user
rely in rsync to pressume actual file permissions and
let it populate destination files with closer results
to what the original overlay have.

Signed-off-by: José Pekkarinen <jose.pekkarinen@unikie.com>
---
 Makefile         | 6 ++++++
 system/Config.in | 7 +++++++
 system/system.mk | 8 ++++++++
 3 files changed, 21 insertions(+)

diff --git a/Makefile b/Makefile
index 259706719a..3746f276ec 100644
--- a/Makefile
+++ b/Makefile
@@ -814,9 +814,15 @@ ifeq ($(BR2_ROOTFS_MERGED_USR),y)
 
 endif # merged /usr
 
+ifeq ($(BR2_ROOTFS_OVERLAY_PRESERVED_PERMISSION),y))
+	$(foreach d, $(call qstrip,$(BR2_ROOTFS_OVERLAY)), \
+		@$(call MESSAGE,"Copying overlay $(d)")$(sep) \
+		$(Q)$(call SYSTEM_OVERLAY_RSYNC,$(d),$(TARGET_DIR))$(sep))
+else
 	$(foreach d, $(call qstrip,$(BR2_ROOTFS_OVERLAY)), \
 		@$(call MESSAGE,"Copying overlay $(d)")$(sep) \
 		$(Q)$(call SYSTEM_RSYNC,$(d),$(TARGET_DIR))$(sep))
+endif
 
 	$(Q)$(if $(TARGET_DIR_FILES_LISTS), \
 		cat $(TARGET_DIR_FILES_LISTS)) > $(BUILD_DIR)/packages-file-list.txt
diff --git a/system/Config.in b/system/Config.in
index b3abeddd68..a9b27f88b5 100644
--- a/system/Config.in
+++ b/system/Config.in
@@ -533,6 +533,13 @@ config BR2_ROOTFS_OVERLAY
 	  They are copied as-is into the rootfs, excluding files ending
 	  with ~ and .git, .svn and .hg directories.
 
+config BR2_ROOTFS_OVERLAY_PRESERVED_PERMISSION
+	string "Preserve permissions of overlay directories"
+	depends on BR2_ROOTFS_OVERLAY
+	default "n"
+	help
+	  Preserve file permissions of specified overlay.
+
 config BR2_ROOTFS_POST_BUILD_SCRIPT
 	string "Custom scripts to run before creating filesystem images"
 	default ""
diff --git a/system/system.mk b/system/system.mk
index 8fe2c138b0..61b6e1c34e 100644
--- a/system/system.mk
+++ b/system/system.mk
@@ -55,6 +55,14 @@ define SYSTEM_RSYNC
 		$(1)/ $(2)/
 endef
 
+# This function rsyncs the overlay directory in $(1) to the destination
+# in $(2), which should be either $(TARTGET_DIR) or $(STAGING_DIR)
+define SYSTEM_OVERLAY_RSYNC
+	rsync -a --ignore-times $(RSYNC_VCS_EXCLUSIONS) \
+		--exclude .empty --exclude '*~' \
+		$(1)/ $(2)/
+endef
+
 # Make a symlink lib32->lib or lib64->lib as appropriate.
 # MIPS64/n32 requires lib32 even though it's a 64-bit arch. However, since gcc
 # 5.1.0 internal compiler paths in sysroot are relative to lib64, so we must
-- 
2.25.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [Buildroot] [PATCH] system: add optional rsync with preserved permissions
  2021-10-04  6:41 [Buildroot] [PATCH] system: add optional rsync with preserved permissions José Pekkarinen
@ 2021-10-04  7:00 ` Thomas Petazzoni
  2021-10-04  7:08   ` José Pekkarinen
  2021-10-04 15:10   ` Yann E. MORIN
  0 siblings, 2 replies; 6+ messages in thread
From: Thomas Petazzoni @ 2021-10-04  7:00 UTC (permalink / raw)
  To: José Pekkarinen; +Cc: Yann E. MORIN, buildroot

Hello,

On Mon,  4 Oct 2021 09:41:43 +0300
José Pekkarinen <jose.pekkarinen@unikie.com> wrote:

> +config BR2_ROOTFS_OVERLAY_PRESERVED_PERMISSION
> +	string "Preserve permissions of overlay directories"
> +	depends on BR2_ROOTFS_OVERLAY
> +	default "n"
> +	help
> +	  Preserve file permissions of specified overlay.

I'm afraid we probably won't want an option like this.

The question is why in the existing SYSTEM_RSYNC we don't preserve
permissions? I can imagine because sometimes they can be wrong in the
original overlay, for example with version control systems that put all
files read-only.

Yann, Arnout: do you remember why SYSTEM_RSYNC has --chmod=u=rwX,go=rX ?

José: have you considered using a permission table to fixup the
permissions ?

Best regards,

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Buildroot] [PATCH] system: add optional rsync with preserved permissions
  2021-10-04  7:00 ` Thomas Petazzoni
@ 2021-10-04  7:08   ` José Pekkarinen
  2021-10-04  7:30     ` Thomas Petazzoni
  2021-10-04 15:15     ` Yann E. MORIN
  2021-10-04 15:10   ` Yann E. MORIN
  1 sibling, 2 replies; 6+ messages in thread
From: José Pekkarinen @ 2021-10-04  7:08 UTC (permalink / raw)
  To: Thomas Petazzoni; +Cc: Yann E. MORIN, buildroot


[-- Attachment #1.1: Type: text/plain, Size: 1240 bytes --]

On Mon, Oct 4, 2021 at 10:00 AM Thomas Petazzoni <
thomas.petazzoni@bootlin.com> wrote:

> Hello,
>
> On Mon,  4 Oct 2021 09:41:43 +0300
> José Pekkarinen <jose.pekkarinen@unikie.com> wrote:
>
> > +config BR2_ROOTFS_OVERLAY_PRESERVED_PERMISSION
> > +     string "Preserve permissions of overlay directories"
> > +     depends on BR2_ROOTFS_OVERLAY
> > +     default "n"
> > +     help
> > +       Preserve file permissions of specified overlay.
>
> I'm afraid we probably won't want an option like this.
>
> The question is why in the existing SYSTEM_RSYNC we don't preserve
> permissions? I can imagine because sometimes they can be wrong in the
> original overlay, for example with version control systems that put all
> files read-only.
>
> Yann, Arnout: do you remember why SYSTEM_RSYNC has --chmod=u=rwX,go=rX ?
>
> José: have you considered using a permission table to fixup the
> permissions ?
>

Never heard of it, but after a quick check it looks like

it is a better answer to my problem, since I found that, for
example, if I want to provide, an authorized_keys file owned
by root, with 600 permissions, rsync copies the file with 644,
which is closer, but not perfect.

Thanks!


José.

[-- Attachment #1.2: Type: text/html, Size: 2225 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Buildroot] [PATCH] system: add optional rsync with preserved permissions
  2021-10-04  7:08   ` José Pekkarinen
@ 2021-10-04  7:30     ` Thomas Petazzoni
  2021-10-04 15:15     ` Yann E. MORIN
  1 sibling, 0 replies; 6+ messages in thread
From: Thomas Petazzoni @ 2021-10-04  7:30 UTC (permalink / raw)
  To: José Pekkarinen; +Cc: Yann E. MORIN, buildroot

On Mon, 4 Oct 2021 10:08:09 +0300
José Pekkarinen <jose.pekkarinen@unikie.com> wrote:

> Never heard of it, but after a quick check it looks like
> it is a better answer to my problem, since I found that, for
> example, if I want to provide, an authorized_keys file owned
> by root, with 600 permissions, rsync copies the file with 644,
> which is closer, but not perfect.

Yes, with a permission table, you can adjust not just the permissions,
but also the ownership of files.

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Buildroot] [PATCH] system: add optional rsync with preserved permissions
  2021-10-04  7:00 ` Thomas Petazzoni
  2021-10-04  7:08   ` José Pekkarinen
@ 2021-10-04 15:10   ` Yann E. MORIN
  1 sibling, 0 replies; 6+ messages in thread
From: Yann E. MORIN @ 2021-10-04 15:10 UTC (permalink / raw)
  To: Thomas Petazzoni; +Cc: José Pekkarinen, buildroot

Thomas, José, All,

On 2021-10-04 09:00 +0200, Thomas Petazzoni spake thusly:
> On Mon,  4 Oct 2021 09:41:43 +0300
> José Pekkarinen <jose.pekkarinen@unikie.com> wrote:
> 
> > +config BR2_ROOTFS_OVERLAY_PRESERVED_PERMISSION
> > +	string "Preserve permissions of overlay directories"
> > +	depends on BR2_ROOTFS_OVERLAY
> > +	default "n"
> > +	help
> > +	  Preserve file permissions of specified overlay.
> 
> I'm afraid we probably won't want an option like this.

Agreed, we already have two mechanisms for that, see below.

> The question is why in the existing SYSTEM_RSYNC we don't preserve
> permissions? I can imagine because sometimes they can be wrong in the
> original overlay, for example with version control systems that put all
> files read-only.
> 
> Yann, Arnout: do you remember why SYSTEM_RSYNC has --chmod=u=rwX,go=rX ?

The reason is that indeed, we have no way to validate the access modes
and ownership for those files. As the build will (most probably) be made
as a non-root user, there is no way we can set arbitrary modes or owners
to those files when we copy them, even if the source tree had proper
owenrship and modes etc...

So we can only enforce a known mode, that is reproducible, and let the
user provide a permission table (or even a fakeroot-script, although I
would personally highly favour a permission table, which could be
generated in a post-build script if needed).

This applies to file ownership and modes (think set-uid root), but also
to extended attributes (think SELinux security context set as extended
attributes). All of that can be set with a permission table.

> José: have you considered using a permission table to fixup the
> permissions ?

This is *the* mechanism to use to set arbitrary ownership, modes, and
extended attributes to arbitrary files.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [Buildroot] [PATCH] system: add optional rsync with preserved permissions
  2021-10-04  7:08   ` José Pekkarinen
  2021-10-04  7:30     ` Thomas Petazzoni
@ 2021-10-04 15:15     ` Yann E. MORIN
  1 sibling, 0 replies; 6+ messages in thread
From: Yann E. MORIN @ 2021-10-04 15:15 UTC (permalink / raw)
  To: José Pekkarinen; +Cc: Thomas Petazzoni, buildroot

José, All,

On 2021-10-04 10:08 +0300, José Pekkarinen spake thusly:
> On Mon, Oct 4, 2021 at 10:00 AM Thomas Petazzoni < [1]thomas.petazzoni@bootlin.com> wrote:
>   On Mon,  4 Oct 2021 09:41:43 +0300
>   José Pekkarinen < [2]jose.pekkarinen@unikie.com> wrote:
> 
>   > +config BR2_ROOTFS_OVERLAY_PRESERVED_PERMISSION
>   > +     string "Preserve permissions of overlay directories"
>   > +     depends on BR2_ROOTFS_OVERLAY
>   > +     default "n"
>   > +     help
>   > +       Preserve file permissions of specified overlay.
>   José: have you considered using a permission table to fixup the
>   permissions ?
> 
> Never heard of it,

It's irght there in the manual:

    9.5.1. Setting file permissions and ownership and adding custom devices nodes
    https://buildroot.org/downloads/manual/manual.html#rootfs-custom

    Chapter 25. Makedev syntax documentation
    https://buildroot.org/downloads/manual/manual.html#makedev-syntax

;-)

> but after a quick check it looks like
> it is a better answer to my problem, since I found that, for
> example, if I want to provide, an authorized_keys file owned
> by root, with 600 permissions, rsync copies the file with 644,
> which is closer, but not perfect.

Given your feedback, and Thomas' and my comments, I've then marked your
patch as rejected in Patchwork.

Thanks!

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-10-04 15:15 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-04  6:41 [Buildroot] [PATCH] system: add optional rsync with preserved permissions José Pekkarinen
2021-10-04  7:00 ` Thomas Petazzoni
2021-10-04  7:08   ` José Pekkarinen
2021-10-04  7:30     ` Thomas Petazzoni
2021-10-04 15:15     ` Yann E. MORIN
2021-10-04 15:10   ` Yann E. MORIN

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.