[Buildroot] [PATCH] [PATCH][RFC] new target: live filesystem

[Buildroot] [PATCH] [PATCH][RFC] new target: live filesystem

[Jérémy Rosen]

add a new target to deploy a live filesystem to be used by NFS

---

Hello everybody

this patch is not ready for commit, it's a proposal to help me solve the very common use case of
a developement FS deployed on NFS. I am not very happy of how it's done at this point but I would
like to discuss it to see if it has a chance to get merged... What I don't like (and would gladly
have ideas on how to solve this)
* no dependency on host-sudo, i'm not sure how to add that
* using sudo to break out of fakeroot is not clean
* I am abusing the fs building framework, but I don't know of a cleaner way
* filesystem can only be under output/images at this point

so this patch is more to start a discussion than anything, i'll gladly take any comments or plain
'ol rejection if the project doesn't want it... but if there is some interest i'll work it into
something mainline worthy

Regards

J?r?my Rosen
---
 fs/Config.in      |    1 +
 fs/live/Config.in |   16 ++++++++++++++++
 fs/live/live.mk   |   23 +++++++++++++++++++++++
 3 files changed, 40 insertions(+)
 create mode 100644 fs/live/Config.in
 create mode 100644 fs/live/live.mk

diff --git a/fs/Config.in b/fs/Config.in
index 94154ea..de2377e 100644
--- a/fs/Config.in
+++ b/fs/Config.in
@@ -11,5 +11,6 @@ source "fs/cpio/Config.in"
 source "fs/iso9660/Config.in"
 source "fs/initramfs/Config.in"
 source "fs/romfs/Config.in"
+source "fs/live/Config.in"
 
 endmenu
diff --git a/fs/live/Config.in b/fs/live/Config.in
new file mode 100644
index 0000000..bf24754
--- /dev/null
+++ b/fs/live/Config.in
@@ -0,0 +1,16 @@
+config BR2_TARGET_ROOTFS_LIVE
+	bool "live root filesystem"
+	help
+	  uses sudo to create a live image of the filesystem
+	  this is mainly usefull if you want to use your filesystem
+	  over NFS
+config BR2_TARGET_ROOTFS_LIVE_DEST
+	string "directory to put the live image"
+	depends on BR2_TARGET_ROOTFS_LIVE
+	default "live"
+	help
+	  The directory where the image should be stored.
+	  this directory will be emptied and recreated, it is given
+	  relative to the buildroot output/images directory
+
+
diff --git a/fs/live/live.mk b/fs/live/live.mk
new file mode 100644
index 0000000..73bcd3e
--- /dev/null
+++ b/fs/live/live.mk
@@ -0,0 +1,23 @@
+#############################################################
+#
+# Build the live root filesystem directory
+#
+#############################################################
+
+LIVE_OPTS :=
+
+
+ROOTFS_LIVE_DEPENDENCIES = #host-sudo
+
+define ROOTFS_LIVE_CMD
+ sudo cp -r $(TARGET_DIR)/* $(BINARIES_DIR)/$(BR2_TARGET_ROOTFS_LIVE_DEST)/
+endef
+
+define ROOTFS_LIVE_INIT
+	sudo rm -rf $(BINARIES_DIR)/$(BR2_TARGET_ROOTFS_LIVE_DEST)
+	sudo mkdir $(BINARIES_DIR)/$(BR2_TARGET_ROOTFS_LIVE_DEST)
+endef
+
+ROOTFS_LIVE_PRE_GEN_HOOKS += ROOTFS_LIVE_INIT
+
+$(eval $(call ROOTFS_TARGET,live))
-- 
1.7.10.4

[Buildroot] [PATCH][RFC] new target: live filesystem

[Arnout Vandecappelle]

On 27/11/12 09:12, J?r?my Rosen wrote:
> add a new target to deploy a live filesystem to be used by NFS
>
> ---
>
> Hello everybody
>
> this patch is not ready for commit, it's a proposal to help me solve the very common use case of
> a developement FS deployed on NFS. I am not very happy of how it's done at this point but I would
> like to discuss it to see if it has a chance to get merged... What I don't like (and would gladly
> have ideas on how to solve this)
> * no dependency on host-sudo, i'm not sure how to add that

  That's impossible. The sudo executable must be setuid root, and to make
it setuid root you have to be root, so sudo must already exist...

  You could instead add a check in live.mk if sudo actually works.  But since
sudo may ask for a password this can get tricky - you should e.g. probably check
if stdin is a terminal as well.

  That said, I don't know if Peter will accept running sudo within buildroot.

> * using sudo to break out of fakeroot is not clean

  I don't see much of a problem there.

> * I am abusing the fs building framework, but I don't know of a cleaner way

  You're not abusing it.

> * filesystem can only be under output/images at this point

  That's not good but easy to solve.

>
> so this patch is more to start a discussion than anything, i'll gladly take any comments or plain
> 'ol rejection if the project doesn't want it... but if there is some interest i'll work it into
> something mainline worthy
>
> Regards
>
> J?r?my Rosen
> ---
>   fs/Config.in      |    1 +
>   fs/live/Config.in |   16 ++++++++++++++++
>   fs/live/live.mk   |   23 +++++++++++++++++++++++
>   3 files changed, 40 insertions(+)
>   create mode 100644 fs/live/Config.in
>   create mode 100644 fs/live/live.mk
>
> diff --git a/fs/Config.in b/fs/Config.in
> index 94154ea..de2377e 100644
> --- a/fs/Config.in
> +++ b/fs/Config.in
> @@ -11,5 +11,6 @@ source "fs/cpio/Config.in"
>   source "fs/iso9660/Config.in"
>   source "fs/initramfs/Config.in"
>   source "fs/romfs/Config.in"
> +source "fs/live/Config.in"

  Sort alphabetically please.

>
>   endmenu
> diff --git a/fs/live/Config.in b/fs/live/Config.in
> new file mode 100644
> index 0000000..bf24754
> --- /dev/null
> +++ b/fs/live/Config.in
> @@ -0,0 +1,16 @@
> +config BR2_TARGET_ROOTFS_LIVE
> +	bool "live root filesystem"
> +	help
> +	  uses sudo to create a live image of the filesystem
> +	  this is mainly usefull if you want to use your filesystem

  useful

> +	  over NFS

  or as a chroot

> +config BR2_TARGET_ROOTFS_LIVE_DEST
> +	string "directory to put the live image"

  "live image location"

> +	depends on BR2_TARGET_ROOTFS_LIVE
> +	default "live"

  Use an empty default, and give an error in live.mk if it's
empty.

> +	help
> +	  The directory where the image should be stored.
> +	  this directory will be emptied and recreated, it is given
> +	  relative to the buildroot output/images directory

  Make it an absolute path instead.

> +
> diff --git a/fs/live/live.mk b/fs/live/live.mk
> new file mode 100644
> index 0000000..73bcd3e
> --- /dev/null
> +++ b/fs/live/live.mk
> @@ -0,0 +1,23 @@
> +#############################################################
> +#
> +# Build the live root filesystem directory
> +#
> +#############################################################
> +
> +LIVE_OPTS :=

  This is unneeded.

> +
> +
> +ROOTFS_LIVE_DEPENDENCIES = #host-sudo

  This as well.

> +
> +define ROOTFS_LIVE_CMD
> + sudo cp -r $(TARGET_DIR)/* $(BINARIES_DIR)/$(BR2_TARGET_ROOTFS_LIVE_DEST)/

  That won't work, because device nodes, symlinks etc. are not
copied correctly.  Use cp -a.  Or even better, rsync -a --delete-during
(then you don't need to remove it first).

  For the target, just use $(BR2_TARGET_ROOTFS_LIVE_DEST).


  Regards,
  Arnout


> +endef
> +
> +define ROOTFS_LIVE_INIT
> +	sudo rm -rf $(BINARIES_DIR)/$(BR2_TARGET_ROOTFS_LIVE_DEST)
> +	sudo mkdir $(BINARIES_DIR)/$(BR2_TARGET_ROOTFS_LIVE_DEST)
> +endef
> +
> +ROOTFS_LIVE_PRE_GEN_HOOKS += ROOTFS_LIVE_INIT
> +
> +$(eval $(call ROOTFS_TARGET,live))

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH] [PATCH][RFC] new target: live filesystem

[Steve Calfee]

I don't understand why you need this. It is a one line "sudo tar -xf
..." to unpack the tarball into your nfs directory. Either type this
line or add a one line script to your ~/bin directory to do it for
each of your buildroot trees/environments. Don't add clutter to the
buildroot makefile.

Regards, Steve

On Tue, Nov 27, 2012 at 12:12 AM, J?r?my Rosen <jeremy.rosen@openwide.fr> wrote:
> add a new target to deploy a live filesystem to be used by NFS
>
> ---
>
> Hello everybody
>
> this patch is not ready for commit, it's a proposal to help me solve the very common use case of
> a developement FS deployed on NFS. I am not very happy of how it's done at this point but I would
> like to discuss it to see if it has a chance to get merged... What I don't like (and would gladly
> have ideas on how to solve this)
> * no dependency on host-sudo, i'm not sure how to add that
> * using sudo to break out of fakeroot is not clean
> * I am abusing the fs building framework, but I don't know of a cleaner way
> * filesystem can only be under output/images at this point
>
> so this patch is more to start a discussion than anything, i'll gladly take any comments or plain
> 'ol rejection if the project doesn't want it... but if there is some interest i'll work it into
> something mainline worthy
>
> Regards
>
> J?r?my Rosen
> ---
>  fs/Config.in      |    1 +
>  fs/live/Config.in |   16 ++++++++++++++++
>  fs/live/live.mk   |   23 +++++++++++++++++++++++
>  3 files changed, 40 insertions(+)
>  create mode 100644 fs/live/Config.in
>  create mode 100644 fs/live/live.mk
>
> diff --git a/fs/Config.in b/fs/Config.in
> index 94154ea..de2377e 100644
> --- a/fs/Config.in
> +++ b/fs/Config.in
> @@ -11,5 +11,6 @@ source "fs/cpio/Config.in"
>  source "fs/iso9660/Config.in"
>  source "fs/initramfs/Config.in"
>  source "fs/romfs/Config.in"
> +source "fs/live/Config.in"
>
>  endmenu
> diff --git a/fs/live/Config.in b/fs/live/Config.in
> new file mode 100644
> index 0000000..bf24754
> --- /dev/null
> +++ b/fs/live/Config.in
> @@ -0,0 +1,16 @@
> +config BR2_TARGET_ROOTFS_LIVE
> +       bool "live root filesystem"
> +       help
> +         uses sudo to create a live image of the filesystem
> +         this is mainly usefull if you want to use your filesystem
> +         over NFS
> +config BR2_TARGET_ROOTFS_LIVE_DEST
> +       string "directory to put the live image"
> +       depends on BR2_TARGET_ROOTFS_LIVE
> +       default "live"
> +       help
> +         The directory where the image should be stored.
> +         this directory will be emptied and recreated, it is given
> +         relative to the buildroot output/images directory
> +
> +
> diff --git a/fs/live/live.mk b/fs/live/live.mk
> new file mode 100644
> index 0000000..73bcd3e
> --- /dev/null
> +++ b/fs/live/live.mk
> @@ -0,0 +1,23 @@
> +#############################################################
> +#
> +# Build the live root filesystem directory
> +#
> +#############################################################
> +
> +LIVE_OPTS :=
> +
> +
> +ROOTFS_LIVE_DEPENDENCIES = #host-sudo
> +
> +define ROOTFS_LIVE_CMD
> + sudo cp -r $(TARGET_DIR)/* $(BINARIES_DIR)/$(BR2_TARGET_ROOTFS_LIVE_DEST)/
> +endef
> +
> +define ROOTFS_LIVE_INIT
> +       sudo rm -rf $(BINARIES_DIR)/$(BR2_TARGET_ROOTFS_LIVE_DEST)
> +       sudo mkdir $(BINARIES_DIR)/$(BR2_TARGET_ROOTFS_LIVE_DEST)
> +endef
> +
> +ROOTFS_LIVE_PRE_GEN_HOOKS += ROOTFS_LIVE_INIT
> +
> +$(eval $(call ROOTFS_TARGET,live))
> --
> 1.7.10.4
>
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] [PATCH][RFC] new target: live filesystem

[Jeremy Rosen]

> > * no dependency on host-sudo, i'm not sure how to add that
> 
>   That's impossible. The sudo executable must be setuid root, and to
>   make
> it setuid root you have to be root, so sudo must already exist...
> 
>   You could instead add a check in live.mk if sudo actually works.
>    But since
> sudo may ask for a password this can get tricky - you should e.g.
> probably check
> if stdin is a terminal as well.
> 

hmm, good point... I hadn't thought of that i'll add a check for that 

>   That said, I don't know if Peter will accept running sudo within
>   buildroot.
> 

:) yes, that's why I wanted some feedback early...

I think running sudo only for the deployment of the FS is isolated enough to be worth it (it's not like I'm getting rid of fakeroot or anything, it's just to ease the final step of NFS work)

but yes, if that patch is rejected because of philosophical reasons, it would be nice to know early

> 
> > * filesystem can only be under output/images at this point
> 
>   That's not good but easy to solve.
> 

yes that was one of the "first draft not ready for commit" aspect :)

you suggest in the commented patch below to use an absolute path...

I like the idea, if I add a line to the help text hinting that $(WHATEVER) can be used to point to output/image it's probably good enough


only answering the remarks which i'd like to discuss below, I agree with all the others

> > +	depends on BR2_TARGET_ROOTFS_LIVE
> > +	default "live"
> 
>   Use an empty default, and give an error in live.mk if it's
> empty.
> 

I don't like to provide a non-working default (esp since most/all other fs targets work out of the box) 

why not provide one (i'm perfectly fine with another name)

if this project is ok with non-working defaults i'll go that way, but it's simple enough to provide a default under output/images so i'm not sur why not do it...

> > +
> > +define ROOTFS_LIVE_CMD
> > + sudo cp -r $(TARGET_DIR)/*
> > $(BINARIES_DIR)/$(BR2_TARGET_ROOTFS_LIVE_DEST)/
> 
>   That won't work, because device nodes, symlinks etc. are not
> copied correctly.  Use cp -a.  Or even better, rsync -a
> --delete-during
> (then you don't need to remove it first).
> 
>   For the target, just use $(BR2_TARGET_ROOTFS_LIVE_DEST).
> 

ok, i'll check all those cp/rsync options I don't know...


V2 of the patch is under way and will be posted soon

Regards

J?r?my

[Buildroot] [PATCH] [PATCH][RFC] new target: live filesystem

[Arnout Vandecappelle]

On 30/11/12 01:04, Steve Calfee wrote:
> I don't understand why you need this. It is a one line "sudo tar -xf
> ..." to unpack the tarball into your nfs directory. Either type this
> line or add a one line script to your ~/bin directory to do it for
> each of your buildroot trees/environments. Don't add clutter to the
> buildroot makefile.

  If that is the reasoning, we can also remove the compression of the
.tar file, or the -dirclean targets, etc. etc.

  In addition, this target also cleans up the NFS directory first, so
it's slightly more than just untarring.

  Regards,
  Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH] [PATCH][RFC] new target: live filesystem

[Steve Calfee]

On Fri, Nov 30, 2012 at 1:15 AM, Arnout Vandecappelle <arnout@mind.be> wrote:
> On 30/11/12 01:04, Steve Calfee wrote:
>>
>> I don't understand why you need this. It is a one line "sudo tar -xf
>> ..." to unpack the tarball into your nfs directory. Either type this
>> line or add a one line script to your ~/bin directory to do it for
>> each of your buildroot trees/environments. Don't add clutter to the
>> buildroot makefile.
>
>
>  If that is the reasoning, we can also remove the compression of the
> .tar file, or the -dirclean targets, etc. etc.
>
>  In addition, this target also cleans up the NFS directory first, so
> it's slightly more than just untarring.
>
Hi  Arnout,

I guess, but buildroot is all about building, not deploying. What is
next, an automatic copy to my usb or sd drive? Stuff that is highly
user specific should not be in the makefile, especially stuff that can
easily be added to a simple script - my opinion only,

Steve

[Buildroot] [PATCH] [PATCH][RFC] new target: live filesystem

[Jeremy Rosen]

well, it's more a question of what you consider the live filesystem to be...

I see it a as just another filesystem format (basically a NFS target) which is why my original patch had it in output/images/live 

if you see it as a deployment, it does not really make sense to add it to buildroot (buildroot doesn't flash the filesystem directly) I can understand that both point of view make sense

however having this target makes things simpler in a very common use case (arguably the most common use case since it's the case where you use buildroot repeatedly to prepare the fs, not the case where you run it once to build the final FS)

yes it's a two line script, and yes it's a minor change, but I think it makes sense and it's simple enough for it not to be a big deal...



    Regards

    J?r?my Rosen

fight key loggers : write some perl using vim

----- Mail original -----
> De: "Steve Calfee" <stevecalfee@gmail.com>
> ?: "Arnout Vandecappelle" <arnout@mind.be>
> Cc: "J?r?my Rosen" <jeremy.rosen@openwide.fr>, buildroot at busybox.net
> Envoy?: Vendredi 30 Novembre 2012 22:46:53
> Objet: Re: [Buildroot] [PATCH] [PATCH][RFC] new target: live filesystem
> 
> On Fri, Nov 30, 2012 at 1:15 AM, Arnout Vandecappelle
> <arnout@mind.be> wrote:
> > On 30/11/12 01:04, Steve Calfee wrote:
> >>
> >> I don't understand why you need this. It is a one line "sudo tar
> >> -xf
> >> ..." to unpack the tarball into your nfs directory. Either type
> >> this
> >> line or add a one line script to your ~/bin directory to do it for
> >> each of your buildroot trees/environments. Don't add clutter to
> >> the
> >> buildroot makefile.
> >
> >
> >  If that is the reasoning, we can also remove the compression of
> >  the
> > .tar file, or the -dirclean targets, etc. etc.
> >
> >  In addition, this target also cleans up the NFS directory first,
> >  so
> > it's slightly more than just untarring.
> >
> Hi  Arnout,
> 
> I guess, but buildroot is all about building, not deploying. What is
> next, an automatic copy to my usb or sd drive? Stuff that is highly
> user specific should not be in the makefile, especially stuff that
> can
> easily be added to a simple script - my opinion only,
> 
> Steve
> 

[Buildroot] [PATCH] [PATCH][RFC] new target: live filesystem

[Samuel Martin]

Hi Jeremy, all,

2012/12/3 Jeremy Rosen <jeremy.rosen@openwide.fr>:
> well, it's more a question of what you consider the live filesystem to be...
>
> I see it a as just another filesystem format (basically a NFS target) which is why my original patch had it in output/images/live
>
> if you see it as a deployment, it does not really make sense to add it to buildroot (buildroot doesn't flash the filesystem directly) I can understand that both point of view make sense
>
> however having this target makes things simpler in a very common use case (arguably the most common use case since it's the case where you use buildroot repeatedly to prepare the fs, not the case where you run it once to build the final FS)

That's a controversial topic...
But, at least, you cat start filling the gap in the doc ;-),
especially this section:
http://buildroot.org/downloads/manual/manual.html#_beyond_buildroot


-- 
Sam

[Buildroot] [PATCH] reorder fs alphabetically

[Jérémy Rosen]

Signed-off-by: J?r?my Rosen <jeremy.rosen@openwide.fr>
---
 fs/Config.in |   12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/fs/Config.in b/fs/Config.in
index 94154ea..da4c5ff 100644
--- a/fs/Config.in
+++ b/fs/Config.in
@@ -1,15 +1,15 @@
 menu "Filesystem images"
 
-source "fs/cramfs/Config.in"
 source "fs/cloop/Config.in"
+source "fs/cpio/Config.in"
+source "fs/cramfs/Config.in"
 source "fs/ext2/Config.in"
+source "fs/initramfs/Config.in"
+source "fs/iso9660/Config.in"
 source "fs/jffs2/Config.in"
-source "fs/ubifs/Config.in"
+source "fs/romfs/Config.in"
 source "fs/squashfs/Config.in"
 source "fs/tar/Config.in"
-source "fs/cpio/Config.in"
-source "fs/iso9660/Config.in"
-source "fs/initramfs/Config.in"
-source "fs/romfs/Config.in"
+source "fs/ubifs/Config.in"
 
 endmenu
-- 
1.7.10.4

[Buildroot] [PATCH] reorder fs alphabetically

[Jeremy Rosen]

Oops, I didn't intend to resent that patch, I wasn't on the correct branch


sorry about that

    Cordialement

    J?r?my Rosen

fight key loggers : write some perl using vim

----- Mail original -----
> De: "J?r?my Rosen" <jeremy.rosen@openwide.fr>
> ?: buildroot at busybox.net
> Cc: "J?r?my Rosen" <jeremy.rosen@openwide.fr>
> Envoy?: Lundi 3 D?cembre 2012 11:21:49
> Objet: [PATCH] reorder fs alphabetically
> 
> Signed-off-by: J?r?my Rosen <jeremy.rosen@openwide.fr>
> ---
>  fs/Config.in |   12 ++++++------
>  1 file changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/fs/Config.in b/fs/Config.in
> index 94154ea..da4c5ff 100644
> --- a/fs/Config.in
> +++ b/fs/Config.in
> @@ -1,15 +1,15 @@
>  menu "Filesystem images"
>  
> -source "fs/cramfs/Config.in"
>  source "fs/cloop/Config.in"
> +source "fs/cpio/Config.in"
> +source "fs/cramfs/Config.in"
>  source "fs/ext2/Config.in"
> +source "fs/initramfs/Config.in"
> +source "fs/iso9660/Config.in"
>  source "fs/jffs2/Config.in"
> -source "fs/ubifs/Config.in"
> +source "fs/romfs/Config.in"
>  source "fs/squashfs/Config.in"
>  source "fs/tar/Config.in"
> -source "fs/cpio/Config.in"
> -source "fs/iso9660/Config.in"
> -source "fs/initramfs/Config.in"
> -source "fs/romfs/Config.in"
> +source "fs/ubifs/Config.in"
>  
>  endmenu
> --
> 1.7.10.4
> 
> 

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Jérémy Rosen]

add a new target to deploy a live filesystem to be used with NFS or as a chroot

Signed-off-by: J?r?my Rosen <jeremy.rosen@openwide.fr>
---
v2 : implement Arnoult's suggestion, update manual entry
---
 docs/manual/beyond-buildroot.txt |   16 +++++++---------
 fs/Config.in                     |    1 +
 fs/live/Config.in                |   16 ++++++++++++++++
 fs/live/live.mk                  |   20 ++++++++++++++++++++
 4 files changed, 44 insertions(+), 9 deletions(-)
 create mode 100644 fs/live/Config.in
 create mode 100644 fs/live/live.mk

diff --git a/docs/manual/beyond-buildroot.txt b/docs/manual/beyond-buildroot.txt
index e7b902d..adf3e83 100644
--- a/docs/manual/beyond-buildroot.txt
+++ b/docs/manual/beyond-buildroot.txt
@@ -9,17 +9,15 @@ Boot the generated images
 NFS boot
 ~~~~~~~~
 
-To achieve NFS-boot, enable _tar root filesystem_ in the _Filesystem
-images_ menu.
+To achieve NFS-boot, enable _live root filesystem_ in the _Filesystem
+images_ menu and select a _live image location_ to choose where the live
+filesystem will be deployed. you can use _$(BINARIES_DIR)_ to easily 
+build in +/path/to/output_dir/+
 
-After a complete build, just run the following commands to setup the
-NFS-root directory:
+You will be asked for a password during the build. This is needed to create
+device entries in the target filesystem
 
--------------------
-sudo tar -xavf /path/to/output_dir/rootfs.tar -C /path/to/nfs_root_dir
--------------------
-
-Then, you can execute a NFS-boot from your target.
+You will need to add the target path  to +/etc/exports+.
 
 Chroot
 ------
diff --git a/fs/Config.in b/fs/Config.in
index 94154ea..de2377e 100644
--- a/fs/Config.in
+++ b/fs/Config.in
@@ -11,5 +11,6 @@ source "fs/cpio/Config.in"
 source "fs/iso9660/Config.in"
 source "fs/initramfs/Config.in"
 source "fs/romfs/Config.in"
+source "fs/live/Config.in"
 
 endmenu
diff --git a/fs/live/Config.in b/fs/live/Config.in
new file mode 100644
index 0000000..60d03a7
--- /dev/null
+++ b/fs/live/Config.in
@@ -0,0 +1,16 @@
+config BR2_TARGET_ROOTFS_LIVE
+	bool "live root filesystem"
+	help
+	  uses sudo to create a live image of the filesystem
+	  this is mainly useful if you want to use your filesystem
+	  over NFS or a chroot
+config BR2_TARGET_ROOTFS_LIVE_DEST
+	string "live image location"
+	depends on BR2_TARGET_ROOTFS_LIVE
+	default "$(BINARIES_DIR)/live"
+	help
+	  The directory where the image should be stored.
+	  this directory will be emptied and recreated, it is given
+	  relative to the buildroot output/images directory
+
+
diff --git a/fs/live/live.mk b/fs/live/live.mk
new file mode 100644
index 0000000..33fe515
--- /dev/null
+++ b/fs/live/live.mk
@@ -0,0 +1,20 @@
+#############################################################
+#
+# Build the live root filesystem directory
+#
+#############################################################
+
+
+define ROOTFS_LIVE_CMD
+ sudo rsync -a --no-o --no-g --delete-during $(TARGET_DIR)/ $(BR2_TARGET_ROOTFS_LIVE_DEST)/
+endef
+
+define ROOTFS_LIVE_INIT
+  if [ -z $(shell which sudo) ] ; then echo "sudo seems to not be installed on the host system" ; false ; fi ; \
+  if [ ! -t 0 ] ; then echo "live filesystem must be generated interactively" ; false ; fi ; \
+  if [ ! -t 2 ] ; then echo "live filesystem must be generated interactively" ; false ; fi ; 
+endef
+
+ROOTFS_LIVE_PRE_GEN_HOOKS += ROOTFS_LIVE_INIT
+
+$(eval $(call ROOTFS_TARGET,live))
-- 
1.7.10.4

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Thomas Petazzoni]

Dear J?r?my Rosen,

On Mon,  3 Dec 2012 11:40:03 +0100, J?r?my Rosen wrote:

> diff --git a/fs/live/live.mk b/fs/live/live.mk
> new file mode 100644
> index 0000000..33fe515
> --- /dev/null
> +++ b/fs/live/live.mk
> @@ -0,0 +1,20 @@
> +#############################################################
> +#
> +# Build the live root filesystem directory
> +#
> +#############################################################
> +
> +
> +define ROOTFS_LIVE_CMD
> + sudo rsync -a --no-o --no-g --delete-during $(TARGET_DIR)/ $(BR2_TARGET_ROOTFS_LIVE_DEST)/
> +endef

Do we really want to include sudo commands in Buildroot? I'm not sure
we want.

What about instead a post-image script hook, that users can use to
extract automatically the tarball image somewhere?

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Jeremy Rosen]

> > +
> > +define ROOTFS_LIVE_CMD
> > + sudo rsync -a --no-o --no-g --delete-during $(TARGET_DIR)/
> > $(BR2_TARGET_ROOTFS_LIVE_DEST)/
> > +endef
> 
> Do we really want to include sudo commands in Buildroot? I'm not sure
> we want.

that's the question I wanted answered by sending this patch early... I'm still quite new in this community and I am not sure how the buildroot philosophy is seen

note that I check for buildroot properly in the PRE_GEN_HOOKS and that I don't try to install host-sudo (which is not possible anyway)

I personally think that building target/ with sudo instead of fakeroot would be a stupid idea, but that's not what this patch does. This patch simply adds a new optional target to ease NFS deployment,

> 
> What about instead a post-image script hook, that users can use to
> extract automatically the tarball image somewhere?
> 

yes, that would work too, I can do that locally to solve my particular problem, but I don't think you would accept it upstream either. If you consider that deploying the live filesystem is not buildroot's job, fine I'll probably do it with post-build scripts instead, but I personally believe that NFS is the most common target and that doing it in buildroot-core is more convinient and makes more sense

pos-install scripts needs to be rewritten by each user, and doing it manually is a bit error prone (especially if you have lots of devs sharing a buildroot project through git, you can have weird side effects with people not cleaning before deplying the new FS) 


again, I personally think this use-case makes sense, especially in big teams where some members don't want to learn what NFS is (yeah, sounds stupid, but it does happen) and I don't think there are any major drawback (we test sudo properly, we only call it for a specific use-case where it is really needed) except the philosophical question of "should a build tool use a tool giving root-priv like sudo" which in the case of NFS doesn't change much since the user will have to use it himself (either directly or trough post-image script). 

I'd really think it makes sense in a "type make, reboot the target, test your change" philosophy which matches really well the idea behind NFS based developement, but it's your call as with any core change...

Regards
J?r?my


> Thomas
> --
> Thomas Petazzoni, Free Electrons
> Kernel, drivers, real-time and embedded Linux
> development, consulting, training and support.
> http://free-electrons.com
> 

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Arnout Vandecappelle]

On 03/12/12 11:40, J?r?my Rosen wrote:
> add a new target to deploy a live filesystem to be used with NFS or as a chroot
>
> Signed-off-by: J?r?my Rosen<jeremy.rosen@openwide.fr>
> ---
> v2 : implement Arnoult's suggestion, update manual entry
> ---
>   docs/manual/beyond-buildroot.txt |   16 +++++++---------
>   fs/Config.in                     |    1 +
>   fs/live/Config.in                |   16 ++++++++++++++++
>   fs/live/live.mk                  |   20 ++++++++++++++++++++
>   4 files changed, 44 insertions(+), 9 deletions(-)
>   create mode 100644 fs/live/Config.in
>   create mode 100644 fs/live/live.mk
>
> diff --git a/docs/manual/beyond-buildroot.txt b/docs/manual/beyond-buildroot.txt
> index e7b902d..adf3e83 100644
> --- a/docs/manual/beyond-buildroot.txt
> +++ b/docs/manual/beyond-buildroot.txt
> @@ -9,17 +9,15 @@ Boot the generated images
>   NFS boot
>   ~~~~~~~~
>
> -To achieve NFS-boot, enable _tar root filesystem_ in the _Filesystem
> -images_ menu.
> +To achieve NFS-boot, enable _live root filesystem_ in the _Filesystem
> +images_ menu and select a _live image location_ to choose where the live
> +filesystem will be deployed. you can use _$(BINARIES_DIR)_ to easily
> +build in +/path/to/output_dir/+

  +/path/to/output_dir/images/+

>
> -After a complete build, just run the following commands to setup the
> -NFS-root directory:
> +You will be asked for a password during the build. This is needed to create
> +device entries in the target filesystem
>
> --------------------
> -sudo tar -xavf /path/to/output_dir/rootfs.tar -C /path/to/nfs_root_dir
> --------------------
> -
> -Then, you can execute a NFS-boot from your target.
> +You will need to add the target path  to +/etc/exports+.

  You will need to add the _live image location_ to +/etc/exports+.

>
>   Chroot
>   ------
> diff --git a/fs/Config.in b/fs/Config.in
> index 94154ea..de2377e 100644
> --- a/fs/Config.in
> +++ b/fs/Config.in
> @@ -11,5 +11,6 @@ source "fs/cpio/Config.in"
>   source "fs/iso9660/Config.in"
>   source "fs/initramfs/Config.in"
>   source "fs/romfs/Config.in"
> +source "fs/live/Config.in"
>
>   endmenu
> diff --git a/fs/live/Config.in b/fs/live/Config.in
> new file mode 100644
> index 0000000..60d03a7
> --- /dev/null
> +++ b/fs/live/Config.in
> @@ -0,0 +1,16 @@
> +config BR2_TARGET_ROOTFS_LIVE
> +	bool "live root filesystem"
> +	help
> +	  uses sudo to create a live image of the filesystem
> +	  this is mainly useful if you want to use your filesystem
> +	  over NFS or a chroot

  Help text:

Create a live image of the root filesystem that is directly
usable as over NFS or chroot.

  Leave an empty line between this and the next config.

> +config BR2_TARGET_ROOTFS_LIVE_DEST
> +	string "live image location"
> +	depends on BR2_TARGET_ROOTFS_LIVE
> +	default "$(BINARIES_DIR)/live"
> +	help
> +	  The directory where the image should be stored.
> +	  this directory will be emptied and recreated, it is given

  this -> This

> +	  relative to the buildroot output/images directory

  The second part of this statement is redundant (it's true for all
paths in the config and is not mentioned anywhere else)

> +
> +

  Don't put empty lines at the end of the file (but do make sure
it ends with a newline).

> diff --git a/fs/live/live.mk b/fs/live/live.mk
> new file mode 100644
> index 0000000..33fe515
> --- /dev/null
> +++ b/fs/live/live.mk
> @@ -0,0 +1,20 @@
> +#############################################################
> +#
> +# Build the live root filesystem directory
> +#
> +#############################################################
> +
> +
> +define ROOTFS_LIVE_CMD
> + sudo rsync -a --no-o --no-g --delete-during $(TARGET_DIR)/ $(BR2_TARGET_ROOTFS_LIVE_DEST)/

  Start commands with a tab - that makes it clearer that it's a
command.

  Split the long line.

> +endef
> +
> +define ROOTFS_LIVE_INIT
> +  if [ -z $(shell which sudo) ] ; then echo "sudo seems to not be installed on the host system" ; false ; fi ; \

  This really should be checked in support/dependencies/dependencies.sh.

> +  if [ ! -t 0 ] ; then echo "live filesystem must be generated interactively" ; false ; fi ; \
> +  if [ ! -t 2 ] ; then echo "live filesystem must be generated interactively" ; false ; fi ;

  Don't concatenate commands with \ but just start each command on a new
line starting with a tab.  (Of course, when you split the condition
over several lines, then you have to use \).

  This test is not actually correct, because sudo will use the
controlling terminal (/dev/tty) to ask a password, i.e. it still
works when input and output are redirected. It will however not
work when there is no controlling terminal, e.g. when it's invoked
directly from a window manager menu.  In addition, if an askpass
program is specified, sudo doesn't even need a controlling terminal.

  So maybe it's better to drop the check after all. The sudo command
itself will just fail with 'sudo: no tty present and no askpass
program specified', which is clear enough.

  (I realize I asked for this check in the first place - but only
idiots never change their mind :-)


  Regards,
  Arnout

> +endef
> +
> +ROOTFS_LIVE_PRE_GEN_HOOKS += ROOTFS_LIVE_INIT
> +
> +$(eval $(call ROOTFS_TARGET,live))

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Arnout Vandecappelle]

On 03/12/12 14:42, Jeremy Rosen wrote:
>
>>> +
>>> +define ROOTFS_LIVE_CMD
>>> + sudo rsync -a --no-o --no-g --delete-during $(TARGET_DIR)/
>>> $(BR2_TARGET_ROOTFS_LIVE_DEST)/
>>> +endef
>>
>> Do we really want to include sudo commands in Buildroot? I'm not sure
>> we want.

  I don't see a fundamental problem with using sudo within buildroot,
as long as it's not for things that can be fakerooted.

[snip]
>>
>> What about instead a post-image script hook, that users can use to
>> extract automatically the tarball image somewhere?
>>
>
> yes, that would work too, I can do that locally to solve my particular problem, but I don't think you would accept it upstream either. If you consider that deploying the live filesystem is not buildroot's job, fine I'll probably do it with post-build scripts instead, but I personally believe that NFS is the most common target and that doing it in buildroot-core is more convinient and makes more sense
>
> pos-install scripts needs to be rewritten by each user, and doing it manually is a bit error prone (especially if you have lots of devs sharing a buildroot project through git, you can have weird side effects with people not cleaning before deplying the new FS)

  +1 to that.

  For me, buildroot should make it *easy* to:

- get an initial system up and running (quickly);

- develop for the target board;

- offer a reproducible and shareable build system.

  We already do very well on all three counts, but we can
always improve.


  In addition, the NFS stuff is a FAQ.  And FAQs are bugs,
really.


  Regards,
  Arnout


> again, I personally think this use-case makes sense, especially in big teams where some members don't want to learn what NFS is (yeah, sounds stupid, but it does happen) and I don't think there are any major drawback (we test sudo properly, we only call it for a specific use-case where it is really needed) except the philosophical question of "should a build tool use a tool giving root-priv like sudo" which in the case of NFS doesn't change much since the user will have to use it himself (either directly or trough post-image script).
>
> I'd really think it makes sense in a "type make, reboot the target, test your change" philosophy which matches really well the idea behind NFS based developement, but it's your call as with any core change...
>
> Regards
> J?r?my

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Jeremy Rosen]

V3 is on the way....

I skip items I agree with and applied

> 
> > +config BR2_TARGET_ROOTFS_LIVE_DEST
> > +	string "live image location"
> > +	depends on BR2_TARGET_ROOTFS_LIVE
> > +	default "$(BINARIES_DIR)/live"
> > +	help
> > +	  The directory where the image should be stored.
> > +	  this directory will be emptied and recreated, it is given
> 
>   this -> This
> 
> > +	  relative to the buildroot output/images directory
> 
>   The second part of this statement is redundant (it's true for all
> paths in the config and is not mentioned anywhere else)
> 

and the whole thing is false since that path is now absolute (which makes more sense for NFS deployement)

unless you want me to make it relative again... I believe absolute (whith the possibility to use env vars to find the buildroot location) makes most sense but that might be worth discussing


> 
> > +endef
> > +
> > +define ROOTFS_LIVE_INIT
> > +  if [ -z $(shell which sudo) ] ; then echo "sudo seems to not be
> > installed on the host system" ; false ; fi ; \
> 
>   This really should be checked in
>   support/dependencies/dependencies.sh.
> 

I didn't know about that file, but it seems to be more about dependencies for buildroot core than dependencies for a particular config option...

otoh it's the only place where we check for stuff installed on the host (not compiled for host, natively installed on host)

maybe the cleanest way to do it would be to have a virtual target

ROOTFS_DEPENDS=native-xxx

that would just check that "which xxx" exists on the system...

that's a different patch I could have a look at if people think it's a good idea (I am not good at makefile so I have no idea if it's one line of code or if i'm going into makefile hell here...)


> > +  if [ ! -t 0 ] ; then echo "live filesystem must be generated
> > interactively" ; false ; fi ; \
> > +  if [ ! -t 2 ] ; then echo "live filesystem must be generated
> > interactively" ; false ; fi ;
> 
>   Don't concatenate commands with \ but just start each command on a
>   new
> line starting with a tab.  (Of course, when you split the condition
> over several lines, then you have to use \).
> 

ok

>   This test is not actually correct, because sudo will use the
> controlling terminal (/dev/tty) to ask a password, i.e. it still
> works when input and output are redirected. It will however not
> work when there is no controlling terminal, e.g. when it's invoked
> directly from a window manager menu.  In addition, if an askpass
> program is specified, sudo doesn't even need a controlling terminal.
> 
>   So maybe it's better to drop the check after all. The sudo command
> itself will just fail with 'sudo: no tty present and no askpass
> program specified', which is clear enough.
> 

i'll test that

>   (I realize I asked for this check in the first place - but only
> idiots never change their mind :-)


fair enough, nobody gets the first idea on the first try

I could also run sudo with "fail instead of asking for a password" option, and have people add the proper line in the sudo config file to allow untar without password, but i'm not sure if it's better or worse

i'll probably wait for your answer before sending V3

Thx for the proofreading

Regards
J?r?my Rosen

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Arnout Vandecappelle]

On 05/12/12 10:28, Jeremy Rosen wrote:
> V3 is on the way....
>
> I skip items I agree with and applied
>
>>
>>> +config BR2_TARGET_ROOTFS_LIVE_DEST
>>> +	string "live image location"
>>> +	depends on BR2_TARGET_ROOTFS_LIVE
>>> +	default "$(BINARIES_DIR)/live"
>>> +	help
>>> +	  The directory where the image should be stored.
>>> +	  this directory will be emptied and recreated, it is given
>>
>>    this ->  This
>>
>>> +	  relative to the buildroot output/images directory
>>
>>    The second part of this statement is redundant (it's true for all
>> paths in the config and is not mentioned anywhere else)
>>
>
> and the whole thing is false since that path is now absolute (which makes more sense for NFS deployement)
>
> unless you want me to make it relative again... I believe absolute (whith the possibility to use env vars to find the buildroot location) makes most sense but that might be worth discussing

  Like most other paths in buildroot, it's relative to $(TOP_DIR)
(i.e. the buildroot directory).


>>> +endef
>>> +
>>> +define ROOTFS_LIVE_INIT
>>> +  if [ -z $(shell which sudo) ] ; then echo "sudo seems to not be
>>> installed on the host system" ; false ; fi ; \
>>
>>    This really should be checked in
>>    support/dependencies/dependencies.sh.
>>
>
> I didn't know about that file, but it seems to be more about dependencies for buildroot core than dependencies for a particular config option...

  I forgot to add: similar to the java dependencies for classpath.


>
> otoh it's the only place where we check for stuff installed on the host (not compiled for host, natively installed on host)
>
> maybe the cleanest way to do it would be to have a virtual target
>
> ROOTFS_DEPENDS=native-xxx
>
> that would just check that "which xxx" exists on the system...
>
> that's a different patch I could have a look at if people think it's a good idea (I am not good at makefile so I have no idea if it's one line of code or if i'm going into makefile hell here...)

  That's not how we work - all native dependencies are checked in
support/dependencies.


[snip]
> I could also run sudo with "fail instead of asking for a password" option, and have people add the proper line in the sudo config file to allow untar without password, but i'm not sure if it's better or worse

  No, we don't want to force people to change their host config just so they
can use buildroot.


  Regards,
  Arnout

>
> i'll probably wait for your answer before sending V3
>
> Thx for the proofreading
>
> Regards
> J?r?my Rosen
>

-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Jeremy Rosen]

> > unless you want me to make it relative again... I believe absolute
> > (whith the possibility to use env vars to find the buildroot
> > location) makes most sense but that might be worth discussing
> 
>   Like most other paths in buildroot, it's relative to $(TOP_DIR)
> (i.e. the buildroot directory).
> 

my bad, you're right about that, i'll correct that

> 
> >>> +endef
> >>> +
> >>> +define ROOTFS_LIVE_INIT
> >>> +  if [ -z $(shell which sudo) ] ; then echo "sudo seems to not
> >>> be
> >>> installed on the host system" ; false ; fi ; \
> >>
> >>    This really should be checked in
> >>    support/dependencies/dependencies.sh.
> >>
> >
> > I didn't know about that file, but it seems to be more about
> > dependencies for buildroot core than dependencies for a particular
> > config option...
> 
>   I forgot to add: similar to the java dependencies for classpath.
> 
> 

I didn't find anything like that in buildroot... there are some config-dependant checks in dependencies.sh (checking is locale is installed) but nothing class path related

am I missing something or should I do it in a way similar to locale ?

Regards

J?r?my Rosen

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Jérémy Rosen]

add a new target to deploy a live filesystem to be used with NFS or as a chroot

Signed-off-by: J?r?my Rosen <jeremy.rosen@openwide.fr>
---
v2 : implement Arnoult's suggestion, update manual entry
v3 : improve documentation for the chroot case, more suggestions by Arnoult
---
 docs/manual/beyond-buildroot.txt     |   20 ++++++++------------
 fs/Config.in                         |    1 +
 fs/live/Config.in                    |   14 ++++++++++++++
 fs/live/live.mk                      |   19 +++++++++++++++++++
 support/dependencies/dependencies.sh |    8 ++++++++
 5 files changed, 50 insertions(+), 12 deletions(-)
 create mode 100644 fs/live/Config.in
 create mode 100644 fs/live/live.mk

diff --git a/docs/manual/beyond-buildroot.txt b/docs/manual/beyond-buildroot.txt
index a87b584..17ccc1a 100644
--- a/docs/manual/beyond-buildroot.txt
+++ b/docs/manual/beyond-buildroot.txt
@@ -9,19 +9,15 @@ Boot the generated images
 NFS boot
 ~~~~~~~~
 
-To achieve NFS-boot, enable _tar root filesystem_ in the _Filesystem
-images_ menu.
+To achieve NFS-boot, enable _live root filesystem_ in the _Filesystem
+images_ menu and select a _live image location_ to choose where the live
+filesystem will be deployed. you can use _$(BINARIES_DIR)_ to easily 
+build in +/path/to/output_dir/images+
 
-After a complete build, just run the following commands to setup the
-NFS-root directory:
+You will be asked for a password during the build. This is needed to create
+device entries in the target filesystem
 
--------------------
-sudo tar -xavf /path/to/output_dir/rootfs.tar -C /path/to/nfs_root_dir
--------------------
-
-Remember to add this path to +/etc/exports+.
-
-Then, you can execute a NFS-boot from your target.
+You will need to add the _live image location_  to +/etc/exports+.
 
 Chroot
 ------
@@ -29,7 +25,7 @@ Chroot
 If you want to chroot in a generated image, then there are few thing
 you should be aware of:
 
-* you should setup the new root from the _tar root filesystem_ image;
+* you should use the _live root filesystem_ image;
 
 * either the selected target architecture is compatible with your host
   machine, or you should use some +qemu-*+ binary and correctly set it
diff --git a/fs/Config.in b/fs/Config.in
index da4c5ff..664d2f6 100644
--- a/fs/Config.in
+++ b/fs/Config.in
@@ -11,5 +11,6 @@ source "fs/romfs/Config.in"
 source "fs/squashfs/Config.in"
 source "fs/tar/Config.in"
 source "fs/ubifs/Config.in"
+source "fs/live/Config.in"
 
 endmenu
diff --git a/fs/live/Config.in b/fs/live/Config.in
new file mode 100644
index 0000000..a79f1dc
--- /dev/null
+++ b/fs/live/Config.in
@@ -0,0 +1,14 @@
+config BR2_TARGET_ROOTFS_LIVE
+	bool "live root filesystem"
+	help
+	  Create a live image of the root filesystem that is directly
+	  usable as over NFS or chroot.
+
+config BR2_TARGET_ROOTFS_LIVE_DEST
+	string "live image location"
+	depends on BR2_TARGET_ROOTFS_LIVE
+	default "$(BINARIES_DIR)/live"
+	help
+	  The directory where the image should be stored.
+	  This directory will be emptied and recreated
+
diff --git a/fs/live/live.mk b/fs/live/live.mk
new file mode 100644
index 0000000..52f7444
--- /dev/null
+++ b/fs/live/live.mk
@@ -0,0 +1,19 @@
+#############################################################
+#
+# Build the live root filesystem directory
+#
+#############################################################
+
+
+define ROOTFS_LIVE_CMD
+	sudo rsync -a --no-o --no-g --delete-during $(TARGET_DIR)/ \
+		$(BR2_TARGET_ROOTFS_LIVE_DEST)/
+endef
+
+define ROOTFS_LIVE_INIT
+  if [ -z $(shell which sudo) ] ; then echo "sudo seems to not be installed on the host system" ; false ; fi 
+endef
+
+ROOTFS_LIVE_PRE_GEN_HOOKS += ROOTFS_LIVE_INIT
+
+$(eval $(call ROOTFS_TARGET,live))
diff --git a/support/dependencies/dependencies.sh b/support/dependencies/dependencies.sh
index 7a02512..ebaabbb 100755
--- a/support/dependencies/dependencies.sh
+++ b/support/dependencies/dependencies.sh
@@ -158,6 +158,7 @@ if grep ^BR2_TOOLCHAIN_BUILDROOT=y $CONFIG_FILE > /dev/null && \
        exit 1 ;
    fi
 fi
+
 if grep -q ^BR2_PACKAGE_CLASSPATH=y $CONFIG_FILE ; then
     for prog in javac jar; do
 	if ! which $prog > /dev/null ; then
@@ -166,3 +167,10 @@ if grep -q ^BR2_PACKAGE_CLASSPATH=y $CONFIG_FILE ; then
 	fi
     done
 fi
+
+if grep ^BR2_TARGET_ROOTFS_LIVE=y $CONFIG_FILE > /dev/null ; then
+   if ! which sudo > /dev/null ; then
+       /bin/echo -e "\nYou need sudo installed on your build machine to build a live filesystem\n"
+       exit 1 ;
+   fi
+fi
-- 
1.7.10.4

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Jeremy Rosen]

> > 
> >   I forgot to add: similar to the java dependencies for classpath.
> > 
> > 
> 
> I didn't find anything like that in buildroot... there are some
> config-dependant checks in dependencies.sh (checking is locale is
> installed) but nothing class path related
> 
> am I missing something or should I do it in a way similar to locale ?
> 

ok, again it's my fault, I hadn't rebased in the last couple of days, v3 should be ready now

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Arnout Vandecappelle]

  Peter, can you make the call if this has a chance of being accepted?
Otherwise J?r?my is wasting his time...


On 06/12/12 14:36, J?r?my Rosen wrote:
> add a new target to deploy a live filesystem to be used with NFS or as a chroot

  The commit message should be word-wrapped at +- 75 characters IIRC.

>
> Signed-off-by: J?r?my Rosen<jeremy.rosen@openwide.fr>
> ---
> v2 : implement Arnoult's suggestion, update manual entry

  Arnout

> v3 : improve documentation for the chroot case, more suggestions by Arnoult
> ---
[snip]
> diff --git a/fs/Config.in b/fs/Config.in
> index da4c5ff..664d2f6 100644
> --- a/fs/Config.in
> +++ b/fs/Config.in
> @@ -11,5 +11,6 @@ source "fs/romfs/Config.in"
>   source "fs/squashfs/Config.in"
>   source "fs/tar/Config.in"
>   source "fs/ubifs/Config.in"
> +source "fs/live/Config.in"

  Was there a problem with sorting this alphabetically?

>
>   endmenu
> diff --git a/fs/live/Config.in b/fs/live/Config.in
> new file mode 100644
> index 0000000..a79f1dc
> --- /dev/null
> +++ b/fs/live/Config.in
> @@ -0,0 +1,14 @@
> +config BR2_TARGET_ROOTFS_LIVE
> +	bool "live root filesystem"
> +	help
> +	  Create a live image of the root filesystem that is directly
> +	  usable as over NFS or chroot.

  usable over NFS or chroot.

> +
> +config BR2_TARGET_ROOTFS_LIVE_DEST
> +	string "live image location"
> +	depends on BR2_TARGET_ROOTFS_LIVE
> +	default "$(BINARIES_DIR)/live"
> +	help
> +	  The directory where the image should be stored.
> +	  This directory will be emptied and recreated

  How about:

The directory where the live root filesystem will be
installed. Any changes to this tree are lost.

> +

  Redundant empty line.

> diff --git a/fs/live/live.mk b/fs/live/live.mk
> new file mode 100644
> index 0000000..52f7444
> --- /dev/null
> +++ b/fs/live/live.mk
> @@ -0,0 +1,19 @@
> +#############################################################
> +#
> +# Build the live root filesystem directory
> +#
> +#############################################################
> +
> +
> +define ROOTFS_LIVE_CMD
> +	sudo rsync -a --no-o --no-g --delete-during $(TARGET_DIR)/ \
> +		$(BR2_TARGET_ROOTFS_LIVE_DEST)/
> +endef
> +
> +define ROOTFS_LIVE_INIT
> +  if [ -z $(shell which sudo) ] ; then echo "sudo seems to not be installed on the host system" ; false ; fi
> +endef
> +
> +ROOTFS_LIVE_PRE_GEN_HOOKS += ROOTFS_LIVE_INIT

  Since it's already in dependencies.sh, this piece is redundant.

> +
> +$(eval $(call ROOTFS_TARGET,live))
> diff --git a/support/dependencies/dependencies.sh b/support/dependencies/dependencies.sh
> index 7a02512..ebaabbb 100755
> --- a/support/dependencies/dependencies.sh
> +++ b/support/dependencies/dependencies.sh
> @@ -158,6 +158,7 @@ if grep ^BR2_TOOLCHAIN_BUILDROOT=y $CONFIG_FILE>  /dev/null&&  \
>          exit 1 ;
>      fi
>   fi
> +
>   if grep -q ^BR2_PACKAGE_CLASSPATH=y $CONFIG_FILE ; then
>       for prog in javac jar; do
>   	if ! which $prog>  /dev/null ; then
> @@ -166,3 +167,10 @@ if grep -q ^BR2_PACKAGE_CLASSPATH=y $CONFIG_FILE ; then
>   	fi
>       done
>   fi
> +
> +if grep ^BR2_TARGET_ROOTFS_LIVE=y $CONFIG_FILE>  /dev/null ; then

  We use 'grep -q' rather than redirecting to /dev/null.

> +   if ! which sudo>  /dev/null ; then
> +       /bin/echo -e "\nYou need sudo installed on your build machine to build a live filesystem\n"
> +       exit 1 ;
> +   fi
> +fi

  Maybe, in addition to the mere existence of sudo, you should also
check if it the user is actually in sudoers.  'sudo -l rsync' should
do the trick.

  Regards,
  Arnout
-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Jeremy Rosen]

As usual, sniped what I agree with and v4 is on the way

> > diff --git a/fs/Config.in b/fs/Config.in
> > index da4c5ff..664d2f6 100644
> > --- a/fs/Config.in
> > +++ b/fs/Config.in
> > @@ -11,5 +11,6 @@ source "fs/romfs/Config.in"
> >   source "fs/squashfs/Config.in"
> >   source "fs/tar/Config.in"
> >   source "fs/ubifs/Config.in"
> > +source "fs/live/Config.in"
> 
>   Was there a problem with sorting this alphabetically?

just a mixup between my "sort alphabetically" and my "live" patches... fixing that

> > +
> > +if grep ^BR2_TARGET_ROOTFS_LIVE=y $CONFIG_FILE>  /dev/null ; then
> 
>   We use 'grep -q' rather than redirecting to /dev/null.
> 

I copied how it was done for locale that redirect to /dev/null, thus my mistake

I'll fix that in v4, but not the locale ones (to avoid mixup in my patch)



Thx for the comments, i'll wait a little to see if more comments come and then i'll send a V4

I also need to see what to do  with "make distclean" when the live filesystem is in output/

currently there are all sorts of permission-denied errors, and i'm not sure what to do here...

* I could use the config options to run "sudo rm" but i'm not sure we want to read the config option in distclean
* I could just leave it as is and just say the user is smart enough to run "sudo make distclean" in that case
* I could blindly run "sudo rm" in output but that sounds a bit dangerous


I tend to think the second option is the best, but i'd gladly get feedback on that one

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Jeremy Rosen]

Hello everybody

there is a little bit more work to do on this patch (mainly updating a few more places in the documentation) but i'd like to have a go/no-go before I do that...

Peter, could I have a final call on this ?

    Regards

    J?r?my Rosen

fight key loggers : write some perl using vim

----- Mail original -----
> De: "J?r?my Rosen" <jeremy.rosen@openwide.fr>
> ?: buildroot at busybox.net
> Cc: "J?r?my Rosen" <jeremy.rosen@openwide.fr>
> Envoy?: Jeudi 6 D?cembre 2012 14:36:07
> Objet: [PATCH] [RFC] new target: live filesystem
> 
> add a new target to deploy a live filesystem to be used with NFS or
> as a chroot
> 
> Signed-off-by: J?r?my Rosen <jeremy.rosen@openwide.fr>
> ---
> v2 : implement Arnoult's suggestion, update manual entry
> v3 : improve documentation for the chroot case, more suggestions by
> Arnoult
> ---
>  docs/manual/beyond-buildroot.txt     |   20 ++++++++------------
>  fs/Config.in                         |    1 +
>  fs/live/Config.in                    |   14 ++++++++++++++
>  fs/live/live.mk                      |   19 +++++++++++++++++++
>  support/dependencies/dependencies.sh |    8 ++++++++
>  5 files changed, 50 insertions(+), 12 deletions(-)
>  create mode 100644 fs/live/Config.in
>  create mode 100644 fs/live/live.mk
> 
> diff --git a/docs/manual/beyond-buildroot.txt
> b/docs/manual/beyond-buildroot.txt
> index a87b584..17ccc1a 100644
> --- a/docs/manual/beyond-buildroot.txt
> +++ b/docs/manual/beyond-buildroot.txt
> @@ -9,19 +9,15 @@ Boot the generated images
>  NFS boot
>  ~~~~~~~~
>  
> -To achieve NFS-boot, enable _tar root filesystem_ in the _Filesystem
> -images_ menu.
> +To achieve NFS-boot, enable _live root filesystem_ in the
> _Filesystem
> +images_ menu and select a _live image location_ to choose where the
> live
> +filesystem will be deployed. you can use _$(BINARIES_DIR)_ to easily
> +build in +/path/to/output_dir/images+
>  
> -After a complete build, just run the following commands to setup the
> -NFS-root directory:
> +You will be asked for a password during the build. This is needed to
> create
> +device entries in the target filesystem
>  
> --------------------
> -sudo tar -xavf /path/to/output_dir/rootfs.tar -C
> /path/to/nfs_root_dir
> --------------------
> -
> -Remember to add this path to +/etc/exports+.
> -
> -Then, you can execute a NFS-boot from your target.
> +You will need to add the _live image location_  to +/etc/exports+.
>  
>  Chroot
>  ------
> @@ -29,7 +25,7 @@ Chroot
>  If you want to chroot in a generated image, then there are few thing
>  you should be aware of:
>  
> -* you should setup the new root from the _tar root filesystem_
> image;
> +* you should use the _live root filesystem_ image;
>  
>  * either the selected target architecture is compatible with your
>  host
>    machine, or you should use some +qemu-*+ binary and correctly set
>    it
> diff --git a/fs/Config.in b/fs/Config.in
> index da4c5ff..664d2f6 100644
> --- a/fs/Config.in
> +++ b/fs/Config.in
> @@ -11,5 +11,6 @@ source "fs/romfs/Config.in"
>  source "fs/squashfs/Config.in"
>  source "fs/tar/Config.in"
>  source "fs/ubifs/Config.in"
> +source "fs/live/Config.in"
>  
>  endmenu
> diff --git a/fs/live/Config.in b/fs/live/Config.in
> new file mode 100644
> index 0000000..a79f1dc
> --- /dev/null
> +++ b/fs/live/Config.in
> @@ -0,0 +1,14 @@
> +config BR2_TARGET_ROOTFS_LIVE
> +	bool "live root filesystem"
> +	help
> +	  Create a live image of the root filesystem that is directly
> +	  usable as over NFS or chroot.
> +
> +config BR2_TARGET_ROOTFS_LIVE_DEST
> +	string "live image location"
> +	depends on BR2_TARGET_ROOTFS_LIVE
> +	default "$(BINARIES_DIR)/live"
> +	help
> +	  The directory where the image should be stored.
> +	  This directory will be emptied and recreated
> +
> diff --git a/fs/live/live.mk b/fs/live/live.mk
> new file mode 100644
> index 0000000..52f7444
> --- /dev/null
> +++ b/fs/live/live.mk
> @@ -0,0 +1,19 @@
> +#############################################################
> +#
> +# Build the live root filesystem directory
> +#
> +#############################################################
> +
> +
> +define ROOTFS_LIVE_CMD
> +	sudo rsync -a --no-o --no-g --delete-during $(TARGET_DIR)/ \
> +		$(BR2_TARGET_ROOTFS_LIVE_DEST)/
> +endef
> +
> +define ROOTFS_LIVE_INIT
> +  if [ -z $(shell which sudo) ] ; then echo "sudo seems to not be
> installed on the host system" ; false ; fi
> +endef
> +
> +ROOTFS_LIVE_PRE_GEN_HOOKS += ROOTFS_LIVE_INIT
> +
> +$(eval $(call ROOTFS_TARGET,live))
> diff --git a/support/dependencies/dependencies.sh
> b/support/dependencies/dependencies.sh
> index 7a02512..ebaabbb 100755
> --- a/support/dependencies/dependencies.sh
> +++ b/support/dependencies/dependencies.sh
> @@ -158,6 +158,7 @@ if grep ^BR2_TOOLCHAIN_BUILDROOT=y $CONFIG_FILE >
> /dev/null && \
>         exit 1 ;
>     fi
>  fi
> +
>  if grep -q ^BR2_PACKAGE_CLASSPATH=y $CONFIG_FILE ; then
>      for prog in javac jar; do
>  	if ! which $prog > /dev/null ; then
> @@ -166,3 +167,10 @@ if grep -q ^BR2_PACKAGE_CLASSPATH=y $CONFIG_FILE
> ; then
>  	fi
>      done
>  fi
> +
> +if grep ^BR2_TARGET_ROOTFS_LIVE=y $CONFIG_FILE > /dev/null ; then
> +   if ! which sudo > /dev/null ; then
> +       /bin/echo -e "\nYou need sudo installed on your build machine
> to build a live filesystem\n"
> +       exit 1 ;
> +   fi
> +fi
> --
> 1.7.10.4
> 
> 

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Richard Genoud]

2012/12/17 Jeremy Rosen <jeremy.rosen@openwide.fr>:
> Hello everybody
>
> there is a little bit more work to do on this patch (mainly updating a few more places in the documentation) but i'd like to have a go/no-go before I do that...

I was looking for that kind of feature, I hope it will be accepted.
For now, I use the output/target tree and those parameters in the
/etc/exports file:
*(rw,all_squash,no_subtree_check,anonuid=my_uid,anongid=my_gid)

But that means that all the FS is owned by my user.

Regards,
Richard.

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Samuel Martin]

Hi Richard,

2012/12/17 Richard Genoud <richard.genoud@gmail.com>:
> 2012/12/17 Jeremy Rosen <jeremy.rosen@openwide.fr>:
>> Hello everybody
>>
>> there is a little bit more work to do on this patch (mainly updating a few more places in the documentation) but i'd like to have a go/no-go before I do that...
>
> I was looking for that kind of feature, I hope it will be accepted.
> For now, I use the output/target tree and those parameters in the
> /etc/exports file:
> *(rw,all_squash,no_subtree_check,anonuid=my_uid,anongid=my_gid)
>
> But that means that all the FS is owned by my user.

There are plenty of good reasons to not use output/target as root for
a chroot or a NFS root, as explained here:
http://buildroot.org/downloads/manual/manual.html#faq-why-not-use-target-as-chroot

And here, you'll find the right way to do this (so far):
http://buildroot.org/downloads/manual/manual.html#_nfs_boot


Regards,

-- 
Sam

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Richard Genoud]

2012/12/17 Samuel Martin <s.martin49@gmail.com>:
> Hi Richard,
> There are plenty of good reasons to not use output/target as root for
> a chroot or a NFS root, as explained here:
> http://buildroot.org/downloads/manual/manual.html#faq-why-not-use-target-as-chroot
>
> And here, you'll find the right way to do this (so far):
> http://buildroot.org/downloads/manual/manual.html#_nfs_boot
>
Yes, I know...
It's just more convenient (fast) for me like that:
When I recompile a small software (with make mysoft-reconfigure), I'd
like it to be fast, and de-tar-ing the whole RFS takes some time.
But, it's true that it's not the right way to do it.
(I also always forgot to sudo when I want to copy something to the
nfsroot, so, like that, the tree is mine and there's no cp problem)

Thanks for the links though !

Regards,
Richard.

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Arnout Vandecappelle]

  More comments on top of what I sent before...

On 12/06/12 14:36, J?r?my Rosen wrote:
> add a new target to deploy a live filesystem to be used with NFS or as a chroot
>
> Signed-off-by: J?r?my Rosen<jeremy.rosen@openwide.fr>

Tested-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

  I used this patch (after some more adaptation) to rsync the rootfs to a 
running target over ssh. This is ideal for development. It's 
unfortunately a bit hackish because ssh tries to use /root/.ssh/... under 
fakeroot.

[snip]
> ---
> v2 : implement Arnoult's suggestion, update manual entry
> v3 : improve documentation for the chroot case, more suggestions by Arnoult
> ---
>   docs/manual/beyond-buildroot.txt     |   20 ++++++++------------
>   fs/Config.in                         |    1 +
>   fs/live/Config.in                    |   14 ++++++++++++++
>   fs/live/live.mk                      |   19 +++++++++++++++++++
>   support/dependencies/dependencies.sh |    8 ++++++++
>   5 files changed, 50 insertions(+), 12 deletions(-)
>   create mode 100644 fs/live/Config.in
>   create mode 100644 fs/live/live.mk
>
> diff --git a/docs/manual/beyond-buildroot.txt b/docs/manual/beyond-buildroot.txt
> index a87b584..17ccc1a 100644
> --- a/docs/manual/beyond-buildroot.txt
> +++ b/docs/manual/beyond-buildroot.txt
> @@ -9,19 +9,15 @@ Boot the generated images
>   NFS boot
>   ~~~~~~~~
>
> -To achieve NFS-boot, enable _tar root filesystem_ in the _Filesystem
> -images_ menu.
> +To achieve NFS-boot, enable _live root filesystem_ in the _Filesystem
> +images_ menu and select a _live image location_ to choose where the live
> +filesystem will be deployed. you can use _$(BINARIES_DIR)_ to easily

  Trailing whitespace.


> +build in +/path/to/output_dir/images+
>
> -After a complete build, just run the following commands to setup the
> -NFS-root directory:
> +You will be asked for a password during the build. This is needed to create
> +device entries in the target filesystem

  Note exactly correct.  Something like:

Since the NFS root must be installed as the real root user, you need to 
have sudo access. You will be asked for a password during the build.

>
> --------------------
> -sudo tar -xavf /path/to/output_dir/rootfs.tar -C /path/to/nfs_root_dir
> --------------------
> -
> -Remember to add this path to +/etc/exports+.
> -
> -Then, you can execute a NFS-boot from your target.
> +You will need to add the _live image location_  to +/etc/exports+.
>

[snip]

> +define ROOTFS_LIVE_INIT
> +  if [ -z $(shell which sudo) ] ; then echo "sudo seems to not be installed on the host system" ; false ; fi

  Trailing whitespace.

> +endef
> +
> +ROOTFS_LIVE_PRE_GEN_HOOKS += ROOTFS_LIVE_INIT
> +
> +$(eval $(call ROOTFS_TARGET,live))
[snip]


-- 
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 +32-16-286540
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7CB5 E4CC 6C2E EFD4 6E3D A754 F963 ECAB 2450 2F1F

[Buildroot] [PATCH] [RFC] new target: live filesystem

[Jeremy Rosen]

unfortunately I had the final word and this patch won't be included in master...

I'd still be interested in your changes if they make sense to add in my patch,

I might post that patch one last time if you are interested though...

    Regards

    J?r?my Rosen

fight key loggers : write some perl using vim