[PATCH 0/4] Support persistent /var/log

[PATCH 0/4] Support persistent /var/log

[Chen Qi]

The following changes since commit d3fcfe346b3472170641d9194243c4bbe06b8585:

  systemd: fix firmware path to match linux-firmware (2015-03-23 11:27:13 +0000)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib ChenQi/persistent-var-log
  http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=ChenQi/persistent-var-log

Chen Qi (4):
  bitbake.conf: add VOLATILE_LOG_DIR variable
  base-files: respect VOLATILE_LOG_DIR
  initscripts: support persistent /var/log
  package.bbclass: support persistent /var/log

 meta/classes/package.bbclass                       |  2 +-
 meta/conf/bitbake.conf                             |  4 ++
 meta/files/fs-perms-persistent-log.txt             | 66 ++++++++++++++++++++++
 meta/recipes-core/base-files/base-files_3.0.14.bb  |  4 +-
 .../initscripts/initscripts-1.0/volatiles          |  1 -
 meta/recipes-core/initscripts/initscripts_1.0.bb   |  3 +
 6 files changed, 76 insertions(+), 4 deletions(-)
 create mode 100644 meta/files/fs-perms-persistent-log.txt

-- 
1.9.1

[PATCH 1/4] bitbake.conf: add VOLATILE_LOG_DIR variable

[Chen Qi]

The default value is "log" which results in the same /var/log to be
a link pointing to /var/volatile/log.

The user could override this value to "" which causes /var/log to be
a directory on persistent storage.

[YOCTO #6132]

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/conf/bitbake.conf | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index f3586da..19b904c 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -77,6 +77,10 @@ USRBINPATH_class-nativesdk = "/usr/bin"
 # Root home directory
 ROOT_HOME ??= "/home/root"
 
+# Volatile log direcotry
+# If it's set to "", then the log directory on the final image is on persistent storage.
+VOLATILE_LOG_DIR ?= "log"
+
 ##################################################################
 # Architecture-dependent build variables.
 ##################################################################
-- 
1.9.1

[PATCH 2/4] base-files: respect VOLATILE_LOG_DIR

[Chen Qi]

Respect VOLATILE_LOG_DIR variable. In this way, if the user overrides
this variable to be "", /var/log on the final image would reside on
persistent storage.

[YOCTO #6132]

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/recipes-core/base-files/base-files_3.0.14.bb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/base-files/base-files_3.0.14.bb b/meta/recipes-core/base-files/base-files_3.0.14.bb
index f2d254e..f519bb2 100644
--- a/meta/recipes-core/base-files/base-files_3.0.14.bb
+++ b/meta/recipes-core/base-files/base-files_3.0.14.bb
@@ -42,7 +42,7 @@ dirs755 = "/bin /boot /dev ${sysconfdir} ${sysconfdir}/default \
            ${localstatedir}/backups ${localstatedir}/lib \
            /sys ${localstatedir}/lib/misc ${localstatedir}/spool \
            ${localstatedir}/volatile \
-           ${localstatedir}/volatile/log \
+           ${localstatedir}/${@['', 'volatile/'][d.getVar('VOLATILE_LOG_DIR', True) == "log"]}log \
            /home ${prefix}/src ${localstatedir}/local \
            /media"
 
@@ -53,7 +53,7 @@ dirs755-lsb = "/srv  \
                ${prefix}/lib/locale"
 dirs2775-lsb = "/var/mail"
 
-volatiles = "log tmp"
+volatiles = "tmp ${VOLATILE_LOG_DIR}"
 conffiles = "${sysconfdir}/debian_version ${sysconfdir}/host.conf \
              ${sysconfdir}/inputrc ${sysconfdir}/issue /${sysconfdir}/issue.net \
              ${sysconfdir}/nsswitch.conf ${sysconfdir}/profile \
-- 
1.9.1

[PATCH 3/4] initscripts: support persistent /var/log

[Chen Qi]

Respect VOLATILE_VAR_LOG variable so that if it's set to "", we could
have persistent /var/log on the final image.

[YOCTO #6132]

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/recipes-core/initscripts/initscripts-1.0/volatiles | 1 -
 meta/recipes-core/initscripts/initscripts_1.0.bb        | 3 +++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-core/initscripts/initscripts-1.0/volatiles b/meta/recipes-core/initscripts/initscripts-1.0/volatiles
index 297245d..6cccab7 100644
--- a/meta/recipes-core/initscripts/initscripts-1.0/volatiles
+++ b/meta/recipes-core/initscripts/initscripts-1.0/volatiles
@@ -25,7 +25,6 @@ d root root 1777 /run/lock none
 d root root 0755 /var/volatile/log none
 d root root 1777 /var/volatile/tmp none
 l root root 1777 /var/lock /run/lock
-l root root 0755 /var/log /var/volatile/log
 l root root 0755 /var/run /run
 l root root 1777 /var/tmp /var/volatile/tmp
 l root root 1777 /tmp /var/tmp
diff --git a/meta/recipes-core/initscripts/initscripts_1.0.bb b/meta/recipes-core/initscripts/initscripts_1.0.bb
index dfb75b2..14f4167 100644
--- a/meta/recipes-core/initscripts/initscripts_1.0.bb
+++ b/meta/recipes-core/initscripts/initscripts_1.0.bb
@@ -98,6 +98,9 @@ do_install () {
 	install -m 0755    ${WORKDIR}/read-only-rootfs-hook.sh ${D}${sysconfdir}/init.d
 	install -m 0755    ${WORKDIR}/save-rtc.sh	${D}${sysconfdir}/init.d
 	install -m 0644    ${WORKDIR}/volatiles		${D}${sysconfdir}/default/volatiles/00_core
+	if [ "${VOLATILE_LOG_DIR}" = "log" ]; then
+		echo "l root root 0755 /var/log /var/volatile/log" >> ${D}${sysconfdir}/default/volatiles/00_core
+	fi
 	install -m 0755    ${WORKDIR}/dmesg.sh		${D}${sysconfdir}/init.d
 	install -m 0644    ${WORKDIR}/logrotate-dmesg.conf ${D}${sysconfdir}/
 
-- 
1.9.1

[PATCH 4/4] package.bbclass: support persistent /var/log

[Chen Qi]

Add a new file, fs-perms-persistent-log.txt, which treats /var/log
as a directory instead of a link.
Modify package.bbclass to use this file if VOLATILE_LOG_DIR is set to "".

[YOCTO #6132]

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/classes/package.bbclass           |  2 +-
 meta/files/fs-perms-persistent-log.txt | 66 ++++++++++++++++++++++++++++++++++
 2 files changed, 67 insertions(+), 1 deletion(-)
 create mode 100644 meta/files/fs-perms-persistent-log.txt

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 9f64ed7..63a70b7 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -699,7 +699,7 @@ python fixup_perms () {
         bbpath = d.getVar('BBPATH', True)
         fs_perms_tables = d.getVar('FILESYSTEM_PERMS_TABLES', True)
         if not fs_perms_tables:
-            fs_perms_tables = 'files/fs-perms.txt'
+            fs_perms_tables = ['files/fs-perms-persistent-log.txt', 'files/fs-perms.txt'][d.getVar('VOLATILE_LOG_DIR', True) == 'log']
         for conf_file in fs_perms_tables.split():
             str += " %s" % bb.utils.which(bbpath, conf_file)
         return str
diff --git a/meta/files/fs-perms-persistent-log.txt b/meta/files/fs-perms-persistent-log.txt
new file mode 100644
index 0000000..fdeeb02
--- /dev/null
+++ b/meta/files/fs-perms-persistent-log.txt
@@ -0,0 +1,66 @@
+# This file contains a list of files and directories with known permissions.
+# It is used by the packaging class to ensure that the permissions, owners and
+# group of listed files and directories are in sync across the system.
+#
+# The format of this file 
+#
+#<path>	<mode>	<uid>	<gid>	<walk>	<fmode>	<fuid>	<fgid>
+#
+# or
+#
+#<path> link <target>
+#
+# <path>: directory path
+# <mode>: mode for directory
+# <uid>:  uid for directory
+# <gid>:  gid for directory
+# <walk>: recursively walk the directory?  true or false
+# <fmode>: if walking, new mode for files
+# <fuid>:  if walking, new uid for files
+# <fgid>:  if walking, new gid for files
+# <target>: turn the directory into a symlink point to target
+#
+# in mode, uid or gid, a "-" means don't change any existing values
+#
+# /usr/src		0755	root	root	false	-	-	-
+# /usr/share/man	0755	root	root	true	0644	root	root
+
+# Note: all standard config directories are automatically assigned "0755 root root false - - -"
+
+# Documentation should always be corrected
+${mandir}		0755	root	root	true	0644	root	root
+${infodir}		0755	root	root	true	0644	root	root
+${docdir}		0755	root	root	true	0644	root	root
+${datadir}/gtk-doc	0755	root	root	true	0644	root	root
+
+# Fixup locales
+${datadir}/locale	0755	root	root	true	0644	root	root
+
+# Cleanup headers
+${includedir}		0755	root	root	true	0644	root	root
+${oldincludedir}	0755	root	root	true	0644	root	root
+
+# Cleanup debug src
+/usr/src/debug		0755	root	root	true	-	root	root
+
+# Items from base-files
+# Links
+${localstatedir}/run	link	/run
+${localstatedir}/lock	link	/run/lock
+${localstatedir}/tmp	link	volatile/tmp
+
+/home				0755	root	root	false - - -
+/srv				0755	root	root	false - - -
+${prefix}/src			0755	root	root	false - - -
+${localstatedir}/local		0755	root	root	false - - -
+
+# Special permissions from base-files
+# Set 1777
+/tmp				01777	root	root	false - - -
+${localstatedir}/volatile/tmp	01777	root	root	false - - -
+
+# Set 755-lsb
+/srv				0755	root	root	false - - -
+
+# Set 2775-lsb
+/var/mail			02775	root	mail	false - - -
-- 
1.9.1

Re: [PATCH 0/4] Support persistent /var/log

[ChenQi]

ping

Any comment on this one?

//Chen Qi

On 03/24/2015 04:42 PM, Chen Qi wrote:
> The following changes since commit d3fcfe346b3472170641d9194243c4bbe06b8585:
>
>    systemd: fix firmware path to match linux-firmware (2015-03-23 11:27:13 +0000)
>
> are available in the git repository at:
>
>    git://git.openembedded.org/openembedded-core-contrib ChenQi/persistent-var-log
>    http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=ChenQi/persistent-var-log
>
> Chen Qi (4):
>    bitbake.conf: add VOLATILE_LOG_DIR variable
>    base-files: respect VOLATILE_LOG_DIR
>    initscripts: support persistent /var/log
>    package.bbclass: support persistent /var/log
>
>   meta/classes/package.bbclass                       |  2 +-
>   meta/conf/bitbake.conf                             |  4 ++
>   meta/files/fs-perms-persistent-log.txt             | 66 ++++++++++++++++++++++
>   meta/recipes-core/base-files/base-files_3.0.14.bb  |  4 +-
>   .../initscripts/initscripts-1.0/volatiles          |  1 -
>   meta/recipes-core/initscripts/initscripts_1.0.bb   |  3 +
>   6 files changed, 76 insertions(+), 4 deletions(-)
>   create mode 100644 meta/files/fs-perms-persistent-log.txt
>

Re: [PATCH 0/4] Support persistent /var/log

[ChenQi]

ping ....

*ANY* comment would be welcome, really ...

Thanks,
Chen Qi


On 03/24/2015 04:42 PM, Chen Qi wrote:
> The following changes since commit d3fcfe346b3472170641d9194243c4bbe06b8585:
>
>    systemd: fix firmware path to match linux-firmware (2015-03-23 11:27:13 +0000)
>
> are available in the git repository at:
>
>    git://git.openembedded.org/openembedded-core-contrib ChenQi/persistent-var-log
>    http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=ChenQi/persistent-var-log
>
> Chen Qi (4):
>    bitbake.conf: add VOLATILE_LOG_DIR variable
>    base-files: respect VOLATILE_LOG_DIR
>    initscripts: support persistent /var/log
>    package.bbclass: support persistent /var/log
>
>   meta/classes/package.bbclass                       |  2 +-
>   meta/conf/bitbake.conf                             |  4 ++
>   meta/files/fs-perms-persistent-log.txt             | 66 ++++++++++++++++++++++
>   meta/recipes-core/base-files/base-files_3.0.14.bb  |  4 +-
>   .../initscripts/initscripts-1.0/volatiles          |  1 -
>   meta/recipes-core/initscripts/initscripts_1.0.bb   |  3 +
>   6 files changed, 76 insertions(+), 4 deletions(-)
>   create mode 100644 meta/files/fs-perms-persistent-log.txt
>

Re: [PATCH 0/4] Support persistent /var/log

[Paul Eggleton]

Randy, Khem, Otavio - this area was the subject of a discussion involving you 
around the 1.8 release, if you could review this set and comment that would be 
great.

Thanks,
Paul

On Tuesday 26 May 2015 13:43:58 ChenQi wrote:
> ping ....
> 
> *ANY* comment would be welcome, really ...
> 
> Thanks,
> Chen Qi
> 
> On 03/24/2015 04:42 PM, Chen Qi wrote:
> > The following changes since commit 
d3fcfe346b3472170641d9194243c4bbe06b8585:
> >    systemd: fix firmware path to match linux-firmware (2015-03-23 11:27:13
> >    +0000)> 
> > are available in the git repository at:
> >    git://git.openembedded.org/openembedded-core-contrib
> >    ChenQi/persistent-var-log
> >    http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h
> >    =ChenQi/persistent-var-log> 
> > Chen Qi (4):
> >    bitbake.conf: add VOLATILE_LOG_DIR variable
> >    base-files: respect VOLATILE_LOG_DIR
> >    initscripts: support persistent /var/log
> >    package.bbclass: support persistent /var/log
> >   
> >   meta/classes/package.bbclass                       |  2 +-
> >   meta/conf/bitbake.conf                             |  4 ++
> >   meta/files/fs-perms-persistent-log.txt             | 66
> >   ++++++++++++++++++++++
> >   meta/recipes-core/base-files/base-files_3.0.14.bb  |  4 +-
> >   .../initscripts/initscripts-1.0/volatiles          |  1 -
> >   meta/recipes-core/initscripts/initscripts_1.0.bb   |  3 +
> >   6 files changed, 76 insertions(+), 4 deletions(-)
> >   create mode 100644 meta/files/fs-perms-persistent-log.txt

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: [PATCH 0/4] Support persistent /var/log

[Khem Raj]

On Tue, May 26, 2015 at 2:15 AM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> Randy, Khem, Otavio - this area was the subject of a discussion involving you
> around the 1.8 release, if you could review this set and comment that would be
> great.


Looked at it but it didnt sound like it considered read only rootfs case at all.

>
> Thanks,
> Paul
>
> On Tuesday 26 May 2015 13:43:58 ChenQi wrote:
>> ping ....
>>
>> *ANY* comment would be welcome, really ...
>>
>> Thanks,
>> Chen Qi
>>
>> On 03/24/2015 04:42 PM, Chen Qi wrote:
>> > The following changes since commit
> d3fcfe346b3472170641d9194243c4bbe06b8585:
>> >    systemd: fix firmware path to match linux-firmware (2015-03-23 11:27:13
>> >    +0000)>
>> > are available in the git repository at:
>> >    git://git.openembedded.org/openembedded-core-contrib
>> >    ChenQi/persistent-var-log
>> >    http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h
>> >    =ChenQi/persistent-var-log>
>> > Chen Qi (4):
>> >    bitbake.conf: add VOLATILE_LOG_DIR variable
>> >    base-files: respect VOLATILE_LOG_DIR
>> >    initscripts: support persistent /var/log
>> >    package.bbclass: support persistent /var/log
>> >
>> >   meta/classes/package.bbclass                       |  2 +-
>> >   meta/conf/bitbake.conf                             |  4 ++
>> >   meta/files/fs-perms-persistent-log.txt             | 66
>> >   ++++++++++++++++++++++
>> >   meta/recipes-core/base-files/base-files_3.0.14.bb  |  4 +-
>> >   .../initscripts/initscripts-1.0/volatiles          |  1 -
>> >   meta/recipes-core/initscripts/initscripts_1.0.bb   |  3 +
>> >   6 files changed, 76 insertions(+), 4 deletions(-)
>> >   create mode 100644 meta/files/fs-perms-persistent-log.txt
>
> --
>
> Paul Eggleton
> Intel Open Source Technology Centre

Re: [PATCH 0/4] Support persistent /var/log

[ChenQi]

On 05/28/2015 12:55 PM, Khem Raj wrote:
> On Tue, May 26, 2015 at 2:15 AM, Paul Eggleton
> <paul.eggleton@linux.intel.com> wrote:
>> Randy, Khem, Otavio - this area was the subject of a discussion involving you
>> around the 1.8 release, if you could review this set and comment that would be
>> great.
>
> Looked at it but it didnt sound like it considered read only rootfs case at all.


read-only rootfs and persistent /var/log are all custom configurations.
If a user wants a read-only rootfs but a persistent /var/log, he'll need 
to do more custom configurations like putting /var/ on a seperate 
partition or something else.

The purpose of this patch is to provide an easy way to user to keep 
their logs between reboots.

//Chen Qi

>> Thanks,
>> Paul
>>
>> On Tuesday 26 May 2015 13:43:58 ChenQi wrote:
>>> ping ....
>>>
>>> *ANY* comment would be welcome, really ...
>>>
>>> Thanks,
>>> Chen Qi
>>>
>>> On 03/24/2015 04:42 PM, Chen Qi wrote:
>>>> The following changes since commit
>> d3fcfe346b3472170641d9194243c4bbe06b8585:
>>>>     systemd: fix firmware path to match linux-firmware (2015-03-23 11:27:13
>>>>     +0000)>
>>>> are available in the git repository at:
>>>>     git://git.openembedded.org/openembedded-core-contrib
>>>>     ChenQi/persistent-var-log
>>>>     http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h
>>>>     =ChenQi/persistent-var-log>
>>>> Chen Qi (4):
>>>>     bitbake.conf: add VOLATILE_LOG_DIR variable
>>>>     base-files: respect VOLATILE_LOG_DIR
>>>>     initscripts: support persistent /var/log
>>>>     package.bbclass: support persistent /var/log
>>>>
>>>>    meta/classes/package.bbclass                       |  2 +-
>>>>    meta/conf/bitbake.conf                             |  4 ++
>>>>    meta/files/fs-perms-persistent-log.txt             | 66
>>>>    ++++++++++++++++++++++
>>>>    meta/recipes-core/base-files/base-files_3.0.14.bb  |  4 +-
>>>>    .../initscripts/initscripts-1.0/volatiles          |  1 -
>>>>    meta/recipes-core/initscripts/initscripts_1.0.bb   |  3 +
>>>>    6 files changed, 76 insertions(+), 4 deletions(-)
>>>>    create mode 100644 meta/files/fs-perms-persistent-log.txt
>> --
>>
>> Paul Eggleton
>> Intel Open Source Technology Centre
>

[PATCH 1/4] bitbake.conf: add VOLATILE_LOG_DIR variable

[Chen Qi]

The default value is "yes" which results in the /var/log being a link
pointing to /var/volatile/log which is on tmpfs.

Setting valid boolean false value ('no', 'n', 'false', 'f', '0') would make
/var/log to be a directory on persistent storage.

[YOCTO #6132]

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/conf/bitbake.conf | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 3110b9c..334ba23 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -85,6 +85,10 @@ USRBINPATH_class-nativesdk = "/usr/bin"
 # Root home directory
 ROOT_HOME ??= "/home/root"
 
+# If set to boolean true ('yes', 'y', 'true', 't', '1'), /var/log links to /var/volatile/log.
+# If set to boolean false ('no', 'n', 'false', 'f', '0'), /var/log is on persistent storage.
+VOLATILE_LOG_DIR ?= "yes"
+
 ##################################################################
 # Architecture-dependent build variables.
 ##################################################################
-- 
1.9.1

[PATCH 1/4] bitbake.conf: add VOLATILE_LOG_DIR variable

[Chen Qi]

The default value is "yes" which results in the /var/log being a link
pointing to /var/volatile/log which is on tmpfs.

The user could override this value to "no" which causes /var/log to be
a directory on persistent storage.

[YOCTO #6132]

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
---
 meta/conf/bitbake.conf | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 1472e8f..794f422 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -83,6 +83,9 @@ USRBINPATH_class-nativesdk = "/usr/bin"
 # Root home directory
 ROOT_HOME ??= "/home/root"
 
+# If set to "yes", /var/log links to /var/volatile/log; otherwise, /var/log is on persistent storage.
+VOLATILE_LOG_DIR ?= "yes"
+
 ##################################################################
 # Architecture-dependent build variables.
 ##################################################################
-- 
1.9.1