All of lore.kernel.org
 help / color / mirror / Atom feed
From: <Jack.Fewx@dell.com>
To: <yocto@yoctoproject.org>
Subject: [pseudo] Pseudo 1.8+ xattr sqlite corruption
Date: Mon, 20 Aug 2018 18:45:09 +0000	[thread overview]
Message-ID: <6a084eda5fcb4423a647bb998471e26d@AUSX13MPC104.AMER.DELL.COM> (raw)

We are encountering a build problem after migrating to Poky 2.3 and Pseudo 1.8.1, and need help to resolve this.
It is hampering our development efforts, forcing us to rebuild images frequently.

Background:
Our build applies SELinux file contexts, during build time since our rootfs is read-only
In Poky 2.0, using Pseudo 1.6.2 this works perfectly 100% of the time

Problem:
Since the upgrade to 2.3 there is a 33%+ chance that the SELinux context labels will be corrupt at the end of the build.
The chance is random.  Cleaning and Rebuilding a bad image target results in success/failure of equal likelihood. We can go days without an error, or like this weekend all 12 builds failed!

Failure mode:
We have learned to identify the failure and mark builds bad based on the contents of the Pseudo SQLite database generated by the image build.

A good build will have unique Inode to xattr references in the "xattrs" table.  We prove pass/fail by doing a query of All entries and unique entries and verify the counts match.
Example of a good result, sorted by "ino":
Id		dev		ino		name			value
"1"		"64773"	"251402120"	"security.selinux"	system_u:object_r:root_t
"10012"	"64773"	"251402121"	"security.selinux"	system_u:object_r:var_t
"7293"		"64773"	"251402124"	"security.selinux"	system_u:object_r:lib_t
"19"		"64773"	"251402133"	"security.selinux"	system_u:object_r:var_run_t

On a bad build, there will be numerous duplicates in this table.  Why this causes the failure I do not know, but this is just what we found is indicative to failure without flashing the image on something.
Example of a bad result, again sorted by "ino":

Id		dev	ino		name			value
"10067"	"45"	"2293256211"	"security.selinux"	system_u:object_r:usr_t
"31918"	"45"	"2293256211"	"security.selinux"	system_u:object_r:usr_t
"59307"	"45"	"2293256211"	"security.selinux"	system_u:object_r:usr_t
"61317"	"45"	"2293256211"	"security.selinux"	system_u:object_r:usr_t
"61737"	"45"	"2293256211"	"security.selinux"	system_u:object_r:usr_t
"61793"	"45"	"2293256211"	"security.selinux"	system_u:object_r:usr_t
"11849"	"45"	"2293250079"	"security.selinux"	system_u:object_r:var_spool_t
"66928"	"45"	"2293250079"	"security.selinux"	system_u:object_r:var_spool_t
"66948"	"45"	"2293250079"	"security.selinux"	system_u:object_r:var_spool_t

Any help would be greatly appreciated.

Jack Fewx
Software Senior Principal Engineer
Dell EMC | Server and Infrastructure Systems
jack_fewx@dell.com



             reply	other threads:[~2018-08-20 18:46 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-08-20 18:45 Jack.Fewx [this message]
2018-08-22  8:41 ` [pseudo] Pseudo 1.8+ xattr sqlite corruption Alexander Kanavin
2018-08-22 13:58   ` Jack.Fewx
2018-08-22 14:41     ` Joshua Watt
2018-08-22 14:54       ` Jack.Fewx
2018-08-22 15:09         ` Richard Purdie
2018-08-22 15:32           ` Jack.Fewx
2018-09-18 20:26           ` Jack.Fewx
2018-09-18 21:09             ` Seebs
2018-09-18 21:16               ` Joshua Watt
2018-09-18 21:20                 ` Seebs
2018-09-19 11:33                   ` Burton, Ross
2018-09-19 14:39                     ` Seebs
2018-09-19 16:25                       ` Jack.Fewx
2018-09-20 19:16                     ` Seebs
2018-09-20 20:41                       ` Jack.Fewx
2018-09-20 20:46                         ` Seebs
2018-09-20 20:50                         ` Seebs
2018-09-21 12:50                           ` Burton, Ross
2018-09-23 13:23                             ` Martin Jansa
2018-08-22 16:41         ` Seebs
2018-08-22 14:44     ` Alexander Kanavin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6a084eda5fcb4423a647bb998471e26d@AUSX13MPC104.AMER.DELL.COM \
    --to=jack.fewx@dell.com \
    --cc=yocto@yoctoproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.