Continuing discussion on Handling persistent files (/etc/group)during BMC updates

Continuing discussion on Handling persistent files (/etc/group)during BMC updates

[Raviteja Bailapudi]

Hi all,

We are looking for thoughts and perspectives on the way the persistent 
files are managed across BMC code updates.

The problem is regarding the code update where, as a part of code update 
we don't touch any of the persistent files like /etc/group or /etc/password.

what if the new BMC image has an application which requires some changes 
to be present in these persistent files ?

For Example:
Let's say the BMC image which is used for code update has a new 
feature(ex:avahi) and it requires avahi user and
the group to be present in the /etc/passwd and /etc/group to even kick 
start it's daemon.

How to update such persistent files where customer data might exist?

One of the quick solution i could think of is:
- We can identify what is missing and then write a service override file 
to make the necessary changes in the persistent file.

But i am still looking community thoughts to fix this issue? There was 
already a mail sent regarding same, please find it below:
https://lists.ozlabs.org/pipermail/openbmc/2018-March/011162.html

Thanks!
Raviteja

Re: Continuing discussion on Handling persistent files (/etc/group)during BMC updates

[Alexander A. Filippov]

On Fri, Aug 09, 2019 at 12:19:32PM +0530, Raviteja Bailapudi wrote:
> Hi all,
> 
> We are looking for thoughts and perspectives on the way the persistent files
> are managed across BMC code updates.
> 
> The problem is regarding the code update where, as a part of code update we
> don't touch any of the persistent files like /etc/group or /etc/password.
> 
> what if the new BMC image has an application which requires some changes to
> be present in these persistent files ?
> 
> For Example:
> Let's say the BMC image which is used for code update has a new
> feature(ex:avahi) and it requires avahi user and
> the group to be present in the /etc/passwd and /etc/group to even kick start
> it's daemon.

We had faced with the similar issue several months ago and don't find good
solution yet.
https://github.com/openbmc/openbmc/issues/3468

> 
> How to update such persistent files where customer data might exist?
> 
> One of the quick solution i could think of is:
> - We can identify what is missing and then write a service override file to
> make the necessary changes in the persistent file.
> 
> But i am still looking community thoughts to fix this issue? There was
> already a mail sent regarding same, please find it below:
> https://lists.ozlabs.org/pipermail/openbmc/2018-March/011162.html
> 
> Thanks!
> Raviteja
> 

Re: Continuing discussion on Handling persistent files (/etc/group)during BMC updates

[Ed Tanous]

On 8/9/19 12:33 AM, Alexander A. Filippov wrote:
> On Fri, Aug 09, 2019 at 12:19:32PM +0530, Raviteja Bailapudi wrote:
>> Hi all,
>>
>> We are looking for thoughts and perspectives on the way the persistent files
>> are managed across BMC code updates.
>>
>> The problem is regarding the code update where, as a part of code update we
>> don't touch any of the persistent files like /etc/group or /etc/passwor
>>
>> what if the new BMC image has an application which requires some changes to
>> be present in these persistent files ?
>>
>> For Example:
>> Let's say the BMC image which is used for code update has a new
>> feature(ex:avahi) and it requires avahi user and
>> the group to be present in the /etc/passwd and /etc/group to even kick start
>> it's daemon.

One way that I've seen work quite well in other production firmwares is
to treat it like you would a database schema, and simply generate the
"new" files via patching the existing file in a script that executes on
first new firmware startup.  In general, when you're messing with these
files, you're either appending a new line, which would be done with a
script like:
if grep -Fxqv "avahi:x:41:" /etc/group
then
cat "avahi:x:41:" >> /etc/group
fi

or you're modifying an existing line, which would be done with a sed
replace.

In the above, we're checking to see if the file has been changed
appropriately before actually appending the line, to make sure that the
update only gets applied once.  I've seen other cases where it makes
more sense to simply leave a separate marker file, with an arbitrary
revision number so that the BMC knows which updates it needs to apply.
It really all depends on what you're attempting to add and whether or
not it could conflict.

If you're careful about how you craft the matching regexes, and keep the
scripts around forever, you can generally transition files like that
from one firmware update to another without any ill effects.

The only case where it doesn't work is in the downgrade case, which gets
really tricky really fast, and why most vendors give a "your mileage may
vary" warning on downgrades, and recommend defaulting the configuration
after a downgrade.

Re: Continuing discussion on Handling persistent files (/etc/group)during BMC updates

[Raviteja Bailapudi]

On 09-08-2019 21:13, Ed Tanous wrote:
> On 8/9/19 12:33 AM, Alexander A. Filippov wrote:
>> On Fri, Aug 09, 2019 at 12:19:32PM +0530, Raviteja Bailapudi wrote:
>>> Hi all,
>>>
>>> We are looking for thoughts and perspectives on the way the persistent files
>>> are managed across BMC code updates.
>>>
>>> The problem is regarding the code update where, as a part of code update we
>>> don't touch any of the persistent files like /etc/group or /etc/passwor
>>>
>>> what if the new BMC image has an application which requires some changes to
>>> be present in these persistent files ?
>>>
>>> For Example:
>>> Let's say the BMC image which is used for code update has a new
>>> feature(ex:avahi) and it requires avahi user and
>>> the group to be present in the /etc/passwd and /etc/group to even kick start
>>> it's daemon.
> One way that I've seen work quite well in other production firmwares is
> to treat it like you would a database schema, and simply generate the
> "new" files via patching the existing file in a script that executes on
> first new firmware startup.  In general, when you're messing with these
> files, you're either appending a new line, which would be done with a
> script like:
> if grep -Fxqv "avahi:x:41:" /etc/group
> then
> cat "avahi:x:41:" >> /etc/group
> fi

We implemented the same behavior through systemd override, where the 
service can check the

prerequisite for the service to start and add the group/password if it 
is not there.

Here is my commit
https://gerrit.openbmc-project.xyz/c/openbmc/meta-phosphor/+/24271/

> or you're modifying an existing line, which would be done with a sed
> replace.
>
> In the above, we're checking to see if the file has been changed
> appropriately before actually appending the line, to make sure that the
> update only gets applied once.  I've seen other cases where it makes
> more sense to simply leave a separate marker file, with an arbitrary
> revision number so that the BMC knows which updates it needs to apply.
> It really all depends on what you're attempting to add and whether or
> not it could conflict.
>
> If you're careful about how you craft the matching regexes, and keep the
> scripts around forever, you can generally transition files like that
> from one firmware update to another without any ill effects.
>
> The only case where it doesn't work is in the downgrade case, which gets
> really tricky really fast, and why most vendors give a "your mileage may
> vary" warning on downgrades, and recommend defaulting the configuration
> after a downgrade.
in the above mentioned commit, there this service file always get 
restarted and make sure that the prerequisite met.