Cleanly moving files from one package to another

Cleanly moving files from one package to another

[Bryan Evenson]

All,

I'm on poky/dizzy-1.7.3 and I'm using opkg for a package manager.  I have a few script files that are part of one package that really belong with another.  So I updated both recipes, removed the file from one recipe and added it to the other recipe.  I built everything, made the opkg feed repo and tried to upgrade with "opkg update; opkg upgrade" to see how clean the upgrade was.  For each script file I moved from one package to the other I get an error message of the form:

* check_data_file_clashes: Package A wants to install file /path/to/script/file
        But that file is already provided by package  * B

I know I can do "opkg upgrade --force-overwrite", but I'd like to make recipe changes so that it upgrades cleanly.  All I've done in the recipes is remove the script files from the FILES variable and the do_install step of one recipe and added the script to the FILES variable and do_install step of the other recipe.  I know update-alternatives can be used if more than one package provides the same binary, but is that necessary in this case when transferring ownership to another package?  Is there anything else I can do in the recipes to prevent the opkg install errors?

Thanks,
Bryan

Re: Cleanly moving files from one package to another

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1566 bytes --]

On Wed, Dec 09, 2015 at 08:19:40PM +0000, Bryan Evenson wrote:
> All,
> 
> I'm on poky/dizzy-1.7.3 and I'm using opkg for a package manager.  I have a few script files that are part of one package that really belong with another.  So I updated both recipes, removed the file from one recipe and added it to the other recipe.  I built everything, made the opkg feed repo and tried to upgrade with "opkg update; opkg upgrade" to see how clean the upgrade was.  For each script file I moved from one package to the other I get an error message of the form:
> 
> * check_data_file_clashes: Package A wants to install file /path/to/script/file
>         But that file is already provided by package  * B
> 
> I know I can do "opkg upgrade --force-overwrite", but I'd like to make recipe changes so that it upgrades cleanly.  All I've done in the recipes is remove the script files from the FILES variable and the do_install step of one recipe and added the script to the FILES variable and do_install step of the other recipe.  I know update-alternatives can be used if more than one package provides the same binary, but is that necessary in this case when transferring ownership to another package?  Is there anything else I can do in the recipes to prevent the opkg install errors?

Have you tried to set
RREPLACES_<pkg-new> = "<pkg-old>"
?

If the old package should be removed completely during opkg upgrade,
then you'll need to set whole combo, that is
RREPLACES,RCONFLICTS,RPROVIDES.

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: Cleanly moving files from one package to another

[Bryan Evenson]

Martin,

> -----Original Message-----
> From: Martin Jansa [mailto:martin.jansa@gmail.com]
> Sent: Thursday, December 10, 2015 4:20 AM
> To: Bryan Evenson <bevenson@melinkcorp.com>
> Cc: yocto@yoctoproject.org
> Subject: Re: [yocto] Cleanly moving files from one package to another
> 
> On Wed, Dec 09, 2015 at 08:19:40PM +0000, Bryan Evenson wrote:
> > All,
> >
> > I'm on poky/dizzy-1.7.3 and I'm using opkg for a package manager.  I have a
> few script files that are part of one package that really belong with another.
> So I updated both recipes, removed the file from one recipe and added it to
> the other recipe.  I built everything, made the opkg feed repo and tried to
> upgrade with "opkg update; opkg upgrade" to see how clean the upgrade
> was.  For each script file I moved from one package to the other I get an error
> message of the form:
> >
> > * check_data_file_clashes: Package A wants to install file
> /path/to/script/file
> >         But that file is already provided by package  * B
> >
> > I know I can do "opkg upgrade --force-overwrite", but I'd like to make
> recipe changes so that it upgrades cleanly.  All I've done in the recipes is
> remove the script files from the FILES variable and the do_install step of one
> recipe and added the script to the FILES variable and do_install step of the
> other recipe.  I know update-alternatives can be used if more than one
> package provides the same binary, but is that necessary in this case when
> transferring ownership to another package?  Is there anything else I can do in
> the recipes to prevent the opkg install errors?
> 
> Have you tried to set
> RREPLACES_<pkg-new> = "<pkg-old>"
> ?

I haven't tried that because I'm not replacing a package.  Both packages still exist on the system and are both needed, it's just that ownership of one of the files is being transferred from one package to another.  Would I still use RREPLACES even if I want to keep both packages on the system?

Specifically in this case, I have some script files for scheduled tasks that were part of one of my self-made packages.  However, they really belong with cronie since this is what is calling the scripts.  So I removed the script files from my self-made package and added them to cronie.  My package has other files still associated with it (that do belong with it) installed on my system.  So I want to keep both packages on my system, but I just want to change ownership of these script files from my package to cronie.  Is RREPLACES the tool for this job?

Thanks,
Bryan

> 
> If the old package should be removed completely during opkg upgrade, then
> you'll need to set whole combo, that is RREPLACES,RCONFLICTS,RPROVIDES.
> 
> --
> Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

Re: Cleanly moving files from one package to another

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3093 bytes --]

On Thu, Dec 10, 2015 at 02:15:05PM +0000, Bryan Evenson wrote:
> Martin,
> 
> > -----Original Message-----
> > From: Martin Jansa [mailto:martin.jansa@gmail.com]
> > Sent: Thursday, December 10, 2015 4:20 AM
> > To: Bryan Evenson <bevenson@melinkcorp.com>
> > Cc: yocto@yoctoproject.org
> > Subject: Re: [yocto] Cleanly moving files from one package to another
> > 
> > On Wed, Dec 09, 2015 at 08:19:40PM +0000, Bryan Evenson wrote:
> > > All,
> > >
> > > I'm on poky/dizzy-1.7.3 and I'm using opkg for a package manager.  I have a
> > few script files that are part of one package that really belong with another.
> > So I updated both recipes, removed the file from one recipe and added it to
> > the other recipe.  I built everything, made the opkg feed repo and tried to
> > upgrade with "opkg update; opkg upgrade" to see how clean the upgrade
> > was.  For each script file I moved from one package to the other I get an error
> > message of the form:
> > >
> > > * check_data_file_clashes: Package A wants to install file
> > /path/to/script/file
> > >         But that file is already provided by package  * B
> > >
> > > I know I can do "opkg upgrade --force-overwrite", but I'd like to make
> > recipe changes so that it upgrades cleanly.  All I've done in the recipes is
> > remove the script files from the FILES variable and the do_install step of one
> > recipe and added the script to the FILES variable and do_install step of the
> > other recipe.  I know update-alternatives can be used if more than one
> > package provides the same binary, but is that necessary in this case when
> > transferring ownership to another package?  Is there anything else I can do in
> > the recipes to prevent the opkg install errors?
> > 
> > Have you tried to set
> > RREPLACES_<pkg-new> = "<pkg-old>"
> > ?
> 
> I haven't tried that because I'm not replacing a package.  Both packages still exist on the system and are both needed, it's just that ownership of one of the files is being transferred from one package to another.  Would I still use RREPLACES even if I want to keep both packages on the system?
> 
> Specifically in this case, I have some script files for scheduled tasks that were part of one of my self-made packages.  However, they really belong with cronie since this is what is calling the scripts.  So I removed the script files from my self-made package and added them to cronie.  My package has other files still associated with it (that do belong with it) installed on my system.  So I want to keep both packages on my system, but I just want to change ownership of these script files from my package to cronie.  Is RREPLACES the tool for this job?

> Is RREPLACES the tool for this job?

Yes, see
https://www.debian.org/doc/debian-policy/ch-relationships.html#s-replaces

opkg should work the same.

> > If the old package should be removed completely during opkg upgrade, then
> > you'll need to set whole combo, that is RREPLACES,RCONFLICTS,RPROVIDES.

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: Cleanly moving files from one package to another

[Bryan Evenson]

Martin,

> -----Original Message-----
> From: Martin Jansa [mailto:martin.jansa@gmail.com]
> Sent: Thursday, December 10, 2015 9:39 AM
> To: Bryan Evenson <bevenson@melinkcorp.com>
> Cc: yocto@yoctoproject.org
> Subject: Re: [yocto] Cleanly moving files from one package to another
> 
> On Thu, Dec 10, 2015 at 02:15:05PM +0000, Bryan Evenson wrote:
> > Martin,
> >
> > > -----Original Message-----
> > > From: Martin Jansa [mailto:martin.jansa@gmail.com]
> > > Sent: Thursday, December 10, 2015 4:20 AM
> > > To: Bryan Evenson <bevenson@melinkcorp.com>
> > > Cc: yocto@yoctoproject.org
> > > Subject: Re: [yocto] Cleanly moving files from one package to
> > > another
> > >
> > > On Wed, Dec 09, 2015 at 08:19:40PM +0000, Bryan Evenson wrote:
> > > > All,
> > > >
> > > > I'm on poky/dizzy-1.7.3 and I'm using opkg for a package manager.
> > > > I have a
> > > few script files that are part of one package that really belong with
> another.
> > > So I updated both recipes, removed the file from one recipe and
> > > added it to the other recipe.  I built everything, made the opkg
> > > feed repo and tried to upgrade with "opkg update; opkg upgrade" to
> > > see how clean the upgrade was.  For each script file I moved from
> > > one package to the other I get an error message of the form:
> > > >
> > > > * check_data_file_clashes: Package A wants to install file
> > > /path/to/script/file
> > > >         But that file is already provided by package  * B
> > > >
> > > > I know I can do "opkg upgrade --force-overwrite", but I'd like to
> > > > make
> > > recipe changes so that it upgrades cleanly.  All I've done in the
> > > recipes is remove the script files from the FILES variable and the
> > > do_install step of one recipe and added the script to the FILES
> > > variable and do_install step of the other recipe.  I know
> > > update-alternatives can be used if more than one package provides
> > > the same binary, but is that necessary in this case when
> > > transferring ownership to another package?  Is there anything else I can
> do in the recipes to prevent the opkg install errors?
> > >
> > > Have you tried to set
> > > RREPLACES_<pkg-new> = "<pkg-old>"
> > > ?
> >
> > I haven't tried that because I'm not replacing a package.  Both packages still
> exist on the system and are both needed, it's just that ownership of one of
> the files is being transferred from one package to another.  Would I still use
> RREPLACES even if I want to keep both packages on the system?
> >
> > Specifically in this case, I have some script files for scheduled tasks that
> were part of one of my self-made packages.  However, they really belong
> with cronie since this is what is calling the scripts.  So I removed the script files
> from my self-made package and added them to cronie.  My package has
> other files still associated with it (that do belong with it) installed on my
> system.  So I want to keep both packages on my system, but I just want to
> change ownership of these script files from my package to cronie.  Is
> RREPLACES the tool for this job?
> 
> > Is RREPLACES the tool for this job?
> 
> Yes, see
> https://www.debian.org/doc/debian-policy/ch-relationships.html#s-
> replaces
> 
> opkg should work the same.

Now I'm having something happen that I haven't seen before with the PR service.  I updated my .bbappend to add the RREPLACES as you suggested, but bitbake didn't rebuild the package.  I tried doing bitbake -c clean and bitbake -c cleansstate and both times the built package had the same full version number as it did before I made the RREPLACES change.  I opened the ipk with Archive Manager and looked at DEBIAN/control, and there was no "Depends: " line in the control file.  Then I noticed that this .bbappend had "inherit pkgconfig" and it isn't needed in the .bbappend.  So while I was at it I deleted the line "inherit pkgconfig" and tried to build the package again (without issuing a bitbake -c clean first).  This time it built the package and the PR service incremented the version.  I opened the resulting ipk and looked at DEBIAN/control again, and this time there was a "Depends:" line in the control file.

Is changing RREPLACES not being seen as a metadata change by bitbake?  Is this a bug or is there something else I'm missing?

Thanks,
Bryan

> 
> > > If the old package should be removed completely during opkg upgrade,
> > > then you'll need to set whole combo, that is
> RREPLACES,RCONFLICTS,RPROVIDES.
> 
> --
> Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

Re: Cleanly moving files from one package to another

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 4955 bytes --]

On Thu, Dec 10, 2015 at 04:20:45PM +0000, Bryan Evenson wrote:
> Martin,
> 
> > -----Original Message-----
> > From: Martin Jansa [mailto:martin.jansa@gmail.com]
> > Sent: Thursday, December 10, 2015 9:39 AM
> > To: Bryan Evenson <bevenson@melinkcorp.com>
> > Cc: yocto@yoctoproject.org
> > Subject: Re: [yocto] Cleanly moving files from one package to another
> > 
> > On Thu, Dec 10, 2015 at 02:15:05PM +0000, Bryan Evenson wrote:
> > > Martin,
> > >
> > > > -----Original Message-----
> > > > From: Martin Jansa [mailto:martin.jansa@gmail.com]
> > > > Sent: Thursday, December 10, 2015 4:20 AM
> > > > To: Bryan Evenson <bevenson@melinkcorp.com>
> > > > Cc: yocto@yoctoproject.org
> > > > Subject: Re: [yocto] Cleanly moving files from one package to
> > > > another
> > > >
> > > > On Wed, Dec 09, 2015 at 08:19:40PM +0000, Bryan Evenson wrote:
> > > > > All,
> > > > >
> > > > > I'm on poky/dizzy-1.7.3 and I'm using opkg for a package manager.
> > > > > I have a
> > > > few script files that are part of one package that really belong with
> > another.
> > > > So I updated both recipes, removed the file from one recipe and
> > > > added it to the other recipe.  I built everything, made the opkg
> > > > feed repo and tried to upgrade with "opkg update; opkg upgrade" to
> > > > see how clean the upgrade was.  For each script file I moved from
> > > > one package to the other I get an error message of the form:
> > > > >
> > > > > * check_data_file_clashes: Package A wants to install file
> > > > /path/to/script/file
> > > > >         But that file is already provided by package  * B
> > > > >
> > > > > I know I can do "opkg upgrade --force-overwrite", but I'd like to
> > > > > make
> > > > recipe changes so that it upgrades cleanly.  All I've done in the
> > > > recipes is remove the script files from the FILES variable and the
> > > > do_install step of one recipe and added the script to the FILES
> > > > variable and do_install step of the other recipe.  I know
> > > > update-alternatives can be used if more than one package provides
> > > > the same binary, but is that necessary in this case when
> > > > transferring ownership to another package?  Is there anything else I can
> > do in the recipes to prevent the opkg install errors?
> > > >
> > > > Have you tried to set
> > > > RREPLACES_<pkg-new> = "<pkg-old>"
> > > > ?
> > >
> > > I haven't tried that because I'm not replacing a package.  Both packages still
> > exist on the system and are both needed, it's just that ownership of one of
> > the files is being transferred from one package to another.  Would I still use
> > RREPLACES even if I want to keep both packages on the system?
> > >
> > > Specifically in this case, I have some script files for scheduled tasks that
> > were part of one of my self-made packages.  However, they really belong
> > with cronie since this is what is calling the scripts.  So I removed the script files
> > from my self-made package and added them to cronie.  My package has
> > other files still associated with it (that do belong with it) installed on my
> > system.  So I want to keep both packages on my system, but I just want to
> > change ownership of these script files from my package to cronie.  Is
> > RREPLACES the tool for this job?
> > 
> > > Is RREPLACES the tool for this job?
> > 
> > Yes, see
> > https://www.debian.org/doc/debian-policy/ch-relationships.html#s-
> > replaces
> > 
> > opkg should work the same.
> 
> Now I'm having something happen that I haven't seen before with the PR service.  I updated my .bbappend to add the RREPLACES as you suggested, but bitbake didn't rebuild the package.  I tried doing bitbake -c clean and bitbake -c cleansstate and both times the built package had the same full version number as it did before I made the RREPLACES change.  I opened the ipk with Archive Manager and looked at DEBIAN/control, and there was no "Depends: " line in the control file.  Then I noticed that this .bbappend had "inherit pkgconfig" and it isn't needed in the .bbappend.  So while I was at it I deleted the line "inherit pkgconfig" and tried to build the package again (without issuing a bitbake -c clean first).  This time it built the package and the PR service incremented the version.  I opened the resulting ipk and looked at DEBIAN/control again, and this time there was a "Depends:" line in the control file.
> 
> Is changing RREPLACES not being seen as a metadata change by bitbake?  Is this a bug or is there something else I'm missing?

It should invalidate sstate signature for do_package* task
(maybe just do_package_write_ipk), if it doesn't then it's a bug.

Best way to test this is to use bitbake -S package, before and after
such change and bitbake-diffsigs on generated .sigdata files to see if
it was noticed by bitbake.

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 188 bytes --]

Re: Cleanly moving files from one package to another

[Bryan Evenson]

Martin,

> -----Original Message-----
> From: Martin Jansa [mailto:martin.jansa@gmail.com]
> Sent: Thursday, December 10, 2015 12:03 PM
> To: Bryan Evenson <bevenson@melinkcorp.com>
> Cc: yocto@yoctoproject.org
> Subject: Re: [yocto] Cleanly moving files from one package to another
> 
> On Thu, Dec 10, 2015 at 04:20:45PM +0000, Bryan Evenson wrote:
> > Martin,
> >
> > > -----Original Message-----
> > > From: Martin Jansa [mailto:martin.jansa@gmail.com]
> > > Sent: Thursday, December 10, 2015 9:39 AM
> > > To: Bryan Evenson <bevenson@melinkcorp.com>
> > > Cc: yocto@yoctoproject.org
> > > Subject: Re: [yocto] Cleanly moving files from one package to another
> > >
> > > On Thu, Dec 10, 2015 at 02:15:05PM +0000, Bryan Evenson wrote:
> > > > Martin,
> > > >
> > > > > -----Original Message-----
> > > > > From: Martin Jansa [mailto:martin.jansa@gmail.com]
> > > > > Sent: Thursday, December 10, 2015 4:20 AM
> > > > > To: Bryan Evenson <bevenson@melinkcorp.com>
> > > > > Cc: yocto@yoctoproject.org
> > > > > Subject: Re: [yocto] Cleanly moving files from one package to
> > > > > another
> > > > >
> > > > > On Wed, Dec 09, 2015 at 08:19:40PM +0000, Bryan Evenson wrote:
> > > > > > All,
> > > > > >
> > > > > > I'm on poky/dizzy-1.7.3 and I'm using opkg for a package manager.
> > > > > > I have a
> > > > > few script files that are part of one package that really belong with
> > > another.
> > > > > So I updated both recipes, removed the file from one recipe and
> > > > > added it to the other recipe.  I built everything, made the opkg
> > > > > feed repo and tried to upgrade with "opkg update; opkg upgrade" to
> > > > > see how clean the upgrade was.  For each script file I moved from
> > > > > one package to the other I get an error message of the form:
> > > > > >
> > > > > > * check_data_file_clashes: Package A wants to install file
> > > > > /path/to/script/file
> > > > > >         But that file is already provided by package  * B
> > > > > >
> > > > > > I know I can do "opkg upgrade --force-overwrite", but I'd like to
> > > > > > make
> > > > > recipe changes so that it upgrades cleanly.  All I've done in the
> > > > > recipes is remove the script files from the FILES variable and the
> > > > > do_install step of one recipe and added the script to the FILES
> > > > > variable and do_install step of the other recipe.  I know
> > > > > update-alternatives can be used if more than one package provides
> > > > > the same binary, but is that necessary in this case when
> > > > > transferring ownership to another package?  Is there anything else I
> can
> > > do in the recipes to prevent the opkg install errors?
> > > > >
> > > > > Have you tried to set
> > > > > RREPLACES_<pkg-new> = "<pkg-old>"
> > > > > ?
> > > >
> > > > I haven't tried that because I'm not replacing a package.  Both packages
> still
> > > exist on the system and are both needed, it's just that ownership of one
> of
> > > the files is being transferred from one package to another.  Would I still
> use
> > > RREPLACES even if I want to keep both packages on the system?
> > > >
> > > > Specifically in this case, I have some script files for scheduled tasks that
> > > were part of one of my self-made packages.  However, they really belong
> > > with cronie since this is what is calling the scripts.  So I removed the script
> files
> > > from my self-made package and added them to cronie.  My package has
> > > other files still associated with it (that do belong with it) installed on my
> > > system.  So I want to keep both packages on my system, but I just want
> to
> > > change ownership of these script files from my package to cronie.  Is
> > > RREPLACES the tool for this job?
> > >
> > > > Is RREPLACES the tool for this job?
> > >
> > > Yes, see
> > > https://www.debian.org/doc/debian-policy/ch-relationships.html#s-
> > > replaces
> > >
> > > opkg should work the same.
> >
> > Now I'm having something happen that I haven't seen before with the PR
> service.  I updated my .bbappend to add the RREPLACES as you suggested,
> but bitbake didn't rebuild the package.  I tried doing bitbake -c clean and
> bitbake -c cleansstate and both times the built package had the same full
> version number as it did before I made the RREPLACES change.  I opened the
> ipk with Archive Manager and looked at DEBIAN/control, and there was no
> "Depends: " line in the control file.  Then I noticed that this .bbappend had
> "inherit pkgconfig" and it isn't needed in the .bbappend.  So while I was at it I
> deleted the line "inherit pkgconfig" and tried to build the package again
> (without issuing a bitbake -c clean first).  This time it built the package and the
> PR service incremented the version.  I opened the resulting ipk and looked at
> DEBIAN/control again, and this time there was a "Depends:" line in the
> control file.
> >
> > Is changing RREPLACES not being seen as a metadata change by bitbake?  Is
> this a bug or is there something else I'm missing?
> 
> It should invalidate sstate signature for do_package* task
> (maybe just do_package_write_ipk), if it doesn't then it's a bug.
> 
> Best way to test this is to use bitbake -S package, before and after
> such change and bitbake-diffsigs on generated .sigdata files to see if
> it was noticed by bitbake.

I did this test and the sstate signatures never changed no matter what I did to RREPLACES.  I just filed a bug about the problem here: https://bugzilla.yoctoproject.org/show_bug.cgi?id=8788.

Regards,
Bryan 

> 
> --
> Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com