[PATCH 0/1] Additional license wrangling functionality

[PATCH 0/1] Additional license wrangling functionality

[Flanagan, Elizabeth]

Resubmitting this without manifest functionality. It's a longer discussion that
should be had. This has some basic functionality additions, bug fixes
and corrects
non-standard indentation.

The following changes since commit 819f18f8bc000f13b644edc194d2a12b4ea5fecf:

  Move architecture specific TARGET_OS mangling into tune files
(2011-07-25 14:36:30 +0100)

are available in the git repository at:
  git://git.pokylinux.org/poky-contrib eflanagan/lic_sans_manifest
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=eflanagan/license_sans_manifest

Beth Flanagan (1):
  license.bbclass: Splitting out licenses

 meta/classes/license.bbclass |  140 ++++++++++++++++++++++++------------------
 1 files changed, 81 insertions(+), 59 deletions(-)

[PATCH 0/1] Additional license wrangling functionality

[Flanagan, Elizabeth]

Requested by a member of the community, this fixes a few issues with license
wrangling. Specifically,

- licenses are now wrangled into:
  ${DEPLOY_DIR}/licenses/${IMAGE_NAME}/datetimestamp
  This keeps multiple runs of a build from polluting the licenses of a
previous build
- remove all wrangling of -native and -cross licenses:
  if it's not on the image, then having the license isn't useful and
in fact, is harmful
- create a manifest
  on BuildCompleted we take all the wrangled licenses and give a
manifest with all the
  *known* common licenses. If license.bbclass doesn't know the common, we don't
  include it.
- Since we have + licenses, we can now deal with them correctly.

The following changes since commit 2c79c9eb7ef8ef0aef8c3096c3c4387e28e56ea2:

  pulseaudio: add 0.9.23 (2011-07-07 13:45:32 +0100)

are available in the git repository at:
  git://git.pokylinux.org/poky-contrib eflanagan/license_pn
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=eflanagan/license_pn

Beth Flanagan (1):
  license.bbclass: License Manifests and more.

 meta/classes/license.bbclass |  179 ++++++++++++++++++++++++++++--------------
 1 files changed, 120 insertions(+), 59 deletions(-)

Re: [PATCH 0/1] Additional license wrangling functionality

[Cliff Brake]

[-- Attachment #1: Type: text/plain, Size: 4727 bytes --]

Many thanks for implementing these improvements!

The manifest file for core-image-minimal is attached.

A few thoughts after testing this ...

I like the way the license information is being cleaned up in the recipes.

In the past, I have collected licenses by using the following
procedure directly on a device:

cd /usr/lib/opkg/info
grep License *

In the past (OE Classic build from a year ago or so), the Control file
in a packaged looked like:

Package: libc6
Version: 2.9-r37.3.6
Description: GNU C Library
Section: libs
Priority: required
Maintainer: Angstrom Developers <angstrom-distro-devel@linuxtogo.org>
License: LGPL
Architecture: armv5te
OE: glibc
Homepage: http://www.gnu.org/software/libc/libc.html
Depends: update-rc.d, libcidn1
Source: ...

Now, in OE core they look like this:

Package: libc6
Version: 2.13-r6+svnr14157
Description: Embedded GLIBC (GNU C Library)
 Embedded GLIBC (EGLIBC) is a variant of the GNU C Library (GLIBC) that is
 designed to work well on embedded systems. EGLIBC strives to be source
 and binary compatible with GLIBC. EGLIBC's goals include reduced
 footprint, configurable components, better support for cross-compilation
 and cross-testing.
Section: libs
Priority: optional
Maintainer: OE-Core Developers <openembedded-core@lists.openembedded.org>
Architecture: i586
OE: eglibc
Homepage: http://www.eglibc.org/home
Provides: glibc
Source:  ...

Why is the License field no longer included in the control file?

One of the biggest issues with the current license mechanism is the
license directory seems to only populated at recipe build time, not at
image rootfs generation.  Therefore, you have to do a clean build to
get a list of licenses.  It seems like it would be more optimal if the
License information was stored in the packages (as it was in the
past), and then extracted from each package during the rootfs phase.
The license manifest could then be created similar to the way the
testlab stuff is done in OE:

http://cgit.openembedded.org/cgit.cgi/meta-openembedded/tree/meta-oe/classes/testlab.bbclass

As a side note, I find the testlab information very useful, so perhaps
it should be considered for including in oe-core.

I'm also not convinced that license information for every package is
being shown in the manifest.  For example, during the rootfs phase, I
see things like:

Installing kernel-module-cfbcopyarea
Installing kernel-module-uvesafb

But I don't see anything offhand in the manifests file that would
correlate to the above kernel modules.  So in summary, there are two
possible improvements that I think would be useful:

1) make sure license information for _every_ package that gets
installed at the rootfs phase is included in the report.  If there is
not License information, perhaps this should be a fatal error.

2) be able to generate the report at rootfs time, not recipe build time.

3) put the License field back into the package Control file, so any
tool that generates an image from packages can use this.

Thanks,
Cliff

-- 
=================
http://bec-systems.com


On Mon, Jul 11, 2011 at 7:11 PM, Flanagan, Elizabeth
<elizabeth.flanagan@intel.com> wrote:
> Requested by a member of the community, this fixes a few issues with license
> wrangling. Specifically,
>
> - licenses are now wrangled into:
>  ${DEPLOY_DIR}/licenses/${IMAGE_NAME}/datetimestamp
>  This keeps multiple runs of a build from polluting the licenses of a
> previous build
> - remove all wrangling of -native and -cross licenses:
>  if it's not on the image, then having the license isn't useful and
> in fact, is harmful
> - create a manifest
>  on BuildCompleted we take all the wrangled licenses and give a
> manifest with all the
>  *known* common licenses. If license.bbclass doesn't know the common, we don't
>  include it.
> - Since we have + licenses, we can now deal with them correctly.
>
> The following changes since commit 2c79c9eb7ef8ef0aef8c3096c3c4387e28e56ea2:
>
>  pulseaudio: add 0.9.23 (2011-07-07 13:45:32 +0100)
>
> are available in the git repository at:
>  git://git.pokylinux.org/poky-contrib eflanagan/license_pn
>  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=eflanagan/license_pn
>
> Beth Flanagan (1):
>  license.bbclass: License Manifests and more.
>
>  meta/classes/license.bbclass |  179 ++++++++++++++++++++++++++++--------------
>  1 files changed, 120 insertions(+), 59 deletions(-)
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
>

[-- Attachment #2: 201107121817_license_manifest --]
[-- Type: application/octet-stream, Size: 4197 bytes --]

Package Name: python
----------------------------

Package Name: libgpg-error
----------------------------
GPL-2+

Package Name: util-macros
----------------------------
MIT-style
MIT

Package Name: acl
----------------------------
GPL-2+

Package Name: netbase
----------------------------
GPL-2

Package Name: linux-libc-headers
----------------------------
GPL-2

Package Name: libxau
----------------------------
MIT-style

Package Name: expat
----------------------------
MIT

Package Name: zlib
----------------------------
Zlib

Package Name: dbus
----------------------------
GPL-2+
AFL-2

Package Name: usbutils
----------------------------
GPL-2+

Package Name: libx11-trim
----------------------------
MIT-style
BSD
MIT

Package Name: perl
----------------------------

Package Name: run-postinsts
----------------------------
MIT

Package Name: ncurses
----------------------------
MIT

Package Name: sqlite3
----------------------------
PD

Package Name: readline
----------------------------
GPL-3+

Package Name: update-modules
----------------------------
GPL-2

Package Name: gdbm
----------------------------
GPL-2+

Package Name: xcb-proto
----------------------------
MIT

Package Name: busybox
----------------------------
GPL-2

Package Name: libice
----------------------------
MIT-style

Package Name: libpthread-stubs
----------------------------
MIT

Package Name: libsm
----------------------------
MIT-style

Package Name: e2fsprogs
----------------------------
GPL-2
LGPL-2
BSD
MIT

Package Name: base-files
----------------------------
GPL-2

Package Name: udev
----------------------------
GPL-2+

Package Name: opkg
----------------------------
GPL-2+

Package Name: opkg-config-base
----------------------------
MIT

Package Name: initscripts
----------------------------
GPL-2

Package Name: elfutils
----------------------------
GPL-2

Package Name: pciutils
----------------------------
GPL-2+

Package Name: kbproto
----------------------------
MIT-style

Package Name: tinylogin
----------------------------
GPL-2

Package Name: attr
----------------------------
GPL-2+

Package Name: modutils-initscripts
----------------------------
PD

Package Name: bzip2
----------------------------

Package Name: glib-2.0
----------------------------
BSD
PD

Package Name: libxcb
----------------------------
MIT

Package Name: xproto
----------------------------
MIT-style
MIT

Package Name: libgcc
----------------------------
GPL-3.0-with-GCC-exception
GPL-3

Package Name: zip
----------------------------

Package Name: base-passwd
----------------------------
GPL-2+

Package Name: inputproto
----------------------------
MIT-style
MIT

Package Name: common-licenses
----------------------------
MIT-style

Package Name: xextproto
----------------------------
MIT-style
MIT

Package Name: bash
----------------------------
GPL-3+

Package Name: gcc-runtime
----------------------------
GPL-3.0-with-GCC-exception
GPL-3

Package Name: xf86bigfontproto
----------------------------
MIT

Package Name: gettext
----------------------------
GPL-3

Package Name: libusb-compat
----------------------------

Package Name: xtrans
----------------------------
MIT-style
MIT

Package Name: linux-yocto
----------------------------
GPL-1

Package Name: core-image-minimal
----------------------------
MIT

Package Name: udev-extraconf
----------------------------
GPL-2

Package Name: libusb1
----------------------------

Package Name: eglibc
----------------------------
LGPL-2

Package Name: libtool
----------------------------
GPL-2
LGPL-2.1

Package Name: db
----------------------------
Sleepycat

Package Name: sysvinit
----------------------------
GPL-2+

Package Name: util-linux
----------------------------
GPL-2+
BSD

Package Name: task-core-boot
----------------------------
MIT

Package Name: module-init-tools
----------------------------
GPL-2+

Package Name: openssl
----------------------------
Openssl

Package Name: update-rc.d
----------------------------
GPL-2+

Package Name: sysvinit-inittab
----------------------------
GPL-2

Package Name: v86d
----------------------------
GPL-2

Package Name: eglibc-initial
----------------------------
LGPL-2

Re: [PATCH 0/1] Additional license wrangling functionality

[Phil Blundell]

On Wed, 2011-07-13 at 10:06 -0400, Cliff Brake wrote:
> Why is the License field no longer included in the control file?

Looks like it was only added to OE.dev relatively recently (in 2008)
which was after oe-core was forked.  The commit in question from oe.dev
seems to be e35d1ffad1553f259b084578992f15d10f590f98 and I expect it
would apply to oe-core without too much trouble.

p.

Re: [PATCH 0/1] Additional license wrangling functionality

[Koen Kooi]

Op 13 jul 2011, om 16:06 heeft Cliff Brake het volgende geschreven:

> Many thanks for implementing these improvements!
> 
> The manifest file for core-image-minimal is attached.
> 
> A few thoughts after testing this ...
> 
> I like the way the license information is being cleaned up in the recipes.
> 
> In the past, I have collected licenses by using the following
> procedure directly on a device:
> 
> cd /usr/lib/opkg/info
> grep License *
> 
> In the past (OE Classic build from a year ago or so), the Control file
> in a packaged looked like:
> 
> Package: libc6
> Version: 2.9-r37.3.6
> Description: GNU C Library
> Section: libs
> Priority: required
> Maintainer: Angstrom Developers <angstrom-distro-devel@linuxtogo.org>
> License: LGPL
> Architecture: armv5te
> OE: glibc
> Homepage: http://www.gnu.org/software/libc/libc.html
> Depends: update-rc.d, libcidn1
> Source: ...
> 
> Now, in OE core they look like this:
> 
> Package: libc6
> Version: 2.13-r6+svnr14157
> Description: Embedded GLIBC (GNU C Library)
> Embedded GLIBC (EGLIBC) is a variant of the GNU C Library (GLIBC) that is
> designed to work well on embedded systems. EGLIBC strives to be source
> and binary compatible with GLIBC. EGLIBC's goals include reduced
> footprint, configurable components, better support for cross-compilation
> and cross-testing.
> Section: libs
> Priority: optional
> Maintainer: OE-Core Developers <openembedded-core@lists.openembedded.org>
> Architecture: i586
> OE: eglibc
> Homepage: http://www.eglibc.org/home
> Provides: glibc
> Source:  ...
> 
> Why is the License field no longer included in the control file?
> 
> One of the biggest issues with the current license mechanism is the
> license directory seems to only populated at recipe build time, not at
> image rootfs generation.  Therefore, you have to do a clean build to
> get a list of licenses.  It seems like it would be more optimal if the
> License information was stored in the packages (as it was in the
> past), and then extracted from each package during the rootfs phase.
> The license manifest could then be created similar to the way the
> testlab stuff is done in OE:
> 
> http://cgit.openembedded.org/cgit.cgi/meta-openembedded/tree/meta-oe/classes/testlab.bbclass

I ported bits of that to narcissus to generate a manifest, have a look at:

http://git.angstrom-distribution.org/cgi-bin/cgit.cgi/narcissus/tree/scripts/extract-metadata.sh
http://git.angstrom-distribution.org/cgi-bin/cgit.cgi/narcissus/tree/scripts/assemble-image.sh#n275

Narcissus will generate a html page that looks like the manifest we use internally in TI to get things approved by the opensource reviewboard, so if things look strange, that's why :)

The angstrom autobuilders log a subset of testlab into a git repo: http://git.angstrom-distribution.org/cgi-bin/cgit.cgi/testlab/log/?h=yocto

regards,

Koen