linux-media.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Philippe Ombredanne <pombredanne@nexb.com>
To: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Cc: "Takiguchi, Yasunari" <Yasunari.Takiguchi@sony.com>,
	"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"devicetree@vger.kernel.org" <devicetree@vger.kernel.org>,
	"linux-media@vger.kernel.org" <linux-media@vger.kernel.org>,
	"tbird20d@gmail.com" <tbird20d@gmail.com>,
	"frowand.list@gmail.com" <frowand.list@gmail.com>,
	"Yamamoto, Masayuki" <Masayuki.Yamamoto@sony.com>,
	"Nozawa, Hideki (STWN)" <Hideki.Nozawa@sony.com>,
	"Yonezawa, Kota" <Kota.Yonezawa@sony.com>,
	"Matsumoto, Toshihiko" <Toshihiko.Matsumoto@sony.com>,
	"Watanabe, Satoshi (SSS)" <Satoshi.C.Watanabe@sony.com>,
	"Bird, Timothy" <Tim.Bird@sony.com>
Subject: Re: [PATCH v4 00/12] [dt-bindings] [media] Add document file and driver for Sony CXD2880 DVB-T2/T tuner + demodulator
Date: Thu, 14 Dec 2017 15:25:13 +0100	[thread overview]
Message-ID: <CAOFm3uEYfMH8Zj8uEx-D9yYrTyDMTG_j02619esHu-j0brQKaA@mail.gmail.com> (raw)
In-Reply-To: <20171214085503.289f06f8@vento.lan>

Dear Mauro,

On Thu, Dec 14, 2017 at 11:55 AM, Mauro Carvalho Chehab
<mchehab@s-opensource.com> wrote:

> SPDX is a new requirement that started late on Kernel 4.14 development
> cycle (and whose initial changes were merged directly at Linus tree).
> Not all existing files have it yet, as identifying the right license
> on existing files is a complex task, but if you do a:
>
>         $ git grep SPDX $(find . -name Makefile) $(find . -name Kconfig)
>
> You'll see that lot of such files have it already.

FWIW, short of having SPDX tags, identifying the right license on
existing files is not a super complex task: if boils down to running
many diffs.

Take the ~60K files in kernel, and about 6K license and notices
reference texts. Then compute a pairwise diff of each of the 60K file
against the 6K reference texts. Repeat the pairwise diff a few more
times, say 10 times, as multiple licenses may appear in any given
kernel file. And keep the diffs that have the fewest
difference/highest similarity with the reference texts as the detected
license. Done!

The only complex thing is that if you have a fast diff that runs at
0.1 millisec end-to-end per diff, you still have 3.6B diffs to do and
this would take about 250 days on one thread. Even with a beefy 250
core CPU, that would still be a day (and quite few kilo watts) . So
the whole trick is to avoid doing a diffs if not really needed. This
is what I do in my scancode-toolkit (that I used/use to help Greg and
Thomas with kernel license scans). Net effect is that on a laptop on 8
threads it takes ~20 minutes to scan a whole kernel using this
diff-based approach and obtain a fairly accurate license detection.
-- 
Cordially
Philippe Ombredanne

  reply	other threads:[~2017-12-14 14:25 UTC|newest]

Thread overview: 43+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-10-13  5:46 [PATCH v4 00/12] [dt-bindings] [media] Add document file and driver for Sony CXD2880 DVB-T2/T tuner + demodulator Yasunari.Takiguchi
2017-10-13  5:58 ` [PATCH v4 01/12] [dt-bindings] [media] Add document file for CXD2880 SPI I/F Yasunari.Takiguchi
2017-10-13  5:59 ` [PATCH v4 02/12] [media] cxd2880-spi: Add support for CXD2880 SPI interface Yasunari.Takiguchi
2017-10-13  8:44   ` Honza Petrouš
2017-10-13  9:10     ` Takiguchi, Yasunari
2017-10-13 10:26       ` Honza Petrouš
2017-10-16  1:35         ` Takiguchi, Yasunari
2017-12-13 17:54   ` Mauro Carvalho Chehab
2017-12-20  0:54     ` Takiguchi, Yasunari
2017-10-13  6:02 ` [PATCH v4 03/12] [media] cxd2880: Add common files for the driver Yasunari.Takiguchi
2017-12-13 18:02   ` Mauro Carvalho Chehab
2017-10-13  6:05 ` [PATCH v4 04/12] [media] cxd2880: Add spi device IO routines Yasunari.Takiguchi
2017-10-13  6:07 ` [PATCH v4 05/12] [media] cxd2880: Add tuner part of the driver Yasunari.Takiguchi
2017-12-13 18:40   ` Mauro Carvalho Chehab
2017-12-20  1:01     ` Takiguchi, Yasunari
2017-10-13  6:08 ` [PATCH v4 06/12] [media] cxd2880: Add integration layer for " Yasunari.Takiguchi
2017-12-03 18:23   ` Sean Young
2017-12-13 19:13   ` Mauro Carvalho Chehab
2017-12-20  1:06     ` Takiguchi, Yasunari
2017-10-13  6:09 ` [PATCH v4 07/12] [media] cxd2880: Add top level of " Yasunari.Takiguchi
2017-12-03 22:59   ` Sean Young
2017-12-05 11:47     ` Takiguchi, Yasunari
2017-12-13 19:25   ` Mauro Carvalho Chehab
2017-12-20  1:18     ` Takiguchi, Yasunari
2017-10-13  6:10 ` [PATCH v4 08/12] [media] cxd2880: Add DVB-T control functions " Yasunari.Takiguchi
2017-10-13  6:11 ` [PATCH v4 09/12] [media] cxd2880: Add DVB-T monitor functions Yasunari.Takiguchi
2017-10-13  6:13 ` [PATCH v4 10/12] [media] cxd2880: Add DVB-T2 control functions for the driver Yasunari.Takiguchi
2017-10-13  6:15 ` [PATCH v4 11/12] [media] cxd2880: Add DVB-T2 monitor functions Yasunari.Takiguchi
2017-10-13  6:17 ` [PATCH v4 12/12] [media] cxd2880: Add all Makefile, Kconfig files and Update MAINTAINERS file for the driver Yasunari.Takiguchi
2017-11-16  1:07 ` [PATCH v4 00/12] [dt-bindings] [media] Add document file and driver for Sony CXD2880 DVB-T2/T tuner + demodulator Takiguchi, Yasunari
2017-11-22  4:17   ` Takiguchi, Yasunari
2017-11-29  7:38     ` Mauro Carvalho Chehab
2017-11-29  8:21       ` Takiguchi, Yasunari
2017-12-13 19:36 ` Mauro Carvalho Chehab
2017-12-14  9:59   ` Takiguchi, Yasunari
2017-12-14 10:55     ` Mauro Carvalho Chehab
2017-12-14 14:25       ` Philippe Ombredanne [this message]
2017-12-14 17:32         ` Bird, Timothy
2017-12-14 18:04           ` Mauro Carvalho Chehab
2017-12-14 18:30             ` Philippe Ombredanne
2018-01-15  0:49             ` Takiguchi, Yasunari
2018-01-15  1:27               ` Takiguchi, Yasunari
2017-12-14 18:22           ` Philippe Ombredanne

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAOFm3uEYfMH8Zj8uEx-D9yYrTyDMTG_j02619esHu-j0brQKaA@mail.gmail.com \
    --to=pombredanne@nexb.com \
    --cc=Hideki.Nozawa@sony.com \
    --cc=Kota.Yonezawa@sony.com \
    --cc=Masayuki.Yamamoto@sony.com \
    --cc=Satoshi.C.Watanabe@sony.com \
    --cc=Tim.Bird@sony.com \
    --cc=Toshihiko.Matsumoto@sony.com \
    --cc=Yasunari.Takiguchi@sony.com \
    --cc=akpm@linux-foundation.org \
    --cc=devicetree@vger.kernel.org \
    --cc=frowand.list@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-media@vger.kernel.org \
    --cc=mchehab@s-opensource.com \
    --cc=tbird20d@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).