From: Philippe Ombredanne <pombredanne@nexb.com>
To: Mauro Carvalho Chehab <mchehab@s-opensource.com>
Cc: "Takiguchi, Yasunari" <Yasunari.Takiguchi@sony.com>,
"akpm@linux-foundation.org" <akpm@linux-foundation.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"devicetree@vger.kernel.org" <devicetree@vger.kernel.org>,
"linux-media@vger.kernel.org" <linux-media@vger.kernel.org>,
"tbird20d@gmail.com" <tbird20d@gmail.com>,
"frowand.list@gmail.com" <frowand.list@gmail.com>,
"Yamamoto, Masayuki" <Masayuki.Yamamoto@sony.com>,
"Nozawa, Hideki (STWN)" <Hideki.Nozawa@sony.com>,
"Yonezawa, Kota" <Kota.Yonezawa@sony.com>,
"Matsumoto, Toshihiko" <Toshihiko.Matsumoto@sony.com>,
"Watanabe, Satoshi (SSS)" <Satoshi.C.Watanabe@sony.com>,
"Bird, Timothy" <Tim.Bird@sony.com>
Subject: Re: [PATCH v4 00/12] [dt-bindings] [media] Add document file and driver for Sony CXD2880 DVB-T2/T tuner + demodulator
Date: Thu, 14 Dec 2017 15:25:13 +0100 [thread overview]
Message-ID: <CAOFm3uEYfMH8Zj8uEx-D9yYrTyDMTG_j02619esHu-j0brQKaA@mail.gmail.com> (raw)
In-Reply-To: <20171214085503.289f06f8@vento.lan>
Dear Mauro,
On Thu, Dec 14, 2017 at 11:55 AM, Mauro Carvalho Chehab
<mchehab@s-opensource.com> wrote:
> SPDX is a new requirement that started late on Kernel 4.14 development
> cycle (and whose initial changes were merged directly at Linus tree).
> Not all existing files have it yet, as identifying the right license
> on existing files is a complex task, but if you do a:
>
> $ git grep SPDX $(find . -name Makefile) $(find . -name Kconfig)
>
> You'll see that lot of such files have it already.
FWIW, short of having SPDX tags, identifying the right license on
existing files is not a super complex task: if boils down to running
many diffs.
Take the ~60K files in kernel, and about 6K license and notices
reference texts. Then compute a pairwise diff of each of the 60K file
against the 6K reference texts. Repeat the pairwise diff a few more
times, say 10 times, as multiple licenses may appear in any given
kernel file. And keep the diffs that have the fewest
difference/highest similarity with the reference texts as the detected
license. Done!
The only complex thing is that if you have a fast diff that runs at
0.1 millisec end-to-end per diff, you still have 3.6B diffs to do and
this would take about 250 days on one thread. Even with a beefy 250
core CPU, that would still be a day (and quite few kilo watts) . So
the whole trick is to avoid doing a diffs if not really needed. This
is what I do in my scancode-toolkit (that I used/use to help Greg and
Thomas with kernel license scans). Net effect is that on a laptop on 8
threads it takes ~20 minutes to scan a whole kernel using this
diff-based approach and obtain a fairly accurate license detection.
--
Cordially
Philippe Ombredanne
next prev parent reply other threads:[~2017-12-14 14:25 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-10-13 5:46 [PATCH v4 00/12] [dt-bindings] [media] Add document file and driver for Sony CXD2880 DVB-T2/T tuner + demodulator Yasunari.Takiguchi
2017-10-13 5:58 ` [PATCH v4 01/12] [dt-bindings] [media] Add document file for CXD2880 SPI I/F Yasunari.Takiguchi
2017-10-13 5:59 ` [PATCH v4 02/12] [media] cxd2880-spi: Add support for CXD2880 SPI interface Yasunari.Takiguchi
2017-10-13 8:44 ` Honza Petrouš
2017-10-13 9:10 ` Takiguchi, Yasunari
2017-10-13 10:26 ` Honza Petrouš
2017-10-16 1:35 ` Takiguchi, Yasunari
2017-12-13 17:54 ` Mauro Carvalho Chehab
2017-12-20 0:54 ` Takiguchi, Yasunari
2017-10-13 6:02 ` [PATCH v4 03/12] [media] cxd2880: Add common files for the driver Yasunari.Takiguchi
2017-12-13 18:02 ` Mauro Carvalho Chehab
2017-10-13 6:05 ` [PATCH v4 04/12] [media] cxd2880: Add spi device IO routines Yasunari.Takiguchi
2017-10-13 6:07 ` [PATCH v4 05/12] [media] cxd2880: Add tuner part of the driver Yasunari.Takiguchi
2017-12-13 18:40 ` Mauro Carvalho Chehab
2017-12-20 1:01 ` Takiguchi, Yasunari
2017-10-13 6:08 ` [PATCH v4 06/12] [media] cxd2880: Add integration layer for " Yasunari.Takiguchi
2017-12-03 18:23 ` Sean Young
2017-12-13 19:13 ` Mauro Carvalho Chehab
2017-12-20 1:06 ` Takiguchi, Yasunari
2017-10-13 6:09 ` [PATCH v4 07/12] [media] cxd2880: Add top level of " Yasunari.Takiguchi
2017-12-03 22:59 ` Sean Young
2017-12-05 11:47 ` Takiguchi, Yasunari
2017-12-13 19:25 ` Mauro Carvalho Chehab
2017-12-20 1:18 ` Takiguchi, Yasunari
2017-10-13 6:10 ` [PATCH v4 08/12] [media] cxd2880: Add DVB-T control functions " Yasunari.Takiguchi
2017-10-13 6:11 ` [PATCH v4 09/12] [media] cxd2880: Add DVB-T monitor functions Yasunari.Takiguchi
2017-10-13 6:13 ` [PATCH v4 10/12] [media] cxd2880: Add DVB-T2 control functions for the driver Yasunari.Takiguchi
2017-10-13 6:15 ` [PATCH v4 11/12] [media] cxd2880: Add DVB-T2 monitor functions Yasunari.Takiguchi
2017-10-13 6:17 ` [PATCH v4 12/12] [media] cxd2880: Add all Makefile, Kconfig files and Update MAINTAINERS file for the driver Yasunari.Takiguchi
2017-11-16 1:07 ` [PATCH v4 00/12] [dt-bindings] [media] Add document file and driver for Sony CXD2880 DVB-T2/T tuner + demodulator Takiguchi, Yasunari
2017-11-22 4:17 ` Takiguchi, Yasunari
2017-11-29 7:38 ` Mauro Carvalho Chehab
2017-11-29 8:21 ` Takiguchi, Yasunari
2017-12-13 19:36 ` Mauro Carvalho Chehab
2017-12-14 9:59 ` Takiguchi, Yasunari
2017-12-14 10:55 ` Mauro Carvalho Chehab
2017-12-14 14:25 ` Philippe Ombredanne [this message]
2017-12-14 17:32 ` Bird, Timothy
2017-12-14 18:04 ` Mauro Carvalho Chehab
2017-12-14 18:30 ` Philippe Ombredanne
2018-01-15 0:49 ` Takiguchi, Yasunari
2018-01-15 1:27 ` Takiguchi, Yasunari
2017-12-14 18:22 ` Philippe Ombredanne
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAOFm3uEYfMH8Zj8uEx-D9yYrTyDMTG_j02619esHu-j0brQKaA@mail.gmail.com \
--to=pombredanne@nexb.com \
--cc=Hideki.Nozawa@sony.com \
--cc=Kota.Yonezawa@sony.com \
--cc=Masayuki.Yamamoto@sony.com \
--cc=Satoshi.C.Watanabe@sony.com \
--cc=Tim.Bird@sony.com \
--cc=Toshihiko.Matsumoto@sony.com \
--cc=Yasunari.Takiguchi@sony.com \
--cc=akpm@linux-foundation.org \
--cc=devicetree@vger.kernel.org \
--cc=frowand.list@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=mchehab@s-opensource.com \
--cc=tbird20d@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).