Fwd: scripts/spdxcheck.py's third-party dependencies

Fwd: scripts/spdxcheck.py's third-party dependencies

[Nicky Chorley]

Resending to linux-spdx.

---------- Forwarded message ---------
From: Nicky Chorley <ndchorley@gmail.com>
Date: Sun, 25 Jul 2021 at 20:35
Subject: scripts/spdxcheck.py's third-party dependencies
To: <tglx@linutronix.de>, <gregkh@linuxfoundation.org>


Hi folks,

On linux-next (tag: next-20210723), running scripts/checkpatch.pl
complains, with e.g.

Traceback (most recent call last):
   File "scripts/spdxcheck.py", line 10, in <module>
     import git
ModuleNotFoundError: No module named 'git'

and the same for ply if you don't have those libraries installed. Would it
be OK to add a pip requirements.txt file, so that people can use that to
install those and not have to manually go hunt them down? In future,
if other Python scripts needed third-party libraries, those could be
listed in the file too (I had a quick look and didn't see any used in the
others).

Apologies if this has already been covered somewhere - I couldn't see
archive links for linux-spdx on vger.kernel.org.

Best,

Nicky

Re: Fwd: scripts/spdxcheck.py's third-party dependencies

[Greg KH]

On Tue, Jul 27, 2021 at 07:42:21AM +0100, Nicky Chorley wrote:
> Resending to linux-spdx.
> 
> ---------- Forwarded message ---------
> From: Nicky Chorley <ndchorley@gmail.com>
> Date: Sun, 25 Jul 2021 at 20:35
> Subject: scripts/spdxcheck.py's third-party dependencies
> To: <tglx@linutronix.de>, <gregkh@linuxfoundation.org>
> 
> 
> Hi folks,
> 
> On linux-next (tag: next-20210723), running scripts/checkpatch.pl
> complains, with e.g.
> 
> Traceback (most recent call last):
>    File "scripts/spdxcheck.py", line 10, in <module>
>      import git
> ModuleNotFoundError: No module named 'git'
> 
> and the same for ply if you don't have those libraries installed. Would it
> be OK to add a pip requirements.txt file, so that people can use that to
> install those and not have to manually go hunt them down? In future,
> if other Python scripts needed third-party libraries, those could be
> listed in the file too (I had a quick look and didn't see any used in the
> others).
> 
> Apologies if this has already been covered somewhere - I couldn't see
> archive links for linux-spdx on vger.kernel.org.

How can you add a requirements.txt file for a single script in a
directory of other scripts?

thanks,

greg k-h

Re: Fwd: scripts/spdxcheck.py's third-party dependencies

[Nicky Chorley]

On Tue, 27 Jul 2021 at 07:47, Greg KH <gregkh@linuxfoundation.org> wrote:
> How can you add a requirements.txt file for a single script in a
> directory of other scripts?

You can name them (e.g. requirements-spdxcheck.txt), as pip lets you
specify which one to use with its -r option. In addition, if one wants
to have a different set of libraries installed for different scripts
(or projects, generally), Python's virtual environments offer a
lightweight way to do that.

Best,

Nicky

Re: Fwd: scripts/spdxcheck.py's third-party dependencies

[Greg KH]

On Sat, Jul 31, 2021 at 12:57:43PM +0100, Nicky Chorley wrote:
> On Tue, 27 Jul 2021 at 07:47, Greg KH <gregkh@linuxfoundation.org> wrote:
> > How can you add a requirements.txt file for a single script in a
> > directory of other scripts?
> 
> You can name them (e.g. requirements-spdxcheck.txt), as pip lets you
> specify which one to use with its -r option. In addition, if one wants
> to have a different set of libraries installed for different scripts
> (or projects, generally), Python's virtual environments offer a
> lightweight way to do that.

But we do not use pip to do kernel builds, so what is this going to help
with?

I'm sorry, I do not understand the problem here...

thanks,

greg k-h

Re: Fwd: scripts/spdxcheck.py's third-party dependencies

[Nicky Chorley]

On Thu, 5 Aug 2021, Greg KH wrote:

> But we do not use pip to do kernel builds, so what is this going to help
> with?

It's just about making people's lives easier for running the spdxcheck.py 
script - not everyone will have the third-party libraries installed, so 
listing them means they can be installed easily. For example:

diff --git a/scripts/README.spdxcheck b/scripts/README.spdxcheck
new file mode 100644
index 000000000000..80bb6308e5c5
--- /dev/null
+++ b/scripts/README.spdxcheck
@@ -0,0 +1,6 @@
+The script spdxcheck.py requires installation of some third party
+libraries. These are listed in ``requirements-spdxcheck.txt`` and can
+be installed with
+
+``pip install -r requirements-spdxcheck.txt``
+
diff --git a/scripts/requirements-spdxcheck.txt b/scripts/requirements-spdxcheck.txt
new file mode 100644
index 000000000000..9a4633040897
--- /dev/null
+++ b/scripts/requirements-spdxcheck.txt
@@ -0,0 +1,3 @@
+ply
+GitPython
+

I'll reformat as a proper patch with any changes if this is useful.

Best,

Nicky

RE: Fwd: scripts/spdxcheck.py's third-party dependencies

[Zavras, Alexios]

Wouldn't the instructions be better if they were simply saying
``pip install ply git``
(or whatever), so no extra requirements file is needed?

-- zvr

-----Original Message-----
From: Nicky Chorley <ndchorley@gmail.com> 
Sent: Friday, 6 August, 2021 08:45
To: Greg KH <gregkh@linuxfoundation.org>
Cc: Nicky Chorley <ndchorley@gmail.com>; linux-spdx@vger.kernel.org
Subject: Re: Fwd: scripts/spdxcheck.py's third-party dependencies

On Thu, 5 Aug 2021, Greg KH wrote:

> But we do not use pip to do kernel builds, so what is this going to 
> help with?

It's just about making people's lives easier for running the spdxcheck.py script - not everyone will have the third-party libraries installed, so listing them means they can be installed easily. For example:

diff --git a/scripts/README.spdxcheck b/scripts/README.spdxcheck new file mode 100644 index 000000000000..80bb6308e5c5
--- /dev/null
+++ b/scripts/README.spdxcheck
@@ -0,0 +1,6 @@
+The script spdxcheck.py requires installation of some third party 
+libraries. These are listed in ``requirements-spdxcheck.txt`` and can 
+be installed with
+
+``pip install -r requirements-spdxcheck.txt``
+
diff --git a/scripts/requirements-spdxcheck.txt b/scripts/requirements-spdxcheck.txt
new file mode 100644
index 000000000000..9a4633040897
--- /dev/null
+++ b/scripts/requirements-spdxcheck.txt
@@ -0,0 +1,3 @@
+ply
+GitPython
+

I'll reformat as a proper patch with any changes if this is useful.

Best,

Nicky
Intel Deutschland GmbH
Registered Address: Am Campeon 10, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de <http://www.intel.de>
Managing Directors: Christin Eisenschmid, Sharon Heck, Tiffany Doon Silva  
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928

Re: Fwd: scripts/spdxcheck.py's third-party dependencies

[Christoph Hellwig]

On Fri, Aug 06, 2021 at 07:44:41AM +0100, Nicky Chorley wrote:
> On Thu, 5 Aug 2021, Greg KH wrote:
> 
> > But we do not use pip to do kernel builds, so what is this going to help
> > with?
> 
> It's just about making people's lives easier for running the spdxcheck.py
> script - not everyone will have the third-party libraries installed, so
> listing them means they can be installed easily. For example:

I think the most useful thing here would be to find a way to make
spdxcheck just work without the non-standard python module.  As far
as I can tell it uses the module only to do the recursive travesal
of the current tree anyway, which seems rather pointless to start with.

Re: Fwd: scripts/spdxcheck.py's third-party dependencies

[Thomas Gleixner]

On Tue, Aug 10 2021 at 17:57, Christoph Hellwig wrote:

> On Fri, Aug 06, 2021 at 07:44:41AM +0100, Nicky Chorley wrote:
>> On Thu, 5 Aug 2021, Greg KH wrote:
>> 
>> > But we do not use pip to do kernel builds, so what is this going to help
>> > with?
>> 
>> It's just about making people's lives easier for running the spdxcheck.py
>> script - not everyone will have the third-party libraries installed, so
>> listing them means they can be installed easily. For example:
>
> I think the most useful thing here would be to find a way to make
> spdxcheck just work without the non-standard python module.  As far
> as I can tell it uses the module only to do the recursive travesal
> of the current tree anyway, which seems rather pointless to start with.

Yes, it's just the tree traversal and it turned out to be convenient to
just use the git python bindings to avoid scanning random crap in the
source tree. It should be simple enough to make it use the output of a
subprocess running 'git ls-files'.

But python-git is packaged in most distros, so when I wrote this script
I really could not be bothered to deal with parsing subprocess output
and aside of that resolving:

  ModuleNotFoundError: No module named 'git'

is not rocket science by any means.

Thanks,

        tglx