[Buildroot] [PATCH 0/2] support/run-test: move pacakges' tests to packages' directories

[Buildroot] [PATCH 0/2] support/run-test: move pacakges' tests to packages' directories

[Yann E. MORIN]

Hello All!

It looks more logical to have packages' tests right next to the packages
own descriptions, side by side with the Config.in and the .mk files.

If this solution is accepted, then it will also allow to easily add
tests directly in the filesystems and boot directories.


Regards,
Yann E. MORIN.


The following changes since commit c665c7c9cd6646b135cdd9aa7036809f7771ab80

  netsnmp: install all MIB files (2017-09-13 23:23:40 +0200)


are available in the git repository at:

  git://git.buildroot.org/~ymorin/git/buildroot.git

for you to fetch changes up to 5d8ea4ebb51023d3943fb4e5a1678f8cfba70527

  testing: add python-cryptography tests (2017-09-16 22:20:52 +0200)


----------------------------------------------------------------
Yann E. MORIN (1):
      support/run-tests: move packages tests to packages directories

Yegor Yefremov (1):
      testing: add python-cryptography tests

 .gitignore                                         |  1 +
 .gitlab-ci.yml                                     |  2 ++
 .../dropbear/dropbear.py                           |  0
 package/python-cryptography/python-cryptography.py | 31 ++++++++++++++++++++++
 .../python-ipython/python-ipython.py               |  0
 .../test_python.py => package/python/python.py     |  0
 support/testing/run-tests                          | 22 +++++++++++++++
 support/testing/tests/package/__init__.py          |  0
 8 files changed, 56 insertions(+)
 rename support/testing/tests/package/test_dropbear.py => package/dropbear/dropbear.py (100%)
 create mode 100644 package/python-cryptography/python-cryptography.py
 rename support/testing/tests/package/test_ipython.py => package/python-ipython/python-ipython.py (100%)
 rename support/testing/tests/package/test_python.py => package/python/python.py (100%)
 delete mode 100644 support/testing/tests/package/__init__.py

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/2] support/run-tests: move packages tests to packages directories

[Yann E. MORIN]

Move the existing few packages' tests to the correspnding package
directories.

If we were to add the package/ directory to nose2's search path, we
would have to add __init__.py files about everywhere, at each level of
the directory hierarchy, because nose2 only includes tests that are in a
python pacakge (a directory with a __init__.py file is a python package).
This is far from ideal, especially since we have absolutely nothing to
put in those __init__.py files.

An alternative is to just scan the package/ directory ourselves, and
create symlinks in a sub-directory of support/tessting/tests/ so that
nose2 does find them. We have no two packages with the same name, we
don't risk having two symlinks with the same name.

That second solution is what we choose to do, at the cost of a slight
increase in complexity in the run-test script.

Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Arnout Vandecappelle <arnout@mind.be>
Cc: Samuel Martin <s.martin49@gmail.com>
---
 .gitignore                                         |  1 +
 .../dropbear/dropbear.py                           |  0
 .../python-ipython/python-ipython.py               |  0
 .../test_python.py => package/python/python.py     |  0
 support/testing/run-tests                          | 22 ++++++++++++++++++++++
 support/testing/tests/package/__init__.py          |  0
 6 files changed, 23 insertions(+)
 rename support/testing/tests/package/test_dropbear.py => package/dropbear/dropbear.py (100%)
 rename support/testing/tests/package/test_ipython.py => package/python-ipython/python-ipython.py (100%)
 rename support/testing/tests/package/test_python.py => package/python/python.py (100%)
 delete mode 100644 support/testing/tests/package/__init__.py

diff --git a/.gitignore b/.gitignore
index bb02d9f572..e85c07f99a 100644
--- a/.gitignore
+++ b/.gitignore
@@ -9,6 +9,7 @@
 *.o
 /*.patch
 /*.diff
+/support/testing/tests/package/
 *.orig
 *.rej
 *~
diff --git a/support/testing/tests/package/test_dropbear.py b/package/dropbear/dropbear.py
similarity index 100%
rename from support/testing/tests/package/test_dropbear.py
rename to package/dropbear/dropbear.py
diff --git a/support/testing/tests/package/test_ipython.py b/package/python-ipython/python-ipython.py
similarity index 100%
rename from support/testing/tests/package/test_ipython.py
rename to package/python-ipython/python-ipython.py
diff --git a/support/testing/tests/package/test_python.py b/package/python/python.py
similarity index 100%
rename from support/testing/tests/package/test_python.py
rename to package/python/python.py
diff --git a/support/testing/run-tests b/support/testing/run-tests
index ae0bd336b5..67736331ae 100755
--- a/support/testing/run-tests
+++ b/support/testing/run-tests
@@ -2,6 +2,9 @@
 import argparse
 import sys
 import os
+import errno
+import shutil
+import re
 import nose2
 import multiprocessing
 
@@ -36,6 +39,9 @@ def main():
     script_path = os.path.realpath(__file__)
     test_dir = os.path.dirname(script_path)
 
+    gatheradditionaltests(os.path.abspath("."),
+                          os.path.join(test_dir,"tests","package"))
+
     if args.stdout:
         BRTest.logtofile = False
 
@@ -116,5 +122,21 @@ def main():
 
     nose2.discover(argv=nose2_args)
 
+def gatheradditionaltests(search_dir, symlink_dir):
+    try:
+        shutil.rmtree(symlink_dir)
+    except OSError as err:
+        if not err.errno == errno.ENOENT:
+            raise err
+        pass
+    os.mkdir(symlink_dir)
+    open(os.path.join(symlink_dir,"__init__.py"), "w").close()
+    for dir, _, files in os.walk(os.path.abspath(search_dir)):
+        package = os.path.basename(dir)
+        for file in files:
+            if re.match("^{}.py$".format(package),file):
+                os.symlink(os.path.join(dir,file),
+                           os.path.join(symlink_dir,"test_{}.py".format(package)))
+
 if __name__ == "__main__":
     sys.exit(main())
diff --git a/support/testing/tests/package/__init__.py b/support/testing/tests/package/__init__.py
deleted file mode 100644
index e69de29bb2..0000000000
-- 
2.11.0

[Buildroot] [PATCH 2/2] testing: add python-cryptography tests

[Yann E. MORIN]

From: Yegor Yefremov <yegorslists@googlemail.com>

Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[yann.morin.1998 at free.fr:
  - move to package directory,
  - indent the config fragment
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>

---
Note:
This is exactly the same as https://patchwork.ozlabs.org/patch/811011/
but with the test moved to the python-cryptography package and the
config fragment indented.
---
 .gitlab-ci.yml                                     |  2 ++
 package/python-cryptography/python-cryptography.py | 31 ++++++++++++++++++++++
 2 files changed, 33 insertions(+)
 create mode 100644 package/python-cryptography/python-cryptography.py

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index b44c935f2a..091b016191 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -250,6 +250,8 @@ tests.package.test_ipython.TestIPythonPy2: *runtime_test
 tests.package.test_ipython.TestIPythonPy3: *runtime_test
 tests.package.test_python.TestPython2: *runtime_test
 tests.package.test_python.TestPython3: *runtime_test
+tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
+tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
 tests.toolchain.test_external.TestExternalToolchainBuildrootMusl: *runtime_test
 tests.toolchain.test_external.TestExternalToolchainBuildrootuClibc: *runtime_test
 tests.toolchain.test_external.TestExternalToolchainCCache: *runtime_test
diff --git a/package/python-cryptography/python-cryptography.py b/package/python-cryptography/python-cryptography.py
new file mode 100644
index 0000000000..907baaafa6
--- /dev/null
+++ b/package/python-cryptography/python-cryptography.py
@@ -0,0 +1,31 @@
+import os
+
+from tests.package.test_python import TestPythonBase
+
+class TestPythonCryptography(TestPythonBase):
+    def fernet_test(self, timeout=-1):
+        cmd = self.interpreter + " -c 'from cryptography.fernet import Fernet;"
+        cmd += "key = Fernet.generate_key();"
+        cmd += "f = Fernet(key)'"
+        _, exit_code = self.emulator.run(cmd, timeout)
+        self.assertEqual(exit_code, 0)
+
+class TestPythonPy2Cryptography(TestPythonCryptography):
+    config = TestPythonBase.config + \
+        """
+        BR2_PACKAGE_PYTHON=y
+        BR2_PACKAGE_PYTHON_CRYPTOGRAPHY=y
+        """
+    def test_run(self):
+        self.login()
+        self.fernet_test(40)
+
+class TestPythonPy3Cryptography(TestPythonCryptography):
+    config = TestPythonBase.config + \
+        """
+        BR2_PACKAGE_PYTHON3=y
+        BR2_PACKAGE_PYTHON_CRYPTOGRAPHY=y
+        """
+    def test_run(self):
+        self.login()
+        self.fernet_test(40)
-- 
2.11.0

[Buildroot] [PATCH 1/2] support/run-tests: move packages tests to packages directories

[Ricardo Martincoski]

Hello,

On Sat, Sep 16, 2017 at 05:38 PM, Yann E. MORIN wrote:

> Move the existing few packages' tests to the correspnding package

typo:                                          corresponding

> directories.
> 
> If we were to add the package/ directory to nose2's search path, we
> would have to add __init__.py files about everywhere, at each level of
> the directory hierarchy, because nose2 only includes tests that are in a
> python pacakge (a directory with a __init__.py file is a python package).

typo:    package

> This is far from ideal, especially since we have absolutely nothing to
> put in those __init__.py files.
> 
> An alternative is to just scan the package/ directory ourselves, and
> create symlinks in a sub-directory of support/tessting/tests/ so that

typo:                                           testing

> nose2 does find them. We have no two packages with the same name, we
> don't risk having two symlinks with the same name.
> 
> That second solution is what we choose to do, at the cost of a slight
> increase in complexity in the run-test script.
> 
> Signed-off-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
> Cc: Ricardo Martincoski <ricardo.martincoski@gmail.com>
> Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> Cc: Arnout Vandecappelle <arnout@mind.be>
> Cc: Samuel Martin <s.martin49@gmail.com>
> ---
>  .gitignore                                         |  1 +
>  .../dropbear/dropbear.py                           |  0
>  .../python-ipython/python-ipython.py               |  0
>  .../test_python.py => package/python/python.py     |  0
>  support/testing/run-tests                          | 22 ++++++++++++++++++++++
>  support/testing/tests/package/__init__.py          |  0
>  6 files changed, 23 insertions(+)
>  rename support/testing/tests/package/test_dropbear.py => package/dropbear/dropbear.py (100%)
>  rename support/testing/tests/package/test_ipython.py => package/python-ipython/python-ipython.py (100%)

.gitlab-ci.yml needs to be generated again because of this rename.

As I mentioned in another thread, having Python files with '-' in their name
AFAIK is not a problem as long we don't want to import them.
We currently import only test_python.py and it seems unlikely to have runtime
tests importing each other.

And this series is really changing the filename convention for runtime tests of
packages, making the pre-existing convention of package files beat the
convention inside the test infra. I am OK with this change, but I think it worth
mentioning. Perhaps it fits in the commit log?

[snip]
> @@ -36,6 +39,9 @@ def main():
>      script_path = os.path.realpath(__file__)
>      test_dir = os.path.dirname(script_path)
>  
> +    gatheradditionaltests(os.path.abspath("."),

Below you run all files in this path (base dir of buildroot) to match against
the regex. It's bad because one can have output/, test-output/, test-dl/,
outgoing/, ...

Also os.getcwd() should give the same result and it is already used in 
support/testing/infra/__init__.py

So for now I suggest to change to:
gather_additional_tests(os.path.join(os.getcwd(), "package"),
And if fs tests gets moved, just add another call to the same method.
We don't have that many subdirectories in the root directory.

Other way around is to pass an array of relative subdirectories
gather_additional_tests(["package", "fs"],
and compose the absolute path inside the method while iterating over the
entries.

> +                          os.path.join(test_dir,"tests","package"))

Here and all over the file there are missing whitespace after ','
Can you use flake8 to find them and fix it?

If you use vim you can do
:set makeprg=flake8\ %
:make
and it will jump the cursor to the right place

> +
>      if args.stdout:
>          BRTest.logtofile = False
>  
> @@ -116,5 +122,21 @@ def main():
>  
>      nose2.discover(argv=nose2_args)
>  
> +def gatheradditionaltests(search_dir, symlink_dir):

You could call it gather_additional_tests.

> +    try:
> +        shutil.rmtree(symlink_dir)
> +    except OSError as err:
> +        if not err.errno == errno.ENOENT:
> +            raise err
> +        pass

Could we do just this? (like is done in builder.py)
    if os.path.exists(symlink_dir):
        shutil.rmtree(symlink_dir)
If you change this, errno doesn't need to be imported.

> +    os.mkdir(symlink_dir)
> +    open(os.path.join(symlink_dir,"__init__.py"), "w").close()
> +    for dir, _, files in os.walk(os.path.abspath(search_dir)):

I first thought, why not glob.glob? But in Python 2 it is not recursive.
So indeed os.walk is the way to go.

> +        package = os.path.basename(dir)
> +        for file in files:

You can import fnmatch to drastically reduce the number of regex compilations:
        for file in fnmatch.filter(files, '*.py'):

> +            if re.match("^{}.py$".format(package),file):
> +                os.symlink(os.path.join(dir,file),
> +                           os.path.join(symlink_dir,"test_{}.py".format(package)))

Regards,
Ricardo

[Buildroot] [PATCH 2/2] testing: add python-cryptography tests

[Ricardo Martincoski]

Hello,

On Sat, Sep 16, 2017 at 05:38 PM, Yann E. MORIN wrote:

> testing: add python-cryptography tests

This patch could now be renamed to:
python-cryptography: add runtime tests
or something like this

> From: Yegor Yefremov <yegorslists@googlemail.com>
> 
> Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
> Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
> [yann.morin.1998 at free.fr:
>   - move to package directory,

  - rename the test file to follow the package convention,

I am OK with this change, but I think it worth mentioning.

>   - indent the config fragment
> ]
> Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
> 
> ---
> Note:
> This is exactly the same as https://patchwork.ozlabs.org/patch/811011/
> but with the test moved to the python-cryptography package and the
> config fragment indented.
> ---
>  .gitlab-ci.yml                                     |  2 ++
>  package/python-cryptography/python-cryptography.py | 31 ++++++++++++++++++++++
>  2 files changed, 33 insertions(+)
>  create mode 100644 package/python-cryptography/python-cryptography.py

.gitlab-ci.yml needs to be generated again because of this rename (to use '-'
in the name again).

> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index b44c935f2a..091b016191 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -250,6 +250,8 @@ tests.package.test_ipython.TestIPythonPy2: *runtime_test
>  tests.package.test_ipython.TestIPythonPy3: *runtime_test
>  tests.package.test_python.TestPython2: *runtime_test
>  tests.package.test_python.TestPython3: *runtime_test
> +tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
> +tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
>  tests.toolchain.test_external.TestExternalToolchainBuildrootMusl: *runtime_test
>  tests.toolchain.test_external.TestExternalToolchainBuildrootuClibc: *runtime_test
>  tests.toolchain.test_external.TestExternalToolchainCCache: *runtime_test
[snip]

Regards,
Ricardo

[Buildroot] [PATCH 1/2] support/run-tests: move packages tests to packages directories

[Yann E. MORIN]

Ricardo, All,

On 2017-09-17 02:20 -0300, Ricardo Martincoski spake thusly:
> On Sat, Sep 16, 2017 at 05:38 PM, Yann E. MORIN wrote:
> > ---
> >  .gitignore                                         |  1 +
> >  .../dropbear/dropbear.py                           |  0
> >  .../python-ipython/python-ipython.py               |  0
> >  .../test_python.py => package/python/python.py     |  0
> >  support/testing/run-tests                          | 22 ++++++++++++++++++++++
> >  support/testing/tests/package/__init__.py          |  0
> >  6 files changed, 23 insertions(+)
> >  rename support/testing/tests/package/test_dropbear.py => package/dropbear/dropbear.py (100%)
> >  rename support/testing/tests/package/test_ipython.py => package/python-ipython/python-ipython.py (100%)
> As I mentioned in another thread, having Python files with '-' in their name
> AFAIK is not a problem as long we don't want to import them.
> We currently import only test_python.py and it seems unlikely to have runtime
> tests importing each other.

Yet we don't know what the future will be made of, so we should squash
the dasj into an underscore, so as to allow this kind of situation.

> And this series is really changing the filename convention for runtime tests of
> packages, making the pre-existing convention of package files beat the
> convention inside the test infra. I am OK with this change, but I think it worth
> mentioning. Perhaps it fits in the commit log?

Yes, it does change the naming convention. I was about to add it to our
manual, in the section about adding new packages, to document the .py
file in addition to Config.in and the .mk and .hash files. But then I
noticed we had nothing about the runtime tests in the manual, so I could
not refer to that section to explain the .py file.

So I deferred updating the manual for later.

> [snip]
> > @@ -36,6 +39,9 @@ def main():
> >      script_path = os.path.realpath(__file__)
> >      test_dir = os.path.dirname(script_path)
> >  
> > +    gatheradditionaltests(os.path.abspath("."),
> 
> Below you run all files in this path (base dir of buildroot) to match against
> the regex. It's bad because one can have output/, test-output/, test-dl/,
> outgoing/, ...

Arg, I initially wanted to only scan package/ in fact.

> Also os.getcwd() should give the same result and it is already used in 
> support/testing/infra/__init__.py
> 
> So for now I suggest to change to:
> gather_additional_tests(os.path.join(os.getcwd(), "package"),

Except that both your solution and mine break when the script is not
called from the Buildroot top directory. We must derive the top
directory from the script path.

> And if fs tests gets moved, just add another call to the same method.
> We don't have that many subdirectories in the root directory.
> 
> Other way around is to pass an array of relative subdirectories
> gather_additional_tests(["package", "fs"],
> and compose the absolute path inside the method while iterating over the
> entries.

I think the origin directory should be passed as argument to the
function, for the above reason.

And yes, we could pass a list of directories.

> > +                          os.path.join(test_dir,"tests","package"))
> 
> Here and all over the file there are missing whitespace after ','
> Can you use flake8 to find them and fix it?

Yup, will do.

> If you use vim you can do
> :set makeprg=flake8\ %
> :make
> and it will jump the cursor to the right place
> 
> > +
> >      if args.stdout:
> >          BRTest.logtofile = False
> >  
> > @@ -116,5 +122,21 @@ def main():
> >  
> >      nose2.discover(argv=nose2_args)
> >  
> > +def gatheradditionaltests(search_dir, symlink_dir):
> 
> You could call it gather_additional_tests.

I never know whatThe_prefered_Solutionis. ;-)

> > +    try:
> > +        shutil.rmtree(symlink_dir)
> > +    except OSError as err:
> > +        if not err.errno == errno.ENOENT:
> > +            raise err
> > +        pass
> 
> Could we do just this? (like is done in builder.py)
>     if os.path.exists(symlink_dir):
>         shutil.rmtree(symlink_dir)
> If you change this, errno doesn't need to be imported.

So I am no Python expert, and this shows. Thanks for the hints! :-)

> > +    os.mkdir(symlink_dir)
> > +    open(os.path.join(symlink_dir,"__init__.py"), "w").close()
> > +    for dir, _, files in os.walk(os.path.abspath(search_dir)):
> 
> I first thought, why not glob.glob? But in Python 2 it is not recursive.
> So indeed os.walk is the way to go.
> 
> > +        package = os.path.basename(dir)
> > +        for file in files:
> 
> You can import fnmatch to drastically reduce the number of regex compilations:
>         for file in fnmatch.filter(files, '*.py'):

Wee! :-)

> > +            if re.match("^{}.py$".format(package),file):
> > +                os.symlink(os.path.join(dir,file),
> > +                           os.path.join(symlink_dir,"test_{}.py".format(package)))
> 
> Regards,
> Ricardo


-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 2/2] testing: add python-cryptography tests

[Yann E. MORIN]

Ricardo, All,

On 2017-09-17 02:26 -0300, Ricardo Martincoski spake thusly:
> On Sat, Sep 16, 2017 at 05:38 PM, Yann E. MORIN wrote:
> 
> > testing: add python-cryptography tests
> 
> This patch could now be renamed to:
> python-cryptography: add runtime tests
> or something like this

Yep, good point.

> > From: Yegor Yefremov <yegorslists@googlemail.com>
> > 
> > Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
> > Reviewed-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
> > [yann.morin.1998 at free.fr:
> >   - move to package directory,
> 
>   - rename the test file to follow the package convention,
> 
> I am OK with this change, but I think it worth mentioning.

Ack.

Regards,
Yann E. MORIN.

> >   - indent the config fragment
> > ]
> > Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
> > 
> > ---
> > Note:
> > This is exactly the same as https://patchwork.ozlabs.org/patch/811011/
> > but with the test moved to the python-cryptography package and the
> > config fragment indented.
> > ---
> >  .gitlab-ci.yml                                     |  2 ++
> >  package/python-cryptography/python-cryptography.py | 31 ++++++++++++++++++++++
> >  2 files changed, 33 insertions(+)
> >  create mode 100644 package/python-cryptography/python-cryptography.py
> 
> .gitlab-ci.yml needs to be generated again because of this rename (to use '-'
> in the name again).
> 
> > 
> > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > index b44c935f2a..091b016191 100644
> > --- a/.gitlab-ci.yml
> > +++ b/.gitlab-ci.yml
> > @@ -250,6 +250,8 @@ tests.package.test_ipython.TestIPythonPy2: *runtime_test
> >  tests.package.test_ipython.TestIPythonPy3: *runtime_test
> >  tests.package.test_python.TestPython2: *runtime_test
> >  tests.package.test_python.TestPython3: *runtime_test
> > +tests.package.test_python_cryptography.TestPythonPy2Cryptography: *runtime_test
> > +tests.package.test_python_cryptography.TestPythonPy3Cryptography: *runtime_test
> >  tests.toolchain.test_external.TestExternalToolchainBuildrootMusl: *runtime_test
> >  tests.toolchain.test_external.TestExternalToolchainBuildrootuClibc: *runtime_test
> >  tests.toolchain.test_external.TestExternalToolchainCCache: *runtime_test
> [snip]
> 
> Regards,
> Ricardo


-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/2] support/run-tests: move packages tests to packages directories

[Yann E. MORIN]

Ricardo, All,

On 2017-09-17 09:18 +0200, Yann E. MORIN spake thusly:
> On 2017-09-17 02:20 -0300, Ricardo Martincoski spake thusly:
> > On Sat, Sep 16, 2017 at 05:38 PM, Yann E. MORIN wrote:
[--SNIP--]
> > > +        package = os.path.basename(dir)
> > > +        for file in files:
> > 
> > You can import fnmatch to drastically reduce the number of regex compilations:
> >         for file in fnmatch.filter(files, '*.py'):
> Wee! :-)

In fact, no. At least, not in this state, because we do not want to
match _any_ .py file; instead, we want to match only .py files that are
named after the package.

But I guess I can make it to work with fnmatch nonetheless. ;-)

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/2] support/run-tests: move packages tests to packages directories

[Ricardo Martincoski]

Hello,

On Sun, Sep 17, 2017 at 04:52 AM, Yann E. MORIN wrote:

> On 2017-09-17 09:18 +0200, Yann E. MORIN spake thusly:
>> On 2017-09-17 02:20 -0300, Ricardo Martincoski spake thusly:
>> > On Sat, Sep 16, 2017 at 05:38 PM, Yann E. MORIN wrote:
> [--SNIP--]
>> > > +        package = os.path.basename(dir)
>> > > +        for file in files:
>> > 
>> > You can import fnmatch to drastically reduce the number of regex compilations:
>> >         for file in fnmatch.filter(files, '*.py'):
>> Wee! :-)
> 
> In fact, no. At least, not in this state, because we do not want to
> match _any_ .py file; instead, we want to match only .py files that are
> named after the package.

We could use 2 filters. Instead of 8k+ regex compilations (one for each file
inside a package directory) we would run 2k+ fnmatch.filter (that probably
compile one regex, one per directory) and one regex compilation for each test
file in the package tree.

         for file in fnmatch.filter(files, '*.py'):
             if re.match("^{}.py$".format(package),file):

But well... the default python interpreter has a cache for regex (I don't know
its internals, just that it exists) and perhaps the change I suggested won't
make any difference in performance.

> 
> But I guess I can make it to work with fnmatch nonetheless. ;-)

BTW, could you change the DEVELOPERS file in the same patch?
+F:     package/*/*.py
 F:     support/testing/

I know it won't work for all possible files. 99% is enough for me.

Regards,
Ricardo

[Buildroot] [PATCH 1/2] support/run-tests: move packages tests to packages directories

[Yann E. MORIN]

Ricardo, All,

On 2017-10-01 21:54 -0300, Ricardo Martincoski spake thusly:
> On Sun, Sep 17, 2017 at 04:52 AM, Yann E. MORIN wrote:
> > On 2017-09-17 09:18 +0200, Yann E. MORIN spake thusly:
> >> On 2017-09-17 02:20 -0300, Ricardo Martincoski spake thusly:
> >> > On Sat, Sep 16, 2017 at 05:38 PM, Yann E. MORIN wrote:
> > [--SNIP--]
> >> > > +        package = os.path.basename(dir)
> >> > > +        for file in files:
> >> > 
> >> > You can import fnmatch to drastically reduce the number of regex compilations:
> >> >         for file in fnmatch.filter(files, '*.py'):
> >> Wee! :-)
> > 
> > In fact, no. At least, not in this state, because we do not want to
> > match _any_ .py file; instead, we want to match only .py files that are
> > named after the package.
> 
> We could use 2 filters. Instead of 8k+ regex compilations (one for each file
> inside a package directory) we would run 2k+ fnmatch.filter (that probably
> compile one regex, one per directory) and one regex compilation for each test
> file in the package tree.
> 
>          for file in fnmatch.filter(files, '*.py'):
>              if re.match("^{}.py$".format(package),file):

Err... I already changed it to:

    for file in fnmatch.filter(files, '{}.py'.format(package)):

Is there something wrong with that? ;-)

> But well... the default python interpreter has a cache for regex (I don't know
> its internals, just that it exists) and perhaps the change I suggested won't
> make any difference in performance.
> 
> > 
> > But I guess I can make it to work with fnmatch nonetheless. ;-)
> 
> BTW, could you change the DEVELOPERS file in the same patch?
> +F:     package/*/*.py
>  F:     support/testing/

You meant, add it to your entry ?

In fact, I would expect to assign the test files to the corresponding
package custodian (now imp[licit by their location).

But yes, I can assign them to you as well.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH 1/2] support/run-tests: move packages tests to packages directories

[Ricardo Martincoski]

Hello,

On Mon, Oct 02, 2017 at 02:32 AM, Yann E. MORIN wrote:

> On 2017-10-01 21:54 -0300, Ricardo Martincoski spake thusly:
>> On Sun, Sep 17, 2017 at 04:52 AM, Yann E. MORIN wrote:
>> > On 2017-09-17 09:18 +0200, Yann E. MORIN spake thusly:
>> >> On 2017-09-17 02:20 -0300, Ricardo Martincoski spake thusly:
>> >> > On Sat, Sep 16, 2017 at 05:38 PM, Yann E. MORIN wrote:
>> > [--SNIP--]
>> >> > > +        package = os.path.basename(dir)
>> >> > > +        for file in files:
>> >> > 
>> >> > You can import fnmatch to drastically reduce the number of regex compilations:
>> >> >         for file in fnmatch.filter(files, '*.py'):
>> >> Wee! :-)
>> > 
>> > In fact, no. At least, not in this state, because we do not want to
>> > match _any_ .py file; instead, we want to match only .py files that are
>> > named after the package.
>> 
>> We could use 2 filters. Instead of 8k+ regex compilations (one for each file
>> inside a package directory) we would run 2k+ fnmatch.filter (that probably
>> compile one regex, one per directory) and one regex compilation for each test
>> file in the package tree.
>> 
>>          for file in fnmatch.filter(files, '*.py'):
>>              if re.match("^{}.py$".format(package),file):
> 
> Err... I already changed it to:
> 
>     for file in fnmatch.filter(files, '{}.py'.format(package)):
> 
> Is there something wrong with that? ;-)

Nothing wrong. It's better this (your) way IMO.

> 
>> But well... the default python interpreter has a cache for regex (I don't know
>> its internals, just that it exists) and perhaps the change I suggested won't
>> make any difference in performance.
>> 
>> > 
>> > But I guess I can make it to work with fnmatch nonetheless. ;-)
>> 
>> BTW, could you change the DEVELOPERS file in the same patch?
>> +F:     package/*/*.py
>>  F:     support/testing/
> 
> You meant, add it to your entry ?

Yes.

> 
> In fact, I would expect to assign the test files to the corresponding
> package custodian (now imp[licit by their location).
> 
> But yes, I can assign them to you as well.

Thanks.

Regards,
Ricardo