Missing prettytable during ceph-disk runs

Missing prettytable during ceph-disk runs

[Willem Jan Withagen]

Hi,

During run-tox tests, ceph-disk complains that it is missing prettytable.
It is however installed on my system, and I can load it. So that part
works. But one way or another it is not loaded into the tox environment.

How or what are all other dependancies loaded?
Is there a config file where I need to add that this test needs prettytable?

--WjW

Re: Missing prettytable during ceph-disk runs

[Alfredo Deza]

On Fri, Jun 2, 2017 at 6:42 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> Hi,
>
> During run-tox tests, ceph-disk complains that it is missing prettytable.
> It is however installed on my system, and I can load it. So that part
> works. But one way or another it is not loaded into the tox environment.
>
> How or what are all other dependancies loaded?
> Is there a config file where I need to add that this test needs prettytable?

I am not sure what tox file you are running, but tox will create a
virtual environment where it installs dependencies
for the project you are testing. These dependencies should be defined
by the project, the tox file, or by configuring tox to
use dependencies installed system-wide.

For example, in ceph-installer, we tell tox to install a dependency
that is only needed for tests like this:

https://github.com/ceph/ceph-installer/blob/master/tox.ini#L5-L6

I think that prettytable is a Python dependency so you can go ahead
and add it there and it should work.

This also sounds like a bug :)
>
> --WjW
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Missing prettytable during ceph-disk runs

[Willem Jan Withagen]

On 2-6-2017 16:36, Alfredo Deza wrote:
> On Fri, Jun 2, 2017 at 6:42 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>> Hi,
>>
>> During run-tox tests, ceph-disk complains that it is missing prettytable.
>> It is however installed on my system, and I can load it. So that part
>> works. But one way or another it is not loaded into the tox environment.
>>
>> How or what are all other dependancies loaded?
>> Is there a config file where I need to add that this test needs prettytable?
> 
> I am not sure what tox file you are running, but tox will create a
> virtual environment where it installs dependencies
> for the project you are testing. These dependencies should be defined
> by the project, the tox file, or by configuring tox to
> use dependencies installed system-wide.
> 
> For example, in ceph-installer, we tell tox to install a dependency
> that is only needed for tests like this:
> 
> https://github.com/ceph/ceph-installer/blob/master/tox.ini#L5-L6
> 
> I think that prettytable is a Python dependency so you can go ahead
> and add it there and it should work.
> 
> This also sounds like a bug :)

I agree that it could a bug, but I also guess that Dan added it for
ansible purposes. How that passes jenkins, I do not know.
But on my FreeBSD/jenkins end it dropped on the floor.

--WjW

Re: Missing prettytable during ceph-disk runs

[Willem Jan Withagen]

On 2-6-2017 16:36, Alfredo Deza wrote:
> On Fri, Jun 2, 2017 at 6:42 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>> Hi,
>>
>> During run-tox tests, ceph-disk complains that it is missing prettytable.
>> It is however installed on my system, and I can load it. So that part
>> works. But one way or another it is not loaded into the tox environment.
>>
>> How or what are all other dependancies loaded?
>> Is there a config file where I need to add that this test needs prettytable?
> 
> I am not sure what tox file you are running, but tox will create a
> virtual environment where it installs dependencies
> for the project you are testing. These dependencies should be defined
> by the project, the tox file, or by configuring tox to
> use dependencies installed system-wide.
> 
> For example, in ceph-installer, we tell tox to install a dependency
> that is only needed for tests like this:
> 
> https://github.com/ceph/ceph-installer/blob/master/tox.ini#L5-L6
> 
> I think that prettytable is a Python dependency so you can go ahead
> and add it there and it should work.
> 
> This also sounds like a bug :)

tox.ini contains:
[testenv]
setenv =
       VIRTUAL_ENV={envdir}
       CEPH_DISK={envbindir}/coverage run --append --source=ceph_disk --
{envbindir}/ceph-disk
usedevelop = true
deps =
  {env:NO_INDEX:}
  --use-wheel
  --find-links=file://{toxinidir}/wheelhouse
  -r{toxinidir}/requirements.txt
  -r{toxinidir}/test-requirements.txt
  ../ceph-detect-init

So I added it to requirements.txt, and ran make tests.

After that every thing is A-oke.

Thanx,
--WjW

Re: Missing prettytable during ceph-disk runs

[Sage Weil]

On Fri, 2 Jun 2017, Alfredo Deza wrote:
> On Fri, Jun 2, 2017 at 6:42 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
> > Hi,
> >
> > During run-tox tests, ceph-disk complains that it is missing prettytable.
> > It is however installed on my system, and I can load it. So that part
> > works. But one way or another it is not loaded into the tox environment.
> >
> > How or what are all other dependancies loaded?
> > Is there a config file where I need to add that this test needs prettytable?
> 
> I am not sure what tox file you are running, but tox will create a
> virtual environment where it installs dependencies
> for the project you are testing. These dependencies should be defined
> by the project, the tox file, or by configuring tox to
> use dependencies installed system-wide.
> 
> For example, in ceph-installer, we tell tox to install a dependency
> that is only needed for tests like this:
> 
> https://github.com/ceph/ceph-installer/blob/master/tox.ini#L5-L6
> 
> I think that prettytable is a Python dependency so you can go ahead
> and add it there and it should work.
> 
> This also sounds like a bug :)

I've been having the ceph-disk tox test fail on my local machine for a 
week or two. I haven't been able to figure out why (prettytable is also 
installed here).  I've tried blowing away my build/ directory but that 
doesn't seem to help..

sage

Re: Missing prettytable during ceph-disk runs

[Willem Jan Withagen]

On 2-6-2017 17:39, Sage Weil wrote:
> On Fri, 2 Jun 2017, Alfredo Deza wrote:
>> On Fri, Jun 2, 2017 at 6:42 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>>> Hi,
>>>
>>> During run-tox tests, ceph-disk complains that it is missing prettytable.
>>> It is however installed on my system, and I can load it. So that part
>>> works. But one way or another it is not loaded into the tox environment.
>>>
>>> How or what are all other dependancies loaded?
>>> Is there a config file where I need to add that this test needs prettytable?
>>
>> I am not sure what tox file you are running, but tox will create a
>> virtual environment where it installs dependencies
>> for the project you are testing. These dependencies should be defined
>> by the project, the tox file, or by configuring tox to
>> use dependencies installed system-wide.
>>
>> For example, in ceph-installer, we tell tox to install a dependency
>> that is only needed for tests like this:
>>
>> https://github.com/ceph/ceph-installer/blob/master/tox.ini#L5-L6
>>
>> I think that prettytable is a Python dependency so you can go ahead
>> and add it there and it should work.
>>
>> This also sounds like a bug :)
> 
> I've been having the ceph-disk tox test fail on my local machine for a 
> week or two. I haven't been able to figure out why (prettytable is also 
> installed here).  I've tried blowing away my build/ directory but that 
> doesn't seem to help..

I have a soluution that works for me.

ceph-disk: add required py-module for tox-testing #15433

It that is the way to follow, I do not know, because there are a zillion
place to add a requirement. And I just think this is the one.
Asuming that it also tells tox that prettytable is required durign
regular running.
But that is open for any better suggestions.

Once you add this, you atleast need to run "make tests" to rebuild the
tox env.

--WjW

Re: Missing prettytable during ceph-disk runs

[Brad Hubbard]

On Sat, Jun 3, 2017 at 2:01 AM, Willem Jan Withagen <wjw@digiware.nl> wrote:
>

Hi Willem,

Sorry to bump an old thread but I was wondering if you could test
something for me?

> I have a soluution that works for me.
>
> ceph-disk: add required py-module for tox-testing #15433
>
> It that is the way to follow, I do not know, because there are a zillion
> place to add a requirement. And I just think this is the one.
> Asuming that it also tells tox that prettytable is required durign
> regular running.
> But that is open for any better suggestions.

Would you mind testing replacing "sitepackages=True" with
"whitelist_externals = prettyprint".

When "sitepackages=True" on fc27 and you already have python2-pytest
and python2-coverage installed (as I currently do) the environment
tries to use the system packages and fails horribly.

>
> Once you add this, you atleast need to run "make tests" to rebuild the
> tox env.

You should be able to just...

$ rm ../src/ceph-disk/.tox -rf
$ ctest -R ceph-disk --output-on-failure

> --WjW

-- 
Cheers,
Brad