Redmine problem

Redmine problem

[Nathan Cutler]

Hi David:

In order to automate creation of backport tracker issues, we use a tool 
written by Loic that is essentially a glorified python script. It has 
worked flawlessly for years. Today, all of a sudden, it started failing 
with

requests.exceptions.ConnectionError: ('Connection aborted.', error(104, 
'Connection reset by peer'))

This error message is emanating from the python-requests module, which 
is used to communicate with Redmine (probably via its REST API) over 
http. Apparently, the other side (i.e. tracker.ceph.com) is resetting 
the connection without warning.

Did something change recently on the server side? Like, in the last two 
or three days? If so, I'm curious to hear what, since it would probably 
help me to debug this issue.

Thanks,
Nathan

Re: Redmine problem

[David Galloway]

On 11/08/2017 04:54 PM, Nathan Cutler wrote:
> Hi David:
> 
> In order to automate creation of backport tracker issues, we use a tool
> written by Loic that is essentially a glorified python script. It has
> worked flawlessly for years. Today, all of a sudden, it started failing
> with
> 
> requests.exceptions.ConnectionError: ('Connection aborted.', error(104,
> 'Connection reset by peer'))
> 
> This error message is emanating from the python-requests module, which
> is used to communicate with Redmine (probably via its REST API) over
> http. Apparently, the other side (i.e. tracker.ceph.com) is resetting
> the connection without warning.
> 
> Did something change recently on the server side? Like, in the last two
> or three days? If so, I'm curious to hear what, since it would probably
> help me to debug this issue.
> 

Could it possibly be because Loic no longer has Redmine Administrator
privileges per his request?

Re: Redmine problem

[Nathan Cutler]

> Could it possibly be because Loic no longer has Redmine Administrator
> privileges per his request?

I don't think so. Was the revocation of Loic's admin privs done within 
the past 72 hours?

(The script uses the Redmine key provided on the command line, and I'm 
providing my own key since that's the only one I have.)

Nathan

Re: Redmine problem

[David Galloway]

On 11/08/2017 05:03 PM, Nathan Cutler wrote:
>> Could it possibly be because Loic no longer has Redmine Administrator
>> privileges per his request?
> 
> I don't think so. Was the revocation of Loic's admin privs done within
> the past 72 hours?

Yes.

> 
> (The script uses the Redmine key provided on the command line, and I'm
> providing my own key since that's the only one I have.)
> 

Can you share the script without the API key and maybe some timestamps
so I can check out the logs?

Re: Redmine problem

[Nathan Cutler]

> Can you share the script without the API key and maybe some timestamps
> so I can check out the logs?

The Redmine bits of the script are in this file: 
http://ceph-workbench.dachary.org/root/ceph-workbench/blob/master/ceph_workbench/wbredmine.py

I first started seeing this at around 11:40 a.m. CET (10:40 UTC) and it 
was happening all afternoon - the script would run for some time and 
then fail with the same error, but not at the same spot. Sometimes it 
would process 10 or 20 pending backport issues, other times it would get 
through more.

I just ran it again and it completed twice in a row without error, so I 
guess it was a transient issue. Timestamp after completing the second run:

$ date --utc
Wed Nov  8 22:48:34 UTC 2017

Thanks for looking into it.

Nathan