Re: Regarding Outreachy project on Improving CR Dashboard

[Priya <vppriya9@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 4046 bytes --]

Hello,

Forgot to CC the Xen-devel mailing list.

I have changed the code to add the tests, and now it is like you need to
provide the whole link of the mbox file rather than just the file name.

So,  $python3 createjson.py --mbox
http://lists.xenproject.org/archives/html/mbox/advisory-board-2013-05-2014-02--output
new.json should work instead of

 $ python3 createjson.py --mbox advisory-board-2014-02 --output new.json

I forgot to update the README file, and I have done that now. I will
try k['data']['Message-ID'] instead, and I would update soon. I'm working
on the testing part and I'll complete it soon.



*Priya V*
Amrita University
LinkedIn
<https://www.linkedin.com/in/priya-v-195560b6?trk=nav_responsive_tab_profile>
| GitHub <https://github.com/priya299> | Bitbucket
<https://bitbucket.org/priya299/>


On Mon, Apr 11, 2016 at 1:23 PM, Jesus M. Gonzalez-Barahona <
jgb@bitergia.com> wrote:

> On Fri, 2016-04-08 at 19:33 +0530, Priya wrote:
> > Hello,
> >
> > I tried running the same command in new version of perceval.  I found
> > the following missing message id errors in perceval_mbox_parse.log
> > file. I am working on the testing part and I will be able to finish
> > it in one or two days.
> >
> > You can see the errors here [1]
> >
> > [1]:http://imgur.com/yVsIoCT
>
> Hi, Priya. I'm not sure about what exactly is causing your messages,
> since I cannot reproduce them (see below). But I still suspect that
> they may happen because in current versions of Perceval the data parsed
> from an mbox is no longer stored as first level key/data in the
> dictionary returned by Perceval for each message, but in data for key
> "data", which is itself a dictionary.
>
> In particular, in the code:
>
> -------------
>               for k in msg_json:
>                     try:
>                         if key == k['Message-ID'].strip('<>'):
>                             k['property'] = key
> -------------
>
> probably you should be checking for k['data']['Message-ID'] instead of
> just k['Message-ID'].
>
> Please, have a look at how recent versions of Perceval produce the
> dictionaries for each message...
>
> But as I said, I cannot reproduce your error. When running your most
> recent code right now (9a5abc47bbab3b06550) with the most recent
> Perceval/master code (53efc14001c806f0452) I get:
>
> --------------------
> (perceval)jgb@expisito:~/src/outreachy/Dashboard/dashboard$ python3
> createjson.py --mbox advisory-board-2014-02 --output new.json
> Traceback (most recent call last):
>   File "createjson.py", line 96, in <module>
>     main()
>   File "createjson.py", line 92, in main
>     mparser.create_json(args.mbox,args.output)
>   File "createjson.py", line 59, in create_json
>     messages = th.message_details(mbox_files)
>   File "/home/jgb/src/outreachy/Dashboard/dashboard/jwzthreading_r.py",
> line 338, in message_details
>     urllib.request.urlretrieve(filename, 'mbox')
>   File "/usr/lib/python3.4/urllib/request.py", line 186, in urlretrieve
>     with contextlib.closing(urlopen(url, data)) as fp:
>   File "/usr/lib/python3.4/urllib/request.py", line 161, in urlopen
>     return opener.open(url, data, timeout)
>   File "/usr/lib/python3.4/urllib/request.py", line 449, in open
>     req = Request(fullurl, data)
>   File "/usr/lib/python3.4/urllib/request.py", line 267, in __init__
>     self.full_url = url
>   File "/usr/lib/python3.4/urllib/request.py", line 293, in full_url
>     self._parse()
>   File "/usr/lib/python3.4/urllib/request.py", line 322, in _parse
>     raise ValueError("unknown url type: %r" % self.full_url)
> ValueError: unknown url type: 'advisory-board-2014-02'
> -----------------------------
>
> Could you please try to checkout and install exactly the same version
> of Perceval I'm using, and see if you get the same error? And if the
> above problem with the format returned by Perceval persists, maybe you
> can fix that too.
>
> Saludos,
>
>         Jesus.
>
> --
> Bitergia: http://bitergia.com
> /me at Twitter: https://twitter.com/jgbarah
>
>

[-- Attachment #1.2: Type: text/html, Size: 6453 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel