xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
From: "Jesus M. Gonzalez-Barahona" <jgb@bitergia.com>
To: Priya <vppriya9@gmail.com>
Cc: Lars Kurth <lars.kurth@xenproject.org>,
	Xen-devel <xen-devel@lists.xen.org>,
	Daniel Izquierdo <dizquierdo@bitergia.com>
Subject: Re: Regarding Outreachy project on Improving CR Dashboard
Date: Mon, 11 Apr 2016 09:53:36 +0200	[thread overview]
Message-ID: <1460361216.7498.242.camel@bitergia.com> (raw)
In-Reply-To: <CAGwjOLM+Qa4xm7UL6E+Bt47Y9mf0gCCjX7fJO1MuPngB+MGrNQ@mail.gmail.com>

On Fri, 2016-04-08 at 19:33 +0530, Priya wrote:
> Hello,
> 
> I tried running the same command in new version of perceval.  I found
> the following missing message id errors in perceval_mbox_parse.log
> file. I am working on the testing part and I will be able to finish
> it in one or two days.
> 
> You can see the errors here [1]
> 
> [1]:http://imgur.com/yVsIoCT

Hi, Priya. I'm not sure about what exactly is causing your messages,
since I cannot reproduce them (see below). But I still suspect that
they may happen because in current versions of Perceval the data parsed
from an mbox is no longer stored as first level key/data in the
dictionary returned by Perceval for each message, but in data for key
"data", which is itself a dictionary.

In particular, in the code:

-------------
              for k in msg_json:
                    try:
                        if key == k['Message-ID'].strip('<>'):
                            k['property'] = key
-------------

probably you should be checking for k['data']['Message-ID'] instead of
just k['Message-ID'].

Please, have a look at how recent versions of Perceval produce the
dictionaries for each message...

But as I said, I cannot reproduce your error. When running your most
recent code right now (9a5abc47bbab3b06550) with the most recent
Perceval/master code (53efc14001c806f0452) I get:

--------------------
(perceval)jgb@expisito:~/src/outreachy/Dashboard/dashboard$ python3
createjson.py --mbox advisory-board-2014-02 --output new.json
Traceback (most recent call last):
  File "createjson.py", line 96, in <module>
    main()
  File "createjson.py", line 92, in main
    mparser.create_json(args.mbox,args.output)
  File "createjson.py", line 59, in create_json
    messages = th.message_details(mbox_files)
  File "/home/jgb/src/outreachy/Dashboard/dashboard/jwzthreading_r.py",
line 338, in message_details
    urllib.request.urlretrieve(filename, 'mbox')
  File "/usr/lib/python3.4/urllib/request.py", line 186, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "/usr/lib/python3.4/urllib/request.py", line 161, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.4/urllib/request.py", line 449, in open
    req = Request(fullurl, data)
  File "/usr/lib/python3.4/urllib/request.py", line 267, in __init__
    self.full_url = url
  File "/usr/lib/python3.4/urllib/request.py", line 293, in full_url
    self._parse()
  File "/usr/lib/python3.4/urllib/request.py", line 322, in _parse
    raise ValueError("unknown url type: %r" % self.full_url)
ValueError: unknown url type: 'advisory-board-2014-02'
-----------------------------

Could you please try to checkout and install exactly the same version
of Perceval I'm using, and see if you get the same error? And if the
above problem with the format returned by Perceval persists, maybe you
can fix that too.

Saludos,

	Jesus.

-- 
Bitergia: http://bitergia.com
/me at Twitter: https://twitter.com/jgbarah


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

  reply	other threads:[~2016-04-11  7:53 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-04-05 16:35 Regarding Outreachy project on Improving CR Dashboard Priya
2016-04-05 23:23 ` Jesus M. Gonzalez-Barahona
2016-04-06 12:00   ` Priya
2016-04-06 21:59     ` Jesus M. Gonzalez-Barahona
2016-04-07 12:27       ` Priya
2016-04-07 17:57         ` Jesus M. Gonzalez-Barahona
2016-04-08 14:03           ` Priya
2016-04-11  7:53             ` Jesus M. Gonzalez-Barahona [this message]
2016-04-13 16:33               ` Priya
2016-04-14 17:11               ` Priya
2016-04-14 22:41                 ` Jesus M. Gonzalez-Barahona
  -- strict thread matches above, loose matches on Subject: below --
2016-03-19  9:54 Priya

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1460361216.7498.242.camel@bitergia.com \
    --to=jgb@bitergia.com \
    --cc=dizquierdo@bitergia.com \
    --cc=lars.kurth@xenproject.org \
    --cc=vppriya9@gmail.com \
    --cc=xen-devel@lists.xen.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).