From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from eggs.gnu.org ([208.118.235.92]:49280)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <tomoki.sekiyama@hds.com>) id 1Ufx9I-0006g7-IT
	for qemu-devel@nongnu.org; Fri, 24 May 2013 14:55:36 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <tomoki.sekiyama@hds.com>) id 1Ufx9F-0008L7-I4
	for qemu-devel@nongnu.org; Fri, 24 May 2013 14:55:32 -0400
Received: from usindpps06.hds.com ([207.126.252.19]:40625)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <tomoki.sekiyama@hds.com>) id 1Ufx9F-0008Ko-64
	for qemu-devel@nongnu.org; Fri, 24 May 2013 14:55:29 -0400
From: Tomoki Sekiyama <tomoki.sekiyama@hds.com>
Date: Fri, 24 May 2013 18:55:21 +0000
Message-ID: <CDC529E9.29CB%Tomoki.Sekiyama@hds.com>
In-Reply-To: <20130523123008.GM9093@stefanha-thinkpad.redhat.com>
Content-Language: en-US
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <0E8C58E4B0DF5C4AB6FCE8B9BAA3C8E5@hds.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Subject: Re: [Qemu-devel] [RFC PATCH v3 11/11] QMP/qmp.py: set locale for
 exceptions to display non-ascii messages correctly
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: Stefan Hajnoczi <stefanha@gmail.com>
Cc: "mdroth@linux.vnet.ibm.com" <mdroth@linux.vnet.ibm.com>, "qemu-devel@nongnu.org" <qemu-devel@nongnu.org>, "lcapitulino@redhat.com" <lcapitulino@redhat.com>, "vrozenfe@redhat.com" <vrozenfe@redhat.com>, "pbonzini@redhat.com" <pbonzini@redhat.com>, Seiji Aguchi <seiji.aguchi@hds.com>, "areis@redhat.com" <areis@redhat.com>

On 5/23/13 8:30 , "Stefan Hajnoczi" <stefanha@gmail.com> wrote:

>On Tue, May 21, 2013 at 11:34:16AM -0400, Tomoki Sekiyama wrote:
>> qemu-ga in Windows may return error message with multibyte characters
>> when the guest OS language is set to other than English. To display such
>> messages correctly, this encodes the message based on the locale
>>settings.
>>=20
>> Signed-off-by: Tomoki Sekiyama <tomoki.sekiyama@hds.com>
>> ---
>>  QMP/qmp.py |    4 +++-
>>  1 file changed, 3 insertions(+), 1 deletion(-)
>>=20
>> diff --git a/QMP/qmp.py b/QMP/qmp.py
>> index c551df1..ee21819 100644
>> --- a/QMP/qmp.py
>> +++ b/QMP/qmp.py
>> @@ -11,6 +11,7 @@
>>  import json
>>  import errno
>>  import socket
>> +import locale
>> =20
>>  class QMPError(Exception):
>>      pass
>> @@ -133,7 +134,8 @@ class QEMUMonitorProtocol:
>>      def command(self, cmd, **kwds):
>>          ret =3D self.cmd(cmd, kwds)
>>          if ret.has_key('error'):
>> -            raise Exception(ret['error']['desc'])
>> +            enc =3D locale.getpreferredencoding()
>> +            raise Exception(ret['error']['desc'].encode(enc))
>
>You should not need to explicitly encode the error descriptor.  The
>error description should be UTF-8 on the wire and a Unicode Python
>string in this script.
>
>I think the real problem is:
>
>1. Guest qga is writing strings in local encoding onto the wire.

The error description Guest qga writes is encoded like:

  {"error": {"class": "GenericError", "desc": "\u64CD\u4F5C\u306F..."}}

I feel this is correct (\u64CD is a correct representation of single
character in Unicode).


>or
>
>2. qmp.py isn't UTF-8-decoding strings received over the wire.

And qmp.py can decode this correctly, and ret['error']['desc'] is
Unicode Python string.

The problem looks like in python Exception, that cannot print out
the message at all if it contains non-ascii Unicode string:

% python

>>> raise Exception(u"abc")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception: abc

>>> raise Exception(u"abc=FC")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception

while the "print" doesn't have this issue:

>>> print(u"abc=FC")
abc=FC



>Either or both bugs could be present.  Once they are fixed you shouldn't
>see encoding problems.
>
>Stefan

Anyway, as I understood this patch for qmp.py is not correct way to fix
this issue, I'm going to drop it from the series.

Thanks,

Tomoki Sekiyama