* [PATCH] buildinfohelper: work around unicode exceptions
@ 2016-04-05 11:02 Joshua Lock
2016-04-06 12:51 ` Smith, Elliot
0 siblings, 1 reply; 3+ messages in thread
From: Joshua Lock @ 2016-04-05 11:02 UTC (permalink / raw)
To: toaster
We have been seeing UnicodeDecodeError's when handling the
ImagePkgList MetadataEvent in ORMWrapper's
save_target_file_information() if the event includes filenames
that include non-ASCII characters.
In the short term work around this by converting paths to the
unicode type when passing them to Django's ORM. This is a bit of
a hack but it's too late in the cycle to do anything more invasive.
[YOCTO #9142]
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
---
bitbake/lib/bb/ui/buildinfohelper.py | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/bitbake/lib/bb/ui/buildinfohelper.py b/bitbake/lib/bb/ui/buildinfohelper.py
index ed43f4c..c211ec6 100644
--- a/bitbake/lib/bb/ui/buildinfohelper.py
+++ b/bitbake/lib/bb/ui/buildinfohelper.py
@@ -483,7 +483,7 @@ class ORMWrapper(object):
parent_obj = self._cached_get(Target_File, target = target_obj, path = parent_path, inodetype = Target_File.ITYPE_DIRECTORY)
tf_obj = Target_File.objects.create(
target = target_obj,
- path = path,
+ path = unicode(path, 'utf-8'),
size = size,
inodetype = Target_File.ITYPE_DIRECTORY,
permission = permission,
@@ -508,7 +508,7 @@ class ORMWrapper(object):
tf_obj = Target_File.objects.create(
target = target_obj,
- path = path,
+ path = unicode(path, 'utf-8'),
size = size,
inodetype = inodetype,
permission = permission,
@@ -539,7 +539,9 @@ class ORMWrapper(object):
filetarget_path = "/".join(fcpl)
try:
- filetarget_obj = Target_File.objects.get(target = target_obj, path = filetarget_path)
+ filetarget_obj = Target_File.objects.get(
+ target = target_obj,
+ path = unicode(filetarget_path, 'utf-8'))
except Target_File.DoesNotExist:
# we might have an invalid link; no way to detect this. just set it to None
filetarget_obj = None
@@ -548,7 +550,7 @@ class ORMWrapper(object):
tf_obj = Target_File.objects.create(
target = target_obj,
- path = path,
+ path = unicode(path, 'utf-8'),
size = size,
inodetype = Target_File.ITYPE_SYMLINK,
permission = permission,
--
2.5.5
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] buildinfohelper: work around unicode exceptions
2016-04-05 11:02 [PATCH] buildinfohelper: work around unicode exceptions Joshua Lock
@ 2016-04-06 12:51 ` Smith, Elliot
0 siblings, 0 replies; 3+ messages in thread
From: Smith, Elliot @ 2016-04-06 12:51 UTC (permalink / raw)
To: Joshua Lock; +Cc: toaster
[-- Attachment #1: Type: text/plain, Size: 3257 bytes --]
Submitted to bitbake-devel and pushed to toaster-next.
Thanks.
Elliot
On 5 April 2016 at 12:02, Joshua Lock <joshua.g.lock@intel.com> wrote:
> We have been seeing UnicodeDecodeError's when handling the
> ImagePkgList MetadataEvent in ORMWrapper's
> save_target_file_information() if the event includes filenames
> that include non-ASCII characters.
>
> In the short term work around this by converting paths to the
> unicode type when passing them to Django's ORM. This is a bit of
> a hack but it's too late in the cycle to do anything more invasive.
>
> [YOCTO #9142]
>
> Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
> ---
> bitbake/lib/bb/ui/buildinfohelper.py | 10 ++++++----
> 1 file changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/bitbake/lib/bb/ui/buildinfohelper.py
> b/bitbake/lib/bb/ui/buildinfohelper.py
> index ed43f4c..c211ec6 100644
> --- a/bitbake/lib/bb/ui/buildinfohelper.py
> +++ b/bitbake/lib/bb/ui/buildinfohelper.py
> @@ -483,7 +483,7 @@ class ORMWrapper(object):
> parent_obj = self._cached_get(Target_File, target =
> target_obj, path = parent_path, inodetype = Target_File.ITYPE_DIRECTORY)
> tf_obj = Target_File.objects.create(
> target = target_obj,
> - path = path,
> + path = unicode(path, 'utf-8'),
> size = size,
> inodetype = Target_File.ITYPE_DIRECTORY,
> permission = permission,
> @@ -508,7 +508,7 @@ class ORMWrapper(object):
>
> tf_obj = Target_File.objects.create(
> target = target_obj,
> - path = path,
> + path = unicode(path, 'utf-8'),
> size = size,
> inodetype = inodetype,
> permission = permission,
> @@ -539,7 +539,9 @@ class ORMWrapper(object):
> filetarget_path = "/".join(fcpl)
>
> try:
> - filetarget_obj = Target_File.objects.get(target =
> target_obj, path = filetarget_path)
> + filetarget_obj = Target_File.objects.get(
> + target = target_obj,
> + path = unicode(filetarget_path,
> 'utf-8'))
> except Target_File.DoesNotExist:
> # we might have an invalid link; no way to detect this.
> just set it to None
> filetarget_obj = None
> @@ -548,7 +550,7 @@ class ORMWrapper(object):
>
> tf_obj = Target_File.objects.create(
> target = target_obj,
> - path = path,
> + path = unicode(path, 'utf-8'),
> size = size,
> inodetype = Target_File.ITYPE_SYMLINK,
> permission = permission,
> --
> 2.5.5
>
> --
> _______________________________________________
> toaster mailing list
> toaster@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/toaster
>
--
Elliot Smith
Software Engineer
Intel Open Source Technology Centre
[-- Attachment #2: Type: text/html, Size: 4466 bytes --]
^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH] buildinfohelper: work around unicode exceptions
@ 2016-04-06 12:50 Elliot Smith
0 siblings, 0 replies; 3+ messages in thread
From: Elliot Smith @ 2016-04-06 12:50 UTC (permalink / raw)
To: bitbake-devel; +Cc: Joshua Lock
From: Joshua Lock <joshua.g.lock@intel.com>
We have been seeing UnicodeDecodeErrors when handling the
ImagePkgList MetadataEvent in ORMWrapper's
save_target_file_information() if the event includes filenames
that include non-ASCII characters.
In the short term work around this by converting paths to the
unicode type when passing them to Django's ORM. This is a bit of
a hack but it's too late in the cycle to do anything more invasive.
[YOCTO #9142]
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
---
bitbake/lib/bb/ui/buildinfohelper.py | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/bitbake/lib/bb/ui/buildinfohelper.py b/bitbake/lib/bb/ui/buildinfohelper.py
index ed43f4c..c211ec6 100644
--- a/bitbake/lib/bb/ui/buildinfohelper.py
+++ b/bitbake/lib/bb/ui/buildinfohelper.py
@@ -483,7 +483,7 @@ class ORMWrapper(object):
parent_obj = self._cached_get(Target_File, target = target_obj, path = parent_path, inodetype = Target_File.ITYPE_DIRECTORY)
tf_obj = Target_File.objects.create(
target = target_obj,
- path = path,
+ path = unicode(path, 'utf-8'),
size = size,
inodetype = Target_File.ITYPE_DIRECTORY,
permission = permission,
@@ -508,7 +508,7 @@ class ORMWrapper(object):
tf_obj = Target_File.objects.create(
target = target_obj,
- path = path,
+ path = unicode(path, 'utf-8'),
size = size,
inodetype = inodetype,
permission = permission,
@@ -539,7 +539,9 @@ class ORMWrapper(object):
filetarget_path = "/".join(fcpl)
try:
- filetarget_obj = Target_File.objects.get(target = target_obj, path = filetarget_path)
+ filetarget_obj = Target_File.objects.get(
+ target = target_obj,
+ path = unicode(filetarget_path, 'utf-8'))
except Target_File.DoesNotExist:
# we might have an invalid link; no way to detect this. just set it to None
filetarget_obj = None
@@ -548,7 +550,7 @@ class ORMWrapper(object):
tf_obj = Target_File.objects.create(
target = target_obj,
- path = path,
+ path = unicode(path, 'utf-8'),
size = size,
inodetype = Target_File.ITYPE_SYMLINK,
permission = permission,
--
1.9.3
---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2016-04-06 12:51 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-04-05 11:02 [PATCH] buildinfohelper: work around unicode exceptions Joshua Lock
2016-04-06 12:51 ` Smith, Elliot
2016-04-06 12:50 Elliot Smith
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.