toaster.lists.yoctoproject.org archive mirror
 help / color / mirror / Atom feed
* Toaster database error
@ 2018-10-03 12:53 Muhammad Shakeel Khan
  2018-10-03 15:39 ` Reyna, David
  0 siblings, 1 reply; 4+ messages in thread
From: Muhammad Shakeel Khan @ 2018-10-03 12:53 UTC (permalink / raw)
  To: toaster

[-- Attachment #1: Type: text/plain, Size: 681 bytes --]

Hi,

I have setup Toaster production instance using Rocko 2.4.2 and it works
fine usually.

Some of our builds have mono libraries and related files (with very long
paths of files to be installed) and for that build I see the following
error:

ERROR: (1406, "Data too long for column 'path' at row 1")

And with this error the Toaster fails to show correct status of a command
line build.

I have worked around this by setting Mysql non-strict mode but that doesn't
look like a good solution and may lead to other errors.

Do you think we can change the SQL entry that specifies the length of
'path' element? What is the limit on this?

Best Regards,
Shakeel

[-- Attachment #2: Type: text/html, Size: 893 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Toaster database error
  2018-10-03 12:53 Toaster database error Muhammad Shakeel Khan
@ 2018-10-03 15:39 ` Reyna, David
  2018-10-04  9:00   ` Muhammad Shakeel Khan
  0 siblings, 1 reply; 4+ messages in thread
From: Reyna, David @ 2018-10-03 15:39 UTC (permalink / raw)
  To: Muhammad Shakeel Khan, toaster

[-- Attachment #1: Type: text/plain, Size: 2253 bytes --]

Hi Shakeel,

It appears that you are referring to:
  class Target_File(models.Model):
    path = models.FilePathField()

Looking at the documentation for this field type  (https://docs.djangoproject.com/en/2.1/ref/models/fields/), I see: “FilePathField<https://docs.djangoproject.com/en/2.1/ref/models/fields/#django.db.models.FilePathField> instances are created in your database as varchar columns with a default max length of 100 characters. As with other fields, you can change the maximum length using the max_length<https://docs.djangoproject.com/en/2.1/ref/models/fields/#django.db.models.CharField.max_length> argument.”

Clearly 100 characters is very limited, as revealed in your issue. Here are the choices:

  (a) We could go with a TextField redefinition, and have no limit. The problem is that we lose the associated methods of FIlePathField. I can review the Toaster code to see if we are actually leveraging those methods, either directly in the Python code or implicitly in the template code.

  (b) Increase the “max_length” to some larger value, for example simply doubling the default to 200 characters. We could still hit that limit, but perhaps vanishingly.

Can you give guidance as to what path lengths you are requiring? That will help give us a real world use case.

Thanks,
David





From: toaster-bounces@yoctoproject.org [mailto:toaster-bounces@yoctoproject.org] On Behalf Of Muhammad Shakeel Khan
Sent: Wednesday, October 03, 2018 5:53 AM
To: toaster@yoctoproject.org
Subject: [Toaster] Toaster database error

Hi,

I have setup Toaster production instance using Rocko 2.4.2 and it works fine usually.

Some of our builds have mono libraries and related files (with very long paths of files to be installed) and for that build I see the following error:

ERROR: (1406, "Data too long for column 'path' at row 1")

And with this error the Toaster fails to show correct status of a command line build.

I have worked around this by setting Mysql non-strict mode but that doesn't look like a good solution and may lead to other errors.

Do you think we can change the SQL entry that specifies the length of 'path' element? What is the limit on this?

Best Regards,
Shakeel

[-- Attachment #2: Type: text/html, Size: 11849 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Toaster database error
  2018-10-03 15:39 ` Reyna, David
@ 2018-10-04  9:00   ` Muhammad Shakeel Khan
  2018-10-05  5:42     ` Reyna, David
  0 siblings, 1 reply; 4+ messages in thread
From: Muhammad Shakeel Khan @ 2018-10-04  9:00 UTC (permalink / raw)
  To: Reyna, David; +Cc: toaster

[-- Attachment #1: Type: text/plain, Size: 3319 bytes --]

Hi David,

Thanks for your detailed response. At this point I believe (b) is going to
be more appropriate. I have looked at the path that is generating this
error and that is in range of 120-140. So setting default to 200 should be
enough. Could you please let me know where exactly I need to make this
change? I can do a complete build after this change and let you know if
this solves the problem.

Also we need this change for database used in 'save_target_kernel_file'. I
saw the following error for a dtb file whose complete path was exceeding
100.

django.db.utils.DataError: (1406, "Data too long for column 'file_name' at
row 1")

Thanks for your help.

Best Regards,
Shakeel

On Wed, Oct 3, 2018 at 8:39 PM Reyna, David <david.reyna@windriver.com>
wrote:

> Hi Shakeel,
>
>
>
> It appears that you are referring to:
>
>   class Target_File(models.Model):
>
>     path = models.FilePathField()
>
>
>
> Looking at the documentation for this field type  (
> https://docs.djangoproject.com/en/2.1/ref/models/fields/), I see:*
> “FilePathField
> <https://docs.djangoproject.com/en/2.1/ref/models/fields/#django.db.models.FilePathField>
> instances are created in your database as **varchar** columns with a
> default max length of 100 characters. As with other fields, you can change
> the maximum length using the max_length
> <https://docs.djangoproject.com/en/2.1/ref/models/fields/#django.db.models.CharField.max_length>
> argument.”*
>
>
>
> Clearly 100 characters is very limited, as revealed in your issue. Here
> are the choices:
>
>
>
>   (a) We could go with a TextField redefinition, and have no limit. The
> problem is that we lose the associated methods of FIlePathField. I can
> review the Toaster code to see if we are actually leveraging those methods,
> either directly in the Python code or implicitly in the template code.
>
>
>
>   (b) Increase the “max_length” to some larger value, for example simply
> doubling the default to 200 characters. We could still hit that limit, but
> perhaps vanishingly.
>
>
>
> Can you give guidance as to what path lengths you are requiring? That will
> help give us a real world use case.
>
>
>
> Thanks,
>
> David
>
>
>
>
>
>
>
>
>
>
>
> *From:* toaster-bounces@yoctoproject.org [mailto:
> toaster-bounces@yoctoproject.org] *On Behalf Of *Muhammad Shakeel Khan
> *Sent:* Wednesday, October 03, 2018 5:53 AM
> *To:* toaster@yoctoproject.org
> *Subject:* [Toaster] Toaster database error
>
>
>
> Hi,
>
>
>
> I have setup Toaster production instance using Rocko 2.4.2 and it works
> fine usually.
>
>
>
> Some of our builds have mono libraries and related files (with very long
> paths of files to be installed) and for that build I see the following
> error:
>
>
>
> ERROR: (1406, "Data too long for column 'path' at row 1")
>
>
>
> And with this error the Toaster fails to show correct status of a command
> line build.
>
>
>
> I have worked around this by setting Mysql non-strict mode but that
> doesn't look like a good solution and may lead to other errors.
>
>
>
> Do you think we can change the SQL entry that specifies the length of
> 'path' element? What is the limit on this?
>
>
>
> Best Regards,
>
> Shakeel
>

[-- Attachment #2: Type: text/html, Size: 9002 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: Toaster database error
  2018-10-04  9:00   ` Muhammad Shakeel Khan
@ 2018-10-05  5:42     ` Reyna, David
  0 siblings, 0 replies; 4+ messages in thread
From: Reyna, David @ 2018-10-05  5:42 UTC (permalink / raw)
  To: Muhammad Shakeel Khan; +Cc: toaster

[-- Attachment #1: Type: text/plain, Size: 5218 bytes --]

Hi Shakeel

Here is a high level description on how one would make this change.

1. Make these changes in your 'models.py':

class Target_File(models.Model):
    path = models.FilePathField(max_length=254)

class TargetKernelFile(models.Model):
    file_name = models.FilePathField(max_length=254)

class TargetSDKFile(models.Model):
    file_name = models.FilePathField(max_length=254)

2. Create a new migration file under "orm/migrations" with the following changes:

  operations = [
    migrations.AlterField('target_file', 'path', models.TextField(max_length=254))
  ]
  operations = [
    migrations.AlterField('targetkernelfile', 'file_name', models.TextField(max_length=254))
  ]
  operations = [
    migrations.AlterField('targetsdkfile', 'file_name', models.TextField(max_length=254))
  ]

You would create a new migration file numbered after the last migration file already there, and set your new migration file dependent on that last one. There are many examples of migration files there.

3. Restart Toaster

What should happen is that when Toaster starts up it will see the new migration file, apply it, and continue. This is fair simple change, so it should work, but I would try first with a blank database, and before trying a live migration on an existing database back up your toaster.sqlite first.

4. As for ‘save_target_kernel_file’, that is here:

Bitbake/lib/bb/ui/buildinfohelper.py:
    def save_target_kernel_file(self, target_obj, file_name, file_size):
        """
        Save kernel file (bzImage, modules*) information for a Target target_obj.
        """
        TargetKernelFile.objects.create(target=target_obj,
            file_name=file_name, file_size=file_size)

This is why would want to update the TargetKernelFile class as well.

- David


From: Muhammad Shakeel Khan [mailto:email.mshakeel@gmail.com]
Sent: Thursday, October 04, 2018 2:01 AM
To: Reyna, David
Cc: toaster@yoctoproject.org
Subject: Re: [Toaster] Toaster database error

Hi David,

Thanks for your detailed response. At this point I believe (b) is going to be more appropriate. I have looked at the path that is generating this error and that is in range of 120-140. So setting default to 200 should be enough. Could you please let me know where exactly I need to make this change? I can do a complete build after this change and let you know if this solves the problem.

Also we need this change for database used in 'save_target_kernel_file'. I saw the following error for a dtb file whose complete path was exceeding 100.

django.db.utils.DataError: (1406, "Data too long for column 'file_name' at row 1")

Thanks for your help.

Best Regards,
Shakeel

On Wed, Oct 3, 2018 at 8:39 PM Reyna, David <david.reyna@windriver.com<mailto:david.reyna@windriver.com>> wrote:
Hi Shakeel,

It appears that you are referring to:
  class Target_File(models.Model):
    path = models.FilePathField()

Looking at the documentation for this field type  (https://docs.djangoproject.com/en/2.1/ref/models/fields/), I see: “FilePathField<https://docs.djangoproject.com/en/2.1/ref/models/fields/#django.db.models.FilePathField> instances are created in your database as varchar columns with a default max length of 100 characters. As with other fields, you can change the maximum length using the max_length<https://docs.djangoproject.com/en/2.1/ref/models/fields/#django.db.models.CharField.max_length> argument.”

Clearly 100 characters is very limited, as revealed in your issue. Here are the choices:

  (a) We could go with a TextField redefinition, and have no limit. The problem is that we lose the associated methods of FIlePathField. I can review the Toaster code to see if we are actually leveraging those methods, either directly in the Python code or implicitly in the template code.

  (b) Increase the “max_length” to some larger value, for example simply doubling the default to 200 characters. We could still hit that limit, but perhaps vanishingly.

Can you give guidance as to what path lengths you are requiring? That will help give us a real world use case.

Thanks,
David





From: toaster-bounces@yoctoproject.org<mailto:toaster-bounces@yoctoproject.org> [mailto:toaster-bounces@yoctoproject.org<mailto:toaster-bounces@yoctoproject.org>] On Behalf Of Muhammad Shakeel Khan
Sent: Wednesday, October 03, 2018 5:53 AM
To: toaster@yoctoproject.org<mailto:toaster@yoctoproject.org>
Subject: [Toaster] Toaster database error

Hi,

I have setup Toaster production instance using Rocko 2.4.2 and it works fine usually.

Some of our builds have mono libraries and related files (with very long paths of files to be installed) and for that build I see the following error:

ERROR: (1406, "Data too long for column 'path' at row 1")

And with this error the Toaster fails to show correct status of a command line build.

I have worked around this by setting Mysql non-strict mode but that doesn't look like a good solution and may lead to other errors.

Do you think we can change the SQL entry that specifies the length of 'path' element? What is the limit on this?

Best Regards,
Shakeel

[-- Attachment #2: Type: text/html, Size: 23181 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2018-10-05  5:43 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-10-03 12:53 Toaster database error Muhammad Shakeel Khan
2018-10-03 15:39 ` Reyna, David
2018-10-04  9:00   ` Muhammad Shakeel Khan
2018-10-05  5:42     ` Reyna, David

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).