Patches for 9808

Patches for 9808

[Reyna, David]

[-- Attachment #1: Type: text/plain, Size: 651 bytes --]

Hi Libby,

I found what I think are the two patches for 9808:

1) The original patch, at poky-contrib:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=libby/toaster_tests&id=ff71f577d0d8f67591dea017410f5c39ecd6ac85

2) A second patch that cleans up formatting in the test ID names, at Patchworks:

https://patchwork.openembedded.org/patch/137605/


Is this the full set? I just want to confirm I have all the content, because I could not find the original patch in my Toaster mail box.

I look forward to the test instructions for using 9808, that will be a help to confirm I have it correctly.

Thanks!
David



[-- Attachment #2: Type: text/html, Size: 3223 bytes --]

Re: Patches for 9808

[Cruz, Libertad]

Hello David,

Sorry for the delay. I hate to say and I am embarrassed to say that I totally forgot the whole instructions on how to run the test cases. So, spend some time fixing the errors shown because of missing steps in the instructions.

But here they are...


First I did have a version of them under poky_contrib/libby. But then went out for maternity leave and Alimon started to work on them under his branch poky_contrib/alimon/toaster_tests. When I got back he had the helpers class and the initial setup with the DJANGO framework on his branch. So, I decided to import the test cases to this branch and changed them based on the comments received by Michael. So, I continued to work on Alimon's branch.


1. git clone http://git.yoctoproject.org/git/poky-contrib
2. cd poky-contrib/
3. git checkout alimon/toaster_tests
4. cd bitbake/lib/toaster/
5. export TOASTER_TESTS_BROWSER='firefox'
6. export DJANGO_SETTINGS_MODULE='toastermain.settings_test'
7. /manage.py test tests.functional


Again sorry for the delay. 



Best Regards
Libby G.
________________________________________
From: Reyna, David [david.reyna@windriver.com]
Sent: Wednesday, April 05, 2017 10:27 AM
To: Cruz, Libertad




Cc: toaster@yoctoproject.org
Subject: Patches for 9808

Hi Libby,

I found what I think are the two patches for 9808:

1) The original patch, at poky-contrib:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=libby/toaster_tests&id=ff71f577d0d8f67591dea017410f5c39ecd6ac85

2) A second patch that cleans up formatting in the test ID names, at Patchworks:

https://patchwork.openembedded.org/patch/137605/


Is this the full set? I just want to confirm I have all the content, because I could not find the original patch in my Toaster mail box.

I look forward to the test instructions for using 9808, that will be a help to confirm I have it correctly.

Thanks!
David

Re: Patches for 9808

[Reyna, David]

[-- Attachment #1: Type: text/plain, Size: 3270 bytes --]

Hi Libby,

Thank you for the information.

1) I have examined "alimon/toaster_tests" and it appears that the cumulative change is:

  * Separate out "load_build_environment ()" in "builds/buildtest.py"
  * Add "functional/functional_helpers.py"
  * Add "functional/test_functional_basic.py"
  * Add "functional/__init__.py"

I have made those changes to a clean installation, and the test you indicated passed (it was fun to see the browser got though all its moves!)

I am going to create a new cumulative patch, include you and Aníbal as sign offs, and send it upstream.

BTW, I did need to source "oe-init-build-env" before running the tests, because they depend on "toaster" being in the path.

2) I also ran the standard test just to see, and I got several timeout errors (see attached).

  TOASTER_TESTS_BROWSER=phantomjs bitbake/lib/toaster/manage.py test orm toastergui tests.browser

I think that this should be unrelated to the new patch, so I will not delay that. 

Perhaps the errors is related to other recent changes? Do you have ideas what is going on?

Thanks!
David



-----Original Message-----
From: Cruz, Libertad [mailto:libertad.cruz@intel.com] 
Sent: Wednesday, April 05, 2017 10:25 AM
To: Reyna, David
Cc: toaster@yoctoproject.org
Subject: RE: Patches for 9808

Hello David,

Sorry for the delay. I hate to say and I am embarrassed to say that I totally forgot the whole instructions on how to run the test cases. So, spend some time fixing the errors shown because of missing steps in the instructions.

But here they are...


First I did have a version of them under poky_contrib/libby. But then went out for maternity leave and Alimon started to work on them under his branch poky_contrib/alimon/toaster_tests. When I got back he had the helpers class and the initial setup with the DJANGO framework on his branch. So, I decided to import the test cases to this branch and changed them based on the comments received by Michael. So, I continued to work on Alimon's branch.


1. git clone http://git.yoctoproject.org/git/poky-contrib
2. cd poky-contrib/
3. git checkout alimon/toaster_tests
4. cd bitbake/lib/toaster/
5. export TOASTER_TESTS_BROWSER='firefox'
6. export DJANGO_SETTINGS_MODULE='toastermain.settings_test'
7. /manage.py test tests.functional


Again sorry for the delay. 



Best Regards
Libby G.
________________________________________
From: Reyna, David [david.reyna@windriver.com]
Sent: Wednesday, April 05, 2017 10:27 AM
To: Cruz, Libertad




Cc: toaster@yoctoproject.org
Subject: Patches for 9808

Hi Libby,

I found what I think are the two patches for 9808:

1) The original patch, at poky-contrib:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=libby/toaster_tests&id=ff71f577d0d8f67591dea017410f5c39ecd6ac85

2) A second patch that cleans up formatting in the test ID names, at Patchworks:

https://patchwork.openembedded.org/patch/137605/


Is this the full set? I just want to confirm I have all the content, because I could not find the original patch in my Toaster mail box.

I look forward to the test instructions for using 9808, that will be a help to confirm I have it correctly.

Thanks!
David



[-- Attachment #2: toaster_tests_browser_errors.txt --]
[-- Type: text/plain, Size: 8037 bytes --]

======================================================================
ERROR: test_build_recipes_columns (tests.browser.test_builddashboard_page_recipes.TestBuilddashboardPageRecipes)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/test_builddashboard_page_recipes.py", line 59, in test_build_recipes_columns
    self.wait_until_visible('#edit-columns-button')
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py", line 198, in wait_until_visible
    Wait(self.driver).until(is_visible, msg)
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py", line 107, in until
    raise TimeoutException(message)
selenium.common.exceptions.TimeoutException: Message: An element matching "#edit-columns-button" should be visible


======================================================================
ERROR: test_build_tasks_columns (tests.browser.test_builddashboard_page_tasks.TestBuilddashboardPageTasks)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/test_builddashboard_page_tasks.py", line 58, in test_build_tasks_columns
    self.wait_until_visible('#edit-columns-button')
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py", line 198, in wait_until_visible
    Wait(self.driver).until(is_visible, msg)
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py", line 107, in until
    raise TimeoutException(message)
selenium.common.exceptions.TimeoutException: Message: An element matching "#edit-columns-button" should be visible


======================================================================
ERROR: test_delete_layer (tests.browser.test_layerdetails_page.TestLayerDetailsPage)
Delete the layer
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/test_layerdetails_page.py", line 177, in test_delete_layer
    self.wait_until_visible("#delete-layer-modal")
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py", line 198, in wait_until_visible
    Wait(self.driver).until(is_visible, msg)
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py", line 107, in until
    raise TimeoutException(message)
selenium.common.exceptions.TimeoutException: Message: An element matching "#delete-layer-modal" should be visible


======================================================================
ERROR: test_states_to_failure (tests.browser.test_most_recent_builds_states.TestMostRecentBuildsStates)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/test_most_recent_builds_states.py", line 167, in test_states_to_failure
    self._check_build_states(build_request)
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/test_most_recent_builds_states.py", line 92, in _check_build_states
    element = Wait(self.driver).until(recipe_bar_updated, msg)
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py", line 107, in until
    raise TimeoutException(message)
selenium.common.exceptions.TimeoutException: Message: recipe parse progress bar should update to 50%


======================================================================
ERROR: test_states_to_success (tests.browser.test_most_recent_builds_states.TestMostRecentBuildsStates)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/test_most_recent_builds_states.py", line 149, in test_states_to_success
    self._check_build_states(build_request)
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/test_most_recent_builds_states.py", line 92, in _check_build_states
    element = Wait(self.driver).until(recipe_bar_updated, msg)
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py", line 107, in until
    raise TimeoutException(message)
selenium.common.exceptions.TimeoutException: Message: recipe parse progress bar should update to 50%


======================================================================
ERROR: test_create_new_project (tests.browser.test_new_project_page.TestNewProjectPage)
Test creating a project
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/test_new_project_page.py", line 68, in test_create_new_project
    self.click("#create-project-button")
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py", line 222, in click
    element.click()
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/webelement.py", line 72, in click
    self._execute(Command.CLICK_ELEMENT)
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/webelement.py", line 461, in _execute
    return self._parent.execute(command, params)
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/webdriver.py", line 236, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.4/dist-packages/selenium/webdriver/remote/errorhandler.py", line 192, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Error Message => 'Click failed: TypeError: 'null' is not an object (evaluating 'prjInfo.machine.name')'
 caused by Request => {"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:44713","User-Agent":"Python-urllib/3.4"},"httpVersion":"1.1","method":"POST","post":"{\"id\": \":wdc:1491426846737\", \"sessionId\": \"cd607b00-1a44-11e7-adf1-e969406af363\"}","url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/cd607b00-1a44-11e7-adf1-e969406af363/element/%3Awdc%3A1491426846737/click"}
Screenshot: available via screen


======================================================================
ERROR: test_revert_orderby (tests.browser.test_toastertable_ui.TestToasterTableUI)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/test_toastertable_ui.py", line 116, in test_revert_orderby
    self.click('#edit-columns-button')
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py", line 221, in click
    element = self.wait_until_visible(selector)
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py", line 198, in wait_until_visible
    Wait(self.driver).until(is_visible, msg)
  File "/opt/dreyna/toaster_newer/poky/bitbake/lib/toaster/tests/browser/selenium_helpers_base.py", line 107, in until
    raise TimeoutException(message)
selenium.common.exceptions.TimeoutException: Message: An element matching "#edit-columns-button" should be visible


----------------------------------------------------------------------
Ran 54 tests in 131.044s

FAILED (errors=7)
Destroying test database for alias 'default'...
[poky]$ 

Re: Patches for 9808

[Cruz, Libertad]

Hey.

Well let me tell you that in the original automated test cases done by Rumania there were comments that timeouts happened only with Chrome.  To this they added the sleep function on initializing web driver and when they changed views. Obviously this was wrong so, when they created testcases in the Django framework, done by Eliot and some others, they implemented methods like wait_until_visible, is_element_visible etc etc.  

The point is, from what I read on the documented code only Firefox seemed to not have timeout issues, this had to do with how this browser handled the selenium web driver. Maybe the problem is around this. But cant be 100% sure.



Best Regards
Libertad G.
________________________________________
From: Reyna, David [david.reyna@windriver.com]
Sent: Wednesday, April 05, 2017 4:48 PM
To: Cruz, Libertad
Cc: toaster@yoctoproject.org
Subject: RE: Patches for 9808

Hi Libby,

Thank you for the information.

1) I have examined "alimon/toaster_tests" and it appears that the cumulative change is:

  * Separate out "load_build_environment ()" in "builds/buildtest.py"
  * Add "functional/functional_helpers.py"
  * Add "functional/test_functional_basic.py"
  * Add "functional/__init__.py"

I have made those changes to a clean installation, and the test you indicated passed (it was fun to see the browser got though all its moves!)

I am going to create a new cumulative patch, include you and Aníbal as sign offs, and send it upstream.

BTW, I did need to source "oe-init-build-env" before running the tests, because they depend on "toaster" being in the path.

2) I also ran the standard test just to see, and I got several timeout errors (see attached).

  TOASTER_TESTS_BROWSER=phantomjs bitbake/lib/toaster/manage.py test orm toastergui tests.browser

I think that this should be unrelated to the new patch, so I will not delay that.

Perhaps the errors is related to other recent changes? Do you have ideas what is going on?

Thanks!
David



-----Original Message-----
From: Cruz, Libertad [mailto:libertad.cruz@intel.com]
Sent: Wednesday, April 05, 2017 10:25 AM
To: Reyna, David
Cc: toaster@yoctoproject.org
Subject: RE: Patches for 9808

Hello David,

Sorry for the delay. I hate to say and I am embarrassed to say that I totally forgot the whole instructions on how to run the test cases. So, spend some time fixing the errors shown because of missing steps in the instructions.

But here they are...


First I did have a version of them under poky_contrib/libby. But then went out for maternity leave and Alimon started to work on them under his branch poky_contrib/alimon/toaster_tests. When I got back he had the helpers class and the initial setup with the DJANGO framework on his branch. So, I decided to import the test cases to this branch and changed them based on the comments received by Michael. So, I continued to work on Alimon's branch.


1. git clone http://git.yoctoproject.org/git/poky-contrib
2. cd poky-contrib/
3. git checkout alimon/toaster_tests
4. cd bitbake/lib/toaster/
5. export TOASTER_TESTS_BROWSER='firefox'
6. export DJANGO_SETTINGS_MODULE='toastermain.settings_test'
7. /manage.py test tests.functional


Again sorry for the delay.



Best Regards
Libby G.
________________________________________
From: Reyna, David [david.reyna@windriver.com]
Sent: Wednesday, April 05, 2017 10:27 AM
To: Cruz, Libertad




Cc: toaster@yoctoproject.org
Subject: Patches for 9808

Hi Libby,

I found what I think are the two patches for 9808:

1) The original patch, at poky-contrib:

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=libby/toaster_tests&id=ff71f577d0d8f67591dea017410f5c39ecd6ac85

2) A second patch that cleans up formatting in the test ID names, at Patchworks:

https://patchwork.openembedded.org/patch/137605/


Is this the full set? I just want to confirm I have all the content, because I could not find the original patch in my Toaster mail box.

I look forward to the test instructions for using 9808, that will be a help to confirm I have it correctly.

Thanks!
David