* [review-request]Fixed old UI tests @ 2016-04-08 14:15 Mihail, StanciuX 2016-04-11 11:39 ` Smith, Elliot 2016-04-11 16:43 ` Smith, Elliot 0 siblings, 2 replies; 7+ messages in thread From: Mihail, StanciuX @ 2016-04-08 14:15 UTC (permalink / raw) To: toaster; +Cc: Hangan, ValentinX, Roman, Alexandru CostinX [-- Attachment #1: Type: text/plain, Size: 273 bytes --] Hello, We've fixed the older UI tests to work with the latest changes to IDs. You can find the code here: http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=vhangan/toaster_tests_fixed Please let us know if anything needs updating. Regards, Mihai [-- Attachment #2: Type: text/html, Size: 2339 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [review-request]Fixed old UI tests 2016-04-08 14:15 [review-request]Fixed old UI tests Mihail, StanciuX @ 2016-04-11 11:39 ` Smith, Elliot 2016-04-11 16:43 ` Smith, Elliot 1 sibling, 0 replies; 7+ messages in thread From: Smith, Elliot @ 2016-04-11 11:39 UTC (permalink / raw) To: Mihail, StanciuX; +Cc: Hangan, ValentinX, Roman, Alexandru CostinX, toaster [-- Attachment #1: Type: text/plain, Size: 2204 bytes --] I have one major comment on these tests: the use of time.sleep(X) isn't a great way of waiting for page loads. It would be much better if you used a wait_until() style API, as provided by Selenium. That way you just wait for an element to become visible on the page, or present in its source (or whatever), before continuing with the next step in the test. If you are interested in how to approach this, I've repurposed some code for Toaster which was originally written by an Intel colleague (Damien Lespiau) working on automated tests for patchwork, which you can see here: https://github.com/dlespiau/patchwork/blob/master/patchwork/tests/browser.py My variation on this code is in the Toaster code base at: http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/tree/bitbake/lib/toaster/tests/browser/selenium_helpers.py (The main differences are that I removed the screenshot method, only initialise the Selenium driver once per test class rather than once per test, and have added a couple more convenience methods like get_page_source().) You can see an example of how to use this in a Selenium test at: http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/tree/bitbake/lib/toaster/tests/browser/test_all_builds_page.py (see the test_show_tasks_with_suffix() test method, which uses wait_until_present()) I would recommend using this approach as it is much less fragile than an arbitrary sleep. If you need any assistance in figuring out how to convert to that API, please let me know. Thanks. Elliot On 8 April 2016 at 15:15, Mihail, StanciuX <stanciux.mihail@intel.com> wrote: > Hello, > > > > We’ve fixed the older UI tests to work with the latest changes to IDs. > > You can find the code here: > > > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=vhangan/toaster_tests_fixed > > > > Please let us know if anything needs updating. > > > > Regards, > > Mihai > > > > -- > _______________________________________________ > toaster mailing list > toaster@yoctoproject.org > https://lists.yoctoproject.org/listinfo/toaster > > -- Elliot Smith Software Engineer Intel Open Source Technology Centre [-- Attachment #2: Type: text/html, Size: 3891 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [review-request]Fixed old UI tests 2016-04-08 14:15 [review-request]Fixed old UI tests Mihail, StanciuX 2016-04-11 11:39 ` Smith, Elliot @ 2016-04-11 16:43 ` Smith, Elliot 2016-04-12 7:45 ` Mihail, StanciuX 2016-04-12 15:15 ` Mihail, StanciuX 1 sibling, 2 replies; 7+ messages in thread From: Smith, Elliot @ 2016-04-11 16:43 UTC (permalink / raw) To: Mihail, StanciuX; +Cc: Hangan, ValentinX, Roman, Alexandru CostinX, toaster [-- Attachment #1: Type: text/plain, Size: 1202 bytes --] Does this branch supersede vhangan/automated_toaster_ui_tests <http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=vhangan/automated_toaster_ui_tests> ? I tried to cherry pick the commits off vhangan/automated_toaster_ui_tests <http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=vhangan/automated_toaster_ui_tests> then the commits off vhangan/toaster_tests_fixed <http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=vhangan/toaster_tests_fixed>, but they conflict with each other. Thanks. Elliot On 8 April 2016 at 15:15, Mihail, StanciuX <stanciux.mihail@intel.com> wrote: > Hello, > > > > We’ve fixed the older UI tests to work with the latest changes to IDs. > > You can find the code here: > > > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=vhangan/toaster_tests_fixed > > > > Please let us know if anything needs updating. > > > > Regards, > > Mihai > > > > -- > _______________________________________________ > toaster mailing list > toaster@yoctoproject.org > https://lists.yoctoproject.org/listinfo/toaster > > -- Elliot Smith Software Engineer Intel Open Source Technology Centre [-- Attachment #2: Type: text/html, Size: 2553 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [review-request]Fixed old UI tests 2016-04-11 16:43 ` Smith, Elliot @ 2016-04-12 7:45 ` Mihail, StanciuX 2016-04-12 15:15 ` Mihail, StanciuX 1 sibling, 0 replies; 7+ messages in thread From: Mihail, StanciuX @ 2016-04-12 7:45 UTC (permalink / raw) To: Smith, Elliot; +Cc: Hangan, ValentinX, Roman, Alexandru CostinX, toaster [-- Attachment #1: Type: text/plain, Size: 1700 bytes --] Hi Elliot, There shouldn’t have been conflicts between them, aside from removing the deprecated tests. I’ll investigate and get back to you. Would it be ok if I rebase them on top of each other and give you back only one mammoth-sized commit? Mihai From: Smith, Elliot [mailto:elliot.smith@intel.com] Sent: Monday, April 11, 2016 7:43 PM To: Mihail, StanciuX Cc: toaster@yoctoproject.org; Hangan, ValentinX; Roman, Alexandru CostinX Subject: Re: [Toaster] [review-request]Fixed old UI tests Does this branch supersede vhangan/automated_toaster_ui_tests<http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=vhangan/automated_toaster_ui_tests> ? I tried to cherry pick the commits off vhangan/automated_toaster_ui_tests<http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=vhangan/automated_toaster_ui_tests> then the commits off vhangan/toaster_tests_fixed<http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=vhangan/toaster_tests_fixed>, but they conflict with each other. Thanks. Elliot On 8 April 2016 at 15:15, Mihail, StanciuX <stanciux.mihail@intel.com<mailto:stanciux.mihail@intel.com>> wrote: Hello, We’ve fixed the older UI tests to work with the latest changes to IDs. You can find the code here: http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=vhangan/toaster_tests_fixed Please let us know if anything needs updating. Regards, Mihai -- _______________________________________________ toaster mailing list toaster@yoctoproject.org<mailto:toaster@yoctoproject.org> https://lists.yoctoproject.org/listinfo/toaster -- Elliot Smith Software Engineer Intel Open Source Technology Centre [-- Attachment #2: Type: text/html, Size: 6900 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [review-request]Fixed old UI tests 2016-04-11 16:43 ` Smith, Elliot 2016-04-12 7:45 ` Mihail, StanciuX @ 2016-04-12 15:15 ` Mihail, StanciuX 2016-04-12 19:26 ` Brian Avery 1 sibling, 1 reply; 7+ messages in thread From: Mihail, StanciuX @ 2016-04-12 15:15 UTC (permalink / raw) To: Smith, Elliot; +Cc: Hangan, ValentinX, Roman, Alexandru CostinX, toaster [-- Attachment #1: Type: text/plain, Size: 1696 bytes --] Hello, I’ve resolved the conflicts here: http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=mstanciu/merged_ui_tests They should rebase over master with no issues. Please let us know if more changes are needed. Regards, Mihai From: Smith, Elliot [mailto:elliot.smith@intel.com] Sent: Monday, April 11, 2016 7:43 PM To: Mihail, StanciuX Cc: toaster@yoctoproject.org; Hangan, ValentinX; Roman, Alexandru CostinX Subject: Re: [Toaster] [review-request]Fixed old UI tests Does this branch supersede vhangan/automated_toaster_ui_tests<http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=vhangan/automated_toaster_ui_tests> ? I tried to cherry pick the commits off vhangan/automated_toaster_ui_tests<http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=vhangan/automated_toaster_ui_tests> then the commits off vhangan/toaster_tests_fixed<http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=vhangan/toaster_tests_fixed>, but they conflict with each other. Thanks. Elliot On 8 April 2016 at 15:15, Mihail, StanciuX <stanciux.mihail@intel.com<mailto:stanciux.mihail@intel.com>> wrote: Hello, We’ve fixed the older UI tests to work with the latest changes to IDs. You can find the code here: http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=vhangan/toaster_tests_fixed Please let us know if anything needs updating. Regards, Mihai -- _______________________________________________ toaster mailing list toaster@yoctoproject.org<mailto:toaster@yoctoproject.org> https://lists.yoctoproject.org/listinfo/toaster -- Elliot Smith Software Engineer Intel Open Source Technology Centre [-- Attachment #2: Type: text/html, Size: 7727 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [review-request]Fixed old UI tests 2016-04-12 15:15 ` Mihail, StanciuX @ 2016-04-12 19:26 ` Brian Avery 2016-04-13 7:15 ` Smith, Elliot 0 siblings, 1 reply; 7+ messages in thread From: Brian Avery @ 2016-04-12 19:26 UTC (permalink / raw) To: Mihail, StanciuX; +Cc: toaster, Hangan, ValentinX, Roman, Alexandru CostinX Hi, A couple of comments: 1) I strongly agree with Elliot that the time.sleep should go away. Since we are migrating alot of tests to different machines/places, assuming that we know the machine characteristics is certainly going to go poorly. The wait_until API makes more sense. 2) Selenium tests in a QA framework for tests that require actual builds make sense. For anything else that can be done with fixture data, it would be much much better to extend elliot's Django tests in bitbake/lib/toaster/tests/browser. These are fast to run since they don't need actual bitbake interactions. This means that the devs can/will run them and will also be able to keep them up to date. This way, the tests can perform a CI use for the devs and, since QA can also wrap these tests, it would let QA use them too. If we use the tests as is, it seems very very likely that they will become stale very soon, especially considering the theme changes that are ongoing at the moment. I've added the bitbake/lib/toaster/tests/browser tests to the Contribute to Toaster page so that people remember to run/keep them up to date! 3) For merging into the bitbake/lib/toaster/tests/browser tests, these need some work for readability/maintainability. -bavery an Intel employee On Tue, Apr 12, 2016 at 8:15 AM, Mihail, StanciuX <stanciux.mihail@intel.com> wrote: > Hello, > > > > I’ve resolved the conflicts here: > > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=mstanciu/merged_ui_tests > > > > They should rebase over master with no issues. > > > > Please let us know if more changes are needed. > > > > Regards, > > Mihai > > > > From: Smith, Elliot [mailto:elliot.smith@intel.com] > Sent: Monday, April 11, 2016 7:43 PM > To: Mihail, StanciuX > Cc: toaster@yoctoproject.org; Hangan, ValentinX; Roman, Alexandru CostinX > Subject: Re: [Toaster] [review-request]Fixed old UI tests > > > > Does this branch supersede vhangan/automated_toaster_ui_tests ? > > > > I tried to cherry pick the commits off vhangan/automated_toaster_ui_tests > then the commits off vhangan/toaster_tests_fixed, but they conflict with > each other. > > > > Thanks. > > Elliot > > > > On 8 April 2016 at 15:15, Mihail, StanciuX <stanciux.mihail@intel.com> > wrote: > > Hello, > > > > We’ve fixed the older UI tests to work with the latest changes to IDs. > > You can find the code here: > > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=vhangan/toaster_tests_fixed > > > > Please let us know if anything needs updating. > > > > Regards, > > Mihai > > > > > -- > _______________________________________________ > toaster mailing list > toaster@yoctoproject.org > https://lists.yoctoproject.org/listinfo/toaster > > > > > > -- > > Elliot Smith > Software Engineer > Intel Open Source Technology Centre > > > -- > _______________________________________________ > toaster mailing list > toaster@yoctoproject.org > https://lists.yoctoproject.org/listinfo/toaster > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [review-request]Fixed old UI tests 2016-04-12 19:26 ` Brian Avery @ 2016-04-13 7:15 ` Smith, Elliot 0 siblings, 0 replies; 7+ messages in thread From: Smith, Elliot @ 2016-04-13 7:15 UTC (permalink / raw) To: Brian Avery; +Cc: toaster, Hangan, ValentinX, Roman, Alexandru CostinX [-- Attachment #1: Type: text/plain, Size: 4402 bytes --] On 12 April 2016 at 20:26, Brian Avery <avery.brian@gmail.com> wrote: > A couple of comments: > 1) I strongly agree with Elliot that the time.sleep should go away. > Since we are migrating alot of tests to different machines/places, > assuming that we know the machine characteristics is certainly going > to go poorly. The wait_until API makes more sense. > I am available to assist with translating any QA tests to Toaster's Selenium test suite. I can also write a wiki page or something to provide guidance in how to do this. As Brian said, it's much quicker to run UI tests using test data, as you don't need to run entire builds. The difficulty is in setting up the test data, but most of the time I can figure out what needs to go where. The rest of it is fairly standard Selenium, though the helpers already in the code base can make your code smaller and easier to follow. They also provide the wait_unti() functionality I mentioned previously (to replace time.sleep()). Splitting the code over multiple files makes it easier to find/fix tests, too. As Brian also said, if the tests are brought into our main browser test suite, we will also help to maintain them. Let me know what I can do to help. Thanks. Elliot > > 2) Selenium tests in a QA framework for tests that require actual > builds make sense. For anything else that can be done with fixture > data, it would be much much better to extend elliot's Django tests in > bitbake/lib/toaster/tests/browser. These are fast to run since they > don't need actual bitbake interactions. This means that the devs > can/will run them and will also be able to keep them up to date. This > way, the tests can perform a CI use for the devs and, since QA can > also wrap these tests, it would let QA use them too. If we use the > tests as is, it seems very very likely that they will become stale > very soon, especially considering the theme changes that are ongoing > at the moment. I've added the bitbake/lib/toaster/tests/browser tests > to the Contribute to Toaster page so that people remember to run/keep > them up to date! > > 3) For merging into the bitbake/lib/toaster/tests/browser tests, these > need some work for readability/maintainability. > > > -bavery > an Intel employee > > On Tue, Apr 12, 2016 at 8:15 AM, Mihail, StanciuX > <stanciux.mihail@intel.com> wrote: > > Hello, > > > > > > > > I’ve resolved the conflicts here: > > > > > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=mstanciu/merged_ui_tests > > > > > > > > They should rebase over master with no issues. > > > > > > > > Please let us know if more changes are needed. > > > > > > > > Regards, > > > > Mihai > > > > > > > > From: Smith, Elliot [mailto:elliot.smith@intel.com] > > Sent: Monday, April 11, 2016 7:43 PM > > To: Mihail, StanciuX > > Cc: toaster@yoctoproject.org; Hangan, ValentinX; Roman, Alexandru > CostinX > > Subject: Re: [Toaster] [review-request]Fixed old UI tests > > > > > > > > Does this branch supersede vhangan/automated_toaster_ui_tests ? > > > > > > > > I tried to cherry pick the commits off vhangan/automated_toaster_ui_tests > > then the commits off vhangan/toaster_tests_fixed, but they conflict with > > each other. > > > > > > > > Thanks. > > > > Elliot > > > > > > > > On 8 April 2016 at 15:15, Mihail, StanciuX <stanciux.mihail@intel.com> > > wrote: > > > > Hello, > > > > > > > > We’ve fixed the older UI tests to work with the latest changes to IDs. > > > > You can find the code here: > > > > > http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=vhangan/toaster_tests_fixed > > > > > > > > Please let us know if anything needs updating. > > > > > > > > Regards, > > > > Mihai > > > > > > > > > > -- > > _______________________________________________ > > toaster mailing list > > toaster@yoctoproject.org > > https://lists.yoctoproject.org/listinfo/toaster > > > > > > > > > > > > -- > > > > Elliot Smith > > Software Engineer > > Intel Open Source Technology Centre > > > > > > -- > > _______________________________________________ > > toaster mailing list > > toaster@yoctoproject.org > > https://lists.yoctoproject.org/listinfo/toaster > > > -- Elliot Smith Software Engineer Intel Open Source Technology Centre [-- Attachment #2: Type: text/html, Size: 6867 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2016-04-13 7:16 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-04-08 14:15 [review-request]Fixed old UI tests Mihail, StanciuX 2016-04-11 11:39 ` Smith, Elliot 2016-04-11 16:43 ` Smith, Elliot 2016-04-12 7:45 ` Mihail, StanciuX 2016-04-12 15:15 ` Mihail, StanciuX 2016-04-12 19:26 ` Brian Avery 2016-04-13 7:15 ` Smith, Elliot
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.