* [Fuego] Please switch server to fuegotest.org (was RE: poll request scheduling)
@ 2020-12-16 3:29 Bird, Tim
2020-12-16 3:59 ` Srivatsan S
2020-12-16 15:46 ` Pooja Sanjay More
0 siblings, 2 replies; 8+ messages in thread
From: Bird, Tim @ 2020-12-16 3:29 UTC (permalink / raw)
To: Pooja Sanjay More, tbird20d, fuego
My Internet provider currently has some routing issues that prevent me
from accessing birdcloud.org. People outside my network have no problems,
but I can't access from my home network. I've been working to get the
issues resolved, but currently, I cannot submit jobs to the Fuego server
on birdcloud.org.
Therefore, I have transitioned my work to the main public Fuego
server on fuegotest.org.
Can you please switch your Fuego configuration to use the fuegotest.org
server, instead of birdcloud.org?
Please change the value of "server_domain" in your fuego-ro/conf/fuego.conf
file to the following (which is the original default value for a Fuego installation):
server_domain=fuegotest.org/cgi-bin
I have added the boards humanshu_wks:rpi and pptlab:rpi to that server.
There was also a board for fuegohost:rpi, which looks like it may have been
related to your lab. Can you let me know what's going on with the humanshu_wks
host entry and board? Is that someone from your team?
(If not, it could be time to roll out some security for the server.)
Let me know when you have switched your configuration, and I will send
some test requests your way. Actually, I'll post a test request on fuegotest.org
(for pptlab:rpi) to see if your host picks it up.
Thanks. Sorry about this issue.
-- Tim
> -----Original Message-----
> From: Pooja Sanjay More <pooja.sm@pathpartnertech.com>
>
> Hi Tim,
>
> Jenkins job has been added to schedule a poll request to run it for 12 hrs on a daily basis and 15 minutes wait time is given. Previously
> rpireboot.py was failing through jenkins job because of permission issues. Now jenkins is added to the dialout group so reboot is also
> working as jenkins job.
>
>
>
>
>
>
>
>
>
>
> Thanks and regards,
> Pooja More
>
>
>
>
> This message contains confidential information and is intended only for the individual(s) named. If you are not the intended recipient, you
> are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this mail and attached file/s is strictly
> prohibited. Please notify the sender immediately and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be
> secured or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The
> sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail
> transmission.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Fuego] Please switch server to fuegotest.org (was RE: poll request scheduling) 2020-12-16 3:29 [Fuego] Please switch server to fuegotest.org (was RE: poll request scheduling) Bird, Tim @ 2020-12-16 3:59 ` Srivatsan S 2020-12-16 15:46 ` Pooja Sanjay More 1 sibling, 0 replies; 8+ messages in thread From: Srivatsan S @ 2020-12-16 3:59 UTC (permalink / raw) To: Bird, Tim; +Cc: tbird20d, Pooja Sanjay More, fuego [-- Attachment #1: Type: text/plain, Size: 3823 bytes --] Tim, Replies inline... On Wed, Dec 16, 2020 at 9:00 AM Bird, Tim <Tim.Bird@sony.com> wrote: > My Internet provider currently has some routing issues that prevent me > from accessing birdcloud.org. People outside my network have no problems, > but I can't access from my home network. I've been working to get the > issues resolved, but currently, I cannot submit jobs to the Fuego server > on birdcloud.org. > > Therefore, I have transitioned my work to the main public Fuego > server on fuegotest.org. > > Can you please switch your Fuego configuration to use the fuegotest.org > server, instead of birdcloud.org? > > Please change the value of "server_domain" in your fuego-ro/conf/fuego.conf > file to the following (which is the original default value for a Fuego > installation): > > server_domain=fuegotest.org/cgi-bin [Srivatsan] Sure.. We will update you after changing the configuration. > > > I have added the boards humanshu_wks:rpi and pptlab:rpi to that server. > There was also a board for fuegohost:rpi, which looks like it may have been > related to your lab. Can you let me know what's going on with the > humanshu_wks > host entry and board? Is that someone from your team? > [Srivatsan] Yes.. This is a request from one of our team members to test the poll_requests. Thanks, Srivatsan > > (If not, it could be time to roll out some security for the server.) > > Let me know when you have switched your configuration, and I will send > some test requests your way. Actually, I'll post a test request on > fuegotest.org > (for pptlab:rpi) to see if your host picks it up. > > Thanks. Sorry about this issue. > -- Tim > > > > -----Original Message----- > > From: Pooja Sanjay More <pooja.sm@pathpartnertech.com> > > > > Hi Tim, > > > > Jenkins job has been added to schedule a poll request to run it for 12 > hrs on a daily basis and 15 minutes wait time is given. Previously > > rpireboot.py was failing through jenkins job because of permission > issues. Now jenkins is added to the dialout group so reboot is also > > working as jenkins job. > > > > > > > > > > > > > > > > > > > > > > Thanks and regards, > > Pooja More > > > > > > > > > > This message contains confidential information and is intended only for > the individual(s) named. If you are not the intended recipient, you > > are notified that disclosing, copying, distributing or taking any action > in reliance on the contents of this mail and attached file/s is strictly > > prohibited. Please notify the sender immediately and delete this e-mail > from your system. E-mail transmission cannot be guaranteed to be > > secured or error-free as information could be intercepted, corrupted, > lost, destroyed, arrive late or incomplete, or contain viruses. The > > sender therefore does not accept liability for any errors or omissions > in the contents of this message, which arise as a result of e-mail > > transmission. > > _______________________________________________ > Fuego mailing list > Fuego@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/fuego > -- This message contains confidential information and is intended only for the individual(s) named. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this mail and attached file/s is strictly prohibited. Please notify the sender immediately and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secured or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. [-- Attachment #2: Type: text/html, Size: 5674 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Fuego] Please switch server to fuegotest.org (was RE: poll request scheduling) 2020-12-16 3:29 [Fuego] Please switch server to fuegotest.org (was RE: poll request scheduling) Bird, Tim 2020-12-16 3:59 ` Srivatsan S @ 2020-12-16 15:46 ` Pooja Sanjay More 2020-12-21 13:13 ` Pooja Sanjay More 1 sibling, 1 reply; 8+ messages in thread From: Pooja Sanjay More @ 2020-12-16 15:46 UTC (permalink / raw) To: Bird, Tim; +Cc: tbird20d, fuego [-- Attachment #1: Type: text/plain, Size: 3776 bytes --] Hi Tim, We have configured the fuego to fuegotest.org server. We tried put-request and run-request for rpi board after configuring it to fuegotest.org, it's working fine. But the test run does not create a link for test result object on fserver. We have scheduled a poll request for the rpi board. Thanks and Regards, Pooja More On Wed, Dec 16, 2020 at 8:59 AM Bird, Tim <Tim.Bird@sony.com> wrote: > My Internet provider currently has some routing issues that prevent me > from accessing birdcloud.org. People outside my network have no problems, > but I can't access from my home network. I've been working to get the > issues resolved, but currently, I cannot submit jobs to the Fuego server > on birdcloud.org. > > Therefore, I have transitioned my work to the main public Fuego > server on fuegotest.org. > > Can you please switch your Fuego configuration to use the fuegotest.org > server, instead of birdcloud.org? > > Please change the value of "server_domain" in your fuego-ro/conf/fuego.conf > file to the following (which is the original default value for a Fuego > installation): > > server_domain=fuegotest.org/cgi-bin > > I have added the boards humanshu_wks:rpi and pptlab:rpi to that server. > There was also a board for fuegohost:rpi, which looks like it may have been > related to your lab. Can you let me know what's going on with the > humanshu_wks > host entry and board? Is that someone from your team? > > (If not, it could be time to roll out some security for the server.) > > Let me know when you have switched your configuration, and I will send > some test requests your way. Actually, I'll post a test request on > fuegotest.org > (for pptlab:rpi) to see if your host picks it up. > > Thanks. Sorry about this issue. > -- Tim > > > > -----Original Message----- > > From: Pooja Sanjay More <pooja.sm@pathpartnertech.com> > > > > Hi Tim, > > > > Jenkins job has been added to schedule a poll request to run it for 12 > hrs on a daily basis and 15 minutes wait time is given. Previously > > rpireboot.py was failing through jenkins job because of permission > issues. Now jenkins is added to the dialout group so reboot is also > > working as jenkins job. > > > > > > > > > > > > > > > > > > > > > > Thanks and regards, > > Pooja More > > > > > > > > > > This message contains confidential information and is intended only for > the individual(s) named. If you are not the intended recipient, you > > are notified that disclosing, copying, distributing or taking any action > in reliance on the contents of this mail and attached file/s is strictly > > prohibited. Please notify the sender immediately and delete this e-mail > from your system. E-mail transmission cannot be guaranteed to be > > secured or error-free as information could be intercepted, corrupted, > lost, destroyed, arrive late or incomplete, or contain viruses. The > > sender therefore does not accept liability for any errors or omissions > in the contents of this message, which arise as a result of e-mail > > transmission. > > -- This message contains confidential information and is intended only for the individual(s) named. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this mail and attached file/s is strictly prohibited. Please notify the sender immediately and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secured or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. [-- Attachment #2: Type: text/html, Size: 5229 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Fuego] Please switch server to fuegotest.org (was RE: poll request scheduling) 2020-12-16 15:46 ` Pooja Sanjay More @ 2020-12-21 13:13 ` Pooja Sanjay More 2020-12-22 23:32 ` Tim.Bird 0 siblings, 1 reply; 8+ messages in thread From: Pooja Sanjay More @ 2020-12-21 13:13 UTC (permalink / raw) To: Bird, Tim; +Cc: tbird20d, fuego [-- Attachment #1: Type: text/plain, Size: 6012 bytes --] Hi Tim, Run test still doesn't update the result object on fserver. *poll request log:* root@cpu-366U:/# fuego-core/scripts/poll_requests.sh == Polling server "fuegotest.org/cgi-bin" for requests == Processing requests for any board Waiting 60 seconds between polls of the server. Running forever (no specified duration) Type Ctrl-C to exit Checking Running request: request-2020-12-21_11:22:58.84-pptlab:rpi Trying to get request 'request-2020-12-21_11:22:58.84-pptlab:rpi' from server Executing test Functional.hello_world on board rpi (using spec default) Running test 'Functional.hello_world' on board 'rpi' using spec 'default' ===== doing fuego phase: pre_test ===== ===== doing fuego phase: build ===== The test is already built Fuego test_build duration=0 seconds ===== doing fuego phase: deploy ===== ===== doing fuego phase: snapshot ===== Firmware revision: 5.4.83-v7-51+ ===== doing fuego phase: run ===== ------------------------------------------------- Hello World! This is a test program, to demonstrate test success and failure Test result: SUCCESS ------------------------------------------------- ===== doing fuego phase: post_test ===== ===== doing fuego phase: processing ===== log_compare: pattern 'SUCCESS' found 1 times (expected greater or equal than 1) chart config not found. Using default values. Fuego: requested test phases complete! Packaging run 'Functional.hello_world-default-104-rpi' run/ run/machine-snapshot.txt run/syslog.before.txt run/devlog.txt run/testlog.txt run/run.json run/syslog.after.txt run/prolog.sh run/spec.json run/build.xml run/consolelog.txt Run packaged successfully, and is at: /tmp/run-Functional.hello_world-default-104-on-pptlab:rpi.frp Can't put run to server Server returned message: Run Functional.hello_world-default-104-rpi put to server Checking Waiting .................................. Also we have installed "ttc" outside the container and configured it for basic commands like console and login, it's working for rpi board. Thanks and regards, Pooja More On Wed, Dec 16, 2020 at 9:16 PM Pooja Sanjay More < pooja.sm@pathpartnertech.com> wrote: > Hi Tim, > > We have configured the fuego to fuegotest.org server. > We tried put-request and run-request for rpi board after > configuring it to fuegotest.org, it's working fine. > But the test run does not create a link for test result object on fserver. > We have scheduled a poll request for the rpi board. > > > Thanks and Regards, > Pooja More > > > On Wed, Dec 16, 2020 at 8:59 AM Bird, Tim <Tim.Bird@sony.com> wrote: > >> My Internet provider currently has some routing issues that prevent me >> from accessing birdcloud.org. People outside my network have no >> problems, >> but I can't access from my home network. I've been working to get the >> issues resolved, but currently, I cannot submit jobs to the Fuego server >> on birdcloud.org. >> >> Therefore, I have transitioned my work to the main public Fuego >> server on fuegotest.org. >> >> Can you please switch your Fuego configuration to use the fuegotest.org >> server, instead of birdcloud.org? >> >> Please change the value of "server_domain" in your >> fuego-ro/conf/fuego.conf >> file to the following (which is the original default value for a Fuego >> installation): >> >> server_domain=fuegotest.org/cgi-bin >> >> I have added the boards humanshu_wks:rpi and pptlab:rpi to that server. >> There was also a board for fuegohost:rpi, which looks like it may have >> been >> related to your lab. Can you let me know what's going on with the >> humanshu_wks >> host entry and board? Is that someone from your team? >> >> (If not, it could be time to roll out some security for the server.) >> >> Let me know when you have switched your configuration, and I will send >> some test requests your way. Actually, I'll post a test request on >> fuegotest.org >> (for pptlab:rpi) to see if your host picks it up. >> >> Thanks. Sorry about this issue. >> -- Tim >> >> >> > -----Original Message----- >> > From: Pooja Sanjay More <pooja.sm@pathpartnertech.com> >> > >> > Hi Tim, >> > >> > Jenkins job has been added to schedule a poll request to run it for 12 >> hrs on a daily basis and 15 minutes wait time is given. Previously >> > rpireboot.py was failing through jenkins job because of permission >> issues. Now jenkins is added to the dialout group so reboot is also >> > working as jenkins job. >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > Thanks and regards, >> > Pooja More >> > >> > >> > >> > >> > This message contains confidential information and is intended only for >> the individual(s) named. If you are not the intended recipient, you >> > are notified that disclosing, copying, distributing or taking any >> action in reliance on the contents of this mail and attached file/s is >> strictly >> > prohibited. Please notify the sender immediately and delete this e-mail >> from your system. E-mail transmission cannot be guaranteed to be >> > secured or error-free as information could be intercepted, corrupted, >> lost, destroyed, arrive late or incomplete, or contain viruses. The >> > sender therefore does not accept liability for any errors or omissions >> in the contents of this message, which arise as a result of e-mail >> > transmission. >> >> -- This message contains confidential information and is intended only for the individual(s) named. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this mail and attached file/s is strictly prohibited. Please notify the sender immediately and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secured or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. [-- Attachment #2: Type: text/html, Size: 8174 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Fuego] Please switch server to fuegotest.org (was RE: poll request scheduling) 2020-12-21 13:13 ` Pooja Sanjay More @ 2020-12-22 23:32 ` Tim.Bird 2020-12-24 14:44 ` Pooja Sanjay More 0 siblings, 1 reply; 8+ messages in thread From: Tim.Bird @ 2020-12-22 23:32 UTC (permalink / raw) To: pooja.sm; +Cc: tbird20d, fuego Pooja, Thanks for the updates. I spent quite a bit of time trying to debug this. From what I can tell, this is a problem on the client side. The CGI script (fserver.py) calls the python module 'cgi' to parse the form data for the POST operation which uploads the run data to the server. Inside the cgi module, in the cgi.FieldStorage() class initializer, the code reads from the CGI program's standard input. In some cases, the standard input appears truncated, and the code cannot read the full data in order to create the file. When this happens, the code returns the a message about not being able to find the 'file1' attribute in order to process the file. In other cases, the standard input appears to be completely empty. This causes the cgi.FieldStorage() to block internally on a read request. Apparently in this case the web server eventually times out the request, and the client receives no result data, which ends up showing the message: Server returned message: (with no message). I did notice that the 'requests' module used in your setup is different from the one I'm using. The client string for mine is: "python-requests/2.12.4", whereas the client string for your submissions is: "python-requests/2.25.1". I don't think this should make a difference, but if you could try installing the specific 2.12.4 version of the 'requests' python module, it would help us eliminate that difference between the two setups. My other theory is that there's a firewall that's messing up the data (somehow truncating the http request body in the POST message) between the client and server. I don't have the capability to sniff the network traffic at the server. Can you try to run wireshark or some other packet tracer, and see if the data being sent by ftc to the network looks correct? The http request should include the data for the packaged run file, in a multipart section in the body of the request. Finally - I saw one other thing that was quite strange. When the server CGI script hangs (blocks reading the request data), I would have expected that 'ftc' would wait for the response before proceeding. But I often see the next operation from ftc (after the put_run operation), which is an update_request, come to the server within less than a second. I'm not sure why this is, and I have no explanation for it. The calls to the 'request' module by ftc should be completely synchronous - so I should not be seeing the update_request operation at the server until the client times out on the previous 'put_run' request. Anyway - I've run out of time to work on this for now, and I don't really have a solution. Based on what I've seen so far, I don't think this is a server-side issue. Please check on your side and see if the version of the request module, something to do with the firewall, or examining the network packet data, can provide any additional insights into the problem. -- Tim > -----Original Message----- > From: Pooja Sanjay More <pooja.sm@pathpartnertech.com> > Hi Tim, > > Run test still doesn't update the result object on fserver. > > poll request log: > > root@cpu-366U:/# fuego-core/scripts/poll_requests.sh > == Polling server "fuegotest.org/cgi-bin <http://fuegotest.org/cgi-bin> " for requests == > Processing requests for any board > Waiting 60 seconds between polls of the server. > Running forever (no specified duration) > Type Ctrl-C to exit > Checking > Running request: request-2020-12-21_11:22:58.84-pptlab:rpi > Trying to get request 'request-2020-12-21_11:22:58.84-pptlab:rpi' from server > Executing test Functional.hello_world on board rpi (using spec default) > Running test 'Functional.hello_world' on board 'rpi' using spec 'default' > ===== doing fuego phase: pre_test ===== > ===== doing fuego phase: build ===== > The test is already built > Fuego test_build duration=0 seconds > ===== doing fuego phase: deploy ===== > ===== doing fuego phase: snapshot ===== > Firmware revision: 5.4.83-v7-51+ > ===== doing fuego phase: run ===== > ------------------------------------------------- > Hello World! > > This is a test program, to demonstrate test success and failure > Test result: SUCCESS > ------------------------------------------------- > ===== doing fuego phase: post_test ===== > ===== doing fuego phase: processing ===== > log_compare: pattern 'SUCCESS' found 1 times (expected greater or equal than 1) > chart config not found. Using default values. > Fuego: requested test phases complete! > Packaging run 'Functional.hello_world-default-104-rpi' > run/ > run/machine-snapshot.txt > run/syslog.before.txt > run/devlog.txt > run/testlog.txt > run/run.json > run/syslog.after.txt > run/prolog.sh > run/spec.json > run/build.xml > run/consolelog.txt > Run packaged successfully, and is at: /tmp/run-Functional.hello_world-default-104-on-pptlab:rpi.frp > Can't put run to server > Server returned message: > Run Functional.hello_world-default-104-rpi put to server > Checking Waiting .................................. > > > Also we have installed "ttc" outside the container and configured it for basic commands like > > console and login, it's working for rpi board. > > > > Thanks and regards, > Pooja More > > > On Wed, Dec 16, 2020 at 9:16 PM Pooja Sanjay More <pooja.sm@pathpartnertech.com <mailto:pooja.sm@pathpartnertech.com> > > wrote: > > > Hi Tim, > > We have configured the fuego to fuegotest.org <http://fuegotest.org> server. > We tried put-request and run-request for rpi board after > configuring it to fuegotest.org <http://fuegotest.org> , it's working fine. > But the test run does not create a link for test result object on fserver. > We have scheduled a poll request for the rpi board. > > > Thanks and Regards, > Pooja More > > > On Wed, Dec 16, 2020 at 8:59 AM Bird, Tim <Tim.Bird@sony.com <mailto:Tim.Bird@sony.com> > wrote: > > > My Internet provider currently has some routing issues that prevent me > from accessing birdcloud.org <http://birdcloud.org> . People outside my network have no problems, > but I can't access from my home network. I've been working to get the > issues resolved, but currently, I cannot submit jobs to the Fuego server > on birdcloud.org <http://birdcloud.org> . > > Therefore, I have transitioned my work to the main public Fuego > server on fuegotest.org <http://fuegotest.org> . > > Can you please switch your Fuego configuration to use the fuegotest.org <http://fuegotest.org> > server, instead of birdcloud.org <http://birdcloud.org> ? > > Please change the value of "server_domain" in your fuego-ro/conf/fuego.conf > file to the following (which is the original default value for a Fuego installation): > > server_domain=fuegotest.org/cgi-bin <http://fuegotest.org/cgi-bin> > > I have added the boards humanshu_wks:rpi and pptlab:rpi to that server. > There was also a board for fuegohost:rpi, which looks like it may have been > related to your lab. Can you let me know what's going on with the humanshu_wks > host entry and board? Is that someone from your team? > > (If not, it could be time to roll out some security for the server.) > > Let me know when you have switched your configuration, and I will send > some test requests your way. Actually, I'll post a test request on fuegotest.org <http://fuegotest.org> > (for pptlab:rpi) to see if your host picks it up. > > Thanks. Sorry about this issue. > -- Tim > > > > -----Original Message----- > > From: Pooja Sanjay More <pooja.sm@pathpartnertech.com <mailto:pooja.sm@pathpartnertech.com> > > > > > Hi Tim, > > > > Jenkins job has been added to schedule a poll request to run it for 12 hrs on a daily basis and 15 minutes wait time is > given. Previously > > rpireboot.py was failing through jenkins job because of permission issues. Now jenkins is added to the dialout group so > reboot is also > > working as jenkins job. > > > > > > > > > > > > > > > > > > > > > > Thanks and regards, > > Pooja More > > > > > > > > > > This message contains confidential information and is intended only for the individual(s) named. If you are not the > intended recipient, you > > are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this mail and > attached file/s is strictly > > prohibited. Please notify the sender immediately and delete this e-mail from your system. E-mail transmission cannot be > guaranteed to be > > secured or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or > contain viruses. The > > sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a > result of e-mail > > transmission. > > > > > This message contains confidential information and is intended only for the individual(s) named. If you are not the intended recipient, you > are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this mail and attached file/s is strictly > prohibited. Please notify the sender immediately and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be > secured or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The > sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail > transmission. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Fuego] Please switch server to fuegotest.org (was RE: poll request scheduling) 2020-12-22 23:32 ` Tim.Bird @ 2020-12-24 14:44 ` Pooja Sanjay More 2021-01-04 22:32 ` Tim.Bird 0 siblings, 1 reply; 8+ messages in thread From: Pooja Sanjay More @ 2020-12-24 14:44 UTC (permalink / raw) To: Bird, Tim; +Cc: tbird20d, fuego [-- Attachment #1.1.1: Type: text/plain, Size: 14204 bytes --] Hi Tim, We checked with the IT team on the result upload issue and it was indeed a firewall configuration which was blocking it. This issue has been resolved now and we have successfully verified the result object updates on the server. Logs for a test : root@cpu-366U:~# ftc put-request pptlab:rpi Functional.hello_world Request was accepted by the server. request_id=request-2020-12-24_12:39:23.93-pptlab:rpi root@cpu-366U:~# ftc run-request --put-run request-2020-12-24_12:39:23.93-pptlab:rpi Trying to get request 'request-2020-12-24_12:39:23.93-pptlab:rpi' from server Executing test Functional.hello_world on board rpi (using spec default) Running test 'Functional.hello_world' on board 'rpi' using spec 'default' ===== doing fuego phase: pre_test ===== ===== doing fuego phase: build ===== The test is already built Fuego test_build duration=0 seconds ===== doing fuego phase: deploy ===== ===== doing fuego phase: snapshot ===== Firmware revision: 5.4.83-v7-52+ ===== doing fuego phase: run ===== ------------------------------------------------- Hello World! This is a test program, to demonstrate test success and failure Test result: SUCCESS ------------------------------------------------- ===== doing fuego phase: post_test ===== ===== doing fuego phase: processing ===== log_compare: pattern 'SUCCESS' found 1 times (expected greater or equal than 1) chart config not found. Using default values. Fuego: requested test phases complete! Packaging run 'Functional.hello_world-default-130-rpi' run/ run/machine-snapshot.txt run/syslog.before.txt run/devlog.txt run/testlog.txt run/run.json run/syslog.after.txt run/prolog.sh run/spec.json run/build.xml run/consolelog.txt Run packaged successfully, and is at: /tmp/run-Functional.hello_world-default-130-on-pptlab:rpi.frp Run package run-Functional.hello_world-default-130-on-pptlab:rpi.frp was accepted by the server. Run Functional.hello_world-default-130-rpi put to server Fserver Output:[image: image.png] [image: image.png] We ran multiple tests and found that when there are tests which do not have test.yaml files give an error for "eprint". [image: image.png] eprint() was not defined in ftc. We added it to ftc then it is working fine. Changes in ftc: diff --git a/scripts/ftc b/scripts/ftc index ae9ab2f..e70847d 100755 --- a/scripts/ftc +++ b/scripts/ftc @@ -616,6 +616,9 @@ def wprint(msg): global ll_warning if ll_warning: print("### WARNING: " + msg) +# always print errors +def eprint(msg): + print("!!! ERROR: " + msg) class config_class: def __init__(self, config_path): Patch file is attached below. *Thanks and regards:* * Pooja More* On Wed, Dec 23, 2020 at 5:02 AM <Tim.Bird@sony.com> wrote: > Pooja, > > Thanks for the updates. > > I spent quite a bit of time trying to debug this. From what I can tell, > this is a problem on the client side. The CGI script (fserver.py) calls > the python module 'cgi' to parse the form data for the POST operation > which uploads the run data to the server. Inside the cgi module, > in the cgi.FieldStorage() class initializer, the code reads from the > CGI program's standard input. In some cases, the standard input > appears truncated, and the code cannot read the full data in order > to create the file. When this happens, the code returns the > a message about not being able to find the 'file1' attribute in > order to process the file. In other cases, the standard input appears > to be completely empty. This causes the cgi.FieldStorage() to block > internally on a read request. Apparently in this case the web > server eventually times out the request, and the client receives > no result data, which ends up showing the message: > > Server returned message: > > (with no message). > > I did notice that the 'requests' module used in your setup is different > from the > one I'm using. The client string for mine is: "python-requests/2.12.4", > whereas > the client string for your submissions is: "python-requests/2.25.1". I > don't think > this should make a difference, but if you could try installing the > specific 2.12.4 > version of the 'requests' python module, it would help us eliminate that > difference > between the two setups. My other theory is that there's a firewall that's > messing > up the data (somehow truncating the http request body in the POST message) > between the client and server. I don't have > the capability to sniff the network traffic at the server. Can you try > to run wireshark or some other packet tracer, and see if the data being > sent > by ftc to the network looks correct? The http request should include the > data for > the packaged run file, in a multipart section in the body of the request. > > Finally - I saw one other thing that was quite strange. When the server > CGI > script hangs (blocks reading the request data), I would have expected that > 'ftc' > would wait for the response before proceeding. But I often see the next > operation from ftc (after the put_run operation), which is an > update_request, > come to the server within less than a second. I'm not sure why this is, > and > I have no explanation for it. The calls to the 'request' module by ftc > should > be completely synchronous - so I should not be seeing the update_request > operation at the server until the client times out on the previous > 'put_run' > request. > > Anyway - I've run out of time to work on this for now, and I don't really > have > a solution. Based on what I've seen so far, I don't think this is a > server-side issue. > Please check on your side and see if the version of the request module, > something to do with the firewall, or examining the network packet data, > can > provide any additional insights into the problem. > > -- Tim > > > > -----Original Message----- > > From: Pooja Sanjay More <pooja.sm@pathpartnertech.com> > > Hi Tim, > > > > Run test still doesn't update the result object on fserver. > > > > poll request log: > > > > root@cpu-366U:/# fuego-core/scripts/poll_requests.sh > > == Polling server "fuegotest.org/cgi-bin <http://fuegotest.org/cgi-bin> > " for requests == > > Processing requests for any board > > Waiting 60 seconds between polls of the server. > > Running forever (no specified duration) > > Type Ctrl-C to exit > > Checking > > Running request: request-2020-12-21_11:22:58.84-pptlab:rpi > > Trying to get request 'request-2020-12-21_11:22:58.84-pptlab:rpi' from > server > > Executing test Functional.hello_world on board rpi (using spec default) > > Running test 'Functional.hello_world' on board 'rpi' using spec 'default' > > ===== doing fuego phase: pre_test ===== > > ===== doing fuego phase: build ===== > > The test is already built > > Fuego test_build duration=0 seconds > > ===== doing fuego phase: deploy ===== > > ===== doing fuego phase: snapshot ===== > > Firmware revision: 5.4.83-v7-51+ > > ===== doing fuego phase: run ===== > > ------------------------------------------------- > > Hello World! > > > > This is a test program, to demonstrate test success and failure > > Test result: SUCCESS > > ------------------------------------------------- > > ===== doing fuego phase: post_test ===== > > ===== doing fuego phase: processing ===== > > log_compare: pattern 'SUCCESS' found 1 times (expected greater or equal > than 1) > > chart config not found. Using default values. > > Fuego: requested test phases complete! > > Packaging run 'Functional.hello_world-default-104-rpi' > > run/ > > run/machine-snapshot.txt > > run/syslog.before.txt > > run/devlog.txt > > run/testlog.txt > > run/run.json > > run/syslog.after.txt > > run/prolog.sh > > run/spec.json > > run/build.xml > > run/consolelog.txt > > Run packaged successfully, and is at: > /tmp/run-Functional.hello_world-default-104-on-pptlab:rpi.frp > > Can't put run to server > > Server returned message: > > Run Functional.hello_world-default-104-rpi put to server > > Checking Waiting .................................. > > > > > > Also we have installed "ttc" outside the container and configured it for > basic commands like > > > > console and login, it's working for rpi board. > > > > > > > > Thanks and regards, > > Pooja More > > > > > > On Wed, Dec 16, 2020 at 9:16 PM Pooja Sanjay More < > pooja.sm@pathpartnertech.com <mailto:pooja.sm@pathpartnertech.com> > > > wrote: > > > > > > Hi Tim, > > > > We have configured the fuego to fuegotest.org < > http://fuegotest.org> server. > > We tried put-request and run-request for rpi board after > > configuring it to fuegotest.org <http://fuegotest.org> , it's > working fine. > > But the test run does not create a link for test result object on > fserver. > > We have scheduled a poll request for the rpi board. > > > > > > Thanks and Regards, > > Pooja More > > > > > > On Wed, Dec 16, 2020 at 8:59 AM Bird, Tim <Tim.Bird@sony.com > <mailto:Tim.Bird@sony.com> > wrote: > > > > > > My Internet provider currently has some routing issues > that prevent me > > from accessing birdcloud.org <http://birdcloud.org> . > People outside my network have no problems, > > but I can't access from my home network. I've been > working to get the > > issues resolved, but currently, I cannot submit jobs to > the Fuego server > > on birdcloud.org <http://birdcloud.org> . > > > > Therefore, I have transitioned my work to the main public > Fuego > > server on fuegotest.org <http://fuegotest.org> . > > > > Can you please switch your Fuego configuration to use the > fuegotest.org <http://fuegotest.org> > > server, instead of birdcloud.org <http://birdcloud.org> ? > > > > Please change the value of "server_domain" in your > fuego-ro/conf/fuego.conf > > file to the following (which is the original default value > for a Fuego installation): > > > > server_domain=fuegotest.org/cgi-bin < > http://fuegotest.org/cgi-bin> > > > > I have added the boards humanshu_wks:rpi and pptlab:rpi to > that server. > > There was also a board for fuegohost:rpi, which looks like > it may have been > > related to your lab. Can you let me know what's going on > with the humanshu_wks > > host entry and board? Is that someone from your team? > > > > (If not, it could be time to roll out some security for > the server.) > > > > Let me know when you have switched your configuration, and > I will send > > some test requests your way. Actually, I'll post a test > request on fuegotest.org <http://fuegotest.org> > > (for pptlab:rpi) to see if your host picks it up. > > > > Thanks. Sorry about this issue. > > -- Tim > > > > > > > -----Original Message----- > > > From: Pooja Sanjay More <pooja.sm@pathpartnertech.com > <mailto:pooja.sm@pathpartnertech.com> > > > > > > > Hi Tim, > > > > > > Jenkins job has been added to schedule a poll request > to run it for 12 hrs on a daily basis and 15 minutes wait time is > > given. Previously > > > rpireboot.py was failing through jenkins job because of > permission issues. Now jenkins is added to the dialout group so > > reboot is also > > > working as jenkins job. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks and regards, > > > Pooja More > > > > > > > > > > > > > > > This message contains confidential information and is > intended only for the individual(s) named. If you are not the > > intended recipient, you > > > are notified that disclosing, copying, distributing or > taking any action in reliance on the contents of this mail and > > attached file/s is strictly > > > prohibited. Please notify the sender immediately and > delete this e-mail from your system. E-mail transmission cannot be > > guaranteed to be > > > secured or error-free as information could be > intercepted, corrupted, lost, destroyed, arrive late or incomplete, or > > contain viruses. The > > > sender therefore does not accept liability for any > errors or omissions in the contents of this message, which arise as a > > result of e-mail > > > transmission. > > > > > > > > > > This message contains confidential information and is intended only for > the individual(s) named. If you are not the intended recipient, you > > are notified that disclosing, copying, distributing or taking any action > in reliance on the contents of this mail and attached file/s is strictly > > prohibited. Please notify the sender immediately and delete this e-mail > from your system. E-mail transmission cannot be guaranteed to be > > secured or error-free as information could be intercepted, corrupted, > lost, destroyed, arrive late or incomplete, or contain viruses. The > > sender therefore does not accept liability for any errors or omissions > in the contents of this message, which arise as a result of e-mail > > transmission. > > -- This message contains confidential information and is intended only for the individual(s) named. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this mail and attached file/s is strictly prohibited. Please notify the sender immediately and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secured or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. [-- Attachment #1.1.2: Type: text/html, Size: 19560 bytes --] [-- Attachment #1.2: image.png --] [-- Type: image/png, Size: 39370 bytes --] [-- Attachment #1.3: image.png --] [-- Type: image/png, Size: 57277 bytes --] [-- Attachment #1.4: image.png --] [-- Type: image/png, Size: 97484 bytes --] [-- Attachment #2: eprint_define.patch --] [-- Type: application/octet-stream, Size: 357 bytes --] diff --git a/scripts/ftc b/scripts/ftc index ae9ab2f..e70847d 100755 --- a/scripts/ftc +++ b/scripts/ftc @@ -616,6 +616,9 @@ def wprint(msg): global ll_warning if ll_warning: print("### WARNING: " + msg) +# always print errors +def eprint(msg): + print("!!! ERROR: " + msg) class config_class: def __init__(self, config_path): ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [Fuego] Please switch server to fuegotest.org (was RE: poll request scheduling) 2020-12-24 14:44 ` Pooja Sanjay More @ 2021-01-04 22:32 ` Tim.Bird 2021-01-06 7:08 ` Pooja Sanjay More 0 siblings, 1 reply; 8+ messages in thread From: Tim.Bird @ 2021-01-04 22:32 UTC (permalink / raw) To: pooja.sm; +Cc: tbird20d, fuego Hey everyone, I'm back from vacation. Sorry this took so long to respond to, but I was away from work since Dec. 22. Please see my comments inline below. > -----Original Message----- > From: Pooja Sanjay More <pooja.sm@pathpartnertech.com> > > Hi Tim, > > We checked with the IT team on the result upload issue and it was indeed a firewall > configuration which was blocking it. This issue has been resolved now and we have > successfully verified the result object updates on the server. That is very good to hear! Thanks for getting this resolved on your end. > > > Logs for a test : > > > root@cpu-366U:~# ftc put-request pptlab:rpi Functional.hello_world > Request was accepted by the server. > request_id=request-2020-12-24_12:39:23.93-pptlab:rpi > > root@cpu-366U:~# ftc run-request --put-run request-2020-12-24_12:39:23.93-pptlab:rpi > Trying to get request 'request-2020-12-24_12:39:23.93-pptlab:rpi' from server > Executing test Functional.hello_world on board rpi (using spec default) > Running test 'Functional.hello_world' on board 'rpi' using spec 'default' > ===== doing fuego phase: pre_test ===== > ===== doing fuego phase: build ===== > The test is already built > Fuego test_build duration=0 seconds > ===== doing fuego phase: deploy ===== > ===== doing fuego phase: snapshot ===== > Firmware revision: 5.4.83-v7-52+ > ===== doing fuego phase: run ===== > ------------------------------------------------- > Hello World! > > This is a test program, to demonstrate test success and failure > Test result: SUCCESS > ------------------------------------------------- > ===== doing fuego phase: post_test ===== > ===== doing fuego phase: processing ===== > log_compare: pattern 'SUCCESS' found 1 times (expected greater or equal than 1) > chart config not found. Using default values. > Fuego: requested test phases complete! > Packaging run 'Functional.hello_world-default-130-rpi' > run/ > run/machine-snapshot.txt > run/syslog.before.txt > run/devlog.txt > run/testlog.txt > run/run.json > run/syslog.after.txt > run/prolog.sh > run/spec.json > run/build.xml > run/consolelog.txt > Run packaged successfully, and is at: /tmp/run-Functional.hello_world-default-130-on-pptlab:rpi.frp > Run package run-Functional.hello_world-default-130-on-pptlab:rpi.frp was accepted by the server. > Run Functional.hello_world-default-130-rpi put to server This looks good. > > Fserver Output: > > > > > We ran multiple tests and found that when there are tests which do not have test.yaml files > give an error for "eprint". > > > > > > eprint() was not defined in ftc. We added it to ftc then it is working fine. > > Changes in ftc: > > diff --git a/scripts/ftc b/scripts/ftc > index ae9ab2f..e70847d 100755 > --- a/scripts/ftc > +++ b/scripts/ftc > @@ -616,6 +616,9 @@ def wprint(msg): > global ll_warning > if ll_warning: > print("### WARNING: " + msg) > +# always print errors > +def eprint(msg): > + print("!!! ERROR: " + msg) > > class config_class: > def __init__(self, config_path): > > Patch file is attached below. I applied the patch. It was not a well-formed patch for integration with the project. It was missing the author, commit text, signed-off-by line and other meta-information required for a good commit. I added this material, and added my own "Signed-off-by" line. Please note that I added a "Signed-off-by" line for you (Pooja). Please let me know if there is any problem with this. Since the code looks like it was taken from parser/common.py, I guessed that it was OK to add your signed-off-by for the patch. I found some other bugs in my testing, and added some other patches. These have all been pushed to the master branch on bitbucket. Please 'git pull' the latest material, and let me know if you see any problems. It is really nice to get these bugs fixed. Thanks for the testing and the patch!! I have some other comments that I'll put in another thread. Thanks, -- Tim ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Fuego] Please switch server to fuegotest.org (was RE: poll request scheduling) 2021-01-04 22:32 ` Tim.Bird @ 2021-01-06 7:08 ` Pooja Sanjay More 0 siblings, 0 replies; 8+ messages in thread From: Pooja Sanjay More @ 2021-01-06 7:08 UTC (permalink / raw) To: Bird, Tim; +Cc: tbird20d, fuego [-- Attachment #1: Type: text/plain, Size: 5182 bytes --] Hi, We have tested test-requests from fserver after pulling the latest changes. Tests are working fine and also updates test results on fserver. Thanks and regards, Pooja More On Tue, Jan 5, 2021 at 4:02 AM <Tim.Bird@sony.com> wrote: > Hey everyone, > > I'm back from vacation. Sorry this took so long to respond to, > but I was away from work since Dec. 22. Please see my comments > inline below. > > > -----Original Message----- > > From: Pooja Sanjay More <pooja.sm@pathpartnertech.com> > > > > Hi Tim, > > > > We checked with the IT team on the result upload issue and it was indeed > a firewall > > configuration which was blocking it. This issue has been resolved now > and we have > > successfully verified the result object updates on the server. > > That is very good to hear! Thanks for getting this resolved on your end. > > > > > > > Logs for a test : > > > > > > root@cpu-366U:~# ftc put-request pptlab:rpi Functional.hello_world > > Request was accepted by the server. > > request_id=request-2020-12-24_12:39:23.93-pptlab:rpi > > > > root@cpu-366U:~# ftc run-request --put-run > request-2020-12-24_12:39:23.93-pptlab:rpi > > Trying to get request 'request-2020-12-24_12:39:23.93-pptlab:rpi' from > server > > Executing test Functional.hello_world on board rpi (using spec default) > > Running test 'Functional.hello_world' on board 'rpi' using spec 'default' > > ===== doing fuego phase: pre_test ===== > > ===== doing fuego phase: build ===== > > The test is already built > > Fuego test_build duration=0 seconds > > ===== doing fuego phase: deploy ===== > > ===== doing fuego phase: snapshot ===== > > Firmware revision: 5.4.83-v7-52+ > > ===== doing fuego phase: run ===== > > ------------------------------------------------- > > Hello World! > > > > This is a test program, to demonstrate test success and failure > > Test result: SUCCESS > > ------------------------------------------------- > > ===== doing fuego phase: post_test ===== > > ===== doing fuego phase: processing ===== > > log_compare: pattern 'SUCCESS' found 1 times (expected greater or equal > than 1) > > chart config not found. Using default values. > > Fuego: requested test phases complete! > > Packaging run 'Functional.hello_world-default-130-rpi' > > run/ > > run/machine-snapshot.txt > > run/syslog.before.txt > > run/devlog.txt > > run/testlog.txt > > run/run.json > > run/syslog.after.txt > > run/prolog.sh > > run/spec.json > > run/build.xml > > run/consolelog.txt > > Run packaged successfully, and is at: > /tmp/run-Functional.hello_world-default-130-on-pptlab:rpi.frp > > Run package run-Functional.hello_world-default-130-on-pptlab:rpi.frp was > accepted by the server. > > Run Functional.hello_world-default-130-rpi put to server > > This looks good. > > > > > Fserver Output: > > > > > > > > > > We ran multiple tests and found that when there are tests which do not > have test.yaml files > > give an error for "eprint". > > > > > > > > > > > > eprint() was not defined in ftc. We added it to ftc then it is working > fine. > > > > Changes in ftc: > > > > diff --git a/scripts/ftc b/scripts/ftc > > index ae9ab2f..e70847d 100755 > > --- a/scripts/ftc > > +++ b/scripts/ftc > > @@ -616,6 +616,9 @@ def wprint(msg): > > global ll_warning > > if ll_warning: > > print("### WARNING: " + msg) > > +# always print errors > > +def eprint(msg): > > + print("!!! ERROR: " + msg) > > > > class config_class: > > def __init__(self, config_path): > > > > Patch file is attached below. > > I applied the patch. It was not a well-formed patch for integration with > the project. It was missing the author, commit text, signed-off-by line > and other meta-information required for a good commit. > > I added this material, and added my own "Signed-off-by" line. Please > note that I added a "Signed-off-by" line for you (Pooja). Please let > me know if there is any problem with this. Since the code looks like it > was taken from parser/common.py, I guessed that it was OK to add > your signed-off-by for the patch. > > I found some other bugs in my testing, and added some other patches. > These have all been pushed to the master branch on bitbucket. > > Please 'git pull' the latest material, and let me know if you see any > problems. > > It is really nice to get these bugs fixed. Thanks > for the testing and the patch!! > > I have some other comments that I'll put in another thread. > > Thanks, > -- Tim > > -- This message contains confidential information and is intended only for the individual(s) named. If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this mail and attached file/s is strictly prohibited. Please notify the sender immediately and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secured or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. [-- Attachment #2: Type: text/html, Size: 6484 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-01-06 7:08 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-12-16 3:29 [Fuego] Please switch server to fuegotest.org (was RE: poll request scheduling) Bird, Tim 2020-12-16 3:59 ` Srivatsan S 2020-12-16 15:46 ` Pooja Sanjay More 2020-12-21 13:13 ` Pooja Sanjay More 2020-12-22 23:32 ` Tim.Bird 2020-12-24 14:44 ` Pooja Sanjay More 2021-01-04 22:32 ` Tim.Bird 2021-01-06 7:08 ` Pooja Sanjay More
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.