From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-ua1-f43.google.com (mail-ua1-f43.google.com [209.85.222.43]) by mx.groups.io with SMTP id smtpd.web10.14332.1622236216133408160 for ; Fri, 28 May 2021 14:10:16 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20161025 header.b=p8ifETeB; spf=pass (domain: gmail.com, ip: 209.85.222.43, mailfrom: alex.kanavin@gmail.com) Received: by mail-ua1-f43.google.com with SMTP id x1so2886512uau.1 for ; Fri, 28 May 2021 14:10:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=4+VRHOjCZncDy/5ZsmG82c4GYAKa+dg5upNx9VYjLVs=; b=p8ifETeBqegdWMQigo9hRLbZIZ6nPOuKR1lROtVkajX6ycNiy0RxPjUjR3i0MhjmUa anlFeCZfIszY1tXbnBy9rVFA0Rq7uZvEKQsT5/fE9rj/84a04hHj2vQBFxWFV6qn6J1b IMj7ShuLXdVhq3evzBMu9hzwJCgGmqqAvNMhuDEbkyZnl3+qzbq+9XIKiePpskJxfpD/ Weah3EniXq5oyv9hcwjZCLNbi0m4a4sNxt8ibmBirpkHLNeDJcp2lo0Cw6GNhJ+J+wu5 QSBkpjxR6Q74LtPpAaPKyDV9ENp3FuObJ0UnEgXVkxcAaPr0A3Pfm2+DWgvzh8//B2Zm lL+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=4+VRHOjCZncDy/5ZsmG82c4GYAKa+dg5upNx9VYjLVs=; b=RX8oj5udDD5HC84JAY4t1aCDB0DOySp998QetMHfrGFx77c9GFSTB/Slv0D90YvnFe jG2lGsUYkS775CsbMj0fR+kW3xv68mtcU+vfwEkhVlUmnmnN8jero+NfOHYtaFOxK+Fm yH5fd62mpgdm+U3Jz8XjJeuXDKNtVc48QVxmG75091EqDTfi0m+k8XOUTHd1tMlYZSbN 1A80TSqVU5vG6pAkGoSGnk90SLMC54iu4q+4b0FKXf6bLmuxMwwJcgiwFb80oxpKKSlP BLz8tITQqRGKw/dzxOnT7R2vuKTsr3zjQ5a+11nujFUVYNVnOpgR1OkdYgYJtxcI/IE/ u14g== X-Gm-Message-State: AOAM531Md881JQyFq/YwZKfjHHb1DeWI87HQpo52gxFIsZToqDBkveFG x7NCoAQRy++qOoT1plM1h2x9Dhw6w22+sYFmKlc= X-Google-Smtp-Source: ABdhPJxKUrqiC+me2kb4Mo6zZfIWmm9up7fjWkDYjgMmoZ2elco4OVZLbW0WQ8a9860xGQpteaYni38wKSfSqDUxv5E= X-Received: by 2002:a1f:b217:: with SMTP id b23mr8561461vkf.16.1622236215269; Fri, 28 May 2021 14:10:15 -0700 (PDT) MIME-Version: 1.0 References: <20210527081100.1213973-1-vinay.m.engg@gmail.com> <20210527081100.1213973-4-vinay.m.engg@gmail.com> In-Reply-To: From: "Alexander Kanavin" Date: Fri, 28 May 2021 23:10:03 +0200 Message-ID: Subject: Re: [PATCH v5 4/5] oeqa/selftest/cases/rust.py: Rust oe-selftest script. To: Vinay Kumar Cc: OE-core , Richard Purdie , Konrad Weihmann , Randy MacLeod , umesh kalappa0 , vinay.kumar@blackfigtech.com Content-Type: multipart/alternative; boundary="0000000000005a5c6a05c36a4deb" --0000000000005a5c6a05c36a4deb Content-Type: text/plain; charset="UTF-8" Thanks Vinay, the code looks nice and clean now, and I have no further comments about it. I would only ask that you put the amount of time it takes to (successfully) execute as a comment just before the class definition, so that anyone who needs to allocate CI resources for this test knows what is the reasonable way to run it. Alex On Fri, 28 May 2021 at 18:13, Vinay Kumar wrote: > Hi Alex, > > Please ignore the v6 patch set. As there is minor correction in rust.py > - cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test > codegen %s --target %s ;" % (builddir, testargs, targetsys) > + cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s > --target %s ;" % (builddir, testargs, targetsys) > > Sent v7 patches with corrections of runCmd and ssh. > > For copying "remote-test-server" used copy_to from sshcontrol.py, and > for ssh background execution used "subprocess" module. > > Regarding result.output, whenever there is failure it gets printed > test summary and also captured in "oe-selftest-results.log". > And when there is failure with runCmd then no test summary details or > printed. This is taken care of by runCmd() itself from commands.py. > > Verified with 2 independent test modules "compile-fail" and "codegen" > which are part of rust testing. > > compile-test : All tests get passed and no errors thrown, so > oe-selftes does not print the test results. > > codegen : Some of the tests fail, making the shell exit. In this case > oe-selftest prints test details with stdout and stderr. > > Regards, > Vinay > > > On Thu, May 27, 2021 at 7:05 PM Alexander Kanavin > wrote: > > > > Hello Vinay, > > > > sure, no problem. I see that SSHControl.run() isn't designed for > starting background processes on the target. But I still think it's then > better to use subprocess module for starting the server over ssh, rather > than bundle everything into one gigantic runCmd(). > > > > Alex > > > > On Thu, 27 May 2021 at 14:18, Vinay Kumar > wrote: > >> > >> Hi Alex, > >> > >> The reason why I changed this is, we have to execute > >> "remote-test-server" on the image through background ssh. > >> So that the "remote-test-server" will be alive waiting for connection. > >> Then in the rust.py control passed to the next command (cmd) for > >> execution. > >> If we use the sshcontrol.py method we have to set a timeout of 600 > >> otherwise default timeout is 300. Even if the test execution completes > >> early due to failure then we have to wait till this timeout. > >> > >> I tried reducing the timeout to 60 seconds and some of the tests got > skipped. > >> > >> If that is ok I will use that copy_to and run from sshcontrol.py as we > >> did in v4 patches and work for the result.output > >> > >> Regards, > >> Vinay > >> > >> On Thu, May 27, 2021 at 5:33 PM Alexander Kanavin > >> wrote: > >> > > >> > On Thu, 27 May 2021 at 10:11, Vinay Kumar > wrote: > >> >> > >> >> + # To kill remote-test-server executing > through background ssh > >> >> + killcommand = "kill -9 \$(ps | grep > remote-test-server | grep -v \"grep\" | awk '{print \$1}')" > >> >> + cmd = cmd + "ssh %s root@%s \"%s\";" % > (sshargs, qemu.ip, killcommand) > >> > > >> > > >> > I don't think you actually need to kill the remote test server. As > qemu will be stopped anyway, and with it, all the processes that run inside > qemu. > >> > > >> > Also, it's better to use cmd only to execute the tests. Starting the > server is better done through SSHControl.run(), and copying the binary via > SSHControl.copy_to() - see examples in selfttest's imagefeatures.py or > runtime_test.py. > >> > > >> > Alex > >> > > --0000000000005a5c6a05c36a4deb Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks Vinay, the code looks nice and clean now, and = I have no further comments about it.

I would only = ask that you put the amount of time it takes to (successfully) execute as a= comment just before the class definition, so that anyone who needs to allo= cate CI resources for this test knows what is the reasonable way to run it.=

Alex

On Fri, 28 May 2021 at 18:13, = Vinay Kumar <vinay.m.engg@gmai= l.com> wrote:
Hi Alex,

Please ignore the v6 patch set. As there is minor correction in rust.py
- cmd =3D cmd + " cd %s;=C2=A0 python3 src/bootstrap/bootstrap.py test=
codegen %s --target %s ;" % (builddir, testargs, targetsys)
+ cmd =3D cmd + " cd %s;=C2=A0 python3 src/bootstrap/bootstrap.py test= %s
--target %s ;" % (builddir, testargs, targetsys)

Sent v7 patches with corrections of runCmd and ssh.

For copying "remote-test-server" used copy_to from sshcontrol.py,= and
for ssh background execution used "subprocess" module.

Regarding result.output, whenever there is failure it gets printed
test summary and also captured in "oe-selftest-results.log".
And when there is failure with runCmd then no test summary details or
printed.=C2=A0 This is taken care of by runCmd() itself from commands.py.
Verified with 2 independent test modules "compile-fail" and "= ;codegen"
which are part of rust testing.

compile-test : All tests get passed and no errors thrown, so
oe-selftes does not print the test results.

codegen : Some of the tests fail, making the shell exit. In this case
oe-selftest prints test details with stdout and stderr.

Regards,
Vinay


On Thu, May 27, 2021 at 7:05 PM Alexander Kanavin
<alex.kanavi= n@gmail.com> wrote:
>
> Hello Vinay,
>
> sure, no problem. I see that SSHControl.run() isn't designed for s= tarting background processes on the target. But I still think it's then= better to use subprocess module for starting the server over ssh, rather t= han bundle everything into one gigantic runCmd().
>
> Alex
>
> On Thu, 27 May 2021 at 14:18, Vinay Kumar <vinay.m.engg@gmail.com> wrote: >>
>> Hi Alex,
>>
>> The reason why I changed this is, we have to execute
>> "remote-test-server" on the image through background ssh= .
>> So that the "remote-test-server" will be alive waiting f= or connection.
>> Then in the rust.py=C2=A0 control passed to the next command (cmd)= for
>> execution.
>> If we use the sshcontrol.py method we have to set a timeout of 600=
>> otherwise default timeout is 300. Even if the test execution compl= etes
>> early due to failure then we have to wait till this timeout.
>>
>> I tried reducing the timeout to 60 seconds and some of the tests g= ot skipped.
>>
>> If that is ok I will use that copy_to and run from sshcontrol.py a= s we
>> did in v4 patches and work for the result.output
>>
>> Regards,
>> Vinay
>>
>> On Thu, May 27, 2021 at 5:33 PM Alexander Kanavin
>> <al= ex.kanavin@gmail.com> wrote:
>> >
>> > On Thu, 27 May 2021 at 10:11, Vinay Kumar <vinay.m.engg@gmail.com>= wrote:
>> >>
>> >> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0# To kill remote-test-server executing through b= ackground ssh
>> >> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0killcommand =3D "kill -9 \$(ps | grep remot= e-test-server |=C2=A0 grep -v \"grep\" | awk '{print \$1}'= ;)"
>> >> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0cmd =3D cmd + "ssh %s root@%s \"%s\&qu= ot;;" % (sshargs, qemu.ip, killcommand)
>> >
>> >
>> > I don't think you actually need to kill the remote test s= erver. As qemu will be stopped anyway, and with it, all the processes that = run inside qemu.
>> >
>> > Also, it's better to use cmd only to execute the tests. S= tarting the server is better done through SSHControl.run(), and copying the= binary via SSHControl.copy_to() - see examples in selfttest's imagefea= tures.py or runtime_test.py.
>> >
>> > Alex
>> >
--0000000000005a5c6a05c36a4deb--