From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-vs1-f48.google.com (mail-vs1-f48.google.com [209.85.217.48]) by mx.groups.io with SMTP id smtpd.web09.19282.1622281771799327343 for ; Sat, 29 May 2021 02:49:32 -0700 Authentication-Results: mx.groups.io; dkim=pass header.i=@gmail.com header.s=20161025 header.b=moSnht4i; spf=pass (domain: gmail.com, ip: 209.85.217.48, mailfrom: alex.kanavin@gmail.com) Received: by mail-vs1-f48.google.com with SMTP id j67so626650vsc.13 for ; Sat, 29 May 2021 02:49:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=bceENIL6FCKm4GVU4DHZJHKQ/zDznG3T8DVkOLS2z24=; b=moSnht4ipIZDQmoGVzMnfDshp5x6/EVa3eBq+ww3idMBcQnA7NuuMOYS9a402wWEGO EHBecUdkUXKI6+/i8A4GOmbiR6YwBimBChvsgrAGbIOX2ufleTNJAc7/1exFApECR9Xa KgdyaGx7Aga82OiaONmyiFotSkjMJO4c5bV0fT12lmByUCTdITuJQawMURzLEI5auIUK kFFGeHnDZM30ZLkN8JohkUQZas9ffP/79WqFiPKWyHTORX0PNNbc+5xRmilMeoUds576 Zpide9yrEIDRwvT8ULvBFrRr/5PzBM+oyb0W3aAoAgN3rieS3+TJa+j7c6s/rm09A1wp Q6BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=bceENIL6FCKm4GVU4DHZJHKQ/zDznG3T8DVkOLS2z24=; b=HuG0SVHiX14VF2RxGe+CppKbA9Ew0VJEd2JwkHv+8YcujGqLwOoy55nvKkCWfoFW2m GHBpqnZlKVAsWAZvcV8QOr6PyBxocrVGOWy4sE59gDcVEF7STz1e7EBV2QueXabDhpf/ YptPu7Ep8nV536sqgnusbDrgZtGr6YEwJtm1+01VkdhbePE8SpkHTL7Bvxq+3GYNdfvh fAcQ4e5ElNWvrssPM5QG0/4MozDyWcqva9txZGmY0zRqFPn6ipHwZGViX3cetNKBFPhK yQFvF/J5P/t7J8lL8hpTZP0+hsntpSfGonGvPNFAtTHu8OIj7kBgaK0Kcou/zqyLqNyu YjXw== X-Gm-Message-State: AOAM531FCSKwvZtEQMalOJ9nCRU6H4zvqFWTowRozF6uKzjlpYjAm2Pd zGbpyiLiGGYahwHl7OVHAJEvlBTyflLeoA7Pcn8= X-Google-Smtp-Source: ABdhPJzfb8NHwjPRFVmkANLBt+ZMFMwIV/C5Pe0/RJnSRE8WyPptNX92Y72s/LtnMu7m1E9jy95ge8Sm3ZADRXDvhCI= X-Received: by 2002:a67:f04a:: with SMTP id q10mr9672038vsm.41.1622281770783; Sat, 29 May 2021 02:49:30 -0700 (PDT) MIME-Version: 1.0 References: <20210527081100.1213973-1-vinay.m.engg@gmail.com> <20210527081100.1213973-4-vinay.m.engg@gmail.com> In-Reply-To: From: "Alexander Kanavin" Date: Sat, 29 May 2021 11:49:19 +0200 Message-ID: Subject: Re: [PATCH v5 4/5] oeqa/selftest/cases/rust.py: Rust oe-selftest script. To: Vinay Kumar Cc: Konrad Weihmann , OE-core , Randy MacLeod , Richard Purdie , umesh kalappa0 , vinay.kumar@blackfigtech.com Content-Type: multipart/alternative; boundary="000000000000ac574c05c374e899" --000000000000ac574c05c374e899 Content-Type: text/plain; charset="UTF-8" Thanks, that is fine and please resend once more the whole set. Alex On Sat 29. May 2021 at 11.18, Vinay Kumar wrote: > Hi Alex, > > Is the below comment about execution time is ok ? > > +# Total time taken for testing is of about 2hr 20min, with > PARALLEL_MAKE set to 40 number of jobs. > class RustSelfTestBase(OESelftestTestCase, OEPTestResultTestCase): > > And, should I send v8 version of patches (all 5 patches) or only the > patch of rust.py with this comment as v8 patch ? > > Regards, > Vinay > > On Sat, May 29, 2021 at 2:40 AM Alexander Kanavin > wrote: > > > > Thanks Vinay, the code looks nice and clean now, and I have no further > comments about it. > > > > I would only ask that you put the amount of time it takes to > (successfully) execute as a comment just before the class definition, so > that anyone who needs to allocate CI resources for this test knows what is > the reasonable way to run it. > > > > Alex > > > > On Fri, 28 May 2021 at 18:13, Vinay Kumar > wrote: > >> > >> Hi Alex, > >> > >> Please ignore the v6 patch set. As there is minor correction in rust.py > >> - cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test > >> codegen %s --target %s ;" % (builddir, testargs, targetsys) > >> + cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s > >> --target %s ;" % (builddir, testargs, targetsys) > >> > >> Sent v7 patches with corrections of runCmd and ssh. > >> > >> For copying "remote-test-server" used copy_to from sshcontrol.py, and > >> for ssh background execution used "subprocess" module. > >> > >> Regarding result.output, whenever there is failure it gets printed > >> test summary and also captured in "oe-selftest-results.log". > >> And when there is failure with runCmd then no test summary details or > >> printed. This is taken care of by runCmd() itself from commands.py. > >> > >> Verified with 2 independent test modules "compile-fail" and "codegen" > >> which are part of rust testing. > >> > >> compile-test : All tests get passed and no errors thrown, so > >> oe-selftes does not print the test results. > >> > >> codegen : Some of the tests fail, making the shell exit. In this case > >> oe-selftest prints test details with stdout and stderr. > >> > >> Regards, > >> Vinay > >> > >> > >> On Thu, May 27, 2021 at 7:05 PM Alexander Kanavin > >> wrote: > >> > > >> > Hello Vinay, > >> > > >> > sure, no problem. I see that SSHControl.run() isn't designed for > starting background processes on the target. But I still think it's then > better to use subprocess module for starting the server over ssh, rather > than bundle everything into one gigantic runCmd(). > >> > > >> > Alex > >> > > >> > On Thu, 27 May 2021 at 14:18, Vinay Kumar > wrote: > >> >> > >> >> Hi Alex, > >> >> > >> >> The reason why I changed this is, we have to execute > >> >> "remote-test-server" on the image through background ssh. > >> >> So that the "remote-test-server" will be alive waiting for > connection. > >> >> Then in the rust.py control passed to the next command (cmd) for > >> >> execution. > >> >> If we use the sshcontrol.py method we have to set a timeout of 600 > >> >> otherwise default timeout is 300. Even if the test execution > completes > >> >> early due to failure then we have to wait till this timeout. > >> >> > >> >> I tried reducing the timeout to 60 seconds and some of the tests got > skipped. > >> >> > >> >> If that is ok I will use that copy_to and run from sshcontrol.py as > we > >> >> did in v4 patches and work for the result.output > >> >> > >> >> Regards, > >> >> Vinay > >> >> > >> >> On Thu, May 27, 2021 at 5:33 PM Alexander Kanavin > >> >> wrote: > >> >> > > >> >> > On Thu, 27 May 2021 at 10:11, Vinay Kumar > wrote: > >> >> >> > >> >> >> + # To kill remote-test-server executing > through background ssh > >> >> >> + killcommand = "kill -9 \$(ps | grep > remote-test-server | grep -v \"grep\" | awk '{print \$1}')" > >> >> >> + cmd = cmd + "ssh %s root@%s \"%s\";" % > (sshargs, qemu.ip, killcommand) > >> >> > > >> >> > > >> >> > I don't think you actually need to kill the remote test server. As > qemu will be stopped anyway, and with it, all the processes that run inside > qemu. > >> >> > > >> >> > Also, it's better to use cmd only to execute the tests. Starting > the server is better done through SSHControl.run(), and copying the binary > via SSHControl.copy_to() - see examples in selfttest's imagefeatures.py or > runtime_test.py. > >> >> > > >> >> > Alex > >> >> > > --000000000000ac574c05c374e899 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks, that is fine and please resend once more the whol= e set.


Alex

On Sat 29. May 2021 at 11.18, Vinay Kumar <vinay.m.engg@gmail.com> wrote:
Hi Alex,

Is the below comment about execution time is ok ?

+# Total time taken for testing is of about 2hr 20min, with
PARALLEL_MAKE set to 40 number of jobs.
=C2=A0class RustSelfTestBase(OESelftestTestCase, OEPTestResultTestCase):
And, should I send v8 version of patches (all 5 patches) or only the
patch of rust.py=C2=A0 with this comment as v8 patch ?

Regards,
Vinay

On Sat, May 29, 2021 at 2:40 AM Alexander Kanavin
<alex.kanavi= n@gmail.com> wrote:
>
> Thanks Vinay, the code looks nice and clean now, and I have no further= comments about it.
>
> I would only ask that you put the amount of time it takes to (successf= ully) execute as a comment just before the class definition, so that anyone= who needs to allocate CI resources for this test knows what is the reasona= ble way to run it.
>
> Alex
>
> On Fri, 28 May 2021 at 18:13, Vinay Kumar <vinay.m.engg@gmail.com> wrote: >>
>> Hi Alex,
>>
>> Please ignore the v6 patch set. As there is minor correction in ru= st.py
>> - cmd =3D cmd + " cd %s;=C2=A0 python3 src/bootstrap/bootstra= p.py test
>> codegen %s --target %s ;" % (builddir, testargs, targetsys) >> + cmd =3D cmd + " cd %s;=C2=A0 python3 src/bootstrap/bootstra= p.py test %s
>> --target %s ;" % (builddir, testargs, targetsys)
>>
>> Sent v7 patches with corrections of runCmd and ssh.
>>
>> For copying "remote-test-server" used copy_to from sshco= ntrol.py, and
>> for ssh background execution used "subprocess" module. >>
>> Regarding result.output, whenever there is failure it gets printed=
>> test summary and also captured in "oe-selftest-results.log&qu= ot;.
>> And when there is failure with runCmd then no test summary details= or
>> printed.=C2=A0 This is taken care of by runCmd() itself from comma= nds.py.
>>
>> Verified with 2 independent test modules "compile-fail" = and "codegen"
>> which are part of rust testing.
>>
>> compile-test : All tests get passed and no errors thrown, so
>> oe-selftes does not print the test results.
>>
>> codegen : Some of the tests fail, making the shell exit. In this c= ase
>> oe-selftest prints test details with stdout and stderr.
>>
>> Regards,
>> Vinay
>>
>>
>> On Thu, May 27, 2021 at 7:05 PM Alexander Kanavin
>> <al= ex.kanavin@gmail.com> wrote:
>> >
>> > Hello Vinay,
>> >
>> > sure, no problem. I see that SSHControl.run() isn't desig= ned for starting background processes on the target. But I still think it&#= 39;s then better to use subprocess module for starting the server over ssh,= rather than bundle everything into one gigantic runCmd().
>> >
>> > Alex
>> >
>> > On Thu, 27 May 2021 at 14:18, Vinay Kumar <vinay.m.engg@gmail.com>= wrote:
>> >>
>> >> Hi Alex,
>> >>
>> >> The reason why I changed this is, we have to execute
>> >> "remote-test-server" on the image through backg= round ssh.
>> >> So that the "remote-test-server" will be alive = waiting for connection.
>> >> Then in the rust.py=C2=A0 control passed to the next comm= and (cmd) for
>> >> execution.
>> >> If we use the sshcontrol.py method we have to set a timeo= ut of 600
>> >> otherwise default timeout is 300. Even if the test execut= ion completes
>> >> early due to failure then we have to wait till this timeo= ut.
>> >>
>> >> I tried reducing the timeout to 60 seconds and some of th= e tests got skipped.
>> >>
>> >> If that is ok I will use that copy_to and run from sshcon= trol.py as we
>> >> did in v4 patches and work for the result.output
>> >>
>> >> Regards,
>> >> Vinay
>> >>
>> >> On Thu, May 27, 2021 at 5:33 PM Alexander Kanavin
>> >> <alex.kanavin@gmail.com> wrote:
>> >> >
>> >> > On Thu, 27 May 2021 at 10:11, Vinay Kumar <vinay.m.engg@gmail.co= m> wrote:
>> >> >>
>> >> >> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0# To kill remote-test-server executin= g through background ssh
>> >> >> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0killcommand =3D "kill -9 \$(ps |= grep remote-test-server |=C2=A0 grep -v \"grep\" | awk '{pri= nt \$1}')"
>> >> >> +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0cmd =3D cmd + "ssh %s root@%s \&= quot;%s\";" % (sshargs, qemu.ip, killcommand)
>> >> >
>> >> >
>> >> > I don't think you actually need to kill the remo= te test server. As qemu will be stopped anyway, and with it, all the proces= ses that run inside qemu.
>> >> >
>> >> > Also, it's better to use cmd only to execute the= tests. Starting the server is better done through SSHControl.run(), and co= pying the binary via SSHControl.copy_to() - see examples in selfttest's= imagefeatures.py or runtime_test.py.
>> >> >
>> >> > Alex
>> >> >
--000000000000ac574c05c374e899--