[kvm-unit-tests] its-migration segmentation fault

[kvm-unit-tests] its-migration segmentation fault

[Alexandru Elisei]

When running all the tests with taskset -c 0-3 ./run_tests.sh on a rockpro64 (on
the Cortex-a53 cores) the its-migration test hangs. In the log file I see:

run_migration timeout -k 1s --foreground 90s /usr/bin/qemu-system-aarch64
-nodefaults -machine virt,gic-version=host,accel=kvm -cpu host -device
virtio-serial-device -device virtconsole,chardev=ctd -chardev testdev,id=ctd
-device pci-testdev -display none -serial stdio -kernel arm/gic.flat -smp 6
-machine gic-version=3 -append its-migration # -initrd /tmp/tmp.OrlQiorBpY
ITS: MAPD devid=2 size = 0x8 itt=0x40420000 valid=1
ITS: MAPD devid=7 size = 0x8 itt=0x40430000 valid=1
MAPC col_id=3 target_addr = 0x30000 valid=1
MAPC col_id=2 target_addr = 0x20000 valid=1
INVALL col_id=2
INVALL col_id=3
MAPTI dev_id=2 event_id=20 -> phys_id=8195, col_id=3
MAPTI dev_id=7 event_id=255 -> phys_id=8196, col_id=2
Now migrate the VM, then press a key to continue...
scripts/arch-run.bash: line 103: 48549 Done                    echo '{ "execute":
"qmp_capabilities" }{ "execute":' "$2" '}'
     48550 Segmentation fault      (core dumped) | ncat -U $1
scripts/arch-run.bash: line 103: 48568 Done                    echo '{ "execute":
"qmp_capabilities" }{ "execute":' "$2" '}'
     48569 Segmentation fault      (core dumped) | ncat -U $1
scripts/arch-run.bash: line 103: 48583 Done                    echo '{ "execute":
"qmp_capabilities" }{ "execute":' "$2" '}'
     48584 Segmentation fault      (core dumped) | ncat -U $1
[..]
scripts/arch-run.bash: line 103: 49414 Done                    echo '{ "execute":
"qmp_capabilities" }{ "execute":' "$2" '}'
     49415 Segmentation fault      (core dumped) | ncat -U $1
qemu-system-aarch64: terminating on signal 15 from pid 48496 (timeout)
qemu-system-aarch64: terminating on signal 15 from pid 48504 (timeout)
scripts/arch-run.bash: line 103: 49430 Done                    echo '{ "execute":
"qmp_capabilities" }{ "execute":' "$2" '}'
     49431 Segmentation fault      (core dumped) | ncat -U $1
scripts/arch-run.bash: line 103: 49445 Done                    echo '{ "execute":
"qmp_capabilities" }{ "execute":' "$2" '}'
[..]

If I run the test manually:

$ taskset -c 0-3 ./arm-run arm/gic.flat -smp 4 -machine gic-version=3 -append
'its-migration'

/usr/bin/qemu-system-aarch64 -nodefaults -machine virt,gic-version=host,accel=kvm
-cpu host -device virtio-serial-device -device virtconsole,chardev=ctd -chardev
testdev,id=ctd -device pci-testdev -display none -serial stdio -kernel
arm/gic.flat -smp 4 -machine gic-version=3 -append its-migration # -initrd
/tmp/tmp.OtsTj3QD4J
ITS: MAPD devid=2 size = 0x8 itt=0x403a0000 valid=1
ITS: MAPD devid=7 size = 0x8 itt=0x403b0000 valid=1
MAPC col_id=3 target_addr = 0x30000 valid=1
MAPC col_id=2 target_addr = 0x20000 valid=1
INVALL col_id=2
INVALL col_id=3
MAPTI dev_id=2 event_id=20 -> phys_id=8195, col_id=3
MAPTI dev_id=7 event_id=255 -> phys_id=8196, col_id=2
Now migrate the VM, then press a key to continue...

And the test hangs here after I press a key.

Package versions:

$ ncat --version
Ncat: Version 7.91 ( https://nmap.org/ncat )

$ /usr/bin/qemu-system-aarch64 --version
QEMU emulator version 5.1.0
Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers

$ uname -a
Linux rockpro 5.10.0-rc4 #33 SMP PREEMPT Thu Nov 19 15:58:57 GMT 2020 aarch64
GNU/Linux

Thanks,

Alex

Re: [kvm-unit-tests] its-migration segmentation fault

[Andrew Jones]

On Fri, Nov 20, 2020 at 12:02:10PM +0000, Alexandru Elisei wrote:
> When running all the tests with taskset -c 0-3 ./run_tests.sh on a rockpro64 (on
> the Cortex-a53 cores) the its-migration test hangs. In the log file I see:
> 
> run_migration timeout -k 1s --foreground 90s /usr/bin/qemu-system-aarch64
> -nodefaults -machine virt,gic-version=host,accel=kvm -cpu host -device
> virtio-serial-device -device virtconsole,chardev=ctd -chardev testdev,id=ctd
> -device pci-testdev -display none -serial stdio -kernel arm/gic.flat -smp 6
> -machine gic-version=3 -append its-migration # -initrd /tmp/tmp.OrlQiorBpY
> ITS: MAPD devid=2 size = 0x8 itt=0x40420000 valid=1
> ITS: MAPD devid=7 size = 0x8 itt=0x40430000 valid=1
> MAPC col_id=3 target_addr = 0x30000 valid=1
> MAPC col_id=2 target_addr = 0x20000 valid=1
> INVALL col_id=2
> INVALL col_id=3
> MAPTI dev_id=2 event_id=20 -> phys_id=8195, col_id=3
> MAPTI dev_id=7 event_id=255 -> phys_id=8196, col_id=2
> Now migrate the VM, then press a key to continue...
> scripts/arch-run.bash: line 103: 48549 Done                    echo '{ "execute":
> "qmp_capabilities" }{ "execute":' "$2" '}'
>      48550 Segmentation fault      (core dumped) | ncat -U $1
> scripts/arch-run.bash: line 103: 48568 Done                    echo '{ "execute":
> "qmp_capabilities" }{ "execute":' "$2" '}'
>      48569 Segmentation fault      (core dumped) | ncat -U $1
> scripts/arch-run.bash: line 103: 48583 Done                    echo '{ "execute":
> "qmp_capabilities" }{ "execute":' "$2" '}'
>      48584 Segmentation fault      (core dumped) | ncat -U $1
> [..]
> scripts/arch-run.bash: line 103: 49414 Done                    echo '{ "execute":
> "qmp_capabilities" }{ "execute":' "$2" '}'
>      49415 Segmentation fault      (core dumped) | ncat -U $1
> qemu-system-aarch64: terminating on signal 15 from pid 48496 (timeout)
> qemu-system-aarch64: terminating on signal 15 from pid 48504 (timeout)
> scripts/arch-run.bash: line 103: 49430 Done                    echo '{ "execute":
> "qmp_capabilities" }{ "execute":' "$2" '}'
>      49431 Segmentation fault      (core dumped) | ncat -U $1
> scripts/arch-run.bash: line 103: 49445 Done                    echo '{ "execute":
> "qmp_capabilities" }{ "execute":' "$2" '}'
> [..]

Is your ncat segfaulting? It looks like it from this output. Have you
tried running your ncat with a UNIX socket independently of this test?

Is this the first time you've tried this test in this environment, or
is this a regression for you?

> 
> If I run the test manually:
> 
> $ taskset -c 0-3 ./arm-run arm/gic.flat -smp 4 -machine gic-version=3 -append
> 'its-migration'

This won't work because we need run_tests.sh to setup the run_migration()
call. The only ways to run migration tests separately are

 $ ./run_tests.sh its-migration

and

 $ tests/its-migration

For the second one you need to do 'make standalone' first.


> 
> /usr/bin/qemu-system-aarch64 -nodefaults -machine virt,gic-version=host,accel=kvm
> -cpu host -device virtio-serial-device -device virtconsole,chardev=ctd -chardev
> testdev,id=ctd -device pci-testdev -display none -serial stdio -kernel
> arm/gic.flat -smp 4 -machine gic-version=3 -append its-migration # -initrd
> /tmp/tmp.OtsTj3QD4J
> ITS: MAPD devid=2 size = 0x8 itt=0x403a0000 valid=1
> ITS: MAPD devid=7 size = 0x8 itt=0x403b0000 valid=1
> MAPC col_id=3 target_addr = 0x30000 valid=1
> MAPC col_id=2 target_addr = 0x20000 valid=1
> INVALL col_id=2
> INVALL col_id=3
> MAPTI dev_id=2 event_id=20 -> phys_id=8195, col_id=3
> MAPTI dev_id=7 event_id=255 -> phys_id=8196, col_id=2
> Now migrate the VM, then press a key to continue...
> 
> And the test hangs here after I press a key.

The test doesn't get your input because of the '</dev/null' in run_qemu(),
which ./arm-run calls. So it's not hanging it's just waiting forever on
the key press.

Thanks,
drew

> 
> Package versions:
> 
> $ ncat --version
> Ncat: Version 7.91 ( https://nmap.org/ncat )
> 
> $ /usr/bin/qemu-system-aarch64 --version
> QEMU emulator version 5.1.0
> Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers
> 
> $ uname -a
> Linux rockpro 5.10.0-rc4 #33 SMP PREEMPT Thu Nov 19 15:58:57 GMT 2020 aarch64
> GNU/Linux
> 
> Thanks,
> 
> Alex
> 

Re: [kvm-unit-tests] its-migration segmentation fault

[Alexandru Elisei]

Hi,

On 11/20/20 12:34 PM, Andrew Jones wrote:
> On Fri, Nov 20, 2020 at 12:02:10PM +0000, Alexandru Elisei wrote:
>> When running all the tests with taskset -c 0-3 ./run_tests.sh on a rockpro64 (on
>> the Cortex-a53 cores) the its-migration test hangs.[..]
> Is your ncat segfaulting? It looks like it from this output. Have you
> tried running your ncat with a UNIX socket independently of this test?
>
> Is this the first time you've tried this test in this environment, or
> is this a regression for you?

Looks like it's a regression in nmap. I downgraded to nmap 7.80 and the test
passes, thank you for the help!

>
>> If I run the test manually:
>>
>> $ taskset -c 0-3 ./arm-run arm/gic.flat -smp 4 -machine gic-version=3 -append
>> 'its-migration'
> This won't work because we need run_tests.sh to setup the run_migration()
> call. The only ways to run migration tests separately are
>
>  $ ./run_tests.sh its-migration
>
> and
>
>  $ tests/its-migration
>
> For the second one you need to do 'make standalone' first.
>
>
>> /usr/bin/qemu-system-aarch64 -nodefaults -machine virt,gic-version=host,accel=kvm
>> -cpu host -device virtio-serial-device -device virtconsole,chardev=ctd -chardev
>> testdev,id=ctd -device pci-testdev -display none -serial stdio -kernel
>> arm/gic.flat -smp 4 -machine gic-version=3 -append its-migration # -initrd
>> /tmp/tmp.OtsTj3QD4J
>> ITS: MAPD devid=2 size = 0x8 itt=0x403a0000 valid=1
>> ITS: MAPD devid=7 size = 0x8 itt=0x403b0000 valid=1
>> MAPC col_id=3 target_addr = 0x30000 valid=1
>> MAPC col_id=2 target_addr = 0x20000 valid=1
>> INVALL col_id=2
>> INVALL col_id=3
>> MAPTI dev_id=2 event_id=20 -> phys_id=8195, col_id=3
>> MAPTI dev_id=7 event_id=255 -> phys_id=8196, col_id=2
>> Now migrate the VM, then press a key to continue...
>>
>> And the test hangs here after I press a key.
> The test doesn't get your input because of the '</dev/null' in run_qemu(),
> which ./arm-run calls. So it's not hanging it's just waiting forever on
> the key press.

I see, thanks for the explanation!

Thanks,
Alex

Re: [kvm-unit-tests] its-migration segmentation fault

[Po-Hsu Lin]

On Fri, Nov 20, 2020 at 8:35 PM Andrew Jones <drjones@redhat.com> wrote:
>
> On Fri, Nov 20, 2020 at 12:02:10PM +0000, Alexandru Elisei wrote:
> > When running all the tests with taskset -c 0-3 ./run_tests.sh on a rockpro64 (on
> > the Cortex-a53 cores) the its-migration test hangs. In the log file I see:
> >
> > run_migration timeout -k 1s --foreground 90s /usr/bin/qemu-system-aarch64
> > -nodefaults -machine virt,gic-version=host,accel=kvm -cpu host -device
> > virtio-serial-device -device virtconsole,chardev=ctd -chardev testdev,id=ctd
> > -device pci-testdev -display none -serial stdio -kernel arm/gic.flat -smp 6
> > -machine gic-version=3 -append its-migration # -initrd /tmp/tmp.OrlQiorBpY
> > ITS: MAPD devid=2 size = 0x8 itt=0x40420000 valid=1
> > ITS: MAPD devid=7 size = 0x8 itt=0x40430000 valid=1
> > MAPC col_id=3 target_addr = 0x30000 valid=1
> > MAPC col_id=2 target_addr = 0x20000 valid=1
> > INVALL col_id=2
> > INVALL col_id=3
> > MAPTI dev_id=2 event_id=20 -> phys_id=8195, col_id=3
> > MAPTI dev_id=7 event_id=255 -> phys_id=8196, col_id=2
> > Now migrate the VM, then press a key to continue...
> > scripts/arch-run.bash: line 103: 48549 Done                    echo '{ "execute":
> > "qmp_capabilities" }{ "execute":' "$2" '}'
> >      48550 Segmentation fault      (core dumped) | ncat -U $1
> > scripts/arch-run.bash: line 103: 48568 Done                    echo '{ "execute":
> > "qmp_capabilities" }{ "execute":' "$2" '}'
> >      48569 Segmentation fault      (core dumped) | ncat -U $1
> > scripts/arch-run.bash: line 103: 48583 Done                    echo '{ "execute":
> > "qmp_capabilities" }{ "execute":' "$2" '}'
> >      48584 Segmentation fault      (core dumped) | ncat -U $1
> > [..]
> > scripts/arch-run.bash: line 103: 49414 Done                    echo '{ "execute":
> > "qmp_capabilities" }{ "execute":' "$2" '}'
> >      49415 Segmentation fault      (core dumped) | ncat -U $1
> > qemu-system-aarch64: terminating on signal 15 from pid 48496 (timeout)
> > qemu-system-aarch64: terminating on signal 15 from pid 48504 (timeout)
> > scripts/arch-run.bash: line 103: 49430 Done                    echo '{ "execute":
> > "qmp_capabilities" }{ "execute":' "$2" '}'
> >      49431 Segmentation fault      (core dumped) | ncat -U $1
> > scripts/arch-run.bash: line 103: 49445 Done                    echo '{ "execute":
> > "qmp_capabilities" }{ "execute":' "$2" '}'
> > [..]
>
> Is your ncat segfaulting? It looks like it from this output. Have you
> tried running your ncat with a UNIX socket independently of this test?
>
> Is this the first time you've tried this test in this environment, or
> is this a regression for you?
>
> >
> > If I run the test manually:
> >
> > $ taskset -c 0-3 ./arm-run arm/gic.flat -smp 4 -machine gic-version=3 -append
> > 'its-migration'
>
> This won't work because we need run_tests.sh to setup the run_migration()
> call. The only ways to run migration tests separately are
>
>  $ ./run_tests.sh its-migration
>
> and
>
>  $ tests/its-migration
>
> For the second one you need to do 'make standalone' first.
>
>
> >
> > /usr/bin/qemu-system-aarch64 -nodefaults -machine virt,gic-version=host,accel=kvm
> > -cpu host -device virtio-serial-device -device virtconsole,chardev=ctd -chardev
> > testdev,id=ctd -device pci-testdev -display none -serial stdio -kernel
> > arm/gic.flat -smp 4 -machine gic-version=3 -append its-migration # -initrd
> > /tmp/tmp.OtsTj3QD4J
> > ITS: MAPD devid=2 size = 0x8 itt=0x403a0000 valid=1
> > ITS: MAPD devid=7 size = 0x8 itt=0x403b0000 valid=1
> > MAPC col_id=3 target_addr = 0x30000 valid=1
> > MAPC col_id=2 target_addr = 0x20000 valid=1
> > INVALL col_id=2
> > INVALL col_id=3
> > MAPTI dev_id=2 event_id=20 -> phys_id=8195, col_id=3
> > MAPTI dev_id=7 event_id=255 -> phys_id=8196, col_id=2
> > Now migrate the VM, then press a key to continue...
> >
> > And the test hangs here after I press a key.
>
> The test doesn't get your input because of the '</dev/null' in run_qemu(),
> which ./arm-run calls. So it's not hanging it's just waiting forever on
> the key press.
Hello Andrew,
We have found this waiting for key press issue on our side as well
[1], the test will fail with TIMEOUT, it looks like it's not getting
my input like you mentioned here.
I would like to ask what is the expected behaviour of these migration
related tests (its-pending-migration / its-migration /
its-migrate-unmapped-collection)? Should they pass right after the
tester hit a key?
Also, if these test would require user interaction, should they be
moved to some special group like 'nodefault' to prevent it from
failing with timeout in automated tests?

I tried to remove '</dev/null' in 'errors=$("${@}" $INITRD </dev/null
2> >(tee /dev/stderr) > /dev/fd/$stdout)' from script/arch-run.bash
and run it again, and found out that if you wait to see that 'Now
migrate the VM, then press a key to continue...' prompt, it's too late
for the test to catch your key press. You will have to press any key
right after the test started. However although the test will pass, it
won't be terminated properly but keep complaining about "Ncat:
Connection refused." until I hit ctrl + c. Not sure if this is
expected?

$ uname -a
Linux kuzzle 5.11.0-18-generic #19-Ubuntu SMP Fri May 7 14:21:20 UTC
2021 aarch64 aarch64 aarch64 GNU/Linux
$ sudo ./its-migration
BUILD_HEAD=90a7d30e
k (my key press)
run_migration timeout -k 1s --foreground 90s
/usr/bin/qemu-system-aarch64 -nodefaults -machine
virt,gic-version=host,accel=kvm -cpu host -device virtio-serial-device
-device virtconsole,chardev=ctd -chardev testdev,id=ctd -device
pci-testdev -display none -serial stdio -kernel /tmp/tmp.HhCjbIcns7
-smp 32 -machine gic-version=3 -append its-migration # -initrd
/tmp/tmp.AyirrSboiF
ITS: MAPD devid=2 size = 0x8 itt=0x408d0000 valid=1
ITS: MAPD devid=7 size = 0x8 itt=0x408e0000 valid=1
MAPC col_id=3 target_addr = 0x30000 valid=1
MAPC col_id=2 target_addr = 0x20000 valid=1
INVALL col_id=2
INVALL col_id=3
MAPTI dev_id=2 event_id=20 -> phys_id=8195, col_id=3
MAPTI dev_id=7 event_id=255 -> phys_id=8196, col_id=2
Now migrate the VM, then press a key to continue...
INFO: gicv3: its-migration: Migration complete
INT dev_id=2 event_id=20
PASS: gicv3: its-migration: dev2/eventid=20 triggers LPI 8195 on PE #3
after migration
INT dev_id=7 event_id=255
PASS: gicv3: its-migration: dev7/eventid=255 triggers LPI 8196 on PE
#2 after migration
SUMMARY: 2 tests
Ncat: Connection refused.
Ncat: Connection refused.
Ncat: Connection refused.

Ncat: Version 7.80

Thanks!
PHLin

[1] https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/1931680

>
> Thanks,
> drew
>
> >
> > Package versions:
> >
> > $ ncat --version
> > Ncat: Version 7.91 ( https://nmap.org/ncat )
> >
> > $ /usr/bin/qemu-system-aarch64 --version
> > QEMU emulator version 5.1.0
> > Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers
> >
> > $ uname -a
> > Linux rockpro 5.10.0-rc4 #33 SMP PREEMPT Thu Nov 19 15:58:57 GMT 2020 aarch64
> > GNU/Linux
> >
> > Thanks,
> >
> > Alex
> >
>

Re: [kvm-unit-tests] its-migration segmentation fault

[Andrew Jones]

On Tue, Jun 15, 2021 at 11:21:05AM +0800, Po-Hsu Lin wrote:
> On Fri, Nov 20, 2020 at 8:35 PM Andrew Jones <drjones@redhat.com> wrote:
> >
> > On Fri, Nov 20, 2020 at 12:02:10PM +0000, Alexandru Elisei wrote:
> > > When running all the tests with taskset -c 0-3 ./run_tests.sh on a rockpro64 (on
> > > the Cortex-a53 cores) the its-migration test hangs. In the log file I see:
> > >
> > > run_migration timeout -k 1s --foreground 90s /usr/bin/qemu-system-aarch64
> > > -nodefaults -machine virt,gic-version=host,accel=kvm -cpu host -device
> > > virtio-serial-device -device virtconsole,chardev=ctd -chardev testdev,id=ctd
> > > -device pci-testdev -display none -serial stdio -kernel arm/gic.flat -smp 6
> > > -machine gic-version=3 -append its-migration # -initrd /tmp/tmp.OrlQiorBpY
> > > ITS: MAPD devid=2 size = 0x8 itt=0x40420000 valid=1
> > > ITS: MAPD devid=7 size = 0x8 itt=0x40430000 valid=1
> > > MAPC col_id=3 target_addr = 0x30000 valid=1
> > > MAPC col_id=2 target_addr = 0x20000 valid=1
> > > INVALL col_id=2
> > > INVALL col_id=3
> > > MAPTI dev_id=2 event_id=20 -> phys_id=8195, col_id=3
> > > MAPTI dev_id=7 event_id=255 -> phys_id=8196, col_id=2
> > > Now migrate the VM, then press a key to continue...
> > > scripts/arch-run.bash: line 103: 48549 Done                    echo '{ "execute":
> > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > >      48550 Segmentation fault      (core dumped) | ncat -U $1
> > > scripts/arch-run.bash: line 103: 48568 Done                    echo '{ "execute":
> > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > >      48569 Segmentation fault      (core dumped) | ncat -U $1
> > > scripts/arch-run.bash: line 103: 48583 Done                    echo '{ "execute":
> > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > >      48584 Segmentation fault      (core dumped) | ncat -U $1
> > > [..]
> > > scripts/arch-run.bash: line 103: 49414 Done                    echo '{ "execute":
> > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > >      49415 Segmentation fault      (core dumped) | ncat -U $1
> > > qemu-system-aarch64: terminating on signal 15 from pid 48496 (timeout)
> > > qemu-system-aarch64: terminating on signal 15 from pid 48504 (timeout)
> > > scripts/arch-run.bash: line 103: 49430 Done                    echo '{ "execute":
> > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > >      49431 Segmentation fault      (core dumped) | ncat -U $1
> > > scripts/arch-run.bash: line 103: 49445 Done                    echo '{ "execute":
> > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > > [..]
> >
> > Is your ncat segfaulting? It looks like it from this output. Have you
> > tried running your ncat with a UNIX socket independently of this test?
> >
> > Is this the first time you've tried this test in this environment, or
> > is this a regression for you?
> >
> > >
> > > If I run the test manually:
> > >
> > > $ taskset -c 0-3 ./arm-run arm/gic.flat -smp 4 -machine gic-version=3 -append
> > > 'its-migration'
> >
> > This won't work because we need run_tests.sh to setup the run_migration()
> > call. The only ways to run migration tests separately are
> >
> >  $ ./run_tests.sh its-migration
> >
> > and
> >
> >  $ tests/its-migration
> >
> > For the second one you need to do 'make standalone' first.
> >
> >
> > >
> > > /usr/bin/qemu-system-aarch64 -nodefaults -machine virt,gic-version=host,accel=kvm
> > > -cpu host -device virtio-serial-device -device virtconsole,chardev=ctd -chardev
> > > testdev,id=ctd -device pci-testdev -display none -serial stdio -kernel
> > > arm/gic.flat -smp 4 -machine gic-version=3 -append its-migration # -initrd
> > > /tmp/tmp.OtsTj3QD4J
> > > ITS: MAPD devid=2 size = 0x8 itt=0x403a0000 valid=1
> > > ITS: MAPD devid=7 size = 0x8 itt=0x403b0000 valid=1
> > > MAPC col_id=3 target_addr = 0x30000 valid=1
> > > MAPC col_id=2 target_addr = 0x20000 valid=1
> > > INVALL col_id=2
> > > INVALL col_id=3
> > > MAPTI dev_id=2 event_id=20 -> phys_id=8195, col_id=3
> > > MAPTI dev_id=7 event_id=255 -> phys_id=8196, col_id=2
> > > Now migrate the VM, then press a key to continue...
> > >
> > > And the test hangs here after I press a key.
> >
> > The test doesn't get your input because of the '</dev/null' in run_qemu(),
> > which ./arm-run calls. So it's not hanging it's just waiting forever on
> > the key press.
> Hello Andrew,
> We have found this waiting for key press issue on our side as well
> [1], the test will fail with TIMEOUT, it looks like it's not getting
> my input like you mentioned here.
> I would like to ask what is the expected behaviour of these migration
> related tests (its-pending-migration / its-migration /
> its-migrate-unmapped-collection)? Should they pass right after the
> tester hit a key?

They should, but normally users don't need to press a key, because the
script uses ncat to do it for them.

> Also, if these test would require user interaction, should they be
> moved to some special group like 'nodefault' to prevent it from
> failing with timeout in automated tests?

The tests shouldn't be a problem when ncat does its job.



I still think we have a script/ncat issue here. Please provide

 qemu version:
 bash version:
 ncat version:

And the distro and distro version might also be helpful.

Thanks,
drew

Re: [kvm-unit-tests] its-migration segmentation fault

[Po-Hsu Lin]

On Tue, Jun 15, 2021 at 2:37 PM Andrew Jones <drjones@redhat.com> wrote:
>
> On Tue, Jun 15, 2021 at 11:21:05AM +0800, Po-Hsu Lin wrote:
> > On Fri, Nov 20, 2020 at 8:35 PM Andrew Jones <drjones@redhat.com> wrote:
> > >
> > > On Fri, Nov 20, 2020 at 12:02:10PM +0000, Alexandru Elisei wrote:
> > > > When running all the tests with taskset -c 0-3 ./run_tests.sh on a rockpro64 (on
> > > > the Cortex-a53 cores) the its-migration test hangs. In the log file I see:
> > > >
> > > > run_migration timeout -k 1s --foreground 90s /usr/bin/qemu-system-aarch64
> > > > -nodefaults -machine virt,gic-version=host,accel=kvm -cpu host -device
> > > > virtio-serial-device -device virtconsole,chardev=ctd -chardev testdev,id=ctd
> > > > -device pci-testdev -display none -serial stdio -kernel arm/gic.flat -smp 6
> > > > -machine gic-version=3 -append its-migration # -initrd /tmp/tmp.OrlQiorBpY
> > > > ITS: MAPD devid=2 size = 0x8 itt=0x40420000 valid=1
> > > > ITS: MAPD devid=7 size = 0x8 itt=0x40430000 valid=1
> > > > MAPC col_id=3 target_addr = 0x30000 valid=1
> > > > MAPC col_id=2 target_addr = 0x20000 valid=1
> > > > INVALL col_id=2
> > > > INVALL col_id=3
> > > > MAPTI dev_id=2 event_id=20 -> phys_id=8195, col_id=3
> > > > MAPTI dev_id=7 event_id=255 -> phys_id=8196, col_id=2
> > > > Now migrate the VM, then press a key to continue...
> > > > scripts/arch-run.bash: line 103: 48549 Done                    echo '{ "execute":
> > > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > > >      48550 Segmentation fault      (core dumped) | ncat -U $1
> > > > scripts/arch-run.bash: line 103: 48568 Done                    echo '{ "execute":
> > > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > > >      48569 Segmentation fault      (core dumped) | ncat -U $1
> > > > scripts/arch-run.bash: line 103: 48583 Done                    echo '{ "execute":
> > > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > > >      48584 Segmentation fault      (core dumped) | ncat -U $1
> > > > [..]
> > > > scripts/arch-run.bash: line 103: 49414 Done                    echo '{ "execute":
> > > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > > >      49415 Segmentation fault      (core dumped) | ncat -U $1
> > > > qemu-system-aarch64: terminating on signal 15 from pid 48496 (timeout)
> > > > qemu-system-aarch64: terminating on signal 15 from pid 48504 (timeout)
> > > > scripts/arch-run.bash: line 103: 49430 Done                    echo '{ "execute":
> > > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > > >      49431 Segmentation fault      (core dumped) | ncat -U $1
> > > > scripts/arch-run.bash: line 103: 49445 Done                    echo '{ "execute":
> > > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > > > [..]
> > >
> > > Is your ncat segfaulting? It looks like it from this output. Have you
> > > tried running your ncat with a UNIX socket independently of this test?
> > >
> > > Is this the first time you've tried this test in this environment, or
> > > is this a regression for you?
> > >
> > > >
> > > > If I run the test manually:
> > > >
> > > > $ taskset -c 0-3 ./arm-run arm/gic.flat -smp 4 -machine gic-version=3 -append
> > > > 'its-migration'
> > >
> > > This won't work because we need run_tests.sh to setup the run_migration()
> > > call. The only ways to run migration tests separately are
> > >
> > >  $ ./run_tests.sh its-migration
> > >
> > > and
> > >
> > >  $ tests/its-migration
> > >
> > > For the second one you need to do 'make standalone' first.
> > >
> > >
> > > >
> > > > /usr/bin/qemu-system-aarch64 -nodefaults -machine virt,gic-version=host,accel=kvm
> > > > -cpu host -device virtio-serial-device -device virtconsole,chardev=ctd -chardev
> > > > testdev,id=ctd -device pci-testdev -display none -serial stdio -kernel
> > > > arm/gic.flat -smp 4 -machine gic-version=3 -append its-migration # -initrd
> > > > /tmp/tmp.OtsTj3QD4J
> > > > ITS: MAPD devid=2 size = 0x8 itt=0x403a0000 valid=1
> > > > ITS: MAPD devid=7 size = 0x8 itt=0x403b0000 valid=1
> > > > MAPC col_id=3 target_addr = 0x30000 valid=1
> > > > MAPC col_id=2 target_addr = 0x20000 valid=1
> > > > INVALL col_id=2
> > > > INVALL col_id=3
> > > > MAPTI dev_id=2 event_id=20 -> phys_id=8195, col_id=3
> > > > MAPTI dev_id=7 event_id=255 -> phys_id=8196, col_id=2
> > > > Now migrate the VM, then press a key to continue...
> > > >
> > > > And the test hangs here after I press a key.
> > >
> > > The test doesn't get your input because of the '</dev/null' in run_qemu(),
> > > which ./arm-run calls. So it's not hanging it's just waiting forever on
> > > the key press.
> > Hello Andrew,
> > We have found this waiting for key press issue on our side as well
> > [1], the test will fail with TIMEOUT, it looks like it's not getting
> > my input like you mentioned here.
> > I would like to ask what is the expected behaviour of these migration
> > related tests (its-pending-migration / its-migration /
> > its-migrate-unmapped-collection)? Should they pass right after the
> > tester hit a key?
>
> They should, but normally users don't need to press a key, because the
> script uses ncat to do it for them.
>
> > Also, if these test would require user interaction, should they be
> > moved to some special group like 'nodefault' to prevent it from
> > failing with timeout in automated tests?
>
> The tests shouldn't be a problem when ncat does its job.
>
>
>
> I still think we have a script/ncat issue here. Please provide
>
>  qemu version:
>  bash version:
>  ncat version:
>
> And the distro and distro version might also be helpful.

Hi Andrew,
thanks for the info, here is my system information:
* Ubuntu Hirsute 21.04 (5.11.0-18-generic)
* qemu version (qemu-system-arm) - QEMU emulator version 5.2.0 (Debian
1:5.2+dfsg-9ubuntu3)
* bash version - GNU bash, version 5.1.4(1)-release (aarch64-unknown-linux-gnu)
* ncat version - Ncat: Version 7.80 ( https://nmap.org/ncat )

Cheers

>
> Thanks,
> drew
>

Re: [kvm-unit-tests] its-migration segmentation fault

[Po-Hsu Lin]

On Tue, Jun 15, 2021 at 3:11 PM Po-Hsu Lin <po-hsu.lin@canonical.com> wrote:
>
> On Tue, Jun 15, 2021 at 2:37 PM Andrew Jones <drjones@redhat.com> wrote:
> >
> > On Tue, Jun 15, 2021 at 11:21:05AM +0800, Po-Hsu Lin wrote:
> > > On Fri, Nov 20, 2020 at 8:35 PM Andrew Jones <drjones@redhat.com> wrote:
> > > >
> > > > On Fri, Nov 20, 2020 at 12:02:10PM +0000, Alexandru Elisei wrote:
> > > > > When running all the tests with taskset -c 0-3 ./run_tests.sh on a rockpro64 (on
> > > > > the Cortex-a53 cores) the its-migration test hangs. In the log file I see:
> > > > >
> > > > > run_migration timeout -k 1s --foreground 90s /usr/bin/qemu-system-aarch64
> > > > > -nodefaults -machine virt,gic-version=host,accel=kvm -cpu host -device
> > > > > virtio-serial-device -device virtconsole,chardev=ctd -chardev testdev,id=ctd
> > > > > -device pci-testdev -display none -serial stdio -kernel arm/gic.flat -smp 6
> > > > > -machine gic-version=3 -append its-migration # -initrd /tmp/tmp.OrlQiorBpY
> > > > > ITS: MAPD devid=2 size = 0x8 itt=0x40420000 valid=1
> > > > > ITS: MAPD devid=7 size = 0x8 itt=0x40430000 valid=1
> > > > > MAPC col_id=3 target_addr = 0x30000 valid=1
> > > > > MAPC col_id=2 target_addr = 0x20000 valid=1
> > > > > INVALL col_id=2
> > > > > INVALL col_id=3
> > > > > MAPTI dev_id=2 event_id=20 -> phys_id=8195, col_id=3
> > > > > MAPTI dev_id=7 event_id=255 -> phys_id=8196, col_id=2
> > > > > Now migrate the VM, then press a key to continue...
> > > > > scripts/arch-run.bash: line 103: 48549 Done                    echo '{ "execute":
> > > > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > > > >      48550 Segmentation fault      (core dumped) | ncat -U $1
> > > > > scripts/arch-run.bash: line 103: 48568 Done                    echo '{ "execute":
> > > > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > > > >      48569 Segmentation fault      (core dumped) | ncat -U $1
> > > > > scripts/arch-run.bash: line 103: 48583 Done                    echo '{ "execute":
> > > > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > > > >      48584 Segmentation fault      (core dumped) | ncat -U $1
> > > > > [..]
> > > > > scripts/arch-run.bash: line 103: 49414 Done                    echo '{ "execute":
> > > > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > > > >      49415 Segmentation fault      (core dumped) | ncat -U $1
> > > > > qemu-system-aarch64: terminating on signal 15 from pid 48496 (timeout)
> > > > > qemu-system-aarch64: terminating on signal 15 from pid 48504 (timeout)
> > > > > scripts/arch-run.bash: line 103: 49430 Done                    echo '{ "execute":
> > > > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > > > >      49431 Segmentation fault      (core dumped) | ncat -U $1
> > > > > scripts/arch-run.bash: line 103: 49445 Done                    echo '{ "execute":
> > > > > "qmp_capabilities" }{ "execute":' "$2" '}'
> > > > > [..]
> > > >
> > > > Is your ncat segfaulting? It looks like it from this output. Have you
> > > > tried running your ncat with a UNIX socket independently of this test?
> > > >
> > > > Is this the first time you've tried this test in this environment, or
> > > > is this a regression for you?
> > > >
> > > > >
> > > > > If I run the test manually:
> > > > >
> > > > > $ taskset -c 0-3 ./arm-run arm/gic.flat -smp 4 -machine gic-version=3 -append
> > > > > 'its-migration'
> > > >
> > > > This won't work because we need run_tests.sh to setup the run_migration()
> > > > call. The only ways to run migration tests separately are
> > > >
> > > >  $ ./run_tests.sh its-migration
> > > >
> > > > and
> > > >
> > > >  $ tests/its-migration
> > > >
> > > > For the second one you need to do 'make standalone' first.
> > > >
> > > >
> > > > >
> > > > > /usr/bin/qemu-system-aarch64 -nodefaults -machine virt,gic-version=host,accel=kvm
> > > > > -cpu host -device virtio-serial-device -device virtconsole,chardev=ctd -chardev
> > > > > testdev,id=ctd -device pci-testdev -display none -serial stdio -kernel
> > > > > arm/gic.flat -smp 4 -machine gic-version=3 -append its-migration # -initrd
> > > > > /tmp/tmp.OtsTj3QD4J
> > > > > ITS: MAPD devid=2 size = 0x8 itt=0x403a0000 valid=1
> > > > > ITS: MAPD devid=7 size = 0x8 itt=0x403b0000 valid=1
> > > > > MAPC col_id=3 target_addr = 0x30000 valid=1
> > > > > MAPC col_id=2 target_addr = 0x20000 valid=1
> > > > > INVALL col_id=2
> > > > > INVALL col_id=3
> > > > > MAPTI dev_id=2 event_id=20 -> phys_id=8195, col_id=3
> > > > > MAPTI dev_id=7 event_id=255 -> phys_id=8196, col_id=2
> > > > > Now migrate the VM, then press a key to continue...
> > > > >
> > > > > And the test hangs here after I press a key.
> > > >
> > > > The test doesn't get your input because of the '</dev/null' in run_qemu(),
> > > > which ./arm-run calls. So it's not hanging it's just waiting forever on
> > > > the key press.
> > > Hello Andrew,
> > > We have found this waiting for key press issue on our side as well
> > > [1], the test will fail with TIMEOUT, it looks like it's not getting
> > > my input like you mentioned here.
> > > I would like to ask what is the expected behaviour of these migration
> > > related tests (its-pending-migration / its-migration /
> > > its-migrate-unmapped-collection)? Should they pass right after the
> > > tester hit a key?
> >
> > They should, but normally users don't need to press a key, because the
> > script uses ncat to do it for them.
> >
> > > Also, if these test would require user interaction, should they be
> > > moved to some special group like 'nodefault' to prevent it from
> > > failing with timeout in automated tests?
> >
> > The tests shouldn't be a problem when ncat does its job.
> >
> >
> >
> > I still think we have a script/ncat issue here. Please provide
> >
> >  qemu version:
> >  bash version:
> >  ncat version:
> >
> > And the distro and distro version might also be helpful.
>
> Hi Andrew,
> thanks for the info, here is my system information:
> * Ubuntu Hirsute 21.04 (5.11.0-18-generic)
> * qemu version (qemu-system-arm) - QEMU emulator version 5.2.0 (Debian
> 1:5.2+dfsg-9ubuntu3)
> * bash version - GNU bash, version 5.1.4(1)-release (aarch64-unknown-linux-gnu)
> * ncat version - Ncat: Version 7.80 ( https://nmap.org/ncat )
>
> Cheers
>
Hello Andrew,
is there anything I can test here for this issue?
Thanks!

> >
> > Thanks,
> > drew
> >