branch xen-unstable-smoke xenbranch xen-unstable-smoke job build-amd64 testid xen-build Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git Tree: qemuu git://xenbits.xen.org/qemu-xen.git Tree: xen git://xenbits.xen.org/xen.git *** Found and reproduced problem changeset *** Bug is in tree: xen git://xenbits.xen.org/xen.git Bug introduced: d162f36848c4a98a782cc05820b0aa7ec1ae297d Bug not present: 6131dab5f2c8059a0fc7fd884bc6d4ff78ba44c2 Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/157774/ commit d162f36848c4a98a782cc05820b0aa7ec1ae297d Author: Andrew Cooper Date: Mon Sep 28 15:25:44 2020 +0100 xen/x86: Fix memory leak in vcpu_create() error path Various paths in vcpu_create() end up calling paging_update_paging_modes(), which eventually allocate a monitor pagetable if one doesn't exist. However, an error in vcpu_create() results in the vcpu being cleaned up locally, and not put onto the domain's vcpu list. Therefore, the monitor table is not freed by {hap,shadow}_teardown()'s loop. This is caught by assertions later that we've successfully freed the entire hap/shadow memory pool. The per-vcpu loops in domain teardown logic is conceptually wrong, but exist due to insufficient existing structure in the existing logic. Break paging_vcpu_teardown() out of paging_teardown(), with mirrored breakouts in the hap/shadow code, and use it from arch_vcpu_create()'s error path. This fixes the memory leak. The new {hap,shadow}_vcpu_teardown() must be idempotent, and are written to be as tolerable as possible, with the minimum number of safety checks possible. In particular, drop the mfn_valid() check - if these fields are junk, then Xen is going to explode anyway. Reported-by: Michał Leszczyński Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich For bisection revision-tuple graph see: http://logs.test-lab.xenproject.org/osstest/results/bisect/xen-unstable-smoke/build-amd64.xen-build.html Revision IDs in each graph node refer, respectively, to the Trees above. ---------------------------------------- Running cs-bisection-step --graph-out=/home/logs/results/bisect/xen-unstable-smoke/build-amd64.xen-build --summary-out=tmp/157774.bisection-summary --basis-template=157696 --blessings=real,real-bisect,real-retry xen-unstable-smoke build-amd64 xen-build Searching for failure / basis pass: 157761 fail [host=himrod1] / 157696 ok. Failure / basis pass flights: 157761 / 157696 (tree with no url: minios) (tree with no url: ovmf) (tree with no url: seabios) Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git Tree: qemuu git://xenbits.xen.org/qemu-xen.git Tree: xen git://xenbits.xen.org/xen.git Latest 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 7ea428895af2840d85c524f0bd11a38aac308308 d162f36848c4a98a782cc05820b0aa7ec1ae297d Basis pass 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 7ea428895af2840d85c524f0bd11a38aac308308 357db96a66e47e609c3b14768f1062e13eedbd93 Generating revisions with ./adhoc-revtuple-generator git://xenbits.xen.org/qemu-xen-traditional.git#3d273dd05e51e5a1ffba3d98c7437ee84e8f8764-3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 git://xenbits.xen.org/qemu-xen.git#7ea428895af2840d85c524f0bd11a38aac308308-7ea428895af2840d85c524f0bd11a38aac308308 git://xenbits.xen.org/xen.git#357db96a66e47e609c3b14768f1062e13eedbd93-d162f36848c4a98a782cc05820b0aa7ec1ae297d Loaded 5001 nodes in revision graph Searching for test results: 157691 [host=himrod2] 157692 [host=himrod2] 157694 [host=himrod2] 157695 pass irrelevant 157697 fail irrelevant 157698 pass irrelevant 157699 pass irrelevant 157700 pass irrelevant 157701 pass irrelevant 157702 pass irrelevant 157703 fail irrelevant 157704 pass irrelevant 157706 fail irrelevant 157696 pass 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 7ea428895af2840d85c524f0bd11a38aac308308 357db96a66e47e609c3b14768f1062e13eedbd93 157707 pass irrelevant 157761 fail 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 7ea428895af2840d85c524f0bd11a38aac308308 d162f36848c4a98a782cc05820b0aa7ec1ae297d 157765 pass 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 7ea428895af2840d85c524f0bd11a38aac308308 357db96a66e47e609c3b14768f1062e13eedbd93 157767 fail 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 7ea428895af2840d85c524f0bd11a38aac308308 d162f36848c4a98a782cc05820b0aa7ec1ae297d 157768 pass 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 7ea428895af2840d85c524f0bd11a38aac308308 6131dab5f2c8059a0fc7fd884bc6d4ff78ba44c2 157769 fail 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 7ea428895af2840d85c524f0bd11a38aac308308 d162f36848c4a98a782cc05820b0aa7ec1ae297d 157770 pass 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 7ea428895af2840d85c524f0bd11a38aac308308 6131dab5f2c8059a0fc7fd884bc6d4ff78ba44c2 157771 fail 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 7ea428895af2840d85c524f0bd11a38aac308308 d162f36848c4a98a782cc05820b0aa7ec1ae297d 157773 pass 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 7ea428895af2840d85c524f0bd11a38aac308308 6131dab5f2c8059a0fc7fd884bc6d4ff78ba44c2 157774 fail 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 7ea428895af2840d85c524f0bd11a38aac308308 d162f36848c4a98a782cc05820b0aa7ec1ae297d Searching for interesting versions Result found: flight 157696 (pass), for basis pass For basis failure, parent search stopping at 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 7ea428895af2840d85c524f0bd11a38aac308308 6131dab5f2c8059a0fc7fd884bc6d4ff78ba44c2, results HASH(0x55feabd515e0) HASH(0x55feabd4ab20) HASH(0x55feabd56098) For basis failure, parent search stopping at 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 7ea428895af2840d85c524f0bd11a38aac308308 357db96a66e47e609c3b14768f1062e13eedbd93, results HASH(0x55feabd3f660) HASH(0x55feabd521e0) Result found: flight 157761 (fail), for \ basis failure (at ancestor ~710) Repro found: flight 157765 (pass), for basis pass Repro found: flight 157767 (fail), for basis failure 0 revisions at 3d273dd05e51e5a1ffba3d98c7437ee84e8f8764 7ea428895af2840d85c524f0bd11a38aac308308 6131dab5f2c8059a0fc7fd884bc6d4ff78ba44c2 No revisions left to test, checking graph state. Result found: flight 157768 (pass), for last pass Result found: flight 157769 (fail), for first failure Repro found: flight 157770 (pass), for last pass Repro found: flight 157771 (fail), for first failure Repro found: flight 157773 (pass), for last pass Repro found: flight 157774 (fail), for first failure *** Found and reproduced problem changeset *** Bug is in tree: xen git://xenbits.xen.org/xen.git Bug introduced: d162f36848c4a98a782cc05820b0aa7ec1ae297d Bug not present: 6131dab5f2c8059a0fc7fd884bc6d4ff78ba44c2 Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/157774/ commit d162f36848c4a98a782cc05820b0aa7ec1ae297d Author: Andrew Cooper Date: Mon Sep 28 15:25:44 2020 +0100 xen/x86: Fix memory leak in vcpu_create() error path Various paths in vcpu_create() end up calling paging_update_paging_modes(), which eventually allocate a monitor pagetable if one doesn't exist. However, an error in vcpu_create() results in the vcpu being cleaned up locally, and not put onto the domain's vcpu list. Therefore, the monitor table is not freed by {hap,shadow}_teardown()'s loop. This is caught by assertions later that we've successfully freed the entire hap/shadow memory pool. The per-vcpu loops in domain teardown logic is conceptually wrong, but exist due to insufficient existing structure in the existing logic. Break paging_vcpu_teardown() out of paging_teardown(), with mirrored breakouts in the hap/shadow code, and use it from arch_vcpu_create()'s error path. This fixes the memory leak. The new {hap,shadow}_vcpu_teardown() must be idempotent, and are written to be as tolerable as possible, with the minimum number of safety checks possible. In particular, drop the mfn_valid() check - if these fields are junk, then Xen is going to explode anyway. Reported-by: Michał Leszczyński Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich Revision graph left in /home/logs/results/bisect/xen-unstable-smoke/build-amd64.xen-build.{dot,ps,png,html,svg}. ---------------------------------------- 157774: tolerable ALL FAIL flight 157774 xen-unstable-smoke real-bisect [real] http://logs.test-lab.xenproject.org/osstest/logs/157774/ Failures :-/ but no regressions. Tests which did not succeed, including tests which could not be run: build-amd64 6 xen-build fail baseline untested jobs: build-amd64 fail ------------------------------------------------------------ sg-report-flight on osstest.test-lab.xenproject.org logs: /home/logs/logs images: /home/logs/images Logs, config files, etc. are available at http://logs.test-lab.xenproject.org/osstest/logs Explanation of these reports, and of osstest in general, is at http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master Test harness code can be found at http://xenbits.xen.org/gitweb?p=osstest.git;a=summary