From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S944831AbcJSSp7 (ORCPT ); Wed, 19 Oct 2016 14:45:59 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:52651 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933236AbcJSSpv (ORCPT ); Wed, 19 Oct 2016 14:45:51 -0400 Subject: Re: [PATCH v4 5/5] x86, kvm: support vcpu preempted check To: =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , Pan Xinhui References: <1476872416-42752-1-git-send-email-xinhui.pan@linux.vnet.ibm.com> <1476872416-42752-6-git-send-email-xinhui.pan@linux.vnet.ibm.com> <20161019172403.GA9240@potion> Cc: linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, virtualization@lists.linux-foundation.org, linux-s390@vger.kernel.org, xen-devel-request@lists.xenproject.org, kvm@vger.kernel.org, benh@kernel.crashing.org, paulus@samba.org, mpe@ellerman.id.au, mingo@redhat.com, peterz@infradead.org, paulmck@linux.vnet.ibm.com, will.deacon@arm.com, kernellwp@gmail.com, jgross@suse.com, pbonzini@redhat.com, bsingharora@gmail.com, boqun.feng@gmail.com, borntraeger@de.ibm.com From: Pan Xinhui Date: Thu, 20 Oct 2016 02:45:33 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <20161019172403.GA9240@potion> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16101918-0028-0000-0000-000005D8AF9E X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00005941; HX=3.00000240; KW=3.00000007; PH=3.00000004; SC=3.00000187; SDB=6.00770285; UDB=6.00369204; IPR=6.00546791; BA=6.00004820; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00013046; XFM=3.00000011; UTC=2016-10-19 18:45:48 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 16101918-0029-0000-0000-0000302A7F0D Message-Id: <34aefe78-af86-19b3-9e2f-cb3ee6b5f735@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-10-19_15:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1609300000 definitions=main-1610190334 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 在 2016/10/20 01:24, Radim Krčmář 写道: > 2016-10-19 06:20-0400, Pan Xinhui: >> This is to fix some lock holder preemption issues. Some other locks >> implementation do a spin loop before acquiring the lock itself. >> Currently kernel has an interface of bool vcpu_is_preempted(int cpu). It >> takes the cpu as parameter and return true if the cpu is preempted. Then >> kernel can break the spin loops upon on the retval of vcpu_is_preempted. >> >> As kernel has used this interface, So lets support it. >> >> We use one field of struct kvm_steal_time to indicate that if one vcpu >> is running or not. >> >> unix benchmark result: >> host: kernel 4.8.1, i5-4570, 4 cpus >> guest: kernel 4.8.1, 8 vcpus >> >> test-case after-patch before-patch >> Execl Throughput | 18307.9 lps | 11701.6 lps >> File Copy 1024 bufsize 2000 maxblocks | 1352407.3 KBps | 790418.9 KBps >> File Copy 256 bufsize 500 maxblocks | 367555.6 KBps | 222867.7 KBps >> File Copy 4096 bufsize 8000 maxblocks | 3675649.7 KBps | 1780614.4 KBps >> Pipe Throughput | 11872208.7 lps | 11855628.9 lps >> Pipe-based Context Switching | 1495126.5 lps | 1490533.9 lps >> Process Creation | 29881.2 lps | 28572.8 lps >> Shell Scripts (1 concurrent) | 23224.3 lpm | 22607.4 lpm >> Shell Scripts (8 concurrent) | 3531.4 lpm | 3211.9 lpm >> System Call Overhead | 10385653.0 lps | 10419979.0 lps >> >> Signed-off-by: Pan Xinhui >> --- >> diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h >> @@ -98,6 +98,10 @@ struct pv_time_ops { >> unsigned long long (*steal_clock)(int cpu); >> }; >> >> +struct pv_vcpu_ops { >> + bool (*vcpu_is_preempted)(int cpu); >> +}; >> + > > (I would put it into pv_lock_ops to save the plumbing.) > hi, Radim thanks for your reply. yes, a new struct leads patch into unnecessary lines changed. I do that just because I am not sure which existing xxx_ops I should place the vcpu_is_preempted in. >> diff --git a/arch/x86/include/uapi/asm/kvm_para.h b/arch/x86/include/uapi/asm/kvm_para.h >> @@ -45,7 +45,8 @@ struct kvm_steal_time { >> __u64 steal; >> __u32 version; >> __u32 flags; >> - __u32 pad[12]; >> + __u32 preempted; > > Why __u32 instead of __u8? > I thought it is 32-bits aligned... yes, u8 is good to store the preempt status. >> + __u32 pad[11]; >> }; > > Please document the change in Documentation/virtual/kvm/msr.txt, section > MSR_KVM_STEAL_TIME. > okay, I totally forgot to do that. thanks! >> diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c >> @@ -415,6 +415,15 @@ void kvm_disable_steal_time(void) >> +static bool kvm_vcpu_is_preempted(int cpu) >> +{ >> + struct kvm_steal_time *src; >> + >> + src = &per_cpu(steal_time, cpu); >> + >> + return !!src->preempted; >> +} >> + >> #ifdef CONFIG_SMP >> static void __init kvm_smp_prepare_boot_cpu(void) >> { >> @@ -488,6 +497,8 @@ void __init kvm_guest_init(void) >> kvm_guest_cpu_init(); >> #endif >> >> + pv_vcpu_ops.vcpu_is_preempted = kvm_vcpu_is_preempted; > > Would be nicer to assign conditionally in the KVM_FEATURE_STEAL_TIME > block. The steal_time structure has to be zeroed, so this code would > work, but the native function (return false) is better if we know that > the kvm_vcpu_is_preempted() would always return false anway. > yes, agree. Will do that. I once thought we can patch the code runtime. we replace binary code "call 0xXXXXXXXX #pv_vcpu_ops.vcpu_is_preempted" with "xor eax, eax" however it is not worth doing that. the performace improvements might be very small. > Old KVMs won't have the feature, so we could also assign only when KVM > reports it, but that requires extra definitions and the performance gain > is fairly small, so I'm ok with this. > >> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c >> @@ -2057,6 +2057,8 @@ static void record_steal_time(struct kvm_vcpu *vcpu) >> &vcpu->arch.st.steal, sizeof(struct kvm_steal_time)))) >> return; >> >> + vcpu->arch.st.steal.preempted = 0; >> + >> if (vcpu->arch.st.steal.version & 1) >> vcpu->arch.st.steal.version += 1; /* first time write, random junk */ >> >> @@ -2812,6 +2814,16 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) >> >> void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu) >> { >> + if (vcpu->arch.st.msr_val & KVM_MSR_ENABLED) >> + if (kvm_read_guest_cached(vcpu->kvm, &vcpu->arch.st.stime, >> + &vcpu->arch.st.steal, >> + sizeof(struct kvm_steal_time)) == 0) { >> + vcpu->arch.st.steal.preempted = 1; >> + kvm_write_guest_cached(vcpu->kvm, &vcpu->arch.st.stime, >> + &vcpu->arch.st.steal, >> + sizeof(struct kvm_steal_time)); >> + } > > Please name this block of code. Something like > kvm_steal_time_set_preempted(vcpu); > yep, my code style is ugly. will do that. thanks xinhui > Thanks. > From mboxrd@z Thu Jan 1 00:00:00 1970 From: Pan Xinhui Subject: Re: [PATCH v4 5/5] x86, kvm: support vcpu preempted check Date: Thu, 20 Oct 2016 02:45:33 +0800 Message-ID: <34aefe78-af86-19b3-9e2f-cb3ee6b5f735@linux.vnet.ibm.com> References: <1476872416-42752-1-git-send-email-xinhui.pan@linux.vnet.ibm.com> <1476872416-42752-6-git-send-email-xinhui.pan@linux.vnet.ibm.com> <20161019172403.GA9240@potion> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <20161019172403.GA9240@potion> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org List-Archive: List-Post: To: =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , Pan Xinhui Cc: kernellwp@gmail.com, linux-s390@vger.kernel.org, jgross@suse.com, kvm@vger.kernel.org, peterz@infradead.org, xen-devel-request@lists.xenproject.org, will.deacon@arm.com, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, mingo@redhat.com, paulus@samba.org, mpe@ellerman.id.au, benh@kernel.crashing.org, pbonzini@redhat.com, paulmck@linux.vnet.ibm.com, linuxppc-dev@lists.ozlabs.org, boqun.feng@gmail.com List-ID: CuWcqCAyMDE2LzEwLzIwIDAxOjI0LCBSYWRpbSBLcsSNbcOhxZkg5YaZ6YGTOgo+IDIwMTYtMTAt MTkgMDY6MjAtMDQwMCwgUGFuIFhpbmh1aToKPj4gVGhpcyBpcyB0byBmaXggc29tZSBsb2NrIGhv bGRlciBwcmVlbXB0aW9uIGlzc3Vlcy4gU29tZSBvdGhlciBsb2Nrcwo+PiBpbXBsZW1lbnRhdGlv biBkbyBhIHNwaW4gbG9vcCBiZWZvcmUgYWNxdWlyaW5nIHRoZSBsb2NrIGl0c2VsZi4KPj4gQ3Vy cmVudGx5IGtlcm5lbCBoYXMgYW4gaW50ZXJmYWNlIG9mIGJvb2wgdmNwdV9pc19wcmVlbXB0ZWQo aW50IGNwdSkuIEl0Cj4+IHRha2VzIHRoZSBjcHUgYXMgcGFyYW1ldGVyIGFuZCByZXR1cm4gdHJ1 ZSBpZiB0aGUgY3B1IGlzIHByZWVtcHRlZC4gIFRoZW4KPj4ga2VybmVsIGNhbiBicmVhayB0aGUg c3BpbiBsb29wcyB1cG9uIG9uIHRoZSByZXR2YWwgb2YgdmNwdV9pc19wcmVlbXB0ZWQuCj4+Cj4+ IEFzIGtlcm5lbCBoYXMgdXNlZCB0aGlzIGludGVyZmFjZSwgU28gbGV0cyBzdXBwb3J0IGl0Lgo+ Pgo+PiBXZSB1c2Ugb25lIGZpZWxkIG9mIHN0cnVjdCBrdm1fc3RlYWxfdGltZSB0byBpbmRpY2F0 ZSB0aGF0IGlmIG9uZSB2Y3B1Cj4+IGlzIHJ1bm5pbmcgb3Igbm90Lgo+Pgo+PiB1bml4IGJlbmNo bWFyayByZXN1bHQ6Cj4+IGhvc3Q6ICBrZXJuZWwgNC44LjEsIGk1LTQ1NzAsIDQgY3B1cwo+PiBn dWVzdDoga2VybmVsIDQuOC4xLCA4IHZjcHVzCj4+Cj4+IAl0ZXN0LWNhc2UJCQlhZnRlci1wYXRj aAkgIGJlZm9yZS1wYXRjaAo+PiBFeGVjbCBUaHJvdWdocHV0ICAgICAgICAgICAgICAgICAgICAg ICB8ICAgIDE4MzA3LjkgbHBzICB8ICAgIDExNzAxLjYgbHBzCj4+IEZpbGUgQ29weSAxMDI0IGJ1 ZnNpemUgMjAwMCBtYXhibG9ja3MgIHwgIDEzNTI0MDcuMyBLQnBzIHwgICA3OTA0MTguOSBLQnBz Cj4+IEZpbGUgQ29weSAyNTYgYnVmc2l6ZSA1MDAgbWF4YmxvY2tzICAgIHwgICAzNjc1NTUuNiBL QnBzIHwgICAyMjI4NjcuNyBLQnBzCj4+IEZpbGUgQ29weSA0MDk2IGJ1ZnNpemUgODAwMCBtYXhi bG9ja3MgIHwgIDM2NzU2NDkuNyBLQnBzIHwgIDE3ODA2MTQuNCBLQnBzCj4+IFBpcGUgVGhyb3Vn aHB1dCAgICAgICAgICAgICAgICAgICAgICAgIHwgMTE4NzIyMDguNyBscHMgIHwgMTE4NTU2Mjgu OSBscHMKPj4gUGlwZS1iYXNlZCBDb250ZXh0IFN3aXRjaGluZyAgICAgICAgICAgfCAgMTQ5NTEy Ni41IGxwcyAgfCAgMTQ5MDUzMy45IGxwcwo+PiBQcm9jZXNzIENyZWF0aW9uICAgICAgICAgICAg ICAgICAgICAgICB8ICAgIDI5ODgxLjIgbHBzICB8ICAgIDI4NTcyLjggbHBzCj4+IFNoZWxsIFNj cmlwdHMgKDEgY29uY3VycmVudCkgICAgICAgICAgIHwgICAgMjMyMjQuMyBscG0gIHwgICAgMjI2 MDcuNCBscG0KPj4gU2hlbGwgU2NyaXB0cyAoOCBjb25jdXJyZW50KSAgICAgICAgICAgfCAgICAg MzUzMS40IGxwbSAgfCAgICAgMzIxMS45IGxwbQo+PiBTeXN0ZW0gQ2FsbCBPdmVyaGVhZCAgICAg ICAgICAgICAgICAgICB8IDEwMzg1NjUzLjAgbHBzICB8IDEwNDE5OTc5LjAgbHBzCj4+Cj4+IFNp Z25lZC1vZmYtYnk6IFBhbiBYaW5odWkgPHhpbmh1aS5wYW5AbGludXgudm5ldC5pYm0uY29tPgo+ PiAtLS0KPj4gZGlmZiAtLWdpdCBhL2FyY2gveDg2L2luY2x1ZGUvYXNtL3BhcmF2aXJ0X3R5cGVz LmggYi9hcmNoL3g4Ni9pbmNsdWRlL2FzbS9wYXJhdmlydF90eXBlcy5oCj4+IEBAIC05OCw2ICs5 OCwxMCBAQCBzdHJ1Y3QgcHZfdGltZV9vcHMgewo+PiAgCXVuc2lnbmVkIGxvbmcgbG9uZyAoKnN0 ZWFsX2Nsb2NrKShpbnQgY3B1KTsKPj4gIH07Cj4+Cj4+ICtzdHJ1Y3QgcHZfdmNwdV9vcHMgewo+ PiArCWJvb2wgKCp2Y3B1X2lzX3ByZWVtcHRlZCkoaW50IGNwdSk7Cj4+ICt9Owo+PiArCj4KPiAo SSB3b3VsZCBwdXQgaXQgaW50byBwdl9sb2NrX29wcyB0byBzYXZlIHRoZSBwbHVtYmluZy4pCj4K aGksIFJhZGltCgl0aGFua3MgZm9yIHlvdXIgcmVwbHkuCgp5ZXMsIGEgbmV3IHN0cnVjdCBsZWFk cyBwYXRjaCBpbnRvIHVubmVjZXNzYXJ5IGxpbmVzIGNoYW5nZWQuCkkgZG8gdGhhdCBqdXN0IGJl Y2F1c2UgSSBhbSBub3Qgc3VyZSB3aGljaCBleGlzdGluZyB4eHhfb3BzIEkgc2hvdWxkIHBsYWNl IHRoZSB2Y3B1X2lzX3ByZWVtcHRlZCBpbi4KCj4+IGRpZmYgLS1naXQgYS9hcmNoL3g4Ni9pbmNs dWRlL3VhcGkvYXNtL2t2bV9wYXJhLmggYi9hcmNoL3g4Ni9pbmNsdWRlL3VhcGkvYXNtL2t2bV9w YXJhLmgKPj4gQEAgLTQ1LDcgKzQ1LDggQEAgc3RydWN0IGt2bV9zdGVhbF90aW1lIHsKPj4gIAlf X3U2NCBzdGVhbDsKPj4gIAlfX3UzMiB2ZXJzaW9uOwo+PiAgCV9fdTMyIGZsYWdzOwo+PiAtCV9f dTMyIHBhZFsxMl07Cj4+ICsJX191MzIgcHJlZW1wdGVkOwo+Cj4gV2h5IF9fdTMyIGluc3RlYWQg b2YgX191OD8KPgpJIHRob3VnaHQgaXQgaXMgMzItYml0cyBhbGlnbmVkLi4uCnllcywgdTggaXMg Z29vZCB0byBzdG9yZSB0aGUgcHJlZW1wdCBzdGF0dXMuCgo+PiArCV9fdTMyIHBhZFsxMV07Cj4+ ICB9Owo+Cj4gUGxlYXNlIGRvY3VtZW50IHRoZSBjaGFuZ2UgaW4gRG9jdW1lbnRhdGlvbi92aXJ0 dWFsL2t2bS9tc3IudHh0LCBzZWN0aW9uCj4gTVNSX0tWTV9TVEVBTF9USU1FLgo+Cm9rYXksIEkg dG90YWxseSBmb3Jnb3QgdG8gZG8gdGhhdC4gdGhhbmtzIQoKPj4gZGlmZiAtLWdpdCBhL2FyY2gv eDg2L2tlcm5lbC9rdm0uYyBiL2FyY2gveDg2L2tlcm5lbC9rdm0uYwo+PiBAQCAtNDE1LDYgKzQx NSwxNSBAQCB2b2lkIGt2bV9kaXNhYmxlX3N0ZWFsX3RpbWUodm9pZCkKPj4gK3N0YXRpYyBib29s IGt2bV92Y3B1X2lzX3ByZWVtcHRlZChpbnQgY3B1KQo+PiArewo+PiArCXN0cnVjdCBrdm1fc3Rl YWxfdGltZSAqc3JjOwo+PiArCj4+ICsJc3JjID0gJnBlcl9jcHUoc3RlYWxfdGltZSwgY3B1KTsK Pj4gKwo+PiArCXJldHVybiAhIXNyYy0+cHJlZW1wdGVkOwo+PiArfQo+PiArCj4+ICAjaWZkZWYg Q09ORklHX1NNUAo+PiAgc3RhdGljIHZvaWQgX19pbml0IGt2bV9zbXBfcHJlcGFyZV9ib290X2Nw dSh2b2lkKQo+PiAgewo+PiBAQCAtNDg4LDYgKzQ5Nyw4IEBAIHZvaWQgX19pbml0IGt2bV9ndWVz dF9pbml0KHZvaWQpCj4+ICAJa3ZtX2d1ZXN0X2NwdV9pbml0KCk7Cj4+ICAjZW5kaWYKPj4KPj4g Kwlwdl92Y3B1X29wcy52Y3B1X2lzX3ByZWVtcHRlZCA9IGt2bV92Y3B1X2lzX3ByZWVtcHRlZDsK Pgo+IFdvdWxkIGJlIG5pY2VyIHRvIGFzc2lnbiBjb25kaXRpb25hbGx5IGluIHRoZSBLVk1fRkVB VFVSRV9TVEVBTF9USU1FCj4gYmxvY2suICBUaGUgc3RlYWxfdGltZSBzdHJ1Y3R1cmUgaGFzIHRv IGJlIHplcm9lZCwgc28gdGhpcyBjb2RlIHdvdWxkCj4gd29yaywgYnV0IHRoZSBuYXRpdmUgZnVu Y3Rpb24gKHJldHVybiBmYWxzZSkgaXMgYmV0dGVyIGlmIHdlIGtub3cgdGhhdAo+IHRoZSBrdm1f dmNwdV9pc19wcmVlbXB0ZWQoKSB3b3VsZCBhbHdheXMgcmV0dXJuIGZhbHNlIGFud2F5Lgo+Cnll cywgYWdyZWUuIFdpbGwgZG8gdGhhdC4KCkkgb25jZSB0aG91Z2h0IHdlIGNhbiBwYXRjaCB0aGUg Y29kZSBydW50aW1lLgp3ZSByZXBsYWNlIGJpbmFyeSBjb2RlCiJjYWxsIDB4WFhYWFhYWFggI3B2 X3ZjcHVfb3BzLnZjcHVfaXNfcHJlZW1wdGVkIgp3aXRoCiJ4b3IgZWF4LCBlYXgiCmhvd2V2ZXIg aXQgaXMgbm90IHdvcnRoIGRvaW5nIHRoYXQuIHRoZSBwZXJmb3JtYWNlIGltcHJvdmVtZW50cyBt aWdodCBiZSB2ZXJ5IHNtYWxsLgoKPiBPbGQgS1ZNcyB3b24ndCBoYXZlIHRoZSBmZWF0dXJlLCBz byB3ZSBjb3VsZCBhbHNvIGFzc2lnbiBvbmx5IHdoZW4gS1ZNCj4gcmVwb3J0cyBpdCwgYnV0IHRo YXQgcmVxdWlyZXMgZXh0cmEgZGVmaW5pdGlvbnMgYW5kIHRoZSBwZXJmb3JtYW5jZSBnYWluCj4g aXMgZmFpcmx5IHNtYWxsLCBzbyBJJ20gb2sgd2l0aCB0aGlzLgo+Cj4+IGRpZmYgLS1naXQgYS9h cmNoL3g4Ni9rdm0veDg2LmMgYi9hcmNoL3g4Ni9rdm0veDg2LmMKPj4gQEAgLTIwNTcsNiArMjA1 Nyw4IEBAIHN0YXRpYyB2b2lkIHJlY29yZF9zdGVhbF90aW1lKHN0cnVjdCBrdm1fdmNwdSAqdmNw dSkKPj4gIAkJJnZjcHUtPmFyY2guc3Quc3RlYWwsIHNpemVvZihzdHJ1Y3Qga3ZtX3N0ZWFsX3Rp bWUpKSkpCj4+ICAJCXJldHVybjsKPj4KPj4gKwl2Y3B1LT5hcmNoLnN0LnN0ZWFsLnByZWVtcHRl ZCA9IDA7Cj4+ICsKPj4gIAlpZiAodmNwdS0+YXJjaC5zdC5zdGVhbC52ZXJzaW9uICYgMSkKPj4g IAkJdmNwdS0+YXJjaC5zdC5zdGVhbC52ZXJzaW9uICs9IDE7ICAvKiBmaXJzdCB0aW1lIHdyaXRl LCByYW5kb20ganVuayAqLwo+Pgo+PiBAQCAtMjgxMiw2ICsyODE0LDE2IEBAIHZvaWQga3ZtX2Fy Y2hfdmNwdV9sb2FkKHN0cnVjdCBrdm1fdmNwdSAqdmNwdSwgaW50IGNwdSkKPj4KPj4gIHZvaWQg a3ZtX2FyY2hfdmNwdV9wdXQoc3RydWN0IGt2bV92Y3B1ICp2Y3B1KQo+PiAgewo+PiArCWlmICh2 Y3B1LT5hcmNoLnN0Lm1zcl92YWwgJiBLVk1fTVNSX0VOQUJMRUQpCj4+ICsJCWlmIChrdm1fcmVh ZF9ndWVzdF9jYWNoZWQodmNwdS0+a3ZtLCAmdmNwdS0+YXJjaC5zdC5zdGltZSwKPj4gKwkJCQkJ JnZjcHUtPmFyY2guc3Quc3RlYWwsCj4+ICsJCQkJCXNpemVvZihzdHJ1Y3Qga3ZtX3N0ZWFsX3Rp bWUpKSA9PSAwKSB7Cj4+ICsJCQl2Y3B1LT5hcmNoLnN0LnN0ZWFsLnByZWVtcHRlZCA9IDE7Cj4+ ICsJCQlrdm1fd3JpdGVfZ3Vlc3RfY2FjaGVkKHZjcHUtPmt2bSwgJnZjcHUtPmFyY2guc3Quc3Rp bWUsCj4+ICsJCQkJCSZ2Y3B1LT5hcmNoLnN0LnN0ZWFsLAo+PiArCQkJCQlzaXplb2Yoc3RydWN0 IGt2bV9zdGVhbF90aW1lKSk7Cj4+ICsJCX0KPgo+IFBsZWFzZSBuYW1lIHRoaXMgYmxvY2sgb2Yg Y29kZS4gIFNvbWV0aGluZyBsaWtlCj4gICBrdm1fc3RlYWxfdGltZV9zZXRfcHJlZW1wdGVkKHZj cHUpOwo+CnllcCwgbXkgY29kZSBzdHlsZSBpcyB1Z2x5Lgp3aWxsIGRvIHRoYXQuCgp0aGFua3MK eGluaHVpCgoKPiBUaGFua3MuCj4KCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fClZpcnR1YWxpemF0aW9uIG1haWxpbmcgbGlzdApWaXJ0dWFsaXphdGlvbkBs aXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZwpodHRwczovL2xpc3RzLmxpbnV4Zm91bmRhdGlvbi5v cmcvbWFpbG1hbi9saXN0aW5mby92aXJ0dWFsaXphdGlvbg==