From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7671AC7618B for ; Thu, 25 Jul 2019 13:26:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4BDFA2190F for ; Thu, 25 Jul 2019 13:26:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727287AbfGYN05 (ORCPT ); Thu, 25 Jul 2019 09:26:57 -0400 Received: from mail-qk1-f169.google.com ([209.85.222.169]:37637 "EHLO mail-qk1-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726973AbfGYN04 (ORCPT ); Thu, 25 Jul 2019 09:26:56 -0400 Received: by mail-qk1-f169.google.com with SMTP id d15so36352908qkl.4 for ; Thu, 25 Jul 2019 06:26:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=KjjBu7BMfk8xmpq06aF6LWsxATJ7zX54K3XmB9nU8ns=; b=FKktEgdqRr/upzCiXoLoNdrCmSbaIy7TpjnhpttZZjp1BIgpDuGLAJnfFnGnjuzBv0 qd8XwkCqqfQZGppcvVqHj9ZYm2ljd/eyZom1DYb6FpuhBx7yzEmGr3gVIGmWp27Mn2qB nJWZUnfpy3sySNJKZX/5AP84L7JY/FQwjjX5Kr0a4lRzrnHG052M34T9MGiPnMzStMKS PqRPPxW2m0BSU7HdI0loN96RGBhTJusqnL75is9PtKTr8tMpfG0vD6YLCGQrdjCNUW4X xeWb3F8FFvhZKYZUrZKP2D6Gw+bdZkuQIHcuhgqvnMzYcdCAc3rM1RPa2y3yXPTqKWzt 1s/A== X-Gm-Message-State: APjAAAWvqdm1NriWq5QI4bZtBV1KInuz56+CWyA0e+ACoED4WpgHN03I Zjk66bEi+MlSlno20hFkA77OPw== X-Google-Smtp-Source: APXvYqxOAsHTrNRpwq6vcJ7tGMHpKurQRE42kOps0/crxS2PgQFMsqddCIqZtIPX1c6ngHTE5rQAdg== X-Received: by 2002:a05:620a:1447:: with SMTP id i7mr59552516qkl.254.1564061215681; Thu, 25 Jul 2019 06:26:55 -0700 (PDT) Received: from redhat.com (bzq-79-181-91-42.red.bezeqint.net. [79.181.91.42]) by smtp.gmail.com with ESMTPSA id 47sm29804600qtw.90.2019.07.25.06.26.48 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 25 Jul 2019 06:26:54 -0700 (PDT) Date: Thu, 25 Jul 2019 09:26:46 -0400 From: "Michael S. Tsirkin" To: Jason Wang Cc: syzbot , aarcange@redhat.com, akpm@linux-foundation.org, christian@brauner.io, davem@davemloft.net, ebiederm@xmission.com, elena.reshetova@intel.com, guro@fb.com, hch@infradead.org, james.bottomley@hansenpartnership.com, jglisse@redhat.com, keescook@chromium.org, ldv@altlinux.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-parisc@vger.kernel.org, luto@amacapital.net, mhocko@suse.com, mingo@kernel.org, namit@vmware.com, peterz@infradead.org, syzkaller-bugs@googlegroups.com, viro@zeniv.linux.org.uk, wad@chromium.org Subject: Re: WARNING in __mmdrop Message-ID: <20190725092332-mutt-send-email-mst@kernel.org> References: <20190723032024-mutt-send-email-mst@kernel.org> <1d14de4d-0133-1614-9f64-3ded381de04e@redhat.com> <20190723035725-mutt-send-email-mst@kernel.org> <3f4178f1-0d71-e032-0f1f-802428ceca59@redhat.com> <20190723051828-mutt-send-email-mst@kernel.org> <20190725012149-mutt-send-email-mst@kernel.org> <55e8930c-2695-365f-a07b-3ad169654d28@redhat.com> <20190725042651-mutt-send-email-mst@kernel.org> <84bb2e31-0606-adff-cf2a-e1878225a847@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <84bb2e31-0606-adff-cf2a-e1878225a847@redhat.com> Sender: linux-parisc-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-parisc@vger.kernel.org On Thu, Jul 25, 2019 at 09:21:22PM +0800, Jason Wang wrote: > > On 2019/7/25 下午4:28, Michael S. Tsirkin wrote: > > On Thu, Jul 25, 2019 at 03:43:41PM +0800, Jason Wang wrote: > > > On 2019/7/25 下午1:52, Michael S. Tsirkin wrote: > > > > On Tue, Jul 23, 2019 at 09:31:35PM +0800, Jason Wang wrote: > > > > > On 2019/7/23 下午5:26, Michael S. Tsirkin wrote: > > > > > > On Tue, Jul 23, 2019 at 04:49:01PM +0800, Jason Wang wrote: > > > > > > > On 2019/7/23 下午4:10, Michael S. Tsirkin wrote: > > > > > > > > On Tue, Jul 23, 2019 at 03:53:06PM +0800, Jason Wang wrote: > > > > > > > > > On 2019/7/23 下午3:23, Michael S. Tsirkin wrote: > > > > > > > > > > > > Really let's just use kfree_rcu. It's way cleaner: fire and forget. > > > > > > > > > > > Looks not, you need rate limit the fire as you've figured out? > > > > > > > > > > See the discussion that followed. Basically no, it's good enough > > > > > > > > > > already and is only going to be better. > > > > > > > > > > > > > > > > > > > > > And in fact, > > > > > > > > > > > the synchronization is not even needed, does it help if I leave a comment to > > > > > > > > > > > explain? > > > > > > > > > > Let's try to figure it out in the mail first. I'm pretty sure the > > > > > > > > > > current logic is wrong. > > > > > > > > > Here is what the code what to achieve: > > > > > > > > > > > > > > > > > > - The map was protected by RCU > > > > > > > > > > > > > > > > > > - Writers are: MMU notifier invalidation callbacks, file operations (ioctls > > > > > > > > > etc), meta_prefetch (datapath) > > > > > > > > > > > > > > > > > > - Readers are: memory accessor > > > > > > > > > > > > > > > > > > Writer are synchronized through mmu_lock. RCU is used to synchronized > > > > > > > > > between writers and readers. > > > > > > > > > > > > > > > > > > The synchronize_rcu() in vhost_reset_vq_maps() was used to synchronized it > > > > > > > > > with readers (memory accessors) in the path of file operations. But in this > > > > > > > > > case, vq->mutex was already held, this means it has been serialized with > > > > > > > > > memory accessor. That's why I think it could be removed safely. > > > > > > > > > > > > > > > > > > Anything I miss here? > > > > > > > > > > > > > > > > > So invalidate callbacks need to reset the map, and they do > > > > > > > > not have vq mutex. How can they do this and free > > > > > > > > the map safely? They need synchronize_rcu or kfree_rcu right? > > > > > > > Invalidation callbacks need but file operations (e.g ioctl) not. > > > > > > > > > > > > > > > > > > > > > > And I worry somewhat that synchronize_rcu in an MMU notifier > > > > > > > > is a problem, MMU notifiers are supposed to be quick: > > > > > > > Looks not, since it can allow to be blocked and lots of driver depends on > > > > > > > this. (E.g mmu_notifier_range_blockable()). > > > > > > Right, they can block. So why don't we take a VQ mutex and be > > > > > > done with it then? No RCU tricks. > > > > > This is how I want to go with RFC and V1. But I end up with deadlock between > > > > > vq locks and some MM internal locks. So I decide to use RCU which is 100% > > > > > under the control of vhost. > > > > > > > > > > Thanks > > > > And I guess the deadlock is because GUP is taking mmu locks which are > > > > taken on mmu notifier path, right? > > > > > > Yes, but it's not the only lock. I don't remember the details, but I can > > > confirm I meet issues with one or two other locks. > > > > > > > > > > How about we add a seqlock and take > > > > that in invalidate callbacks? We can then drop the VQ lock before GUP, > > > > and take it again immediately after. > > > > > > > > something like > > > > if (!vq_meta_mapped(vq)) { > > > > vq_meta_setup(&uaddrs); > > > > mutex_unlock(vq->mutex) > > > > vq_meta_map(&uaddrs); > > > > > > The problem is the vq address could be changed at this time. > > > > > > > > > > mutex_lock(vq->mutex) > > > > > > > > /* recheck both sock->private_data and seqlock count. */ > > > > if changed - bail out > > > > } > > > > > > > > And also requires that VQ uaddrs is defined like this: > > > > - writers must have both vq mutex and dev mutex > > > > - readers must have either vq mutex or dev mutex > > > > > > > > > > > > That's a big change though. For now, how about switching to a per-vq SRCU? > > > > That is only a little bit more expensive than RCU, and we > > > > can use synchronize_srcu_expedited. > > > > > > > Consider we switch to use kfree_rcu(), what's the advantage of per-vq SRCU? > > > > > > Thanks > > > > I thought we established that notifiers must wait for > > all readers to finish before they mark page dirty, to > > prevent page from becoming dirty after address > > has been invalidated. > > Right? > > > Exactly, and that's the reason actually I use synchronize_rcu() there. > > So the concern is still the possible synchronize_expedited()? I think synchronize_srcu_expedited. synchronize_expedited sends lots of IPI and is bad for realtime VMs. > Can I do this > on through another series on top of the incoming V2? > > Thanks > The question is this: is this still a gain if we switch to the more expensive srcu? If yes then we can keep the feature on, if not we'll put it off until next release and think of better solutions. rcu->srcu is just a find and replace, don't see why we need to defer that. can be a separate patch for sure, but we need to know how well it works. -- MST From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 700F6C7618B for ; Thu, 25 Jul 2019 13:27:16 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4550222C7C for ; Thu, 25 Jul 2019 13:27:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="LdV9N+vz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4550222C7C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=0i5YbhJviKa6Rw3otk/oLGoahGG3sGtF+SCXTjcoQtc=; b=LdV9N+vzfW7O4d dYn12ZwXh3VP3Th1kMDffpu3e3/cxthNziQ7tVgO6soPiPtl2hKtze3sxlWpe7cc+3OE7baj1H3nP EoRq8zs30kM9YB59FQAN3wz8X8Aq7CQ8nfcFw/fet/aFgGfMiQUIPQZ1sDdP+jTUP2mEspsnrTGIr 3sYlAIjDgPwPtahz4AFVtCvcaok3m6VpjfXa3LPTIlVgvokVNSTnyN+90waodcrdgPgbB0ou7DJp5 dMxifcW7mnmMYHn9Gw2dg+0JA5IBwMVD4Zc1JFK5iqeU+NgszZVBedciLQ6dwMmFpEUg7JemPTJ/+ g7Cl9iq4FUFhrizMAlSA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1hqdlz-00036b-WE; Thu, 25 Jul 2019 13:27:08 +0000 Received: from mail-qk1-f171.google.com ([209.85.222.171]) by bombadil.infradead.org with esmtps (Exim 4.92 #3 (Red Hat Linux)) id 1hqdlo-00030w-PV for linux-arm-kernel@lists.infradead.org; Thu, 25 Jul 2019 13:26:58 +0000 Received: by mail-qk1-f171.google.com with SMTP id w190so36360176qkc.6 for ; Thu, 25 Jul 2019 06:26:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=KjjBu7BMfk8xmpq06aF6LWsxATJ7zX54K3XmB9nU8ns=; b=DeOSo2SxvcBKomPd3se27GYTgrkp188yQH0zT3/wMQfzX2l8fwcGSUDUcn9jHtyqFm +3WBynCl0FIKtIqaIm9DKFcMB3rVZ2YDOt3Dn3mDJ5zZUNC4rDYSY/uS5vu6s8Mp/05g faAbVFCYjLbqTw51xm611nbyW0LLZxerkcWFP7j6Clkt/6Smi88Kxi/4gLhJq+yj4KNi FVA3WqTFcQWXFbJZBCwjKbj2LHhNZ0jK+48vSv06fmjkj5+Isj6ZEfJ+eKGVPDwgkEyh g9XclMwrQ4xJw2YQu7YwffS0BlOxMQ6NZtT7KoSQSEaC9POj4C8zcwZbTRfH4iyUxIW5 f+JQ== X-Gm-Message-State: APjAAAUiOVAiiTe928FVhXvEbHSqdG7puVM88l5/uVb8fm/ujehYTeJN 0uZxuofmnd7DDN9N1LfrJHS6fw== X-Google-Smtp-Source: APXvYqxOAsHTrNRpwq6vcJ7tGMHpKurQRE42kOps0/crxS2PgQFMsqddCIqZtIPX1c6ngHTE5rQAdg== X-Received: by 2002:a05:620a:1447:: with SMTP id i7mr59552516qkl.254.1564061215681; Thu, 25 Jul 2019 06:26:55 -0700 (PDT) Received: from redhat.com (bzq-79-181-91-42.red.bezeqint.net. [79.181.91.42]) by smtp.gmail.com with ESMTPSA id 47sm29804600qtw.90.2019.07.25.06.26.48 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Thu, 25 Jul 2019 06:26:54 -0700 (PDT) Date: Thu, 25 Jul 2019 09:26:46 -0400 From: "Michael S. Tsirkin" To: Jason Wang Subject: Re: WARNING in __mmdrop Message-ID: <20190725092332-mutt-send-email-mst@kernel.org> References: <20190723032024-mutt-send-email-mst@kernel.org> <1d14de4d-0133-1614-9f64-3ded381de04e@redhat.com> <20190723035725-mutt-send-email-mst@kernel.org> <3f4178f1-0d71-e032-0f1f-802428ceca59@redhat.com> <20190723051828-mutt-send-email-mst@kernel.org> <20190725012149-mutt-send-email-mst@kernel.org> <55e8930c-2695-365f-a07b-3ad169654d28@redhat.com> <20190725042651-mutt-send-email-mst@kernel.org> <84bb2e31-0606-adff-cf2a-e1878225a847@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <84bb2e31-0606-adff-cf2a-e1878225a847@redhat.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190725_062656_869675_7F500837 X-CRM114-Status: GOOD ( 35.57 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mhocko@suse.com, peterz@infradead.org, ldv@altlinux.org, james.bottomley@hansenpartnership.com, linux-mm@kvack.org, namit@vmware.com, mingo@kernel.org, elena.reshetova@intel.com, keescook@chromium.org, aarcange@redhat.com, davem@davemloft.net, hch@infradead.org, christian@brauner.io, syzbot , syzkaller-bugs@googlegroups.com, jglisse@redhat.com, viro@zeniv.linux.org.uk, linux-arm-kernel@lists.infradead.org, wad@chromium.org, linux-parisc@vger.kernel.org, linux-kernel@vger.kernel.org, luto@amacapital.net, ebiederm@xmission.com, akpm@linux-foundation.org, guro@fb.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gVGh1LCBKdWwgMjUsIDIwMTkgYXQgMDk6MjE6MjJQTSArMDgwMCwgSmFzb24gV2FuZyB3cm90 ZToKPiAKPiBPbiAyMDE5LzcvMjUg5LiL5Y2INDoyOCwgTWljaGFlbCBTLiBUc2lya2luIHdyb3Rl Ogo+ID4gT24gVGh1LCBKdWwgMjUsIDIwMTkgYXQgMDM6NDM6NDFQTSArMDgwMCwgSmFzb24gV2Fu ZyB3cm90ZToKPiA+ID4gT24gMjAxOS83LzI1IOS4i+WNiDE6NTIsIE1pY2hhZWwgUy4gVHNpcmtp biB3cm90ZToKPiA+ID4gPiBPbiBUdWUsIEp1bCAyMywgMjAxOSBhdCAwOTozMTozNVBNICswODAw LCBKYXNvbiBXYW5nIHdyb3RlOgo+ID4gPiA+ID4gT24gMjAxOS83LzIzIOS4i+WNiDU6MjYsIE1p Y2hhZWwgUy4gVHNpcmtpbiB3cm90ZToKPiA+ID4gPiA+ID4gT24gVHVlLCBKdWwgMjMsIDIwMTkg YXQgMDQ6NDk6MDFQTSArMDgwMCwgSmFzb24gV2FuZyB3cm90ZToKPiA+ID4gPiA+ID4gPiBPbiAy MDE5LzcvMjMg5LiL5Y2INDoxMCwgTWljaGFlbCBTLiBUc2lya2luIHdyb3RlOgo+ID4gPiA+ID4g PiA+ID4gT24gVHVlLCBKdWwgMjMsIDIwMTkgYXQgMDM6NTM6MDZQTSArMDgwMCwgSmFzb24gV2Fu ZyB3cm90ZToKPiA+ID4gPiA+ID4gPiA+ID4gT24gMjAxOS83LzIzIOS4i+WNiDM6MjMsIE1pY2hh ZWwgUy4gVHNpcmtpbiB3cm90ZToKPiA+ID4gPiA+ID4gPiA+ID4gPiA+ID4gUmVhbGx5IGxldCdz IGp1c3QgdXNlIGtmcmVlX3JjdS4gSXQncyB3YXkgY2xlYW5lcjogZmlyZSBhbmQgZm9yZ2V0Lgo+ ID4gPiA+ID4gPiA+ID4gPiA+ID4gTG9va3Mgbm90LCB5b3UgbmVlZCByYXRlIGxpbWl0IHRoZSBm aXJlIGFzIHlvdSd2ZSBmaWd1cmVkIG91dD8KPiA+ID4gPiA+ID4gPiA+ID4gPiBTZWUgdGhlIGRp c2N1c3Npb24gdGhhdCBmb2xsb3dlZC4gQmFzaWNhbGx5IG5vLCBpdCdzIGdvb2QgZW5vdWdoCj4g PiA+ID4gPiA+ID4gPiA+ID4gYWxyZWFkeSBhbmQgaXMgb25seSBnb2luZyB0byBiZSBiZXR0ZXIu Cj4gPiA+ID4gPiA+ID4gPiA+ID4gCj4gPiA+ID4gPiA+ID4gPiA+ID4gPiBBbmQgaW4gZmFjdCwK PiA+ID4gPiA+ID4gPiA+ID4gPiA+IHRoZSBzeW5jaHJvbml6YXRpb24gaXMgbm90IGV2ZW4gbmVl ZGVkLCBkb2VzIGl0IGhlbHAgaWYgSSBsZWF2ZSBhIGNvbW1lbnQgdG8KPiA+ID4gPiA+ID4gPiA+ ID4gPiA+IGV4cGxhaW4/Cj4gPiA+ID4gPiA+ID4gPiA+ID4gTGV0J3MgdHJ5IHRvIGZpZ3VyZSBp dCBvdXQgaW4gdGhlIG1haWwgZmlyc3QuIEknbSBwcmV0dHkgc3VyZSB0aGUKPiA+ID4gPiA+ID4g PiA+ID4gPiBjdXJyZW50IGxvZ2ljIGlzIHdyb25nLgo+ID4gPiA+ID4gPiA+ID4gPiBIZXJlIGlz IHdoYXQgdGhlIGNvZGUgd2hhdCB0byBhY2hpZXZlOgo+ID4gPiA+ID4gPiA+ID4gPiAKPiA+ID4g PiA+ID4gPiA+ID4gLSBUaGUgbWFwIHdhcyBwcm90ZWN0ZWQgYnkgUkNVCj4gPiA+ID4gPiA+ID4g PiA+IAo+ID4gPiA+ID4gPiA+ID4gPiAtIFdyaXRlcnMgYXJlOiBNTVUgbm90aWZpZXIgaW52YWxp ZGF0aW9uIGNhbGxiYWNrcywgZmlsZSBvcGVyYXRpb25zIChpb2N0bHMKPiA+ID4gPiA+ID4gPiA+ ID4gZXRjKSwgbWV0YV9wcmVmZXRjaCAoZGF0YXBhdGgpCj4gPiA+ID4gPiA+ID4gPiA+IAo+ID4g PiA+ID4gPiA+ID4gPiAtIFJlYWRlcnMgYXJlOiBtZW1vcnkgYWNjZXNzb3IKPiA+ID4gPiA+ID4g PiA+ID4gCj4gPiA+ID4gPiA+ID4gPiA+IFdyaXRlciBhcmUgc3luY2hyb25pemVkIHRocm91Z2gg bW11X2xvY2suIFJDVSBpcyB1c2VkIHRvIHN5bmNocm9uaXplZAo+ID4gPiA+ID4gPiA+ID4gPiBi ZXR3ZWVuIHdyaXRlcnMgYW5kIHJlYWRlcnMuCj4gPiA+ID4gPiA+ID4gPiA+IAo+ID4gPiA+ID4g PiA+ID4gPiBUaGUgc3luY2hyb25pemVfcmN1KCkgaW4gdmhvc3RfcmVzZXRfdnFfbWFwcygpIHdh cyB1c2VkIHRvIHN5bmNocm9uaXplZCBpdAo+ID4gPiA+ID4gPiA+ID4gPiB3aXRoIHJlYWRlcnMg KG1lbW9yeSBhY2Nlc3NvcnMpIGluIHRoZSBwYXRoIG9mIGZpbGUgb3BlcmF0aW9ucy4gQnV0IGlu IHRoaXMKPiA+ID4gPiA+ID4gPiA+ID4gY2FzZSwgdnEtPm11dGV4IHdhcyBhbHJlYWR5IGhlbGQs IHRoaXMgbWVhbnMgaXQgaGFzIGJlZW4gc2VyaWFsaXplZCB3aXRoCj4gPiA+ID4gPiA+ID4gPiA+ IG1lbW9yeSBhY2Nlc3Nvci4gVGhhdCdzIHdoeSBJIHRoaW5rIGl0IGNvdWxkIGJlIHJlbW92ZWQg c2FmZWx5Lgo+ID4gPiA+ID4gPiA+ID4gPiAKPiA+ID4gPiA+ID4gPiA+ID4gQW55dGhpbmcgSSBt aXNzIGhlcmU/Cj4gPiA+ID4gPiA+ID4gPiA+IAo+ID4gPiA+ID4gPiA+ID4gU28gaW52YWxpZGF0 ZSBjYWxsYmFja3MgbmVlZCB0byByZXNldCB0aGUgbWFwLCBhbmQgdGhleSBkbwo+ID4gPiA+ID4g PiA+ID4gbm90IGhhdmUgdnEgbXV0ZXguIEhvdyBjYW4gdGhleSBkbyB0aGlzIGFuZCBmcmVlCj4g PiA+ID4gPiA+ID4gPiB0aGUgbWFwIHNhZmVseT8gVGhleSBuZWVkIHN5bmNocm9uaXplX3JjdSBv ciBrZnJlZV9yY3UgcmlnaHQ/Cj4gPiA+ID4gPiA+ID4gSW52YWxpZGF0aW9uIGNhbGxiYWNrcyBu ZWVkIGJ1dCBmaWxlIG9wZXJhdGlvbnMgKGUuZyBpb2N0bCkgbm90Lgo+ID4gPiA+ID4gPiA+IAo+ ID4gPiA+ID4gPiA+IAo+ID4gPiA+ID4gPiA+ID4gQW5kIEkgd29ycnkgc29tZXdoYXQgdGhhdCBz eW5jaHJvbml6ZV9yY3UgaW4gYW4gTU1VIG5vdGlmaWVyCj4gPiA+ID4gPiA+ID4gPiBpcyBhIHBy b2JsZW0sIE1NVSBub3RpZmllcnMgYXJlIHN1cHBvc2VkIHRvIGJlIHF1aWNrOgo+ID4gPiA+ID4g PiA+IExvb2tzIG5vdCwgc2luY2UgaXQgY2FuIGFsbG93IHRvIGJlIGJsb2NrZWQgYW5kIGxvdHMg b2YgZHJpdmVyIGRlcGVuZHMgb24KPiA+ID4gPiA+ID4gPiB0aGlzLiAoRS5nIG1tdV9ub3RpZmll cl9yYW5nZV9ibG9ja2FibGUoKSkuCj4gPiA+ID4gPiA+IFJpZ2h0LCB0aGV5IGNhbiBibG9jay4g U28gd2h5IGRvbid0IHdlIHRha2UgYSBWUSBtdXRleCBhbmQgYmUKPiA+ID4gPiA+ID4gZG9uZSB3 aXRoIGl0IHRoZW4/IE5vIFJDVSB0cmlja3MuCj4gPiA+ID4gPiBUaGlzIGlzIGhvdyBJIHdhbnQg dG8gZ28gd2l0aCBSRkMgYW5kIFYxLiBCdXQgSSBlbmQgdXAgd2l0aCBkZWFkbG9jayBiZXR3ZWVu Cj4gPiA+ID4gPiB2cSBsb2NrcyBhbmQgc29tZSBNTSBpbnRlcm5hbCBsb2Nrcy4gU28gSSBkZWNp ZGUgdG8gdXNlIFJDVSB3aGljaCBpcyAxMDAlCj4gPiA+ID4gPiB1bmRlciB0aGUgY29udHJvbCBv ZiB2aG9zdC4KPiA+ID4gPiA+IAo+ID4gPiA+ID4gVGhhbmtzCj4gPiA+ID4gQW5kIEkgZ3Vlc3Mg dGhlIGRlYWRsb2NrIGlzIGJlY2F1c2UgR1VQIGlzIHRha2luZyBtbXUgbG9ja3Mgd2hpY2ggYXJl Cj4gPiA+ID4gdGFrZW4gb24gbW11IG5vdGlmaWVyIHBhdGgsIHJpZ2h0Pwo+ID4gPiAKPiA+ID4g WWVzLCBidXQgaXQncyBub3QgdGhlIG9ubHkgbG9jay4gSSBkb24ndCByZW1lbWJlciB0aGUgZGV0 YWlscywgYnV0IEkgY2FuCj4gPiA+IGNvbmZpcm0gSSBtZWV0IGlzc3VlcyB3aXRoIG9uZSBvciB0 d28gb3RoZXIgbG9ja3MuCj4gPiA+IAo+ID4gPiAKPiA+ID4gPiAgICAgSG93IGFib3V0IHdlIGFk ZCBhIHNlcWxvY2sgYW5kIHRha2UKPiA+ID4gPiB0aGF0IGluIGludmFsaWRhdGUgY2FsbGJhY2tz PyAgV2UgY2FuIHRoZW4gZHJvcCB0aGUgVlEgbG9jayBiZWZvcmUgR1VQLAo+ID4gPiA+IGFuZCB0 YWtlIGl0IGFnYWluIGltbWVkaWF0ZWx5IGFmdGVyLgo+ID4gPiA+IAo+ID4gPiA+IHNvbWV0aGlu ZyBsaWtlCj4gPiA+ID4gCWlmICghdnFfbWV0YV9tYXBwZWQodnEpKSB7Cj4gPiA+ID4gCQl2cV9t ZXRhX3NldHVwKCZ1YWRkcnMpOwo+ID4gPiA+IAkJbXV0ZXhfdW5sb2NrKHZxLT5tdXRleCkKPiA+ ID4gPiAJCXZxX21ldGFfbWFwKCZ1YWRkcnMpOwo+ID4gPiAKPiA+ID4gVGhlIHByb2JsZW0gaXMg dGhlIHZxIGFkZHJlc3MgY291bGQgYmUgY2hhbmdlZCBhdCB0aGlzIHRpbWUuCj4gPiA+IAo+ID4g PiAKPiA+ID4gPiAJCW11dGV4X2xvY2sodnEtPm11dGV4KQo+ID4gPiA+IAo+ID4gPiA+IAkJLyog cmVjaGVjayBib3RoIHNvY2stPnByaXZhdGVfZGF0YSBhbmQgc2VxbG9jayBjb3VudC4gKi8KPiA+ ID4gPiAJCWlmIGNoYW5nZWQgLSBiYWlsIG91dAo+ID4gPiA+IAl9Cj4gPiA+ID4gCj4gPiA+ID4g QW5kIGFsc28gcmVxdWlyZXMgdGhhdCBWUSB1YWRkcnMgaXMgZGVmaW5lZCBsaWtlIHRoaXM6Cj4g PiA+ID4gLSB3cml0ZXJzIG11c3QgaGF2ZSBib3RoIHZxIG11dGV4IGFuZCBkZXYgbXV0ZXgKPiA+ ID4gPiAtIHJlYWRlcnMgbXVzdCBoYXZlIGVpdGhlciB2cSBtdXRleCBvciBkZXYgbXV0ZXgKPiA+ ID4gPiAKPiA+ID4gPiAKPiA+ID4gPiBUaGF0J3MgYSBiaWcgY2hhbmdlIHRob3VnaC4gRm9yIG5v dywgaG93IGFib3V0IHN3aXRjaGluZyB0byBhIHBlci12cSBTUkNVPwo+ID4gPiA+IFRoYXQgaXMg b25seSBhIGxpdHRsZSBiaXQgbW9yZSBleHBlbnNpdmUgdGhhbiBSQ1UsIGFuZCB3ZQo+ID4gPiA+ IGNhbiB1c2Ugc3luY2hyb25pemVfc3JjdV9leHBlZGl0ZWQuCj4gPiA+ID4gCj4gPiA+IENvbnNp ZGVyIHdlIHN3aXRjaCB0byB1c2Uga2ZyZWVfcmN1KCksIHdoYXQncyB0aGUgYWR2YW50YWdlIG9m IHBlci12cSBTUkNVPwo+ID4gPiAKPiA+ID4gVGhhbmtzCj4gPiAKPiA+IEkgdGhvdWdodCB3ZSBl c3RhYmxpc2hlZCB0aGF0IG5vdGlmaWVycyBtdXN0IHdhaXQgZm9yCj4gPiBhbGwgcmVhZGVycyB0 byBmaW5pc2ggYmVmb3JlIHRoZXkgbWFyayBwYWdlIGRpcnR5LCB0bwo+ID4gcHJldmVudCBwYWdl IGZyb20gYmVjb21pbmcgZGlydHkgYWZ0ZXIgYWRkcmVzcwo+ID4gaGFzIGJlZW4gaW52YWxpZGF0 ZWQuCj4gPiBSaWdodD8KPiAKPiAKPiBFeGFjdGx5LCBhbmQgdGhhdCdzIHRoZSByZWFzb24gYWN0 dWFsbHkgSSB1c2Ugc3luY2hyb25pemVfcmN1KCkgdGhlcmUuCj4gCj4gU28gdGhlIGNvbmNlcm4g aXMgc3RpbGwgdGhlIHBvc3NpYmxlIHN5bmNocm9uaXplX2V4cGVkaXRlZCgpPwoKCkkgdGhpbmsg c3luY2hyb25pemVfc3JjdV9leHBlZGl0ZWQuCgpzeW5jaHJvbml6ZV9leHBlZGl0ZWQgc2VuZHMg bG90cyBvZiBJUEkgYW5kIGlzIGJhZCBmb3IgcmVhbHRpbWUgVk1zLgoKPiBDYW4gSSBkbyB0aGlz Cj4gb24gdGhyb3VnaCBhbm90aGVyIHNlcmllcyBvbiB0b3Agb2YgdGhlIGluY29taW5nIFYyPwo+ IAo+IFRoYW5rcwo+IAoKVGhlIHF1ZXN0aW9uIGlzIHRoaXM6IGlzIHRoaXMgc3RpbGwgYSBnYWlu IGlmIHdlIHN3aXRjaCB0byB0aGUKbW9yZSBleHBlbnNpdmUgc3JjdT8gSWYgeWVzIHRoZW4gd2Ug Y2FuIGtlZXAgdGhlIGZlYXR1cmUgb24sCmlmIG5vdCB3ZSdsbCBwdXQgaXQgb2ZmIHVudGlsIG5l eHQgcmVsZWFzZSBhbmQgdGhpbmsKb2YgYmV0dGVyIHNvbHV0aW9ucy4gcmN1LT5zcmN1IGlzIGp1 c3QgYSBmaW5kIGFuZCByZXBsYWNlLApkb24ndCBzZWUgd2h5IHdlIG5lZWQgdG8gZGVmZXIgdGhh dC4gY2FuIGJlIGEgc2VwYXJhdGUgcGF0Y2gKZm9yIHN1cmUsIGJ1dCB3ZSBuZWVkIHRvIGtub3cg aG93IHdlbGwgaXQgd29ya3MuCgotLSAKTVNUCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxpbmcgbGlzdApsaW51eC1h cm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcv bWFpbG1hbi9saXN0aW5mby9saW51eC1hcm0ta2VybmVsCg==