From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D000C433EF for ; Tue, 8 Feb 2022 08:20:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349297AbiBHIUP (ORCPT ); Tue, 8 Feb 2022 03:20:15 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52986 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349157AbiBHITr (ORCPT ); Tue, 8 Feb 2022 03:19:47 -0500 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 528ADC03FED3 for ; Tue, 8 Feb 2022 00:19:39 -0800 (PST) Received: by mail-yb1-xb49.google.com with SMTP id 3-20020a250103000000b0061d99b7d0b8so13338794ybb.13 for ; Tue, 08 Feb 2022 00:19:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=lRNGv978IZUzuZtn7+GJfB1JhMWHf1r6lSn8z5WI1mY=; b=INp8EKCPNwNrzFQ2lW8+ZqMYhAtdAEXCXLlJVllljVDNsoGTRm2ZuYxrtOcHVsp/hJ qmssG44HpUq64mP3uLn/4AVI1ej7zkoKRZWpxXkOxhxavN0fQ+I9m7cy7g8P97wLPGBw NY0xshp2tLyOPGdrOUuB5HoeJdMea7eAJE8C4wvmp6nNQciiFuE+WjSFzsZM5l3dFI8A 4dQU9Ou2fEJQqJbBAdBnKNy6nOdDXSgR7uVtCvlcRb795L8OEurajARCMGpgCZxjqeQP sEetj61pS6mz0Xca5MGDgAbECJWqr+tKRTa9ZXEmvfYTizKbzD38q8OtNntJi9hD/hoC tL4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=lRNGv978IZUzuZtn7+GJfB1JhMWHf1r6lSn8z5WI1mY=; b=8OBP+sBDusAx18seC31c2MZvsiSz1OOogU8JXP3UzhJ/jD3YUkLZfBVqP9ASuqd46s Jws2o1BHR8rkZ0r5Cu4NNzNfv+sp9mNx3RwO34sIXjx8/7jB7VVVxnAuJCKOtel54bVM toZJH2FnLSlrdOAk3igGP99sJfJtDOfqWOrGUOT8D6xw1FlJrThti6RSfuNru8pYrJe8 jqkpm8GnzTtsK4m6pZMU3/jgDwk4O0Axk6w47+LhK4brmHEiY2MVqE1EaKpGuEj8QBJa mRTyNy4vk0VzkPIZDpdIHrrH0BC85slhqdrgb1/58suiFJ08NUCF9vKf0aIabE9kKvB8 lEZg== X-Gm-Message-State: AOAM5316kBMUfYF3FZvM9jWgGCDGWJ9UzwGCqjmNLMXSQ6EK7+tLBCLG fmp5n/8tiW+WpFXbgZBLusaqRNP0+2s= X-Google-Smtp-Source: ABdhPJxJg59mt99jNZX53wlCHH+5BfJiYUEn9IVqtde0TbSBN1Ev7xHE2KNmWqJImN86wGIDNgk+93Hy6EU= X-Received: from yuzhao.bld.corp.google.com ([2620:15c:183:200:5f31:19c3:21f5:7300]) (user=yuzhao job=sendgmr) by 2002:a81:3346:: with SMTP id z67mr3736711ywz.187.1644308378463; Tue, 08 Feb 2022 00:19:38 -0800 (PST) Date: Tue, 8 Feb 2022 01:18:58 -0700 In-Reply-To: <20220208081902.3550911-1-yuzhao@google.com> Message-Id: <20220208081902.3550911-9-yuzhao@google.com> Mime-Version: 1.0 References: <20220208081902.3550911-1-yuzhao@google.com> X-Mailer: git-send-email 2.35.0.263.gb82422642f-goog Subject: [PATCH v7 08/12] mm: multigenerational LRU: optimize multiple memcgs From: Yu Zhao To: Andrew Morton , Johannes Weiner , Mel Gorman , Michal Hocko Cc: Andi Kleen , Aneesh Kumar , Barry Song <21cnbao@gmail.com>, Catalin Marinas , Dave Hansen , Hillf Danton , Jens Axboe , Jesse Barnes , Jonathan Corbet , Linus Torvalds , Matthew Wilcox , Michael Larabel , Mike Rapoport , Rik van Riel , Vlastimil Babka , Will Deacon , Ying Huang , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, page-reclaim@google.com, x86@kernel.org, Yu Zhao , Brian Geffon , Jan Alexander Steffens , Oleksandr Natalenko , Steven Barrett , Suleiman Souhlal , Daniel Byrne , Donald Carr , "=?UTF-8?q?Holger=20Hoffst=C3=A4tte?=" , Konstantin Kharlamov , Shuang Zhai , Sofia Trinh Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org When multiple memcgs are available, it's possible to improve the overall performance under global memory pressure by making better choices based on generations and tiers. This patch adds a rudimentary optimization to select memcgs that can drop single-use unmapped clean pages first, and thus it reduces the chance of going into the aging path or swapping, which can be costly. Its goal is to improve the overall performance when there are mixed types of workloads, e.g., heavy anon workload in one memcg and heavy buffered I/O workload in the other. Though this optimization can be applied to both kswapd and direct reclaim, it's only added to kswapd to keep the patchset manageable. Later improvements will cover the direct reclaim path. Server benchmark results: Mixed workloads: fio (buffered I/O): -[28, 30]% IOPS BW patch1-7: 3117k 11.9GiB/s patch1-8: 2217k 8661MiB/s memcached (anon): +[247, 251]% Ops/sec KB/sec patch1-7: 563772.35 21900.01 patch1-8: 1968343.76 76461.24 Mixed workloads: fio (buffered I/O): -[4, 6]% IOPS BW 5.17-rc2: 2338k 9133MiB/s patch1-8: 2217k 8661MiB/s memcached (anon): +[524, 530]% Ops/sec KB/sec 5.17-rc2: 313821.65 12190.55 patch1-8: 1968343.76 76461.24 Configurations: (changes since patch 5) cat combined.sh modprobe brd rd_nr=3D2 rd_size=3D56623104 swapoff -a mkswap /dev/ram0 swapon /dev/ram0 mkfs.ext4 /dev/ram1 mount -t ext4 /dev/ram1 /mnt memtier_benchmark -S /var/run/memcached/memcached.sock \ -P memcache_binary -n allkeys --key-minimum=3D1 \ --key-maximum=3D50000000 --key-pattern=3DP:P -c 1 -t 36 \ --ratio 1:0 --pipeline 8 -d 2000 fio -name=3Dmglru --numjobs=3D36 --directory=3D/mnt --size=3D1408m \ --buffered=3D1 --ioengine=3Dio_uring --iodepth=3D128 \ --iodepth_batch_submit=3D32 --iodepth_batch_complete=3D32 \ --rw=3Drandread --random_distribution=3Drandom --norandommap \ --time_based --ramp_time=3D10m --runtime=3D90m --group_reporting & pid=3D$! sleep 200 memtier_benchmark -S /var/run/memcached/memcached.sock \ -P memcache_binary -n allkeys --key-minimum=3D1 \ --key-maximum=3D50000000 --key-pattern=3DR:R -c 1 -t 36 \ --ratio 0:1 --pipeline 8 --randomize --distinct-client-seed kill -INT $pid wait Client benchmark results: no change (CONFIG_MEMCG=3Dn) Signed-off-by: Yu Zhao Acked-by: Brian Geffon Acked-by: Jan Alexander Steffens (heftig) Acked-by: Oleksandr Natalenko Acked-by: Steven Barrett Acked-by: Suleiman Souhlal Tested-by: Daniel Byrne Tested-by: Donald Carr Tested-by: Holger Hoffst=C3=A4tte Tested-by: Konstantin Kharlamov Tested-by: Shuang Zhai Tested-by: Sofia Trinh --- mm/vmscan.c | 45 +++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 41 insertions(+), 4 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 5ab6cd332fcc..fc09b6c10624 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -127,6 +127,13 @@ struct scan_control { /* Always discard instead of demoting to lower tier memory */ unsigned int no_demotion:1; =20 +#ifdef CONFIG_LRU_GEN + /* help make better choices when multiple memcgs are available */ + unsigned int memcgs_need_aging:1; + unsigned int memcgs_need_swapping:1; + unsigned int memcgs_avoid_swapping:1; +#endif + /* Allocation order */ s8 order; =20 @@ -4343,6 +4350,22 @@ static void lru_gen_age_node(struct pglist_data *pgd= at, struct scan_control *sc) =20 VM_BUG_ON(!current_is_kswapd()); =20 + /* + * To reduce the chance of going into the aging path or swapping, which + * can be costly, optimistically skip them unless their corresponding + * flags were cleared in the eviction path. This improves the overall + * performance when multiple memcgs are available. + */ + if (!sc->memcgs_need_aging) { + sc->memcgs_need_aging =3D true; + sc->memcgs_avoid_swapping =3D !sc->memcgs_need_swapping; + sc->memcgs_need_swapping =3D true; + return; + } + + sc->memcgs_need_swapping =3D true; + sc->memcgs_avoid_swapping =3D true; + current->reclaim_state->mm_walk =3D &pgdat->mm_walk; =20 memcg =3D mem_cgroup_iter(NULL, NULL, NULL); @@ -4745,7 +4768,8 @@ static int isolate_folios(struct lruvec *lruvec, stru= ct scan_control *sc, int sw return scanned; } =20 -static int evict_folios(struct lruvec *lruvec, struct scan_control *sc, in= t swappiness) +static int evict_folios(struct lruvec *lruvec, struct scan_control *sc, in= t swappiness, + bool *swapped) { int type; int scanned; @@ -4810,6 +4834,9 @@ static int evict_folios(struct lruvec *lruvec, struct= scan_control *sc, int swap =20 sc->nr_reclaimed +=3D reclaimed; =20 + if (!type && swapped) + *swapped =3D true; + return scanned; } =20 @@ -4838,8 +4865,10 @@ static long get_nr_to_scan(struct lruvec *lruvec, st= ruct scan_control *sc, bool if (!nr_to_scan) return 0; =20 - if (!need_aging) + if (!need_aging) { + sc->memcgs_need_aging =3D false; return nr_to_scan; + } =20 /* leave the work to lru_gen_age_node() */ if (current_is_kswapd()) @@ -4861,6 +4890,8 @@ static void lru_gen_shrink_lruvec(struct lruvec *lruv= ec, struct scan_control *sc { struct blk_plug plug; long scanned =3D 0; + bool swapped =3D false; + unsigned long reclaimed =3D sc->nr_reclaimed; struct mem_cgroup *memcg =3D lruvec_memcg(lruvec); struct pglist_data *pgdat =3D lruvec_pgdat(lruvec); =20 @@ -4887,13 +4918,19 @@ static void lru_gen_shrink_lruvec(struct lruvec *lr= uvec, struct scan_control *sc if (!nr_to_scan) break; =20 - delta =3D evict_folios(lruvec, sc, swappiness); + delta =3D evict_folios(lruvec, sc, swappiness, &swapped); if (!delta) break; =20 + if (sc->memcgs_avoid_swapping && swappiness < 200 && swapped) + break; + scanned +=3D delta; - if (scanned >=3D nr_to_scan) + if (scanned >=3D nr_to_scan) { + if (!swapped && sc->nr_reclaimed - reclaimed >=3D MIN_LRU_BATCH) + sc->memcgs_need_swapping =3D false; break; + } =20 cond_resched(); } --=20 2.35.0.263.gb82422642f-goog From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8336DC433EF for ; Tue, 8 Feb 2022 08:22:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=fMYSsUFWV8U9IGbAziPLMatwP9M35coGTQ+thmXdZSs=; b=mot0StJMygkgCNmViN6rEw9CcE pJBUU+SgfpCfRS9ii1YKBH7cVWQ3tD1/vtKj+Dehx6WmXNCidt2x2Sh8Wm4l8ddOOexs+p7dthvOH HHrvRY8biIPxvoKBmpPfydyCcG5UkjhV1wZU6Jjyw2gAsfZSDoXzaHhP5EyCHotjdke9jQXgBk496 wc0qHGTBgr1Ch5yVPi3PqOeyb9uMHYHbxvjcpfmNiHsjJQKGRIzDv8kQ69Bg1lratjElPHkPj+DCO KwjI8YAcPb70ZDsOneLz7WRDvGs+jYnxZViN/qgSV+trdha9acPl1SJxIoQHjSy/kP+KN19hXzRTT rAj5K9tQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nHLja-00CuFz-6j; Tue, 08 Feb 2022 08:20:22 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nHLiu-00Ctyt-1C for linux-arm-kernel@lists.infradead.org; Tue, 08 Feb 2022 08:19:49 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id b12-20020a056902030c00b0061d720e274aso18396411ybs.20 for ; Tue, 08 Feb 2022 00:19:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=lRNGv978IZUzuZtn7+GJfB1JhMWHf1r6lSn8z5WI1mY=; b=INp8EKCPNwNrzFQ2lW8+ZqMYhAtdAEXCXLlJVllljVDNsoGTRm2ZuYxrtOcHVsp/hJ qmssG44HpUq64mP3uLn/4AVI1ej7zkoKRZWpxXkOxhxavN0fQ+I9m7cy7g8P97wLPGBw NY0xshp2tLyOPGdrOUuB5HoeJdMea7eAJE8C4wvmp6nNQciiFuE+WjSFzsZM5l3dFI8A 4dQU9Ou2fEJQqJbBAdBnKNy6nOdDXSgR7uVtCvlcRb795L8OEurajARCMGpgCZxjqeQP sEetj61pS6mz0Xca5MGDgAbECJWqr+tKRTa9ZXEmvfYTizKbzD38q8OtNntJi9hD/hoC tL4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=lRNGv978IZUzuZtn7+GJfB1JhMWHf1r6lSn8z5WI1mY=; b=WaIkops9LKo23n7viX2EEzgUkeKSSx0fBji6lHMPjTKM2rAOPFVjtx/cUYaEzof/z7 W4h9Y9l5Mk02oGXlaFQhuJfxKbAz9xW4FDiEHs7+y+fIo8n9zfuDoh1XMudApwKA7HKC JdtX+DqYms7wkFsi+BKSlF2yeUyo4jgiDkAOw/MpynGWT1URNTkery7SOSTGMLhCf4Z0 jzwpe/Bex4IFIJsT4+Io7Dx1KbrwE9yK1V7drR5nL+7U3NyfRDDLqJM7SVtnP0k/XtL3 bscV1G6vp10fMIQMq1QaDmipRRJvK7t6kN/Q0GlDDVNe08K86vLM7wFXVIIQykxR75Gi zbrg== X-Gm-Message-State: AOAM53224UwBqXNpbIuD5grmIZLKZlt2ZbNpLxFAAClUb0seAyRlpwwA Bst5DzmEdL3Y2zU2DFR3hQvpw3COFPU= X-Google-Smtp-Source: ABdhPJxJg59mt99jNZX53wlCHH+5BfJiYUEn9IVqtde0TbSBN1Ev7xHE2KNmWqJImN86wGIDNgk+93Hy6EU= X-Received: from yuzhao.bld.corp.google.com ([2620:15c:183:200:5f31:19c3:21f5:7300]) (user=yuzhao job=sendgmr) by 2002:a81:3346:: with SMTP id z67mr3736711ywz.187.1644308378463; Tue, 08 Feb 2022 00:19:38 -0800 (PST) Date: Tue, 8 Feb 2022 01:18:58 -0700 In-Reply-To: <20220208081902.3550911-1-yuzhao@google.com> Message-Id: <20220208081902.3550911-9-yuzhao@google.com> Mime-Version: 1.0 References: <20220208081902.3550911-1-yuzhao@google.com> X-Mailer: git-send-email 2.35.0.263.gb82422642f-goog Subject: [PATCH v7 08/12] mm: multigenerational LRU: optimize multiple memcgs From: Yu Zhao To: Andrew Morton , Johannes Weiner , Mel Gorman , Michal Hocko Cc: Andi Kleen , Aneesh Kumar , Barry Song <21cnbao@gmail.com>, Catalin Marinas , Dave Hansen , Hillf Danton , Jens Axboe , Jesse Barnes , Jonathan Corbet , Linus Torvalds , Matthew Wilcox , Michael Larabel , Mike Rapoport , Rik van Riel , Vlastimil Babka , Will Deacon , Ying Huang , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, page-reclaim@google.com, x86@kernel.org, Yu Zhao , Brian Geffon , Jan Alexander Steffens , Oleksandr Natalenko , Steven Barrett , Suleiman Souhlal , Daniel Byrne , Donald Carr , "=?UTF-8?q?Holger=20Hoffst=C3=A4tte?=" , Konstantin Kharlamov , Shuang Zhai , Sofia Trinh X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220208_001940_120246_B2E1CBEA X-CRM114-Status: GOOD ( 19.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org V2hlbiBtdWx0aXBsZSBtZW1jZ3MgYXJlIGF2YWlsYWJsZSwgaXQncyBwb3NzaWJsZSB0byBpbXBy b3ZlIHRoZQpvdmVyYWxsIHBlcmZvcm1hbmNlIHVuZGVyIGdsb2JhbCBtZW1vcnkgcHJlc3N1cmUg YnkgbWFraW5nIGJldHRlcgpjaG9pY2VzIGJhc2VkIG9uIGdlbmVyYXRpb25zIGFuZCB0aWVycy4g VGhpcyBwYXRjaCBhZGRzIGEgcnVkaW1lbnRhcnkKb3B0aW1pemF0aW9uIHRvIHNlbGVjdCBtZW1j Z3MgdGhhdCBjYW4gZHJvcCBzaW5nbGUtdXNlIHVubWFwcGVkIGNsZWFuCnBhZ2VzIGZpcnN0LCBh bmQgdGh1cyBpdCByZWR1Y2VzIHRoZSBjaGFuY2Ugb2YgZ29pbmcgaW50byB0aGUgYWdpbmcKcGF0 aCBvciBzd2FwcGluZywgd2hpY2ggY2FuIGJlIGNvc3RseS4gSXRzIGdvYWwgaXMgdG8gaW1wcm92 ZSB0aGUKb3ZlcmFsbCBwZXJmb3JtYW5jZSB3aGVuIHRoZXJlIGFyZSBtaXhlZCB0eXBlcyBvZiB3 b3JrbG9hZHMsIGUuZy4sCmhlYXZ5IGFub24gd29ya2xvYWQgaW4gb25lIG1lbWNnIGFuZCBoZWF2 eSBidWZmZXJlZCBJL08gd29ya2xvYWQgaW4KdGhlIG90aGVyLgoKVGhvdWdoIHRoaXMgb3B0aW1p emF0aW9uIGNhbiBiZSBhcHBsaWVkIHRvIGJvdGgga3N3YXBkIGFuZCBkaXJlY3QKcmVjbGFpbSwg aXQncyBvbmx5IGFkZGVkIHRvIGtzd2FwZCB0byBrZWVwIHRoZSBwYXRjaHNldCBtYW5hZ2VhYmxl LgpMYXRlciBpbXByb3ZlbWVudHMgd2lsbCBjb3ZlciB0aGUgZGlyZWN0IHJlY2xhaW0gcGF0aC4K ClNlcnZlciBiZW5jaG1hcmsgcmVzdWx0czoKICBNaXhlZCB3b3JrbG9hZHM6CiAgICBmaW8gKGJ1 ZmZlcmVkIEkvTyk6IC1bMjgsIDMwXSUKICAgICAgICAgICAgICAgIElPUFMgICAgICAgICBCVwog ICAgICBwYXRjaDEtNzogMzExN2sgICAgICAgIDExLjlHaUIvcwogICAgICBwYXRjaDEtODogMjIx N2sgICAgICAgIDg2NjFNaUIvcwoKICAgIG1lbWNhY2hlZCAoYW5vbik6ICtbMjQ3LCAyNTFdJQog ICAgICAgICAgICAgICAgT3BzL3NlYyAgICAgIEtCL3NlYwogICAgICBwYXRjaDEtNzogNTYzNzcy LjM1ICAgIDIxOTAwLjAxCiAgICAgIHBhdGNoMS04OiAxOTY4MzQzLjc2ICAgNzY0NjEuMjQKCiAg TWl4ZWQgd29ya2xvYWRzOgogICAgZmlvIChidWZmZXJlZCBJL08pOiAtWzQsIDZdJQogICAgICAg ICAgICAgICAgSU9QUyAgICAgICAgIEJXCiAgICAgIDUuMTctcmMyOiAyMzM4ayAgICAgICAgOTEz M01pQi9zCiAgICAgIHBhdGNoMS04OiAyMjE3ayAgICAgICAgODY2MU1pQi9zCgogICAgbWVtY2Fj aGVkIChhbm9uKTogK1s1MjQsIDUzMF0lCiAgICAgICAgICAgICAgICBPcHMvc2VjICAgICAgS0Iv c2VjCiAgICAgIDUuMTctcmMyOiAzMTM4MjEuNjUgICAgMTIxOTAuNTUKICAgICAgcGF0Y2gxLTg6 IDE5NjgzNDMuNzYgICA3NjQ2MS4yNAoKICBDb25maWd1cmF0aW9uczoKICAgIChjaGFuZ2VzIHNp bmNlIHBhdGNoIDUpCgogICAgY2F0IGNvbWJpbmVkLnNoCiAgICBtb2Rwcm9iZSBicmQgcmRfbnI9 MiByZF9zaXplPTU2NjIzMTA0CgogICAgc3dhcG9mZiAtYQogICAgbWtzd2FwIC9kZXYvcmFtMAog ICAgc3dhcG9uIC9kZXYvcmFtMAoKICAgIG1rZnMuZXh0NCAvZGV2L3JhbTEKICAgIG1vdW50IC10 IGV4dDQgL2Rldi9yYW0xIC9tbnQKCiAgICBtZW10aWVyX2JlbmNobWFyayAtUyAvdmFyL3J1bi9t ZW1jYWNoZWQvbWVtY2FjaGVkLnNvY2sgXAogICAgICAtUCBtZW1jYWNoZV9iaW5hcnkgLW4gYWxs a2V5cyAtLWtleS1taW5pbXVtPTEgXAogICAgICAtLWtleS1tYXhpbXVtPTUwMDAwMDAwIC0ta2V5 LXBhdHRlcm49UDpQIC1jIDEgLXQgMzYgXAogICAgICAtLXJhdGlvIDE6MCAtLXBpcGVsaW5lIDgg LWQgMjAwMAoKICAgIGZpbyAtbmFtZT1tZ2xydSAtLW51bWpvYnM9MzYgLS1kaXJlY3Rvcnk9L21u dCAtLXNpemU9MTQwOG0gXAogICAgICAtLWJ1ZmZlcmVkPTEgLS1pb2VuZ2luZT1pb191cmluZyAt LWlvZGVwdGg9MTI4IFwKICAgICAgLS1pb2RlcHRoX2JhdGNoX3N1Ym1pdD0zMiAtLWlvZGVwdGhf YmF0Y2hfY29tcGxldGU9MzIgXAogICAgICAtLXJ3PXJhbmRyZWFkIC0tcmFuZG9tX2Rpc3RyaWJ1 dGlvbj1yYW5kb20gLS1ub3JhbmRvbW1hcCBcCiAgICAgIC0tdGltZV9iYXNlZCAtLXJhbXBfdGlt ZT0xMG0gLS1ydW50aW1lPTkwbSAtLWdyb3VwX3JlcG9ydGluZyAmCiAgICBwaWQ9JCEKCiAgICBz bGVlcCAyMDAKCiAgICBtZW10aWVyX2JlbmNobWFyayAtUyAvdmFyL3J1bi9tZW1jYWNoZWQvbWVt Y2FjaGVkLnNvY2sgXAogICAgICAtUCBtZW1jYWNoZV9iaW5hcnkgLW4gYWxsa2V5cyAtLWtleS1t aW5pbXVtPTEgXAogICAgICAtLWtleS1tYXhpbXVtPTUwMDAwMDAwIC0ta2V5LXBhdHRlcm49UjpS IC1jIDEgLXQgMzYgXAogICAgICAtLXJhdGlvIDA6MSAtLXBpcGVsaW5lIDggLS1yYW5kb21pemUg LS1kaXN0aW5jdC1jbGllbnQtc2VlZAoKICAgIGtpbGwgLUlOVCAkcGlkCiAgICB3YWl0CgpDbGll bnQgYmVuY2htYXJrIHJlc3VsdHM6CiAgbm8gY2hhbmdlIChDT05GSUdfTUVNQ0c9bikKClNpZ25l ZC1vZmYtYnk6IFl1IFpoYW8gPHl1emhhb0Bnb29nbGUuY29tPgpBY2tlZC1ieTogQnJpYW4gR2Vm Zm9uIDxiZ2VmZm9uQGdvb2dsZS5jb20+CkFja2VkLWJ5OiBKYW4gQWxleGFuZGVyIFN0ZWZmZW5z IChoZWZ0aWcpIDxoZWZ0aWdAYXJjaGxpbnV4Lm9yZz4KQWNrZWQtYnk6IE9sZWtzYW5kciBOYXRh bGVua28gPG9sZWtzYW5kckBuYXRhbGVua28ubmFtZT4KQWNrZWQtYnk6IFN0ZXZlbiBCYXJyZXR0 IDxzdGV2ZW5AbGlxdW9yaXgubmV0PgpBY2tlZC1ieTogU3VsZWltYW4gU291aGxhbCA8c3VsZWlt YW5AZ29vZ2xlLmNvbT4KVGVzdGVkLWJ5OiBEYW5pZWwgQnlybmUgPGRqYnlybmVAbXR1LmVkdT4K VGVzdGVkLWJ5OiBEb25hbGQgQ2FyciA8ZEBjaGFvcy1yZWlucy5jb20+ClRlc3RlZC1ieTogSG9s Z2VyIEhvZmZzdMOkdHRlIDxob2xnZXJAYXBwbGllZC1hc3luY2hyb255LmNvbT4KVGVzdGVkLWJ5 OiBLb25zdGFudGluIEtoYXJsYW1vdiA8SGktQW5nZWxAeWFuZGV4LnJ1PgpUZXN0ZWQtYnk6IFNo dWFuZyBaaGFpIDxzemhhaTJAY3Mucm9jaGVzdGVyLmVkdT4KVGVzdGVkLWJ5OiBTb2ZpYSBUcmlu aCA8c29maWEudHJpbmhAZWRpLndvcmtzPgotLS0KIG1tL3Ztc2Nhbi5jIHwgNDUgKysrKysrKysr KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKystLS0tCiAxIGZpbGUgY2hhbmdlZCwgNDEg aW5zZXJ0aW9ucygrKSwgNCBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQgYS9tbS92bXNjYW4uYyBi L21tL3Ztc2Nhbi5jCmluZGV4IDVhYjZjZDMzMmZjYy4uZmMwOWI2YzEwNjI0IDEwMDY0NAotLS0g YS9tbS92bXNjYW4uYworKysgYi9tbS92bXNjYW4uYwpAQCAtMTI3LDYgKzEyNywxMyBAQCBzdHJ1 Y3Qgc2Nhbl9jb250cm9sIHsKIAkvKiBBbHdheXMgZGlzY2FyZCBpbnN0ZWFkIG9mIGRlbW90aW5n IHRvIGxvd2VyIHRpZXIgbWVtb3J5ICovCiAJdW5zaWduZWQgaW50IG5vX2RlbW90aW9uOjE7CiAK KyNpZmRlZiBDT05GSUdfTFJVX0dFTgorCS8qIGhlbHAgbWFrZSBiZXR0ZXIgY2hvaWNlcyB3aGVu IG11bHRpcGxlIG1lbWNncyBhcmUgYXZhaWxhYmxlICovCisJdW5zaWduZWQgaW50IG1lbWNnc19u ZWVkX2FnaW5nOjE7CisJdW5zaWduZWQgaW50IG1lbWNnc19uZWVkX3N3YXBwaW5nOjE7CisJdW5z aWduZWQgaW50IG1lbWNnc19hdm9pZF9zd2FwcGluZzoxOworI2VuZGlmCisKIAkvKiBBbGxvY2F0 aW9uIG9yZGVyICovCiAJczggb3JkZXI7CiAKQEAgLTQzNDMsNiArNDM1MCwyMiBAQCBzdGF0aWMg dm9pZCBscnVfZ2VuX2FnZV9ub2RlKHN0cnVjdCBwZ2xpc3RfZGF0YSAqcGdkYXQsIHN0cnVjdCBz Y2FuX2NvbnRyb2wgKnNjKQogCiAJVk1fQlVHX09OKCFjdXJyZW50X2lzX2tzd2FwZCgpKTsKIAor CS8qCisJICogVG8gcmVkdWNlIHRoZSBjaGFuY2Ugb2YgZ29pbmcgaW50byB0aGUgYWdpbmcgcGF0 aCBvciBzd2FwcGluZywgd2hpY2gKKwkgKiBjYW4gYmUgY29zdGx5LCBvcHRpbWlzdGljYWxseSBz a2lwIHRoZW0gdW5sZXNzIHRoZWlyIGNvcnJlc3BvbmRpbmcKKwkgKiBmbGFncyB3ZXJlIGNsZWFy ZWQgaW4gdGhlIGV2aWN0aW9uIHBhdGguIFRoaXMgaW1wcm92ZXMgdGhlIG92ZXJhbGwKKwkgKiBw ZXJmb3JtYW5jZSB3aGVuIG11bHRpcGxlIG1lbWNncyBhcmUgYXZhaWxhYmxlLgorCSAqLworCWlm ICghc2MtPm1lbWNnc19uZWVkX2FnaW5nKSB7CisJCXNjLT5tZW1jZ3NfbmVlZF9hZ2luZyA9IHRy dWU7CisJCXNjLT5tZW1jZ3NfYXZvaWRfc3dhcHBpbmcgPSAhc2MtPm1lbWNnc19uZWVkX3N3YXBw aW5nOworCQlzYy0+bWVtY2dzX25lZWRfc3dhcHBpbmcgPSB0cnVlOworCQlyZXR1cm47CisJfQor CisJc2MtPm1lbWNnc19uZWVkX3N3YXBwaW5nID0gdHJ1ZTsKKwlzYy0+bWVtY2dzX2F2b2lkX3N3 YXBwaW5nID0gdHJ1ZTsKKwogCWN1cnJlbnQtPnJlY2xhaW1fc3RhdGUtPm1tX3dhbGsgPSAmcGdk YXQtPm1tX3dhbGs7CiAKIAltZW1jZyA9IG1lbV9jZ3JvdXBfaXRlcihOVUxMLCBOVUxMLCBOVUxM KTsKQEAgLTQ3NDUsNyArNDc2OCw4IEBAIHN0YXRpYyBpbnQgaXNvbGF0ZV9mb2xpb3Moc3RydWN0 IGxydXZlYyAqbHJ1dmVjLCBzdHJ1Y3Qgc2Nhbl9jb250cm9sICpzYywgaW50IHN3CiAJcmV0dXJu IHNjYW5uZWQ7CiB9CiAKLXN0YXRpYyBpbnQgZXZpY3RfZm9saW9zKHN0cnVjdCBscnV2ZWMgKmxy dXZlYywgc3RydWN0IHNjYW5fY29udHJvbCAqc2MsIGludCBzd2FwcGluZXNzKQorc3RhdGljIGlu dCBldmljdF9mb2xpb3Moc3RydWN0IGxydXZlYyAqbHJ1dmVjLCBzdHJ1Y3Qgc2Nhbl9jb250cm9s ICpzYywgaW50IHN3YXBwaW5lc3MsCisJCQlib29sICpzd2FwcGVkKQogewogCWludCB0eXBlOwog CWludCBzY2FubmVkOwpAQCAtNDgxMCw2ICs0ODM0LDkgQEAgc3RhdGljIGludCBldmljdF9mb2xp b3Moc3RydWN0IGxydXZlYyAqbHJ1dmVjLCBzdHJ1Y3Qgc2Nhbl9jb250cm9sICpzYywgaW50IHN3 YXAKIAogCXNjLT5ucl9yZWNsYWltZWQgKz0gcmVjbGFpbWVkOwogCisJaWYgKCF0eXBlICYmIHN3 YXBwZWQpCisJCSpzd2FwcGVkID0gdHJ1ZTsKKwogCXJldHVybiBzY2FubmVkOwogfQogCkBAIC00 ODM4LDggKzQ4NjUsMTAgQEAgc3RhdGljIGxvbmcgZ2V0X25yX3RvX3NjYW4oc3RydWN0IGxydXZl YyAqbHJ1dmVjLCBzdHJ1Y3Qgc2Nhbl9jb250cm9sICpzYywgYm9vbAogCWlmICghbnJfdG9fc2Nh bikKIAkJcmV0dXJuIDA7CiAKLQlpZiAoIW5lZWRfYWdpbmcpCisJaWYgKCFuZWVkX2FnaW5nKSB7 CisJCXNjLT5tZW1jZ3NfbmVlZF9hZ2luZyA9IGZhbHNlOwogCQlyZXR1cm4gbnJfdG9fc2NhbjsK Kwl9CiAKIAkvKiBsZWF2ZSB0aGUgd29yayB0byBscnVfZ2VuX2FnZV9ub2RlKCkgKi8KIAlpZiAo Y3VycmVudF9pc19rc3dhcGQoKSkKQEAgLTQ4NjEsNiArNDg5MCw4IEBAIHN0YXRpYyB2b2lkIGxy dV9nZW5fc2hyaW5rX2xydXZlYyhzdHJ1Y3QgbHJ1dmVjICpscnV2ZWMsIHN0cnVjdCBzY2FuX2Nv bnRyb2wgKnNjCiB7CiAJc3RydWN0IGJsa19wbHVnIHBsdWc7CiAJbG9uZyBzY2FubmVkID0gMDsK Kwlib29sIHN3YXBwZWQgPSBmYWxzZTsKKwl1bnNpZ25lZCBsb25nIHJlY2xhaW1lZCA9IHNjLT5u cl9yZWNsYWltZWQ7CiAJc3RydWN0IG1lbV9jZ3JvdXAgKm1lbWNnID0gbHJ1dmVjX21lbWNnKGxy dXZlYyk7CiAJc3RydWN0IHBnbGlzdF9kYXRhICpwZ2RhdCA9IGxydXZlY19wZ2RhdChscnV2ZWMp OwogCkBAIC00ODg3LDEzICs0OTE4LDE5IEBAIHN0YXRpYyB2b2lkIGxydV9nZW5fc2hyaW5rX2xy dXZlYyhzdHJ1Y3QgbHJ1dmVjICpscnV2ZWMsIHN0cnVjdCBzY2FuX2NvbnRyb2wgKnNjCiAJCWlm ICghbnJfdG9fc2NhbikKIAkJCWJyZWFrOwogCi0JCWRlbHRhID0gZXZpY3RfZm9saW9zKGxydXZl Yywgc2MsIHN3YXBwaW5lc3MpOworCQlkZWx0YSA9IGV2aWN0X2ZvbGlvcyhscnV2ZWMsIHNjLCBz d2FwcGluZXNzLCAmc3dhcHBlZCk7CiAJCWlmICghZGVsdGEpCiAJCQlicmVhazsKIAorCQlpZiAo c2MtPm1lbWNnc19hdm9pZF9zd2FwcGluZyAmJiBzd2FwcGluZXNzIDwgMjAwICYmIHN3YXBwZWQp CisJCQlicmVhazsKKwogCQlzY2FubmVkICs9IGRlbHRhOwotCQlpZiAoc2Nhbm5lZCA+PSBucl90 b19zY2FuKQorCQlpZiAoc2Nhbm5lZCA+PSBucl90b19zY2FuKSB7CisJCQlpZiAoIXN3YXBwZWQg JiYgc2MtPm5yX3JlY2xhaW1lZCAtIHJlY2xhaW1lZCA+PSBNSU5fTFJVX0JBVENIKQorCQkJCXNj LT5tZW1jZ3NfbmVlZF9zd2FwcGluZyA9IGZhbHNlOwogCQkJYnJlYWs7CisJCX0KIAogCQljb25k X3Jlc2NoZWQoKTsKIAl9Ci0tIAoyLjM1LjAuMjYzLmdiODI0MjI2NDJmLWdvb2cKCgpfX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpsaW51eC1hcm0ta2VybmVs IG1haWxpbmcgbGlzdApsaW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDov L2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1hcm0ta2VybmVsCg==