From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 077F3C43603 for ; Sat, 14 Dec 2019 00:23:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E6BBB214AF for ; Sat, 14 Dec 2019 00:23:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1576282987; bh=ve6z4xzprm/oNsIdz3QJy0RXU4y2Vma9DSO/NgDPyQ8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:In-Reply-To: References:List-ID:From; b=pljuR2vaVwlhBtd1p9xUSVKDXeLPNEMT4QYNe99kTDMydxFrdsk1RN2v/tfNtqijM gEPIhAsacRgeR3xLQuJX330wou+ZIsupTj/k2XXRis2j1HhtBTgwH99n2NukeSG7yw B1vuXZvH+Twxu5DRX76vu2EbIb9m+s6ja8hkpkIQ= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726939AbfLNAXF (ORCPT ); Fri, 13 Dec 2019 19:23:05 -0500 Received: from mail-pg1-f193.google.com ([209.85.215.193]:38731 "EHLO mail-pg1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726865AbfLNAXC (ORCPT ); Fri, 13 Dec 2019 19:23:02 -0500 Received: by mail-pg1-f193.google.com with SMTP id a33so279823pgm.5 for ; Fri, 13 Dec 2019 16:23:02 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:in-reply-to:references; bh=MqX4YrE1Mit8QmVppoCrUPY+0r38mzKs9y2W2gE8I/k=; b=iWIeUXlB72+aCrqflwfhpror5Vee5fPIqOaqHXEcOTwcDSDo2uM1kBSRrTmpqkeKo4 yKN17eaXBT+8FlYZWD2YAgel6X0WYUfiiFUn6nZiThiU0lk1PY7C/HWQ0OJZbAIu8JhC laf62fVFOuioRVn+/h6tLZGWH5Bqo1PTupjb0PrlFz8Rfpi+R8JbIzoTQXy64Jt8ubOY 0ZMnWkIVlxU49uDSU9CMWOYr5ghFxWOlN8hRr1v9GbjvBE/X2IXWQmrIiGMSLx019DNx 07qMtLGe6ZxPADIrPUvy2ZwGcGxGRRP7k+9Q2+x928+IKodfoCg501TneQ8JBh+9EVG4 sr9A== X-Gm-Message-State: APjAAAVfu40+hF8H7fIzNUVXuAgpVEb8I0imZUnEIIBHmnY4eT9aOVgR 9hrPL/PA777zV3zF8EnWLqg= X-Google-Smtp-Source: APXvYqxTSGNXLxpu3AVS/Qv3GFbcJkqpV9iNlSYWPXMdNN4CA0Dj7qdYF09gF9G4Brw5Ea/qotA9Cw== X-Received: by 2002:a63:6704:: with SMTP id b4mr2656908pgc.424.1576282981954; Fri, 13 Dec 2019 16:23:01 -0800 (PST) Received: from dennisz-mbp.thefacebook.com ([199.201.64.138]) by smtp.gmail.com with ESMTPSA id m12sm11911430pgr.87.2019.12.13.16.23.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 13 Dec 2019 16:23:01 -0800 (PST) From: Dennis Zhou To: David Sterba , Chris Mason , Josef Bacik , Omar Sandoval Cc: kernel-team@fb.com, linux-btrfs@vger.kernel.org, Dennis Zhou Subject: [PATCH 21/22] btrfs: increase the metadata allowance for the free_space_cache Date: Fri, 13 Dec 2019 16:22:30 -0800 Message-Id: <6c434fa53aae2b4bbe8eaf2a9f2c96b0070b6977.1576195673.git.dennis@kernel.org> X-Mailer: git-send-email 2.13.5 In-Reply-To: References: In-Reply-To: References: Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Archived-At: List-Archive: List-Post: Currently, there is no way for the free space cache to recover from being serviced by purely bitmaps because the extent threshold is set to 0 in recalculate_thresholds() when we surpass the metadata allowance. This adds a recovery mechanism by keeping large extents out of the bitmaps and increases the metadata upper bound to 64KB. The recovery mechanism bypasses this upper bound, thus making it a soft upper bound. But, with the bypass being 1MB or greater, it shouldn't add unbounded overhead. Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik --- fs/btrfs/free-space-cache.c | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c index 612726a91e98..6a313946a8da 100644 --- a/fs/btrfs/free-space-cache.c +++ b/fs/btrfs/free-space-cache.c @@ -24,7 +24,8 @@ #include "discard.h" #define BITS_PER_BITMAP (PAGE_SIZE * 8UL) -#define MAX_CACHE_BYTES_PER_GIG SZ_32K +#define MAX_CACHE_BYTES_PER_GIG SZ_64K +#define FORCE_EXTENT_THRESHOLD SZ_1M struct btrfs_trim_range { u64 start; @@ -1695,26 +1696,17 @@ static void recalculate_thresholds(struct btrfs_free_space_ctl *ctl) ASSERT(ctl->total_bitmaps <= max_bitmaps); /* - * The goal is to keep the total amount of memory used per 1gb of space - * at or below 32k, so we need to adjust how much memory we allow to be - * used by extent based free space tracking + * We are trying to keep the total amount of memory used per 1gb of + * space to be MAX_CACHE_BYTES_PER_GIG. However, with a reclamation + * mechanism of pulling extents >= FORCE_EXTENT_THRESHOLD out of + * bitmaps, we may end up using more memory than this. */ if (size < SZ_1G) max_bytes = MAX_CACHE_BYTES_PER_GIG; else max_bytes = MAX_CACHE_BYTES_PER_GIG * div_u64(size, SZ_1G); - /* - * we want to account for 1 more bitmap than what we have so we can make - * sure we don't go over our overall goal of MAX_CACHE_BYTES_PER_GIG as - * we add more bitmaps. - */ - bitmap_bytes = (ctl->total_bitmaps + 1) * ctl->unit; - - if (bitmap_bytes >= max_bytes) { - ctl->extents_thresh = 0; - return; - } + bitmap_bytes = ctl->total_bitmaps * ctl->unit; /* * we want the extent entry threshold to always be at most 1/2 the max @@ -2100,6 +2092,10 @@ static bool use_bitmap(struct btrfs_free_space_ctl *ctl, forced = true; #endif + /* This is a way to reclaim large regions from the bitmaps. */ + if (!forced && info->bytes >= FORCE_EXTENT_THRESHOLD) + return false; + /* * If we are below the extents threshold then we can add this as an * extent, and don't have to deal with the bitmap -- 2.17.1