From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E384CC433EF for ; Wed, 4 May 2022 00:39:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244729AbiEDAnJ (ORCPT ); Tue, 3 May 2022 20:43:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236921AbiEDAnI (ORCPT ); Tue, 3 May 2022 20:43:08 -0400 Received: from ams.source.kernel.org (ams.source.kernel.org [IPv6:2604:1380:4601:e00::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1510F38BC6; Tue, 3 May 2022 17:39:34 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id C2E02B8227E; Wed, 4 May 2022 00:39:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8EC3EC385A4; Wed, 4 May 2022 00:39:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1651624771; bh=xlKxpLDsGDGqIEBxZuWucqMujX6BG0fM6Gh35+FdMGY=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=b9JoHDXlj3YVviGODLUCXCiwap0JGZceDZFB8LsU8AHsYwzyjBJ4Xo6aoKC84aqPk xaQ+ZoXuRr+DrZV/j+/BD6Rtnlg3tXACE5zvfF1xmaRGhDnl5zX5K9qpLDj5Qe0LGX OpCIEJ5wgQmw71F/3NlylbGuRq1L50tx8R1aCOUk6kzEXsOpD9aMPG4Vs93tOSqo6R Spz7Tz9DqEi8k6sErFcR/PKNTPVhmtMO0qPtdyny7r3aO5iYlIQGSXrA/skflGN9xC j0JxoZ2S5J325sCuBXMXzHbFPyNMUdeYR1KLaW8mR/LkHrDACOgRAqmaFzF2YDkrpI wj6ha0aV+rwLg== Message-ID: <6c0e4596-1698-25d5-7b59-c0ee6ef7eefd@kernel.org> Date: Wed, 4 May 2022 08:39:12 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: Re: [PATCH] f2fs: fix deadloop in foreground GC Content-Language: en-US To: Jaegeuk Kim Cc: linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org, stable@vger.kernel.org, Ming Yan , Chao Yu References: <20220429204631.7241-1-chao@kernel.org> From: Chao Yu In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2022/5/4 6:25, Jaegeuk Kim wrote: > On 04/30, Chao Yu wrote: >> As Yanming reported in bugzilla: >> >> https://bugzilla.kernel.org/show_bug.cgi?id=215914 >> >> The root cause is: in a very small sized image, it's very easy to >> exceed threshold of foreground GC, if we calculate free space and >> dirty data based on section granularity, in corner case, >> has_not_enough_free_secs() will always return true, result in >> deadloop in f2fs_gc(). > > Performance regression was reported. Can we check this for very small sized > image only? I noticed that, I've fixed the issue in v2, could you please take a look? Thanks, > >> >> So this patch refactors has_not_enough_free_secs() as below to fix >> this issue: >> 1. calculate needed space based on block granularity, and separate >> all blocks to two parts, section part, and block part, comparing >> section part to free section, and comparing block part to free space >> in openned log. >> 2. account F2FS_DIRTY_NODES, F2FS_DIRTY_IMETA and F2FS_DIRTY_DENTS >> as node block consumer; >> 3. account F2FS_DIRTY_DENTS as data block consumer; >> >> Cc: stable@vger.kernel.org >> Reported-by: Ming Yan >> Signed-off-by: Chao Yu >> --- >> fs/f2fs/segment.h | 30 +++++++++++++++++------------- >> 1 file changed, 17 insertions(+), 13 deletions(-) >> >> diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h >> index 8a591455d796..28f7aa9b40bf 100644 >> --- a/fs/f2fs/segment.h >> +++ b/fs/f2fs/segment.h >> @@ -575,11 +575,10 @@ static inline int reserved_sections(struct f2fs_sb_info *sbi) >> return GET_SEC_FROM_SEG(sbi, reserved_segments(sbi)); >> } >> >> -static inline bool has_curseg_enough_space(struct f2fs_sb_info *sbi) >> +static inline bool has_curseg_enough_space(struct f2fs_sb_info *sbi, >> + unsigned int node_blocks, unsigned int dent_blocks) >> { >> - unsigned int node_blocks = get_pages(sbi, F2FS_DIRTY_NODES) + >> - get_pages(sbi, F2FS_DIRTY_DENTS); >> - unsigned int dent_blocks = get_pages(sbi, F2FS_DIRTY_DENTS); >> + >> unsigned int segno, left_blocks; >> int i; >> >> @@ -605,19 +604,24 @@ static inline bool has_curseg_enough_space(struct f2fs_sb_info *sbi) >> static inline bool has_not_enough_free_secs(struct f2fs_sb_info *sbi, >> int freed, int needed) >> { >> - int node_secs = get_blocktype_secs(sbi, F2FS_DIRTY_NODES); >> - int dent_secs = get_blocktype_secs(sbi, F2FS_DIRTY_DENTS); >> - int imeta_secs = get_blocktype_secs(sbi, F2FS_DIRTY_IMETA); >> + unsigned int total_node_blocks = get_pages(sbi, F2FS_DIRTY_NODES) + >> + get_pages(sbi, F2FS_DIRTY_DENTS) + >> + get_pages(sbi, F2FS_DIRTY_IMETA); >> + unsigned int total_dent_blocks = get_pages(sbi, F2FS_DIRTY_DENTS); >> + unsigned int node_secs = total_node_blocks / BLKS_PER_SEC(sbi); >> + unsigned int dent_secs = total_dent_blocks / BLKS_PER_SEC(sbi); >> + unsigned int node_blocks = total_node_blocks % BLKS_PER_SEC(sbi); >> + unsigned int dent_blocks = total_dent_blocks % BLKS_PER_SEC(sbi); >> >> if (unlikely(is_sbi_flag_set(sbi, SBI_POR_DOING))) >> return false; >> >> - if (free_sections(sbi) + freed == reserved_sections(sbi) + needed && >> - has_curseg_enough_space(sbi)) >> - return false; >> - return (free_sections(sbi) + freed) <= >> - (node_secs + 2 * dent_secs + imeta_secs + >> - reserved_sections(sbi) + needed); >> + if (free_sections(sbi) + freed <= >> + node_secs + dent_secs + reserved_sections(sbi) + needed) >> + return true; >> + if (!has_curseg_enough_space(sbi, node_blocks, dent_blocks)) >> + return true; >> + return false; >> } >> >> static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi) >> -- >> 2.32.0 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.sourceforge.net (lists.sourceforge.net [216.105.38.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7312FC433EF for ; Wed, 4 May 2022 00:39:57 +0000 (UTC) Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.94.2) (envelope-from ) id 1nm33a-0003Zn-Pj; Wed, 04 May 2022 00:39:53 +0000 Received: from [172.30.20.202] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1nm33Z-0003Zd-AG for linux-f2fs-devel@lists.sourceforge.net; Wed, 04 May 2022 00:39:52 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:Content-Type:In-Reply-To: From:References:Cc:To:Subject:MIME-Version:Date:Message-ID:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=Z9IqWh5DpQgv2Qs7N2b906ZeMmf/9mM3Kqzxbti2ikQ=; b=BrHIWMdYZ1kf0NEFu9xHtJsvOQ CMdmzhhgtyvfI2RtcbTQm67B1JkduNcl4+kAibSORpwyHBLUnNbwYFv2TMYK0cUeol2uFngheucbf ER/ETOmmajO4VDECZxFe3jnL+gsQqjpNAi1MEep4YF2H4rKLdQQdi8F9+i1MasCa3I+I=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:References:Cc:To: Subject:MIME-Version:Date:Message-ID:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=Z9IqWh5DpQgv2Qs7N2b906ZeMmf/9mM3Kqzxbti2ikQ=; b=MRSFvJIC6p//YP0CkdIvT4CorZ wmmPSKxke3lZq2BZTelcQVTBcVkIZMDUyf0+iRlpcV7V7QSGF//R/30jZ8Mzyeqpvk10XjMmJ6IDS ImoYDnmx6J4P717aPcoy2Xw0p1fbOlMfI1JVfBvkXUi/tGvCuHLZ/Z/RseMesCHk7avs=; Received: from ams.source.kernel.org ([145.40.68.75]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.94.2) id 1nm33S-0006l5-W6 for linux-f2fs-devel@lists.sourceforge.net; Wed, 04 May 2022 00:39:52 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id A4D87B80DB7; Wed, 4 May 2022 00:39:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8EC3EC385A4; Wed, 4 May 2022 00:39:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1651624771; bh=xlKxpLDsGDGqIEBxZuWucqMujX6BG0fM6Gh35+FdMGY=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=b9JoHDXlj3YVviGODLUCXCiwap0JGZceDZFB8LsU8AHsYwzyjBJ4Xo6aoKC84aqPk xaQ+ZoXuRr+DrZV/j+/BD6Rtnlg3tXACE5zvfF1xmaRGhDnl5zX5K9qpLDj5Qe0LGX OpCIEJ5wgQmw71F/3NlylbGuRq1L50tx8R1aCOUk6kzEXsOpD9aMPG4Vs93tOSqo6R Spz7Tz9DqEi8k6sErFcR/PKNTPVhmtMO0qPtdyny7r3aO5iYlIQGSXrA/skflGN9xC j0JxoZ2S5J325sCuBXMXzHbFPyNMUdeYR1KLaW8mR/LkHrDACOgRAqmaFzF2YDkrpI wj6ha0aV+rwLg== Message-ID: <6c0e4596-1698-25d5-7b59-c0ee6ef7eefd@kernel.org> Date: Wed, 4 May 2022 08:39:12 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Content-Language: en-US To: Jaegeuk Kim References: <20220429204631.7241-1-chao@kernel.org> From: Chao Yu In-Reply-To: X-Headers-End: 1nm33S-0006l5-W6 Subject: Re: [f2fs-dev] [PATCH] f2fs: fix deadloop in foreground GC X-BeenThere: linux-f2fs-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ming Yan , linux-kernel@vger.kernel.org, stable@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: linux-f2fs-devel-bounces@lists.sourceforge.net On 2022/5/4 6:25, Jaegeuk Kim wrote: > On 04/30, Chao Yu wrote: >> As Yanming reported in bugzilla: >> >> https://bugzilla.kernel.org/show_bug.cgi?id=215914 >> >> The root cause is: in a very small sized image, it's very easy to >> exceed threshold of foreground GC, if we calculate free space and >> dirty data based on section granularity, in corner case, >> has_not_enough_free_secs() will always return true, result in >> deadloop in f2fs_gc(). > > Performance regression was reported. Can we check this for very small sized > image only? I noticed that, I've fixed the issue in v2, could you please take a look? Thanks, > >> >> So this patch refactors has_not_enough_free_secs() as below to fix >> this issue: >> 1. calculate needed space based on block granularity, and separate >> all blocks to two parts, section part, and block part, comparing >> section part to free section, and comparing block part to free space >> in openned log. >> 2. account F2FS_DIRTY_NODES, F2FS_DIRTY_IMETA and F2FS_DIRTY_DENTS >> as node block consumer; >> 3. account F2FS_DIRTY_DENTS as data block consumer; >> >> Cc: stable@vger.kernel.org >> Reported-by: Ming Yan >> Signed-off-by: Chao Yu >> --- >> fs/f2fs/segment.h | 30 +++++++++++++++++------------- >> 1 file changed, 17 insertions(+), 13 deletions(-) >> >> diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h >> index 8a591455d796..28f7aa9b40bf 100644 >> --- a/fs/f2fs/segment.h >> +++ b/fs/f2fs/segment.h >> @@ -575,11 +575,10 @@ static inline int reserved_sections(struct f2fs_sb_info *sbi) >> return GET_SEC_FROM_SEG(sbi, reserved_segments(sbi)); >> } >> >> -static inline bool has_curseg_enough_space(struct f2fs_sb_info *sbi) >> +static inline bool has_curseg_enough_space(struct f2fs_sb_info *sbi, >> + unsigned int node_blocks, unsigned int dent_blocks) >> { >> - unsigned int node_blocks = get_pages(sbi, F2FS_DIRTY_NODES) + >> - get_pages(sbi, F2FS_DIRTY_DENTS); >> - unsigned int dent_blocks = get_pages(sbi, F2FS_DIRTY_DENTS); >> + >> unsigned int segno, left_blocks; >> int i; >> >> @@ -605,19 +604,24 @@ static inline bool has_curseg_enough_space(struct f2fs_sb_info *sbi) >> static inline bool has_not_enough_free_secs(struct f2fs_sb_info *sbi, >> int freed, int needed) >> { >> - int node_secs = get_blocktype_secs(sbi, F2FS_DIRTY_NODES); >> - int dent_secs = get_blocktype_secs(sbi, F2FS_DIRTY_DENTS); >> - int imeta_secs = get_blocktype_secs(sbi, F2FS_DIRTY_IMETA); >> + unsigned int total_node_blocks = get_pages(sbi, F2FS_DIRTY_NODES) + >> + get_pages(sbi, F2FS_DIRTY_DENTS) + >> + get_pages(sbi, F2FS_DIRTY_IMETA); >> + unsigned int total_dent_blocks = get_pages(sbi, F2FS_DIRTY_DENTS); >> + unsigned int node_secs = total_node_blocks / BLKS_PER_SEC(sbi); >> + unsigned int dent_secs = total_dent_blocks / BLKS_PER_SEC(sbi); >> + unsigned int node_blocks = total_node_blocks % BLKS_PER_SEC(sbi); >> + unsigned int dent_blocks = total_dent_blocks % BLKS_PER_SEC(sbi); >> >> if (unlikely(is_sbi_flag_set(sbi, SBI_POR_DOING))) >> return false; >> >> - if (free_sections(sbi) + freed == reserved_sections(sbi) + needed && >> - has_curseg_enough_space(sbi)) >> - return false; >> - return (free_sections(sbi) + freed) <= >> - (node_secs + 2 * dent_secs + imeta_secs + >> - reserved_sections(sbi) + needed); >> + if (free_sections(sbi) + freed <= >> + node_secs + dent_secs + reserved_sections(sbi) + needed) >> + return true; >> + if (!has_curseg_enough_space(sbi, node_blocks, dent_blocks)) >> + return true; >> + return false; >> } >> >> static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi) >> -- >> 2.32.0 _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel