From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3AE4C47E49 for ; Sun, 27 Oct 2019 15:19:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7847221850 for ; Sun, 27 Oct 2019 15:19:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XC50yoVi" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727099AbfJ0PTo (ORCPT ); Sun, 27 Oct 2019 11:19:44 -0400 Received: from mail-wm1-f67.google.com ([209.85.128.67]:38075 "EHLO mail-wm1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726814AbfJ0PTo (ORCPT ); Sun, 27 Oct 2019 11:19:44 -0400 Received: by mail-wm1-f67.google.com with SMTP id 22so6476535wms.3 for ; Sun, 27 Oct 2019 08:19:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=YJb+vmjy0wXs/Kws6kGlO2P4oTjtM6GMyzuyiFSYUAw=; b=XC50yoViEgR0WRfNzML48D2gPidFTyG1XtoDnEkJbVmCMDviRzCAslgZpX+0TOlS0+ IpGFH6Uh/iseGG8GBLWlAgOfAntZhRlsfXP5bIGGqmlLkAy/2a6zTJ3Rs4SlYzOWgQD+ 05QPNF3UQyQql5tmz8hGcFpAEg/YDXSgi7duW9OuIJs235LmbWkK6BSpHkmwML9twwfm iL6uPTInoFJX97RBvWBEjAKxq6z0nHnaRkBlO2rrsRedpmqpVWCqdRNjMDoZL5eeR6oH 8yj3CAX5uo8gomb+wXToknmOkwAuTgZlVmdwFXYnZ7YjtaY9pUoxESbZbxftZBRSQD1o XiJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=YJb+vmjy0wXs/Kws6kGlO2P4oTjtM6GMyzuyiFSYUAw=; b=dlynsQjqG7EyBau8BdokGV6Dthak/w8oi/74paHOWjc8GEeE23UpmkW+Z84twbYGEc 8hOttqWAGBURk1wyfABjvVpu7ecFBVsXTJvilpopWUZoIQmCTPtJb8Oo8xK2M5ejO96t iaJF3vv/b6kRsTFg2mlGe8qq4ynSGnKmu/XtfYcBfHjFG79BX1Uvji4+YzrxSWaawgLg y8oezjuVr3jHzpRvvayGxcxgWpe675d+DSlp0gezoix2Ch+ii8RrCnFT0GepsPQsUR3q LXOEEirV4SMWAuYDH8rXvpXlUCwHer6nQAU/ssIQAgl4iNz1j6KomjTZlSO8FuigqCcR 61zQ== X-Gm-Message-State: APjAAAW//JyQdf75JI7s7uYJcbtLTnijdp+Krp7xCd9OvKI21yRg2wHN EVeHDRa2P4JCKS8CoeICNhwskOKAHIgnRtYfiIPFp9F9704= X-Google-Smtp-Source: APXvYqzr9jwHprsLxL423U57iA3LUzWbjxci6LeJZDHrEV2mCBO4bJqjV39jHtq45ucmEkxqUrf4y8ww8FOWLrMMAiw= X-Received: by 2002:a7b:c049:: with SMTP id u9mr11674388wmc.12.1572189582197; Sun, 27 Oct 2019 08:19:42 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Atemu Date: Sun, 27 Oct 2019 16:19:31 +0100 Message-ID: Subject: Re: BUG: btrfs send: Kernel's memory usage rises until OOM kernel panic after sending ~37GiB To: Qu Wenruo Cc: linux-btrfs@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org > It's really hard to determine, you could try the following command to > determine: > # btrfs ins dump-tree -t extent --bfs /dev/nvme/btrfs |\ > grep "(.*_ITEM.*)" | awk '{print $4" "$5" "$6" size "$10}' > > Then which key is the most shown one and its size. > > If a key's objectid (the first value) shows up multiple times, it's a > kinda heavily shared extent. > > Then search that objectid in the full extent tree dump, to find out how > it's shared. I analyzed it a bit differently but this should be the information we wanted: https://gist.github.com/Atemu/206c44cd46474458c083721e49d84a42 Yeah... Is there any way to "unshare" these worst cases without having to btrfs defragment everything? I also uploaded the (compressed) extent tree dump if you want to take a look yourself (205MB, expires in 7 days): https://send.firefox.com/download/a729c57a94fcd89e/#w51BjzRmGnCg2qKNs39UNw -Atemu