From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45F9CCA9EC0 for ; Mon, 28 Oct 2019 13:44:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1908C214B2 for ; Mon, 28 Oct 2019 13:44:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="D/PyAIoP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730143AbfJ1Nod (ORCPT ); Mon, 28 Oct 2019 09:44:33 -0400 Received: from mail-wr1-f47.google.com ([209.85.221.47]:40180 "EHLO mail-wr1-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725829AbfJ1Nod (ORCPT ); Mon, 28 Oct 2019 09:44:33 -0400 Received: by mail-wr1-f47.google.com with SMTP id o28so9946752wro.7 for ; Mon, 28 Oct 2019 06:44:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=DNiOV+6+GU/KYL9atfQs9NOj0c1poYrF1KJUfSYG+HY=; b=D/PyAIoPi6WzND0pRBPIJOaCxIbNNSedGBxhSsIXr/qz8mpeQl1GgmayEl+ZUSCttt 6mt24oVC4Fnzj8JZi7k6AlThJ2ClHgzkMDAA/vqE3/HIGq7E1pPUJzZ1EYqWM16uf/IV M1J0UIQQC3rcayQugi9ITAhVAXnEFBSgaHJedGkay+CtpM5atkQ9sc93BjU7OVrSXX9M nu5flZjaJ1zJtsvqnmT7lkh66WddQs8x61s11LoJGf0DcOhJcU8ExX5aplSTU/G+XYcz SPtmBlOochbPyKLcEyjyP3XKgqwQdNBi9nIl6pKtqwHTGr8o+9Lh/nBS7+/RIMf9YM0G 7hgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=DNiOV+6+GU/KYL9atfQs9NOj0c1poYrF1KJUfSYG+HY=; b=f9CtlgkMW4Emx2PU4bhm0unu5reJbpk2d+2JTvaZcuYAvTatgZuqzp4FsmIWMvsH3N lRAf1rCTe894ndrN/0sFv1ouBX9/D4um5y30UVrjGzr9rLttEtaqRfFFXP9HB7FWVY5f vfHJHGcaHC7ar5ys31941TE8CLNFgrCNQtyMGVHl2hbe/8VBx3cxBJgJzAQjVhRr3Ohm sVtl6oR8eqpeAEHIzKjC5nmM57Hwi1PmSDPn6e70u9t3DkVWyqR0VDqwuYLEIzwrZTHD Hr6dDoDpP0RO44eSgUrh9Z+KxFp9RkWWnbN0bJ/Jx7XOEzkgx4ia0qHyzZQztc0VqpUO 7Yyw== X-Gm-Message-State: APjAAAUGkoku1CkreOWqVNxTaW+Oav70QOn7AkFgRwXLwKjPpo1Cl2Kl 4cUfg4six6bz+qwSJqd59WEZ7jvfLPb1pwpspII= X-Google-Smtp-Source: APXvYqwlXrp3q/qbQ/uhuEqXb6SuGJ9ymGgcBCXaLEucFiADJoQQrp0ScnYPb80Oz1ZLisNkWuM/a35Ro8ciyQ62cyE= X-Received: by 2002:adf:e5cf:: with SMTP id a15mr15981099wrn.143.1572270271170; Mon, 28 Oct 2019 06:44:31 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Atemu Date: Mon, 28 Oct 2019 14:44:19 +0100 Message-ID: Subject: Re: BUG: btrfs send: Kernel's memory usage rises until OOM kernel panic after sending ~37GiB To: fdmanana@gmail.com Cc: Qu Wenruo , linux-btrfs Content-Type: text/plain; charset="UTF-8" Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org > You can run 'slabtop' while doing the send operation. > That might be enough. > > It's very likely the backreference walking code, due to huge ulists > (kmalloc-N slab), lots of btrfs_prelim_ref structures > (btrfs_prelim_ref slab), etc. I actually did run slabtop once but couldn't remember the exact name of the top entry, so I didn't mention it. Now that you mentioned the options though, I'm pretty sure it was kmalloc-N. N was probably 64 but that I'm not sure about. > Yes, it's likely a different bug. I don't think it's related either. I have only seen these warnings after the bug triggered though, reading the file under normal conditions doesn't produce them. What would be the best way to get more information on how btrfs comes to the conclusion that this file is corrupt? > Ideally yes. But that's a lot harder to do for several reasons and in > the end might not be worth it. I see, thanks! -Atemu