From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 192E5C433DB for ; Wed, 17 Feb 2021 08:12:40 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9967764DE9 for ; Wed, 17 Feb 2021 08:12:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9967764DE9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=zededa.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.86139.161345 (Exim 4.92) (envelope-from ) id 1lCHwa-0001Ip-KF; Wed, 17 Feb 2021 08:12:20 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 86139.161345; Wed, 17 Feb 2021 08:12:20 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lCHwa-0001Ii-H3; Wed, 17 Feb 2021 08:12:20 +0000 Received: by outflank-mailman (input) for mailman id 86139; Wed, 17 Feb 2021 08:12:19 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lCHwZ-0001Id-Q6 for xen-devel@lists.xenproject.org; Wed, 17 Feb 2021 08:12:19 +0000 Received: from mail-qk1-x72b.google.com (unknown [2607:f8b0:4864:20::72b]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id e3a59a0a-b68a-40c0-8488-1526509153c7; Wed, 17 Feb 2021 08:12:17 +0000 (UTC) Received: by mail-qk1-x72b.google.com with SMTP id c3so11340750qkj.11 for ; Wed, 17 Feb 2021 00:12:17 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e3a59a0a-b68a-40c0-8488-1526509153c7 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zededa.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=VTjdFI+DhbdiZu1W84z4/bry4Ozmd78oQz1jcG+yAz8=; b=axOw/BlkT6+Fgu929G7QhIJhl+AsLB9nEhtRfHwYvE2H5ceqtkinWuagD51jTVbVCR AKUe9tVsk9liOydGR2Fgtq9kcStxAsgXTKw1tM2P8UnKVx1dSM8QpbePiq48t99DXV0J YSx+kSv7UBur7zrkYkuNlHMNKMDHwNWPfoEz0LKR7t/5GvZP4RnMR3uH6yiNhxV6fIPN yADghDzeYgRTEXnHW55jKoTt+zhWNFK3m630+DKVkI8k0+3KeGu2z7JgICx7onRZd+OW 4MuVpOXV9nyswuXcoTl5/xZp6kb/3s8ggCmvv+jllrOvP6+d+3DAMiN6JaBVGLXnxlnf wVOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=VTjdFI+DhbdiZu1W84z4/bry4Ozmd78oQz1jcG+yAz8=; b=ISTxfW2yT5mEJ9mrb0WhSIsFP8ibYQI4iYCjBMhZyJHBhTGB5k/zEIwV2lwfLcp+nk PLJ/WJmxRuTQSlSHmiKFWv8o9kOSsuvAHiaSwCux5RxY1nXae350swITOLGH0EA1Brzc 58areGHI/PJ4dWY5lm/iUltr+gtZ1W2UtyKESgX0T9Q4ExMdguaDjXkzr9ggCmZkootC yXRt055QMVerHdZflwY0YS/MtD3e9e8ZVXQi3gBZseHqMlH2IKAevlZz8FSGjpzzF3MD +ubb+nBL3nP4c5PglqCzgY+9Z4wpKFgkWGNdC2Hczr8RrzgkjYm9qAZ0luqeNgzvkg6X yfsg== X-Gm-Message-State: AOAM532woVIUQTbYTtOZFKuFMilcbFQYDWxwQJ64BmGbwnyQb/4gVR6I btoN3gwMHi7Hcjz4F0MXHJYVXjlu3bcyCv+igBBAvQ== X-Google-Smtp-Source: ABdhPJyDPOkIN7B3Sy9stn/hUhmweh/Z4xG0/NMCd9Jaw3934wXaTtNnai8UHWvY6Xa6ZuqbUaxvst3ZymC3O68IUR4= X-Received: by 2002:a37:d0b:: with SMTP id 11mr14688599qkn.267.1613549537469; Wed, 17 Feb 2021 00:12:17 -0800 (PST) MIME-Version: 1.0 References: <45b8ef4c-6d36-e91b-ca1a-a82eeca5aaf5@suse.com> In-Reply-To: <45b8ef4c-6d36-e91b-ca1a-a82eeca5aaf5@suse.com> From: Roman Shaposhnik Date: Wed, 17 Feb 2021 00:12:07 -0800 Message-ID: Subject: Re: Linux DomU freezes and dies under heavy memory shuffling To: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= Cc: Stefano Stabellini , Xen-devel , Jan Beulich , Andrew Cooper , =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi J=C3=BCrgen, thanks for taking a look at this. A few comments below: On Tue, Feb 16, 2021 at 10:47 PM J=C3=BCrgen Gro=C3=9F wr= ote: > > On 16.02.21 21:34, Stefano Stabellini wrote: > > + x86 maintainers > > > > It looks like the tlbflush is getting stuck? > > I have seen this case multiple times on customer systems now, but > reproducing it reliably seems to be very hard. It is reliably reproducible under my workload but it take a long time (~3 days of the workload running in the lab). > I suspected fifo events to be blamed, but just yesterday I've been > informed of another case with fifo events disabled in the guest. > > One common pattern seems to be that up to now I have seen this effect > only on systems with Intel Gold cpus. Can it be confirmed to be true > in this case, too? I am pretty sure mine isn't -- I can get you full CPU specs if that's usefu= l. > In case anybody has a reproducer (either in a guest or dom0) with a > setup where a diagnostic kernel can be used, I'd be _very_ interested! I can easily add things to Dom0 and DomU. Whether that will disrupt the experiment is, of course, another matter. Still please let me know what would be helpful to do. Thanks, Roman.