From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDBE4C04EBF for ; Mon, 23 Sep 2019 15:00:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BCF9F20835 for ; Mon, 23 Sep 2019 15:00:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BCF9F20835 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5E7BB6B000A; Mon, 23 Sep 2019 11:00:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 599426B000C; Mon, 23 Sep 2019 11:00:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 487D26B000D; Mon, 23 Sep 2019 11:00:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0149.hostedemail.com [216.40.44.149]) by kanga.kvack.org (Postfix) with ESMTP id 2767C6B000A for ; Mon, 23 Sep 2019 11:00:38 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id AFF48181AC9C9 for ; Mon, 23 Sep 2019 15:00:37 +0000 (UTC) X-FDA: 75966496914.18.spoon20_79835095d5701 X-HE-Tag: spoon20_79835095d5701 X-Filterd-Recvd-Size: 5901 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 15:00:34 +0000 (UTC) Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8246E85536 for ; Mon, 23 Sep 2019 15:00:33 +0000 (UTC) Received: by mail-qt1-f199.google.com with SMTP id r15so17551144qtn.12 for ; Mon, 23 Sep 2019 08:00:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=QNoXmel+GlMigZDcUd1vYpLYfL7zFiKdxBF3l1k0lPQ=; b=Q3QclkKmhlblQ/qvkd3vgX8lo86F69j+X0diurHpchY3hkvVUnHgPws+NGGz/V8sQj T+NnbAFkssbK/sEIGQTIdHZvYOqVBrma9SgsJU0J0NwvHqGPFSJ8zj0YJoqw38wrmoRS oOubsaShjfdBR+/IxcdeHU4Ss/7tvX5KOZbHAeOw8D8N1PYpTR83TdhgNkP25QjQBnoJ w6RLW/06VBipLqnPcy6GM2UXccOHBtuwHHxHUoFm067YfwKuXxI3VBeX2uQVIt+q49/t A+mxazqUC+iy/kd9PDjsm9Zy61+R6gMMGpHY8HFtGg5Ndgp4IIBtfYnmrst2476h461l V71Q== X-Gm-Message-State: APjAAAU//Ybb0XeUSF8FfHiiG4B15BwL/E/Bj8aPdkPmBcfh/BZwQZ5D H2wepylsvYp9v9C8t73qCzNQWSQm6orDSi+X5JhI/wtYesRPbD1H3F9Z/5sx+DMv5r+b5BFWsdq F/wt7wtJBhbo= X-Received: by 2002:ac8:2c50:: with SMTP id e16mr317605qta.257.1569250832851; Mon, 23 Sep 2019 08:00:32 -0700 (PDT) X-Google-Smtp-Source: APXvYqyFUJn6BvXztT9hmWoUunZD0ZUcW4uUuFms3FLLwyr9jPXb7ZlVF254P3OydfcVZPOAJdBaaA== X-Received: by 2002:ac8:2c50:: with SMTP id e16mr317578qta.257.1569250832651; Mon, 23 Sep 2019 08:00:32 -0700 (PDT) Received: from redhat.com (bzq-79-176-40-226.red.bezeqint.net. [79.176.40.226]) by smtp.gmail.com with ESMTPSA id j17sm5330553qki.99.2019.09.23.08.00.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 08:00:31 -0700 (PDT) Date: Mon, 23 Sep 2019 11:00:23 -0400 From: "Michael S. Tsirkin" To: Alexander Duyck Cc: virtio-dev@lists.oasis-open.org, kvm list , David Hildenbrand , Dave Hansen , LKML , Matthew Wilcox , Michal Hocko , linux-mm , Vlastimil Babka , Andrew Morton , Mel Gorman , linux-arm-kernel@lists.infradead.org, Oscar Salvador , Yang Zhang , Pankaj Gupta , Konrad Rzeszutek Wilk , Nitesh Narayan Lal , Rik van Riel , lcapitulino@redhat.com, "Wang, Wei W" , Andrea Arcangeli , Paolo Bonzini , Dan Williams , Alexander Duyck Subject: Re: [PATCH v10 3/6] mm: Introduce Reported pages Message-ID: <20190923105746-mutt-send-email-mst@kernel.org> References: <20190918175109.23474.67039.stgit@localhost.localdomain> <20190918175249.23474.51171.stgit@localhost.localdomain> <20190923041330-mutt-send-email-mst@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Sep 23, 2019 at 07:50:15AM -0700, Alexander Duyck wrote: > > > +static inline void > > > +page_reporting_reset_boundary(struct zone *zone, unsigned int order, int mt) > > > +{ > > > + int index; > > > + > > > + if (order < PAGE_REPORTING_MIN_ORDER) > > > + return; > > > + if (!test_bit(ZONE_PAGE_REPORTING_ACTIVE, &zone->flags)) > > > + return; > > > + > > > + index = get_reporting_index(order, mt); > > > + reported_boundary[index] = &zone->free_area[order].free_list[mt]; > > > +} > > > > So this seems to be costly. > > I'm guessing it's the access to flags: > > > > > > /* zone flags, see below */ > > unsigned long flags; > > > > /* Primarily protects free_area */ > > spinlock_t lock; > > > > > > > > which is in the same cache line as the lock. > > I'm not sure what you mean by this being costly? I've just been wondering why does will it scale report a 1.5% regression with this patch. > Also, at least on my system, pahole seems to indicate they are in > different cache lines. > > /* --- cacheline 3 boundary (192 bytes) --- */ > struct zone_padding _pad1_; /* 192 0 */ > struct free_area free_area[11]; /* 192 1144 */ > /* --- cacheline 20 boundary (1280 bytes) was 56 bytes ago --- */ > long unsigned int flags; /* 1336 8 */ > /* --- cacheline 21 boundary (1344 bytes) --- */ > spinlock_t lock; /* 1344 4 */ > > Basically these flags aren't supposed to be touched unless we are > holding the lock anyway so I am not sure it would be all that costly > for this setup. Basically we are holding the lock when the flag is set > or cleared, and we only set it if it is not already set. If needed > though I suppose I could look at moving the flags if you think that is > an issue. However I would probably need to add some additional padding > to prevent the lock from getting into the same cache line as the > free_area values. -- MST