From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98C67C43331 for ; Thu, 26 Mar 2020 08:51:07 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 70EC520714 for ; Thu, 26 Mar 2020 08:51:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 70EC520714 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jHOE4-0003EB-UM; Thu, 26 Mar 2020 08:50:56 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1jHOE4-0003E5-2o for xen-devel@lists.xenproject.org; Thu, 26 Mar 2020 08:50:56 +0000 X-Inumbo-ID: e7700914-6f3e-11ea-bec1-bc764e2007e4 Received: from mx2.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id e7700914-6f3e-11ea-bec1-bc764e2007e4; Thu, 26 Mar 2020 08:50:55 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id AA350ACCE; Thu, 26 Mar 2020 08:50:54 +0000 (UTC) To: Jan Beulich , Julien Grall References: <20200325105511.20882-1-jgross@suse.com> <20200325105511.20882-3-jgross@suse.com> <1c688233-2f64-9dd0-7d98-4a0840489293@xen.org> <5950cabe-0065-904c-6c61-0e6eab8f8060@suse.com> <7539a91f-6b9a-865d-f8c3-b8124d16f17a@suse.com> From: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= Message-ID: Date: Thu, 26 Mar 2020 09:50:54 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.5.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [Xen-devel] [PATCH v7 2/5] xen/rcu: don't use stop_machine_run() for rcu_barrier() X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Wei Liu , Andrew Cooper , Ian Jackson , George Dunlap , xen-devel@lists.xenproject.org Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" On 26.03.20 09:49, Jan Beulich wrote: > On 26.03.2020 08:24, Jürgen Groß wrote: >> On 26.03.20 07:58, Jan Beulich wrote: >>> On 25.03.2020 17:13, Julien Grall wrote: >>>> On 25/03/2020 10:55, Juergen Gross wrote: >>>>> @@ -143,51 +143,90 @@ static int qhimark = 10000; >>>>>    static int qlowmark = 100; >>>>>    static int rsinterval = 1000; >>>>>    -struct rcu_barrier_data { >>>>> -    struct rcu_head head; >>>>> -    atomic_t *cpu_count; >>>>> -}; >>>>> +/* >>>>> + * rcu_barrier() handling: >>>>> + * Two counters are used to synchronize rcu_barrier() work: >>>>> + * - cpu_count holds the number of cpus required to finish barrier handling. >>>>> + *   It is decremented by each cpu when it has performed all pending rcu calls. >>>>> + * - pending_count shows whether any rcu_barrier() activity is running and >>>>> + *   it is used to synchronize leaving rcu_barrier() only after all cpus >>>>> + *   have finished their processing. pending_count is initialized to nr_cpus + 1 >>>>> + *   and it is decremented by each cpu when it has seen that cpu_count has >>>>> + *   reached 0. The cpu where rcu_barrier() has been called will wait until >>>>> + *   pending_count has been decremented to 1 (so all cpus have seen cpu_count >>>>> + *   reaching 0) and will then set pending_count to 0 indicating there is no >>>>> + *   rcu_barrier() running. >>>>> + * Cpus are synchronized via softirq mechanism. rcu_barrier() is regarded to >>>>> + * be active if pending_count is not zero. In case rcu_barrier() is called on >>>>> + * multiple cpus it is enough to check for pending_count being not zero on entry >>>>> + * and to call process_pending_softirqs() in a loop until pending_count drops to >>>>> + * zero, before starting the new rcu_barrier() processing. >>>>> + */ >>>>> +static atomic_t cpu_count = ATOMIC_INIT(0); >>>>> +static atomic_t pending_count = ATOMIC_INIT(0); >>>>>      static void rcu_barrier_callback(struct rcu_head *head) >>>>>    { >>>>> -    struct rcu_barrier_data *data = container_of( >>>>> -        head, struct rcu_barrier_data, head); >>>>> -    atomic_inc(data->cpu_count); >>>>> +    smp_mb__before_atomic();     /* Make all writes visible to other cpus. */ >>>> >>>> smp_mb__before_atomic() will order both read and write. However, the >>>> comment suggest only the write are required to be ordered. >>>> >>>> So either the barrier is too strong or the comment is incorrect. Can >>>> you clarify it? >>> >>> Neither is the case, I guess: There simply is no smp_wmb__before_atomic() >>> in Linux, and if we want to follow their model we shouldn't have one >>> either. I'd rather take the comment to indicate that if one appeared, it >>> could be used here. >> >> Right. Currently we have the choice of either using >> smp_mb__before_atomic() which is too strong for Arm, or smp_wmb() which >> is too strong for x86. > > For x86 smp_wmb() is actually only very slightly too strong - it expands > to just barrier(), after all. So overall perhaps that's the better > choice here (with a suitable comment)? Fine with me. Juergen