From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S965480AbcKJTiY (ORCPT <rfc822;w@1wt.eu>);
        Thu, 10 Nov 2016 14:38:24 -0500
Received: from bombadil.infradead.org ([198.137.202.9]:38915 "EHLO
        bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S934267AbcKJTiW (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 10 Nov 2016 14:38:22 -0500
Date: Thu, 10 Nov 2016 20:38:22 +0100
From: Peter Zijlstra <peterz@infradead.org>
To: Vitaly Wool <vitalywool@gmail.com>
Cc: Linux-MM <linux-mm@kvack.org>, linux-kernel@vger.kernel.org,
        Dan Streetman <ddstreet@ieee.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Andi Kleen <andi@firstfloor.org>
Subject: Re: [PATCH/RFC] z3fold: use per-page read/write lock
Message-ID: <20161110193822.GK3175@twins.programming.kicks-ass.net>
References: <20161105144946.3b4be0ee799ae61a82e1d918@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20161105144946.3b4be0ee799ae61a82e1d918@gmail.com>
User-Agent: Mutt/1.5.23.1 (2014-03-12)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Sat, Nov 05, 2016 at 02:49:46PM +0100, Vitaly Wool wrote:
> +/* Read-lock a z3fold page */
> +static void z3fold_page_rlock(struct z3fold_header *zhdr)
> +{
> +	while (!atomic_add_unless(&zhdr->page_lock, 1, Z3FOLD_PAGE_WRITE_FLAG))
> +		cpu_relax();
> +	smp_mb();
> +}
> +
> +/* Read-unlock a z3fold page */
> +static void z3fold_page_runlock(struct z3fold_header *zhdr)
> +{
> +	atomic_dec(&zhdr->page_lock);
> +	smp_mb();
> +}
> +
> +/* Write-lock a z3fold page */
> +static void z3fold_page_wlock(struct z3fold_header *zhdr)
> +{
> +	while (atomic_cmpxchg(&zhdr->page_lock, 0, Z3FOLD_PAGE_WRITE_FLAG) != 0)
> +		cpu_relax();
> +	smp_mb();
> +}
> +
> +/* Write-unlock a z3fold page */
> +static void z3fold_page_wunlock(struct z3fold_header *zhdr)
> +{
> +	atomic_sub(Z3FOLD_PAGE_WRITE_FLAG, &zhdr->page_lock);
> +	smp_mb();
> +}

This is trivially broken. What Andi said, don't roll your own locks.


The unlocks want: smp_mb__before_atomic() _before_ the atomic for
'obvious' reasons.

Also, this lock has serious starvation issues and is reader biased.