From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1954507AbdDZGI0 (ORCPT <rfc822;w@1wt.eu>);
        Wed, 26 Apr 2017 02:08:26 -0400
Received: from mail-pf0-f193.google.com ([209.85.192.193]:34559 "EHLO
        mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S1950748AbdDZGIZ (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Wed, 26 Apr 2017 02:08:25 -0400
Date: Wed, 26 Apr 2017 15:08:18 +0900
From: Joonsoo Kim <js1304@gmail.com>
To: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
        Minchan Kim <minchan@kernel.org>,
        Sergey Senozhatsky <sergey.senozhatsky@gmail.com>,
        linux-kernel@vger.kernel.org, kernel-team@lge.com
Subject: Re: [PATCH v4 2/4] zram: implement deduplication in zram
Message-ID: <20170426060816.GD29773@js1304-desktop>
References: <1493167946-10936-1-git-send-email-iamjoonsoo.kim@lge.com>
 <1493167946-10936-3-git-send-email-iamjoonsoo.kim@lge.com>
 <20170426042826.GD673@jagdpanzerIV.localdomain>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170426042826.GD673@jagdpanzerIV.localdomain>
User-Agent: Mutt/1.5.24 (2015-08-30)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Apr 26, 2017 at 01:28:26PM +0900, Sergey Senozhatsky wrote:
> On (04/26/17 09:52), js1304@gmail.com wrote:
> [..]
> > +struct zram_hash {
> > +	spinlock_t lock;
> > +	struct rb_root rb_root;
> >  };
> 
> just a note.
> 
> we can easily have N CPUs spinning on ->lock for __zram_dedup_get() lookup,
> which can invole a potentially slow zcomp_decompress() [zlib, for example,
> with 64k pages] and memcmp(). the larger PAGE_SHIFT is, the more serialized
> IOs become. in theory, at least.
> 
> CPU0				CPU1		...	CPUN
> 
> __zram_bvec_write()	__zram_bvec_write()		__zram_bvec_write()
>  zram_dedup_find()	 zram_dedup_find()		 zram_dedup_find()
>   spin_lock(&hash->lock);
> 			  spin_lock(&hash->lock);	  spin_lock(&hash->lock);
>    __zram_dedup_get()
>     zcomp_decompress()
>      ...
> 
> 
> so may be there is a way to use read-write lock instead on spinlock for hash
> and reduce write/read IO serialization.

In fact, dedup release hash->lock before doing zcomp_decompress(). So,
above contention cannot happen.

However, contention still possible when traversing the rb_tree. If
your fio shows that contention, I will change it to read-write lock.

Thanks.