From mboxrd@z Thu Jan 1 00:00:00 1970 From: Blair Bethwaite Subject: Re: Dramatic performance drop at certain number of objects in pool Date: Mon, 27 Jun 2016 18:12:38 +1000 Message-ID: References: <1450235390.2134.1466084299677@ox.pcextreme.nl> <20160621075856.3ad471d1@batzmaru.gol.ad.jp> <20160624090806.1246b1ff@batzmaru.gol.ad.jp> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0983951030==" Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: ceph-users-bounces-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org Sender: "ceph-users" To: Kyle Bader Cc: "ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org" , Ceph Development List-Id: ceph-devel.vger.kernel.org --===============0983951030== Content-Type: multipart/alternative; boundary=94eb2c0933626fbfe305363e1517 --94eb2c0933626fbfe305363e1517 Content-Type: text/plain; charset=UTF-8 On 25 Jun 2016 6:02 PM, "Kyle Bader" wrote: > fdatasync takes longer when you have more inodes in the slab caches, it's the double edged sword of vfs_cache_pressure. That's a bit sad when, iiuc, it's only journals doing fdatasync in the Ceph write path. I'd have expected the vfs to handle this on a per fs basis (and a journal filesystem would have very little in the inode cache). It's somewhat annoying there isn't a way to favor dentries (and perhaps dentry inodes) over other inodes in the vfs cache. Our experience shows that it's dentry misses that cause the major performance issues (makes sense when you consider the osd is storing all its data in the leafs of the on disk PG structure). This is another discussion that seems to backup the choice to implement bluestore. Cheers, Blair --94eb2c0933626fbfe305363e1517 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On 25 Jun 2016 6:02 PM, "Kyle Bader" <kyle.bader-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> fdatasync takes longer when you have more inodes in the slab caches, i= t's the double edged sword of vfs_cache_pressure.

That's a bit sad when, iiuc, it's only journals doin= g fdatasync in the Ceph write path. I'd have expected the vfs to handle= this on a per fs basis (and a journal filesystem would have very little in= the inode cache).

It's somewhat annoying there isn't a way to favor de= ntries (and perhaps dentry inodes) over other inodes in the vfs cache. Our = experience shows that it's dentry misses that cause the major performan= ce issues (makes sense when you consider the osd is storing all its data in= the leafs of the on disk PG structure).

This is another discussion that seems to backup the choice t= o implement bluestore.

Cheers,
Blair

--94eb2c0933626fbfe305363e1517-- --===============0983951030== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ ceph-users mailing list ceph-users-idqoXFIVOFJgJs9I8MT0rw@public.gmane.org http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com --===============0983951030==--