From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from out1-smtp.messagingengine.com ([66.111.4.25]:43291 "EHLO out1-smtp.messagingengine.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725989AbeLGNiF (ORCPT ); Fri, 7 Dec 2018 08:38:05 -0500 From: Nikolaus Rath To: Miklos Szeredi Cc: linux-fsdevel@vger.kernel.org, fuse-devel Subject: Re: [fuse-devel] [fuse] Speeding up readdir() References: <87va46pgqu.fsf@vostro.rath.org> Date: Fri, 07 Dec 2018 13:38:01 +0000 In-Reply-To: (Miklos Szeredi's message of "Fri, 7 Dec 2018 10:21:41 +0100") Message-ID: <871s6ttq2u.fsf@vostro.rath.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Dec 07 2018, Miklos Szeredi wrote: > On Thu, Dec 6, 2018 at 9:01 PM Nikolaus Rath wrote: >> >> Hello, >> >> I am trying to improve the performance of readdir() requests. I have a >> client application that issues a lot of readdir() requests, and a FUSE >> filesystem that makes extensive use of kernel caching for inode entry >> attributes because retrieving the attributes from the backend is >> relatively expensive. >> >> Unfortunately, it seems to me that currently there is no way to avoid >> having to retrieve the attributes from the backend for every entry that >> is returned by readdir - on every call: >> >> If I am using readdirplus, I have to include the full attributes even if >> the kernel already has them cached. >> >> If I disable readdirplus, I can return just the entry name and its inode >> - but I believe because this doesn't result in a lookup count increase >> of the inode, the kernel can't match this with the existing cached data >> for the inode (is that correct?) and I'm getting a separate lookup() >> request for each entry that I've returned. > > Was the entry timed out? If not, then there shouldn't've been a lookup. And there is no lookup indeed, what I am getting is additional getattr() requests. Sorry for the confusion. Here some more concrete data: When enabling readdirplus(), I have: Operation counts:=20 lookup 1196 create 4 flush 1034 lookup_new 0 fsync 0 fsyncdir 0 getattr 12818 link 2 unlink 4 mknod 0 mkdir 0 forget 2 open 1001 opendir 19635 read 3999 readdir 19393 readdirplus 22654 readlink 0 release 1005 releasedir 19168 rename 2 rmdir 0 setattr 6 statfs 142 symlink 0 write 9 max_fd 1438 When disabling readdirplus, I get: Operation counts: lookup 1298 create 0 flush 1031 lookup_new 0 fsync 0 fsyncdir 0 getattr 22346 link 0 unlink 0 mknod 0 mkdir 0 forget 0 open 1002 opendir 18955 read 3916 readdir 36162 readdirplus 0 readlink 0 release 1002 releasedir 18378 rename 0 rmdir 0 setattr 6 statfs 142 symlink 0 write 5=09=20 i.e. ~10k additional getattr() requests. That is lot less than the number of extra readdir requests (~22k), especially considering that each readdir() returns a number of entries. However, it is still more than expected given that there are only 1298 distinct directory entries that are all cached. Nevertheless, the overall performance is unchanged (variation within measurement noise of 5%). This is odd because my readdirplus() implementation internally calls the getattr() handler so I would have expected big savings from using plan readdir instead. I tried to run the whole filesystem under oprofile, but found that essentially all the time was spent outside the filesystem code: $ opreport --symbols -g | head Using /home/nikratio/readdirplus/oprofile_data/samples/ for samples directo= ry. CPU: AMD64 generic, speed 3500 MHz (estimated) Counted CPU_CLK_UNHALTED events (CPU Clocks not Halted) with a unit mask of= 0x00 (No unit mask) count 100000 samples % linenr info image name app = name symbol name 110313 88.7117 (no location information) no-vmlinux test= fs /no-vmlinux 2341 1.8826 vfscanf.c:277 libc-2.28.so test= fs _IO_vfscanf 1183 0.9513 memcpy-ssse3.S:67 libc-2.28.so test= fs __memcpy_ssse3 803 0.6458 cacheinfo.c:259 libc-2.28.so test= fs handle_intel.constprop.1 784 0.6305 vfscanf.c:277 libc-2.28.so test= fs _IO_vfwscanf Any suggestions? Best, -Nikolaus --=20 GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F =C2=BBTime flies like an arrow, fruit flies like a Banana.=C2= =AB