linux-ext4.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Theodore Y. Ts'o" <tytso@mit.edu>
To: Jan Kara <jack@suse.cz>
Cc: linux-ext4@vger.kernel.org
Subject: Re: [PATCH 0/19 v3] ext4: Fix transaction overflow due to revoke descriptors
Date: Thu, 14 Nov 2019 00:26:52 -0500	[thread overview]
Message-ID: <20191114052652.GB11994@mit.edu> (raw)
In-Reply-To: <20191113094545.GC6367@quack2.suse.cz>

On Wed, Nov 13, 2019 at 10:45:45AM +0100, Jan Kara wrote:
> Thanks for the heads up! I didn't do any performance testing with the jbd2
> changes specifically and our internal performance testing grid only checks
> Linus' kernel so it didn't yet even try to run that code. I'll queue some
> sqlite insert tests internally with my changes to see whether I'm able to
> reproduce. I don't have NVME disks available quickly but I guess SATA SSD
> could do the job as well...

Sorry, false alarm.  What Phoronix was testing was 5.3 versus 5.4-rcX,
using Ubuntu's bleeding-edge kernels.  It wouldn't have any of the
ext4 patches we have queued for the *next* merge window.

That being said, I wasn't able to reproduce performance delta using
upstream kernels, running on a Google Compute Engine VM, machtype
n1-highcpu-8, using a GCE Local SSD (SCSI-attached) for the first
benchmark, which I believe was the pts/sqlite benchmark using a thread
count of 1:

     Phoronix Test Suite 9.0.1
     SQLite 3.30.1
     Threads / Copies: 1
     Seconds < Lower Is Better
     5.3.0 ..................... 225 |===========================================
     5.4.0-rc3 ................. 224 |==========================================
     5.4-rc3-80-gafb2442fa429 .. 227 |===========================================
     5.4.0-rc7 ................. 223 |==========================================

     Processor: Intel Xeon (4 Cores / 8 Threads), Chipset: Intel 440FX
     82441FX PMC, Memory: 1 x 7373 MB RAM, Disk: 11GB PersistentDisk +
     403GB EphemeralDisk, Network: Red Hat Virtio device

     OS: Debian 10, Kernel: 5.4.0-rc3-xfstests (x86_64) 20191113, Compiler:
     GCC 8.3.0, File-System: ext4, System Layer: KVM

This was done using an extension to a gce-xfstests test appliance, to
which I hope to be adding an automation engine where it will kexec
into a series of kernels, run the benchmarks and then spit out the
report somewhere.  For now, the benchmarks are run manually.

(Adding commentary and click-baity titles is left as an exercise to
the reader.  :-)

						- Ted
						
P.S.  For all that I like to make snarky comments about Phoronix.com,
I have to admit Michael Larabel has done a pretty good job with his
performance test engine.  I probably would have choosen a different
implementation than PHP, and I'd have added an explicit way to specify
the file system to be tested other than mounting it on top of
/var/lib/phoronix-test-suite, and at least have the option of placing
the benchmarks' build trees and binaries in a different location than
the file system under test.

But that being said, he's collecting a decent set of benchmark tools,
and it is pretty cool that it has an automated way of collecting the
benchmark results, including the pretty graphs suitable for web
articles and conference slide decks ("and now, we turn to the rigged
benchmarks section of the presentation designed to show my new feature
in the best possible light...").

  parent reply	other threads:[~2019-11-14  5:27 UTC|newest]

Thread overview: 101+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-03 22:05 [PATCH 0/19 v3] ext4: Fix transaction overflow due to revoke descriptors Jan Kara
2019-10-03 22:05 ` [PATCH 01/22] jbd2: Fix possible overflow in jbd2_log_space_left() Jan Kara
2019-10-21  1:08   ` Theodore Y. Ts'o
2019-10-03 22:05 ` [PATCH 02/22] jbd2: Fixup stale comment in commit code Jan Kara
2019-10-21  1:08   ` Theodore Y. Ts'o
2019-10-03 22:05 ` [PATCH 03/22] ext4: Do not iput inode under running transaction in ext4_mkdir() Jan Kara
2019-10-21  1:21   ` Theodore Y. Ts'o
2019-10-24 10:19     ` Jan Kara
2019-10-24 12:09       ` Theodore Y. Ts'o
2019-10-24 13:37         ` Jan Kara
2019-11-04 12:35           ` Theodore Y. Ts'o
2019-10-03 22:05 ` [PATCH 04/22] ext4: Fix credit estimate for final inode freeing Jan Kara
2019-10-21  1:07   ` Theodore Y. Ts'o
2019-10-24 10:30     ` Jan Kara
2019-10-03 22:05 ` [PATCH 05/22] ext4: Fix ext4_should_journal_data() for EA inodes Jan Kara
2019-10-21  1:38   ` Theodore Y. Ts'o
2019-10-23 16:55     ` Jan Kara
2019-10-03 22:05 ` [PATCH 06/22] ext4: Use ext4_journal_extend() instead of jbd2_journal_extend() Jan Kara
2019-10-21  1:39   ` Theodore Y. Ts'o
2019-10-03 22:05 ` [PATCH 07/22] ext4: Avoid unnecessary revokes in ext4_alloc_branch() Jan Kara
2019-10-21 13:39   ` Theodore Y. Ts'o
2019-10-03 22:05 ` [PATCH 08/22] ext4: Provide function to handle transaction restarts Jan Kara
2019-10-21 16:20   ` Theodore Y. Ts'o
2019-10-23 16:25     ` Jan Kara
2019-10-03 22:05 ` [PATCH 09/22] ext4, jbd2: Provide accessor function for handle credits Jan Kara
2019-10-21 16:21   ` Theodore Y. Ts'o
2019-10-03 22:05 ` [PATCH 10/22] ocfs2: Use accessor function for h_buffer_credits Jan Kara
2019-10-21 16:21   ` Theodore Y. Ts'o
2019-10-03 22:05 ` [PATCH 11/22] jbd2: Fix statistics for the number of logged blocks Jan Kara
2019-10-21 16:24   ` Theodore Y. Ts'o
2019-10-03 22:05 ` [PATCH 12/22] jbd2: Reorganize jbd2_journal_stop() Jan Kara
2019-10-21 17:29   ` Theodore Y. Ts'o
2019-10-03 22:05 ` [PATCH 13/22] jbd2: Drop pointless check from jbd2_journal_stop() Jan Kara
2019-10-21 17:30   ` Theodore Y. Ts'o
2019-10-03 22:06 ` [PATCH 14/22] jbd2: Drop pointless wakeup " Jan Kara
2019-10-21 17:34   ` Theodore Y. Ts'o
2019-10-03 22:06 ` [PATCH 15/22] jbd2: Factor out common parts of stopping and restarting a handle Jan Kara
2019-10-21 17:49   ` Theodore Y. Ts'o
2019-10-23 16:17     ` Jan Kara
2019-11-04 12:36       ` Theodore Y. Ts'o
2019-11-04 12:59         ` Jan Kara
2019-10-03 22:06 ` [PATCH 16/22] jbd2: Account descriptor blocks into t_outstanding_credits Jan Kara
2019-10-21 21:04   ` Theodore Y. Ts'o
2019-10-23 13:09     ` Jan Kara
2019-10-03 22:06 ` [PATCH 17/22] jbd2: Drop jbd2_space_needed() Jan Kara
2019-10-21 21:05   ` Theodore Y. Ts'o
2019-10-03 22:06 ` [PATCH 18/22] jbd2: Reserve space for revoke descriptor blocks Jan Kara
2019-10-21 21:47   ` Theodore Y. Ts'o
2019-10-23 13:27     ` Jan Kara
2019-10-03 22:06 ` [PATCH 19/22] jbd2: Rename h_buffer_credits to h_total_credits Jan Kara
2019-10-21 21:48   ` Theodore Y. Ts'o
2019-10-03 22:06 ` [PATCH 20/22] jbd2: Make credit checking more strict Jan Kara
2019-10-21 22:29   ` Theodore Y. Ts'o
2019-10-23 13:30     ` Jan Kara
2019-10-03 22:06 ` [PATCH 21/22] ext4: Reserve revoke credits for freed blocks Jan Kara
2019-10-21 23:18   ` Theodore Y. Ts'o
2019-10-23 16:13     ` Jan Kara
2019-11-04 13:08       ` Theodore Y. Ts'o
2019-11-05  8:31         ` Jan Kara
2019-10-03 22:06 ` [PATCH 22/22] jbd2: Provide trace event for handle restarts Jan Kara
2019-10-21 23:18   ` Theodore Y. Ts'o
2019-10-19 19:19 ` [PATCH 0/19 v3] ext4: Fix transaction overflow due to revoke descriptors Theodore Y. Ts'o
2019-10-24 13:09   ` Jan Kara
2019-10-24 15:12     ` Jan Kara
2019-11-04  3:32 ` Theodore Y. Ts'o
2019-11-04 11:22   ` Jan Kara
2019-11-04 13:09     ` Theodore Y. Ts'o
2019-11-05 16:44 ` [PATCH 0/25 " Jan Kara
2019-11-05 16:44 ` [PATCH 01/25] jbd2: Fix possible overflow in jbd2_log_space_left() Jan Kara
2019-11-05 16:44 ` [PATCH 02/25] jbd2: Fixup stale comment in commit code Jan Kara
2019-11-05 16:44 ` [PATCH 03/25] jbd2: Completely fill journal descriptor blocks Jan Kara
2019-11-05 16:44 ` [PATCH 04/25] ext4: Move marking of handle as sync to ext4_add_nondir() Jan Kara
2019-11-05 16:44 ` [PATCH 05/25] ext4: Do not iput inode under running transaction Jan Kara
2019-11-05 16:44 ` [PATCH 06/25] ext4: Fix credit estimate for final inode freeing Jan Kara
2019-11-05 21:00   ` Theodore Y. Ts'o
2019-11-05 16:44 ` [PATCH 07/25] ext4: Fix ext4_should_journal_data() for EA inodes Jan Kara
2019-11-05 16:44 ` [PATCH 08/25] ext4: Use ext4_journal_extend() instead of jbd2_journal_extend() Jan Kara
2019-11-05 16:44 ` [PATCH 09/25] ext4: Avoid unnecessary revokes in ext4_alloc_branch() Jan Kara
2019-11-05 16:44 ` [PATCH 10/25] ext4: Provide function to handle transaction restarts Jan Kara
2019-11-05 16:44 ` [PATCH 11/25] ext4, jbd2: Provide accessor function for handle credits Jan Kara
2019-11-05 16:44 ` [PATCH 12/25] ocfs2: Use accessor function for h_buffer_credits Jan Kara
2019-11-05 16:44 ` [PATCH 13/25] jbd2: Fix statistics for the number of logged blocks Jan Kara
2019-11-05 16:44 ` [PATCH 14/25] jbd2: Reorganize jbd2_journal_stop() Jan Kara
2019-11-05 16:44 ` [PATCH 15/25] jbd2: Drop pointless check from jbd2_journal_stop() Jan Kara
2019-11-05 16:44 ` [PATCH 16/25] jbd2: Drop pointless wakeup " Jan Kara
2019-11-05 16:44 ` [PATCH 17/25] jbd2: Factor out common parts of stopping and restarting a handle Jan Kara
2019-11-05 16:44 ` [PATCH 18/25] jbd2: Account descriptor blocks into t_outstanding_credits Jan Kara
2019-11-05 16:44 ` [PATCH 19/25] jbd2: Drop jbd2_space_needed() Jan Kara
2019-11-05 16:44 ` [PATCH 20/25] jbd2: Reserve space for revoke descriptor blocks Jan Kara
2019-11-15  7:52   ` Eric Biggers
2019-11-15 10:02     ` Jan Kara
2019-11-15 14:20       ` Theodore Y. Ts'o
2019-11-15 17:10         ` Eric Biggers
2019-11-05 16:44 ` [PATCH 21/25] jbd2: Rename h_buffer_credits to h_total_credits Jan Kara
2019-11-05 16:44 ` [PATCH 22/25] jbd2: Make credit checking more strict Jan Kara
2019-11-05 16:44 ` [PATCH 23/25] ext4: Reserve revoke credits for freed blocks Jan Kara
2019-11-05 16:44 ` [PATCH 24/25] jbd2: Provide trace event for handle restarts Jan Kara
2019-11-05 16:44 ` [PATCH 25/25] jbd2: Fine tune estimate of necessary descriptor blocks Jan Kara
2019-11-05 21:04 ` [PATCH 0/25 v3] ext4: Fix transaction overflow due to revoke descriptors Theodore Y. Ts'o
     [not found] ` <20191112220614.GA11089@mit.edu>
     [not found]   ` <20191113094545.GC6367@quack2.suse.cz>
2019-11-14  5:26     ` Theodore Y. Ts'o [this message]
2019-11-14  8:49       ` [PATCH 0/19 " Jan Kara

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191114052652.GB11994@mit.edu \
    --to=tytso@mit.edu \
    --cc=jack@suse.cz \
    --cc=linux-ext4@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).