From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79593C31E45 for ; Thu, 13 Jun 2019 21:13:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0000321473 for ; Thu, 13 Jun 2019 21:13:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=dilger-ca.20150623.gappssmtp.com header.i=@dilger-ca.20150623.gappssmtp.com header.b="p+G7/1/e" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730425AbfFMVNw (ORCPT ); Thu, 13 Jun 2019 17:13:52 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:39870 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730028AbfFMVNv (ORCPT ); Thu, 13 Jun 2019 17:13:51 -0400 Received: by mail-pf1-f196.google.com with SMTP id j2so38186pfe.6 for ; Thu, 13 Jun 2019 14:13:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dilger-ca.20150623.gappssmtp.com; s=20150623; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=ZAuTZrP5uwQtOWPRneiBhZJlP/UXmquxD1j2HqAKSiM=; b=p+G7/1/ewjdB9HiQK+FdGr9MGLEhptw5EBH1QNWtX8WvwCX0lGsYlBUOW+633P3SMf G3ArR6YWwP4rUw7veR06XTVJAzctOG820blMP5VxSAZBZFJ02/5PGqFAqx2Ea69cNDF8 m8H/DHV48bUmaH1by8iIxDw2aLLdX4k7Z14US2Y3Fj7foRinbkEmHOjrw0Mv8iYxUX2u SsnmpKr0zBlSBALDiCWQAkbXPbmxATJS1dz6FV8v0iPfLr3YmA0gC2LFQ4G5qWGKmQO0 TCgWBx/A7r7kBGD1OntUhFFtvP9WQb0K7Snot0KwcxExRKxjibBGPkSrySsAm0jR9f1R gqGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=ZAuTZrP5uwQtOWPRneiBhZJlP/UXmquxD1j2HqAKSiM=; b=FDGgl8wJeMEoF0jjpc4AZtLGblAtUD8SKZ20Tlwpsh54W9PA7nKhe73QvJH58NmJ+B 4m30dUOlbrep40GcR6qFzs6mfOJ9uCwmwNcZpcGEwKAq0NtD6BeJGwHBLdAt7sF39kYU +cNEFfGWopB1Ylode24Lc6WsmwIztt9lFS2xLDpCIPMrjWVYRW7+Xc5WZ4/EHsoU+fYC O2RZ9IBPydf1aqVP8a842WGR3In8GQfKHzYx0lvUWS7qNysKPv0+j9/ga1OHrvS45z0L UqHKWmDVZTfHjS7DLYN8YIjq4oA0/0LLKdeYkpAw0SOYVHrRi+kP7gb/mQf+ws0WvFDp 7+QQ== X-Gm-Message-State: APjAAAVs4/3Pi5OOx5qPdjUD2MLESUdEVT/SRPpIEG04/ncri6Sldt/N 68w+n9hP98rVZCNn2I17eQCdkA== X-Google-Smtp-Source: APXvYqxgDAHXGwSH9hV9RC35wOp7XY+1n/0PHYoshJaSaX5zOh3oBmTkIPT8VtcOw+MxLgJ8/kF2EQ== X-Received: by 2002:a63:454a:: with SMTP id u10mr30570204pgk.291.1560460427986; Thu, 13 Jun 2019 14:13:47 -0700 (PDT) Received: from cabot-wlan.adilger.int (S0106a84e3fe4b223.cg.shawcable.net. [70.77.216.213]) by smtp.gmail.com with ESMTPSA id f2sm595069pgs.83.2019.06.13.14.13.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 13 Jun 2019 14:13:46 -0700 (PDT) From: Andreas Dilger Message-Id: Content-Type: multipart/signed; boundary="Apple-Mail=_EA59EFD2-F69E-4324-A987-ABC5C770B932"; protocol="application/pgp-signature"; micalg=pgp-sha256 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: pagecache locking (was: bcachefs status update) merged) Date: Thu, 13 Jun 2019 15:13:40 -0600 In-Reply-To: <20190613183625.GA28171@kmo-pixel> Cc: Dave Chinner , Linus Torvalds , Dave Chinner , "Darrick J . Wong" , Christoph Hellwig , Matthew Wilcox , Amir Goldstein , Jan Kara , Linux List Kernel Mailing , linux-xfs , linux-fsdevel , Josef Bacik , Alexander Viro , Andrew Morton To: Kent Overstreet References: <20190610191420.27007-1-kent.overstreet@gmail.com> <20190611011737.GA28701@kmo-pixel> <20190611043336.GB14363@dread.disaster.area> <20190612162144.GA7619@kmo-pixel> <20190612230224.GJ14308@dread.disaster.area> <20190613183625.GA28171@kmo-pixel> X-Mailer: Apple Mail (2.3273) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org --Apple-Mail=_EA59EFD2-F69E-4324-A987-ABC5C770B932 Content-Type: multipart/mixed; boundary="Apple-Mail=_20FAA3B9-CB10-4AA6-8146-ACB6B9D72F27" --Apple-Mail=_20FAA3B9-CB10-4AA6-8146-ACB6B9D72F27 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On Jun 13, 2019, at 12:36 PM, Kent Overstreet = wrote: >=20 > On Thu, Jun 13, 2019 at 09:02:24AM +1000, Dave Chinner wrote: >> On Wed, Jun 12, 2019 at 12:21:44PM -0400, Kent Overstreet wrote: >>> Ok, I'm totally on board with returning EDEADLOCK. >>>=20 >>> Question: Would we be ok with returning EDEADLOCK for any IO where = the buffer is >>> in the same address space as the file being read/written to, even if = the buffer >>> and the IO don't technically overlap? >>=20 >> I'd say that depends on the lock granularity. For a range lock, >> we'd be able to do the IO for non-overlapping ranges. For a normal >> mutex or rwsem, then we risk deadlock if the page fault triggers on >> the same address space host as we already have locked for IO. That's >> the case we currently handle with the second IO lock in XFS, ext4, >> btrfs, etc (XFS_MMAPLOCK_* in XFS). >>=20 >> One of the reasons I'm looking at range locks for XFS is to get rid >> of the need for this second mmap lock, as there is no reason for it >> existing if we can lock ranges and EDEADLOCK inside page faults and >> return errors. >=20 > My concern is that range locks are going to turn out to be both more = complicated > and heavier weight, performance wise, than the approach I've taken of = just a > single lock per address space. >=20 > Reason being range locks only help when you've got multiple operations = going on > simultaneously that don't conflict - i.e. it's really only going to be = useful > for applications that are doing buffered IO and direct IO = simultaneously to the > same file. Personally, I think that would be a pretty gross thing to = do and I'm > not particularly interested in optimizing for that case myself... but, = if you > know of applications that do depend on that I might change my opinion. = If not, I > want to try and get the simpler, one-lock-per-address space approach = to work. There are definitely workloads that require multiple threads doing = non-overlapping writes to a single file in HPC. This is becoming an increasingly common = problem as the number of cores on a single client increase, since there is = typically one thread per core trying to write to a shared file. Using multiple files = (one per core) is possible, but that has file management issues for users when = there are a million cores running on the same job/file (obviously not on the same = client node) dumping data every hour. We were just looking at this exact problem last week, and most of the = threads are spinning in grab_cache_page_nowait->add_to_page_cache_lru() and = set_page_dirty() when writing at 1.9GB/s when they could be writing at 5.8GB/s (when = threads are writing O_DIRECT instead of buffered). Flame graph is attached for = 16-thread case, but high-end systems today easily have 2-4x that many cores. Any approach for range locks can't be worse than spending 80% of time = spinning. Cheers, Andreas --Apple-Mail=_20FAA3B9-CB10-4AA6-8146-ACB6B9D72F27 Content-Disposition: inline; filename=shared_file_write.svg Content-Type: image/svg+xml; x-unix-mode=0666; name="shared_file_write.svg" Content-Transfer-Encoding: 7bit Flame Graph Reset Zoom Search cl_vmpage_page (7 samples, 0.02%) __mod_zone_page_state (4 samples, 0.01%) cl_io_init (4 samples, 0.01%) mem_cgroup_charge_common (300 samples, 0.69%) osc_lru_reclaim (11 samples, 0.03%) native_queued_spin_lock_slowpath (5 samples, 0.01%) radix_tree_descend (34 samples, 0.08%) ldlm_lock_match_with_skip (5 samples, 0.01%) [libpthread-2.17.so] (43,011 samples, 98.40%) [libpthread-2.17.so] cl_page_make_ready (23 samples, 0.05%) cl_page_state_set0 (9 samples, 0.02%) mem_cgroup_charge_statistics.isra.21 (9 samples, 0.02%) [unknown] (4 samples, 0.01%) bit_wait_io (25 samples, 0.06%) native_queued_spin_lock_slowpath (17,339 samples, 39.67%) native_queued_spin_lock_slowpath zone_statistics (17 samples, 0.04%) kmem_cache_alloc (5 samples, 0.01%) cl_io_start (25 samples, 0.06%) ___slab_alloc (4 samples, 0.01%) __list_add (18 samples, 0.04%) radix_tree_descend (13 samples, 0.03%) _raw_spin_lock_irqsave (6 samples, 0.01%) cl_io_top (7 samples, 0.02%) get_task_policy.part.26 (4 samples, 0.01%) __close_nocancel (13 samples, 0.03%) lru_cache_add (150 samples, 0.34%) lov_io_call.isra.7 (25 samples, 0.06%) __css_tryget (5 samples, 0.01%) radix_tree_descend (9 samples, 0.02%) radix_tree_descend (11 samples, 0.03%) [libmlx4-rdmav2.so] (18 samples, 0.04%) rcu_check_callbacks (4 samples, 0.01%) do_notify_resume (9 samples, 0.02%) __rmqueue (4 samples, 0.01%) osc_enqueue_base (5 samples, 0.01%) lov_lock_init (5 samples, 0.01%) __radix_tree_create (54 samples, 0.12%) osc_io_commit_async (20,035 samples, 45.84%) osc_io_commit_async cl_page_list_del (214 samples, 0.49%) __percpu_counter_add (5 samples, 0.01%) __schedule (92 samples, 0.21%) osc_io_fsync_start (218 samples, 0.50%) __radix_tree_preload (16 samples, 0.04%) lovsub_attr_update (4 samples, 0.01%) mca_pml_ob1_progress (10 samples, 0.02%) __libc_fsync (292 samples, 0.67%) osc_enter_cache_try.constprop.36 (98 samples, 0.22%) tick_sched_timer (24 samples, 0.05%) cl_page_state_set0 (8 samples, 0.02%) cl_sync_file_range (25 samples, 0.06%) search_binary_handler (4 samples, 0.01%) find_get_pages_tag (4 samples, 0.01%) cl_lock_cancel (6 samples, 0.01%) native_queued_spin_lock_slowpath (17,192 samples, 39.33%) native_queued_spin_lock_slowpath sys_sched_setaffinity (92 samples, 0.21%) __radix_tree_preload (6 samples, 0.01%) [unknown] (120 samples, 0.27%) lov_io_iter_init (73 samples, 0.17%) wait_for_completion (92 samples, 0.21%) [libopen-pal.so.20.10.5] (13 samples, 0.03%) [libmlx4-rdmav2.so] (38 samples, 0.09%) cl_lock_init (9 samples, 0.02%) iov_iter_copy_from_user_atomic (10 samples, 0.02%) cl_lock_enqueue (7 samples, 0.02%) hrtimer_interrupt (28 samples, 0.06%) set_cpus_allowed_ptr (92 samples, 0.21%) btl_openib_component_progress (22 samples, 0.05%) lov_sub_get (5 samples, 0.01%) ll_merge_attr (11 samples, 0.03%) [unknown] (13 samples, 0.03%) ll_file_io_generic (43,002 samples, 98.38%) ll_file_io_generic [unknown] (43,109 samples, 98.63%) [unknown] ktime_get_ts64 (35 samples, 0.08%) cl_io_top (9 samples, 0.02%) iov_iter_fault_in_readable (14 samples, 0.03%) cl_page_list_add (8 samples, 0.02%) set_page_dirty (5 samples, 0.01%) add_to_page_cache_lru (18,827 samples, 43.07%) add_to_page_cache_lru osc_cache_writeback_range (218 samples, 0.50%) radix_tree_preload (22 samples, 0.05%) opal_progress (5 samples, 0.01%) delete_from_page_cache (7 samples, 0.02%) cl_page_find (1,118 samples, 2.56%) cl.. osc_update_next_shrink (49 samples, 0.11%) cl_page_assume (33 samples, 0.08%) cl_io_start (218 samples, 0.50%) __wake_up_bit (10 samples, 0.02%) __list_del_entry (4 samples, 0.01%) cl_page_put (8 samples, 0.02%) __hrtimer_run_queues (27 samples, 0.06%) cl_page_make_ready (214 samples, 0.49%) cl_io_iter_init (71 samples, 0.16%) finish_task_switch (92 samples, 0.21%) unlock_page (28 samples, 0.06%) cl_page_slice_add (39 samples, 0.09%) __generic_file_write_iter (22,793 samples, 52.15%) __generic_file_write_iter vvp_io_write_start (42,873 samples, 98.09%) vvp_io_write_start discard_pagevec (10 samples, 0.02%) read_tsc (6 samples, 0.01%) osc_lru_alloc (117 samples, 0.27%) _raw_spin_lock_irqsave (17 samples, 0.04%) osc_enter_cache_try.constprop.36 (4 samples, 0.01%) __inc_zone_page_state (9 samples, 0.02%) task_tick_fair (11 samples, 0.03%) osc_prep_async_page (12 samples, 0.03%) cl_io_top (5 samples, 0.01%) ll_writepages (218 samples, 0.50%) _raw_qspin_lock_irq (7 samples, 0.02%) lov_io_end (7 samples, 0.02%) remove_commit_idr_uobject (5 samples, 0.01%) opal_timer_linux_get_cycles_sys_timer (7 samples, 0.02%) __kprobes_text_start (4 samples, 0.01%) __kmalloc (71 samples, 0.16%) queued_spin_lock_slowpath (17,192 samples, 39.33%) queued_spin_lock_slowpath __slab_alloc (50 samples, 0.11%) queued_spin_lock_slowpath (17 samples, 0.04%) do_fsync (292 samples, 0.67%) load_elf_binary (4 samples, 0.01%) next_zones_zonelist (12 samples, 0.03%) __mark_inode_dirty (72 samples, 0.16%) apic_timer_interrupt (43 samples, 0.10%) cl_io_loop (42,989 samples, 98.35%) cl_io_loop setup_new_exec (4 samples, 0.01%) cl_page_put (186 samples, 0.43%) __generic_file_aio_write (22,793 samples, 52.15%) __generic_file_aio_write prepare_to_wait (7 samples, 0.02%) lov_io_call.isra.7 (218 samples, 0.50%) osc_extent_find (6 samples, 0.01%) cl_page_slice_add (14 samples, 0.03%) __list_add (4 samples, 0.01%) generic_file_buffered_write (22,684 samples, 51.90%) generic_file_buffered_write sched_setaffinity@@GLIBC_2.3.4 (93 samples, 0.21%) __test_set_page_writeback (209 samples, 0.48%) cl_pagevec_put (183 samples, 0.42%) [libpthread-2.17.so] (6 samples, 0.01%) lov_io_end_wrapper (6 samples, 0.01%) ompi_coll_libnbc_progress (6 samples, 0.01%) call_softirq (7 samples, 0.02%) __mod_zone_page_state (10 samples, 0.02%) __set_page_dirty_nobuffers (18,267 samples, 41.79%) __set_page_dirty_nobuffers radix_tree_lookup_slot (161 samples, 0.37%) __slab_alloc (4 samples, 0.01%) _raw_spin_lock_irqsave (6 samples, 0.01%) osc_io_iter_init (58 samples, 0.13%) account_page_dirtied (128 samples, 0.29%) cl_io_end (6 samples, 0.01%) vfs_write (5 samples, 0.01%) cl_page_slice_add (6 samples, 0.01%) ior (43,585 samples, 99.71%) ior __mem_cgroup_try_charge (34 samples, 0.08%) stripe_width (10 samples, 0.02%) iov_iter_fault_in_readable (43 samples, 0.10%) alloc_pages_current (327 samples, 0.75%) native_queued_spin_lock_slowpath (48 samples, 0.11%) sys_fsync (292 samples, 0.67%) mark_page_accessed (242 samples, 0.55%) _raw_qspin_lock (49 samples, 0.11%) __inc_zone_state (8 samples, 0.02%) policy_zonelist (6 samples, 0.01%) radix_tree_descend (12 samples, 0.03%) __kprobes_text_start (30 samples, 0.07%) lprocfs_counter_add (4 samples, 0.01%) lookup_page_cgroup (6 samples, 0.01%) lov_lock_enqueue (8 samples, 0.02%) scheduler_tick (10 samples, 0.02%) mca_btl_vader_component_progress (43 samples, 0.10%) ll_file_aio_write (43,004 samples, 98.38%) ll_file_aio_write __find_get_page (7 samples, 0.02%) grab_cache_page_nowait (19,377 samples, 44.33%) grab_cache_page_nowait lov_stripe_offset (10 samples, 0.02%) cl_lock_enqueue (8 samples, 0.02%) osc_page_cache_add (891 samples, 2.04%) o.. smp_apic_timer_interrupt (36 samples, 0.08%) _rdma_remove_commit_uobject (5 samples, 0.01%) __hrtimer_run_queues (28 samples, 0.06%) mem_cgroup_update_lru_size (4 samples, 0.01%) osc_io_fsync_start (25 samples, 0.06%) osc_io_write_iter_init (71 samples, 0.16%) ib_uverbs_write (5 samples, 0.01%) cl_page_disown0 (76 samples, 0.17%) _raw_spin_lock_irqsave (17,538 samples, 40.12%) _raw_spin_lock_irqsave task_work_run (9 samples, 0.02%) native_queued_spin_lock_slowpath (54 samples, 0.12%) set_page_dirty (18,367 samples, 42.02%) set_page_dirty smp_apic_timer_interrupt (4 samples, 0.01%) new_slab (32 samples, 0.07%) vvp_page_make_ready (212 samples, 0.49%) cl_page_put (7 samples, 0.02%) perf (4 samples, 0.01%) tick_sched_timer (23 samples, 0.05%) lov_lock_cancel (6 samples, 0.01%) __list_add (19 samples, 0.04%) _raw_qspin_lock (5 samples, 0.01%) cl_page_alloc (840 samples, 1.92%) c.. _raw_read_lock (8 samples, 0.02%) ____fput (9 samples, 0.02%) pagevec_lru_move_fn (119 samples, 0.27%) apic_timer_interrupt (38 samples, 0.09%) __kprobes_text_start (6 samples, 0.01%) osc_io_unplug0 (218 samples, 0.50%) ktime_get_seconds (44 samples, 0.10%) __inc_zone_page_state (19 samples, 0.04%) tick_sched_handle (19 samples, 0.04%) __cond_resched (92 samples, 0.21%) lov_io_rw_iter_init (73 samples, 0.17%) cfs_cdebug_show.part.1.constprop.38 (13 samples, 0.03%) page_mapping (12 samples, 0.03%) osc_extent_is_overlapped (5 samples, 0.01%) osc_extent_make_ready (215 samples, 0.49%) cl_page_is_vmlocked (10 samples, 0.02%) cl_io_start (218 samples, 0.50%) lov_page_init (509 samples, 1.16%) __x86_indirect_thunk_rax (4 samples, 0.01%) lov_lock_sub_init (4 samples, 0.01%) __do_softirq (5 samples, 0.01%) lov_page_init_composite (490 samples, 1.12%) smp_apic_timer_interrupt (41 samples, 0.09%) ib_uverbs_dereg_mr (5 samples, 0.01%) __test_set_page_writeback (22 samples, 0.05%) mpirun (121 samples, 0.28%) filemap_write_and_wait_range (267 samples, 0.61%) queued_spin_lock_slowpath (192 samples, 0.44%) cl_object_attr_get (4 samples, 0.01%) __radix_tree_create (174 samples, 0.40%) balance_dirty_pages_ratelimited (75 samples, 0.17%) clockevents_program_event (6 samples, 0.01%) native_queued_spin_lock_slowpath (57 samples, 0.13%) get_partial_node.isra.49 (4 samples, 0.01%) native_queued_spin_lock_slowpath (58 samples, 0.13%) lprocfs_counter_add (52 samples, 0.12%) system_call_fastpath (292 samples, 0.67%) __inc_zone_page_state (8 samples, 0.02%) lprocfs_stats_unlock (5 samples, 0.01%) _cond_resched (4 samples, 0.01%) page_waitqueue (8 samples, 0.02%) lov_io_start (218 samples, 0.50%) osc_cache_writeback_range (25 samples, 0.06%) list_del (9 samples, 0.02%) __add_to_page_cache_locked (18,216 samples, 41.67%) __add_to_page_cache_locked __mem_cgroup_commit_charge (243 samples, 0.56%) cl_object_attr_unlock (8 samples, 0.02%) radix_tree_descend (11 samples, 0.03%) cl_io_lock (22 samples, 0.05%) io_schedule_timeout (25 samples, 0.06%) lu_context_key_get (4 samples, 0.01%) opal_progress (34 samples, 0.08%) do_writepages (218 samples, 0.50%) _raw_spin_lock_irqsave (4 samples, 0.01%) _raw_qspin_lock (58 samples, 0.13%) cl_page_discard (8 samples, 0.02%) do_execve_common.isra.24 (4 samples, 0.01%) hrtimer_interrupt (34 samples, 0.08%) osc_page_init (286 samples, 0.65%) do_softirq (8 samples, 0.02%) queued_spin_lock_slowpath (6 samples, 0.01%) system_call_fastpath (4 samples, 0.01%) perf_pmu_enable (8 samples, 0.02%) update_process_times (19 samples, 0.04%) osc_queue_async_io (637 samples, 1.46%) cl_io_top (10 samples, 0.02%) mlx4_ib_dereg_mr (5 samples, 0.01%) cl_page_is_owned (9 samples, 0.02%) cl_io_rw_init (4 samples, 0.01%) irq_exit (8 samples, 0.02%) page_mapping (14 samples, 0.03%) __osc_unreserve_grant (10 samples, 0.02%) _cond_resched (4 samples, 0.01%) __zone_watermark_ok (5 samples, 0.01%) osc_wake_cache_waiters (12 samples, 0.03%) queued_spin_lock_slowpath (54 samples, 0.12%) local_apic_timer_interrupt (28 samples, 0.06%) cl_lock_release (9 samples, 0.02%) cl_io_end (7 samples, 0.02%) osc_lru_reserve (12 samples, 0.03%) osc_queue_async_io (4 samples, 0.01%) _cond_resched (92 samples, 0.21%) system_call_fastpath (5 samples, 0.01%) write_commit_callback (18,524 samples, 42.38%) write_commit_callback cl_pagevec_put (7 samples, 0.02%) __percpu_counter_add (22 samples, 0.05%) lov_lock_init_composite (5 samples, 0.01%) ll_write_begin (20,930 samples, 47.88%) ll_write_begin ___slab_alloc (48 samples, 0.11%) deactivate_task (6 samples, 0.01%) __zone_watermark_ok (4 samples, 0.01%) native_apic_mem_write (9 samples, 0.02%) osc_unreserve_grant (7 samples, 0.02%) cl_offset (8 samples, 0.02%) tick_sched_handle (20 samples, 0.05%) release_pages (13 samples, 0.03%) vvp_page_make_ready (23 samples, 0.05%) cl_object_attr_lock (71 samples, 0.16%) __alloc_pages_nodemask (10 samples, 0.02%) __list_add (12 samples, 0.03%) put_page (19 samples, 0.04%) alloc_pages_current (11 samples, 0.03%) _raw_qspin_lock_irq (17,463 samples, 39.95%) _raw_qspin_lock_irq call_rcu_sched (5 samples, 0.01%) mca_btl_vader_component_progress (11 samples, 0.03%) cl_page_owner_clear.isra.14 (5 samples, 0.01%) system_call_fastpath (43,009 samples, 98.40%) system_call_fastpath page_mapping (16 samples, 0.04%) queued_spin_lock_slowpath (57 samples, 0.13%) rdma_remove_commit_uobject (5 samples, 0.01%) vvp_page_is_vmlocked (6 samples, 0.01%) __inc_zone_state (4 samples, 0.01%) cl_lock_request (17 samples, 0.04%) native_queued_spin_lock_slowpath (192 samples, 0.44%) __alloc_pages_nodemask (300 samples, 0.69%) cl_io_iter_init (73 samples, 0.17%) __find_get_page (165 samples, 0.38%) cl_sync_file_range (218 samples, 0.50%) lprocfs_stats_lock (16 samples, 0.04%) iov_iter_advance (5 samples, 0.01%) vvp_io_write_commit (20,075 samples, 45.93%) vvp_io_write_commit cl_page_is_owned (4 samples, 0.01%) __schedule (18 samples, 0.04%) __list_del_entry (7 samples, 0.02%) [libmlx4-rdmav2.so] (10 samples, 0.02%) cl_object_top (10 samples, 0.02%) _raw_qspin_lock (54 samples, 0.12%) radix_tree_tag_set (265 samples, 0.61%) intel_pmu_handle_irq (6 samples, 0.01%) next_extent (5 samples, 0.01%) __rmqueue (70 samples, 0.16%) system_call_fastpath (4 samples, 0.01%) task_tick_fair (6 samples, 0.01%) _raw_spin_unlock_irqrestore (4 samples, 0.01%) cl_io_start (25 samples, 0.06%) lov_stripe_number (16 samples, 0.04%) ll_write_end (78 samples, 0.18%) intel_bts_enable_local (4 samples, 0.01%) __page_cache_alloc (342 samples, 0.78%) __filemap_fdatawrite_range (218 samples, 0.50%) osc_attr_update (8 samples, 0.02%) cl_io_loop (25 samples, 0.06%) _raw_qspin_lock (101 samples, 0.23%) lov_io_commit_async (20,043 samples, 45.85%) lov_io_commit_async cl_io_unlock (9 samples, 0.02%) memcg_check_events (14 samples, 0.03%) apic_timer_interrupt (4 samples, 0.01%) queued_spin_lock_slowpath (17,339 samples, 39.67%) queued_spin_lock_slowpath osc_lru_shrink (11 samples, 0.03%) cl_object_get (195 samples, 0.45%) __radix_tree_lookup (146 samples, 0.33%) ll_file_write (43,005 samples, 98.39%) ll_file_write _raw_qspin_lock (21 samples, 0.05%) vfs_write (43,008 samples, 98.39%) vfs_write __inc_zone_state (5 samples, 0.01%) osc_extent_make_ready (23 samples, 0.05%) osc_page_transfer_get.isra.18 (26 samples, 0.06%) radix_tree_descend (59 samples, 0.13%) system_call_fastpath (92 samples, 0.21%) __list_add (16 samples, 0.04%) cl_object_top (15 samples, 0.03%) stub_execve (4 samples, 0.01%) tick_program_event (6 samples, 0.01%) scheduler_tick (15 samples, 0.03%) cl_object_attr_update (51 samples, 0.12%) uverbs_free_mr (5 samples, 0.01%) lov_io_start (25 samples, 0.06%) osc_io_unplug0 (25 samples, 0.06%) cl_page_disown (91 samples, 0.21%) mem_cgroup_page_lruvec (12 samples, 0.03%) __inc_zone_state (13 samples, 0.03%) pagevec_lookup_tag (4 samples, 0.01%) lprocfs_stats_lock (8 samples, 0.02%) page_cache_tree_insert (239 samples, 0.55%) cl_page_get_trust (20 samples, 0.05%) kmalloc_slab (7 samples, 0.02%) __fput (8 samples, 0.02%) update_process_times (19 samples, 0.04%) stop_one_cpu (92 samples, 0.21%) cl_io_commit_async (20,041 samples, 45.85%) cl_io_commit_async lov_io_call.isra.7 (7 samples, 0.02%) get_page_from_freelist (9 samples, 0.02%) radix_tree_descend (19 samples, 0.04%) _raw_spin_lock_irqsave (194 samples, 0.44%) radix_tree_maybe_preload (4 samples, 0.01%) cl_object_top (6 samples, 0.01%) poll_device (11 samples, 0.03%) sys_execve (4 samples, 0.01%) vvp_page_discard (8 samples, 0.02%) native_queued_spin_lock_slowpath (17 samples, 0.04%) __filemap_fdatawait_range (49 samples, 0.11%) cl_lock_cancel (5 samples, 0.01%) queued_spin_lock_slowpath (5 samples, 0.01%) cl_io_top (5 samples, 0.01%) osc_lock_cancel (5 samples, 0.01%) cl_io_init0.isra.15 (4 samples, 0.01%) __pagevec_lru_add_fn (70 samples, 0.16%) ll_fsync (292 samples, 0.67%) int_signal (9 samples, 0.02%) opal_hwloc1112_hwloc_linux_set_tid_cpubind (93 samples, 0.21%) get_task_policy.part.26 (6 samples, 0.01%) __memset (7 samples, 0.02%) ldlm_lock_decref (4 samples, 0.01%) sys_lseek (4 samples, 0.01%) io_schedule (25 samples, 0.06%) schedule (18 samples, 0.04%) sched_setaffinity (92 samples, 0.21%) lov_io_layout_at (47 samples, 0.11%) _raw_qspin_lock (179 samples, 0.41%) filemap_fdatawait_range (49 samples, 0.11%) __x86_indirect_thunk_rax (5 samples, 0.01%) vvp_page_init (86 samples, 0.20%) page_waitqueue (14 samples, 0.03%) native_queued_spin_lock_slowpath (6 samples, 0.01%) poll_device (39 samples, 0.09%) __list_add (9 samples, 0.02%) llseek (5 samples, 0.01%) cl_io_loop (218 samples, 0.50%) __call_rcu (5 samples, 0.01%) ll_cl_find (31 samples, 0.07%) _raw_qspin_lock (5 samples, 0.01%) cl_io_start (42,876 samples, 98.09%) cl_io_start cl_page_disown (6 samples, 0.01%) copy_user_enhanced_fast_string (1,158 samples, 2.65%) co.. sys_write (43,009 samples, 98.40%) sys_write finish_task_switch (5 samples, 0.01%) osc_page_touch_at (364 samples, 0.83%) all (43,710 samples, 100%) radix_tree_lookup_slot (5 samples, 0.01%) __list_del_entry (6 samples, 0.01%) queued_spin_lock_slowpath (58 samples, 0.13%) cl_io_commit_async (20,044 samples, 45.86%) cl_io_commit_async __perf_event_task_sched_in (37 samples, 0.08%) get_pageblock_flags_group (12 samples, 0.03%) queued_spin_lock_slowpath (48 samples, 0.11%) radix_tree_maybe_preload (9 samples, 0.02%) osc_io_end (5 samples, 0.01%) ib_dereg_mr (5 samples, 0.01%) __wait_on_bit (40 samples, 0.09%) sys_write (5 samples, 0.01%) mem_cgroup_cache_charge (311 samples, 0.71%) schedule_timeout (18 samples, 0.04%) vvp_page_disown (40 samples, 0.09%) wait_on_page_bit (42 samples, 0.10%) cl_object_top (100 samples, 0.23%) sched_clock (5 samples, 0.01%) __lru_cache_add (147 samples, 0.34%) osc_extent_release (4 samples, 0.01%) osc_unreserve_grant (74 samples, 0.17%) osc_extent_sanity_check0 (41 samples, 0.09%) local_apic_timer_interrupt (34 samples, 0.08%) __radix_tree_insert (84 samples, 0.19%) __inc_zone_state (8 samples, 0.02%) _raw_spin_unlock_irqrestore (13 samples, 0.03%) _raw_qspin_lock (191 samples, 0.44%) __list_del_entry (4 samples, 0.01%) get_page_from_freelist (235 samples, 0.54%) list_del (9 samples, 0.02%) osc_lock_enqueue (7 samples, 0.02%) --Apple-Mail=_20FAA3B9-CB10-4AA6-8146-ACB6B9D72F27-- --Apple-Mail=_EA59EFD2-F69E-4324-A987-ABC5C770B932 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQIzBAEBCAAdFiEEDb73u6ZejP5ZMprvcqXauRfMH+AFAl0CvIUACgkQcqXauRfM H+BypA//c1xErye5WL47N3Lt4Yc8bvp8uJt40eOdt0vbq2xdE1uDgvlzC1NrseeX eflTcMvd9Of0gIspZti8YUhTuG0ha+MF15wGsCwbMbwv6vPWIuUY7Lq7MnELGntD rryknyg5Em6jFdeopVsucAbEJlLqZ1PYcEBhnYATFC/t3GAOxsNF7zXFQiY/EZ05 GK79t6VCKchgJ5Sa4HaQvMq5mPGWgnAWe8qZmlzKvkCw/9jxT0CEtJ4b5yWXjgnM ysyE6h5GSOVMc1j7fNMXiyYJA8fb13lseLjyUtNBQsLl1AYYTCRmFOpuue9DxA5R wa8o5JHBcb+VgpJKAXMRnpfk943DJJA56M//1Q4BaR5s0lVF5YZeGLX0RXIlrSaB CfDn+dpbZOV9huavCz9SmkYLeAygSii80+b4LDw7cIRlO6sf7w0CMOdwV+Jt7MZI BBHyq6RQT9YDOFIvhPha+bWLiZ6QXg9wyD10E+Vok5RYHXvNh2bfzCWIJY5+YXlI AKilr7UXDOPNWnqnFOjIAd3+I/k9XNMa2xiyPSv98IpfBRji0p9Jn7Xwnm0rHIJC Bfqwuv2Jbs5dA0SzJNA1UY92d03iltU+LM7UjekzoRRiKSTndtpw0+rO0LRVThs6 EYrWZgbQqO/5BouFKXNVPb+VcNfp3ekjh3zaVEp703+opdZlp4A= =KMtb -----END PGP SIGNATURE----- --Apple-Mail=_EA59EFD2-F69E-4324-A987-ABC5C770B932--