From: Jens Axboe <axboe@kernel.dk>
To: Ayush Jain <ayush.jain3@amd.com>,
broonie@kernel.org,
Linux Next Mailing List <linux-next@vger.kernel.org>
Cc: Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
Santosh.Shukla@amd.com, Kalpana.Shetty@amd.com,
Narasimhan V <Narasimhan.V@amd.com>,
sfr@canb.auug.org.au, David Howells <dhowells@redhat.com>
Subject: Re: linux-next: Tree for Apr 13
Date: Sat, 15 Apr 2023 14:37:04 -0600 [thread overview]
Message-ID: <e8bc6158-86b2-18e5-efea-c165e2a3f196@kernel.dk> (raw)
In-Reply-To: <3c4b9025-2667-ca8c-7427-502068b8168e@amd.com>
Adding David, who did the splice changes. Always a good idea to CC the
person(s) involved.
On 4/14/23 1:25 AM, Ayush Jain wrote:
> Hello,
>
> On 4/14/2023 10:41 AM, Jain, Ayush wrote:
>> Hello Broonie,
>>
>> When Running fio-test on latest linux-next tree, I noticed that test hung indefinitely, Going back I see that this problem exists since
>> next-20230316 release, After bisecting I landed on the following merge commit by Jens.
>>
>> Commit 097d3ca138f9 ("Merge branch 'for-6.4/splice' into for-next")
>>
>> Running perf I see following trace and call-stack for fio:
>>
>> Overhead Command Shared Object Symbol
>> 25.08% fio [kernel.vmlinux] [k] copy_user_generic_string
>> copy_user_generic_string
>> __do_splice
>> __x64_sys_splice
>> do_syscall_64
>> entry_SYSCALL_64_after_hwframe
>> splice
>> 0x1c44be0
>> ...
>>
>> On a good kernel I see the following perf trace:
>>
>> Overhead Command Shared Object Symbol
>> 49.93% fio fio [.] fio_crc32
>> 7.23% fio fio [.] clock_thread_fn
>> 2.10% fio [kernel.vmlinux] [k] clear_page_rep
>> 1.55% fio fio [.] __fill_random_buf
>> 1.35% fio [kernel.vmlinux] [k] loop_queue_rq
>> 1.05% fio [kernel.vmlinux] [k] copy_user_generic_string
>> ...
>>
>> I see some splice changes being added as the part of merge
>> https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/log/?h=for-6.4/splice
>>
>> I observe this problem on 3 EPYC system(Zen1,3,4), with the following disk architecture
>>
>> Zen1: nvme0n1 931.5G Samsung SSD 970 EVO Plus 1TB
>> Zen4: nvme0n1 232.9G Samsung SSD 960 EVO 250GB
>>
>> I am running fio as follows:
>>
>> $fio fio-simple.job --filename=/dev/test_vg/test_lv
>>
>> where test_lv is mounted as follows:
>>
>> NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
>> loop0 7:0 0 12.2G 0 loop
>> └─md127 9:127 0 12.2G 0 raid0
>> └─test_vg-test_lv 253:3 0 5.5G 0 lvm
>>
>> You can find fio-simple.job at
>> https://github.com/avocado-framework-tests/avocado-misc-tests/blob/master/io/disk/fiotest.py.data/fio-simple.job
>>
>> Fio Version: fio-3.34-25-g07ed
>>
> Also adding to these observations
>
> -If we create a filesystem on the raw disk -- Test completes with a Pass
>
> -If there is no Filesystem on the raw disk(loop, nvme) -- Test hangs with the provided trace
>
>>
>> Regards
>> Ayush Jain
>> > On 4/13/2023 11:55 PM, broonie@kernel.org wrote:
>>> Hi all,
>>>
>>> Changes since 20230411:
>>>
>>> The sh tree gained a conflict with the mm-unstable tree.
>>>
>>> The erofs tree gained a conflict with the vfs-idmapping tree.
>>>
>>> The ext4 tree gained multiple conflicts with the mm-stable tree.
>>>
>>> The net-next tree gained a conflict with the origin tree.
>>>
>>> The bpf-next tree gained a conflict with the net-net tree.
>>>
>>> Non-merge commits (relative to Linus' tree): 10382
>>> 11028 files changed, 580165 insertions(+), 249563 deletions(-)
>>>
>>> ----------------------------------------------------------------------------
>>>
>>> I have created today's linux-next tree at
>>> git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
>>> (patches at http://www.kernel.org/pub/linux/kernel/next/ ). If you
>>> are tracking the linux-next tree using git, you should not use "git pull"
>>> to do so as that will try to merge the new linux-next release with the
>>> old one. You should use "git fetch" and checkout or reset to the new
>>> master.
>>>
>>> You can see which trees have been included by looking in the Next/Trees
>>> file in the source. There is also the merge.log file in the Next
>>> directory. Between each merge, the tree was built with a ppc64_defconfig
>>> for powerpc, an allmodconfig for x86_64, a multi_v7_defconfig for arm
>>> and a native build of tools/perf. After the final fixups (if any), I do
>>> an x86_64 modules_install followed by builds for x86_64 allnoconfig,
>>> powerpc allnoconfig (32 and 64 bit), ppc44x_defconfig, allyesconfig
>>> and pseries_le_defconfig and i386, arm64, s390, sparc and sparc64
>>> defconfig and htmldocs. And finally, a simple boot test of the powerpc
>>> pseries_le_defconfig kernel in qemu (with and without kvm enabled).
>>>
>>> Below is a summary of the state of the merge.
>>>
>>> I am currently merging 357 trees (counting Linus' and 102 trees of bug
>>> fix patches pending for the current merge release).
>>>
>>> Stats about the size of the tree over time can be seen at
>>> http://neuling.org/linux-next-size.html .
>>>
>>> Status of my local build tests will be at
>>> http://kisskb.ellerman.id.au/linux-next . If maintainers want to give
>>> advice about cross compilers/configs that work, we are always open to add
>>> more builds.
>>>
>>> Thanks to Randy Dunlap for doing many randconfig builds. And to Paul
>>> Gortmaker for triage and bug fixes.
>>
>
> Regards,
> Ayush Jain
--
Jens Axboe
next prev parent reply other threads:[~2023-04-15 20:37 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-04-13 18:25 linux-next: Tree for Apr 13 broonie
2023-04-14 5:11 ` Jain, Ayush
2023-04-14 7:25 ` Ayush Jain
2023-04-15 20:37 ` Jens Axboe [this message]
2023-04-17 12:28 ` David Howells
2023-04-17 13:45 ` David Howells
2023-04-17 14:06 ` David Howells
-- strict thread matches above, loose matches on Subject: below --
2022-04-13 5:25 Stephen Rothwell
2021-04-13 10:04 Stephen Rothwell
2020-04-13 2:18 Stephen Rothwell
2018-04-13 3:13 Stephen Rothwell
2017-04-13 5:38 Stephen Rothwell
2016-04-13 5:05 Stephen Rothwell
2015-04-13 12:12 Stephen Rothwell
2012-04-13 5:31 Stephen Rothwell
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e8bc6158-86b2-18e5-efea-c165e2a3f196@kernel.dk \
--to=axboe@kernel.dk \
--cc=Kalpana.Shetty@amd.com \
--cc=Narasimhan.V@amd.com \
--cc=Santosh.Shukla@amd.com \
--cc=ayush.jain3@amd.com \
--cc=broonie@kernel.org \
--cc=dhowells@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-next@vger.kernel.org \
--cc=sfr@canb.auug.org.au \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).