From: Marek Szyprowski <m.szyprowski@samsung.com>
To: Ming Lei <ming.lei@redhat.com>
Cc: Jens Axboe <axboe@kernel.dk>,
linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-mm@kvack.org, Theodore Ts'o <tytso@mit.edu>,
Omar Sandoval <osandov@fb.com>, Sagi Grimberg <sagi@grimberg.me>,
Dave Chinner <dchinner@redhat.com>,
Kent Overstreet <kent.overstreet@gmail.com>,
Mike Snitzer <snitzer@redhat.com>,
dm-devel@redhat.com, Alexander Viro <viro@zeniv.linux.org.uk>,
linux-fsdevel@vger.kernel.org, linux-raid@vger.kernel.org,
David Sterba <dsterba@suse.com>,
linux-btrfs@vger.kernel.org,
"Darrick J . Wong" <darrick.wong@oracle.com>,
linux-xfs@vger.kernel.org, Gao Xiang <gaoxiang25@huawei.com>,
Christoph Hellwig <hch@lst.de>,
linux-ext4@vger.kernel.org, Coly Li <colyli@suse.de>,
linux-bcache@vger.kernel.org, Boaz Harrosh <ooo@electrozaur.com>,
Bob Peterson <rpeterso@redhat.com>,
cluster-devel@redhat.com, Ulf Hansson <ulf.hansson@linaro.org>,
"linux-mmc@vger.kernel.org" <linux-mmc@vger.kernel.org>,
'Linux Samsung SOC' <linux-samsung-soc@vger.kernel.org>,
Krzysztof Kozlowski <krzk@kernel.org>,
Adrian Hunter <adrian.hunter@intel.com>,
Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Subject: Re: [PATCH V15 14/18] block: enable multipage bvecs
Date: Thu, 21 Feb 2019 12:42:22 +0100 [thread overview]
Message-ID: <104e10f8-7aea-54ce-4fea-951768fa81c8@samsung.com> (raw)
In-Reply-To: <20190221103856.GC12448@ming.t460p>
Hi Ming,
On 2019-02-21 11:38, Ming Lei wrote:
> On Thu, Feb 21, 2019 at 11:22:39AM +0100, Marek Szyprowski wrote:
>> On 2019-02-21 11:16, Ming Lei wrote:
>>> On Thu, Feb 21, 2019 at 11:08:19AM +0100, Marek Szyprowski wrote:
>>>> On 2019-02-21 10:57, Ming Lei wrote:
>>>>> On Thu, Feb 21, 2019 at 09:42:59AM +0100, Marek Szyprowski wrote:
>>>>>> On 2019-02-15 12:13, Ming Lei wrote:
>>>>>>> This patch pulls the trigger for multi-page bvecs.
>>>>>>>
>>>>>>> Reviewed-by: Omar Sandoval <osandov@fb.com>
>>>>>>> Signed-off-by: Ming Lei <ming.lei@redhat.com>
>>>>>> Since Linux next-20190218 I've observed problems with block layer on one
>>>>>> of my test devices (Odroid U3 with EXT4 rootfs on SD card). Bisecting
>>>>>> this issue led me to this change. This is also the first linux-next
>>>>>> release with this change merged. The issue is fully reproducible and can
>>>>>> be observed in the following kernel log:
>>>>>>
>>>>>> sdhci: Secure Digital Host Controller Interface driver
>>>>>> sdhci: Copyright(c) Pierre Ossman
>>>>>> s3c-sdhci 12530000.sdhci: clock source 2: mmc_busclk.2 (100000000 Hz)
>>>>>> s3c-sdhci 12530000.sdhci: Got CD GPIO
>>>>>> mmc0: SDHCI controller on samsung-hsmmc [12530000.sdhci] using ADMA
>>>>>> mmc0: new high speed SDHC card at address aaaa
>>>>>> mmcblk0: mmc0:aaaa SL16G 14.8 GiB
>>>>>>
>>>>>> ...
>>>>>>
>>>>>> EXT4-fs (mmcblk0p2): INFO: recovery required on readonly filesystem
>>>>>> EXT4-fs (mmcblk0p2): write access will be enabled during recovery
>>>>>> EXT4-fs (mmcblk0p2): recovery complete
>>>>>> EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
>>>>>> VFS: Mounted root (ext4 filesystem) readonly on device 179:2.
>>>>>> devtmpfs: mounted
>>>>>> Freeing unused kernel memory: 1024K
>>>>>> hub 1-3:1.0: USB hub found
>>>>>> Run /sbin/init as init process
>>>>>> hub 1-3:1.0: 3 ports detected
>>>>>> *** stack smashing detected ***: <unknown> terminated
>>>>>> Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004
>>>>>> CPU: 1 PID: 1 Comm: init Not tainted 5.0.0-rc6-next-20190218 #1546
>>>>>> Hardware name: SAMSUNG EXYNOS (Flattened Device Tree)
>>>>>> [<c01118d0>] (unwind_backtrace) from [<c010d794>] (show_stack+0x10/0x14)
>>>>>> [<c010d794>] (show_stack) from [<c09ff8a4>] (dump_stack+0x90/0xc8)
>>>>>> [<c09ff8a4>] (dump_stack) from [<c0125944>] (panic+0xfc/0x304)
>>>>>> [<c0125944>] (panic) from [<c012bc98>] (do_exit+0xabc/0xc6c)
>>>>>> [<c012bc98>] (do_exit) from [<c012c100>] (do_group_exit+0x3c/0xbc)
>>>>>> [<c012c100>] (do_group_exit) from [<c0138908>] (get_signal+0x130/0xbf4)
>>>>>> [<c0138908>] (get_signal) from [<c010c7a0>] (do_work_pending+0x130/0x618)
>>>>>> [<c010c7a0>] (do_work_pending) from [<c0101034>]
>>>>>> (slow_work_pending+0xc/0x20)
>>>>>> Exception stack(0xe88c3fb0 to 0xe88c3ff8)
>>>>>> 3fa0: 00000000 bea7787c 00000005
>>>>>> b6e8d0b8
>>>>>> 3fc0: bea77a18 b6f92010 b6e8d0b8 00000001 b6e8d0c8 00000001 b6e8c000
>>>>>> bea77b60
>>>>>> 3fe0: 00000020 bea77998 ffffffff b6d52368 60000050 ffffffff
>>>>>> CPU3: stopping
>>>>>>
>>>>>> I would like to help debugging and fixing this issue, but I don't really
>>>>>> have idea where to start. Here are some more detailed information about
>>>>>> my test system:
>>>>>>
>>>>>> 1. Board: ARM 32bit Samsung Exynos4412-based Odroid U3 (device tree
>>>>>> source: arch/arm/boot/dts/exynos4412-odroidu3.dts)
>>>>>>
>>>>>> 2. Block device: MMC/SDHCI/SDHCI-S3C with SD card
>>>>>> (drivers/mmc/host/sdhci-s3c.c driver, sdhci_2 device node in the device
>>>>>> tree)
>>>>>>
>>>>>> 3. Rootfs: Ext4
>>>>>>
>>>>>> 4. Kernel config: arch/arm/configs/exynos_defconfig
>>>>>>
>>>>>> I can gather more logs if needed, just let me which kernel option to
>>>>>> enable. Reverting this commit on top of next-20190218 as well as current
>>>>>> linux-next (tested with next-20190221) fixes this issue and makes the
>>>>>> system bootable again.
>>>>> Could you test the patch in following link and see if it can make a difference?
>>>>>
>>>>> https://marc.info/?l=linux-aio&m=155070355614541&w=2
>>>> I've tested that patch, but it doesn't make any difference on the test
>>>> system. In the log I see no warning added by it.
>>> I guess it might be related with memory corruption, could you enable the
>>> following debug options and post the dmesg log?
>>>
>>> CONFIG_DEBUG_STACKOVERFLOW=y
>>> CONFIG_KASAN=y
>> It won't be that easy as none of the above options is available on ARM
>> 32bit. I will try to apply some ARM KASAN patches floating on the net
>> and let you know the result.
> Hi Marek,
>
> Could you test the following patch?
Yes. Sadly, no change observed.
Best regards
--
Marek Szyprowski, PhD
Samsung R&D Institute Poland
next prev parent reply other threads:[~2019-02-21 11:42 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-15 11:13 [PATCH V15 00/18] block: support multi-page bvec Ming Lei
2019-02-15 11:13 ` [PATCH V15 01/18] btrfs: look at bi_size for repair decisions Ming Lei
2019-02-15 11:13 ` [PATCH V15 02/18] block: don't use bio->bi_vcnt to figure out segment number Ming Lei
2019-02-15 11:13 ` [PATCH V15 03/18] block: remove bvec_iter_rewind() Ming Lei
2019-02-15 11:13 ` [PATCH V15 04/18] block: introduce multi-page bvec helpers Ming Lei
2019-02-15 11:13 ` [PATCH V15 05/18] block: introduce bio_for_each_bvec() and rq_for_each_bvec() Ming Lei
2019-02-15 11:13 ` [PATCH V15 06/18] block: use bio_for_each_bvec() to compute multi-page bvec count Ming Lei
2019-02-15 11:13 ` [PATCH V15 07/18] block: use bio_for_each_bvec() to map sg Ming Lei
2019-02-15 11:13 ` [PATCH V15 08/18] block: introduce mp_bvec_last_segment() Ming Lei
2019-02-15 11:13 ` [PATCH V15 09/18] fs/buffer.c: use bvec iterator to truncate the bio Ming Lei
2019-02-15 11:13 ` [PATCH V15 10/18] btrfs: use mp_bvec_last_segment to get bio's last page Ming Lei
2019-02-15 11:13 ` [PATCH V15 11/18] block: loop: pass multi-page bvec to iov_iter Ming Lei
2019-02-15 11:13 ` [PATCH V15 12/18] bcache: avoid to use bio_for_each_segment_all() in bch_bio_alloc_pages() Ming Lei
2019-02-15 11:13 ` [PATCH V15 13/18] block: allow bio_for_each_segment_all() to iterate over multi-page bvec Ming Lei
2019-02-15 11:13 ` [PATCH V15 14/18] block: enable multipage bvecs Ming Lei
[not found] ` <CGME20190221084301eucas1p11e8841a62b4b1da3cccca661b6f4c29d@eucas1p1.samsung.com>
2019-02-21 8:42 ` Marek Szyprowski
2019-02-21 9:57 ` Ming Lei
2019-02-21 10:08 ` Marek Szyprowski
2019-02-21 10:16 ` Ming Lei
2019-02-21 10:22 ` Marek Szyprowski
2019-02-21 10:38 ` Ming Lei
2019-02-21 11:42 ` Marek Szyprowski [this message]
2019-02-27 20:47 ` Jon Hunter
2019-02-27 23:29 ` Ming Lei
2019-02-28 7:51 ` Marek Szyprowski
2019-02-28 12:39 ` Jon Hunter
2019-02-15 11:13 ` [PATCH V15 15/18] block: always define BIO_MAX_PAGES as 256 Ming Lei
2019-02-15 11:13 ` [PATCH V15 16/18] block: document usage of bio iterator helpers Ming Lei
2019-02-15 11:13 ` [PATCH V15 17/18] block: kill QUEUE_FLAG_NO_SG_MERGE Ming Lei
2019-02-15 11:13 ` [PATCH V15 18/18] block: kill BLK_MQ_F_SG_MERGE Ming Lei
2019-02-15 14:51 ` [PATCH V15 00/18] block: support multi-page bvec Christoph Hellwig
2019-02-17 13:10 ` Ming Lei
2019-02-15 15:49 ` Jens Axboe
2019-02-15 17:14 ` [dm-devel] " Bart Van Assche
2019-02-15 17:59 ` Jens Axboe
2019-02-17 13:13 ` Ming Lei
2019-02-18 7:49 ` Ming Lei
2019-02-17 13:11 ` Ming Lei
2019-02-19 16:28 ` Bart Van Assche
2019-02-20 1:17 ` Ming Lei
2019-02-20 2:37 ` Bart Van Assche
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=104e10f8-7aea-54ce-4fea-951768fa81c8@samsung.com \
--to=m.szyprowski@samsung.com \
--cc=adrian.hunter@intel.com \
--cc=axboe@kernel.dk \
--cc=b.zolnierkie@samsung.com \
--cc=cluster-devel@redhat.com \
--cc=colyli@suse.de \
--cc=darrick.wong@oracle.com \
--cc=dchinner@redhat.com \
--cc=dm-devel@redhat.com \
--cc=dsterba@suse.com \
--cc=gaoxiang25@huawei.com \
--cc=hch@lst.de \
--cc=kent.overstreet@gmail.com \
--cc=krzk@kernel.org \
--cc=linux-bcache@vger.kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-mmc@vger.kernel.org \
--cc=linux-raid@vger.kernel.org \
--cc=linux-samsung-soc@vger.kernel.org \
--cc=linux-xfs@vger.kernel.org \
--cc=ming.lei@redhat.com \
--cc=ooo@electrozaur.com \
--cc=osandov@fb.com \
--cc=rpeterso@redhat.com \
--cc=sagi@grimberg.me \
--cc=snitzer@redhat.com \
--cc=tytso@mit.edu \
--cc=ulf.hansson@linaro.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).