All of lore.kernel.org
 help / color / mirror / Atom feed
From: Emmanuel Florac <eflorac@intellique.com>
To: linux-xfs@vger.kernel.org
Subject: Re: "interesting" crash : 3.18.44, huge xfs, nfs
Date: Tue, 4 Apr 2017 19:24:11 +0200	[thread overview]
Message-ID: <20170404192411.650fb403@harpe.intellique.com> (raw)
In-Reply-To: <20170404150202.6c7d032c@harpe.intellique.com>

[-- Attachment #1: Type: text/plain, Size: 7432 bytes --]

Le Tue, 4 Apr 2017 15:02:02 +0200
Emmanuel Florac <eflorac@intellique.com> écrivait:

> Hi, 
> here is an interesting crash dump. I've never seen this function
> called before : "Fixing recursive fault but reboot is needed!"
> 
> The system is frozen and must be hard rebooted. The filesystem is
> humongous: about 400 TB.
> 
> Running is plain vanilla 3.18.44 (should upgrade...). I wonder if the
> bug is still present though? 
> 
> Context: this is a file server. There was a power failure yesterday,
> so there's probably some corruption hidden somewhere triggering the
> crash.
> 

The machine goes on crashing on disk access... xfs_repair 4.9 is
running now, and after that we'll reboot with a current kernel
(4.4.59). Any advice?

The latest crash trace was still xfs related apparently:


avril 04 16:22:01 Colorstock-01 kernel: BUG: unable to handle kernel NULL pointer dereference at 0000000000000240
avril 04 16:22:01 Colorstock-01 kernel: IP: [<ffffffff811614d5>] iput+0x5/0x190
avril 04 16:22:01 Colorstock-01 kernel: PGD 0
avril 04 16:22:01 Colorstock-01 kernel: Oops: 0000 [#1] SMP
avril 04 16:22:01 Colorstock-01 kernel: Modules linked in: nfsv3 arc4 ecb md4 nfsv4 cifs fscache dm_mod cfg80211 rfkill nfsd auth_rpcgss oid_registry nfs_acl nfs lockd grace sunrpc af_packet bonding joydev evdev x86_pkg_temp_thermal coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd microcode pcspkr myri10ge ast ttm drm_kms_helper drm i2c_algo_bit ixgbe ptp pps_core mdio dca ses enclosure i2c_i801 sg i2c_core lpc_ich mfd_core ipmi_si 8250_fintek ipmi_msghandler rtc_cmos wmi processor thermal_sys acpi_power_meter button md_mod usbhid xhci_pci xhci_hcd ohci_pci ehci_pci ohci_hcd uhci_hcd ehci_hcd uas usb_storage usbcore usb_common fuse ipv6 autofs4 ext4 ext3 xfs reiserfs crc16 jbd2 jbd aacraid ahci libahci ata_generic libata
avril 04 16:22:01 Colorstock-01 kernel: CPU: 19 PID: 822 Comm: kworker/u64:11 Not tainted 3.18.44-storiq64-i7 #1
avril 04 16:22:01 Colorstock-01 kernel: Hardware name: Supermicro Super Server/X10DRD-iNT, BIOS 2.0 12/17/2015
avril 04 16:22:01 Colorstock-01 kernel: Workqueue: writeback bdi_writeback_workfn (flush-253:0)
avril 04 16:22:01 Colorstock-01 kernel: task: ffff88085c19a050 ti: ffff88085c1ac000 task.ti: ffff88085c1ac000
avril 04 16:22:01 Colorstock-01 kernel: RIP: 0010:[<ffffffff811614d5>]  [<ffffffff811614d5>] iput+0x5/0x190
avril 04 16:22:01 Colorstock-01 kernel: RSP: 0018:ffff88085c1af590  EFLAGS: 00010202
avril 04 16:22:01 Colorstock-01 kernel: RAX: 0000000000000000 RBX: ffff880755ea4000 RCX: 000ffffffffe0000
avril 04 16:22:01 Colorstock-01 kernel: RDX: 0000000000000000 RSI: 0000000000000060 RDI: 00000000000001a8
avril 04 16:22:01 Colorstock-01 kernel: RBP: 0000000000000000 R08: ffff88085c1af684 R09: ffff88085c1af750
avril 04 16:22:01 Colorstock-01 kernel: R10: 0000000000000001 R11: 0000000000000000 R12: ffff880652e45c00
avril 04 16:22:01 Colorstock-01 kernel: R13: 0000000000000000 R14: ffff88085c1af801 R15: ffff880755ea4000
avril 04 16:22:01 Colorstock-01 kernel: FS:  0000000000000000(0000) GS:ffff88087f460000(0000) knlGS:0000000000000000
avril 04 16:22:01 Colorstock-01 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
avril 04 16:22:01 Colorstock-01 kernel: CR2: 0000000000000240 CR3: 00000000015e9000 CR4: 00000000003407e0
avril 04 16:22:01 Colorstock-01 kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
avril 04 16:22:01 Colorstock-01 kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
avril 04 16:22:01 Colorstock-01 kernel: Stack:
avril 04 16:22:01 Colorstock-01 kernel:  ffffffffa2c357b6 0000000000001000 ffffffff52e45c60 ffff880652e45c40
avril 04 16:22:01 Colorstock-01 kernel:  ffff88085c1af730 ffff88085c1af730 0000000000000010 0000000000000000
avril 04 16:22:01 Colorstock-01 kernel:  ffffffffa2c0438d ffff88085c8c6c48 ffff88085c1af770 ffff88085c1af7b0
avril 04 16:22:01 Colorstock-01 kernel: Call Trace:
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffffa2c357b6>] ? xfs_filestream_lookup_ag+0x76/0x1b0 [xfs]
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffffa2c0438d>] ? xfs_bmap_btalloc+0x2dd/0x770 [xfs]
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffffa2bfc192>] ? xfs_bmap_search_multi_extents+0xa2/0x120 [xfs]
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffffa2c03a06>] ? xfs_bmap_last_extent+0x56/0x80 [xfs]
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffffa2c03a56>] ? xfs_bmap_isaeof+0x26/0x90 [xfs]
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffffa2c05211>] ? xfs_bmapi_write+0x4b1/0xaa0 [xfs]
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffffa2c3ca61>] ? xfs_iomap_write_allocate+0x131/0x350 [xfs]
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffffa2c283d7>] ? xfs_map_blocks+0x1b7/0x240 [xfs]
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffffa2c291f6>] ? xfs_vm_writepage+0x186/0x5a0 [xfs]
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffff81106ead>] ? __writepage+0xd/0x40
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffff811077af>] ? write_cache_pages+0x1bf/0x430
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffff81106ea0>] ? global_dirtyable_memory+0x50/0x50
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffff81107a58>] ? generic_writepages+0x38/0x50
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffff8116db61>] ? __writeback_single_inode+0x41/0x2b0
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffff811700b8>] ? writeback_sb_inodes+0x1b8/0x470
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffff811703fe>] ? __writeback_inodes_wb+0x8e/0xc0
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffff81170683>] ? wb_writeback+0x253/0x350
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffff81170b8a>] ? bdi_writeback_workfn+0x2ca/0x480
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffff81071b95>] ? process_one_work+0x155/0x440
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffff81072543>] ? worker_thread+0x63/0x490
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffff810724e0>] ? rescuer_thread+0x290/0x290
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffff810769ee>] ? kthread+0xce/0xf0
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffff81076920>] ? kthread_create_on_node+0x180/0x180
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffff81450fd8>] ? ret_from_fork+0x58/0x90
avril 04 16:22:01 Colorstock-01 kernel:  [<ffffffff81076920>] ? kthread_create_on_node+0x180/0x180
avril 04 16:22:01 Colorstock-01 kernel: Code: 08 48 8d b8 00 04 00 00 e8 e9 eb fb ff 84 c0 74 09 65 48 ff 04 25 18 f5 00 00 48 83 c4 08 c3 0f 1f 80 00 00 00 00 48 85 ff 74 3e <f6> 87 98 00 00 00 40 0f 85 fe 00 00 00 41 55 41 54 55 48 8d af
avril 04 16:22:01 Colorstock-01 kernel: RIP  [<ffffffff811614d5>] iput+0x5/0x190
avril 04 16:22:01 Colorstock-01 kernel:  RSP <ffff88085c1af590>
avril 04 16:22:01 Colorstock-01 kernel: CR2: 0000000000000240
avril 04 16:22:01 Colorstock-01 kernel: ---[ end trace a6342d7a0e4dea0f ]---
avril 04 16:22:01 Colorstock-01 kernel: ------------[ cut here ]------------


-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

[-- Attachment #2: Signature digitale OpenPGP --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

  reply	other threads:[~2017-04-04 17:24 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-04-04 13:02 "interesting" crash : 3.18.44, huge xfs, nfs Emmanuel Florac
2017-04-04 17:24 ` Emmanuel Florac [this message]
2017-04-04 17:36   ` Eric Sandeen
2017-04-04 17:43     ` Emmanuel Florac
2017-04-04 17:45       ` Eric Sandeen
2017-04-04 18:07         ` Emmanuel Florac
2017-04-04 18:16           ` Eric Sandeen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170404192411.650fb403@harpe.intellique.com \
    --to=eflorac@intellique.com \
    --cc=linux-xfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.