Re: Oops in cifs_match_super() - linux kernel 4.16.2-1.el7.elrepo.x86_64

[Steve French <smfrench@gmail.com>]

At least it is very easy to build cifs.ko out of tree for old distros
that don't keep up to date with stable patches (e.g. something similar
to: cd into fs/cifs directory then "make C=1 -C
/usr/src/linux-headers-`uname -r` M=`pwd` modules") ... assuming you
can get src rpms for your distro version for the kernel.

On Fri, Jul 17, 2020 at 4:23 PM Vidhesh Ramesh
<vidhesh.ramesh@komprise.com> wrote:
>
> Thanks Steve.
> We have few more bug fixes that we need in 4.19 LT kernel. However I am unable to get a built kernel RPM of latest build 133 of kernel 4.19.
> The latest I could get is build 113 of kernel 4.19 at https://buildlogs.centos.org/c7-kernels.x86_64/kernel/20200330213326/4.19.113-300.el8.x86_64/kernel-4.19.113-300.el7.x86_64.rpm
>
> Vidhesh Ramesh
>
> From: Steve French <smfrench@gmail.com>
> Sent: Friday, July 17, 2020 7:05 AM
> To: Vidhesh Ramesh <vidhesh.ramesh@komprise.com>
> Cc: Kenneth Dsouza <kdsouza@redhat.com>; CIFS <linux-cifs@vger.kernel.org>
> Subject: Re: Oops in cifs_match_super() - linux kernel 4.16.2-1.el7.elrepo.x86_64
>
> I am not familiar with your distro but a Google search matched on this as a more recent kernel rpm download with similar sounding name http://lists.elrepo.org/pipermail/elrepo/2018-May/004282.html
>
> But there should be more recent kernels than that are eg based on newer 4.6.18 stable branch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/log/?h=linux-4.16.y
>
> On Fri, Jul 17, 2020, 04:12 Vidhesh Ramesh <vidhesh.ramesh@komprise.com> wrote:
> Thanks Steve and Kenneth for your responses.
>
> I did look at the commits and found that this has been fixed in a patch of 4.19. Is there an archive where I can download latest 4.19 kernel rpm so that I can verify the fix ?
>
> Vidhesh Ramesh
>
>
>
>
>
>
> From: Steve French <smfrench@gmail.com>
> Sent: Tuesday, July 14, 2020 10:49 AM
> To: Kenneth Dsouza <kdsouza@redhat.com>
> Cc: Vidhesh Ramesh <vidhesh.ramesh@komprise.com>; linux-cifs@vger.kernel.org <linux-cifs@vger.kernel.org>
> Subject: Re: Oops in cifs_match_super() - linux kernel 4.16.2-1.el7.elrepo.x86_64
>
> If you have a particular stable kernel used for your distro you can
> email stable@vger.kernel.org with the commit id and version you want
> it backported for.
>
> On Tue, Jul 14, 2020 at 12:43 PM Kenneth Dsouza <kdsouza@redhat.com> wrote:
> >
> > You are hitting a known bug which has been fixed by below patch and is
> > part of v5.6 and above.
> >
> > commit fe1292686333d1dadaf84091f585ee903b9ddb84
> > Author: Ronnie Sahlberg <lsahlber@redhat.com>
> > Date:   Wed Jan 22 11:07:56 2020 +1000
> >
> >     cifs: fix NULL dereference in match_prepath
> >
> >     RHBZ: 1760879
> >
> >     Fix an oops in match_prepath() by making sure that the prepath string is not
> >     NULL before we pass it into strcmp().
> >
> >     This is similar to other checks we make for example in cifs_root_iget()
> >
> >     Signed-off-by: Ronnie Sahlberg <lsahlber@redhat.com>
> >     Signed-off-by: Steve French <stfrench@microsoft.com>
> >
> > On Tue, Jul 14, 2020 at 11:05 PM Vidhesh Ramesh
> > <vidhesh.ramesh@komprise.com> wrote:
> > >
> > > Adding linux-cifs mailing list.
> > >
> > > Vidhesh Ramesh
> > >
> > >
> > > From: Vidhesh Ramesh <vidhesh.ramesh@komprise.com>
> > > Sent: Tuesday, July 14, 2020 10:18 AM
> > > To: Steve French <sfrench@samba.org>; linux-cifs@vger.kernel.org <linux-cifs@vger.kernel.org>; samba-technical@lists.samba.org <samba-technical@lists.samba.org>
> > > Cc: Ameya Usgaonkar <ameya.usgaonkar@komprise.com>
> > > Subject: Re: Oops in cifs_match_super() - linux kernel 4.16.2-1.el7.elrepo.x86_64
> > >
> > > Hi Steve et all,
> > >
> > > Resending this as a gentle reminder if anyone got a chance to look at the below mentioned oops kernel panic.
> > >
> > >
> > > Vidhesh Ramesh
> > >
> > >
> > > From: Vidhesh Ramesh <vidhesh.ramesh@komprise.com>
> > > Sent: Thursday, July 2, 2020 12:21 AM
> > > To: Steve French <sfrench@samba.org>; linux-cifs@vger.kernel.org <linux-cifs@vger.kernel.org>; samba-technical@lists.samba.org <samba-technical@lists.samba.org>
> > > Cc: Ameya Usgaonkar <ameya.usgaonkar@komprise.com>
> > > Subject: Re: Oops in cifs_match_super() - linux kernel 4.16.2-1.el7.elrepo.x86_64
> > >
> > > Hi Steve et all,
> > >
> > > Resending this as a gentle reminder if anyone got a chance to look at the below mentioned oops kernel panic.
> > >
> > > Vidhesh Ramesh
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > From: Ameya Usgaonkar <ameya.usgaonkar@komprise.com>
> > > Sent: Wednesday, June 24, 2020 10:26 PM
> > > To: Steve French <sfrench@samba.org>; linux-cifs@vger.kernel.org <linux-cifs@vger.kernel.org>; samba-technical@lists.samba.org <samba-technical@lists.samba.org>
> > > Cc: Vidhesh Ramesh <vidhesh.ramesh@komprise.com>
> > > Subject: Oops in cifs_match_super() - linux kernel 4.16.2-1.el7.elrepo.x86_64
> > >
> > > Hi Steve et al,
> > >    My name is Ameya and I work for www.komprise.com.  The linux kernel with above mentioned version has been panicing, though randomly, but the stack trace appears is consistent.  You can find more details in the attachments.  Below is the brief description of the problem -
> > >
> > > [1.] One line summary of the problem:
> > > oops kernel panic
> > > [2.] Full description of the problem/report:
> > > ESX VM hangs with a kernel panic when cifs shares are mounted. Unable to ssh to the VM and the console of the VM is also not responding. With kdump service running and core collected the VM restarts successfully.
> > > [3.] Keywords (i.e., modules, networking, kernel):
> > > cifs, kernel, panic, strcmp, mount
> > > [4.] Kernel information
> > > [4.1.] Kernel version (from /proc/version):
> > > Linux version 4.16.2-1.el7.elrepo.x86_64 (mockbuild@Build64R7) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC)) #1 SMP Thu Apr 12 09:08:05 EDT 2018
> > > [4.2.] Kernel .config file:
> > > Please check the file uploaded
> > > [5.] Most recent kernel version which did not have the bug:
> > > [6.] Output of Oops.. message (if applicable) with symbolic information
> > >      resolved (see Documentation/admin-guide/oops-tracing.rst)
> > > [442282.069937] BUG: unable to handle kernel NULL pointer dereference at 0000000000000000
> > > [442282.071474] IP: strcmp+0xe/0x30
> > > [442282.072892] PGD 0 P4D 0
> > > [442282.074196] Oops: 0000 [#1] SMP PTI
> > > [442282.075561] Modules linked in: binfmt_misc fuse cmac rpcsec_gss_krb5 nfsv4 arc4 md4 nls_utf8 cifs ccm dns_resolver nfsv3 nfs fscache nf_conntrack_netbios_ns nf_conntrack_broadcast xt_CT ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sb_edac crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc ppdev aesni_intel vmw_balloon crypto_simd glue_helper cryptd sg intel_rapl_perf input_leds pcspkr joydev shpchp
> > > [442282.090386]  parport_pc parport i2c_piix4 vmw_vmci nfsd nfs_acl lockd auth_rpcgss grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod pata_acpi crc32c_intel serio_raw vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi scsi_transport_spi ata_piix mptscsih vmxnet3 drm mptbase libata floppy dm_mirror dm_region_hash dm_log dm_mod dax
> > > [442282.095332] CPU: 4 PID: 9273 Comm: mount.cifs Tainted: G        W        4.16.2-1.el7.elrepo.x86_64 #1
> > > [442282.097802] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
> > > [442282.100363] RIP: 0010:strcmp+0xe/0x30
> > > [442282.101645] RSP: 0018:ffffc9001bff7c88 EFLAGS: 00010202
> > > [442282.102919] RAX: 0000000000000001 RBX: ffff8802ad7c2400 RCX: 0000000001240404
> > > [442282.104207] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000001
> > > [442282.105497] RBP: ffffc9001bff7c88 R08: 0000000001240404 R09: 0000000000000001
> > > [442282.106765] R10: ffff88017d2c2b40 R11: ffff8801744c0e50 R12: ffff88011f793000
> > > [442282.108038] R13: ffff88042b105800 R14: ffffc9001bff7d98 R15: ffff8801b63c0f00
> > > [442282.109318] FS:  00007f3e90ee6780(0000) GS:ffff88043fd00000(0000) knlGS:0000000000000000
> > > [442282.110630] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > [442282.111970] CR2: 0000000000000000 CR3: 000000011b2ea000 CR4: 00000000000406e0
> > > [442282.113405] Call Trace:
> > > [442282.114849]  cifs_match_super+0x123/0x210 [cifs]
> > > [442282.116211]  ? cifs_prune_tlinks+0xe0/0xe0 [cifs]
> > > [442282.117541]  sget_userns+0x88/0x4a0
> > > [442282.118877]  ? cifs_kill_sb+0x30/0x30 [cifs]
> > > [442282.120203]  ? cifs_prune_tlinks+0xe0/0xe0 [cifs]
> > > [442282.121502]  sget+0x7d/0xa0
> > > [442282.122781]  ? cifs_kill_sb+0x30/0x30 [cifs]
> > > [442282.124072]  cifs_do_mount+0x168/0x5a0 [cifs]
> > > [442282.125364]  mount_fs+0x3e/0x150
> > > [442282.126627]  vfs_kern_mount+0x67/0x130
> > > [442282.127850]  do_mount+0x1f5/0xca0
> > > [442282.129047]  SyS_mount+0x83/0xd0
> > > [442282.130257]  do_syscall_64+0x79/0x1b0
> > > [442282.131583]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2
> > > [442282.132687] RIP: 0033:0x7f3e907fdfea
> > > [442282.133754] RSP: 002b:00007ffee9139868 EFLAGS: 00000202 ORIG_RAX: 00000000000000a5
> > > [442282.134842] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f3e907fdfea
> > > [442282.135909] RDX: 000055d13f5953b2 RSI: 000055d13f5953f9 RDI: 00007ffee913bb51
> > > [442282.136944] RBP: 00007ffee913bb46 R08: 000055d1415e80d0 R09: 0000000000001000
> > > [442282.137962] R10: 0000000000000000 R11: 0000000000000202 R12: 00007f3e90ef0000
> > > [442282.138957] R13: 000055d1415e80d0 R14: 00007f3e90ef290f R15: 0000000000000000
> > > [442282.139918] Code: 80 3a 00 75 f7 48 83 c6 01 0f b6 4e ff 48 83 c2 01 84 c9 88 4a ff 75 ed 5d c3 0f 1f 00 55 48 89 e5 eb 04 84 c0 74 18 48 83 c7 01 <0f> b6 47 ff 48 83 c6 01 3a 46 ff 74 eb 19 c0 83 c8 01 5d c3 31
> > > [442282.142829] RIP: strcmp+0xe/0x30 RSP: ffffc9001bff7c88
> > > [442282.143759] CR2: 0000000000000000
> > > [7.] A small shell script or example program which triggers the
> > >      problem (if possible)
> > >
> > > My colleague Vidhesh (CC'ed) will be glad to share further details on the test scenario and/or in-house reproduction.  Please let us know of workarounds, if any.
> > >
> > > Thanks and Regards,
> > > = Ameya
> > >
> > >
> > >
> >
>
>
> --
> Thanks,
>
> Steve



-- 
Thanks,

Steve