All of lore.kernel.org
 help / color / mirror / Atom feed
* mount.cifs question: mounting a cifs share located on an IBM z/OS mainframe to a linux machine
@ 2014-04-09  9:22 Nadezhda Ivanova
       [not found] ` <CACERx8gmfuw+B7zkX+hfTOFrpTghEVkSEMxSK+TZ9QsrG9vFkA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  0 siblings, 1 reply; 6+ messages in thread
From: Nadezhda Ivanova @ 2014-04-09  9:22 UTC (permalink / raw)
  To: linux-cifs-u79uwXL29TY76Z2rM5mHXA

[-- Attachment #1: Type: text/plain, Size: 4813 bytes --]

Hi guys,
Has anyone successfully mounted a cifs share from a z/OS mainframe to
a linux machine?
We have a partner who is trying to do that, but the mount fails with a
Input/Output error. The same share is successfully mounted to a
Windows 7 or a Mac. The attached wireshark captures seem to show that
the z/OS returns a malformed Protocol Negotiation request to the
mount.cifs, but returns a correct packet to the Windows machine.
The user is running an Ububtu 12.04 TLS, kernel version 3.11.0-19-generic.
Also tried Ubuntu 10.04 and Centos6.5 with identical results.
The server is a z/OS 1.13

Here are the commands we tried:

root@cnaf-app-00:~# mount --verbose -t cifs //192.182.1.212/samba
/media/samba -o username=mreeves%amoscat0,sec=ntlm
mount.cifs kernel mount options:
ip=192.168.1.212,unc=//192.168.1.212/samba,sec=ntlm,ver=1,user=mreeves,pass=********
mount error(5): Input/output error
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
root@cnaf-app-00:~# mount --verbose -t cifs //192.182.1.212/samba
/media/samba -o username=mreeves%amoscat0,sec=NTLMv2
mount.cifs kernel mount options:
ip=192.168.1.212,unc=//192.168.1.212/samba,sec=NTLMv2,ver=1,user=mreeves,pass=********
mount error(5): Input/output error
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
root@cnaf-app-00:~# mount --verbose -t cifs //192.168.1.212/samba
/media/samba -o username=mreeves%amoscat0,sec=lanman
mount.cifs kernel mount options:
ip=192.168.1.212,unc=//192.168.1.212/samba,sec=lanman,ver=1,user=mreeves,pass=********
mount error(5): Input/output error
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

smbclient appears to work, but we get some garbage:
oot@cnaf-app-00:/etc/samba# smbclient //192.168.1.212/samba -U
mreeves%amoscat0
Domain=[■£Ç■╝Ç■êÇ■¼Ç■£Ç■êÇ■╝Ç■
>
> öÇ■ÇÇ]
> OS=[■îÇ■╝Ç■îÇ■ñÇ■ÇÇ■ÇÇ■ÿÇ■êÇ■êÇ■áÇ■┤Ç■ÇÇ]
> Server=[■ñÇ■êÇ■┤Ç■ÇÇ■îÇ°öDZêDZÿÇ°öDZêÇ■ÇÇ°ÿÇ°╝DZêÇ■ÇÇ■£Ç°ñÇ°╕Ç°ÉÇ°╝DZ£Ç±îÇ■ÇÇ■╕Ç°öDZÉDZ£Ç°╝DZêÇ°¼Ç■ÇÇ■╕Ç°öÇ°ñÇ°£Ç°áÇ°êÇ°╝DZêÇ°áÇ°╝Ç°╝Ç°ÉÇ]
> smb: \> ls
> .                                   D        0  Thu Apr  3 06:43:17 2014
> ..                                  D        0  Thu Apr  3 06:43:17 2014
> ._.DS_Store                               4096  Thu Apr  3 06:43:17 2014
> .DS_Store                                 6148  Thu Apr  3 06:43:39 2014
> shellscript.txt                           3260  Wed Apr  2 19:09:53 2014
>
>              65535 blocks of size 32768. 65535 blocks available
> smb: \>


The following appears in the logs:

Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.808997] CIFS VFS:
RFC1001 size 84 smaller than SMB for mid=1
Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809034] Bad SMB: : dump
of 48 bytes of data at 0xffff880019744540
Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809037]  54000000
424d53ff 00000072 c8018000 . . . T ˇ S M B r . . . . . . »
Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809039]  00000000
00000000 00000000 0e260000 . . . . . . . . . . . . . . & .
Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809041]  00010000
03000211 00010032 0000ffff . . . . . . . . 2 . . . ˇ ˇ . .
Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809107] CIFS VFS:
cifs_mount failed w/return code = -5

This is the z/OS machine configuration:

_IOE_DFS_MODIFY_PATH=/opt/dfslocal/home/dfscntl/modify.rendezvous
_IOE_MVS_DFSDFLT=EMCROOT
_IOE_LFS_SYNC_INTERVAL=30
_IOE_SMB_CLEAR_PW=NOTALLOWED
_IOE_DYNAMIC_EXPORT=ON
_IOE_SMB_TRANSPORTS=BOTH
_IOE_SMB_CONNECT_MSGS=2
_IOE_SERVER_NAME=EMCZPDT1
_IOE_SMB_COMPUTER_NAME=EMCZPDT1
_IOE_SMB_DOMAIN_NAME=WORKGROUP
_IOE_SMB_IDMAP=/opt/dfslocal/home/dfskern/smbidmap
_IOE_PROTOCOL_SMB=ON
_IOE_WIRE_CODEPAGE=ISO8859-1
_EUV_SVC_MSG_LOGGING=CONSOLE_LOGGING
DCE_START_SOCKET_NAME=/opt/dfslocal/home/dfscntl/ioepk.soc
TZ=EST5EDT
_EUV_AUTOLOG=NO
#NLSPATH=/usr/lib/nls/msg/%L/%N:/usr/lpp/Printsrv/En_US/%N
NLSPATH=/usr/lib/nls/msg/%L/%N
LANG=En_US.IBM-1047
#LIBPATH=/usr/lib:/usr/lpp/Printsrv/lib
LIBPATH=/usr/lib
_IOE_SMB_OCSF=OFF

I am attaching wireshark captures of both the successful mount from a
windows machine and the failed one from the linux machine.

After my initial mail on the samba-technical list, I was contacted by
someone in an identical situation who has also been unable to access
their z/OS shares after switching to  RHEL 6.x from Windows, so I
guess this is not an isolated incident.

If anyone has experience with this and has any ideas, any possible
problems with the configuration, I would appreciate it!

Best Regards,
Nadezhda

[-- Attachment #2: ubuntu_cifs --]
[-- Type: application/octet-stream, Size: 2427 bytes --]

[-- Attachment #3: win_cifs.cap --]
[-- Type: application/vnd.tcpdump.pcap, Size: 14364 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: mount.cifs question: mounting a cifs share located on an IBM z/OS mainframe to a linux machine
       [not found] ` <CACERx8gmfuw+B7zkX+hfTOFrpTghEVkSEMxSK+TZ9QsrG9vFkA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2014-04-09 19:38   ` Jeff Layton
       [not found]     ` <20140409153807.05b587fa-9yPaYZwiELC+kQycOl6kW4xkIHaj4LzF@public.gmane.org>
       [not found]     ` <CACERx8gGJQ6ykNp5XGVX_YZdOL9figHG-2v7y39Bs4_ha6f7Hg@mail.gmail.com>
  0 siblings, 2 replies; 6+ messages in thread
From: Jeff Layton @ 2014-04-09 19:38 UTC (permalink / raw)
  To: Nadezhda Ivanova; +Cc: linux-cifs-u79uwXL29TY76Z2rM5mHXA

On Wed, 9 Apr 2014 12:22:38 +0300
Nadezhda Ivanova <nivanova-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:

> Hi guys,
> Has anyone successfully mounted a cifs share from a z/OS mainframe to
> a linux machine?
> We have a partner who is trying to do that, but the mount fails with a
> Input/Output error. The same share is successfully mounted to a
> Windows 7 or a Mac. The attached wireshark captures seem to show that
> the z/OS returns a malformed Protocol Negotiation request to the
> mount.cifs, but returns a correct packet to the Windows machine.
> The user is running an Ububtu 12.04 TLS, kernel version 3.11.0-19-generic.
> Also tried Ubuntu 10.04 and Centos6.5 with identical results.
> The server is a z/OS 1.13
> 
> Here are the commands we tried:
> 
> root@cnaf-app-00:~# mount --verbose -t cifs //192.182.1.212/samba
> /media/samba -o username=mreeves%amoscat0,sec=ntlm
> mount.cifs kernel mount options:
> ip=192.168.1.212,unc=//192.168.1.212/samba,sec=ntlm,ver=1,user=mreeves,pass=********
> mount error(5): Input/output error
> Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
> root@cnaf-app-00:~# mount --verbose -t cifs //192.182.1.212/samba
> /media/samba -o username=mreeves%amoscat0,sec=NTLMv2
> mount.cifs kernel mount options:
> ip=192.168.1.212,unc=//192.168.1.212/samba,sec=NTLMv2,ver=1,user=mreeves,pass=********
> mount error(5): Input/output error
> Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
> root@cnaf-app-00:~# mount --verbose -t cifs //192.168.1.212/samba
> /media/samba -o username=mreeves%amoscat0,sec=lanman
> mount.cifs kernel mount options:
> ip=192.168.1.212,unc=//192.168.1.212/samba,sec=lanman,ver=1,user=mreeves,pass=********
> mount error(5): Input/output error
> Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
> 
> smbclient appears to work, but we get some garbage:
> oot@cnaf-app-00:/etc/samba# smbclient //192.168.1.212/samba -U
> mreeves%amoscat0
> Domain=[■£Ç■╝Ç■êÇ■¼Ç■£Ç■êÇ■╝Ç■
> >
> > öÇ■ÇÇ]
> > OS=[■îÇ■╝Ç■îÇ■ñÇ■ÇÇ■ÇÇ■ÿÇ■êÇ■êÇ■áÇ■┤Ç■ÇÇ]
> > Server=[■ñÇ■êÇ■┤Ç■ÇÇ■îÇ°öDZêDZÿÇ°öDZêÇ■ÇÇ°ÿÇ°╝DZêÇ■ÇÇ■£Ç°ñÇ°╕Ç°ÉÇ°╝DZ£Ç±îÇ■ÇÇ■╕Ç°öDZÉDZ£Ç°╝DZêÇ°¼Ç■ÇÇ■╕Ç°öÇ°ñÇ°£Ç°áÇ°êÇ°╝DZêÇ°áÇ°╝Ç°╝Ç°ÉÇ]
> > smb: \> ls
> > .                                   D        0  Thu Apr  3 06:43:17 2014
> > ..                                  D        0  Thu Apr  3 06:43:17 2014
> > ._.DS_Store                               4096  Thu Apr  3 06:43:17 2014
> > .DS_Store                                 6148  Thu Apr  3 06:43:39 2014
> > shellscript.txt                           3260  Wed Apr  2 19:09:53 2014
> >
> >              65535 blocks of size 32768. 65535 blocks available
> > smb: \>
> 
> 
> The following appears in the logs:
> 
> Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.808997] CIFS VFS:
> RFC1001 size 84 smaller than SMB for mid=1
> Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809034] Bad SMB: : dump
> of 48 bytes of data at 0xffff880019744540
> Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809037]  54000000
> 424d53ff 00000072 c8018000 . . . T ˇ S M B r . . . . . . »
> Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809039]  00000000
> 00000000 00000000 0e260000 . . . . . . . . . . . . . . & .
> Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809041]  00010000
> 03000211 00010032 0000ffff . . . . . . . . 2 . . . ˇ ˇ . .
> Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809107] CIFS VFS:
> cifs_mount failed w/return code = -5
> 
> This is the z/OS machine configuration:
> 
> _IOE_DFS_MODIFY_PATH=/opt/dfslocal/home/dfscntl/modify.rendezvous
> _IOE_MVS_DFSDFLT=EMCROOT
> _IOE_LFS_SYNC_INTERVAL=30
> _IOE_SMB_CLEAR_PW=NOTALLOWED
> _IOE_DYNAMIC_EXPORT=ON
> _IOE_SMB_TRANSPORTS=BOTH
> _IOE_SMB_CONNECT_MSGS=2
> _IOE_SERVER_NAME=EMCZPDT1
> _IOE_SMB_COMPUTER_NAME=EMCZPDT1
> _IOE_SMB_DOMAIN_NAME=WORKGROUP
> _IOE_SMB_IDMAP=/opt/dfslocal/home/dfskern/smbidmap
> _IOE_PROTOCOL_SMB=ON
> _IOE_WIRE_CODEPAGE=ISO8859-1
> _EUV_SVC_MSG_LOGGING=CONSOLE_LOGGING
> DCE_START_SOCKET_NAME=/opt/dfslocal/home/dfscntl/ioepk.soc
> TZ=EST5EDT
> _EUV_AUTOLOG=NO
> #NLSPATH=/usr/lib/nls/msg/%L/%N:/usr/lpp/Printsrv/En_US/%N
> NLSPATH=/usr/lib/nls/msg/%L/%N
> LANG=En_US.IBM-1047
> #LIBPATH=/usr/lib:/usr/lpp/Printsrv/lib
> LIBPATH=/usr/lib
> _IOE_SMB_OCSF=OFF
> 
> I am attaching wireshark captures of both the successful mount from a
> windows machine and the failed one from the linux machine.
> 
> After my initial mail on the samba-technical list, I was contacted by
> someone in an identical situation who has also been unable to access
> their z/OS shares after switching to  RHEL 6.x from Windows, so I
> guess this is not an isolated incident.
> 
> If anyone has experience with this and has any ideas, any possible
> problems with the configuration, I would appreciate it!
> 
> Best Regards,
> Nadezhda

Hi Nadezhda!

This server just looks to be broken.

The server claims to support unicode according to the capability bits
and then sends an ASCII string in the Primary Domain field in the
windows capture -- that's just wrong AFAIK.

In the linux capture, that field is these bytes:

    00 ff 53 4d 42 25 00

...which is also not proper unicode. Even worse, it starts with a NULL,
and then follows with the standard SMB protocol header. Looks maybe
they have some sort of alignment problem there and a trailing SMB got
tacked on to the end of this one. Nasty!

They should really talk to their server vendor ;)

-- 
Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: mount.cifs question: mounting a cifs share located on an IBM z/OS mainframe to a linux machine
       [not found]     ` <20140409153807.05b587fa-9yPaYZwiELC+kQycOl6kW4xkIHaj4LzF@public.gmane.org>
@ 2014-04-09 20:05       ` Nadezhda Ivanova
  0 siblings, 0 replies; 6+ messages in thread
From: Nadezhda Ivanova @ 2014-04-09 20:05 UTC (permalink / raw)
  To: Jeff Layton; +Cc: linux-cifs-u79uwXL29TY76Z2rM5mHXA

Hi Jeff,
Thanks for the answer!
I agree that something is broken with the server. However, since the
server sends correct packets to Windows and Mac (as evidenced by the
other capture), I thought there may be something in the linux client
that exposes a hidden bug on the server, and there may be a
workaround, and someone among the experienced CIFS masters might have
heard of it :).

I may try and find out what that is when I have time...



Regards,

Nadya

On Wed, Apr 9, 2014 at 10:38 PM, Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> On Wed, 9 Apr 2014 12:22:38 +0300
> Nadezhda Ivanova <nivanova-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:
>
>> Hi guys,
>> Has anyone successfully mounted a cifs share from a z/OS mainframe to
>> a linux machine?
>> We have a partner who is trying to do that, but the mount fails with a
>> Input/Output error. The same share is successfully mounted to a
>> Windows 7 or a Mac. The attached wireshark captures seem to show that
>> the z/OS returns a malformed Protocol Negotiation request to the
>> mount.cifs, but returns a correct packet to the Windows machine.
>> The user is running an Ububtu 12.04 TLS, kernel version 3.11.0-19-generic.
>> Also tried Ubuntu 10.04 and Centos6.5 with identical results.
>> The server is a z/OS 1.13
>>
>> Here are the commands we tried:
>>
>> root@cnaf-app-00:~# mount --verbose -t cifs //192.182.1.212/samba
>> /media/samba -o username=mreeves%amoscat0,sec=ntlm
>> mount.cifs kernel mount options:
>> ip=192.168.1.212,unc=//192.168.1.212/samba,sec=ntlm,ver=1,user=mreeves,pass=********
>> mount error(5): Input/output error
>> Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
>> root@cnaf-app-00:~# mount --verbose -t cifs //192.182.1.212/samba
>> /media/samba -o username=mreeves%amoscat0,sec=NTLMv2
>> mount.cifs kernel mount options:
>> ip=192.168.1.212,unc=//192.168.1.212/samba,sec=NTLMv2,ver=1,user=mreeves,pass=********
>> mount error(5): Input/output error
>> Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
>> root@cnaf-app-00:~# mount --verbose -t cifs //192.168.1.212/samba
>> /media/samba -o username=mreeves%amoscat0,sec=lanman
>> mount.cifs kernel mount options:
>> ip=192.168.1.212,unc=//192.168.1.212/samba,sec=lanman,ver=1,user=mreeves,pass=********
>> mount error(5): Input/output error
>> Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
>>
>> smbclient appears to work, but we get some garbage:
>> oot@cnaf-app-00:/etc/samba# smbclient //192.168.1.212/samba -U
>> mreeves%amoscat0
>> Domain=[■£Ç■╝Ç■êÇ■¼Ç■£Ç■êÇ■╝Ç■
>> >
>> > öÇ■ÇÇ]
>> > OS=[■îÇ■╝Ç■îÇ■ñÇ■ÇÇ■ÇÇ■ÿÇ■êÇ■êÇ■áÇ■┤Ç■ÇÇ]
>> > Server=[■ñÇ■êÇ■┤Ç■ÇÇ■îÇ°öDZêDZÿÇ°öDZêÇ■ÇÇ°ÿÇ°╝DZêÇ■ÇÇ■£Ç°ñÇ°╕Ç°ÉÇ°╝DZ£Ç±îÇ■ÇÇ■╕Ç°öDZÉDZ£Ç°╝DZêÇ°¼Ç■ÇÇ■╕Ç°öÇ°ñÇ°£Ç°áÇ°êÇ°╝DZêÇ°áÇ°╝Ç°╝Ç°ÉÇ]
>> > smb: \> ls
>> > .                                   D        0  Thu Apr  3 06:43:17 2014
>> > ..                                  D        0  Thu Apr  3 06:43:17 2014
>> > ._.DS_Store                               4096  Thu Apr  3 06:43:17 2014
>> > .DS_Store                                 6148  Thu Apr  3 06:43:39 2014
>> > shellscript.txt                           3260  Wed Apr  2 19:09:53 2014
>> >
>> >              65535 blocks of size 32768. 65535 blocks available
>> > smb: \>
>>
>>
>> The following appears in the logs:
>>
>> Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.808997] CIFS VFS:
>> RFC1001 size 84 smaller than SMB for mid=1
>> Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809034] Bad SMB: : dump
>> of 48 bytes of data at 0xffff880019744540
>> Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809037]  54000000
>> 424d53ff 00000072 c8018000 . . . T ˇ S M B r . . . . . . »
>> Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809039]  00000000
>> 00000000 00000000 0e260000 . . . . . . . . . . . . . . & .
>> Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809041]  00010000
>> 03000211 00010032 0000ffff . . . . . . . . 2 . . . ˇ ˇ . .
>> Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809107] CIFS VFS:
>> cifs_mount failed w/return code = -5
>>
>> This is the z/OS machine configuration:
>>
>> _IOE_DFS_MODIFY_PATH=/opt/dfslocal/home/dfscntl/modify.rendezvous
>> _IOE_MVS_DFSDFLT=EMCROOT
>> _IOE_LFS_SYNC_INTERVAL=30
>> _IOE_SMB_CLEAR_PW=NOTALLOWED
>> _IOE_DYNAMIC_EXPORT=ON
>> _IOE_SMB_TRANSPORTS=BOTH
>> _IOE_SMB_CONNECT_MSGS=2
>> _IOE_SERVER_NAME=EMCZPDT1
>> _IOE_SMB_COMPUTER_NAME=EMCZPDT1
>> _IOE_SMB_DOMAIN_NAME=WORKGROUP
>> _IOE_SMB_IDMAP=/opt/dfslocal/home/dfskern/smbidmap
>> _IOE_PROTOCOL_SMB=ON
>> _IOE_WIRE_CODEPAGE=ISO8859-1
>> _EUV_SVC_MSG_LOGGING=CONSOLE_LOGGING
>> DCE_START_SOCKET_NAME=/opt/dfslocal/home/dfscntl/ioepk.soc
>> TZ=EST5EDT
>> _EUV_AUTOLOG=NO
>> #NLSPATH=/usr/lib/nls/msg/%L/%N:/usr/lpp/Printsrv/En_US/%N
>> NLSPATH=/usr/lib/nls/msg/%L/%N
>> LANG=En_US.IBM-1047
>> #LIBPATH=/usr/lib:/usr/lpp/Printsrv/lib
>> LIBPATH=/usr/lib
>> _IOE_SMB_OCSF=OFF
>>
>> I am attaching wireshark captures of both the successful mount from a
>> windows machine and the failed one from the linux machine.
>>
>> After my initial mail on the samba-technical list, I was contacted by
>> someone in an identical situation who has also been unable to access
>> their z/OS shares after switching to  RHEL 6.x from Windows, so I
>> guess this is not an isolated incident.
>>
>> If anyone has experience with this and has any ideas, any possible
>> problems with the configuration, I would appreciate it!
>>
>> Best Regards,
>> Nadezhda
>
> Hi Nadezhda!
>
> This server just looks to be broken.
>
> The server claims to support unicode according to the capability bits
> and then sends an ASCII string in the Primary Domain field in the
> windows capture -- that's just wrong AFAIK.
>
> In the linux capture, that field is these bytes:
>
>     00 ff 53 4d 42 25 00
>
> ...which is also not proper unicode. Even worse, it starts with a NULL,
> and then follows with the standard SMB protocol header. Looks maybe
> they have some sort of alignment problem there and a trailing SMB got
> tacked on to the end of this one. Nasty!
>
> They should really talk to their server vendor ;)
>
> --
> Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: mount.cifs question: mounting a cifs share located on an IBM z/OS mainframe to a linux machine
       [not found]       ` <CACERx8gGJQ6ykNp5XGVX_YZdOL9figHG-2v7y39Bs4_ha6f7Hg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2014-04-09 20:27         ` Jeff Layton
       [not found]           ` <20140409162739.65689422-9yPaYZwiELC+kQycOl6kW4xkIHaj4LzF@public.gmane.org>
  0 siblings, 1 reply; 6+ messages in thread
From: Jeff Layton @ 2014-04-09 20:27 UTC (permalink / raw)
  To: Nadezhda Ivanova; +Cc: linux-cifs-u79uwXL29TY76Z2rM5mHXA

On Wed, 9 Apr 2014 23:04:20 +0300
Nadezhda Ivanova <nivanova-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:

> Hi Jeff,
> Thanks for the answer!
> I agree that something is broken with the server. However, since the server
> sends correct packets to Windows and Mac (as evidenced by the other
> capture), I thought there may be something in the linux client that exposes
> a hidden bug on the server, and there may be a workaround, and someone
> among the experienced CIFS masters might have heard of it :).
> 
> I may try and find out what that is when I have time...
>

They're not correct packets, unfortunately, it's just that Windows and
Macs are less strict about checking of frame lengths than the linux
client is.

The problem in your case is coming from the checkSMB function, which
verifies that the lengths in the packet match up. In this case, the
RFC1001 length at the start of the frame is shorter than the the length
we get back from smbCalcSize (which totals up the header length, word
count and byte count).

In point of fact, it looks like the BCC is where things are broken. It
says that it's 18 bytes, but the frame ends 15 bytes later. If we ended
up trusting the BCC in that frame then we'd quite possibly end up
walking off the end of the packet. Since cifs.ko lives in the kernel,
we're a bit paranoid about buffer overruns. ;)

You can relax the checks in there (at your own risk!) but I don't think
we'd be amenable to any patches that do so in the mainline kernel.

> 
> 
> Regards,
> 
> Nadya
> 
> 
> On Wed, Apr 9, 2014 at 10:38 PM, Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> 
> > On Wed, 9 Apr 2014 12:22:38 +0300
> > Nadezhda Ivanova <nivanova-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:
> >
> > > Hi guys,
> > > Has anyone successfully mounted a cifs share from a z/OS mainframe to
> > > a linux machine?
> > > We have a partner who is trying to do that, but the mount fails with a
> > > Input/Output error. The same share is successfully mounted to a
> > > Windows 7 or a Mac. The attached wireshark captures seem to show that
> > > the z/OS returns a malformed Protocol Negotiation request to the
> > > mount.cifs, but returns a correct packet to the Windows machine.
> > > The user is running an Ububtu 12.04 TLS, kernel version
> > 3.11.0-19-generic.
> > > Also tried Ubuntu 10.04 and Centos6.5 with identical results.
> > > The server is a z/OS 1.13
> > >
> > > Here are the commands we tried:
> > >
> > > root@cnaf-app-00:~# mount --verbose -t cifs //192.182.1.212/samba
> > > /media/samba -o username=mreeves%amoscat0,sec=ntlm
> > > mount.cifs kernel mount options:
> > > ip=192.168.1.212,unc=//
> > 192.168.1.212/samba,sec=ntlm,ver=1,user=mreeves,pass=********
> > > mount error(5): Input/output error
> > > Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
> > > root@cnaf-app-00:~# mount --verbose -t cifs //192.182.1.212/samba
> > > /media/samba -o username=mreeves%amoscat0,sec=NTLMv2
> > > mount.cifs kernel mount options:
> > > ip=192.168.1.212,unc=//
> > 192.168.1.212/samba,sec=NTLMv2,ver=1,user=mreeves,pass=********
> > > mount error(5): Input/output error
> > > Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
> > > root@cnaf-app-00:~# mount --verbose -t cifs //192.168.1.212/samba
> > > /media/samba -o username=mreeves%amoscat0,sec=lanman
> > > mount.cifs kernel mount options:
> > > ip=192.168.1.212,unc=//
> > 192.168.1.212/samba,sec=lanman,ver=1,user=mreeves,pass=********
> > > mount error(5): Input/output error
> > > Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
> > >
> > > smbclient appears to work, but we get some garbage:
> > > oot@cnaf-app-00:/etc/samba# smbclient //192.168.1.212/samba -U
> > > mreeves%amoscat0
> > > Domain=[■£Ç■╝Ç■êÇ■¼Ç■£Ç■êÇ■╝Ç■
> > > >
> > > > öÇ■ÇÇ]
> > > > OS=[■îÇ■╝Ç■îÇ■ñÇ■ÇÇ■ÇÇ■ÿÇ■êÇ■êÇ■áÇ■┤Ç■ÇÇ]
> > > >
> > Server=[■ñÇ■êÇ■┤Ç■ÇÇ■îÇ°öDZêDZÿÇ°öDZêÇ■ÇÇ°ÿÇ°╝DZêÇ■ÇÇ■£Ç°ñÇ°╕Ç°ÉÇ°╝DZ£Ç±îÇ■ÇÇ■╕Ç°öDZÉDZ£Ç°╝DZêÇ°¼Ç■ÇÇ■╕Ç°öÇ°ñÇ°£Ç°áÇ°êÇ°╝DZêÇ°áÇ°╝Ç°╝Ç°ÉÇ]
> > > > smb: \> ls
> > > > .                                   D        0  Thu Apr  3 06:43:17
> > 2014
> > > > ..                                  D        0  Thu Apr  3 06:43:17
> > 2014
> > > > ._.DS_Store                               4096  Thu Apr  3 06:43:17
> > 2014
> > > > .DS_Store                                 6148  Thu Apr  3 06:43:39
> > 2014
> > > > shellscript.txt                           3260  Wed Apr  2 19:09:53
> > 2014
> > > >
> > > >              65535 blocks of size 32768. 65535 blocks available
> > > > smb: \>
> > >
> > >
> > > The following appears in the logs:
> > >
> > > Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.808997] CIFS VFS:
> > > RFC1001 size 84 smaller than SMB for mid=1
> > > Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809034] Bad SMB: : dump
> > > of 48 bytes of data at 0xffff880019744540
> > > Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809037]  54000000
> > > 424d53ff 00000072 c8018000 . . . T ˇ S M B r . . . . . . »
> > > Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809039]  00000000
> > > 00000000 00000000 0e260000 . . . . . . . . . . . . . . & .
> > > Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809041]  00010000
> > > 03000211 00010032 0000ffff . . . . . . . . 2 . . . ˇ ˇ . .
> > > Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809107] CIFS VFS:
> > > cifs_mount failed w/return code = -5
> > >
> > > This is the z/OS machine configuration:
> > >
> > > _IOE_DFS_MODIFY_PATH=/opt/dfslocal/home/dfscntl/modify.rendezvous
> > > _IOE_MVS_DFSDFLT=EMCROOT
> > > _IOE_LFS_SYNC_INTERVAL=30
> > > _IOE_SMB_CLEAR_PW=NOTALLOWED
> > > _IOE_DYNAMIC_EXPORT=ON
> > > _IOE_SMB_TRANSPORTS=BOTH
> > > _IOE_SMB_CONNECT_MSGS=2
> > > _IOE_SERVER_NAME=EMCZPDT1
> > > _IOE_SMB_COMPUTER_NAME=EMCZPDT1
> > > _IOE_SMB_DOMAIN_NAME=WORKGROUP
> > > _IOE_SMB_IDMAP=/opt/dfslocal/home/dfskern/smbidmap
> > > _IOE_PROTOCOL_SMB=ON
> > > _IOE_WIRE_CODEPAGE=ISO8859-1
> > > _EUV_SVC_MSG_LOGGING=CONSOLE_LOGGING
> > > DCE_START_SOCKET_NAME=/opt/dfslocal/home/dfscntl/ioepk.soc
> > > TZ=EST5EDT
> > > _EUV_AUTOLOG=NO
> > > #NLSPATH=/usr/lib/nls/msg/%L/%N:/usr/lpp/Printsrv/En_US/%N
> > > NLSPATH=/usr/lib/nls/msg/%L/%N
> > > LANG=En_US.IBM-1047
> > > #LIBPATH=/usr/lib:/usr/lpp/Printsrv/lib
> > > LIBPATH=/usr/lib
> > > _IOE_SMB_OCSF=OFF
> > >
> > > I am attaching wireshark captures of both the successful mount from a
> > > windows machine and the failed one from the linux machine.
> > >
> > > After my initial mail on the samba-technical list, I was contacted by
> > > someone in an identical situation who has also been unable to access
> > > their z/OS shares after switching to  RHEL 6.x from Windows, so I
> > > guess this is not an isolated incident.
> > >
> > > If anyone has experience with this and has any ideas, any possible
> > > problems with the configuration, I would appreciate it!
> > >
> > > Best Regards,
> > > Nadezhda
> >
> > Hi Nadezhda!
> >
> > This server just looks to be broken.
> >
> > The server claims to support unicode according to the capability bits
> > and then sends an ASCII string in the Primary Domain field in the
> > windows capture -- that's just wrong AFAIK.
> >
> > In the linux capture, that field is these bytes:
> >
> >     00 ff 53 4d 42 25 00
> >
> > ...which is also not proper unicode. Even worse, it starts with a NULL,
> > and then follows with the standard SMB protocol header. Looks maybe
> > they have some sort of alignment problem there and a trailing SMB got
> > tacked on to the end of this one. Nasty!
> >
> > They should really talk to their server vendor ;)
> >
> > --
> > Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> >


-- 
Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: mount.cifs question: mounting a cifs share located on an IBM z/OS mainframe to a linux machine
       [not found]           ` <20140409162739.65689422-9yPaYZwiELC+kQycOl6kW4xkIHaj4LzF@public.gmane.org>
@ 2014-04-10 17:41             ` Nadezhda Ivanova
       [not found]               ` <CACERx8iEFmb+y+N4NAeqpcDtqZi+C-EWPuvA01E+1QpR4bKynw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
  0 siblings, 1 reply; 6+ messages in thread
From: Nadezhda Ivanova @ 2014-04-10 17:41 UTC (permalink / raw)
  To: Jeff Layton; +Cc: linux-cifs-u79uwXL29TY76Z2rM5mHXA

Hi Jeff,
Thank you for the detailed explanation! The stricter checking makes
sense to me now.
Can you tell me what is wrong with the Protocol Negotiation response
returned to the windows client (the win_cifs.cap)? I thought it was
correct, as it has indeed 18 bytes after bcc, and you can see that the
primary domain is WORKGROUP, obviously I am missing something...

Regards,
Nadya

On Wed, Apr 9, 2014 at 11:27 PM, Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> On Wed, 9 Apr 2014 23:04:20 +0300
> Nadezhda Ivanova <nivanova-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:
>
>> Hi Jeff,
>> Thanks for the answer!
>> I agree that something is broken with the server. However, since the server
>> sends correct packets to Windows and Mac (as evidenced by the other
>> capture), I thought there may be something in the linux client that exposes
>> a hidden bug on the server, and there may be a workaround, and someone
>> among the experienced CIFS masters might have heard of it :).
>>
>> I may try and find out what that is when I have time...
>>
>
> They're not correct packets, unfortunately, it's just that Windows and
> Macs are less strict about checking of frame lengths than the linux
> client is.
>
> The problem in your case is coming from the checkSMB function, which
> verifies that the lengths in the packet match up. In this case, the
> RFC1001 length at the start of the frame is shorter than the the length
> we get back from smbCalcSize (which totals up the header length, word
> count and byte count).
>
> In point of fact, it looks like the BCC is where things are broken. It
> says that it's 18 bytes, but the frame ends 15 bytes later. If we ended
> up trusting the BCC in that frame then we'd quite possibly end up
> walking off the end of the packet. Since cifs.ko lives in the kernel,
> we're a bit paranoid about buffer overruns. ;)
>
> You can relax the checks in there (at your own risk!) but I don't think
> we'd be amenable to any patches that do so in the mainline kernel.
>
>>
>>
>> Regards,
>>
>> Nadya
>>
>>
>> On Wed, Apr 9, 2014 at 10:38 PM, Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>>
>> > On Wed, 9 Apr 2014 12:22:38 +0300
>> > Nadezhda Ivanova <nivanova-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:
>> >
>> > > Hi guys,
>> > > Has anyone successfully mounted a cifs share from a z/OS mainframe to
>> > > a linux machine?
>> > > We have a partner who is trying to do that, but the mount fails with a
>> > > Input/Output error. The same share is successfully mounted to a
>> > > Windows 7 or a Mac. The attached wireshark captures seem to show that
>> > > the z/OS returns a malformed Protocol Negotiation request to the
>> > > mount.cifs, but returns a correct packet to the Windows machine.
>> > > The user is running an Ububtu 12.04 TLS, kernel version
>> > 3.11.0-19-generic.
>> > > Also tried Ubuntu 10.04 and Centos6.5 with identical results.
>> > > The server is a z/OS 1.13
>> > >
>> > > Here are the commands we tried:
>> > >
>> > > root@cnaf-app-00:~# mount --verbose -t cifs //192.182.1.212/samba
>> > > /media/samba -o username=mreeves%amoscat0,sec=ntlm
>> > > mount.cifs kernel mount options:
>> > > ip=192.168.1.212,unc=//
>> > 192.168.1.212/samba,sec=ntlm,ver=1,user=mreeves,pass=********
>> > > mount error(5): Input/output error
>> > > Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
>> > > root@cnaf-app-00:~# mount --verbose -t cifs //192.182.1.212/samba
>> > > /media/samba -o username=mreeves%amoscat0,sec=NTLMv2
>> > > mount.cifs kernel mount options:
>> > > ip=192.168.1.212,unc=//
>> > 192.168.1.212/samba,sec=NTLMv2,ver=1,user=mreeves,pass=********
>> > > mount error(5): Input/output error
>> > > Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
>> > > root@cnaf-app-00:~# mount --verbose -t cifs //192.168.1.212/samba
>> > > /media/samba -o username=mreeves%amoscat0,sec=lanman
>> > > mount.cifs kernel mount options:
>> > > ip=192.168.1.212,unc=//
>> > 192.168.1.212/samba,sec=lanman,ver=1,user=mreeves,pass=********
>> > > mount error(5): Input/output error
>> > > Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
>> > >
>> > > smbclient appears to work, but we get some garbage:
>> > > oot@cnaf-app-00:/etc/samba# smbclient //192.168.1.212/samba -U
>> > > mreeves%amoscat0
>> > > Domain=[■£Ç■╝Ç■êÇ■¼Ç■£Ç■êÇ■╝Ç■
>> > > >
>> > > > öÇ■ÇÇ]
>> > > > OS=[■îÇ■╝Ç■îÇ■ñÇ■ÇÇ■ÇÇ■ÿÇ■êÇ■êÇ■áÇ■┤Ç■ÇÇ]
>> > > >
>> > Server=[■ñÇ■êÇ■┤Ç■ÇÇ■îÇ°öDZêDZÿÇ°öDZêÇ■ÇÇ°ÿÇ°╝DZêÇ■ÇÇ■£Ç°ñÇ°╕Ç°ÉÇ°╝DZ£Ç±îÇ■ÇÇ■╕Ç°öDZÉDZ£Ç°╝DZêÇ°¼Ç■ÇÇ■╕Ç°öÇ°ñÇ°£Ç°áÇ°êÇ°╝DZêÇ°áÇ°╝Ç°╝Ç°ÉÇ]
>> > > > smb: \> ls
>> > > > .                                   D        0  Thu Apr  3 06:43:17
>> > 2014
>> > > > ..                                  D        0  Thu Apr  3 06:43:17
>> > 2014
>> > > > ._.DS_Store                               4096  Thu Apr  3 06:43:17
>> > 2014
>> > > > .DS_Store                                 6148  Thu Apr  3 06:43:39
>> > 2014
>> > > > shellscript.txt                           3260  Wed Apr  2 19:09:53
>> > 2014
>> > > >
>> > > >              65535 blocks of size 32768. 65535 blocks available
>> > > > smb: \>
>> > >
>> > >
>> > > The following appears in the logs:
>> > >
>> > > Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.808997] CIFS VFS:
>> > > RFC1001 size 84 smaller than SMB for mid=1
>> > > Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809034] Bad SMB: : dump
>> > > of 48 bytes of data at 0xffff880019744540
>> > > Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809037]  54000000
>> > > 424d53ff 00000072 c8018000 . . . T ˇ S M B r . . . . . . »
>> > > Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809039]  00000000
>> > > 00000000 00000000 0e260000 . . . . . . . . . . . . . . & .
>> > > Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809041]  00010000
>> > > 03000211 00010032 0000ffff . . . . . . . . 2 . . . ˇ ˇ . .
>> > > Apr  4 10:27:28 ubu-14-server1 kernel: [ 4652.809107] CIFS VFS:
>> > > cifs_mount failed w/return code = -5
>> > >
>> > > This is the z/OS machine configuration:
>> > >
>> > > _IOE_DFS_MODIFY_PATH=/opt/dfslocal/home/dfscntl/modify.rendezvous
>> > > _IOE_MVS_DFSDFLT=EMCROOT
>> > > _IOE_LFS_SYNC_INTERVAL=30
>> > > _IOE_SMB_CLEAR_PW=NOTALLOWED
>> > > _IOE_DYNAMIC_EXPORT=ON
>> > > _IOE_SMB_TRANSPORTS=BOTH
>> > > _IOE_SMB_CONNECT_MSGS=2
>> > > _IOE_SERVER_NAME=EMCZPDT1
>> > > _IOE_SMB_COMPUTER_NAME=EMCZPDT1
>> > > _IOE_SMB_DOMAIN_NAME=WORKGROUP
>> > > _IOE_SMB_IDMAP=/opt/dfslocal/home/dfskern/smbidmap
>> > > _IOE_PROTOCOL_SMB=ON
>> > > _IOE_WIRE_CODEPAGE=ISO8859-1
>> > > _EUV_SVC_MSG_LOGGING=CONSOLE_LOGGING
>> > > DCE_START_SOCKET_NAME=/opt/dfslocal/home/dfscntl/ioepk.soc
>> > > TZ=EST5EDT
>> > > _EUV_AUTOLOG=NO
>> > > #NLSPATH=/usr/lib/nls/msg/%L/%N:/usr/lpp/Printsrv/En_US/%N
>> > > NLSPATH=/usr/lib/nls/msg/%L/%N
>> > > LANG=En_US.IBM-1047
>> > > #LIBPATH=/usr/lib:/usr/lpp/Printsrv/lib
>> > > LIBPATH=/usr/lib
>> > > _IOE_SMB_OCSF=OFF
>> > >
>> > > I am attaching wireshark captures of both the successful mount from a
>> > > windows machine and the failed one from the linux machine.
>> > >
>> > > After my initial mail on the samba-technical list, I was contacted by
>> > > someone in an identical situation who has also been unable to access
>> > > their z/OS shares after switching to  RHEL 6.x from Windows, so I
>> > > guess this is not an isolated incident.
>> > >
>> > > If anyone has experience with this and has any ideas, any possible
>> > > problems with the configuration, I would appreciate it!
>> > >
>> > > Best Regards,
>> > > Nadezhda
>> >
>> > Hi Nadezhda!
>> >
>> > This server just looks to be broken.
>> >
>> > The server claims to support unicode according to the capability bits
>> > and then sends an ASCII string in the Primary Domain field in the
>> > windows capture -- that's just wrong AFAIK.
>> >
>> > In the linux capture, that field is these bytes:
>> >
>> >     00 ff 53 4d 42 25 00
>> >
>> > ...which is also not proper unicode. Even worse, it starts with a NULL,
>> > and then follows with the standard SMB protocol header. Looks maybe
>> > they have some sort of alignment problem there and a trailing SMB got
>> > tacked on to the end of this one. Nasty!
>> >
>> > They should really talk to their server vendor ;)
>> >
>> > --
>> > Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>> >
>
>
> --
> Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: mount.cifs question: mounting a cifs share located on an IBM z/OS mainframe to a linux machine
       [not found]               ` <CACERx8iEFmb+y+N4NAeqpcDtqZi+C-EWPuvA01E+1QpR4bKynw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2014-04-10 17:47                 ` Jeff Layton
  0 siblings, 0 replies; 6+ messages in thread
From: Jeff Layton @ 2014-04-10 17:47 UTC (permalink / raw)
  To: Nadezhda Ivanova; +Cc: linux-cifs-u79uwXL29TY76Z2rM5mHXA

On Thu, 10 Apr 2014 20:41:02 +0300
Nadezhda Ivanova <nivanova-eUNUBHrolfbYtjvyW6yDsg@public.gmane.org> wrote:

> Hi Jeff,
> Thank you for the detailed explanation! The stricter checking makes
> sense to me now.
> Can you tell me what is wrong with the Protocol Negotiation response
> returned to the windows client (the win_cifs.cap)? I thought it was
> correct, as it has indeed 18 bytes after bcc, and you can see that the
> primary domain is WORKGROUP, obviously I am missing something...
> 
> Regards,
> Nadya
> 

Yeah, the lengths in the windows capture look fine and it's NULL
terminated properly. I'm not sure why wireshark is complaining, but I'd
suspect that it's because the server has set the unicode bit in the
capability bits, but isn't sending that string in unicode.

If the server sent a response that looked like that to the cifs client,
I suspect that it would accept it and move on.

-- 
Jeff Layton <jlayton-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2014-04-10 17:47 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-04-09  9:22 mount.cifs question: mounting a cifs share located on an IBM z/OS mainframe to a linux machine Nadezhda Ivanova
     [not found] ` <CACERx8gmfuw+B7zkX+hfTOFrpTghEVkSEMxSK+TZ9QsrG9vFkA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-04-09 19:38   ` Jeff Layton
     [not found]     ` <20140409153807.05b587fa-9yPaYZwiELC+kQycOl6kW4xkIHaj4LzF@public.gmane.org>
2014-04-09 20:05       ` Nadezhda Ivanova
     [not found]     ` <CACERx8gGJQ6ykNp5XGVX_YZdOL9figHG-2v7y39Bs4_ha6f7Hg@mail.gmail.com>
     [not found]       ` <CACERx8gGJQ6ykNp5XGVX_YZdOL9figHG-2v7y39Bs4_ha6f7Hg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-04-09 20:27         ` Jeff Layton
     [not found]           ` <20140409162739.65689422-9yPaYZwiELC+kQycOl6kW4xkIHaj4LzF@public.gmane.org>
2014-04-10 17:41             ` Nadezhda Ivanova
     [not found]               ` <CACERx8iEFmb+y+N4NAeqpcDtqZi+C-EWPuvA01E+1QpR4bKynw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-04-10 17:47                 ` Jeff Layton

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.