* Re: writing a ceph cliente for MS windows
[not found] <CAME-gASugOE=-ZoY3-sAa4vCzOd+TXBiEavjoUH3gDLG+1K38w@mail.gmail.com>
@ 2013-11-07 12:13 ` Alphe Salas Michels
2013-11-07 14:29 ` Ketor D
0 siblings, 1 reply; 21+ messages in thread
From: Alphe Salas Michels @ 2013-11-07 12:13 UTC (permalink / raw)
To: ceph-devel, d.ketor
Commercial libraries are a pain ...
If we want the more permossive licence offered by callback file system
we have to buy it for 20.000 usd. Then we will have to provide a backbox
that we have no control upon and that will kill our product anytime they
want anf if they decide to stop their commercial activity we will be in
the same situation that with dokanfs but without having the source code
of the black box. If i have to spend 20 000 dollars i would prefere
paying someone to retake dokanfs or to write from scratch a dokanfs
fuselike software make it all shiny and pumpy fantastic and ready to
plug to ceph client.
I would prefere if people have to pay something to get access to
ceph4win that this money goes in ceph main branch pockets... Or as a
gift you donante to ceph 10 dollars you get 2 free registration codes
for ceph4win... or something like that.
If ceph4win as to be comercial then I would prefer delegate the task to
a company like south river technologies and their great product
webdrive. I would mininaly get involved in that project and simply buy
the final product to sell it together with my ceph based product (which
could be a calxeda ceph box or something like that).
I m open anyway to any proposition. But I doubt that callback filesystem
offers us a suitable solution in the way I see ceph4win to be spread and
used... I m maybe wrong. And anything that will be done around ceph4win
will be public documented etc... And licensed the way that if someone
want to build a commercial solution on top of it, that would be a
possibility.
My idea is to giveback somehow to ceph project and at same time forge a
better knowledge in ceph technologies. Because like many in libre world
I think the business is in the services around the software more than on
the software. That the ones writing code should be financed and benefits
from the one selling and giving support of the software at all levels. I
m probably too idealistic. And too optimistic after all I m the one
saying I will do this stuff I have no idea how but well it is
interesting and fun so lets do it.
Regards,
P.S: using commercial backend libraries appart including their own cost
will force you to use commercial IDE like MS VisualStudio because their
library has some kind of drm that only that IDE compiler can use. So
alot of cost and yet there is nothing done. If I had to open a
kickstarter project saying we need 60 000 USD to do ceph4win with that
monney we will buy the right to use and share a commercial copyrighted
library but abandonned punctually to us in public domaine and that we
will eventually produce something out of it. I doubt I will get a dollar.
We still can suggest the idea to Edlos the commercial company that has
the copyright of Callback FS, Or to buy them their product in a blender
way (blender was bought with donation before being put opensource and
public domaine), Or to open source their library. But in commercial
minds opensourcing = death of their technical advantage and death of
their marketing strategy. They will have to invent something more to
retrieve monney from it.
El nov 6, 2013 11:22 p.m., "Ketor D" <d.ketor@gmail.com
<mailto:d.ketor@gmail.com>> escribió:
Hi Alphe,
I think you could try Callback Filesystem dev framework. It
is a commerical dev framework and is maintained by Edlos today.
I have communicated with Edlos to get a try code for
development. To dokan, Callback Filesystem has vary document and maybe
more stabilize.
Regards.
On Thu, Nov 7, 2013 at 10:00 AM, Alphe Salas <asalas@kepler.cl
<mailto:asalas@kepler.cl>> wrote:
> Hello ketor thank you for your interest un ceph4win. Since muy
first mail I
> exposed the lacks of dokanfs and that I m far from being a
specialist un
> filesystems.
> I exposed what i like un dokanfs bit I not a fanátic of it. Muy
goal is to
> have something working quickly.
>
> So I am up to any proposición sure the one with the more docs and
support
> will be the best choice. As for right now what I need is
understand what are
> the files involved what are the interfaces functions and what are
the needed
> library dependencies and if they exist ported to windows with
cygwin. And
> all that is retrieved from source code.
>
> Regards.
>
> El nov 6, 2013 10:34 p.m., "Ketor D" <d.ketor@gmail.com
<mailto:d.ketor@gmail.com>> escribió:
>
>> Hi Alphe,
>> We are taking an interest in your work on Ceph Client for
Windows
>> with Dokan.As we know, the performance of Dokan is not very
good, and it's
>> abandoned 3 years ago.
>> I have learned and used OpenDedup(SDFS) for a long time.
OpenDedup
>> has a Dokan version. And the author of OpenDedup said
>>
>> The Dokan library is quite flakey and testing should be
performed before
>> putting into production
>>
>> So what do you think about this? And if there is another
solution of
>> fuse-like filesystem dev framwork on Windows?
>>
>> Best Wish!
>>
>>
>>
>> On Thu, Nov 7, 2013 at 5:47 AM, Alphe Salas Michels
<asalas@kepler.cl <mailto:asalas@kepler.cl>>
>> wrote:
>>>
>>> Hello I created the github repository for this project
>>> https://github.com/alphe/Ceph4Win
>>>
>>> Regards,
>>>
>>> signature
>>>
>>> *Alphé Salas*
>>> Ingeniero T.I
>>>
>>> asalas@kepler.cl <mailto:asalas@kepler.cl>
>>> *<http://www.kepler.cl>*
>>>
>>> On 11/05/13 21:00, Sage Weil wrote:
>>>>
>>>> Hi Alphe,
>>>>
>>>> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
>>>>>
>>>>> signature *Hi, Sage !
>>>>> thank you for you enthousiast reply.
>>>>> I sure want to make the best use of everything or anything
previously
>>>>> done to
>>>>> tend to
>>>>> write ceph cliente for windows.
>>>>>
>>>>> Apart using libre tools for building the future ceph cliente
I am open
>>>>> to
>>>>> anything.
>>>>> I would recommand eclipse CDT or Code::BLocks they are based
on mingwin
>>>>> open
>>>>> and easyly enhanceable.**
>>>>>
>>>>> more free tools can be found here:
>>>>> http://www.freebyte.com/programming/cpp/#cppcompilers
>>>>>
>>>>>
>>>>> I will read libcephfs source code and take some notes about the
>>>>> protocol.
>>>>
>>>> I think you don't need to worry about hte protocol at all, since
>>>> libcephs
>>>> implements it for you (and will capture any future changes).
>>>>
>>>>> I was more going from what I know and trying to track down how
>>>>> mount.ceph work
>>>>> with the parameters passed to it.
>>>>> since it point finally to Kernel/fs/ceph and that I don t really
>>>>> understand
>>>>> how that module work and that it probably points to some other
>>>>> dependencies
>>>>> Reading libcephfs source code could be a big gain of time.
>>>>
>>>> (I would also ignore mount.ceph as everything it does it
specific to
>>>> how Linux mounts work.)
>>>>
>>>>> basically on the protocol what is need are:
>>>>>
>>>>> 1) open and maintain a connection (socket open, auth, etc )
>>>>> 2) retreive a map of directories and disk Quota (disk sizing
Y TB free,
>>>>> Z TB
>>>>> total)
>>>>> 3) procedure to send files / directories in a maner that it
will allow
>>>>> our
>>>>> client to fit ceph transmission protocols
>>>>> (limit bandwith for stability?, limit connection amount?,
limit cpu
>>>>> use?,
>>>>> Cache for preparing data transfer (a FIFO cache)?)
>>>>> 4)Procedure to retreive files / directory from ceph cluster
>>>>> 5) Management copy/move files /Directories, FS stats,
Connection Stats.
>>>>> logging.
>>>>>
>>>>> My idea to progress is to take those main bulletpoint in ceph
protocol
>>>>> based
>>>>> on general ideas of what ceph file system does and start
identifying
>>>>> parts
>>>>> from libcephfs to match those "needs".
>>>>
>>>> Instead, I would look at include/cephfs/libcephfs.h, the
interface that
>>>> libcephfs provides, and try to map that to what the fuse layer
expects.
>>>> There is both a path-based that I suspsect lends itself well
to the
>>>> Windows interface and (very soon now) a handle based API that is
>>>> targetted
>>>> at the Unix-style VFS layers. I'm mostly guessing, though,
since I've
>>>> never seen any low-level fs code in windows before.
>>>>
>>>> In this case, the analogous code for Linux should be
client/fuse_ll.cc
>>>> itself (and not much else), although there will probably be a
few tricks
>>>> necessary to map cleanly onto how the windows interfaces work.
>>>>
>>>> Does that make sense?
>>>>
>>>> Cheers!
>>>> sage
>>>>
>>>>
>>>>> Any suggestion and contributions are welcome.
>>>>>
>>>>>
>>>>> *
>>>>> On 11/05/13 11:23, Sage Weil wrote:
>>>>>>
>>>>>> Hi Alphe,
>>>>>>
>>>>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
>>>>>>>
>>>>>>> Good day developers!
>>>>>>>
>>>>>>> I would like to propose to the one interested work with me to
>>>>>>> develop a
>>>>>>> ceph
>>>>>>> cliente for MS windows world, Basing us on dokanFS.
>>>>>>>
>>>>>>> My company is a ceph enthousiast that use on a dayly basis
ceph and
>>>>>>> that
>>>>>>> need
>>>>>>> both transfer speed and big expendable and cheap storage.
>>>>>>> My company is specialised in data recovery and we want to
participate
>>>>>>> to
>>>>>>> ceph
>>>>>>> effort by bringing a ceph cliente for windows.
>>>>>>
>>>>>> Awesome!
>>>>>>
>>>>>>> Our experience shows us that the best gateway is each
clientes being
>>>>>>> its
>>>>>>> own
>>>>>>> gateway, instead of having a bottle neck server or a cluster of
>>>>>>> bottle
>>>>>>> neck
>>>>>>> servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
>>>>>>>
>>>>>>> We already did some research in that domain.
>>>>>>>
>>>>>>> Dokan FS is an intent to write an opensource fuse like
cliente for
>>>>>>> MS
>>>>>>> windows.
>>>>>>>
>>>>>>> More information on DOKANFS can be triggered here
>>>>>>> http://dokan-dev.net/en/download/
>>>>>>>
>>>>>>> Positive points of using DOKANFS.
>>>>>>>
>>>>>>> - its opensourced and well licenced mit licence, gpl
licence and lgpl
>>>>>>> licence.
>>>>>>>
>>>>>>> Negative point of using DOKAN FS.
>>>>>>> - unreachable author
>>>>>>> - Poor documentation . Dev comments in japanese.
>>>>>>> - Work in progress so it is unstable and needs to be updated,
>>>>>>> debugged and
>>>>>>> maintained by a MS Windows file system expert developper.
>>>>>>
>>>>>> I am not very familiar with windows storage APIs, but
somebody told me
>>>>>> at once point there were several interfaces against which a
new file
>>>>>> system could be implemented, everything from a full
in-kernel driver
>>>>>> to
>>>>>> something that is explorer-based. Are any of those
suitable? Using a
>>>>>> potentially abandoned fuse-like layer makes me a bit nervous.
>>>>>>
>>>>>> That said,
>>>>>>
>>>>>>>
>>>>>>> I try past year to do a merge from ceph-fuse to dokanfs
>>>>>>> here are what I learnt.
>>>>>>> - Ceph-fuse and related source code is around 60 000 lines
of code.
>>>>>>> - Ceph protocol isn t documented so it is like trying to
draw a map
>>>>>>> of
>>>>>>> america
>>>>>>> using only a sextan and a compass.
>>>>>>>
>>>>>>> Those led me to those conclusions:
>>>>>>> - I can t do it alone.
>>>>>>> - It is easier to draw down the ceph protocol way to work from
>>>>>>> kernel/fs/ceph
>>>>>>> sources and mount.ceph
>>>>>>> - Ceph depending libraries may be unexistant or not up to
date in
>>>>>>> their
>>>>>>> port
>>>>>>> on MS Windows (cygwin)
>>>>>>
>>>>>> I think the most sane path should be to make libcephfs
sufficiently
>>>>>> portable to build on windows (or cygwin). For the bits used
by the
>>>>>> client-side coe, I don't think there should be much in the
way of
>>>>>> dependencies, and the main challenge would be untangling the
build for
>>>>>> the necessary pieces out from the rest of Ceph.
>>>>>>
>>>>>> Have you seen the wip-port portability work that is
currently underway
>>>>>> by
>>>>>> Noah and Alan? That may solve many of the cygwin problems
you are
>>>>>> seeing
>>>>>> today.
>>>>>>
>>>>>>> - MS file system specialist are hard do find in the "open
source
>>>>>>> libre
>>>>>>> world"
>>>>>>> so I will try in the commercial world.
>>>>>>>
>>>>>>> The commercial world has some problems too. They need ceph
protocol
>>>>>>> draft
>>>>>>> to
>>>>>>> implemente it to their own product They will have licencing
>>>>>>> /commercial
>>>>>>> politics that infringe lgpl, and hide that most of the work
is done
>>>>>>> by
>>>>>>> people
>>>>>>> other than them. They will not participate in a financial
way to ceph
>>>>>>> enhancement and growth.
>>>>>>
>>>>>> I don't think reimplementing the client code is an efficient way
>>>>>> forward.
>>>>>> Unless the goal is a pure kernel implementation...but a
significant
>>>>>> ongoing investment in development resources would be needed
for that
>>>>>> going
>>>>>> forward. I suspect that is a challenge for a platform that
does not
>>>>>> typically rally that sort of community effort.
>>>>>>
>>>>>> The easiest thing is of course just to use CIFS and Samba
(which works
>>>>>> today). A fuse-like approach is probably a reasonably
middle ground
>>>>>> (both
>>>>>> in initial effort and maintainability going forward)...
>>>>>>
>>>>>> sage
>>>>>>
>>>>>>
>>>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe
ceph-devel" in
>>> the body of a message to majordomo@vger.kernel.org
<mailto:majordomo@vger.kernel.org>
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>>
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2013-11-07 12:13 ` writing a ceph cliente for MS windows Alphe Salas Michels
@ 2013-11-07 14:29 ` Ketor D
2013-11-07 14:50 ` Matt W. Benjamin
0 siblings, 1 reply; 21+ messages in thread
From: Ketor D @ 2013-11-07 14:29 UTC (permalink / raw)
To: Alphe Salas Michels; +Cc: ceph-devel
Hi Alphe:
Yes Callback Filesystem is very expensive and can't open source.
It's not a good choice for ceph4win.
Another way for ceph4win maybe develop a kernel-mode fs like
pnfs. pnfs has a kernel-mode windows client. I think you can read its
src code and maybe migrating from ceph kernel client to windows kernel
fs is easier than from userspace ceph fuse client.And a kernel-mode fs
client has greater performance than userspace fs like ceph-fuse client
and ceph kernel client.
Regards.
On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas Michels <asalas@kepler.cl> wrote:
>
> Commercial libraries are a pain ...
>
> If we want the more permossive licence offered by callback file system we
> have to buy it for 20.000 usd. Then we will have to provide a backbox that
> we have no control upon and that will kill our product anytime they want anf
> if they decide to stop their commercial activity we will be in the same
> situation that with dokanfs but without having the source code of the black
> box. If i have to spend 20 000 dollars i would prefere paying someone to
> retake dokanfs or to write from scratch a dokanfs fuselike software make it
> all shiny and pumpy fantastic and ready to plug to ceph client.
>
> I would prefere if people have to pay something to get access to ceph4win
> that this money goes in ceph main branch pockets... Or as a gift you donante
> to ceph 10 dollars you get 2 free registration codes for ceph4win... or
> something like that.
>
> If ceph4win as to be comercial then I would prefer delegate the task to a
> company like south river technologies and their great product webdrive. I
> would mininaly get involved in that project and simply buy the final product
> to sell it together with my ceph based product (which could be a calxeda
> ceph box or something like that).
>
> I m open anyway to any proposition. But I doubt that callback filesystem
> offers us a suitable solution in the way I see ceph4win to be spread and
> used... I m maybe wrong. And anything that will be done around ceph4win will
> be public documented etc... And licensed the way that if someone want to
> build a commercial solution on top of it, that would be a possibility.
>
> My idea is to giveback somehow to ceph project and at same time forge a
> better knowledge in ceph technologies. Because like many in libre world I
> think the business is in the services around the software more than on the
> software. That the ones writing code should be financed and benefits from
> the one selling and giving support of the software at all levels. I m
> probably too idealistic. And too optimistic after all I m the one saying I
> will do this stuff I have no idea how but well it is interesting and fun so
> lets do it.
>
> Regards,
>
> P.S: using commercial backend libraries appart including their own cost will
> force you to use commercial IDE like MS VisualStudio because their library
> has some kind of drm that only that IDE compiler can use. So alot of cost
> and yet there is nothing done. If I had to open a kickstarter project saying
> we need 60 000 USD to do ceph4win with that monney we will buy the right to
> use and share a commercial copyrighted library but abandonned punctually to
> us in public domaine and that we will eventually produce something out of
> it. I doubt I will get a dollar.
>
> We still can suggest the idea to Edlos the commercial company that has the
> copyright of Callback FS, Or to buy them their product in a blender way
> (blender was bought with donation before being put opensource and public
> domaine), Or to open source their library. But in commercial minds
> opensourcing = death of their technical advantage and death of their
> marketing strategy. They will have to invent something more to retrieve
> monney from it.
>
> El nov 6, 2013 11:22 p.m., "Ketor D" <d.ketor@gmail.com
> <mailto:d.ketor@gmail.com>> escribió:
>
>
> Hi Alphe,
> I think you could try Callback Filesystem dev framework. It
> is a commerical dev framework and is maintained by Edlos today.
> I have communicated with Edlos to get a try code for
> development. To dokan, Callback Filesystem has vary document and maybe
> more stabilize.
>
> Regards.
>
>
>
> On Thu, Nov 7, 2013 at 10:00 AM, Alphe Salas <asalas@kepler.cl
> <mailto:asalas@kepler.cl>> wrote:
> > Hello ketor thank you for your interest un ceph4win. Since muy
> first mail I
> > exposed the lacks of dokanfs and that I m far from being a
> specialist un
> > filesystems.
> > I exposed what i like un dokanfs bit I not a fanátic of it. Muy
> goal is to
> > have something working quickly.
> >
> > So I am up to any proposición sure the one with the more docs and
> support
> > will be the best choice. As for right now what I need is
> understand what are
> > the files involved what are the interfaces functions and what are
> the needed
> > library dependencies and if they exist ported to windows with
> cygwin. And
> > all that is retrieved from source code.
> >
> > Regards.
> >
> > El nov 6, 2013 10:34 p.m., "Ketor D" <d.ketor@gmail.com
> <mailto:d.ketor@gmail.com>> escribió:
>
> >
> >> Hi Alphe,
> >> We are taking an interest in your work on Ceph Client for
> Windows
> >> with Dokan.As we know, the performance of Dokan is not very
> good, and it's
> >> abandoned 3 years ago.
> >> I have learned and used OpenDedup(SDFS) for a long time.
> OpenDedup
> >> has a Dokan version. And the author of OpenDedup said
> >>
> >> The Dokan library is quite flakey and testing should be
> performed before
> >> putting into production
> >>
> >> So what do you think about this? And if there is another
> solution of
> >> fuse-like filesystem dev framwork on Windows?
> >>
> >> Best Wish!
> >>
> >>
> >>
> >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe Salas Michels
> <asalas@kepler.cl <mailto:asalas@kepler.cl>>
>
> >> wrote:
> >>>
> >>> Hello I created the github repository for this project
> >>> https://github.com/alphe/Ceph4Win
> >>>
> >>> Regards,
> >>>
> >>> signature
> >>>
> >>> *Alphé Salas*
> >>> Ingeniero T.I
> >>>
> >>> asalas@kepler.cl <mailto:asalas@kepler.cl>
>
> >>> *<http://www.kepler.cl>*
> >>>
> >>> On 11/05/13 21:00, Sage Weil wrote:
> >>>>
> >>>> Hi Alphe,
> >>>>
> >>>> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
> >>>>>
> >>>>> signature *Hi, Sage !
> >>>>> thank you for you enthousiast reply.
> >>>>> I sure want to make the best use of everything or anything
> previously
> >>>>> done to
> >>>>> tend to
> >>>>> write ceph cliente for windows.
> >>>>>
> >>>>> Apart using libre tools for building the future ceph cliente
> I am open
> >>>>> to
> >>>>> anything.
> >>>>> I would recommand eclipse CDT or Code::BLocks they are based
> on mingwin
> >>>>> open
> >>>>> and easyly enhanceable.**
> >>>>>
> >>>>> more free tools can be found here:
> >>>>> http://www.freebyte.com/programming/cpp/#cppcompilers
> >>>>>
> >>>>>
> >>>>> I will read libcephfs source code and take some notes about the
> >>>>> protocol.
> >>>>
> >>>> I think you don't need to worry about hte protocol at all, since
> >>>> libcephs
> >>>> implements it for you (and will capture any future changes).
> >>>>
> >>>>> I was more going from what I know and trying to track down how
> >>>>> mount.ceph work
> >>>>> with the parameters passed to it.
> >>>>> since it point finally to Kernel/fs/ceph and that I don t really
> >>>>> understand
> >>>>> how that module work and that it probably points to some other
> >>>>> dependencies
> >>>>> Reading libcephfs source code could be a big gain of time.
> >>>>
> >>>> (I would also ignore mount.ceph as everything it does it
> specific to
> >>>> how Linux mounts work.)
> >>>>
> >>>>> basically on the protocol what is need are:
> >>>>>
> >>>>> 1) open and maintain a connection (socket open, auth, etc )
> >>>>> 2) retreive a map of directories and disk Quota (disk sizing
> Y TB free,
> >>>>> Z TB
> >>>>> total)
> >>>>> 3) procedure to send files / directories in a maner that it
> will allow
> >>>>> our
> >>>>> client to fit ceph transmission protocols
> >>>>> (limit bandwith for stability?, limit connection amount?,
> limit cpu
> >>>>> use?,
> >>>>> Cache for preparing data transfer (a FIFO cache)?)
> >>>>> 4)Procedure to retreive files / directory from ceph cluster
> >>>>> 5) Management copy/move files /Directories, FS stats,
> Connection Stats.
> >>>>> logging.
> >>>>>
> >>>>> My idea to progress is to take those main bulletpoint in ceph
> protocol
> >>>>> based
> >>>>> on general ideas of what ceph file system does and start
> identifying
> >>>>> parts
> >>>>> from libcephfs to match those "needs".
> >>>>
> >>>> Instead, I would look at include/cephfs/libcephfs.h, the
> interface that
> >>>> libcephfs provides, and try to map that to what the fuse layer
> expects.
> >>>> There is both a path-based that I suspsect lends itself well
> to the
> >>>> Windows interface and (very soon now) a handle based API that is
> >>>> targetted
> >>>> at the Unix-style VFS layers. I'm mostly guessing, though,
> since I've
> >>>> never seen any low-level fs code in windows before.
> >>>>
> >>>> In this case, the analogous code for Linux should be
> client/fuse_ll.cc
> >>>> itself (and not much else), although there will probably be a
> few tricks
> >>>> necessary to map cleanly onto how the windows interfaces work.
> >>>>
> >>>> Does that make sense?
> >>>>
> >>>> Cheers!
> >>>> sage
> >>>>
> >>>>
> >>>>> Any suggestion and contributions are welcome.
> >>>>>
> >>>>>
> >>>>> *
> >>>>> On 11/05/13 11:23, Sage Weil wrote:
> >>>>>>
> >>>>>> Hi Alphe,
> >>>>>>
> >>>>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
> >>>>>>>
> >>>>>>> Good day developers!
> >>>>>>>
> >>>>>>> I would like to propose to the one interested work with me to
> >>>>>>> develop a
> >>>>>>> ceph
> >>>>>>> cliente for MS windows world, Basing us on dokanFS.
> >>>>>>>
> >>>>>>> My company is a ceph enthousiast that use on a dayly basis
> ceph and
> >>>>>>> that
> >>>>>>> need
> >>>>>>> both transfer speed and big expendable and cheap storage.
> >>>>>>> My company is specialised in data recovery and we want to
> participate
> >>>>>>> to
> >>>>>>> ceph
> >>>>>>> effort by bringing a ceph cliente for windows.
> >>>>>>
> >>>>>> Awesome!
> >>>>>>
> >>>>>>> Our experience shows us that the best gateway is each
> clientes being
> >>>>>>> its
> >>>>>>> own
> >>>>>>> gateway, instead of having a bottle neck server or a cluster of
> >>>>>>> bottle
> >>>>>>> neck
> >>>>>>> servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
> >>>>>>>
> >>>>>>> We already did some research in that domain.
> >>>>>>>
> >>>>>>> Dokan FS is an intent to write an opensource fuse like
> cliente for
> >>>>>>> MS
> >>>>>>> windows.
> >>>>>>>
> >>>>>>> More information on DOKANFS can be triggered here
> >>>>>>> http://dokan-dev.net/en/download/
> >>>>>>>
> >>>>>>> Positive points of using DOKANFS.
> >>>>>>>
> >>>>>>> - its opensourced and well licenced mit licence, gpl
> licence and lgpl
> >>>>>>> licence.
> >>>>>>>
> >>>>>>> Negative point of using DOKAN FS.
> >>>>>>> - unreachable author
> >>>>>>> - Poor documentation . Dev comments in japanese.
> >>>>>>> - Work in progress so it is unstable and needs to be updated,
> >>>>>>> debugged and
> >>>>>>> maintained by a MS Windows file system expert developper.
> >>>>>>
> >>>>>> I am not very familiar with windows storage APIs, but
> somebody told me
> >>>>>> at once point there were several interfaces against which a
> new file
> >>>>>> system could be implemented, everything from a full
> in-kernel driver
> >>>>>> to
> >>>>>> something that is explorer-based. Are any of those
> suitable? Using a
> >>>>>> potentially abandoned fuse-like layer makes me a bit nervous.
> >>>>>>
> >>>>>> That said,
> >>>>>>
> >>>>>>>
> >>>>>>> I try past year to do a merge from ceph-fuse to dokanfs
> >>>>>>> here are what I learnt.
> >>>>>>> - Ceph-fuse and related source code is around 60 000 lines
> of code.
> >>>>>>> - Ceph protocol isn t documented so it is like trying to
> draw a map
> >>>>>>> of
> >>>>>>> america
> >>>>>>> using only a sextan and a compass.
> >>>>>>>
> >>>>>>> Those led me to those conclusions:
> >>>>>>> - I can t do it alone.
> >>>>>>> - It is easier to draw down the ceph protocol way to work from
> >>>>>>> kernel/fs/ceph
> >>>>>>> sources and mount.ceph
> >>>>>>> - Ceph depending libraries may be unexistant or not up to
> date in
> >>>>>>> their
> >>>>>>> port
> >>>>>>> on MS Windows (cygwin)
> >>>>>>
> >>>>>> I think the most sane path should be to make libcephfs
> sufficiently
> >>>>>> portable to build on windows (or cygwin). For the bits used
> by the
> >>>>>> client-side coe, I don't think there should be much in the
> way of
> >>>>>> dependencies, and the main challenge would be untangling the
> build for
> >>>>>> the necessary pieces out from the rest of Ceph.
> >>>>>>
> >>>>>> Have you seen the wip-port portability work that is
> currently underway
> >>>>>> by
> >>>>>> Noah and Alan? That may solve many of the cygwin problems
> you are
> >>>>>> seeing
> >>>>>> today.
> >>>>>>
> >>>>>>> - MS file system specialist are hard do find in the "open
> source
> >>>>>>> libre
> >>>>>>> world"
> >>>>>>> so I will try in the commercial world.
> >>>>>>>
> >>>>>>> The commercial world has some problems too. They need ceph
> protocol
> >>>>>>> draft
> >>>>>>> to
> >>>>>>> implemente it to their own product They will have licencing
> >>>>>>> /commercial
> >>>>>>> politics that infringe lgpl, and hide that most of the work
> is done
> >>>>>>> by
> >>>>>>> people
> >>>>>>> other than them. They will not participate in a financial
> way to ceph
> >>>>>>> enhancement and growth.
> >>>>>>
> >>>>>> I don't think reimplementing the client code is an efficient way
> >>>>>> forward.
> >>>>>> Unless the goal is a pure kernel implementation...but a
> significant
> >>>>>> ongoing investment in development resources would be needed
> for that
> >>>>>> going
> >>>>>> forward. I suspect that is a challenge for a platform that
> does not
> >>>>>> typically rally that sort of community effort.
> >>>>>>
> >>>>>> The easiest thing is of course just to use CIFS and Samba
> (which works
> >>>>>> today). A fuse-like approach is probably a reasonably
> middle ground
> >>>>>> (both
> >>>>>> in initial effort and maintainability going forward)...
> >>>>>>
> >>>>>> sage
> >>>>>>
> >>>>>>
> >>>>>
> >>>
> >>> --
> >>> To unsubscribe from this list: send the line "unsubscribe
> ceph-devel" in
> >>> the body of a message to majordomo@vger.kernel.org
> <mailto:majordomo@vger.kernel.org>
>
> >>> More majordomo info at http://vger.kernel.org/majordomo-info.html
> >>
> >>
> >
>
>
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2013-11-07 14:29 ` Ketor D
@ 2013-11-07 14:50 ` Matt W. Benjamin
2013-11-07 18:02 ` Alphe Salas Michels
0 siblings, 1 reply; 21+ messages in thread
From: Matt W. Benjamin @ 2013-11-07 14:50 UTC (permalink / raw)
To: Ketor D; +Cc: ceph-devel, Alphe Salas Michels
Hi,
The Window NFS v4.1 client is what we work on, so this may be good for
code sharing. The license is lgplv2, like Ceph's.
Something important to be aware of is that the client uses rdbss, which
is a (partial) fsd abstraction that simplified implementation
quite a bit, kind of like a mini driver. However, Microsoft's support
for rdbss has been in limbo for a bit. For example, to link with
the rdbss symbols you can't use the Windows 8 driver kit--you'll need
to use the one for Windows 7. (There's a private rdbss2 used internally
by Microsoft's SMB implemenation. A the moment, 3rd party drivers
can't use that.)
We've been in communication with Microsoft about this issue, and know of
a few other fsds using it, but it could be a good thing for that lobbying
effort to have another user--or it could be a dead end :(.
There are a couple of other choices if you're looking to go this route,
that I'm aware of (and we may need to take them too, if RDBSS has no
way forward), but the required work could be a lot larger.
Matt
----- "Ketor D" <d.ketor@gmail.com> wrote:
> Hi Alphe:
> Yes Callback Filesystem is very expensive and can't open
> source.
> It's not a good choice for ceph4win.
> Another way for ceph4win maybe develop a kernel-mode fs like
> pnfs. pnfs has a kernel-mode windows client. I think you can read its
> src code and maybe migrating from ceph kernel client to windows
> kernel
> fs is easier than from userspace ceph fuse client.And a kernel-mode
> fs
> client has greater performance than userspace fs like ceph-fuse
> client
> and ceph kernel client.
>
> Regards.
>
>
>
> On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas Michels <asalas@kepler.cl>
> wrote:
> >
> > Commercial libraries are a pain ...
> >
> > If we want the more permossive licence offered by callback file
> system we
> > have to buy it for 20.000 usd. Then we will have to provide a
> backbox that
> > we have no control upon and that will kill our product anytime they
> want anf
> > if they decide to stop their commercial activity we will be in the
> same
> > situation that with dokanfs but without having the source code of
> the black
> > box. If i have to spend 20 000 dollars i would prefere paying
> someone to
> > retake dokanfs or to write from scratch a dokanfs fuselike software
> make it
> > all shiny and pumpy fantastic and ready to plug to ceph client.
> >
> > I would prefere if people have to pay something to get access to
> ceph4win
> > that this money goes in ceph main branch pockets... Or as a gift you
> donante
> > to ceph 10 dollars you get 2 free registration codes for
> ceph4win... or
> > something like that.
> >
> > If ceph4win as to be comercial then I would prefer delegate the task
> to a
> > company like south river technologies and their great product
> webdrive. I
> > would mininaly get involved in that project and simply buy the final
> product
> > to sell it together with my ceph based product (which could be a
> calxeda
> > ceph box or something like that).
> >
> > I m open anyway to any proposition. But I doubt that callback
> filesystem
> > offers us a suitable solution in the way I see ceph4win to be spread
> and
> > used... I m maybe wrong. And anything that will be done around
> ceph4win will
> > be public documented etc... And licensed the way that if someone
> want to
> > build a commercial solution on top of it, that would be a
> possibility.
> >
> > My idea is to giveback somehow to ceph project and at same time
> forge a
> > better knowledge in ceph technologies. Because like many in libre
> world I
> > think the business is in the services around the software more than
> on the
> > software. That the ones writing code should be financed and benefits
> from
> > the one selling and giving support of the software at all levels. I
> m
> > probably too idealistic. And too optimistic after all I m the one
> saying I
> > will do this stuff I have no idea how but well it is interesting and
> fun so
> > lets do it.
> >
> > Regards,
> >
> > P.S: using commercial backend libraries appart including their own
> cost will
> > force you to use commercial IDE like MS VisualStudio because their
> library
> > has some kind of drm that only that IDE compiler can use. So alot of
> cost
> > and yet there is nothing done. If I had to open a kickstarter
> project saying
> > we need 60 000 USD to do ceph4win with that monney we will buy the
> right to
> > use and share a commercial copyrighted library but abandonned
> punctually to
> > us in public domaine and that we will eventually produce something
> out of
> > it. I doubt I will get a dollar.
> >
> > We still can suggest the idea to Edlos the commercial company that
> has the
> > copyright of Callback FS, Or to buy them their product in a blender
> way
> > (blender was bought with donation before being put opensource and
> public
> > domaine), Or to open source their library. But in commercial minds
> > opensourcing = death of their technical advantage and death of
> their
> > marketing strategy. They will have to invent something more to
> retrieve
> > monney from it.
> >
> > El nov 6, 2013 11:22 p.m., "Ketor D" <d.ketor@gmail.com
> > <mailto:d.ketor@gmail.com>> escribió:
> >
> >
> > Hi Alphe,
> > I think you could try Callback Filesystem dev
> framework. It
> > is a commerical dev framework and is maintained by Edlos today.
> > I have communicated with Edlos to get a try code for
> > development. To dokan, Callback Filesystem has vary document and
> maybe
> > more stabilize.
> >
> > Regards.
> >
> >
> >
> > On Thu, Nov 7, 2013 at 10:00 AM, Alphe Salas <asalas@kepler.cl
> > <mailto:asalas@kepler.cl>> wrote:
> > > Hello ketor thank you for your interest un ceph4win. Since
> muy
> > first mail I
> > > exposed the lacks of dokanfs and that I m far from being a
> > specialist un
> > > filesystems.
> > > I exposed what i like un dokanfs bit I not a fanátic of it.
> Muy
> > goal is to
> > > have something working quickly.
> > >
> > > So I am up to any proposición sure the one with the more docs
> and
> > support
> > > will be the best choice. As for right now what I need is
> > understand what are
> > > the files involved what are the interfaces functions and what
> are
> > the needed
> > > library dependencies and if they exist ported to windows with
> > cygwin. And
> > > all that is retrieved from source code.
> > >
> > > Regards.
> > >
> > > El nov 6, 2013 10:34 p.m., "Ketor D" <d.ketor@gmail.com
> > <mailto:d.ketor@gmail.com>> escribió:
> >
> > >
> > >> Hi Alphe,
> > >> We are taking an interest in your work on Ceph Client
> for
> > Windows
> > >> with Dokan.As we know, the performance of Dokan is not very
> > good, and it's
> > >> abandoned 3 years ago.
> > >> I have learned and used OpenDedup(SDFS) for a long
> time.
> > OpenDedup
> > >> has a Dokan version. And the author of OpenDedup said
> > >>
> > >> The Dokan library is quite flakey and testing should be
> > performed before
> > >> putting into production
> > >>
> > >> So what do you think about this? And if there is
> another
> > solution of
> > >> fuse-like filesystem dev framwork on Windows?
> > >>
> > >> Best Wish!
> > >>
> > >>
> > >>
> > >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe Salas Michels
> > <asalas@kepler.cl <mailto:asalas@kepler.cl>>
> >
> > >> wrote:
> > >>>
> > >>> Hello I created the github repository for this project
> > >>> https://github.com/alphe/Ceph4Win
> > >>>
> > >>> Regards,
> > >>>
> > >>> signature
> > >>>
> > >>> *Alphé Salas*
> > >>> Ingeniero T.I
> > >>>
> > >>> asalas@kepler.cl <mailto:asalas@kepler.cl>
> >
> > >>> *<http://www.kepler.cl>*
> > >>>
> > >>> On 11/05/13 21:00, Sage Weil wrote:
> > >>>>
> > >>>> Hi Alphe,
> > >>>>
> > >>>> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
> > >>>>>
> > >>>>> signature *Hi, Sage !
> > >>>>> thank you for you enthousiast reply.
> > >>>>> I sure want to make the best use of everything or
> anything
> > previously
> > >>>>> done to
> > >>>>> tend to
> > >>>>> write ceph cliente for windows.
> > >>>>>
> > >>>>> Apart using libre tools for building the future ceph
> cliente
> > I am open
> > >>>>> to
> > >>>>> anything.
> > >>>>> I would recommand eclipse CDT or Code::BLocks they are
> based
> > on mingwin
> > >>>>> open
> > >>>>> and easyly enhanceable.**
> > >>>>>
> > >>>>> more free tools can be found here:
> > >>>>> http://www.freebyte.com/programming/cpp/#cppcompilers
> > >>>>>
> > >>>>>
> > >>>>> I will read libcephfs source code and take some notes
> about the
> > >>>>> protocol.
> > >>>>
> > >>>> I think you don't need to worry about hte protocol at all,
> since
> > >>>> libcephs
> > >>>> implements it for you (and will capture any future
> changes).
> > >>>>
> > >>>>> I was more going from what I know and trying to track down
> how
> > >>>>> mount.ceph work
> > >>>>> with the parameters passed to it.
> > >>>>> since it point finally to Kernel/fs/ceph and that I don t
> really
> > >>>>> understand
> > >>>>> how that module work and that it probably points to some
> other
> > >>>>> dependencies
> > >>>>> Reading libcephfs source code could be a big gain of
> time.
> > >>>>
> > >>>> (I would also ignore mount.ceph as everything it does it
> > specific to
> > >>>> how Linux mounts work.)
> > >>>>
> > >>>>> basically on the protocol what is need are:
> > >>>>>
> > >>>>> 1) open and maintain a connection (socket open, auth, etc
> )
> > >>>>> 2) retreive a map of directories and disk Quota (disk
> sizing
> > Y TB free,
> > >>>>> Z TB
> > >>>>> total)
> > >>>>> 3) procedure to send files / directories in a maner that
> it
> > will allow
> > >>>>> our
> > >>>>> client to fit ceph transmission protocols
> > >>>>> (limit bandwith for stability?, limit connection amount?,
> > limit cpu
> > >>>>> use?,
> > >>>>> Cache for preparing data transfer (a FIFO cache)?)
> > >>>>> 4)Procedure to retreive files / directory from ceph
> cluster
> > >>>>> 5) Management copy/move files /Directories, FS stats,
> > Connection Stats.
> > >>>>> logging.
> > >>>>>
> > >>>>> My idea to progress is to take those main bulletpoint in
> ceph
> > protocol
> > >>>>> based
> > >>>>> on general ideas of what ceph file system does and start
> > identifying
> > >>>>> parts
> > >>>>> from libcephfs to match those "needs".
> > >>>>
> > >>>> Instead, I would look at include/cephfs/libcephfs.h, the
> > interface that
> > >>>> libcephfs provides, and try to map that to what the fuse
> layer
> > expects.
> > >>>> There is both a path-based that I suspsect lends itself
> well
> > to the
> > >>>> Windows interface and (very soon now) a handle based API
> that is
> > >>>> targetted
> > >>>> at the Unix-style VFS layers. I'm mostly guessing,
> though,
> > since I've
> > >>>> never seen any low-level fs code in windows before.
> > >>>>
> > >>>> In this case, the analogous code for Linux should be
> > client/fuse_ll.cc
> > >>>> itself (and not much else), although there will probably be
> a
> > few tricks
> > >>>> necessary to map cleanly onto how the windows interfaces
> work.
> > >>>>
> > >>>> Does that make sense?
> > >>>>
> > >>>> Cheers!
> > >>>> sage
> > >>>>
> > >>>>
> > >>>>> Any suggestion and contributions are welcome.
> > >>>>>
> > >>>>>
> > >>>>> *
> > >>>>> On 11/05/13 11:23, Sage Weil wrote:
> > >>>>>>
> > >>>>>> Hi Alphe,
> > >>>>>>
> > >>>>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
> > >>>>>>>
> > >>>>>>> Good day developers!
> > >>>>>>>
> > >>>>>>> I would like to propose to the one interested work with
> me to
> > >>>>>>> develop a
> > >>>>>>> ceph
> > >>>>>>> cliente for MS windows world, Basing us on dokanFS.
> > >>>>>>>
> > >>>>>>> My company is a ceph enthousiast that use on a dayly
> basis
> > ceph and
> > >>>>>>> that
> > >>>>>>> need
> > >>>>>>> both transfer speed and big expendable and cheap
> storage.
> > >>>>>>> My company is specialised in data recovery and we want
> to
> > participate
> > >>>>>>> to
> > >>>>>>> ceph
> > >>>>>>> effort by bringing a ceph cliente for windows.
> > >>>>>>
> > >>>>>> Awesome!
> > >>>>>>
> > >>>>>>> Our experience shows us that the best gateway is each
> > clientes being
> > >>>>>>> its
> > >>>>>>> own
> > >>>>>>> gateway, instead of having a bottle neck server or a
> cluster of
> > >>>>>>> bottle
> > >>>>>>> neck
> > >>>>>>> servers as gateway (FTP, samba, SFTP,webdav, s3,
> etc..).
> > >>>>>>>
> > >>>>>>> We already did some research in that domain.
> > >>>>>>>
> > >>>>>>> Dokan FS is an intent to write an opensource fuse like
> > cliente for
> > >>>>>>> MS
> > >>>>>>> windows.
> > >>>>>>>
> > >>>>>>> More information on DOKANFS can be triggered here
> > >>>>>>> http://dokan-dev.net/en/download/
> > >>>>>>>
> > >>>>>>> Positive points of using DOKANFS.
> > >>>>>>>
> > >>>>>>> - its opensourced and well licenced mit licence, gpl
> > licence and lgpl
> > >>>>>>> licence.
> > >>>>>>>
> > >>>>>>> Negative point of using DOKAN FS.
> > >>>>>>> - unreachable author
> > >>>>>>> - Poor documentation . Dev comments in japanese.
> > >>>>>>> - Work in progress so it is unstable and needs to be
> updated,
> > >>>>>>> debugged and
> > >>>>>>> maintained by a MS Windows file system expert
> developper.
> > >>>>>>
> > >>>>>> I am not very familiar with windows storage APIs, but
> > somebody told me
> > >>>>>> at once point there were several interfaces against which
> a
> > new file
> > >>>>>> system could be implemented, everything from a full
> > in-kernel driver
> > >>>>>> to
> > >>>>>> something that is explorer-based. Are any of those
> > suitable? Using a
> > >>>>>> potentially abandoned fuse-like layer makes me a bit
> nervous.
> > >>>>>>
> > >>>>>> That said,
> > >>>>>>
> > >>>>>>>
> > >>>>>>> I try past year to do a merge from ceph-fuse to dokanfs
> > >>>>>>> here are what I learnt.
> > >>>>>>> - Ceph-fuse and related source code is around 60 000
> lines
> > of code.
> > >>>>>>> - Ceph protocol isn t documented so it is like trying
> to
> > draw a map
> > >>>>>>> of
> > >>>>>>> america
> > >>>>>>> using only a sextan and a compass.
> > >>>>>>>
> > >>>>>>> Those led me to those conclusions:
> > >>>>>>> - I can t do it alone.
> > >>>>>>> - It is easier to draw down the ceph protocol way to
> work from
> > >>>>>>> kernel/fs/ceph
> > >>>>>>> sources and mount.ceph
> > >>>>>>> - Ceph depending libraries may be unexistant or not up
> to
> > date in
> > >>>>>>> their
> > >>>>>>> port
> > >>>>>>> on MS Windows (cygwin)
> > >>>>>>
> > >>>>>> I think the most sane path should be to make libcephfs
> > sufficiently
> > >>>>>> portable to build on windows (or cygwin). For the bits
> used
> > by the
> > >>>>>> client-side coe, I don't think there should be much in
> the
> > way of
> > >>>>>> dependencies, and the main challenge would be untangling
> the
> > build for
> > >>>>>> the necessary pieces out from the rest of Ceph.
> > >>>>>>
> > >>>>>> Have you seen the wip-port portability work that is
> > currently underway
> > >>>>>> by
> > >>>>>> Noah and Alan? That may solve many of the cygwin
> problems
> > you are
> > >>>>>> seeing
> > >>>>>> today.
> > >>>>>>
> > >>>>>>> - MS file system specialist are hard do find in the
> "open
> > source
> > >>>>>>> libre
> > >>>>>>> world"
> > >>>>>>> so I will try in the commercial world.
> > >>>>>>>
> > >>>>>>> The commercial world has some problems too. They need
> ceph
> > protocol
> > >>>>>>> draft
> > >>>>>>> to
> > >>>>>>> implemente it to their own product They will have
> licencing
> > >>>>>>> /commercial
> > >>>>>>> politics that infringe lgpl, and hide that most of the
> work
> > is done
> > >>>>>>> by
> > >>>>>>> people
> > >>>>>>> other than them. They will not participate in a
> financial
> > way to ceph
> > >>>>>>> enhancement and growth.
> > >>>>>>
> > >>>>>> I don't think reimplementing the client code is an
> efficient way
> > >>>>>> forward.
> > >>>>>> Unless the goal is a pure kernel implementation...but a
> > significant
> > >>>>>> ongoing investment in development resources would be
> needed
> > for that
> > >>>>>> going
> > >>>>>> forward. I suspect that is a challenge for a platform
> that
> > does not
> > >>>>>> typically rally that sort of community effort.
> > >>>>>>
> > >>>>>> The easiest thing is of course just to use CIFS and
> Samba
> > (which works
> > >>>>>> today). A fuse-like approach is probably a reasonably
> > middle ground
> > >>>>>> (both
> > >>>>>> in initial effort and maintainability going forward)...
> > >>>>>>
> > >>>>>> sage
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> > >>> --
> > >>> To unsubscribe from this list: send the line "unsubscribe
> > ceph-devel" in
> > >>> the body of a message to majordomo@vger.kernel.org
> > <mailto:majordomo@vger.kernel.org>
> >
> > >>> More majordomo info at
> http://vger.kernel.org/majordomo-info.html
> > >>
> > >>
> > >
> >
> >
> >
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI 48104
http://linuxbox.com
tel. 734-761-4689
fax. 734-769-8938
cel. 734-216-5309
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2013-11-07 14:50 ` Matt W. Benjamin
@ 2013-11-07 18:02 ` Alphe Salas Michels
2013-11-07 20:47 ` Alphe Salas Michels
0 siblings, 1 reply; 21+ messages in thread
From: Alphe Salas Michels @ 2013-11-07 18:02 UTC (permalink / raw)
To: Matt W. Benjamin, Ketor D; +Cc: ceph-devel
Hello D.Ketor and Matt Benjamin,
You give me alot to think about and this is great!
I merged your previous post to make a single reply that anyone can
report to easyly
Windows NFS 4.1 is available here:
http://www.citi.umich.edu/projects/nfsv4/windows/readme.html
pnfs is another name for NFS4.X. It is presented as alternative to ceph
and we get known terminology as MDS and OSD but without the self healing
part if I understand well my rapid look on the topic. (when I say rapid
look I mean ... 5 minutes spent in that... which is really small amount
of time to get an accurate view on something)
starting from mount.ceph ... I know that mount.ceph does little but it
is a great hint to know what ceph needs and do things.
Basically mount.ceph modprobe the ceph driver in the linux kernel then
call mount with the line command passed args and the cephfs type as
argument. Then the kernel does the work I don t understand yet what is
the start calls that are made to the ceph driver but it seemed to me
that is was relatively light. (a first impression compared to ceph-fuse.)
I think I will do both isolate source code from ceph-client kernel
(cephfs module for linux kernel) and the one pointed by Sage starting
from client/fuse_ll.cc in ceph master branch. The common files betwin
those 2 extractions will be our core set of mandatory features.
Then we try to compile with cygwin a cephfs client library . Then we
will try to interface with a modified windows nfs 4.1 client or pnfs or
any other that will accept to be compiled with gcc for win32...
the fact that windows 8.1 is and windows 2012 are out of reach at the
moment is not a problem to me.
Our first concern is to understand what is ceph protocol. Then adapt it
to something that can be used on windows prior windows 8.1. Dokan fs if
I remember well use too the WDK (windows driver dev-kit ) for it s
compilation so possibly we will see the same limitations.
We need to multiply our source of information by example regarding
ceph-client (kernel or fuse, radosgw is on a different layer so I will
not try anything around it at first.) And we need to multiply our source
of information by example regarding virtual file system technologies on
windoes OS.
Alot of work but all of those available source code everyone point at me
will make our best solution. And in the end we will choose technologies
knowing what we do and what concequencies they have.
regards,
Regards
signature
*Alphé Salas*
Ingeniero T.I
asalas@kepler.cl
On 11/07/13 11:29, Ketor D wrote:
> Hi Alphe:
> Yes Callback Filesystem is very expensive and can't open source.
> It's not a good choice for ceph4win.
> Another way for ceph4win maybe develop a kernel-mode fs like
> pnfs. pnfs has a kernel-mode windows client. I think you can read its
> src code and maybe migrating from ceph kernel client to windows kernel
> fs is easier than from userspace ceph fuse client.And a kernel-mode fs
> client has greater performance than userspace fs like ceph-fuse client
> and ceph kernel client.
>
> Regards.
>
On 11/07/13 11:50, Matt W. Benjamin wrote:
> Hi,
>
> The Window NFS v4.1 client is what we work on, so this may be good for
> code sharing. The license is lgplv2, like Ceph's.
>
> Something important to be aware of is that the client uses rdbss, which
> is a (partial) fsd abstraction that simplified implementation
> quite a bit, kind of like a mini driver. However, Microsoft's support
> for rdbss has been in limbo for a bit. For example, to link with
> the rdbss symbols you can't use the Windows 8 driver kit--you'll need
> to use the one for Windows 7. (There's a private rdbss2 used internally
> by Microsoft's SMB implemenation. A the moment, 3rd party drivers
> can't use that.)
>
> We've been in communication with Microsoft about this issue, and know of
> a few other fsds using it, but it could be a good thing for that lobbying
> effort to have another user--or it could be a dead end :(.
>
> There are a couple of other choices if you're looking to go this route,
> that I'm aware of (and we may need to take them too, if RDBSS has no
> way forward), but the required work could be a lot larger.
>
> Matt
>
> ----- "Ketor D"<d.ketor@gmail.com> wrote:
>
>> Hi Alphe:
>> Yes Callback Filesystem is very expensive and can't open
>> source.
>> It's not a good choice for ceph4win.
>> Another way for ceph4win maybe develop a kernel-mode fs like
>> pnfs. pnfs has a kernel-mode windows client. I think you can read its
>> src code and maybe migrating from ceph kernel client to windows
>> kernel
>> fs is easier than from userspace ceph fuse client.And a kernel-mode
>> fs
>> client has greater performance than userspace fs like ceph-fuse
>> client
>> and ceph kernel client.
>>
>> Regards.
>>
>>
>>
>> On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas Michels<asalas@kepler.cl>
>> wrote:
>>> Commercial libraries are a pain ...
>>>
>>> If we want the more permossive licence offered by callback file
>> system we
>>> have to buy it for 20.000 usd. Then we will have to provide a
>> backbox that
>>> we have no control upon and that will kill our product anytime they
>> want anf
>>> if they decide to stop their commercial activity we will be in the
>> same
>>> situation that with dokanfs but without having the source code of
>> the black
>>> box. If i have to spend 20 000 dollars i would prefere paying
>> someone to
>>> retake dokanfs or to write from scratch a dokanfs fuselike software
>> make it
>>> all shiny and pumpy fantastic and ready to plug to ceph client.
>>>
>>> I would prefere if people have to pay something to get access to
>> ceph4win
>>> that this money goes in ceph main branch pockets... Or as a gift you
>> donante
>>> to ceph 10 dollars you get 2 free registration codes for
>> ceph4win... or
>>> something like that.
>>>
>>> If ceph4win as to be comercial then I would prefer delegate the task
>> to a
>>> company like south river technologies and their great product
>> webdrive. I
>>> would mininaly get involved in that project and simply buy the final
>> product
>>> to sell it together with my ceph based product (which could be a
>> calxeda
>>> ceph box or something like that).
>>>
>>> I m open anyway to any proposition. But I doubt that callback
>> filesystem
>>> offers us a suitable solution in the way I see ceph4win to be spread
>> and
>>> used... I m maybe wrong. And anything that will be done around
>> ceph4win will
>>> be public documented etc... And licensed the way that if someone
>> want to
>>> build a commercial solution on top of it, that would be a
>> possibility.
>>> My idea is to giveback somehow to ceph project and at same time
>> forge a
>>> better knowledge in ceph technologies. Because like many in libre
>> world I
>>> think the business is in the services around the software more than
>> on the
>>> software. That the ones writing code should be financed and benefits
>> from
>>> the one selling and giving support of the software at all levels. I
>> m
>>> probably too idealistic. And too optimistic after all I m the one
>> saying I
>>> will do this stuff I have no idea how but well it is interesting and
>> fun so
>>> lets do it.
>>>
>>> Regards,
>>>
>>> P.S: using commercial backend libraries appart including their own
>> cost will
>>> force you to use commercial IDE like MS VisualStudio because their
>> library
>>> has some kind of drm that only that IDE compiler can use. So alot of
>> cost
>>> and yet there is nothing done. If I had to open a kickstarter
>> project saying
>>> we need 60 000 USD to do ceph4win with that monney we will buy the
>> right to
>>> use and share a commercial copyrighted library but abandonned
>> punctually to
>>> us in public domaine and that we will eventually produce something
>> out of
>>> it. I doubt I will get a dollar.
>>>
>>> We still can suggest the idea to Edlos the commercial company that
>> has the
>>> copyright of Callback FS, Or to buy them their product in a blender
>> way
>>> (blender was bought with donation before being put opensource and
>> public
>>> domaine), Or to open source their library. But in commercial minds
>>> opensourcing = death of their technical advantage and death of
>> their
>>> marketing strategy. They will have to invent something more to
>> retrieve
>>> monney from it.
>>>
>>> El nov 6, 2013 11:22 p.m., "Ketor D" <d.ketor@gmail.com
>>> <mailto:d.ketor@gmail.com>> escribió:
>>>
>>>
>>> Hi Alphe,
>>> I think you could try Callback Filesystem dev
>> framework. It
>>> is a commerical dev framework and is maintained by Edlos today.
>>> I have communicated with Edlos to get a try code for
>>> development. To dokan, Callback Filesystem has vary document and
>> maybe
>>> more stabilize.
>>>
>>> Regards.
>>>
>>>
>>>
>>> On Thu, Nov 7, 2013 at 10:00 AM, Alphe Salas <asalas@kepler.cl
>>> <mailto:asalas@kepler.cl>> wrote:
>>> > Hello ketor thank you for your interest un ceph4win. Since
>> muy
>>> first mail I
>>> > exposed the lacks of dokanfs and that I m far from being a
>>> specialist un
>>> > filesystems.
>>> > I exposed what i like un dokanfs bit I not a fanátic of it.
>> Muy
>>> goal is to
>>> > have something working quickly.
>>> >
>>> > So I am up to any proposición sure the one with the more docs
>> and
>>> support
>>> > will be the best choice. As for right now what I need is
>>> understand what are
>>> > the files involved what are the interfaces functions and what
>> are
>>> the needed
>>> > library dependencies and if they exist ported to windows with
>>> cygwin. And
>>> > all that is retrieved from source code.
>>> >
>>> > Regards.
>>> >
>>> > El nov 6, 2013 10:34 p.m., "Ketor D" <d.ketor@gmail.com
>>> <mailto:d.ketor@gmail.com>> escribió:
>>>
>>> >
>>> >> Hi Alphe,
>>> >> We are taking an interest in your work on Ceph Client
>> for
>>> Windows
>>> >> with Dokan.As we know, the performance of Dokan is not very
>>> good, and it's
>>> >> abandoned 3 years ago.
>>> >> I have learned and used OpenDedup(SDFS) for a long
>> time.
>>> OpenDedup
>>> >> has a Dokan version. And the author of OpenDedup said
>>> >>
>>> >> The Dokan library is quite flakey and testing should be
>>> performed before
>>> >> putting into production
>>> >>
>>> >> So what do you think about this? And if there is
>> another
>>> solution of
>>> >> fuse-like filesystem dev framwork on Windows?
>>> >>
>>> >> Best Wish!
>>> >>
>>> >>
>>> >>
>>> >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe Salas Michels
>>> <asalas@kepler.cl <mailto:asalas@kepler.cl>>
>>>
>>> >> wrote:
>>> >>>
>>> >>> Hello I created the github repository for this project
>>> >>>https://github.com/alphe/Ceph4Win
>>> >>>
>>> >>> Regards,
>>> >>>
>>> >>> signature
>>> >>>
>>> >>> *Alphé Salas*
>>> >>> Ingeniero T.I
>>> >>>
>>> >>>asalas@kepler.cl <mailto:asalas@kepler.cl>
>>>
>>> >>> *<http://www.kepler.cl>*
>>> >>>
>>> >>> On 11/05/13 21:00, Sage Weil wrote:
>>> >>>>
>>> >>>> Hi Alphe,
>>> >>>>
>>> >>>> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
>>> >>>>>
>>> >>>>> signature *Hi, Sage !
>>> >>>>> thank you for you enthousiast reply.
>>> >>>>> I sure want to make the best use of everything or
>> anything
>>> previously
>>> >>>>> done to
>>> >>>>> tend to
>>> >>>>> write ceph cliente for windows.
>>> >>>>>
>>> >>>>> Apart using libre tools for building the future ceph
>> cliente
>>> I am open
>>> >>>>> to
>>> >>>>> anything.
>>> >>>>> I would recommand eclipse CDT or Code::BLocks they are
>> based
>>> on mingwin
>>> >>>>> open
>>> >>>>> and easyly enhanceable.**
>>> >>>>>
>>> >>>>> more free tools can be found here:
>>> >>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
>>> >>>>>
>>> >>>>>
>>> >>>>> I will read libcephfs source code and take some notes
>> about the
>>> >>>>> protocol.
>>> >>>>
>>> >>>> I think you don't need to worry about hte protocol at all,
>> since
>>> >>>> libcephs
>>> >>>> implements it for you (and will capture any future
>> changes).
>>> >>>>
>>> >>>>> I was more going from what I know and trying to track down
>> how
>>> >>>>> mount.ceph work
>>> >>>>> with the parameters passed to it.
>>> >>>>> since it point finally to Kernel/fs/ceph and that I don t
>> really
>>> >>>>> understand
>>> >>>>> how that module work and that it probably points to some
>> other
>>> >>>>> dependencies
>>> >>>>> Reading libcephfs source code could be a big gain of
>> time.
>>> >>>>
>>> >>>> (I would also ignore mount.ceph as everything it does it
>>> specific to
>>> >>>> how Linux mounts work.)
>>> >>>>
>>> >>>>> basically on the protocol what is need are:
>>> >>>>>
>>> >>>>> 1) open and maintain a connection (socket open, auth, etc
>> )
>>> >>>>> 2) retreive a map of directories and disk Quota (disk
>> sizing
>>> Y TB free,
>>> >>>>> Z TB
>>> >>>>> total)
>>> >>>>> 3) procedure to send files / directories in a maner that
>> it
>>> will allow
>>> >>>>> our
>>> >>>>> client to fit ceph transmission protocols
>>> >>>>> (limit bandwith for stability?, limit connection amount?,
>>> limit cpu
>>> >>>>> use?,
>>> >>>>> Cache for preparing data transfer (a FIFO cache)?)
>>> >>>>> 4)Procedure to retreive files / directory from ceph
>> cluster
>>> >>>>> 5) Management copy/move files /Directories, FS stats,
>>> Connection Stats.
>>> >>>>> logging.
>>> >>>>>
>>> >>>>> My idea to progress is to take those main bulletpoint in
>> ceph
>>> protocol
>>> >>>>> based
>>> >>>>> on general ideas of what ceph file system does and start
>>> identifying
>>> >>>>> parts
>>> >>>>> from libcephfs to match those "needs".
>>> >>>>
>>> >>>> Instead, I would look at include/cephfs/libcephfs.h, the
>>> interface that
>>> >>>> libcephfs provides, and try to map that to what the fuse
>> layer
>>> expects.
>>> >>>> There is both a path-based that I suspsect lends itself
>> well
>>> to the
>>> >>>> Windows interface and (very soon now) a handle based API
>> that is
>>> >>>> targetted
>>> >>>> at the Unix-style VFS layers. I'm mostly guessing,
>> though,
>>> since I've
>>> >>>> never seen any low-level fs code in windows before.
>>> >>>>
>>> >>>> In this case, the analogous code for Linux should be
>>> client/fuse_ll.cc
>>> >>>> itself (and not much else), although there will probably be
>> a
>>> few tricks
>>> >>>> necessary to map cleanly onto how the windows interfaces
>> work.
>>> >>>>
>>> >>>> Does that make sense?
>>> >>>>
>>> >>>> Cheers!
>>> >>>> sage
>>> >>>>
>>> >>>>
>>> >>>>> Any suggestion and contributions are welcome.
>>> >>>>>
>>> >>>>>
>>> >>>>> *
>>> >>>>> On 11/05/13 11:23, Sage Weil wrote:
>>> >>>>>>
>>> >>>>>> Hi Alphe,
>>> >>>>>>
>>> >>>>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
>>> >>>>>>>
>>> >>>>>>> Good day developers!
>>> >>>>>>>
>>> >>>>>>> I would like to propose to the one interested work with
>> me to
>>> >>>>>>> develop a
>>> >>>>>>> ceph
>>> >>>>>>> cliente for MS windows world, Basing us on dokanFS.
>>> >>>>>>>
>>> >>>>>>> My company is a ceph enthousiast that use on a dayly
>> basis
>>> ceph and
>>> >>>>>>> that
>>> >>>>>>> need
>>> >>>>>>> both transfer speed and big expendable and cheap
>> storage.
>>> >>>>>>> My company is specialised in data recovery and we want
>> to
>>> participate
>>> >>>>>>> to
>>> >>>>>>> ceph
>>> >>>>>>> effort by bringing a ceph cliente for windows.
>>> >>>>>>
>>> >>>>>> Awesome!
>>> >>>>>>
>>> >>>>>>> Our experience shows us that the best gateway is each
>>> clientes being
>>> >>>>>>> its
>>> >>>>>>> own
>>> >>>>>>> gateway, instead of having a bottle neck server or a
>> cluster of
>>> >>>>>>> bottle
>>> >>>>>>> neck
>>> >>>>>>> servers as gateway (FTP, samba, SFTP,webdav, s3,
>> etc..).
>>> >>>>>>>
>>> >>>>>>> We already did some research in that domain.
>>> >>>>>>>
>>> >>>>>>> Dokan FS is an intent to write an opensource fuse like
>>> cliente for
>>> >>>>>>> MS
>>> >>>>>>> windows.
>>> >>>>>>>
>>> >>>>>>> More information on DOKANFS can be triggered here
>>> >>>>>>>http://dokan-dev.net/en/download/
>>> >>>>>>>
>>> >>>>>>> Positive points of using DOKANFS.
>>> >>>>>>>
>>> >>>>>>> - its opensourced and well licenced mit licence, gpl
>>> licence and lgpl
>>> >>>>>>> licence.
>>> >>>>>>>
>>> >>>>>>> Negative point of using DOKAN FS.
>>> >>>>>>> - unreachable author
>>> >>>>>>> - Poor documentation . Dev comments in japanese.
>>> >>>>>>> - Work in progress so it is unstable and needs to be
>> updated,
>>> >>>>>>> debugged and
>>> >>>>>>> maintained by a MS Windows file system expert
>> developper.
>>> >>>>>>
>>> >>>>>> I am not very familiar with windows storage APIs, but
>>> somebody told me
>>> >>>>>> at once point there were several interfaces against which
>> a
>>> new file
>>> >>>>>> system could be implemented, everything from a full
>>> in-kernel driver
>>> >>>>>> to
>>> >>>>>> something that is explorer-based. Are any of those
>>> suitable? Using a
>>> >>>>>> potentially abandoned fuse-like layer makes me a bit
>> nervous.
>>> >>>>>>
>>> >>>>>> That said,
>>> >>>>>>
>>> >>>>>>>
>>> >>>>>>> I try past year to do a merge from ceph-fuse to dokanfs
>>> >>>>>>> here are what I learnt.
>>> >>>>>>> - Ceph-fuse and related source code is around 60 000
>> lines
>>> of code.
>>> >>>>>>> - Ceph protocol isn t documented so it is like trying
>> to
>>> draw a map
>>> >>>>>>> of
>>> >>>>>>> america
>>> >>>>>>> using only a sextan and a compass.
>>> >>>>>>>
>>> >>>>>>> Those led me to those conclusions:
>>> >>>>>>> - I can t do it alone.
>>> >>>>>>> - It is easier to draw down the ceph protocol way to
>> work from
>>> >>>>>>> kernel/fs/ceph
>>> >>>>>>> sources and mount.ceph
>>> >>>>>>> - Ceph depending libraries may be unexistant or not up
>> to
>>> date in
>>> >>>>>>> their
>>> >>>>>>> port
>>> >>>>>>> on MS Windows (cygwin)
>>> >>>>>>
>>> >>>>>> I think the most sane path should be to make libcephfs
>>> sufficiently
>>> >>>>>> portable to build on windows (or cygwin). For the bits
>> used
>>> by the
>>> >>>>>> client-side coe, I don't think there should be much in
>> the
>>> way of
>>> >>>>>> dependencies, and the main challenge would be untangling
>> the
>>> build for
>>> >>>>>> the necessary pieces out from the rest of Ceph.
>>> >>>>>>
>>> >>>>>> Have you seen the wip-port portability work that is
>>> currently underway
>>> >>>>>> by
>>> >>>>>> Noah and Alan? That may solve many of the cygwin
>> problems
>>> you are
>>> >>>>>> seeing
>>> >>>>>> today.
>>> >>>>>>
>>> >>>>>>> - MS file system specialist are hard do find in the
>> "open
>>> source
>>> >>>>>>> libre
>>> >>>>>>> world"
>>> >>>>>>> so I will try in the commercial world.
>>> >>>>>>>
>>> >>>>>>> The commercial world has some problems too. They need
>> ceph
>>> protocol
>>> >>>>>>> draft
>>> >>>>>>> to
>>> >>>>>>> implemente it to their own product They will have
>> licencing
>>> >>>>>>> /commercial
>>> >>>>>>> politics that infringe lgpl, and hide that most of the
>> work
>>> is done
>>> >>>>>>> by
>>> >>>>>>> people
>>> >>>>>>> other than them. They will not participate in a
>> financial
>>> way to ceph
>>> >>>>>>> enhancement and growth.
>>> >>>>>>
>>> >>>>>> I don't think reimplementing the client code is an
>> efficient way
>>> >>>>>> forward.
>>> >>>>>> Unless the goal is a pure kernel implementation...but a
>>> significant
>>> >>>>>> ongoing investment in development resources would be
>> needed
>>> for that
>>> >>>>>> going
>>> >>>>>> forward. I suspect that is a challenge for a platform
>> that
>>> does not
>>> >>>>>> typically rally that sort of community effort.
>>> >>>>>>
>>> >>>>>> The easiest thing is of course just to use CIFS and
>> Samba
>>> (which works
>>> >>>>>> today). A fuse-like approach is probably a reasonably
>>> middle ground
>>> >>>>>> (both
>>> >>>>>> in initial effort and maintainability going forward)...
>>> >>>>>>
>>> >>>>>> sage
>>> >>>>>>
>>> >>>>>>
>>> >>>>>
>>> >>>
>>> >>> --
>>> >>> To unsubscribe from this list: send the line "unsubscribe
>>> ceph-devel" in
>>> >>> the body of a message tomajordomo@vger.kernel.org
>>> <mailto:majordomo@vger.kernel.org>
>>>
>>> >>> More majordomo info at
>> http://vger.kernel.org/majordomo-info.html
>>> >>
>>> >>
>>> >
>>>
>>>
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
>> in
>> the body of a message tomajordomo@vger.kernel.org
>> More majordomo info athttp://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2013-11-07 18:02 ` Alphe Salas Michels
@ 2013-11-07 20:47 ` Alphe Salas Michels
2013-11-08 0:11 ` Malcolm Haak
0 siblings, 1 reply; 21+ messages in thread
From: Alphe Salas Michels @ 2013-11-07 20:47 UTC (permalink / raw)
To: Matt W. Benjamin, Ketor D; +Cc: ceph-devel
Hello all I finally finished my first source code extraction that starts
from ceph/src/client/fuse_ll.c
The result is accurate unlike previous provided results. basically the
script start from a file extract all the private includes definitions
#include "something.h" and recursively extract private includes too. the
best way to know who is related with who.
starting from fuse_ll.cc I optain 390 files retreived and 120 000 lines
of code !
involved dirs are : in ceph/src
objclass/, common/, msg/, common/, osdc/, include/, client/, mds/,
global/, json_spirit/, log/, os/, crush/, mon/, osd/, auth/
probably not a good way to analyse what amount of work it means since
most of those directories are the implementation of servers (osd, mon,
mds) and even if only a tiny bit of them is needed at client level. you
need two structures from ./osd/OSD.h and my script by relation will
take into acount the whole directory...
I ran the script with libcephfs.cc as start point and got almost the
same results. 131 000 lines of code and 386 files most of the same dirs
involved.
I think I will spend alot of time doing the manual source code isolation
and understand way each #include is set in the files I read (what
purpose they have do they allow to integrate a crucial data type or not.
The other way around will be to read src/libcephfs.cc. It seems shorter
but without understanding what part is used for each included header I
can t say anything...
I will keep reading the source code and take notes. I think in the case
of libcephfs I will gain alot of time.
signature
*Alphé Salas*
Ingeniero T.I
asalas@kepler.cl
*www.kepler.cl <http://www.kepler.cl>*
On 11/07/13 15:02, Alphe Salas Michels wrote:
> Hello D.Ketor and Matt Benjamin,
> You give me alot to think about and this is great!
> I merged your previous post to make a single reply that anyone can
> report to easyly
>
> Windows NFS 4.1 is available here:
> http://www.citi.umich.edu/projects/nfsv4/windows/readme.html
>
> pnfs is another name for NFS4.X. It is presented as alternative to
> ceph and we get known terminology as MDS and OSD but without the self
> healing part if I understand well my rapid look on the topic. (when I
> say rapid look I mean ... 5 minutes spent in that... which is really
> small amount of time to get an accurate view on something)
>
>
> starting from mount.ceph ... I know that mount.ceph does little but it
> is a great hint to know what ceph needs and do things.
> Basically mount.ceph modprobe the ceph driver in the linux kernel then
> call mount with the line command passed args and the cephfs type as
> argument. Then the kernel does the work I don t understand yet what is
> the start calls that are made to the ceph driver but it seemed to me
> that is was relatively light. (a first impression compared to ceph-fuse.)
>
> I think I will do both isolate source code from ceph-client kernel
> (cephfs module for linux kernel) and the one pointed by Sage starting
> from client/fuse_ll.cc in ceph master branch. The common files betwin
> those 2 extractions will be our core set of mandatory features.
>
> Then we try to compile with cygwin a cephfs client library . Then we
> will try to interface with a modified windows nfs 4.1 client or pnfs
> or any other that will accept to be compiled with gcc for win32...
>
> the fact that windows 8.1 is and windows 2012 are out of reach at the
> moment is not a problem to me.
>
> Our first concern is to understand what is ceph protocol. Then adapt
> it to something that can be used on windows prior windows 8.1. Dokan
> fs if I remember well use too the WDK (windows driver dev-kit ) for it
> s compilation so possibly we will see the same limitations.
>
> We need to multiply our source of information by example regarding
> ceph-client (kernel or fuse, radosgw is on a different layer so I will
> not try anything around it at first.) And we need to multiply our
> source of information by example regarding virtual file system
> technologies on windoes OS.
> Alot of work but all of those available source code everyone point at
> me will make our best solution. And in the end we will choose
> technologies knowing what we do and what concequencies they have.
>
> regards,
>
>
>
>
> Regards
>
> signature
>
> *Alphé Salas*
> Ingeniero T.I
>
> asalas@kepler.cl
>
>
> On 11/07/13 11:29, Ketor D wrote:
>> Hi Alphe:
>> Yes Callback Filesystem is very expensive and can't open source.
>> It's not a good choice for ceph4win.
>> Another way for ceph4win maybe develop a kernel-mode fs like
>> pnfs. pnfs has a kernel-mode windows client. I think you can read its
>> src code and maybe migrating from ceph kernel client to windows kernel
>> fs is easier than from userspace ceph fuse client.And a kernel-mode fs
>> client has greater performance than userspace fs like ceph-fuse client
>> and ceph kernel client.
>>
>> Regards.
>>
> On 11/07/13 11:50, Matt W. Benjamin wrote:
>> Hi,
>>
>> The Window NFS v4.1 client is what we work on, so this may be good for
>> code sharing. The license is lgplv2, like Ceph's.
>>
>> Something important to be aware of is that the client uses rdbss, which
>> is a (partial) fsd abstraction that simplified implementation
>> quite a bit, kind of like a mini driver. However, Microsoft's support
>> for rdbss has been in limbo for a bit. For example, to link with
>> the rdbss symbols you can't use the Windows 8 driver kit--you'll need
>> to use the one for Windows 7. (There's a private rdbss2 used internally
>> by Microsoft's SMB implemenation. A the moment, 3rd party drivers
>> can't use that.)
>>
>> We've been in communication with Microsoft about this issue, and know of
>> a few other fsds using it, but it could be a good thing for that
>> lobbying
>> effort to have another user--or it could be a dead end :(.
>>
>> There are a couple of other choices if you're looking to go this route,
>> that I'm aware of (and we may need to take them too, if RDBSS has no
>> way forward), but the required work could be a lot larger.
>>
>> Matt
>>
>> ----- "Ketor D"<d.ketor@gmail.com> wrote:
>>
>>> Hi Alphe:
>>> Yes Callback Filesystem is very expensive and can't open
>>> source.
>>> It's not a good choice for ceph4win.
>>> Another way for ceph4win maybe develop a kernel-mode fs like
>>> pnfs. pnfs has a kernel-mode windows client. I think you can read its
>>> src code and maybe migrating from ceph kernel client to windows
>>> kernel
>>> fs is easier than from userspace ceph fuse client.And a kernel-mode
>>> fs
>>> client has greater performance than userspace fs like ceph-fuse
>>> client
>>> and ceph kernel client.
>>>
>>> Regards.
>>>
>>>
>>>
>>> On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas Michels<asalas@kepler.cl>
>>> wrote:
>>>> Commercial libraries are a pain ...
>>>>
>>>> If we want the more permossive licence offered by callback file
>>> system we
>>>> have to buy it for 20.000 usd. Then we will have to provide a
>>> backbox that
>>>> we have no control upon and that will kill our product anytime they
>>> want anf
>>>> if they decide to stop their commercial activity we will be in the
>>> same
>>>> situation that with dokanfs but without having the source code of
>>> the black
>>>> box. If i have to spend 20 000 dollars i would prefere paying
>>> someone to
>>>> retake dokanfs or to write from scratch a dokanfs fuselike software
>>> make it
>>>> all shiny and pumpy fantastic and ready to plug to ceph client.
>>>>
>>>> I would prefere if people have to pay something to get access to
>>> ceph4win
>>>> that this money goes in ceph main branch pockets... Or as a gift you
>>> donante
>>>> to ceph 10 dollars you get 2 free registration codes for
>>> ceph4win... or
>>>> something like that.
>>>>
>>>> If ceph4win as to be comercial then I would prefer delegate the task
>>> to a
>>>> company like south river technologies and their great product
>>> webdrive. I
>>>> would mininaly get involved in that project and simply buy the final
>>> product
>>>> to sell it together with my ceph based product (which could be a
>>> calxeda
>>>> ceph box or something like that).
>>>>
>>>> I m open anyway to any proposition. But I doubt that callback
>>> filesystem
>>>> offers us a suitable solution in the way I see ceph4win to be spread
>>> and
>>>> used... I m maybe wrong. And anything that will be done around
>>> ceph4win will
>>>> be public documented etc... And licensed the way that if someone
>>> want to
>>>> build a commercial solution on top of it, that would be a
>>> possibility.
>>>> My idea is to giveback somehow to ceph project and at same time
>>> forge a
>>>> better knowledge in ceph technologies. Because like many in libre
>>> world I
>>>> think the business is in the services around the software more than
>>> on the
>>>> software. That the ones writing code should be financed and benefits
>>> from
>>>> the one selling and giving support of the software at all levels. I
>>> m
>>>> probably too idealistic. And too optimistic after all I m the one
>>> saying I
>>>> will do this stuff I have no idea how but well it is interesting and
>>> fun so
>>>> lets do it.
>>>>
>>>> Regards,
>>>>
>>>> P.S: using commercial backend libraries appart including their own
>>> cost will
>>>> force you to use commercial IDE like MS VisualStudio because their
>>> library
>>>> has some kind of drm that only that IDE compiler can use. So alot of
>>> cost
>>>> and yet there is nothing done. If I had to open a kickstarter
>>> project saying
>>>> we need 60 000 USD to do ceph4win with that monney we will buy the
>>> right to
>>>> use and share a commercial copyrighted library but abandonned
>>> punctually to
>>>> us in public domaine and that we will eventually produce something
>>> out of
>>>> it. I doubt I will get a dollar.
>>>>
>>>> We still can suggest the idea to Edlos the commercial company that
>>> has the
>>>> copyright of Callback FS, Or to buy them their product in a blender
>>> way
>>>> (blender was bought with donation before being put opensource and
>>> public
>>>> domaine), Or to open source their library. But in commercial minds
>>>> opensourcing = death of their technical advantage and death of
>>> their
>>>> marketing strategy. They will have to invent something more to
>>> retrieve
>>>> monney from it.
>>>>
>>>> El nov 6, 2013 11:22 p.m., "Ketor D" <d.ketor@gmail.com
>>>> <mailto:d.ketor@gmail.com>> escribió:
>>>>
>>>>
>>>> Hi Alphe,
>>>> I think you could try Callback Filesystem dev
>>> framework. It
>>>> is a commerical dev framework and is maintained by Edlos today.
>>>> I have communicated with Edlos to get a try code for
>>>> development. To dokan, Callback Filesystem has vary document and
>>> maybe
>>>> more stabilize.
>>>>
>>>> Regards.
>>>>
>>>>
>>>>
>>>> On Thu, Nov 7, 2013 at 10:00 AM, Alphe Salas <asalas@kepler.cl
>>>> <mailto:asalas@kepler.cl>> wrote:
>>>> > Hello ketor thank you for your interest un ceph4win. Since
>>> muy
>>>> first mail I
>>>> > exposed the lacks of dokanfs and that I m far from being a
>>>> specialist un
>>>> > filesystems.
>>>> > I exposed what i like un dokanfs bit I not a fanátic of it.
>>> Muy
>>>> goal is to
>>>> > have something working quickly.
>>>> >
>>>> > So I am up to any proposición sure the one with the more docs
>>> and
>>>> support
>>>> > will be the best choice. As for right now what I need is
>>>> understand what are
>>>> > the files involved what are the interfaces functions and what
>>> are
>>>> the needed
>>>> > library dependencies and if they exist ported to windows with
>>>> cygwin. And
>>>> > all that is retrieved from source code.
>>>> >
>>>> > Regards.
>>>> >
>>>> > El nov 6, 2013 10:34 p.m., "Ketor D" <d.ketor@gmail.com
>>>> <mailto:d.ketor@gmail.com>> escribió:
>>>>
>>>> >
>>>> >> Hi Alphe,
>>>> >> We are taking an interest in your work on Ceph Client
>>> for
>>>> Windows
>>>> >> with Dokan.As we know, the performance of Dokan is not very
>>>> good, and it's
>>>> >> abandoned 3 years ago.
>>>> >> I have learned and used OpenDedup(SDFS) for a long
>>> time.
>>>> OpenDedup
>>>> >> has a Dokan version. And the author of OpenDedup said
>>>> >>
>>>> >> The Dokan library is quite flakey and testing should be
>>>> performed before
>>>> >> putting into production
>>>> >>
>>>> >> So what do you think about this? And if there is
>>> another
>>>> solution of
>>>> >> fuse-like filesystem dev framwork on Windows?
>>>> >>
>>>> >> Best Wish!
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe Salas Michels
>>>> <asalas@kepler.cl <mailto:asalas@kepler.cl>>
>>>>
>>>> >> wrote:
>>>> >>>
>>>> >>> Hello I created the github repository for this project
>>>> >>>https://github.com/alphe/Ceph4Win
>>>> >>>
>>>> >>> Regards,
>>>> >>>
>>>> >>> signature
>>>> >>>
>>>> >>> *Alphé Salas*
>>>> >>> Ingeniero T.I
>>>> >>>
>>>> >>>asalas@kepler.cl <mailto:asalas@kepler.cl>
>>>>
>>>> >>> *<http://www.kepler.cl>*
>>>> >>>
>>>> >>> On 11/05/13 21:00, Sage Weil wrote:
>>>> >>>>
>>>> >>>> Hi Alphe,
>>>> >>>>
>>>> >>>> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
>>>> >>>>>
>>>> >>>>> signature *Hi, Sage !
>>>> >>>>> thank you for you enthousiast reply.
>>>> >>>>> I sure want to make the best use of everything or
>>> anything
>>>> previously
>>>> >>>>> done to
>>>> >>>>> tend to
>>>> >>>>> write ceph cliente for windows.
>>>> >>>>>
>>>> >>>>> Apart using libre tools for building the future ceph
>>> cliente
>>>> I am open
>>>> >>>>> to
>>>> >>>>> anything.
>>>> >>>>> I would recommand eclipse CDT or Code::BLocks they are
>>> based
>>>> on mingwin
>>>> >>>>> open
>>>> >>>>> and easyly enhanceable.**
>>>> >>>>>
>>>> >>>>> more free tools can be found here:
>>>> >>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> I will read libcephfs source code and take some notes
>>> about the
>>>> >>>>> protocol.
>>>> >>>>
>>>> >>>> I think you don't need to worry about hte protocol at all,
>>> since
>>>> >>>> libcephs
>>>> >>>> implements it for you (and will capture any future
>>> changes).
>>>> >>>>
>>>> >>>>> I was more going from what I know and trying to track down
>>> how
>>>> >>>>> mount.ceph work
>>>> >>>>> with the parameters passed to it.
>>>> >>>>> since it point finally to Kernel/fs/ceph and that I don t
>>> really
>>>> >>>>> understand
>>>> >>>>> how that module work and that it probably points to some
>>> other
>>>> >>>>> dependencies
>>>> >>>>> Reading libcephfs source code could be a big gain of
>>> time.
>>>> >>>>
>>>> >>>> (I would also ignore mount.ceph as everything it does it
>>>> specific to
>>>> >>>> how Linux mounts work.)
>>>> >>>>
>>>> >>>>> basically on the protocol what is need are:
>>>> >>>>>
>>>> >>>>> 1) open and maintain a connection (socket open, auth, etc
>>> )
>>>> >>>>> 2) retreive a map of directories and disk Quota (disk
>>> sizing
>>>> Y TB free,
>>>> >>>>> Z TB
>>>> >>>>> total)
>>>> >>>>> 3) procedure to send files / directories in a maner that
>>> it
>>>> will allow
>>>> >>>>> our
>>>> >>>>> client to fit ceph transmission protocols
>>>> >>>>> (limit bandwith for stability?, limit connection amount?,
>>>> limit cpu
>>>> >>>>> use?,
>>>> >>>>> Cache for preparing data transfer (a FIFO cache)?)
>>>> >>>>> 4)Procedure to retreive files / directory from ceph
>>> cluster
>>>> >>>>> 5) Management copy/move files /Directories, FS stats,
>>>> Connection Stats.
>>>> >>>>> logging.
>>>> >>>>>
>>>> >>>>> My idea to progress is to take those main bulletpoint in
>>> ceph
>>>> protocol
>>>> >>>>> based
>>>> >>>>> on general ideas of what ceph file system does and start
>>>> identifying
>>>> >>>>> parts
>>>> >>>>> from libcephfs to match those "needs".
>>>> >>>>
>>>> >>>> Instead, I would look at include/cephfs/libcephfs.h, the
>>>> interface that
>>>> >>>> libcephfs provides, and try to map that to what the fuse
>>> layer
>>>> expects.
>>>> >>>> There is both a path-based that I suspsect lends itself
>>> well
>>>> to the
>>>> >>>> Windows interface and (very soon now) a handle based API
>>> that is
>>>> >>>> targetted
>>>> >>>> at the Unix-style VFS layers. I'm mostly guessing,
>>> though,
>>>> since I've
>>>> >>>> never seen any low-level fs code in windows before.
>>>> >>>>
>>>> >>>> In this case, the analogous code for Linux should be
>>>> client/fuse_ll.cc
>>>> >>>> itself (and not much else), although there will probably be
>>> a
>>>> few tricks
>>>> >>>> necessary to map cleanly onto how the windows interfaces
>>> work.
>>>> >>>>
>>>> >>>> Does that make sense?
>>>> >>>>
>>>> >>>> Cheers!
>>>> >>>> sage
>>>> >>>>
>>>> >>>>
>>>> >>>>> Any suggestion and contributions are welcome.
>>>> >>>>>
>>>> >>>>>
>>>> >>>>> *
>>>> >>>>> On 11/05/13 11:23, Sage Weil wrote:
>>>> >>>>>>
>>>> >>>>>> Hi Alphe,
>>>> >>>>>>
>>>> >>>>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
>>>> >>>>>>>
>>>> >>>>>>> Good day developers!
>>>> >>>>>>>
>>>> >>>>>>> I would like to propose to the one interested work with
>>> me to
>>>> >>>>>>> develop a
>>>> >>>>>>> ceph
>>>> >>>>>>> cliente for MS windows world, Basing us on dokanFS.
>>>> >>>>>>>
>>>> >>>>>>> My company is a ceph enthousiast that use on a dayly
>>> basis
>>>> ceph and
>>>> >>>>>>> that
>>>> >>>>>>> need
>>>> >>>>>>> both transfer speed and big expendable and cheap
>>> storage.
>>>> >>>>>>> My company is specialised in data recovery and we want
>>> to
>>>> participate
>>>> >>>>>>> to
>>>> >>>>>>> ceph
>>>> >>>>>>> effort by bringing a ceph cliente for windows.
>>>> >>>>>>
>>>> >>>>>> Awesome!
>>>> >>>>>>
>>>> >>>>>>> Our experience shows us that the best gateway is each
>>>> clientes being
>>>> >>>>>>> its
>>>> >>>>>>> own
>>>> >>>>>>> gateway, instead of having a bottle neck server or a
>>> cluster of
>>>> >>>>>>> bottle
>>>> >>>>>>> neck
>>>> >>>>>>> servers as gateway (FTP, samba, SFTP,webdav, s3,
>>> etc..).
>>>> >>>>>>>
>>>> >>>>>>> We already did some research in that domain.
>>>> >>>>>>>
>>>> >>>>>>> Dokan FS is an intent to write an opensource fuse like
>>>> cliente for
>>>> >>>>>>> MS
>>>> >>>>>>> windows.
>>>> >>>>>>>
>>>> >>>>>>> More information on DOKANFS can be triggered here
>>>> >>>>>>>http://dokan-dev.net/en/download/
>>>> >>>>>>>
>>>> >>>>>>> Positive points of using DOKANFS.
>>>> >>>>>>>
>>>> >>>>>>> - its opensourced and well licenced mit licence, gpl
>>>> licence and lgpl
>>>> >>>>>>> licence.
>>>> >>>>>>>
>>>> >>>>>>> Negative point of using DOKAN FS.
>>>> >>>>>>> - unreachable author
>>>> >>>>>>> - Poor documentation . Dev comments in japanese.
>>>> >>>>>>> - Work in progress so it is unstable and needs to be
>>> updated,
>>>> >>>>>>> debugged and
>>>> >>>>>>> maintained by a MS Windows file system expert
>>> developper.
>>>> >>>>>>
>>>> >>>>>> I am not very familiar with windows storage APIs, but
>>>> somebody told me
>>>> >>>>>> at once point there were several interfaces against which
>>> a
>>>> new file
>>>> >>>>>> system could be implemented, everything from a full
>>>> in-kernel driver
>>>> >>>>>> to
>>>> >>>>>> something that is explorer-based. Are any of those
>>>> suitable? Using a
>>>> >>>>>> potentially abandoned fuse-like layer makes me a bit
>>> nervous.
>>>> >>>>>>
>>>> >>>>>> That said,
>>>> >>>>>>
>>>> >>>>>>>
>>>> >>>>>>> I try past year to do a merge from ceph-fuse to dokanfs
>>>> >>>>>>> here are what I learnt.
>>>> >>>>>>> - Ceph-fuse and related source code is around 60 000
>>> lines
>>>> of code.
>>>> >>>>>>> - Ceph protocol isn t documented so it is like trying
>>> to
>>>> draw a map
>>>> >>>>>>> of
>>>> >>>>>>> america
>>>> >>>>>>> using only a sextan and a compass.
>>>> >>>>>>>
>>>> >>>>>>> Those led me to those conclusions:
>>>> >>>>>>> - I can t do it alone.
>>>> >>>>>>> - It is easier to draw down the ceph protocol way to
>>> work from
>>>> >>>>>>> kernel/fs/ceph
>>>> >>>>>>> sources and mount.ceph
>>>> >>>>>>> - Ceph depending libraries may be unexistant or not up
>>> to
>>>> date in
>>>> >>>>>>> their
>>>> >>>>>>> port
>>>> >>>>>>> on MS Windows (cygwin)
>>>> >>>>>>
>>>> >>>>>> I think the most sane path should be to make libcephfs
>>>> sufficiently
>>>> >>>>>> portable to build on windows (or cygwin). For the bits
>>> used
>>>> by the
>>>> >>>>>> client-side coe, I don't think there should be much in
>>> the
>>>> way of
>>>> >>>>>> dependencies, and the main challenge would be untangling
>>> the
>>>> build for
>>>> >>>>>> the necessary pieces out from the rest of Ceph.
>>>> >>>>>>
>>>> >>>>>> Have you seen the wip-port portability work that is
>>>> currently underway
>>>> >>>>>> by
>>>> >>>>>> Noah and Alan? That may solve many of the cygwin
>>> problems
>>>> you are
>>>> >>>>>> seeing
>>>> >>>>>> today.
>>>> >>>>>>
>>>> >>>>>>> - MS file system specialist are hard do find in the
>>> "open
>>>> source
>>>> >>>>>>> libre
>>>> >>>>>>> world"
>>>> >>>>>>> so I will try in the commercial world.
>>>> >>>>>>>
>>>> >>>>>>> The commercial world has some problems too. They need
>>> ceph
>>>> protocol
>>>> >>>>>>> draft
>>>> >>>>>>> to
>>>> >>>>>>> implemente it to their own product They will have
>>> licencing
>>>> >>>>>>> /commercial
>>>> >>>>>>> politics that infringe lgpl, and hide that most of the
>>> work
>>>> is done
>>>> >>>>>>> by
>>>> >>>>>>> people
>>>> >>>>>>> other than them. They will not participate in a
>>> financial
>>>> way to ceph
>>>> >>>>>>> enhancement and growth.
>>>> >>>>>>
>>>> >>>>>> I don't think reimplementing the client code is an
>>> efficient way
>>>> >>>>>> forward.
>>>> >>>>>> Unless the goal is a pure kernel implementation...but a
>>>> significant
>>>> >>>>>> ongoing investment in development resources would be
>>> needed
>>>> for that
>>>> >>>>>> going
>>>> >>>>>> forward. I suspect that is a challenge for a platform
>>> that
>>>> does not
>>>> >>>>>> typically rally that sort of community effort.
>>>> >>>>>>
>>>> >>>>>> The easiest thing is of course just to use CIFS and
>>> Samba
>>>> (which works
>>>> >>>>>> today). A fuse-like approach is probably a reasonably
>>>> middle ground
>>>> >>>>>> (both
>>>> >>>>>> in initial effort and maintainability going forward)...
>>>> >>>>>>
>>>> >>>>>> sage
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>
>>>> >>>
>>>> >>> --
>>>> >>> To unsubscribe from this list: send the line "unsubscribe
>>>> ceph-devel" in
>>>> >>> the body of a message tomajordomo@vger.kernel.org
>>>> <mailto:majordomo@vger.kernel.org>
>>>>
>>>> >>> More majordomo info at
>>> http://vger.kernel.org/majordomo-info.html
>>>> >>
>>>> >>
>>>> >
>>>>
>>>>
>>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
>>> in
>>> the body of a message tomajordomo@vger.kernel.org
>>> More majordomo info athttp://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2013-11-07 20:47 ` Alphe Salas Michels
@ 2013-11-08 0:11 ` Malcolm Haak
2013-11-08 0:31 ` Matt W. Benjamin
[not found] ` <CAME-gARbjR++qXsx9Sx4KbuggthONrscHToxCTUKrci_MLMDzw@mail.gmail.com>
0 siblings, 2 replies; 21+ messages in thread
From: Malcolm Haak @ 2013-11-08 0:11 UTC (permalink / raw)
To: Alphe Salas Michels, Matt W. Benjamin, Ketor D; +Cc: ceph-devel
I'm just going to throw these in there.
http://www.acc.umu.se/~bosse/
They are GPLv2 some already use sockets and such from inside the kernel.
Heck you might even be able to mod the HTTP one to use rados gateway.
I don't know as I havent sat down and pulled them apart enough yet.
They might help, but they might be useless. Not sure.
On 08/11/13 06:47, Alphe Salas Michels wrote:
> Hello all I finally finished my first source code extraction that starts
> from ceph/src/client/fuse_ll.c
> The result is accurate unlike previous provided results. basically the
> script start from a file extract all the private includes definitions
> #include "something.h" and recursively extract private includes too. the
> best way to know who is related with who.
>
> starting from fuse_ll.cc I optain 390 files retreived and 120 000 lines
> of code !
> involved dirs are : in ceph/src
> objclass/, common/, msg/, common/, osdc/, include/, client/, mds/,
> global/, json_spirit/, log/, os/, crush/, mon/, osd/, auth/
>
> probably not a good way to analyse what amount of work it means since
> most of those directories are the implementation of servers (osd, mon,
> mds) and even if only a tiny bit of them is needed at client level. you
> need two structures from ./osd/OSD.h and my script by relation will
> take into acount the whole directory...
>
> I ran the script with libcephfs.cc as start point and got almost the
> same results. 131 000 lines of code and 386 files most of the same dirs
> involved.
>
>
>
> I think I will spend alot of time doing the manual source code isolation
> and understand way each #include is set in the files I read (what
> purpose they have do they allow to integrate a crucial data type or not.
>
>
> The other way around will be to read src/libcephfs.cc. It seems shorter
> but without understanding what part is used for each included header I
> can t say anything...
>
>
>
> I will keep reading the source code and take notes. I think in the case
> of libcephfs I will gain alot of time.
>
> signature
>
> *Alphé Salas*
> Ingeniero T.I
>
> asalas@kepler.cl
> *www.kepler.cl <http://www.kepler.cl>*
>
> On 11/07/13 15:02, Alphe Salas Michels wrote:
>> Hello D.Ketor and Matt Benjamin,
>> You give me alot to think about and this is great!
>> I merged your previous post to make a single reply that anyone can
>> report to easyly
>>
>> Windows NFS 4.1 is available here:
>> http://www.citi.umich.edu/projects/nfsv4/windows/readme.html
>>
>> pnfs is another name for NFS4.X. It is presented as alternative to
>> ceph and we get known terminology as MDS and OSD but without the self
>> healing part if I understand well my rapid look on the topic. (when I
>> say rapid look I mean ... 5 minutes spent in that... which is really
>> small amount of time to get an accurate view on something)
>>
>>
>> starting from mount.ceph ... I know that mount.ceph does little but it
>> is a great hint to know what ceph needs and do things.
>> Basically mount.ceph modprobe the ceph driver in the linux kernel then
>> call mount with the line command passed args and the cephfs type as
>> argument. Then the kernel does the work I don t understand yet what is
>> the start calls that are made to the ceph driver but it seemed to me
>> that is was relatively light. (a first impression compared to ceph-fuse.)
>>
>> I think I will do both isolate source code from ceph-client kernel
>> (cephfs module for linux kernel) and the one pointed by Sage starting
>> from client/fuse_ll.cc in ceph master branch. The common files betwin
>> those 2 extractions will be our core set of mandatory features.
>>
>> Then we try to compile with cygwin a cephfs client library . Then we
>> will try to interface with a modified windows nfs 4.1 client or pnfs
>> or any other that will accept to be compiled with gcc for win32...
>>
>> the fact that windows 8.1 is and windows 2012 are out of reach at the
>> moment is not a problem to me.
>>
>> Our first concern is to understand what is ceph protocol. Then adapt
>> it to something that can be used on windows prior windows 8.1. Dokan
>> fs if I remember well use too the WDK (windows driver dev-kit ) for it
>> s compilation so possibly we will see the same limitations.
>>
>> We need to multiply our source of information by example regarding
>> ceph-client (kernel or fuse, radosgw is on a different layer so I will
>> not try anything around it at first.) And we need to multiply our
>> source of information by example regarding virtual file system
>> technologies on windoes OS.
>> Alot of work but all of those available source code everyone point at
>> me will make our best solution. And in the end we will choose
>> technologies knowing what we do and what concequencies they have.
>>
>> regards,
>>
>>
>>
>>
>> Regards
>>
>> signature
>>
>> *Alphé Salas*
>> Ingeniero T.I
>>
>> asalas@kepler.cl
>>
>>
>> On 11/07/13 11:29, Ketor D wrote:
>>> Hi Alphe:
>>> Yes Callback Filesystem is very expensive and can't open source.
>>> It's not a good choice for ceph4win.
>>> Another way for ceph4win maybe develop a kernel-mode fs like
>>> pnfs. pnfs has a kernel-mode windows client. I think you can read its
>>> src code and maybe migrating from ceph kernel client to windows kernel
>>> fs is easier than from userspace ceph fuse client.And a kernel-mode fs
>>> client has greater performance than userspace fs like ceph-fuse client
>>> and ceph kernel client.
>>>
>>> Regards.
>>>
>> On 11/07/13 11:50, Matt W. Benjamin wrote:
>>> Hi,
>>>
>>> The Window NFS v4.1 client is what we work on, so this may be good for
>>> code sharing. The license is lgplv2, like Ceph's.
>>>
>>> Something important to be aware of is that the client uses rdbss, which
>>> is a (partial) fsd abstraction that simplified implementation
>>> quite a bit, kind of like a mini driver. However, Microsoft's support
>>> for rdbss has been in limbo for a bit. For example, to link with
>>> the rdbss symbols you can't use the Windows 8 driver kit--you'll need
>>> to use the one for Windows 7. (There's a private rdbss2 used internally
>>> by Microsoft's SMB implemenation. A the moment, 3rd party drivers
>>> can't use that.)
>>>
>>> We've been in communication with Microsoft about this issue, and know of
>>> a few other fsds using it, but it could be a good thing for that
>>> lobbying
>>> effort to have another user--or it could be a dead end :(.
>>>
>>> There are a couple of other choices if you're looking to go this route,
>>> that I'm aware of (and we may need to take them too, if RDBSS has no
>>> way forward), but the required work could be a lot larger.
>>>
>>> Matt
>>>
>>> ----- "Ketor D"<d.ketor@gmail.com> wrote:
>>>
>>>> Hi Alphe:
>>>> Yes Callback Filesystem is very expensive and can't open
>>>> source.
>>>> It's not a good choice for ceph4win.
>>>> Another way for ceph4win maybe develop a kernel-mode fs like
>>>> pnfs. pnfs has a kernel-mode windows client. I think you can read its
>>>> src code and maybe migrating from ceph kernel client to windows
>>>> kernel
>>>> fs is easier than from userspace ceph fuse client.And a kernel-mode
>>>> fs
>>>> client has greater performance than userspace fs like ceph-fuse
>>>> client
>>>> and ceph kernel client.
>>>>
>>>> Regards.
>>>>
>>>>
>>>>
>>>> On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas Michels<asalas@kepler.cl>
>>>> wrote:
>>>>> Commercial libraries are a pain ...
>>>>>
>>>>> If we want the more permossive licence offered by callback file
>>>> system we
>>>>> have to buy it for 20.000 usd. Then we will have to provide a
>>>> backbox that
>>>>> we have no control upon and that will kill our product anytime they
>>>> want anf
>>>>> if they decide to stop their commercial activity we will be in the
>>>> same
>>>>> situation that with dokanfs but without having the source code of
>>>> the black
>>>>> box. If i have to spend 20 000 dollars i would prefere paying
>>>> someone to
>>>>> retake dokanfs or to write from scratch a dokanfs fuselike software
>>>> make it
>>>>> all shiny and pumpy fantastic and ready to plug to ceph client.
>>>>>
>>>>> I would prefere if people have to pay something to get access to
>>>> ceph4win
>>>>> that this money goes in ceph main branch pockets... Or as a gift you
>>>> donante
>>>>> to ceph 10 dollars you get 2 free registration codes for
>>>> ceph4win... or
>>>>> something like that.
>>>>>
>>>>> If ceph4win as to be comercial then I would prefer delegate the task
>>>> to a
>>>>> company like south river technologies and their great product
>>>> webdrive. I
>>>>> would mininaly get involved in that project and simply buy the final
>>>> product
>>>>> to sell it together with my ceph based product (which could be a
>>>> calxeda
>>>>> ceph box or something like that).
>>>>>
>>>>> I m open anyway to any proposition. But I doubt that callback
>>>> filesystem
>>>>> offers us a suitable solution in the way I see ceph4win to be spread
>>>> and
>>>>> used... I m maybe wrong. And anything that will be done around
>>>> ceph4win will
>>>>> be public documented etc... And licensed the way that if someone
>>>> want to
>>>>> build a commercial solution on top of it, that would be a
>>>> possibility.
>>>>> My idea is to giveback somehow to ceph project and at same time
>>>> forge a
>>>>> better knowledge in ceph technologies. Because like many in libre
>>>> world I
>>>>> think the business is in the services around the software more than
>>>> on the
>>>>> software. That the ones writing code should be financed and benefits
>>>> from
>>>>> the one selling and giving support of the software at all levels. I
>>>> m
>>>>> probably too idealistic. And too optimistic after all I m the one
>>>> saying I
>>>>> will do this stuff I have no idea how but well it is interesting and
>>>> fun so
>>>>> lets do it.
>>>>>
>>>>> Regards,
>>>>>
>>>>> P.S: using commercial backend libraries appart including their own
>>>> cost will
>>>>> force you to use commercial IDE like MS VisualStudio because their
>>>> library
>>>>> has some kind of drm that only that IDE compiler can use. So alot of
>>>> cost
>>>>> and yet there is nothing done. If I had to open a kickstarter
>>>> project saying
>>>>> we need 60 000 USD to do ceph4win with that monney we will buy the
>>>> right to
>>>>> use and share a commercial copyrighted library but abandonned
>>>> punctually to
>>>>> us in public domaine and that we will eventually produce something
>>>> out of
>>>>> it. I doubt I will get a dollar.
>>>>>
>>>>> We still can suggest the idea to Edlos the commercial company that
>>>> has the
>>>>> copyright of Callback FS, Or to buy them their product in a blender
>>>> way
>>>>> (blender was bought with donation before being put opensource and
>>>> public
>>>>> domaine), Or to open source their library. But in commercial minds
>>>>> opensourcing = death of their technical advantage and death of
>>>> their
>>>>> marketing strategy. They will have to invent something more to
>>>> retrieve
>>>>> monney from it.
>>>>>
>>>>> El nov 6, 2013 11:22 p.m., "Ketor D" <d.ketor@gmail.com
>>>>> <mailto:d.ketor@gmail.com>> escribió:
>>>>>
>>>>>
>>>>> Hi Alphe,
>>>>> I think you could try Callback Filesystem dev
>>>> framework. It
>>>>> is a commerical dev framework and is maintained by Edlos today.
>>>>> I have communicated with Edlos to get a try code for
>>>>> development. To dokan, Callback Filesystem has vary document and
>>>> maybe
>>>>> more stabilize.
>>>>>
>>>>> Regards.
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Nov 7, 2013 at 10:00 AM, Alphe Salas <asalas@kepler.cl
>>>>> <mailto:asalas@kepler.cl>> wrote:
>>>>> > Hello ketor thank you for your interest un ceph4win. Since
>>>> muy
>>>>> first mail I
>>>>> > exposed the lacks of dokanfs and that I m far from being a
>>>>> specialist un
>>>>> > filesystems.
>>>>> > I exposed what i like un dokanfs bit I not a fanátic of it.
>>>> Muy
>>>>> goal is to
>>>>> > have something working quickly.
>>>>> >
>>>>> > So I am up to any proposición sure the one with the more docs
>>>> and
>>>>> support
>>>>> > will be the best choice. As for right now what I need is
>>>>> understand what are
>>>>> > the files involved what are the interfaces functions and what
>>>> are
>>>>> the needed
>>>>> > library dependencies and if they exist ported to windows with
>>>>> cygwin. And
>>>>> > all that is retrieved from source code.
>>>>> >
>>>>> > Regards.
>>>>> >
>>>>> > El nov 6, 2013 10:34 p.m., "Ketor D" <d.ketor@gmail.com
>>>>> <mailto:d.ketor@gmail.com>> escribió:
>>>>>
>>>>> >
>>>>> >> Hi Alphe,
>>>>> >> We are taking an interest in your work on Ceph Client
>>>> for
>>>>> Windows
>>>>> >> with Dokan.As we know, the performance of Dokan is not very
>>>>> good, and it's
>>>>> >> abandoned 3 years ago.
>>>>> >> I have learned and used OpenDedup(SDFS) for a long
>>>> time.
>>>>> OpenDedup
>>>>> >> has a Dokan version. And the author of OpenDedup said
>>>>> >>
>>>>> >> The Dokan library is quite flakey and testing should be
>>>>> performed before
>>>>> >> putting into production
>>>>> >>
>>>>> >> So what do you think about this? And if there is
>>>> another
>>>>> solution of
>>>>> >> fuse-like filesystem dev framwork on Windows?
>>>>> >>
>>>>> >> Best Wish!
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe Salas Michels
>>>>> <asalas@kepler.cl <mailto:asalas@kepler.cl>>
>>>>>
>>>>> >> wrote:
>>>>> >>>
>>>>> >>> Hello I created the github repository for this project
>>>>> >>>https://github.com/alphe/Ceph4Win
>>>>> >>>
>>>>> >>> Regards,
>>>>> >>>
>>>>> >>> signature
>>>>> >>>
>>>>> >>> *Alphé Salas*
>>>>> >>> Ingeniero T.I
>>>>> >>>
>>>>> >>>asalas@kepler.cl <mailto:asalas@kepler.cl>
>>>>>
>>>>> >>> *<http://www.kepler.cl>*
>>>>> >>>
>>>>> >>> On 11/05/13 21:00, Sage Weil wrote:
>>>>> >>>>
>>>>> >>>> Hi Alphe,
>>>>> >>>>
>>>>> >>>> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
>>>>> >>>>>
>>>>> >>>>> signature *Hi, Sage !
>>>>> >>>>> thank you for you enthousiast reply.
>>>>> >>>>> I sure want to make the best use of everything or
>>>> anything
>>>>> previously
>>>>> >>>>> done to
>>>>> >>>>> tend to
>>>>> >>>>> write ceph cliente for windows.
>>>>> >>>>>
>>>>> >>>>> Apart using libre tools for building the future ceph
>>>> cliente
>>>>> I am open
>>>>> >>>>> to
>>>>> >>>>> anything.
>>>>> >>>>> I would recommand eclipse CDT or Code::BLocks they are
>>>> based
>>>>> on mingwin
>>>>> >>>>> open
>>>>> >>>>> and easyly enhanceable.**
>>>>> >>>>>
>>>>> >>>>> more free tools can be found here:
>>>>> >>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> I will read libcephfs source code and take some notes
>>>> about the
>>>>> >>>>> protocol.
>>>>> >>>>
>>>>> >>>> I think you don't need to worry about hte protocol at all,
>>>> since
>>>>> >>>> libcephs
>>>>> >>>> implements it for you (and will capture any future
>>>> changes).
>>>>> >>>>
>>>>> >>>>> I was more going from what I know and trying to track down
>>>> how
>>>>> >>>>> mount.ceph work
>>>>> >>>>> with the parameters passed to it.
>>>>> >>>>> since it point finally to Kernel/fs/ceph and that I don t
>>>> really
>>>>> >>>>> understand
>>>>> >>>>> how that module work and that it probably points to some
>>>> other
>>>>> >>>>> dependencies
>>>>> >>>>> Reading libcephfs source code could be a big gain of
>>>> time.
>>>>> >>>>
>>>>> >>>> (I would also ignore mount.ceph as everything it does it
>>>>> specific to
>>>>> >>>> how Linux mounts work.)
>>>>> >>>>
>>>>> >>>>> basically on the protocol what is need are:
>>>>> >>>>>
>>>>> >>>>> 1) open and maintain a connection (socket open, auth, etc
>>>> )
>>>>> >>>>> 2) retreive a map of directories and disk Quota (disk
>>>> sizing
>>>>> Y TB free,
>>>>> >>>>> Z TB
>>>>> >>>>> total)
>>>>> >>>>> 3) procedure to send files / directories in a maner that
>>>> it
>>>>> will allow
>>>>> >>>>> our
>>>>> >>>>> client to fit ceph transmission protocols
>>>>> >>>>> (limit bandwith for stability?, limit connection amount?,
>>>>> limit cpu
>>>>> >>>>> use?,
>>>>> >>>>> Cache for preparing data transfer (a FIFO cache)?)
>>>>> >>>>> 4)Procedure to retreive files / directory from ceph
>>>> cluster
>>>>> >>>>> 5) Management copy/move files /Directories, FS stats,
>>>>> Connection Stats.
>>>>> >>>>> logging.
>>>>> >>>>>
>>>>> >>>>> My idea to progress is to take those main bulletpoint in
>>>> ceph
>>>>> protocol
>>>>> >>>>> based
>>>>> >>>>> on general ideas of what ceph file system does and start
>>>>> identifying
>>>>> >>>>> parts
>>>>> >>>>> from libcephfs to match those "needs".
>>>>> >>>>
>>>>> >>>> Instead, I would look at include/cephfs/libcephfs.h, the
>>>>> interface that
>>>>> >>>> libcephfs provides, and try to map that to what the fuse
>>>> layer
>>>>> expects.
>>>>> >>>> There is both a path-based that I suspsect lends itself
>>>> well
>>>>> to the
>>>>> >>>> Windows interface and (very soon now) a handle based API
>>>> that is
>>>>> >>>> targetted
>>>>> >>>> at the Unix-style VFS layers. I'm mostly guessing,
>>>> though,
>>>>> since I've
>>>>> >>>> never seen any low-level fs code in windows before.
>>>>> >>>>
>>>>> >>>> In this case, the analogous code for Linux should be
>>>>> client/fuse_ll.cc
>>>>> >>>> itself (and not much else), although there will probably be
>>>> a
>>>>> few tricks
>>>>> >>>> necessary to map cleanly onto how the windows interfaces
>>>> work.
>>>>> >>>>
>>>>> >>>> Does that make sense?
>>>>> >>>>
>>>>> >>>> Cheers!
>>>>> >>>> sage
>>>>> >>>>
>>>>> >>>>
>>>>> >>>>> Any suggestion and contributions are welcome.
>>>>> >>>>>
>>>>> >>>>>
>>>>> >>>>> *
>>>>> >>>>> On 11/05/13 11:23, Sage Weil wrote:
>>>>> >>>>>>
>>>>> >>>>>> Hi Alphe,
>>>>> >>>>>>
>>>>> >>>>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
>>>>> >>>>>>>
>>>>> >>>>>>> Good day developers!
>>>>> >>>>>>>
>>>>> >>>>>>> I would like to propose to the one interested work with
>>>> me to
>>>>> >>>>>>> develop a
>>>>> >>>>>>> ceph
>>>>> >>>>>>> cliente for MS windows world, Basing us on dokanFS.
>>>>> >>>>>>>
>>>>> >>>>>>> My company is a ceph enthousiast that use on a dayly
>>>> basis
>>>>> ceph and
>>>>> >>>>>>> that
>>>>> >>>>>>> need
>>>>> >>>>>>> both transfer speed and big expendable and cheap
>>>> storage.
>>>>> >>>>>>> My company is specialised in data recovery and we want
>>>> to
>>>>> participate
>>>>> >>>>>>> to
>>>>> >>>>>>> ceph
>>>>> >>>>>>> effort by bringing a ceph cliente for windows.
>>>>> >>>>>>
>>>>> >>>>>> Awesome!
>>>>> >>>>>>
>>>>> >>>>>>> Our experience shows us that the best gateway is each
>>>>> clientes being
>>>>> >>>>>>> its
>>>>> >>>>>>> own
>>>>> >>>>>>> gateway, instead of having a bottle neck server or a
>>>> cluster of
>>>>> >>>>>>> bottle
>>>>> >>>>>>> neck
>>>>> >>>>>>> servers as gateway (FTP, samba, SFTP,webdav, s3,
>>>> etc..).
>>>>> >>>>>>>
>>>>> >>>>>>> We already did some research in that domain.
>>>>> >>>>>>>
>>>>> >>>>>>> Dokan FS is an intent to write an opensource fuse like
>>>>> cliente for
>>>>> >>>>>>> MS
>>>>> >>>>>>> windows.
>>>>> >>>>>>>
>>>>> >>>>>>> More information on DOKANFS can be triggered here
>>>>> >>>>>>>http://dokan-dev.net/en/download/
>>>>> >>>>>>>
>>>>> >>>>>>> Positive points of using DOKANFS.
>>>>> >>>>>>>
>>>>> >>>>>>> - its opensourced and well licenced mit licence, gpl
>>>>> licence and lgpl
>>>>> >>>>>>> licence.
>>>>> >>>>>>>
>>>>> >>>>>>> Negative point of using DOKAN FS.
>>>>> >>>>>>> - unreachable author
>>>>> >>>>>>> - Poor documentation . Dev comments in japanese.
>>>>> >>>>>>> - Work in progress so it is unstable and needs to be
>>>> updated,
>>>>> >>>>>>> debugged and
>>>>> >>>>>>> maintained by a MS Windows file system expert
>>>> developper.
>>>>> >>>>>>
>>>>> >>>>>> I am not very familiar with windows storage APIs, but
>>>>> somebody told me
>>>>> >>>>>> at once point there were several interfaces against which
>>>> a
>>>>> new file
>>>>> >>>>>> system could be implemented, everything from a full
>>>>> in-kernel driver
>>>>> >>>>>> to
>>>>> >>>>>> something that is explorer-based. Are any of those
>>>>> suitable? Using a
>>>>> >>>>>> potentially abandoned fuse-like layer makes me a bit
>>>> nervous.
>>>>> >>>>>>
>>>>> >>>>>> That said,
>>>>> >>>>>>
>>>>> >>>>>>>
>>>>> >>>>>>> I try past year to do a merge from ceph-fuse to dokanfs
>>>>> >>>>>>> here are what I learnt.
>>>>> >>>>>>> - Ceph-fuse and related source code is around 60 000
>>>> lines
>>>>> of code.
>>>>> >>>>>>> - Ceph protocol isn t documented so it is like trying
>>>> to
>>>>> draw a map
>>>>> >>>>>>> of
>>>>> >>>>>>> america
>>>>> >>>>>>> using only a sextan and a compass.
>>>>> >>>>>>>
>>>>> >>>>>>> Those led me to those conclusions:
>>>>> >>>>>>> - I can t do it alone.
>>>>> >>>>>>> - It is easier to draw down the ceph protocol way to
>>>> work from
>>>>> >>>>>>> kernel/fs/ceph
>>>>> >>>>>>> sources and mount.ceph
>>>>> >>>>>>> - Ceph depending libraries may be unexistant or not up
>>>> to
>>>>> date in
>>>>> >>>>>>> their
>>>>> >>>>>>> port
>>>>> >>>>>>> on MS Windows (cygwin)
>>>>> >>>>>>
>>>>> >>>>>> I think the most sane path should be to make libcephfs
>>>>> sufficiently
>>>>> >>>>>> portable to build on windows (or cygwin). For the bits
>>>> used
>>>>> by the
>>>>> >>>>>> client-side coe, I don't think there should be much in
>>>> the
>>>>> way of
>>>>> >>>>>> dependencies, and the main challenge would be untangling
>>>> the
>>>>> build for
>>>>> >>>>>> the necessary pieces out from the rest of Ceph.
>>>>> >>>>>>
>>>>> >>>>>> Have you seen the wip-port portability work that is
>>>>> currently underway
>>>>> >>>>>> by
>>>>> >>>>>> Noah and Alan? That may solve many of the cygwin
>>>> problems
>>>>> you are
>>>>> >>>>>> seeing
>>>>> >>>>>> today.
>>>>> >>>>>>
>>>>> >>>>>>> - MS file system specialist are hard do find in the
>>>> "open
>>>>> source
>>>>> >>>>>>> libre
>>>>> >>>>>>> world"
>>>>> >>>>>>> so I will try in the commercial world.
>>>>> >>>>>>>
>>>>> >>>>>>> The commercial world has some problems too. They need
>>>> ceph
>>>>> protocol
>>>>> >>>>>>> draft
>>>>> >>>>>>> to
>>>>> >>>>>>> implemente it to their own product They will have
>>>> licencing
>>>>> >>>>>>> /commercial
>>>>> >>>>>>> politics that infringe lgpl, and hide that most of the
>>>> work
>>>>> is done
>>>>> >>>>>>> by
>>>>> >>>>>>> people
>>>>> >>>>>>> other than them. They will not participate in a
>>>> financial
>>>>> way to ceph
>>>>> >>>>>>> enhancement and growth.
>>>>> >>>>>>
>>>>> >>>>>> I don't think reimplementing the client code is an
>>>> efficient way
>>>>> >>>>>> forward.
>>>>> >>>>>> Unless the goal is a pure kernel implementation...but a
>>>>> significant
>>>>> >>>>>> ongoing investment in development resources would be
>>>> needed
>>>>> for that
>>>>> >>>>>> going
>>>>> >>>>>> forward. I suspect that is a challenge for a platform
>>>> that
>>>>> does not
>>>>> >>>>>> typically rally that sort of community effort.
>>>>> >>>>>>
>>>>> >>>>>> The easiest thing is of course just to use CIFS and
>>>> Samba
>>>>> (which works
>>>>> >>>>>> today). A fuse-like approach is probably a reasonably
>>>>> middle ground
>>>>> >>>>>> (both
>>>>> >>>>>> in initial effort and maintainability going forward)...
>>>>> >>>>>>
>>>>> >>>>>> sage
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>
>>>>> >>>
>>>>> >>> --
>>>>> >>> To unsubscribe from this list: send the line "unsubscribe
>>>>> ceph-devel" in
>>>>> >>> the body of a message tomajordomo@vger.kernel.org
>>>>> <mailto:majordomo@vger.kernel.org>
>>>>>
>>>>> >>> More majordomo info at
>>>> http://vger.kernel.org/majordomo-info.html
>>>>> >>
>>>>> >>
>>>>> >
>>>>>
>>>>>
>>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
>>>> in
>>>> the body of a message tomajordomo@vger.kernel.org
>>>> More majordomo info athttp://vger.kernel.org/majordomo-info.html
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2013-11-08 0:11 ` Malcolm Haak
@ 2013-11-08 0:31 ` Matt W. Benjamin
[not found] ` <CAME-gARbjR++qXsx9Sx4KbuggthONrscHToxCTUKrci_MLMDzw@mail.gmail.com>
1 sibling, 0 replies; 21+ messages in thread
From: Matt W. Benjamin @ 2013-11-08 0:31 UTC (permalink / raw)
To: Malcolm Haak; +Cc: ceph-devel, Alphe Salas Michels, Ketor D
They certainly could be. The OpenAFS kdfs driver is particularly complete.
I've built it and worked on it a bit in earlier versions than those shipping
today.
It's a hybrid driver. Networking is actually in userspace (that's true of the
the NFSv4 client too). The two differ greatly in architecture, however, and
the AFS driver has complex licensing. You should be able to use the kernel
components by Kernel Drivers/Secure Endpoints (YFS?), but the legacy AFS code
is not GPL compatible.
(Can't speak for the others.)
Matt
----- "Malcolm Haak" <malcolm@sgi.com> wrote:
> I'm just going to throw these in there.
>
> http://www.acc.umu.se/~bosse/
>
> They are GPLv2 some already use sockets and such from inside the
> kernel.
> Heck you might even be able to mod the HTTP one to use rados
> gateway.
> I don't know as I havent sat down and pulled them apart enough yet.
>
> They might help, but they might be useless. Not sure.
>
> On 08/11/13 06:47, Alphe Salas Michels wrote:
> > Hello all I finally finished my first source code extraction that
> starts
> > from ceph/src/client/fuse_ll.c
> > The result is accurate unlike previous provided results. basically
> the
> > script start from a file extract all the private includes
> definitions
> > #include "something.h" and recursively extract private includes too.
> the
> > best way to know who is related with who.
> >
> > starting from fuse_ll.cc I optain 390 files retreived and 120 000
> lines
> > of code !
> > involved dirs are : in ceph/src
> > objclass/, common/, msg/, common/, osdc/, include/, client/, mds/,
> > global/, json_spirit/, log/, os/, crush/, mon/, osd/, auth/
> >
> > probably not a good way to analyse what amount of work it means
> since
> > most of those directories are the implementation of servers (osd,
> mon,
> > mds) and even if only a tiny bit of them is needed at client level.
> you
> > need two structures from ./osd/OSD.h and my script by relation
> will
> > take into acount the whole directory...
> >
> > I ran the script with libcephfs.cc as start point and got almost
> the
> > same results. 131 000 lines of code and 386 files most of the same
> dirs
> > involved.
> >
> >
> >
> > I think I will spend alot of time doing the manual source code
> isolation
> > and understand way each #include is set in the files I read (what
> > purpose they have do they allow to integrate a crucial data type or
> not.
> >
> >
> > The other way around will be to read src/libcephfs.cc. It seems
> shorter
> > but without understanding what part is used for each included header
> I
> > can t say anything...
> >
> >
> >
> > I will keep reading the source code and take notes. I think in the
> case
> > of libcephfs I will gain alot of time.
> >
> > signature
> >
> > *Alphé Salas*
> > Ingeniero T.I
> >
> > asalas@kepler.cl
> > *www.kepler.cl <http://www.kepler.cl>*
> >
> > On 11/07/13 15:02, Alphe Salas Michels wrote:
> >> Hello D.Ketor and Matt Benjamin,
> >> You give me alot to think about and this is great!
> >> I merged your previous post to make a single reply that anyone can
> >> report to easyly
> >>
> >> Windows NFS 4.1 is available here:
> >> http://www.citi.umich.edu/projects/nfsv4/windows/readme.html
> >>
> >> pnfs is another name for NFS4.X. It is presented as alternative to
> >> ceph and we get known terminology as MDS and OSD but without the
> self
> >> healing part if I understand well my rapid look on the topic. (when
> I
> >> say rapid look I mean ... 5 minutes spent in that... which is
> really
> >> small amount of time to get an accurate view on something)
> >>
> >>
> >> starting from mount.ceph ... I know that mount.ceph does little but
> it
> >> is a great hint to know what ceph needs and do things.
> >> Basically mount.ceph modprobe the ceph driver in the linux kernel
> then
> >> call mount with the line command passed args and the cephfs type
> as
> >> argument. Then the kernel does the work I don t understand yet what
> is
> >> the start calls that are made to the ceph driver but it seemed to
> me
> >> that is was relatively light. (a first impression compared to
> ceph-fuse.)
> >>
> >> I think I will do both isolate source code from ceph-client kernel
> >> (cephfs module for linux kernel) and the one pointed by Sage
> starting
> >> from client/fuse_ll.cc in ceph master branch. The common files
> betwin
> >> those 2 extractions will be our core set of mandatory features.
> >>
> >> Then we try to compile with cygwin a cephfs client library . Then
> we
> >> will try to interface with a modified windows nfs 4.1 client or
> pnfs
> >> or any other that will accept to be compiled with gcc for win32...
> >>
> >> the fact that windows 8.1 is and windows 2012 are out of reach at
> the
> >> moment is not a problem to me.
> >>
> >> Our first concern is to understand what is ceph protocol. Then
> adapt
> >> it to something that can be used on windows prior windows 8.1.
> Dokan
> >> fs if I remember well use too the WDK (windows driver dev-kit ) for
> it
> >> s compilation so possibly we will see the same limitations.
> >>
> >> We need to multiply our source of information by example regarding
> >> ceph-client (kernel or fuse, radosgw is on a different layer so I
> will
> >> not try anything around it at first.) And we need to multiply our
> >> source of information by example regarding virtual file system
> >> technologies on windoes OS.
> >> Alot of work but all of those available source code everyone point
> at
> >> me will make our best solution. And in the end we will choose
> >> technologies knowing what we do and what concequencies they have.
> >>
> >> regards,
> >>
> >>
> >>
> >>
> >> Regards
> >>
> >> signature
> >>
> >> *Alphé Salas*
> >> Ingeniero T.I
> >>
> >> asalas@kepler.cl
> >>
> >>
> >> On 11/07/13 11:29, Ketor D wrote:
> >>> Hi Alphe:
> >>> Yes Callback Filesystem is very expensive and can't open
> source.
> >>> It's not a good choice for ceph4win.
> >>> Another way for ceph4win maybe develop a kernel-mode fs
> like
> >>> pnfs. pnfs has a kernel-mode windows client. I think you can read
> its
> >>> src code and maybe migrating from ceph kernel client to windows
> kernel
> >>> fs is easier than from userspace ceph fuse client.And a
> kernel-mode fs
> >>> client has greater performance than userspace fs like ceph-fuse
> client
> >>> and ceph kernel client.
> >>>
> >>> Regards.
> >>>
> >> On 11/07/13 11:50, Matt W. Benjamin wrote:
> >>> Hi,
> >>>
> >>> The Window NFS v4.1 client is what we work on, so this may be good
> for
> >>> code sharing. The license is lgplv2, like Ceph's.
> >>>
> >>> Something important to be aware of is that the client uses rdbss,
> which
> >>> is a (partial) fsd abstraction that simplified implementation
> >>> quite a bit, kind of like a mini driver. However, Microsoft's
> support
> >>> for rdbss has been in limbo for a bit. For example, to link with
> >>> the rdbss symbols you can't use the Windows 8 driver kit--you'll
> need
> >>> to use the one for Windows 7. (There's a private rdbss2 used
> internally
> >>> by Microsoft's SMB implemenation. A the moment, 3rd party
> drivers
> >>> can't use that.)
> >>>
> >>> We've been in communication with Microsoft about this issue, and
> know of
> >>> a few other fsds using it, but it could be a good thing for that
> >>> lobbying
> >>> effort to have another user--or it could be a dead end :(.
> >>>
> >>> There are a couple of other choices if you're looking to go this
> route,
> >>> that I'm aware of (and we may need to take them too, if RDBSS has
> no
> >>> way forward), but the required work could be a lot larger.
> >>>
> >>> Matt
> >>>
> >>> ----- "Ketor D"<d.ketor@gmail.com> wrote:
> >>>
> >>>> Hi Alphe:
> >>>> Yes Callback Filesystem is very expensive and can't open
> >>>> source.
> >>>> It's not a good choice for ceph4win.
> >>>> Another way for ceph4win maybe develop a kernel-mode fs
> like
> >>>> pnfs. pnfs has a kernel-mode windows client. I think you can read
> its
> >>>> src code and maybe migrating from ceph kernel client to windows
> >>>> kernel
> >>>> fs is easier than from userspace ceph fuse client.And a
> kernel-mode
> >>>> fs
> >>>> client has greater performance than userspace fs like ceph-fuse
> >>>> client
> >>>> and ceph kernel client.
> >>>>
> >>>> Regards.
> >>>>
> >>>>
> >>>>
> >>>> On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas
> Michels<asalas@kepler.cl>
> >>>> wrote:
> >>>>> Commercial libraries are a pain ...
> >>>>>
> >>>>> If we want the more permossive licence offered by callback file
> >>>> system we
> >>>>> have to buy it for 20.000 usd. Then we will have to provide a
> >>>> backbox that
> >>>>> we have no control upon and that will kill our product anytime
> they
> >>>> want anf
> >>>>> if they decide to stop their commercial activity we will be in
> the
> >>>> same
> >>>>> situation that with dokanfs but without having the source code
> of
> >>>> the black
> >>>>> box. If i have to spend 20 000 dollars i would prefere paying
> >>>> someone to
> >>>>> retake dokanfs or to write from scratch a dokanfs fuselike
> software
> >>>> make it
> >>>>> all shiny and pumpy fantastic and ready to plug to ceph client.
> >>>>>
> >>>>> I would prefere if people have to pay something to get access
> to
> >>>> ceph4win
> >>>>> that this money goes in ceph main branch pockets... Or as a gift
> you
> >>>> donante
> >>>>> to ceph 10 dollars you get 2 free registration codes for
> >>>> ceph4win... or
> >>>>> something like that.
> >>>>>
> >>>>> If ceph4win as to be comercial then I would prefer delegate the
> task
> >>>> to a
> >>>>> company like south river technologies and their great product
> >>>> webdrive. I
> >>>>> would mininaly get involved in that project and simply buy the
> final
> >>>> product
> >>>>> to sell it together with my ceph based product (which could be
> a
> >>>> calxeda
> >>>>> ceph box or something like that).
> >>>>>
> >>>>> I m open anyway to any proposition. But I doubt that callback
> >>>> filesystem
> >>>>> offers us a suitable solution in the way I see ceph4win to be
> spread
> >>>> and
> >>>>> used... I m maybe wrong. And anything that will be done around
> >>>> ceph4win will
> >>>>> be public documented etc... And licensed the way that if
> someone
> >>>> want to
> >>>>> build a commercial solution on top of it, that would be a
> >>>> possibility.
> >>>>> My idea is to giveback somehow to ceph project and at same time
> >>>> forge a
> >>>>> better knowledge in ceph technologies. Because like many in
> libre
> >>>> world I
> >>>>> think the business is in the services around the software more
> than
> >>>> on the
> >>>>> software. That the ones writing code should be financed and
> benefits
> >>>> from
> >>>>> the one selling and giving support of the software at all
> levels. I
> >>>> m
> >>>>> probably too idealistic. And too optimistic after all I m the
> one
> >>>> saying I
> >>>>> will do this stuff I have no idea how but well it is interesting
> and
> >>>> fun so
> >>>>> lets do it.
> >>>>>
> >>>>> Regards,
> >>>>>
> >>>>> P.S: using commercial backend libraries appart including their
> own
> >>>> cost will
> >>>>> force you to use commercial IDE like MS VisualStudio because
> their
> >>>> library
> >>>>> has some kind of drm that only that IDE compiler can use. So
> alot of
> >>>> cost
> >>>>> and yet there is nothing done. If I had to open a kickstarter
> >>>> project saying
> >>>>> we need 60 000 USD to do ceph4win with that monney we will buy
> the
> >>>> right to
> >>>>> use and share a commercial copyrighted library but abandonned
> >>>> punctually to
> >>>>> us in public domaine and that we will eventually produce
> something
> >>>> out of
> >>>>> it. I doubt I will get a dollar.
> >>>>>
> >>>>> We still can suggest the idea to Edlos the commercial company
> that
> >>>> has the
> >>>>> copyright of Callback FS, Or to buy them their product in a
> blender
> >>>> way
> >>>>> (blender was bought with donation before being put opensource
> and
> >>>> public
> >>>>> domaine), Or to open source their library. But in commercial
> minds
> >>>>> opensourcing = death of their technical advantage and death of
> >>>> their
> >>>>> marketing strategy. They will have to invent something more to
> >>>> retrieve
> >>>>> monney from it.
> >>>>>
> >>>>> El nov 6, 2013 11:22 p.m., "Ketor D" <d.ketor@gmail.com
> >>>>> <mailto:d.ketor@gmail.com>> escribió:
> >>>>>
> >>>>>
> >>>>> Hi Alphe,
> >>>>> I think you could try Callback Filesystem dev
> >>>> framework. It
> >>>>> is a commerical dev framework and is maintained by Edlos
> today.
> >>>>> I have communicated with Edlos to get a try code
> for
> >>>>> development. To dokan, Callback Filesystem has vary document
> and
> >>>> maybe
> >>>>> more stabilize.
> >>>>>
> >>>>> Regards.
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Thu, Nov 7, 2013 at 10:00 AM, Alphe Salas
> <asalas@kepler.cl
> >>>>> <mailto:asalas@kepler.cl>> wrote:
> >>>>> > Hello ketor thank you for your interest un ceph4win.
> Since
> >>>> muy
> >>>>> first mail I
> >>>>> > exposed the lacks of dokanfs and that I m far from being
> a
> >>>>> specialist un
> >>>>> > filesystems.
> >>>>> > I exposed what i like un dokanfs bit I not a fanátic of
> it.
> >>>> Muy
> >>>>> goal is to
> >>>>> > have something working quickly.
> >>>>> >
> >>>>> > So I am up to any proposición sure the one with the more
> docs
> >>>> and
> >>>>> support
> >>>>> > will be the best choice. As for right now what I need is
> >>>>> understand what are
> >>>>> > the files involved what are the interfaces functions and
> what
> >>>> are
> >>>>> the needed
> >>>>> > library dependencies and if they exist ported to windows
> with
> >>>>> cygwin. And
> >>>>> > all that is retrieved from source code.
> >>>>> >
> >>>>> > Regards.
> >>>>> >
> >>>>> > El nov 6, 2013 10:34 p.m., "Ketor D" <d.ketor@gmail.com
> >>>>> <mailto:d.ketor@gmail.com>> escribió:
> >>>>>
> >>>>> >
> >>>>> >> Hi Alphe,
> >>>>> >> We are taking an interest in your work on Ceph
> Client
> >>>> for
> >>>>> Windows
> >>>>> >> with Dokan.As we know, the performance of Dokan is not
> very
> >>>>> good, and it's
> >>>>> >> abandoned 3 years ago.
> >>>>> >> I have learned and used OpenDedup(SDFS) for a long
> >>>> time.
> >>>>> OpenDedup
> >>>>> >> has a Dokan version. And the author of OpenDedup said
> >>>>> >>
> >>>>> >> The Dokan library is quite flakey and testing should
> be
> >>>>> performed before
> >>>>> >> putting into production
> >>>>> >>
> >>>>> >> So what do you think about this? And if there is
> >>>> another
> >>>>> solution of
> >>>>> >> fuse-like filesystem dev framwork on Windows?
> >>>>> >>
> >>>>> >> Best Wish!
> >>>>> >>
> >>>>> >>
> >>>>> >>
> >>>>> >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe Salas Michels
> >>>>> <asalas@kepler.cl <mailto:asalas@kepler.cl>>
> >>>>>
> >>>>> >> wrote:
> >>>>> >>>
> >>>>> >>> Hello I created the github repository for this project
> >>>>> >>>https://github.com/alphe/Ceph4Win
> >>>>> >>>
> >>>>> >>> Regards,
> >>>>> >>>
> >>>>> >>> signature
> >>>>> >>>
> >>>>> >>> *Alphé Salas*
> >>>>> >>> Ingeniero T.I
> >>>>> >>>
> >>>>> >>>asalas@kepler.cl <mailto:asalas@kepler.cl>
> >>>>>
> >>>>> >>> *<http://www.kepler.cl>*
> >>>>> >>>
> >>>>> >>> On 11/05/13 21:00, Sage Weil wrote:
> >>>>> >>>>
> >>>>> >>>> Hi Alphe,
> >>>>> >>>>
> >>>>> >>>> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
> >>>>> >>>>>
> >>>>> >>>>> signature *Hi, Sage !
> >>>>> >>>>> thank you for you enthousiast reply.
> >>>>> >>>>> I sure want to make the best use of everything or
> >>>> anything
> >>>>> previously
> >>>>> >>>>> done to
> >>>>> >>>>> tend to
> >>>>> >>>>> write ceph cliente for windows.
> >>>>> >>>>>
> >>>>> >>>>> Apart using libre tools for building the future ceph
> >>>> cliente
> >>>>> I am open
> >>>>> >>>>> to
> >>>>> >>>>> anything.
> >>>>> >>>>> I would recommand eclipse CDT or Code::BLocks they
> are
> >>>> based
> >>>>> on mingwin
> >>>>> >>>>> open
> >>>>> >>>>> and easyly enhanceable.**
> >>>>> >>>>>
> >>>>> >>>>> more free tools can be found here:
> >>>>> >>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
> >>>>> >>>>>
> >>>>> >>>>>
> >>>>> >>>>> I will read libcephfs source code and take some
> notes
> >>>> about the
> >>>>> >>>>> protocol.
> >>>>> >>>>
> >>>>> >>>> I think you don't need to worry about hte protocol at
> all,
> >>>> since
> >>>>> >>>> libcephs
> >>>>> >>>> implements it for you (and will capture any future
> >>>> changes).
> >>>>> >>>>
> >>>>> >>>>> I was more going from what I know and trying to track
> down
> >>>> how
> >>>>> >>>>> mount.ceph work
> >>>>> >>>>> with the parameters passed to it.
> >>>>> >>>>> since it point finally to Kernel/fs/ceph and that I
> don t
> >>>> really
> >>>>> >>>>> understand
> >>>>> >>>>> how that module work and that it probably points to
> some
> >>>> other
> >>>>> >>>>> dependencies
> >>>>> >>>>> Reading libcephfs source code could be a big gain of
> >>>> time.
> >>>>> >>>>
> >>>>> >>>> (I would also ignore mount.ceph as everything it does
> it
> >>>>> specific to
> >>>>> >>>> how Linux mounts work.)
> >>>>> >>>>
> >>>>> >>>>> basically on the protocol what is need are:
> >>>>> >>>>>
> >>>>> >>>>> 1) open and maintain a connection (socket open, auth,
> etc
> >>>> )
> >>>>> >>>>> 2) retreive a map of directories and disk Quota
> (disk
> >>>> sizing
> >>>>> Y TB free,
> >>>>> >>>>> Z TB
> >>>>> >>>>> total)
> >>>>> >>>>> 3) procedure to send files / directories in a maner
> that
> >>>> it
> >>>>> will allow
> >>>>> >>>>> our
> >>>>> >>>>> client to fit ceph transmission protocols
> >>>>> >>>>> (limit bandwith for stability?, limit connection
> amount?,
> >>>>> limit cpu
> >>>>> >>>>> use?,
> >>>>> >>>>> Cache for preparing data transfer (a FIFO cache)?)
> >>>>> >>>>> 4)Procedure to retreive files / directory from ceph
> >>>> cluster
> >>>>> >>>>> 5) Management copy/move files /Directories, FS
> stats,
> >>>>> Connection Stats.
> >>>>> >>>>> logging.
> >>>>> >>>>>
> >>>>> >>>>> My idea to progress is to take those main bulletpoint
> in
> >>>> ceph
> >>>>> protocol
> >>>>> >>>>> based
> >>>>> >>>>> on general ideas of what ceph file system does and
> start
> >>>>> identifying
> >>>>> >>>>> parts
> >>>>> >>>>> from libcephfs to match those "needs".
> >>>>> >>>>
> >>>>> >>>> Instead, I would look at include/cephfs/libcephfs.h,
> the
> >>>>> interface that
> >>>>> >>>> libcephfs provides, and try to map that to what the
> fuse
> >>>> layer
> >>>>> expects.
> >>>>> >>>> There is both a path-based that I suspsect lends
> itself
> >>>> well
> >>>>> to the
> >>>>> >>>> Windows interface and (very soon now) a handle based
> API
> >>>> that is
> >>>>> >>>> targetted
> >>>>> >>>> at the Unix-style VFS layers. I'm mostly guessing,
> >>>> though,
> >>>>> since I've
> >>>>> >>>> never seen any low-level fs code in windows before.
> >>>>> >>>>
> >>>>> >>>> In this case, the analogous code for Linux should be
> >>>>> client/fuse_ll.cc
> >>>>> >>>> itself (and not much else), although there will
> probably be
> >>>> a
> >>>>> few tricks
> >>>>> >>>> necessary to map cleanly onto how the windows
> interfaces
> >>>> work.
> >>>>> >>>>
> >>>>> >>>> Does that make sense?
> >>>>> >>>>
> >>>>> >>>> Cheers!
> >>>>> >>>> sage
> >>>>> >>>>
> >>>>> >>>>
> >>>>> >>>>> Any suggestion and contributions are welcome.
> >>>>> >>>>>
> >>>>> >>>>>
> >>>>> >>>>> *
> >>>>> >>>>> On 11/05/13 11:23, Sage Weil wrote:
> >>>>> >>>>>>
> >>>>> >>>>>> Hi Alphe,
> >>>>> >>>>>>
> >>>>> >>>>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
> >>>>> >>>>>>>
> >>>>> >>>>>>> Good day developers!
> >>>>> >>>>>>>
> >>>>> >>>>>>> I would like to propose to the one interested work
> with
> >>>> me to
> >>>>> >>>>>>> develop a
> >>>>> >>>>>>> ceph
> >>>>> >>>>>>> cliente for MS windows world, Basing us on
> dokanFS.
> >>>>> >>>>>>>
> >>>>> >>>>>>> My company is a ceph enthousiast that use on a
> dayly
> >>>> basis
> >>>>> ceph and
> >>>>> >>>>>>> that
> >>>>> >>>>>>> need
> >>>>> >>>>>>> both transfer speed and big expendable and cheap
> >>>> storage.
> >>>>> >>>>>>> My company is specialised in data recovery and we
> want
> >>>> to
> >>>>> participate
> >>>>> >>>>>>> to
> >>>>> >>>>>>> ceph
> >>>>> >>>>>>> effort by bringing a ceph cliente for windows.
> >>>>> >>>>>>
> >>>>> >>>>>> Awesome!
> >>>>> >>>>>>
> >>>>> >>>>>>> Our experience shows us that the best gateway is
> each
> >>>>> clientes being
> >>>>> >>>>>>> its
> >>>>> >>>>>>> own
> >>>>> >>>>>>> gateway, instead of having a bottle neck server or
> a
> >>>> cluster of
> >>>>> >>>>>>> bottle
> >>>>> >>>>>>> neck
> >>>>> >>>>>>> servers as gateway (FTP, samba, SFTP,webdav, s3,
> >>>> etc..).
> >>>>> >>>>>>>
> >>>>> >>>>>>> We already did some research in that domain.
> >>>>> >>>>>>>
> >>>>> >>>>>>> Dokan FS is an intent to write an opensource fuse
> like
> >>>>> cliente for
> >>>>> >>>>>>> MS
> >>>>> >>>>>>> windows.
> >>>>> >>>>>>>
> >>>>> >>>>>>> More information on DOKANFS can be triggered here
> >>>>> >>>>>>>http://dokan-dev.net/en/download/
> >>>>> >>>>>>>
> >>>>> >>>>>>> Positive points of using DOKANFS.
> >>>>> >>>>>>>
> >>>>> >>>>>>> - its opensourced and well licenced mit licence,
> gpl
> >>>>> licence and lgpl
> >>>>> >>>>>>> licence.
> >>>>> >>>>>>>
> >>>>> >>>>>>> Negative point of using DOKAN FS.
> >>>>> >>>>>>> - unreachable author
> >>>>> >>>>>>> - Poor documentation . Dev comments in japanese.
> >>>>> >>>>>>> - Work in progress so it is unstable and needs to
> be
> >>>> updated,
> >>>>> >>>>>>> debugged and
> >>>>> >>>>>>> maintained by a MS Windows file system expert
> >>>> developper.
> >>>>> >>>>>>
> >>>>> >>>>>> I am not very familiar with windows storage APIs,
> but
> >>>>> somebody told me
> >>>>> >>>>>> at once point there were several interfaces against
> which
> >>>> a
> >>>>> new file
> >>>>> >>>>>> system could be implemented, everything from a full
> >>>>> in-kernel driver
> >>>>> >>>>>> to
> >>>>> >>>>>> something that is explorer-based. Are any of those
> >>>>> suitable? Using a
> >>>>> >>>>>> potentially abandoned fuse-like layer makes me a
> bit
> >>>> nervous.
> >>>>> >>>>>>
> >>>>> >>>>>> That said,
> >>>>> >>>>>>
> >>>>> >>>>>>>
> >>>>> >>>>>>> I try past year to do a merge from ceph-fuse to
> dokanfs
> >>>>> >>>>>>> here are what I learnt.
> >>>>> >>>>>>> - Ceph-fuse and related source code is around 60
> 000
> >>>> lines
> >>>>> of code.
> >>>>> >>>>>>> - Ceph protocol isn t documented so it is like
> trying
> >>>> to
> >>>>> draw a map
> >>>>> >>>>>>> of
> >>>>> >>>>>>> america
> >>>>> >>>>>>> using only a sextan and a compass.
> >>>>> >>>>>>>
> >>>>> >>>>>>> Those led me to those conclusions:
> >>>>> >>>>>>> - I can t do it alone.
> >>>>> >>>>>>> - It is easier to draw down the ceph protocol way
> to
> >>>> work from
> >>>>> >>>>>>> kernel/fs/ceph
> >>>>> >>>>>>> sources and mount.ceph
> >>>>> >>>>>>> - Ceph depending libraries may be unexistant or not
> up
> >>>> to
> >>>>> date in
> >>>>> >>>>>>> their
> >>>>> >>>>>>> port
> >>>>> >>>>>>> on MS Windows (cygwin)
> >>>>> >>>>>>
> >>>>> >>>>>> I think the most sane path should be to make
> libcephfs
> >>>>> sufficiently
> >>>>> >>>>>> portable to build on windows (or cygwin). For the
> bits
> >>>> used
> >>>>> by the
> >>>>> >>>>>> client-side coe, I don't think there should be much
> in
> >>>> the
> >>>>> way of
> >>>>> >>>>>> dependencies, and the main challenge would be
> untangling
> >>>> the
> >>>>> build for
> >>>>> >>>>>> the necessary pieces out from the rest of Ceph.
> >>>>> >>>>>>
> >>>>> >>>>>> Have you seen the wip-port portability work that is
> >>>>> currently underway
> >>>>> >>>>>> by
> >>>>> >>>>>> Noah and Alan? That may solve many of the cygwin
> >>>> problems
> >>>>> you are
> >>>>> >>>>>> seeing
> >>>>> >>>>>> today.
> >>>>> >>>>>>
> >>>>> >>>>>>> - MS file system specialist are hard do find in
> the
> >>>> "open
> >>>>> source
> >>>>> >>>>>>> libre
> >>>>> >>>>>>> world"
> >>>>> >>>>>>> so I will try in the commercial world.
> >>>>> >>>>>>>
> >>>>> >>>>>>> The commercial world has some problems too. They
> need
> >>>> ceph
> >>>>> protocol
> >>>>> >>>>>>> draft
> >>>>> >>>>>>> to
> >>>>> >>>>>>> implemente it to their own product They will have
> >>>> licencing
> >>>>> >>>>>>> /commercial
> >>>>> >>>>>>> politics that infringe lgpl, and hide that most of
> the
> >>>> work
> >>>>> is done
> >>>>> >>>>>>> by
> >>>>> >>>>>>> people
> >>>>> >>>>>>> other than them. They will not participate in a
> >>>> financial
> >>>>> way to ceph
> >>>>> >>>>>>> enhancement and growth.
> >>>>> >>>>>>
> >>>>> >>>>>> I don't think reimplementing the client code is an
> >>>> efficient way
> >>>>> >>>>>> forward.
> >>>>> >>>>>> Unless the goal is a pure kernel
> implementation...but a
> >>>>> significant
> >>>>> >>>>>> ongoing investment in development resources would
> be
> >>>> needed
> >>>>> for that
> >>>>> >>>>>> going
> >>>>> >>>>>> forward. I suspect that is a challenge for a
> platform
> >>>> that
> >>>>> does not
> >>>>> >>>>>> typically rally that sort of community effort.
> >>>>> >>>>>>
> >>>>> >>>>>> The easiest thing is of course just to use CIFS and
> >>>> Samba
> >>>>> (which works
> >>>>> >>>>>> today). A fuse-like approach is probably a
> reasonably
> >>>>> middle ground
> >>>>> >>>>>> (both
> >>>>> >>>>>> in initial effort and maintainability going
> forward)...
> >>>>> >>>>>>
> >>>>> >>>>>> sage
> >>>>> >>>>>>
> >>>>> >>>>>>
> >>>>> >>>>>
> >>>>> >>>
> >>>>> >>> --
> >>>>> >>> To unsubscribe from this list: send the line
> "unsubscribe
> >>>>> ceph-devel" in
> >>>>> >>> the body of a message tomajordomo@vger.kernel.org
> >>>>> <mailto:majordomo@vger.kernel.org>
> >>>>>
> >>>>> >>> More majordomo info at
> >>>> http://vger.kernel.org/majordomo-info.html
> >>>>> >>
> >>>>> >>
> >>>>> >
> >>>>>
> >>>>>
> >>>>>
> >>>> --
> >>>> To unsubscribe from this list: send the line "unsubscribe
> ceph-devel"
> >>>> in
> >>>> the body of a message tomajordomo@vger.kernel.org
> >>>> More majordomo info athttp://vger.kernel.org/majordomo-info.html
> >>
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe
> ceph-devel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI 48104
http://linuxbox.com
tel. 734-761-4689
fax. 734-769-8938
cel. 734-216-5309
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
[not found] ` <CAME-gARbjR++qXsx9Sx4KbuggthONrscHToxCTUKrci_MLMDzw@mail.gmail.com>
@ 2013-11-08 14:15 ` Alphe Salas Michels
2014-12-26 17:10 ` Ketor D
0 siblings, 1 reply; 21+ messages in thread
From: Alphe Salas Michels @ 2013-11-08 14:15 UTC (permalink / raw)
To: ceph-devel, Ketor D, Matt W. Benjamin
Hello malcom and matt thank you for apporting some more information
source. OpenAFS is sure interesting httpfs too.
I hope it will help us on deciding the best path to follow in our
interface with window.
Actually I still trying to isolate the needed client code in the
shortest way possible.
Regards.
Alphe Salas
El nov 7, 2013 9:11 p.m., "Malcolm Haak" <malcolm@sgi.com
<mailto:malcolm@sgi.com>> escribió:
I'm just going to throw these in there.
http://www.acc.umu.se/~bosse/ <http://www.acc.umu.se/%7Ebosse/>
They are GPLv2 some already use sockets and such from inside the
kernel. Heck you might even be able to mod the HTTP one to use
rados gateway. I don't know as I havent sat down and pulled them
apart enough yet.
They might help, but they might be useless. Not sure.
On 08/11/13 06:47, Alphe Salas Michels wrote:
Hello all I finally finished my first source code extraction
that starts
from ceph/src/client/fuse_ll.c
The result is accurate unlike previous provided results.
basically the
script start from a file extract all the private includes
definitions
#include "something.h" and recursively extract private includes
too. the
best way to know who is related with who.
starting from fuse_ll.cc I optain 390 files retreived and 120
000 lines
of code !
involved dirs are : in ceph/src
objclass/, common/, msg/, common/, osdc/, include/, client/, mds/,
global/, json_spirit/, log/, os/, crush/, mon/, osd/, auth/
probably not a good way to analyse what amount of work it means
since
most of those directories are the implementation of servers
(osd, mon,
mds) and even if only a tiny bit of them is needed at client
level. you
need two structures from ./osd/OSD.h and my script by relation will
take into acount the whole directory...
I ran the script with libcephfs.cc as start point and got almost the
same results. 131 000 lines of code and 386 files most of the
same dirs
involved.
I think I will spend alot of time doing the manual source code
isolation
and understand way each #include is set in the files I read (what
purpose they have do they allow to integrate a crucial data type
or not.
The other way around will be to read src/libcephfs.cc. It seems
shorter
but without understanding what part is used for each included
header I
can t say anything...
I will keep reading the source code and take notes. I think in
the case
of libcephfs I will gain alot of time.
signature
*Alphé Salas*
Ingeniero T.I
asalas@kepler.cl <mailto:asalas@kepler.cl>
*www.kepler.cl <http://www.kepler.cl> <http://www.kepler.cl>*
On 11/07/13 15:02, Alphe Salas Michels wrote:
Hello D.Ketor and Matt Benjamin,
You give me alot to think about and this is great!
I merged your previous post to make a single reply that
anyone can
report to easyly
Windows NFS 4.1 is available here:
http://www.citi.umich.edu/projects/nfsv4/windows/readme.html
pnfs is another name for NFS4.X. It is presented as
alternative to
ceph and we get known terminology as MDS and OSD but without
the self
healing part if I understand well my rapid look on the
topic. (when I
say rapid look I mean ... 5 minutes spent in that... which
is really
small amount of time to get an accurate view on something)
starting from mount.ceph ... I know that mount.ceph does
little but it
is a great hint to know what ceph needs and do things.
Basically mount.ceph modprobe the ceph driver in the linux
kernel then
call mount with the line command passed args and the cephfs
type as
argument. Then the kernel does the work I don t understand
yet what is
the start calls that are made to the ceph driver but it
seemed to me
that is was relatively light. (a first impression compared
to ceph-fuse.)
I think I will do both isolate source code from ceph-client
kernel
(cephfs module for linux kernel) and the one pointed by Sage
starting
from client/fuse_ll.cc in ceph master branch. The common
files betwin
those 2 extractions will be our core set of mandatory features.
Then we try to compile with cygwin a cephfs client library .
Then we
will try to interface with a modified windows nfs 4.1 client
or pnfs
or any other that will accept to be compiled with gcc for
win32...
the fact that windows 8.1 is and windows 2012 are out of
reach at the
moment is not a problem to me.
Our first concern is to understand what is ceph protocol.
Then adapt
it to something that can be used on windows prior windows
8.1. Dokan
fs if I remember well use too the WDK (windows driver
dev-kit ) for it
s compilation so possibly we will see the same limitations.
We need to multiply our source of information by example
regarding
ceph-client (kernel or fuse, radosgw is on a different layer
so I will
not try anything around it at first.) And we need to
multiply our
source of information by example regarding virtual file system
technologies on windoes OS.
Alot of work but all of those available source code everyone
point at
me will make our best solution. And in the end we will choose
technologies knowing what we do and what concequencies they
have.
regards,
Regards
signature
*Alphé Salas*
Ingeniero T.I
asalas@kepler.cl <mailto:asalas@kepler.cl>
On 11/07/13 11:29, Ketor D wrote:
Hi Alphe:
Yes Callback Filesystem is very expensive and
can't open source.
It's not a good choice for ceph4win.
Another way for ceph4win maybe develop a
kernel-mode fs like
pnfs. pnfs has a kernel-mode windows client. I think you
can read its
src code and maybe migrating from ceph kernel client to
windows kernel
fs is easier than from userspace ceph fuse client.And a
kernel-mode fs
client has greater performance than userspace fs like
ceph-fuse client
and ceph kernel client.
Regards.
On 11/07/13 11:50, Matt W. Benjamin wrote:
Hi,
The Window NFS v4.1 client is what we work on, so this
may be good for
code sharing. The license is lgplv2, like Ceph's.
Something important to be aware of is that the client
uses rdbss, which
is a (partial) fsd abstraction that simplified
implementation
quite a bit, kind of like a mini driver. However,
Microsoft's support
for rdbss has been in limbo for a bit. For example, to
link with
the rdbss symbols you can't use the Windows 8 driver
kit--you'll need
to use the one for Windows 7. (There's a private rdbss2
used internally
by Microsoft's SMB implemenation. A the moment, 3rd
party drivers
can't use that.)
We've been in communication with Microsoft about this
issue, and know of
a few other fsds using it, but it could be a good thing
for that
lobbying
effort to have another user--or it could be a dead end :(.
There are a couple of other choices if you're looking to
go this route,
that I'm aware of (and we may need to take them too, if
RDBSS has no
way forward), but the required work could be a lot larger.
Matt
----- "Ketor D"<d.ketor@gmail.com
<mailto:d.ketor@gmail.com>> wrote:
Hi Alphe:
Yes Callback Filesystem is very expensive
and can't open
source.
It's not a good choice for ceph4win.
Another way for ceph4win maybe develop a
kernel-mode fs like
pnfs. pnfs has a kernel-mode windows client. I think
you can read its
src code and maybe migrating from ceph kernel client
to windows
kernel
fs is easier than from userspace ceph fuse
client.And a kernel-mode
fs
client has greater performance than userspace fs
like ceph-fuse
client
and ceph kernel client.
Regards.
On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas
Michels<asalas@kepler.cl <mailto:asalas@kepler.cl>>
wrote:
Commercial libraries are a pain ...
If we want the more permossive licence offered
by callback file
system we
have to buy it for 20.000 usd. Then we will have
to provide a
backbox that
we have no control upon and that will kill our
product anytime they
want anf
if they decide to stop their commercial activity
we will be in the
same
situation that with dokanfs but without having
the source code of
the black
box. If i have to spend 20 000 dollars i would
prefere paying
someone to
retake dokanfs or to write from scratch a
dokanfs fuselike software
make it
all shiny and pumpy fantastic and ready to plug
to ceph client.
I would prefere if people have to pay something
to get access to
ceph4win
that this money goes in ceph main branch
pockets... Or as a gift you
donante
to ceph 10 dollars you get 2 free registration
codes for
ceph4win... or
something like that.
If ceph4win as to be comercial then I would
prefer delegate the task
to a
company like south river technologies and their
great product
webdrive. I
would mininaly get involved in that project and
simply buy the final
product
to sell it together with my ceph based product
(which could be a
calxeda
ceph box or something like that).
I m open anyway to any proposition. But I doubt
that callback
filesystem
offers us a suitable solution in the way I see
ceph4win to be spread
and
used... I m maybe wrong. And anything that will
be done around
ceph4win will
be public documented etc... And licensed the way
that if someone
want to
build a commercial solution on top of it, that
would be a
possibility.
My idea is to giveback somehow to ceph project
and at same time
forge a
better knowledge in ceph technologies. Because
like many in libre
world I
think the business is in the services around the
software more than
on the
software. That the ones writing code should be
financed and benefits
from
the one selling and giving support of the
software at all levels. I
m
probably too idealistic. And too optimistic
after all I m the one
saying I
will do this stuff I have no idea how but well
it is interesting and
fun so
lets do it.
Regards,
P.S: using commercial backend libraries appart
including their own
cost will
force you to use commercial IDE like MS
VisualStudio because their
library
has some kind of drm that only that IDE compiler
can use. So alot of
cost
and yet there is nothing done. If I had to open
a kickstarter
project saying
we need 60 000 USD to do ceph4win with that
monney we will buy the
right to
use and share a commercial copyrighted library
but abandonned
punctually to
us in public domaine and that we will
eventually produce something
out of
it. I doubt I will get a dollar.
We still can suggest the idea to Edlos the
commercial company that
has the
copyright of Callback FS, Or to buy them their
product in a blender
way
(blender was bought with donation before being
put opensource and
public
domaine), Or to open source their library. But
in commercial minds
opensourcing = death of their technical
advantage and death of
their
marketing strategy. They will have to invent
something more to
retrieve
monney from it.
El nov 6, 2013 11:22 p.m., "Ketor D"
<d.ketor@gmail.com <mailto:d.ketor@gmail.com>
<mailto:d.ketor@gmail.com
<mailto:d.ketor@gmail.com>>> escribió:
Hi Alphe,
I think you could try Callback
Filesystem dev
framework. It
is a commerical dev framework and is
maintained by Edlos today.
I have communicated with Edlos to
get a try code for
development. To dokan, Callback Filesystem
has vary document and
maybe
more stabilize.
Regards.
On Thu, Nov 7, 2013 at 10:00 AM, Alphe
Salas <asalas@kepler.cl <mailto:asalas@kepler.cl>
<mailto:asalas@kepler.cl
<mailto:asalas@kepler.cl>>> wrote:
> Hello ketor thank you for your interest
un ceph4win. Since
muy
first mail I
> exposed the lacks of dokanfs and that I
m far from being a
specialist un
> filesystems.
> I exposed what i like un dokanfs bit I
not a fanátic of it.
Muy
goal is to
> have something working quickly.
>
> So I am up to any proposición sure the
one with the more docs
and
support
> will be the best choice. As for right
now what I need is
understand what are
> the files involved what are the
interfaces functions and what
are
the needed
> library dependencies and if they exist
ported to windows with
cygwin. And
> all that is retrieved from source code.
>
> Regards.
>
> El nov 6, 2013 10:34 p.m., "Ketor D"
<d.ketor@gmail.com <mailto:d.ketor@gmail.com>
<mailto:d.ketor@gmail.com
<mailto:d.ketor@gmail.com>>> escribió:
>
>> Hi Alphe,
>> We are taking an interest in your
work on Ceph Client
for
Windows
>> with Dokan.As we know, the performance
of Dokan is not very
good, and it's
>> abandoned 3 years ago.
>> I have learned and used
OpenDedup(SDFS) for a long
time.
OpenDedup
>> has a Dokan version. And the author of
OpenDedup said
>>
>> The Dokan library is quite flakey and
testing should be
performed before
>> putting into production
>>
>> So what do you think about this?
And if there is
another
solution of
>> fuse-like filesystem dev framwork on
Windows?
>>
>> Best Wish!
>>
>>
>>
>> On Thu, Nov 7, 2013 at 5:47 AM, Alphe
Salas Michels
<asalas@kepler.cl <mailto:asalas@kepler.cl>
<mailto:asalas@kepler.cl <mailto:asalas@kepler.cl>>>
>> wrote:
>>>
>>> Hello I created the github repository
for this project
>>>https://github.com/alphe/Ceph4Win
>>>
>>> Regards,
>>>
>>> signature
>>>
>>> *Alphé Salas*
>>> Ingeniero T.I
>>>
>>>asalas@kepler.cl
<mailto:asalas@kepler.cl>
<mailto:asalas@kepler.cl <mailto:asalas@kepler.cl>>
>>> *<http://www.kepler.cl>*
>>>
>>> On 11/05/13 21:00, Sage Weil wrote:
>>>>
>>>> Hi Alphe,
>>>>
>>>> On Tue, 5 Nov 2013, Alphe Salas
Michels wrote:
>>>>>
>>>>> signature *Hi, Sage !
>>>>> thank you for you enthousiast reply.
>>>>> I sure want to make the best use of
everything or
anything
previously
>>>>> done to
>>>>> tend to
>>>>> write ceph cliente for windows.
>>>>>
>>>>> Apart using libre tools for building
the future ceph
cliente
I am open
>>>>> to
>>>>> anything.
>>>>> I would recommand eclipse CDT or
Code::BLocks they are
based
on mingwin
>>>>> open
>>>>> and easyly enhanceable.**
>>>>>
>>>>> more free tools can be found here:
>>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
>>>>>
>>>>>
>>>>> I will read libcephfs source code
and take some notes
about the
>>>>> protocol.
>>>>
>>>> I think you don't need to worry about
hte protocol at all,
since
>>>> libcephs
>>>> implements it for you (and will
capture any future
changes).
>>>>
>>>>> I was more going from what I know
and trying to track down
how
>>>>> mount.ceph work
>>>>> with the parameters passed to it.
>>>>> since it point finally to
Kernel/fs/ceph and that I don t
really
>>>>> understand
>>>>> how that module work and that it
probably points to some
other
>>>>> dependencies
>>>>> Reading libcephfs source code could
be a big gain of
time.
>>>>
>>>> (I would also ignore mount.ceph as
everything it does it
specific to
>>>> how Linux mounts work.)
>>>>
>>>>> basically on the protocol what is
need are:
>>>>>
>>>>> 1) open and maintain a connection
(socket open, auth, etc
)
>>>>> 2) retreive a map of directories and
disk Quota (disk
sizing
Y TB free,
>>>>> Z TB
>>>>> total)
>>>>> 3) procedure to send files /
directories in a maner that
it
will allow
>>>>> our
>>>>> client to fit ceph transmission
protocols
>>>>> (limit bandwith for stability?,
limit connection amount?,
limit cpu
>>>>> use?,
>>>>> Cache for preparing data transfer (a
FIFO cache)?)
>>>>> 4)Procedure to retreive files /
directory from ceph
cluster
>>>>> 5) Management copy/move files
/Directories, FS stats,
Connection Stats.
>>>>> logging.
>>>>>
>>>>> My idea to progress is to take those
main bulletpoint in
ceph
protocol
>>>>> based
>>>>> on general ideas of what ceph file
system does and start
identifying
>>>>> parts
>>>>> from libcephfs to match those "needs".
>>>>
>>>> Instead, I would look at
include/cephfs/libcephfs.h, the
interface that
>>>> libcephfs provides, and try to map
that to what the fuse
layer
expects.
>>>> There is both a path-based that I
suspsect lends itself
well
to the
>>>> Windows interface and (very soon now)
a handle based API
that is
>>>> targetted
>>>> at the Unix-style VFS layers. I'm
mostly guessing,
though,
since I've
>>>> never seen any low-level fs code in
windows before.
>>>>
>>>> In this case, the analogous code for
Linux should be
client/fuse_ll.cc
>>>> itself (and not much else), although
there will probably be
a
few tricks
>>>> necessary to map cleanly onto how the
windows interfaces
work.
>>>>
>>>> Does that make sense?
>>>>
>>>> Cheers!
>>>> sage
>>>>
>>>>
>>>>> Any suggestion and contributions are
welcome.
>>>>>
>>>>>
>>>>> *
>>>>> On 11/05/13 11:23, Sage Weil wrote:
>>>>>>
>>>>>> Hi Alphe,
>>>>>>
>>>>>> On Mon, 4 Nov 2013, Alphe Salas
Michels wrote:
>>>>>>>
>>>>>>> Good day developers!
>>>>>>>
>>>>>>> I would like to propose to the one
interested work with
me to
>>>>>>> develop a
>>>>>>> ceph
>>>>>>> cliente for MS windows world,
Basing us on dokanFS.
>>>>>>>
>>>>>>> My company is a ceph enthousiast
that use on a dayly
basis
ceph and
>>>>>>> that
>>>>>>> need
>>>>>>> both transfer speed and big
expendable and cheap
storage.
>>>>>>> My company is specialised in data
recovery and we want
to
participate
>>>>>>> to
>>>>>>> ceph
>>>>>>> effort by bringing a ceph cliente
for windows.
>>>>>>
>>>>>> Awesome!
>>>>>>
>>>>>>> Our experience shows us that the
best gateway is each
clientes being
>>>>>>> its
>>>>>>> own
>>>>>>> gateway, instead of having a
bottle neck server or a
cluster of
>>>>>>> bottle
>>>>>>> neck
>>>>>>> servers as gateway (FTP, samba,
SFTP,webdav, s3,
etc..).
>>>>>>>
>>>>>>> We already did some research in
that domain.
>>>>>>>
>>>>>>> Dokan FS is an intent to write an
opensource fuse like
cliente for
>>>>>>> MS
>>>>>>> windows.
>>>>>>>
>>>>>>> More information on DOKANFS can be
triggered here
>>>>>>>http://dokan-dev.net/en/download/
>>>>>>>
>>>>>>> Positive points of using DOKANFS.
>>>>>>>
>>>>>>> - its opensourced and well
licenced mit licence, gpl
licence and lgpl
>>>>>>> licence.
>>>>>>>
>>>>>>> Negative point of using DOKAN FS.
>>>>>>> - unreachable author
>>>>>>> - Poor documentation . Dev
comments in japanese.
>>>>>>> - Work in progress so it is
unstable and needs to be
updated,
>>>>>>> debugged and
>>>>>>> maintained by a MS Windows file
system expert
developper.
>>>>>>
>>>>>> I am not very familiar with windows
storage APIs, but
somebody told me
>>>>>> at once point there were several
interfaces against which
a
new file
>>>>>> system could be implemented,
everything from a full
in-kernel driver
>>>>>> to
>>>>>> something that is explorer-based.
Are any of those
suitable? Using a
>>>>>> potentially abandoned fuse-like
layer makes me a bit
nervous.
>>>>>>
>>>>>> That said,
>>>>>>
>>>>>>>
>>>>>>> I try past year to do a merge from
ceph-fuse to dokanfs
>>>>>>> here are what I learnt.
>>>>>>> - Ceph-fuse and related source
code is around 60 000
lines
of code.
>>>>>>> - Ceph protocol isn t documented
so it is like trying
to
draw a map
>>>>>>> of
>>>>>>> america
>>>>>>> using only a sextan and a compass.
>>>>>>>
>>>>>>> Those led me to those conclusions:
>>>>>>> - I can t do it alone.
>>>>>>> - It is easier to draw down the
ceph protocol way to
work from
>>>>>>> kernel/fs/ceph
>>>>>>> sources and mount.ceph
>>>>>>> - Ceph depending libraries may be
unexistant or not up
to
date in
>>>>>>> their
>>>>>>> port
>>>>>>> on MS Windows (cygwin)
>>>>>>
>>>>>> I think the most sane path should
be to make libcephfs
sufficiently
>>>>>> portable to build on windows (or
cygwin). For the bits
used
by the
>>>>>> client-side coe, I don't think
there should be much in
the
way of
>>>>>> dependencies, and the main
challenge would be untangling
the
build for
>>>>>> the necessary pieces out from the
rest of Ceph.
>>>>>>
>>>>>> Have you seen the wip-port
portability work that is
currently underway
>>>>>> by
>>>>>> Noah and Alan? That may solve many
of the cygwin
problems
you are
>>>>>> seeing
>>>>>> today.
>>>>>>
>>>>>>> - MS file system specialist are
hard do find in the
"open
source
>>>>>>> libre
>>>>>>> world"
>>>>>>> so I will try in the commercial world.
>>>>>>>
>>>>>>> The commercial world has some
problems too. They need
ceph
protocol
>>>>>>> draft
>>>>>>> to
>>>>>>> implemente it to their own product
They will have
licencing
>>>>>>> /commercial
>>>>>>> politics that infringe lgpl, and
hide that most of the
work
is done
>>>>>>> by
>>>>>>> people
>>>>>>> other than them. They will not
participate in a
financial
way to ceph
>>>>>>> enhancement and growth.
>>>>>>
>>>>>> I don't think reimplementing the
client code is an
efficient way
>>>>>> forward.
>>>>>> Unless the goal is a pure kernel
implementation...but a
significant
>>>>>> ongoing investment in development
resources would be
needed
for that
>>>>>> going
>>>>>> forward. I suspect that is a
challenge for a platform
that
does not
>>>>>> typically rally that sort of
community effort.
>>>>>>
>>>>>> The easiest thing is of course just
to use CIFS and
Samba
(which works
>>>>>> today). A fuse-like approach is
probably a reasonably
middle ground
>>>>>> (both
>>>>>> in initial effort and
maintainability going forward)...
>>>>>>
>>>>>> sage
>>>>>>
>>>>>>
>>>>>
>>>
>>> --
>>> To unsubscribe from this list: send
the line "unsubscribe
ceph-devel" in
>>> the body of a message
tomajordomo@vger.kernel.org
<mailto:tomajordomo@vger.kernel.org>
<mailto:majordomo@vger.kernel.org
<mailto:majordomo@vger.kernel.org>>
>>> More majordomo info at
http://vger.kernel.org/majordomo-info.html
>>
>>
>
--
To unsubscribe from this list: send the line
"unsubscribe ceph-devel"
in
the body of a message tomajordomo@vger.kernel.org
<mailto:tomajordomo@vger.kernel.org>
More majordomo info
athttp://vger.kernel.org/majordomo-info.html
<http://vger.kernel.org/majordomo-info.html>
--
To unsubscribe from this list: send the line "unsubscribe
ceph-devel" in
the body of a message to majordomo@vger.kernel.org
<mailto:majordomo@vger.kernel.org>
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2013-11-08 14:15 ` Alphe Salas Michels
@ 2014-12-26 17:10 ` Ketor D
2014-12-27 5:14 ` Dong Yuan
0 siblings, 1 reply; 21+ messages in thread
From: Ketor D @ 2014-12-26 17:10 UTC (permalink / raw)
To: Alphe Salas Michels; +Cc: ceph-devel, Matt W. Benjamin
Is here any progress on Cephfs Windows Client ?
2013-11-08 22:15 GMT+08:00 Alphe Salas Michels <asalas@kepler.cl>:
> Hello malcom and matt thank you for apporting some more information source.
> OpenAFS is sure interesting httpfs too.
>
> I hope it will help us on deciding the best path to follow in our interface
> with window.
> Actually I still trying to isolate the needed client code in the shortest
> way possible.
>
> Regards.
>
> Alphe Salas
>
> El nov 7, 2013 9:11 p.m., "Malcolm Haak" <malcolm@sgi.com
> <mailto:malcolm@sgi.com>> escribió:
>
> I'm just going to throw these in there.
>
> http://www.acc.umu.se/~bosse/ <http://www.acc.umu.se/%7Ebosse/>
>
>
> They are GPLv2 some already use sockets and such from inside the
> kernel. Heck you might even be able to mod the HTTP one to use
> rados gateway. I don't know as I havent sat down and pulled them
> apart enough yet.
>
> They might help, but they might be useless. Not sure.
>
> On 08/11/13 06:47, Alphe Salas Michels wrote:
>
> Hello all I finally finished my first source code extraction
> that starts
> from ceph/src/client/fuse_ll.c
> The result is accurate unlike previous provided results.
> basically the
> script start from a file extract all the private includes
> definitions
> #include "something.h" and recursively extract private includes
> too. the
> best way to know who is related with who.
>
> starting from fuse_ll.cc I optain 390 files retreived and 120
> 000 lines
> of code !
> involved dirs are : in ceph/src
> objclass/, common/, msg/, common/, osdc/, include/, client/, mds/,
> global/, json_spirit/, log/, os/, crush/, mon/, osd/, auth/
>
> probably not a good way to analyse what amount of work it means
> since
> most of those directories are the implementation of servers
> (osd, mon,
> mds) and even if only a tiny bit of them is needed at client
> level. you
> need two structures from ./osd/OSD.h and my script by relation will
> take into acount the whole directory...
>
> I ran the script with libcephfs.cc as start point and got almost the
> same results. 131 000 lines of code and 386 files most of the
> same dirs
> involved.
>
>
>
> I think I will spend alot of time doing the manual source code
> isolation
> and understand way each #include is set in the files I read (what
> purpose they have do they allow to integrate a crucial data type
> or not.
>
>
> The other way around will be to read src/libcephfs.cc. It seems
> shorter
> but without understanding what part is used for each included
> header I
> can t say anything...
>
>
>
> I will keep reading the source code and take notes. I think in
> the case
> of libcephfs I will gain alot of time.
>
> signature
>
> *Alphé Salas*
> Ingeniero T.I
>
> asalas@kepler.cl <mailto:asalas@kepler.cl>
> *www.kepler.cl <http://www.kepler.cl> <http://www.kepler.cl>*
>
>
> On 11/07/13 15:02, Alphe Salas Michels wrote:
>
> Hello D.Ketor and Matt Benjamin,
> You give me alot to think about and this is great!
> I merged your previous post to make a single reply that
> anyone can
> report to easyly
>
> Windows NFS 4.1 is available here:
> http://www.citi.umich.edu/projects/nfsv4/windows/readme.html
>
> pnfs is another name for NFS4.X. It is presented as
> alternative to
> ceph and we get known terminology as MDS and OSD but without
> the self
> healing part if I understand well my rapid look on the
> topic. (when I
> say rapid look I mean ... 5 minutes spent in that... which
> is really
> small amount of time to get an accurate view on something)
>
>
> starting from mount.ceph ... I know that mount.ceph does
> little but it
> is a great hint to know what ceph needs and do things.
> Basically mount.ceph modprobe the ceph driver in the linux
> kernel then
> call mount with the line command passed args and the cephfs
> type as
> argument. Then the kernel does the work I don t understand
> yet what is
> the start calls that are made to the ceph driver but it
> seemed to me
> that is was relatively light. (a first impression compared
> to ceph-fuse.)
>
> I think I will do both isolate source code from ceph-client
> kernel
> (cephfs module for linux kernel) and the one pointed by Sage
> starting
> from client/fuse_ll.cc in ceph master branch. The common
> files betwin
> those 2 extractions will be our core set of mandatory features.
>
> Then we try to compile with cygwin a cephfs client library .
> Then we
> will try to interface with a modified windows nfs 4.1 client
> or pnfs
> or any other that will accept to be compiled with gcc for
> win32...
>
> the fact that windows 8.1 is and windows 2012 are out of
> reach at the
> moment is not a problem to me.
>
> Our first concern is to understand what is ceph protocol.
> Then adapt
> it to something that can be used on windows prior windows
> 8.1. Dokan
> fs if I remember well use too the WDK (windows driver
> dev-kit ) for it
> s compilation so possibly we will see the same limitations.
>
> We need to multiply our source of information by example
> regarding
> ceph-client (kernel or fuse, radosgw is on a different layer
> so I will
> not try anything around it at first.) And we need to
> multiply our
> source of information by example regarding virtual file system
> technologies on windoes OS.
> Alot of work but all of those available source code everyone
> point at
> me will make our best solution. And in the end we will choose
> technologies knowing what we do and what concequencies they
> have.
>
> regards,
>
>
>
>
> Regards
>
> signature
>
> *Alphé Salas*
> Ingeniero T.I
>
> asalas@kepler.cl <mailto:asalas@kepler.cl>
>
>
>
> On 11/07/13 11:29, Ketor D wrote:
>
> Hi Alphe:
> Yes Callback Filesystem is very expensive and
> can't open source.
> It's not a good choice for ceph4win.
> Another way for ceph4win maybe develop a
> kernel-mode fs like
> pnfs. pnfs has a kernel-mode windows client. I think you
> can read its
> src code and maybe migrating from ceph kernel client to
> windows kernel
> fs is easier than from userspace ceph fuse client.And a
> kernel-mode fs
> client has greater performance than userspace fs like
> ceph-fuse client
> and ceph kernel client.
>
> Regards.
>
> On 11/07/13 11:50, Matt W. Benjamin wrote:
>
> Hi,
>
> The Window NFS v4.1 client is what we work on, so this
> may be good for
> code sharing. The license is lgplv2, like Ceph's.
>
> Something important to be aware of is that the client
> uses rdbss, which
> is a (partial) fsd abstraction that simplified
> implementation
> quite a bit, kind of like a mini driver. However,
> Microsoft's support
> for rdbss has been in limbo for a bit. For example, to
> link with
> the rdbss symbols you can't use the Windows 8 driver
> kit--you'll need
> to use the one for Windows 7. (There's a private rdbss2
> used internally
> by Microsoft's SMB implemenation. A the moment, 3rd
> party drivers
> can't use that.)
>
> We've been in communication with Microsoft about this
> issue, and know of
> a few other fsds using it, but it could be a good thing
> for that
> lobbying
> effort to have another user--or it could be a dead end :(.
>
> There are a couple of other choices if you're looking to
> go this route,
> that I'm aware of (and we may need to take them too, if
> RDBSS has no
> way forward), but the required work could be a lot larger.
>
> Matt
>
> ----- "Ketor D"<d.ketor@gmail.com
> <mailto:d.ketor@gmail.com>> wrote:
>
> Hi Alphe:
> Yes Callback Filesystem is very expensive
> and can't open
> source.
> It's not a good choice for ceph4win.
> Another way for ceph4win maybe develop a
> kernel-mode fs like
> pnfs. pnfs has a kernel-mode windows client. I think
> you can read its
> src code and maybe migrating from ceph kernel client
> to windows
> kernel
> fs is easier than from userspace ceph fuse
> client.And a kernel-mode
> fs
> client has greater performance than userspace fs
> like ceph-fuse
> client
> and ceph kernel client.
>
> Regards.
>
>
>
> On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas
> Michels<asalas@kepler.cl <mailto:asalas@kepler.cl>>
>
> wrote:
>
> Commercial libraries are a pain ...
>
> If we want the more permossive licence offered
> by callback file
>
> system we
>
> have to buy it for 20.000 usd. Then we will have
> to provide a
>
> backbox that
>
> we have no control upon and that will kill our
> product anytime they
>
> want anf
>
> if they decide to stop their commercial activity
> we will be in the
>
> same
>
> situation that with dokanfs but without having
> the source code of
>
> the black
>
> box. If i have to spend 20 000 dollars i would
> prefere paying
>
> someone to
>
> retake dokanfs or to write from scratch a
> dokanfs fuselike software
>
> make it
>
> all shiny and pumpy fantastic and ready to plug
> to ceph client.
>
> I would prefere if people have to pay something
> to get access to
>
> ceph4win
>
> that this money goes in ceph main branch
> pockets... Or as a gift you
>
> donante
>
> to ceph 10 dollars you get 2 free registration
> codes for
>
> ceph4win... or
>
> something like that.
>
> If ceph4win as to be comercial then I would
> prefer delegate the task
>
> to a
>
> company like south river technologies and their
> great product
>
> webdrive. I
>
> would mininaly get involved in that project and
> simply buy the final
>
> product
>
> to sell it together with my ceph based product
> (which could be a
>
> calxeda
>
> ceph box or something like that).
>
> I m open anyway to any proposition. But I doubt
> that callback
>
> filesystem
>
> offers us a suitable solution in the way I see
> ceph4win to be spread
>
> and
>
> used... I m maybe wrong. And anything that will
> be done around
>
> ceph4win will
>
> be public documented etc... And licensed the way
> that if someone
>
> want to
>
> build a commercial solution on top of it, that
> would be a
>
> possibility.
>
> My idea is to giveback somehow to ceph project
> and at same time
>
> forge a
>
> better knowledge in ceph technologies. Because
> like many in libre
>
> world I
>
> think the business is in the services around the
> software more than
>
> on the
>
> software. That the ones writing code should be
> financed and benefits
>
> from
>
> the one selling and giving support of the
> software at all levels. I
>
> m
>
> probably too idealistic. And too optimistic
> after all I m the one
>
> saying I
>
> will do this stuff I have no idea how but well
> it is interesting and
>
> fun so
>
> lets do it.
>
> Regards,
>
> P.S: using commercial backend libraries appart
> including their own
>
> cost will
>
> force you to use commercial IDE like MS
> VisualStudio because their
>
> library
>
> has some kind of drm that only that IDE compiler
> can use. So alot of
>
> cost
>
> and yet there is nothing done. If I had to open
> a kickstarter
>
> project saying
>
> we need 60 000 USD to do ceph4win with that
> monney we will buy the
>
> right to
>
> use and share a commercial copyrighted library
> but abandonned
>
> punctually to
>
> us in public domaine and that we will
> eventually produce something
>
> out of
>
> it. I doubt I will get a dollar.
>
> We still can suggest the idea to Edlos the
> commercial company that
>
> has the
>
> copyright of Callback FS, Or to buy them their
> product in a blender
>
> way
>
> (blender was bought with donation before being
> put opensource and
>
> public
>
> domaine), Or to open source their library. But
> in commercial minds
> opensourcing = death of their technical
> advantage and death of
>
> their
>
> marketing strategy. They will have to invent
> something more to
>
> retrieve
>
> monney from it.
>
> El nov 6, 2013 11:22 p.m., "Ketor D"
> <d.ketor@gmail.com <mailto:d.ketor@gmail.com>
> <mailto:d.ketor@gmail.com
> <mailto:d.ketor@gmail.com>>> escribió:
>
>
> Hi Alphe,
> I think you could try Callback
> Filesystem dev
>
> framework. It
>
> is a commerical dev framework and is
> maintained by Edlos today.
> I have communicated with Edlos to
> get a try code for
> development. To dokan, Callback Filesystem
> has vary document and
>
> maybe
>
> more stabilize.
>
> Regards.
>
>
>
> On Thu, Nov 7, 2013 at 10:00 AM, Alphe
> Salas <asalas@kepler.cl <mailto:asalas@kepler.cl>
> <mailto:asalas@kepler.cl
>
> <mailto:asalas@kepler.cl>>> wrote:
> > Hello ketor thank you for your interest
> un ceph4win. Since
>
> muy
>
> first mail I
> > exposed the lacks of dokanfs and that I
> m far from being a
> specialist un
> > filesystems.
> > I exposed what i like un dokanfs bit I
> not a fanátic of it.
>
> Muy
>
> goal is to
> > have something working quickly.
> >
> > So I am up to any proposición sure the
> one with the more docs
>
> and
>
> support
> > will be the best choice. As for right
> now what I need is
> understand what are
> > the files involved what are the
> interfaces functions and what
>
> are
>
> the needed
> > library dependencies and if they exist
> ported to windows with
> cygwin. And
> > all that is retrieved from source code.
> >
> > Regards.
> >
> > El nov 6, 2013 10:34 p.m., "Ketor D"
> <d.ketor@gmail.com <mailto:d.ketor@gmail.com>
> <mailto:d.ketor@gmail.com
>
> <mailto:d.ketor@gmail.com>>> escribió:
>
> >
> >> Hi Alphe,
> >> We are taking an interest in your
> work on Ceph Client
>
> for
>
> Windows
> >> with Dokan.As we know, the performance
> of Dokan is not very
> good, and it's
> >> abandoned 3 years ago.
> >> I have learned and used
> OpenDedup(SDFS) for a long
>
> time.
>
> OpenDedup
> >> has a Dokan version. And the author of
> OpenDedup said
> >>
> >> The Dokan library is quite flakey and
> testing should be
> performed before
> >> putting into production
> >>
> >> So what do you think about this?
> And if there is
>
> another
>
> solution of
> >> fuse-like filesystem dev framwork on
> Windows?
> >>
> >> Best Wish!
> >>
> >>
> >>
> >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe
> Salas Michels
> <asalas@kepler.cl <mailto:asalas@kepler.cl>
> <mailto:asalas@kepler.cl <mailto:asalas@kepler.cl>>>
>
> >> wrote:
> >>>
> >>> Hello I created the github repository
> for this project
> >>>https://github.com/alphe/Ceph4Win
> >>>
> >>> Regards,
> >>>
> >>> signature
> >>>
> >>> *Alphé Salas*
> >>> Ingeniero T.I
> >>>
> >>>asalas@kepler.cl
> <mailto:asalas@kepler.cl>
> <mailto:asalas@kepler.cl <mailto:asalas@kepler.cl>>
>
>
> >>> *<http://www.kepler.cl>*
> >>>
> >>> On 11/05/13 21:00, Sage Weil wrote:
> >>>>
> >>>> Hi Alphe,
> >>>>
> >>>> On Tue, 5 Nov 2013, Alphe Salas
> Michels wrote:
> >>>>>
> >>>>> signature *Hi, Sage !
> >>>>> thank you for you enthousiast reply.
> >>>>> I sure want to make the best use of
> everything or
>
> anything
>
> previously
> >>>>> done to
> >>>>> tend to
> >>>>> write ceph cliente for windows.
> >>>>>
> >>>>> Apart using libre tools for building
> the future ceph
>
> cliente
>
> I am open
> >>>>> to
> >>>>> anything.
> >>>>> I would recommand eclipse CDT or
> Code::BLocks they are
>
> based
>
> on mingwin
> >>>>> open
> >>>>> and easyly enhanceable.**
> >>>>>
> >>>>> more free tools can be found here:
>
>>>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
> >>>>>
> >>>>>
> >>>>> I will read libcephfs source code
> and take some notes
>
> about the
>
> >>>>> protocol.
> >>>>
> >>>> I think you don't need to worry about
> hte protocol at all,
>
> since
>
> >>>> libcephs
> >>>> implements it for you (and will
> capture any future
>
> changes).
>
> >>>>
> >>>>> I was more going from what I know
> and trying to track down
>
> how
>
> >>>>> mount.ceph work
> >>>>> with the parameters passed to it.
> >>>>> since it point finally to
> Kernel/fs/ceph and that I don t
>
> really
>
> >>>>> understand
> >>>>> how that module work and that it
> probably points to some
>
> other
>
> >>>>> dependencies
> >>>>> Reading libcephfs source code could
> be a big gain of
>
> time.
>
> >>>>
> >>>> (I would also ignore mount.ceph as
> everything it does it
> specific to
> >>>> how Linux mounts work.)
> >>>>
> >>>>> basically on the protocol what is
> need are:
> >>>>>
> >>>>> 1) open and maintain a connection
> (socket open, auth, etc
>
> )
>
> >>>>> 2) retreive a map of directories and
> disk Quota (disk
>
> sizing
>
> Y TB free,
> >>>>> Z TB
> >>>>> total)
> >>>>> 3) procedure to send files /
> directories in a maner that
>
> it
>
> will allow
> >>>>> our
> >>>>> client to fit ceph transmission
> protocols
> >>>>> (limit bandwith for stability?,
> limit connection amount?,
> limit cpu
> >>>>> use?,
> >>>>> Cache for preparing data transfer (a
> FIFO cache)?)
> >>>>> 4)Procedure to retreive files /
> directory from ceph
>
> cluster
>
> >>>>> 5) Management copy/move files
> /Directories, FS stats,
> Connection Stats.
> >>>>> logging.
> >>>>>
> >>>>> My idea to progress is to take those
> main bulletpoint in
>
> ceph
>
> protocol
> >>>>> based
> >>>>> on general ideas of what ceph file
> system does and start
> identifying
> >>>>> parts
> >>>>> from libcephfs to match those "needs".
> >>>>
> >>>> Instead, I would look at
> include/cephfs/libcephfs.h, the
> interface that
> >>>> libcephfs provides, and try to map
> that to what the fuse
>
> layer
>
> expects.
> >>>> There is both a path-based that I
> suspsect lends itself
>
> well
>
> to the
> >>>> Windows interface and (very soon now)
> a handle based API
>
> that is
>
> >>>> targetted
> >>>> at the Unix-style VFS layers. I'm
> mostly guessing,
>
> though,
>
> since I've
> >>>> never seen any low-level fs code in
> windows before.
> >>>>
> >>>> In this case, the analogous code for
> Linux should be
> client/fuse_ll.cc
> >>>> itself (and not much else), although
> there will probably be
>
> a
>
> few tricks
> >>>> necessary to map cleanly onto how the
> windows interfaces
>
> work.
>
> >>>>
> >>>> Does that make sense?
> >>>>
> >>>> Cheers!
> >>>> sage
> >>>>
> >>>>
> >>>>> Any suggestion and contributions are
> welcome.
> >>>>>
> >>>>>
> >>>>> *
> >>>>> On 11/05/13 11:23, Sage Weil wrote:
> >>>>>>
> >>>>>> Hi Alphe,
> >>>>>>
> >>>>>> On Mon, 4 Nov 2013, Alphe Salas
> Michels wrote:
> >>>>>>>
> >>>>>>> Good day developers!
> >>>>>>>
> >>>>>>> I would like to propose to the one
> interested work with
>
> me to
>
> >>>>>>> develop a
> >>>>>>> ceph
> >>>>>>> cliente for MS windows world,
> Basing us on dokanFS.
> >>>>>>>
> >>>>>>> My company is a ceph enthousiast
> that use on a dayly
>
> basis
>
> ceph and
> >>>>>>> that
> >>>>>>> need
> >>>>>>> both transfer speed and big
> expendable and cheap
>
> storage.
>
> >>>>>>> My company is specialised in data
> recovery and we want
>
> to
>
> participate
> >>>>>>> to
> >>>>>>> ceph
> >>>>>>> effort by bringing a ceph cliente
> for windows.
> >>>>>>
> >>>>>> Awesome!
> >>>>>>
> >>>>>>> Our experience shows us that the
> best gateway is each
> clientes being
> >>>>>>> its
> >>>>>>> own
> >>>>>>> gateway, instead of having a
> bottle neck server or a
>
> cluster of
>
> >>>>>>> bottle
> >>>>>>> neck
> >>>>>>> servers as gateway (FTP, samba,
> SFTP,webdav, s3,
>
> etc..).
>
> >>>>>>>
> >>>>>>> We already did some research in
> that domain.
> >>>>>>>
> >>>>>>> Dokan FS is an intent to write an
> opensource fuse like
> cliente for
> >>>>>>> MS
> >>>>>>> windows.
> >>>>>>>
> >>>>>>> More information on DOKANFS can be
> triggered here
> >>>>>>>http://dokan-dev.net/en/download/
> >>>>>>>
> >>>>>>> Positive points of using DOKANFS.
> >>>>>>>
> >>>>>>> - its opensourced and well
> licenced mit licence, gpl
> licence and lgpl
> >>>>>>> licence.
> >>>>>>>
> >>>>>>> Negative point of using DOKAN FS.
> >>>>>>> - unreachable author
> >>>>>>> - Poor documentation . Dev
> comments in japanese.
> >>>>>>> - Work in progress so it is
> unstable and needs to be
>
> updated,
>
> >>>>>>> debugged and
> >>>>>>> maintained by a MS Windows file
> system expert
>
> developper.
>
> >>>>>>
> >>>>>> I am not very familiar with windows
> storage APIs, but
> somebody told me
> >>>>>> at once point there were several
> interfaces against which
>
> a
>
> new file
> >>>>>> system could be implemented,
> everything from a full
> in-kernel driver
> >>>>>> to
> >>>>>> something that is explorer-based.
> Are any of those
> suitable? Using a
> >>>>>> potentially abandoned fuse-like
> layer makes me a bit
>
> nervous.
>
> >>>>>>
> >>>>>> That said,
> >>>>>>
> >>>>>>>
> >>>>>>> I try past year to do a merge from
> ceph-fuse to dokanfs
> >>>>>>> here are what I learnt.
> >>>>>>> - Ceph-fuse and related source
> code is around 60 000
>
> lines
>
> of code.
> >>>>>>> - Ceph protocol isn t documented
> so it is like trying
>
> to
>
> draw a map
> >>>>>>> of
> >>>>>>> america
> >>>>>>> using only a sextan and a compass.
> >>>>>>>
> >>>>>>> Those led me to those conclusions:
> >>>>>>> - I can t do it alone.
> >>>>>>> - It is easier to draw down the
> ceph protocol way to
>
> work from
>
> >>>>>>> kernel/fs/ceph
> >>>>>>> sources and mount.ceph
> >>>>>>> - Ceph depending libraries may be
> unexistant or not up
>
> to
>
> date in
> >>>>>>> their
> >>>>>>> port
> >>>>>>> on MS Windows (cygwin)
> >>>>>>
> >>>>>> I think the most sane path should
> be to make libcephfs
> sufficiently
> >>>>>> portable to build on windows (or
> cygwin). For the bits
>
> used
>
> by the
> >>>>>> client-side coe, I don't think
> there should be much in
>
> the
>
> way of
> >>>>>> dependencies, and the main
> challenge would be untangling
>
> the
>
> build for
> >>>>>> the necessary pieces out from the
> rest of Ceph.
> >>>>>>
> >>>>>> Have you seen the wip-port
> portability work that is
> currently underway
> >>>>>> by
> >>>>>> Noah and Alan? That may solve many
> of the cygwin
>
> problems
>
> you are
> >>>>>> seeing
> >>>>>> today.
> >>>>>>
> >>>>>>> - MS file system specialist are
> hard do find in the
>
> "open
>
> source
> >>>>>>> libre
> >>>>>>> world"
> >>>>>>> so I will try in the commercial world.
> >>>>>>>
> >>>>>>> The commercial world has some
> problems too. They need
>
> ceph
>
> protocol
> >>>>>>> draft
> >>>>>>> to
> >>>>>>> implemente it to their own product
> They will have
>
> licencing
>
> >>>>>>> /commercial
> >>>>>>> politics that infringe lgpl, and
> hide that most of the
>
> work
>
> is done
> >>>>>>> by
> >>>>>>> people
> >>>>>>> other than them. They will not
> participate in a
>
> financial
>
> way to ceph
> >>>>>>> enhancement and growth.
> >>>>>>
> >>>>>> I don't think reimplementing the
> client code is an
>
> efficient way
>
> >>>>>> forward.
> >>>>>> Unless the goal is a pure kernel
> implementation...but a
> significant
> >>>>>> ongoing investment in development
> resources would be
>
> needed
>
> for that
> >>>>>> going
> >>>>>> forward. I suspect that is a
> challenge for a platform
>
> that
>
> does not
> >>>>>> typically rally that sort of
> community effort.
> >>>>>>
> >>>>>> The easiest thing is of course just
> to use CIFS and
>
> Samba
>
> (which works
> >>>>>> today). A fuse-like approach is
> probably a reasonably
> middle ground
> >>>>>> (both
> >>>>>> in initial effort and
> maintainability going forward)...
> >>>>>>
> >>>>>> sage
> >>>>>>
> >>>>>>
> >>>>>
> >>>
> >>> --
> >>> To unsubscribe from this list: send
> the line "unsubscribe
> ceph-devel" in
> >>> the body of a message
> tomajordomo@vger.kernel.org
> <mailto:tomajordomo@vger.kernel.org>
> <mailto:majordomo@vger.kernel.org
> <mailto:majordomo@vger.kernel.org>>
>
> >>> More majordomo info at
>
> http://vger.kernel.org/majordomo-info.html
>
> >>
> >>
> >
>
>
>
> --
> To unsubscribe from this list: send the line
> "unsubscribe ceph-devel"
> in
> the body of a message tomajordomo@vger.kernel.org
> <mailto:tomajordomo@vger.kernel.org>
> More majordomo info
> athttp://vger.kernel.org/majordomo-info.html
> <http://vger.kernel.org/majordomo-info.html>
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe
> ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> <mailto:majordomo@vger.kernel.org>
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2014-12-26 17:10 ` Ketor D
@ 2014-12-27 5:14 ` Dong Yuan
2014-12-27 5:14 ` Dong Yuan
0 siblings, 1 reply; 21+ messages in thread
From: Dong Yuan @ 2014-12-27 5:14 UTC (permalink / raw)
To: Ketor D; +Cc: Alphe Salas Michels, ceph-devel, Matt W. Benjamin
Why don't you push your impl? Maybe I can have a BP first.
On 27 December 2014 at 01:10, Ketor D <d.ketor@gmail.com> wrote:
> Is here any progress on Cephfs Windows Client ?
>
> 2013-11-08 22:15 GMT+08:00 Alphe Salas Michels <asalas@kepler.cl>:
>> Hello malcom and matt thank you for apporting some more information source.
>> OpenAFS is sure interesting httpfs too.
>>
>> I hope it will help us on deciding the best path to follow in our interface
>> with window.
>> Actually I still trying to isolate the needed client code in the shortest
>> way possible.
>>
>> Regards.
>>
>> Alphe Salas
>>
>> El nov 7, 2013 9:11 p.m., "Malcolm Haak" <malcolm@sgi.com
>> <mailto:malcolm@sgi.com>> escribió:
>>
>> I'm just going to throw these in there.
>>
>> http://www.acc.umu.se/~bosse/ <http://www.acc.umu.se/%7Ebosse/>
>>
>>
>> They are GPLv2 some already use sockets and such from inside the
>> kernel. Heck you might even be able to mod the HTTP one to use
>> rados gateway. I don't know as I havent sat down and pulled them
>> apart enough yet.
>>
>> They might help, but they might be useless. Not sure.
>>
>> On 08/11/13 06:47, Alphe Salas Michels wrote:
>>
>> Hello all I finally finished my first source code extraction
>> that starts
>> from ceph/src/client/fuse_ll.c
>> The result is accurate unlike previous provided results.
>> basically the
>> script start from a file extract all the private includes
>> definitions
>> #include "something.h" and recursively extract private includes
>> too. the
>> best way to know who is related with who.
>>
>> starting from fuse_ll.cc I optain 390 files retreived and 120
>> 000 lines
>> of code !
>> involved dirs are : in ceph/src
>> objclass/, common/, msg/, common/, osdc/, include/, client/, mds/,
>> global/, json_spirit/, log/, os/, crush/, mon/, osd/, auth/
>>
>> probably not a good way to analyse what amount of work it means
>> since
>> most of those directories are the implementation of servers
>> (osd, mon,
>> mds) and even if only a tiny bit of them is needed at client
>> level. you
>> need two structures from ./osd/OSD.h and my script by relation will
>> take into acount the whole directory...
>>
>> I ran the script with libcephfs.cc as start point and got almost the
>> same results. 131 000 lines of code and 386 files most of the
>> same dirs
>> involved.
>>
>>
>>
>> I think I will spend alot of time doing the manual source code
>> isolation
>> and understand way each #include is set in the files I read (what
>> purpose they have do they allow to integrate a crucial data type
>> or not.
>>
>>
>> The other way around will be to read src/libcephfs.cc. It seems
>> shorter
>> but without understanding what part is used for each included
>> header I
>> can t say anything...
>>
>>
>>
>> I will keep reading the source code and take notes. I think in
>> the case
>> of libcephfs I will gain alot of time.
>>
>> signature
>>
>> *Alphé Salas*
>> Ingeniero T.I
>>
>> asalas@kepler.cl <mailto:asalas@kepler.cl>
>> *www.kepler.cl <http://www.kepler.cl> <http://www.kepler.cl>*
>>
>>
>> On 11/07/13 15:02, Alphe Salas Michels wrote:
>>
>> Hello D.Ketor and Matt Benjamin,
>> You give me alot to think about and this is great!
>> I merged your previous post to make a single reply that
>> anyone can
>> report to easyly
>>
>> Windows NFS 4.1 is available here:
>> http://www.citi.umich.edu/projects/nfsv4/windows/readme.html
>>
>> pnfs is another name for NFS4.X. It is presented as
>> alternative to
>> ceph and we get known terminology as MDS and OSD but without
>> the self
>> healing part if I understand well my rapid look on the
>> topic. (when I
>> say rapid look I mean ... 5 minutes spent in that... which
>> is really
>> small amount of time to get an accurate view on something)
>>
>>
>> starting from mount.ceph ... I know that mount.ceph does
>> little but it
>> is a great hint to know what ceph needs and do things.
>> Basically mount.ceph modprobe the ceph driver in the linux
>> kernel then
>> call mount with the line command passed args and the cephfs
>> type as
>> argument. Then the kernel does the work I don t understand
>> yet what is
>> the start calls that are made to the ceph driver but it
>> seemed to me
>> that is was relatively light. (a first impression compared
>> to ceph-fuse.)
>>
>> I think I will do both isolate source code from ceph-client
>> kernel
>> (cephfs module for linux kernel) and the one pointed by Sage
>> starting
>> from client/fuse_ll.cc in ceph master branch. The common
>> files betwin
>> those 2 extractions will be our core set of mandatory features.
>>
>> Then we try to compile with cygwin a cephfs client library .
>> Then we
>> will try to interface with a modified windows nfs 4.1 client
>> or pnfs
>> or any other that will accept to be compiled with gcc for
>> win32...
>>
>> the fact that windows 8.1 is and windows 2012 are out of
>> reach at the
>> moment is not a problem to me.
>>
>> Our first concern is to understand what is ceph protocol.
>> Then adapt
>> it to something that can be used on windows prior windows
>> 8.1. Dokan
>> fs if I remember well use too the WDK (windows driver
>> dev-kit ) for it
>> s compilation so possibly we will see the same limitations.
>>
>> We need to multiply our source of information by example
>> regarding
>> ceph-client (kernel or fuse, radosgw is on a different layer
>> so I will
>> not try anything around it at first.) And we need to
>> multiply our
>> source of information by example regarding virtual file system
>> technologies on windoes OS.
>> Alot of work but all of those available source code everyone
>> point at
>> me will make our best solution. And in the end we will choose
>> technologies knowing what we do and what concequencies they
>> have.
>>
>> regards,
>>
>>
>>
>>
>> Regards
>>
>> signature
>>
>> *Alphé Salas*
>> Ingeniero T.I
>>
>> asalas@kepler.cl <mailto:asalas@kepler.cl>
>>
>>
>>
>> On 11/07/13 11:29, Ketor D wrote:
>>
>> Hi Alphe:
>> Yes Callback Filesystem is very expensive and
>> can't open source.
>> It's not a good choice for ceph4win.
>> Another way for ceph4win maybe develop a
>> kernel-mode fs like
>> pnfs. pnfs has a kernel-mode windows client. I think you
>> can read its
>> src code and maybe migrating from ceph kernel client to
>> windows kernel
>> fs is easier than from userspace ceph fuse client.And a
>> kernel-mode fs
>> client has greater performance than userspace fs like
>> ceph-fuse client
>> and ceph kernel client.
>>
>> Regards.
>>
>> On 11/07/13 11:50, Matt W. Benjamin wrote:
>>
>> Hi,
>>
>> The Window NFS v4.1 client is what we work on, so this
>> may be good for
>> code sharing. The license is lgplv2, like Ceph's.
>>
>> Something important to be aware of is that the client
>> uses rdbss, which
>> is a (partial) fsd abstraction that simplified
>> implementation
>> quite a bit, kind of like a mini driver. However,
>> Microsoft's support
>> for rdbss has been in limbo for a bit. For example, to
>> link with
>> the rdbss symbols you can't use the Windows 8 driver
>> kit--you'll need
>> to use the one for Windows 7. (There's a private rdbss2
>> used internally
>> by Microsoft's SMB implemenation. A the moment, 3rd
>> party drivers
>> can't use that.)
>>
>> We've been in communication with Microsoft about this
>> issue, and know of
>> a few other fsds using it, but it could be a good thing
>> for that
>> lobbying
>> effort to have another user--or it could be a dead end :(.
>>
>> There are a couple of other choices if you're looking to
>> go this route,
>> that I'm aware of (and we may need to take them too, if
>> RDBSS has no
>> way forward), but the required work could be a lot larger.
>>
>> Matt
>>
>> ----- "Ketor D"<d.ketor@gmail.com
>> <mailto:d.ketor@gmail.com>> wrote:
>>
>> Hi Alphe:
>> Yes Callback Filesystem is very expensive
>> and can't open
>> source.
>> It's not a good choice for ceph4win.
>> Another way for ceph4win maybe develop a
>> kernel-mode fs like
>> pnfs. pnfs has a kernel-mode windows client. I think
>> you can read its
>> src code and maybe migrating from ceph kernel client
>> to windows
>> kernel
>> fs is easier than from userspace ceph fuse
>> client.And a kernel-mode
>> fs
>> client has greater performance than userspace fs
>> like ceph-fuse
>> client
>> and ceph kernel client.
>>
>> Regards.
>>
>>
>>
>> On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas
>> Michels<asalas@kepler.cl <mailto:asalas@kepler.cl>>
>>
>> wrote:
>>
>> Commercial libraries are a pain ...
>>
>> If we want the more permossive licence offered
>> by callback file
>>
>> system we
>>
>> have to buy it for 20.000 usd. Then we will have
>> to provide a
>>
>> backbox that
>>
>> we have no control upon and that will kill our
>> product anytime they
>>
>> want anf
>>
>> if they decide to stop their commercial activity
>> we will be in the
>>
>> same
>>
>> situation that with dokanfs but without having
>> the source code of
>>
>> the black
>>
>> box. If i have to spend 20 000 dollars i would
>> prefere paying
>>
>> someone to
>>
>> retake dokanfs or to write from scratch a
>> dokanfs fuselike software
>>
>> make it
>>
>> all shiny and pumpy fantastic and ready to plug
>> to ceph client.
>>
>> I would prefere if people have to pay something
>> to get access to
>>
>> ceph4win
>>
>> that this money goes in ceph main branch
>> pockets... Or as a gift you
>>
>> donante
>>
>> to ceph 10 dollars you get 2 free registration
>> codes for
>>
>> ceph4win... or
>>
>> something like that.
>>
>> If ceph4win as to be comercial then I would
>> prefer delegate the task
>>
>> to a
>>
>> company like south river technologies and their
>> great product
>>
>> webdrive. I
>>
>> would mininaly get involved in that project and
>> simply buy the final
>>
>> product
>>
>> to sell it together with my ceph based product
>> (which could be a
>>
>> calxeda
>>
>> ceph box or something like that).
>>
>> I m open anyway to any proposition. But I doubt
>> that callback
>>
>> filesystem
>>
>> offers us a suitable solution in the way I see
>> ceph4win to be spread
>>
>> and
>>
>> used... I m maybe wrong. And anything that will
>> be done around
>>
>> ceph4win will
>>
>> be public documented etc... And licensed the way
>> that if someone
>>
>> want to
>>
>> build a commercial solution on top of it, that
>> would be a
>>
>> possibility.
>>
>> My idea is to giveback somehow to ceph project
>> and at same time
>>
>> forge a
>>
>> better knowledge in ceph technologies. Because
>> like many in libre
>>
>> world I
>>
>> think the business is in the services around the
>> software more than
>>
>> on the
>>
>> software. That the ones writing code should be
>> financed and benefits
>>
>> from
>>
>> the one selling and giving support of the
>> software at all levels. I
>>
>> m
>>
>> probably too idealistic. And too optimistic
>> after all I m the one
>>
>> saying I
>>
>> will do this stuff I have no idea how but well
>> it is interesting and
>>
>> fun so
>>
>> lets do it.
>>
>> Regards,
>>
>> P.S: using commercial backend libraries appart
>> including their own
>>
>> cost will
>>
>> force you to use commercial IDE like MS
>> VisualStudio because their
>>
>> library
>>
>> has some kind of drm that only that IDE compiler
>> can use. So alot of
>>
>> cost
>>
>> and yet there is nothing done. If I had to open
>> a kickstarter
>>
>> project saying
>>
>> we need 60 000 USD to do ceph4win with that
>> monney we will buy the
>>
>> right to
>>
>> use and share a commercial copyrighted library
>> but abandonned
>>
>> punctually to
>>
>> us in public domaine and that we will
>> eventually produce something
>>
>> out of
>>
>> it. I doubt I will get a dollar.
>>
>> We still can suggest the idea to Edlos the
>> commercial company that
>>
>> has the
>>
>> copyright of Callback FS, Or to buy them their
>> product in a blender
>>
>> way
>>
>> (blender was bought with donation before being
>> put opensource and
>>
>> public
>>
>> domaine), Or to open source their library. But
>> in commercial minds
>> opensourcing = death of their technical
>> advantage and death of
>>
>> their
>>
>> marketing strategy. They will have to invent
>> something more to
>>
>> retrieve
>>
>> monney from it.
>>
>> El nov 6, 2013 11:22 p.m., "Ketor D"
>> <d.ketor@gmail.com <mailto:d.ketor@gmail.com>
>> <mailto:d.ketor@gmail.com
>> <mailto:d.ketor@gmail.com>>> escribió:
>>
>>
>> Hi Alphe,
>> I think you could try Callback
>> Filesystem dev
>>
>> framework. It
>>
>> is a commerical dev framework and is
>> maintained by Edlos today.
>> I have communicated with Edlos to
>> get a try code for
>> development. To dokan, Callback Filesystem
>> has vary document and
>>
>> maybe
>>
>> more stabilize.
>>
>> Regards.
>>
>>
>>
>> On Thu, Nov 7, 2013 at 10:00 AM, Alphe
>> Salas <asalas@kepler.cl <mailto:asalas@kepler.cl>
>> <mailto:asalas@kepler.cl
>>
>> <mailto:asalas@kepler.cl>>> wrote:
>> > Hello ketor thank you for your interest
>> un ceph4win. Since
>>
>> muy
>>
>> first mail I
>> > exposed the lacks of dokanfs and that I
>> m far from being a
>> specialist un
>> > filesystems.
>> > I exposed what i like un dokanfs bit I
>> not a fanátic of it.
>>
>> Muy
>>
>> goal is to
>> > have something working quickly.
>> >
>> > So I am up to any proposición sure the
>> one with the more docs
>>
>> and
>>
>> support
>> > will be the best choice. As for right
>> now what I need is
>> understand what are
>> > the files involved what are the
>> interfaces functions and what
>>
>> are
>>
>> the needed
>> > library dependencies and if they exist
>> ported to windows with
>> cygwin. And
>> > all that is retrieved from source code.
>> >
>> > Regards.
>> >
>> > El nov 6, 2013 10:34 p.m., "Ketor D"
>> <d.ketor@gmail.com <mailto:d.ketor@gmail.com>
>> <mailto:d.ketor@gmail.com
>>
>> <mailto:d.ketor@gmail.com>>> escribió:
>>
>> >
>> >> Hi Alphe,
>> >> We are taking an interest in your
>> work on Ceph Client
>>
>> for
>>
>> Windows
>> >> with Dokan.As we know, the performance
>> of Dokan is not very
>> good, and it's
>> >> abandoned 3 years ago.
>> >> I have learned and used
>> OpenDedup(SDFS) for a long
>>
>> time.
>>
>> OpenDedup
>> >> has a Dokan version. And the author of
>> OpenDedup said
>> >>
>> >> The Dokan library is quite flakey and
>> testing should be
>> performed before
>> >> putting into production
>> >>
>> >> So what do you think about this?
>> And if there is
>>
>> another
>>
>> solution of
>> >> fuse-like filesystem dev framwork on
>> Windows?
>> >>
>> >> Best Wish!
>> >>
>> >>
>> >>
>> >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe
>> Salas Michels
>> <asalas@kepler.cl <mailto:asalas@kepler.cl>
>> <mailto:asalas@kepler.cl <mailto:asalas@kepler.cl>>>
>>
>> >> wrote:
>> >>>
>> >>> Hello I created the github repository
>> for this project
>> >>>https://github.com/alphe/Ceph4Win
>> >>>
>> >>> Regards,
>> >>>
>> >>> signature
>> >>>
>> >>> *Alphé Salas*
>> >>> Ingeniero T.I
>> >>>
>> >>>asalas@kepler.cl
>> <mailto:asalas@kepler.cl>
>> <mailto:asalas@kepler.cl <mailto:asalas@kepler.cl>>
>>
>>
>> >>> *<http://www.kepler.cl>*
>> >>>
>> >>> On 11/05/13 21:00, Sage Weil wrote:
>> >>>>
>> >>>> Hi Alphe,
>> >>>>
>> >>>> On Tue, 5 Nov 2013, Alphe Salas
>> Michels wrote:
>> >>>>>
>> >>>>> signature *Hi, Sage !
>> >>>>> thank you for you enthousiast reply.
>> >>>>> I sure want to make the best use of
>> everything or
>>
>> anything
>>
>> previously
>> >>>>> done to
>> >>>>> tend to
>> >>>>> write ceph cliente for windows.
>> >>>>>
>> >>>>> Apart using libre tools for building
>> the future ceph
>>
>> cliente
>>
>> I am open
>> >>>>> to
>> >>>>> anything.
>> >>>>> I would recommand eclipse CDT or
>> Code::BLocks they are
>>
>> based
>>
>> on mingwin
>> >>>>> open
>> >>>>> and easyly enhanceable.**
>> >>>>>
>> >>>>> more free tools can be found here:
>>
>>>>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
>> >>>>>
>> >>>>>
>> >>>>> I will read libcephfs source code
>> and take some notes
>>
>> about the
>>
>> >>>>> protocol.
>> >>>>
>> >>>> I think you don't need to worry about
>> hte protocol at all,
>>
>> since
>>
>> >>>> libcephs
>> >>>> implements it for you (and will
>> capture any future
>>
>> changes).
>>
>> >>>>
>> >>>>> I was more going from what I know
>> and trying to track down
>>
>> how
>>
>> >>>>> mount.ceph work
>> >>>>> with the parameters passed to it.
>> >>>>> since it point finally to
>> Kernel/fs/ceph and that I don t
>>
>> really
>>
>> >>>>> understand
>> >>>>> how that module work and that it
>> probably points to some
>>
>> other
>>
>> >>>>> dependencies
>> >>>>> Reading libcephfs source code could
>> be a big gain of
>>
>> time.
>>
>> >>>>
>> >>>> (I would also ignore mount.ceph as
>> everything it does it
>> specific to
>> >>>> how Linux mounts work.)
>> >>>>
>> >>>>> basically on the protocol what is
>> need are:
>> >>>>>
>> >>>>> 1) open and maintain a connection
>> (socket open, auth, etc
>>
>> )
>>
>> >>>>> 2) retreive a map of directories and
>> disk Quota (disk
>>
>> sizing
>>
>> Y TB free,
>> >>>>> Z TB
>> >>>>> total)
>> >>>>> 3) procedure to send files /
>> directories in a maner that
>>
>> it
>>
>> will allow
>> >>>>> our
>> >>>>> client to fit ceph transmission
>> protocols
>> >>>>> (limit bandwith for stability?,
>> limit connection amount?,
>> limit cpu
>> >>>>> use?,
>> >>>>> Cache for preparing data transfer (a
>> FIFO cache)?)
>> >>>>> 4)Procedure to retreive files /
>> directory from ceph
>>
>> cluster
>>
>> >>>>> 5) Management copy/move files
>> /Directories, FS stats,
>> Connection Stats.
>> >>>>> logging.
>> >>>>>
>> >>>>> My idea to progress is to take those
>> main bulletpoint in
>>
>> ceph
>>
>> protocol
>> >>>>> based
>> >>>>> on general ideas of what ceph file
>> system does and start
>> identifying
>> >>>>> parts
>> >>>>> from libcephfs to match those "needs".
>> >>>>
>> >>>> Instead, I would look at
>> include/cephfs/libcephfs.h, the
>> interface that
>> >>>> libcephfs provides, and try to map
>> that to what the fuse
>>
>> layer
>>
>> expects.
>> >>>> There is both a path-based that I
>> suspsect lends itself
>>
>> well
>>
>> to the
>> >>>> Windows interface and (very soon now)
>> a handle based API
>>
>> that is
>>
>> >>>> targetted
>> >>>> at the Unix-style VFS layers. I'm
>> mostly guessing,
>>
>> though,
>>
>> since I've
>> >>>> never seen any low-level fs code in
>> windows before.
>> >>>>
>> >>>> In this case, the analogous code for
>> Linux should be
>> client/fuse_ll.cc
>> >>>> itself (and not much else), although
>> there will probably be
>>
>> a
>>
>> few tricks
>> >>>> necessary to map cleanly onto how the
>> windows interfaces
>>
>> work.
>>
>> >>>>
>> >>>> Does that make sense?
>> >>>>
>> >>>> Cheers!
>> >>>> sage
>> >>>>
>> >>>>
>> >>>>> Any suggestion and contributions are
>> welcome.
>> >>>>>
>> >>>>>
>> >>>>> *
>> >>>>> On 11/05/13 11:23, Sage Weil wrote:
>> >>>>>>
>> >>>>>> Hi Alphe,
>> >>>>>>
>> >>>>>> On Mon, 4 Nov 2013, Alphe Salas
>> Michels wrote:
>> >>>>>>>
>> >>>>>>> Good day developers!
>> >>>>>>>
>> >>>>>>> I would like to propose to the one
>> interested work with
>>
>> me to
>>
>> >>>>>>> develop a
>> >>>>>>> ceph
>> >>>>>>> cliente for MS windows world,
>> Basing us on dokanFS.
>> >>>>>>>
>> >>>>>>> My company is a ceph enthousiast
>> that use on a dayly
>>
>> basis
>>
>> ceph and
>> >>>>>>> that
>> >>>>>>> need
>> >>>>>>> both transfer speed and big
>> expendable and cheap
>>
>> storage.
>>
>> >>>>>>> My company is specialised in data
>> recovery and we want
>>
>> to
>>
>> participate
>> >>>>>>> to
>> >>>>>>> ceph
>> >>>>>>> effort by bringing a ceph cliente
>> for windows.
>> >>>>>>
>> >>>>>> Awesome!
>> >>>>>>
>> >>>>>>> Our experience shows us that the
>> best gateway is each
>> clientes being
>> >>>>>>> its
>> >>>>>>> own
>> >>>>>>> gateway, instead of having a
>> bottle neck server or a
>>
>> cluster of
>>
>> >>>>>>> bottle
>> >>>>>>> neck
>> >>>>>>> servers as gateway (FTP, samba,
>> SFTP,webdav, s3,
>>
>> etc..).
>>
>> >>>>>>>
>> >>>>>>> We already did some research in
>> that domain.
>> >>>>>>>
>> >>>>>>> Dokan FS is an intent to write an
>> opensource fuse like
>> cliente for
>> >>>>>>> MS
>> >>>>>>> windows.
>> >>>>>>>
>> >>>>>>> More information on DOKANFS can be
>> triggered here
>> >>>>>>>http://dokan-dev.net/en/download/
>> >>>>>>>
>> >>>>>>> Positive points of using DOKANFS.
>> >>>>>>>
>> >>>>>>> - its opensourced and well
>> licenced mit licence, gpl
>> licence and lgpl
>> >>>>>>> licence.
>> >>>>>>>
>> >>>>>>> Negative point of using DOKAN FS.
>> >>>>>>> - unreachable author
>> >>>>>>> - Poor documentation . Dev
>> comments in japanese.
>> >>>>>>> - Work in progress so it is
>> unstable and needs to be
>>
>> updated,
>>
>> >>>>>>> debugged and
>> >>>>>>> maintained by a MS Windows file
>> system expert
>>
>> developper.
>>
>> >>>>>>
>> >>>>>> I am not very familiar with windows
>> storage APIs, but
>> somebody told me
>> >>>>>> at once point there were several
>> interfaces against which
>>
>> a
>>
>> new file
>> >>>>>> system could be implemented,
>> everything from a full
>> in-kernel driver
>> >>>>>> to
>> >>>>>> something that is explorer-based.
>> Are any of those
>> suitable? Using a
>> >>>>>> potentially abandoned fuse-like
>> layer makes me a bit
>>
>> nervous.
>>
>> >>>>>>
>> >>>>>> That said,
>> >>>>>>
>> >>>>>>>
>> >>>>>>> I try past year to do a merge from
>> ceph-fuse to dokanfs
>> >>>>>>> here are what I learnt.
>> >>>>>>> - Ceph-fuse and related source
>> code is around 60 000
>>
>> lines
>>
>> of code.
>> >>>>>>> - Ceph protocol isn t documented
>> so it is like trying
>>
>> to
>>
>> draw a map
>> >>>>>>> of
>> >>>>>>> america
>> >>>>>>> using only a sextan and a compass.
>> >>>>>>>
>> >>>>>>> Those led me to those conclusions:
>> >>>>>>> - I can t do it alone.
>> >>>>>>> - It is easier to draw down the
>> ceph protocol way to
>>
>> work from
>>
>> >>>>>>> kernel/fs/ceph
>> >>>>>>> sources and mount.ceph
>> >>>>>>> - Ceph depending libraries may be
>> unexistant or not up
>>
>> to
>>
>> date in
>> >>>>>>> their
>> >>>>>>> port
>> >>>>>>> on MS Windows (cygwin)
>> >>>>>>
>> >>>>>> I think the most sane path should
>> be to make libcephfs
>> sufficiently
>> >>>>>> portable to build on windows (or
>> cygwin). For the bits
>>
>> used
>>
>> by the
>> >>>>>> client-side coe, I don't think
>> there should be much in
>>
>> the
>>
>> way of
>> >>>>>> dependencies, and the main
>> challenge would be untangling
>>
>> the
>>
>> build for
>> >>>>>> the necessary pieces out from the
>> rest of Ceph.
>> >>>>>>
>> >>>>>> Have you seen the wip-port
>> portability work that is
>> currently underway
>> >>>>>> by
>> >>>>>> Noah and Alan? That may solve many
>> of the cygwin
>>
>> problems
>>
>> you are
>> >>>>>> seeing
>> >>>>>> today.
>> >>>>>>
>> >>>>>>> - MS file system specialist are
>> hard do find in the
>>
>> "open
>>
>> source
>> >>>>>>> libre
>> >>>>>>> world"
>> >>>>>>> so I will try in the commercial world.
>> >>>>>>>
>> >>>>>>> The commercial world has some
>> problems too. They need
>>
>> ceph
>>
>> protocol
>> >>>>>>> draft
>> >>>>>>> to
>> >>>>>>> implemente it to their own product
>> They will have
>>
>> licencing
>>
>> >>>>>>> /commercial
>> >>>>>>> politics that infringe lgpl, and
>> hide that most of the
>>
>> work
>>
>> is done
>> >>>>>>> by
>> >>>>>>> people
>> >>>>>>> other than them. They will not
>> participate in a
>>
>> financial
>>
>> way to ceph
>> >>>>>>> enhancement and growth.
>> >>>>>>
>> >>>>>> I don't think reimplementing the
>> client code is an
>>
>> efficient way
>>
>> >>>>>> forward.
>> >>>>>> Unless the goal is a pure kernel
>> implementation...but a
>> significant
>> >>>>>> ongoing investment in development
>> resources would be
>>
>> needed
>>
>> for that
>> >>>>>> going
>> >>>>>> forward. I suspect that is a
>> challenge for a platform
>>
>> that
>>
>> does not
>> >>>>>> typically rally that sort of
>> community effort.
>> >>>>>>
>> >>>>>> The easiest thing is of course just
>> to use CIFS and
>>
>> Samba
>>
>> (which works
>> >>>>>> today). A fuse-like approach is
>> probably a reasonably
>> middle ground
>> >>>>>> (both
>> >>>>>> in initial effort and
>> maintainability going forward)...
>> >>>>>>
>> >>>>>> sage
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>
>> >>> --
>> >>> To unsubscribe from this list: send
>> the line "unsubscribe
>> ceph-devel" in
>> >>> the body of a message
>> tomajordomo@vger.kernel.org
>> <mailto:tomajordomo@vger.kernel.org>
>> <mailto:majordomo@vger.kernel.org
>> <mailto:majordomo@vger.kernel.org>>
>>
>> >>> More majordomo info at
>>
>> http://vger.kernel.org/majordomo-info.html
>>
>> >>
>> >>
>> >
>>
>>
>>
>> --
>> To unsubscribe from this list: send the line
>> "unsubscribe ceph-devel"
>> in
>> the body of a message tomajordomo@vger.kernel.org
>> <mailto:tomajordomo@vger.kernel.org>
>> More majordomo info
>> athttp://vger.kernel.org/majordomo-info.html
>> <http://vger.kernel.org/majordomo-info.html>
>>
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe
>> ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> <mailto:majordomo@vger.kernel.org>
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Dong Yuan
Email:yuandong1222@gmail.com
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2014-12-27 5:14 ` Dong Yuan
@ 2014-12-27 5:14 ` Dong Yuan
0 siblings, 0 replies; 21+ messages in thread
From: Dong Yuan @ 2014-12-27 5:14 UTC (permalink / raw)
To: Ketor D; +Cc: Alphe Salas Michels, ceph-devel, Matt W. Benjamin
Sorry, I mean YOU can have a BP first.
On 27 December 2014 at 13:14, Dong Yuan <yuandong1222@gmail.com> wrote:
> Why don't you push your impl? Maybe I can have a BP first.
>
> On 27 December 2014 at 01:10, Ketor D <d.ketor@gmail.com> wrote:
>> Is here any progress on Cephfs Windows Client ?
>>
>> 2013-11-08 22:15 GMT+08:00 Alphe Salas Michels <asalas@kepler.cl>:
>>> Hello malcom and matt thank you for apporting some more information source.
>>> OpenAFS is sure interesting httpfs too.
>>>
>>> I hope it will help us on deciding the best path to follow in our interface
>>> with window.
>>> Actually I still trying to isolate the needed client code in the shortest
>>> way possible.
>>>
>>> Regards.
>>>
>>> Alphe Salas
>>>
>>> El nov 7, 2013 9:11 p.m., "Malcolm Haak" <malcolm@sgi.com
>>> <mailto:malcolm@sgi.com>> escribió:
>>>
>>> I'm just going to throw these in there.
>>>
>>> http://www.acc.umu.se/~bosse/ <http://www.acc.umu.se/%7Ebosse/>
>>>
>>>
>>> They are GPLv2 some already use sockets and such from inside the
>>> kernel. Heck you might even be able to mod the HTTP one to use
>>> rados gateway. I don't know as I havent sat down and pulled them
>>> apart enough yet.
>>>
>>> They might help, but they might be useless. Not sure.
>>>
>>> On 08/11/13 06:47, Alphe Salas Michels wrote:
>>>
>>> Hello all I finally finished my first source code extraction
>>> that starts
>>> from ceph/src/client/fuse_ll.c
>>> The result is accurate unlike previous provided results.
>>> basically the
>>> script start from a file extract all the private includes
>>> definitions
>>> #include "something.h" and recursively extract private includes
>>> too. the
>>> best way to know who is related with who.
>>>
>>> starting from fuse_ll.cc I optain 390 files retreived and 120
>>> 000 lines
>>> of code !
>>> involved dirs are : in ceph/src
>>> objclass/, common/, msg/, common/, osdc/, include/, client/, mds/,
>>> global/, json_spirit/, log/, os/, crush/, mon/, osd/, auth/
>>>
>>> probably not a good way to analyse what amount of work it means
>>> since
>>> most of those directories are the implementation of servers
>>> (osd, mon,
>>> mds) and even if only a tiny bit of them is needed at client
>>> level. you
>>> need two structures from ./osd/OSD.h and my script by relation will
>>> take into acount the whole directory...
>>>
>>> I ran the script with libcephfs.cc as start point and got almost the
>>> same results. 131 000 lines of code and 386 files most of the
>>> same dirs
>>> involved.
>>>
>>>
>>>
>>> I think I will spend alot of time doing the manual source code
>>> isolation
>>> and understand way each #include is set in the files I read (what
>>> purpose they have do they allow to integrate a crucial data type
>>> or not.
>>>
>>>
>>> The other way around will be to read src/libcephfs.cc. It seems
>>> shorter
>>> but without understanding what part is used for each included
>>> header I
>>> can t say anything...
>>>
>>>
>>>
>>> I will keep reading the source code and take notes. I think in
>>> the case
>>> of libcephfs I will gain alot of time.
>>>
>>> signature
>>>
>>> *Alphé Salas*
>>> Ingeniero T.I
>>>
>>> asalas@kepler.cl <mailto:asalas@kepler.cl>
>>> *www.kepler.cl <http://www.kepler.cl> <http://www.kepler.cl>*
>>>
>>>
>>> On 11/07/13 15:02, Alphe Salas Michels wrote:
>>>
>>> Hello D.Ketor and Matt Benjamin,
>>> You give me alot to think about and this is great!
>>> I merged your previous post to make a single reply that
>>> anyone can
>>> report to easyly
>>>
>>> Windows NFS 4.1 is available here:
>>> http://www.citi.umich.edu/projects/nfsv4/windows/readme.html
>>>
>>> pnfs is another name for NFS4.X. It is presented as
>>> alternative to
>>> ceph and we get known terminology as MDS and OSD but without
>>> the self
>>> healing part if I understand well my rapid look on the
>>> topic. (when I
>>> say rapid look I mean ... 5 minutes spent in that... which
>>> is really
>>> small amount of time to get an accurate view on something)
>>>
>>>
>>> starting from mount.ceph ... I know that mount.ceph does
>>> little but it
>>> is a great hint to know what ceph needs and do things.
>>> Basically mount.ceph modprobe the ceph driver in the linux
>>> kernel then
>>> call mount with the line command passed args and the cephfs
>>> type as
>>> argument. Then the kernel does the work I don t understand
>>> yet what is
>>> the start calls that are made to the ceph driver but it
>>> seemed to me
>>> that is was relatively light. (a first impression compared
>>> to ceph-fuse.)
>>>
>>> I think I will do both isolate source code from ceph-client
>>> kernel
>>> (cephfs module for linux kernel) and the one pointed by Sage
>>> starting
>>> from client/fuse_ll.cc in ceph master branch. The common
>>> files betwin
>>> those 2 extractions will be our core set of mandatory features.
>>>
>>> Then we try to compile with cygwin a cephfs client library .
>>> Then we
>>> will try to interface with a modified windows nfs 4.1 client
>>> or pnfs
>>> or any other that will accept to be compiled with gcc for
>>> win32...
>>>
>>> the fact that windows 8.1 is and windows 2012 are out of
>>> reach at the
>>> moment is not a problem to me.
>>>
>>> Our first concern is to understand what is ceph protocol.
>>> Then adapt
>>> it to something that can be used on windows prior windows
>>> 8.1. Dokan
>>> fs if I remember well use too the WDK (windows driver
>>> dev-kit ) for it
>>> s compilation so possibly we will see the same limitations.
>>>
>>> We need to multiply our source of information by example
>>> regarding
>>> ceph-client (kernel or fuse, radosgw is on a different layer
>>> so I will
>>> not try anything around it at first.) And we need to
>>> multiply our
>>> source of information by example regarding virtual file system
>>> technologies on windoes OS.
>>> Alot of work but all of those available source code everyone
>>> point at
>>> me will make our best solution. And in the end we will choose
>>> technologies knowing what we do and what concequencies they
>>> have.
>>>
>>> regards,
>>>
>>>
>>>
>>>
>>> Regards
>>>
>>> signature
>>>
>>> *Alphé Salas*
>>> Ingeniero T.I
>>>
>>> asalas@kepler.cl <mailto:asalas@kepler.cl>
>>>
>>>
>>>
>>> On 11/07/13 11:29, Ketor D wrote:
>>>
>>> Hi Alphe:
>>> Yes Callback Filesystem is very expensive and
>>> can't open source.
>>> It's not a good choice for ceph4win.
>>> Another way for ceph4win maybe develop a
>>> kernel-mode fs like
>>> pnfs. pnfs has a kernel-mode windows client. I think you
>>> can read its
>>> src code and maybe migrating from ceph kernel client to
>>> windows kernel
>>> fs is easier than from userspace ceph fuse client.And a
>>> kernel-mode fs
>>> client has greater performance than userspace fs like
>>> ceph-fuse client
>>> and ceph kernel client.
>>>
>>> Regards.
>>>
>>> On 11/07/13 11:50, Matt W. Benjamin wrote:
>>>
>>> Hi,
>>>
>>> The Window NFS v4.1 client is what we work on, so this
>>> may be good for
>>> code sharing. The license is lgplv2, like Ceph's.
>>>
>>> Something important to be aware of is that the client
>>> uses rdbss, which
>>> is a (partial) fsd abstraction that simplified
>>> implementation
>>> quite a bit, kind of like a mini driver. However,
>>> Microsoft's support
>>> for rdbss has been in limbo for a bit. For example, to
>>> link with
>>> the rdbss symbols you can't use the Windows 8 driver
>>> kit--you'll need
>>> to use the one for Windows 7. (There's a private rdbss2
>>> used internally
>>> by Microsoft's SMB implemenation. A the moment, 3rd
>>> party drivers
>>> can't use that.)
>>>
>>> We've been in communication with Microsoft about this
>>> issue, and know of
>>> a few other fsds using it, but it could be a good thing
>>> for that
>>> lobbying
>>> effort to have another user--or it could be a dead end :(.
>>>
>>> There are a couple of other choices if you're looking to
>>> go this route,
>>> that I'm aware of (and we may need to take them too, if
>>> RDBSS has no
>>> way forward), but the required work could be a lot larger.
>>>
>>> Matt
>>>
>>> ----- "Ketor D"<d.ketor@gmail.com
>>> <mailto:d.ketor@gmail.com>> wrote:
>>>
>>> Hi Alphe:
>>> Yes Callback Filesystem is very expensive
>>> and can't open
>>> source.
>>> It's not a good choice for ceph4win.
>>> Another way for ceph4win maybe develop a
>>> kernel-mode fs like
>>> pnfs. pnfs has a kernel-mode windows client. I think
>>> you can read its
>>> src code and maybe migrating from ceph kernel client
>>> to windows
>>> kernel
>>> fs is easier than from userspace ceph fuse
>>> client.And a kernel-mode
>>> fs
>>> client has greater performance than userspace fs
>>> like ceph-fuse
>>> client
>>> and ceph kernel client.
>>>
>>> Regards.
>>>
>>>
>>>
>>> On Thu, Nov 7, 2013 at 8:13 PM, Alphe Salas
>>> Michels<asalas@kepler.cl <mailto:asalas@kepler.cl>>
>>>
>>> wrote:
>>>
>>> Commercial libraries are a pain ...
>>>
>>> If we want the more permossive licence offered
>>> by callback file
>>>
>>> system we
>>>
>>> have to buy it for 20.000 usd. Then we will have
>>> to provide a
>>>
>>> backbox that
>>>
>>> we have no control upon and that will kill our
>>> product anytime they
>>>
>>> want anf
>>>
>>> if they decide to stop their commercial activity
>>> we will be in the
>>>
>>> same
>>>
>>> situation that with dokanfs but without having
>>> the source code of
>>>
>>> the black
>>>
>>> box. If i have to spend 20 000 dollars i would
>>> prefere paying
>>>
>>> someone to
>>>
>>> retake dokanfs or to write from scratch a
>>> dokanfs fuselike software
>>>
>>> make it
>>>
>>> all shiny and pumpy fantastic and ready to plug
>>> to ceph client.
>>>
>>> I would prefere if people have to pay something
>>> to get access to
>>>
>>> ceph4win
>>>
>>> that this money goes in ceph main branch
>>> pockets... Or as a gift you
>>>
>>> donante
>>>
>>> to ceph 10 dollars you get 2 free registration
>>> codes for
>>>
>>> ceph4win... or
>>>
>>> something like that.
>>>
>>> If ceph4win as to be comercial then I would
>>> prefer delegate the task
>>>
>>> to a
>>>
>>> company like south river technologies and their
>>> great product
>>>
>>> webdrive. I
>>>
>>> would mininaly get involved in that project and
>>> simply buy the final
>>>
>>> product
>>>
>>> to sell it together with my ceph based product
>>> (which could be a
>>>
>>> calxeda
>>>
>>> ceph box or something like that).
>>>
>>> I m open anyway to any proposition. But I doubt
>>> that callback
>>>
>>> filesystem
>>>
>>> offers us a suitable solution in the way I see
>>> ceph4win to be spread
>>>
>>> and
>>>
>>> used... I m maybe wrong. And anything that will
>>> be done around
>>>
>>> ceph4win will
>>>
>>> be public documented etc... And licensed the way
>>> that if someone
>>>
>>> want to
>>>
>>> build a commercial solution on top of it, that
>>> would be a
>>>
>>> possibility.
>>>
>>> My idea is to giveback somehow to ceph project
>>> and at same time
>>>
>>> forge a
>>>
>>> better knowledge in ceph technologies. Because
>>> like many in libre
>>>
>>> world I
>>>
>>> think the business is in the services around the
>>> software more than
>>>
>>> on the
>>>
>>> software. That the ones writing code should be
>>> financed and benefits
>>>
>>> from
>>>
>>> the one selling and giving support of the
>>> software at all levels. I
>>>
>>> m
>>>
>>> probably too idealistic. And too optimistic
>>> after all I m the one
>>>
>>> saying I
>>>
>>> will do this stuff I have no idea how but well
>>> it is interesting and
>>>
>>> fun so
>>>
>>> lets do it.
>>>
>>> Regards,
>>>
>>> P.S: using commercial backend libraries appart
>>> including their own
>>>
>>> cost will
>>>
>>> force you to use commercial IDE like MS
>>> VisualStudio because their
>>>
>>> library
>>>
>>> has some kind of drm that only that IDE compiler
>>> can use. So alot of
>>>
>>> cost
>>>
>>> and yet there is nothing done. If I had to open
>>> a kickstarter
>>>
>>> project saying
>>>
>>> we need 60 000 USD to do ceph4win with that
>>> monney we will buy the
>>>
>>> right to
>>>
>>> use and share a commercial copyrighted library
>>> but abandonned
>>>
>>> punctually to
>>>
>>> us in public domaine and that we will
>>> eventually produce something
>>>
>>> out of
>>>
>>> it. I doubt I will get a dollar.
>>>
>>> We still can suggest the idea to Edlos the
>>> commercial company that
>>>
>>> has the
>>>
>>> copyright of Callback FS, Or to buy them their
>>> product in a blender
>>>
>>> way
>>>
>>> (blender was bought with donation before being
>>> put opensource and
>>>
>>> public
>>>
>>> domaine), Or to open source their library. But
>>> in commercial minds
>>> opensourcing = death of their technical
>>> advantage and death of
>>>
>>> their
>>>
>>> marketing strategy. They will have to invent
>>> something more to
>>>
>>> retrieve
>>>
>>> monney from it.
>>>
>>> El nov 6, 2013 11:22 p.m., "Ketor D"
>>> <d.ketor@gmail.com <mailto:d.ketor@gmail.com>
>>> <mailto:d.ketor@gmail.com
>>> <mailto:d.ketor@gmail.com>>> escribió:
>>>
>>>
>>> Hi Alphe,
>>> I think you could try Callback
>>> Filesystem dev
>>>
>>> framework. It
>>>
>>> is a commerical dev framework and is
>>> maintained by Edlos today.
>>> I have communicated with Edlos to
>>> get a try code for
>>> development. To dokan, Callback Filesystem
>>> has vary document and
>>>
>>> maybe
>>>
>>> more stabilize.
>>>
>>> Regards.
>>>
>>>
>>>
>>> On Thu, Nov 7, 2013 at 10:00 AM, Alphe
>>> Salas <asalas@kepler.cl <mailto:asalas@kepler.cl>
>>> <mailto:asalas@kepler.cl
>>>
>>> <mailto:asalas@kepler.cl>>> wrote:
>>> > Hello ketor thank you for your interest
>>> un ceph4win. Since
>>>
>>> muy
>>>
>>> first mail I
>>> > exposed the lacks of dokanfs and that I
>>> m far from being a
>>> specialist un
>>> > filesystems.
>>> > I exposed what i like un dokanfs bit I
>>> not a fanátic of it.
>>>
>>> Muy
>>>
>>> goal is to
>>> > have something working quickly.
>>> >
>>> > So I am up to any proposición sure the
>>> one with the more docs
>>>
>>> and
>>>
>>> support
>>> > will be the best choice. As for right
>>> now what I need is
>>> understand what are
>>> > the files involved what are the
>>> interfaces functions and what
>>>
>>> are
>>>
>>> the needed
>>> > library dependencies and if they exist
>>> ported to windows with
>>> cygwin. And
>>> > all that is retrieved from source code.
>>> >
>>> > Regards.
>>> >
>>> > El nov 6, 2013 10:34 p.m., "Ketor D"
>>> <d.ketor@gmail.com <mailto:d.ketor@gmail.com>
>>> <mailto:d.ketor@gmail.com
>>>
>>> <mailto:d.ketor@gmail.com>>> escribió:
>>>
>>> >
>>> >> Hi Alphe,
>>> >> We are taking an interest in your
>>> work on Ceph Client
>>>
>>> for
>>>
>>> Windows
>>> >> with Dokan.As we know, the performance
>>> of Dokan is not very
>>> good, and it's
>>> >> abandoned 3 years ago.
>>> >> I have learned and used
>>> OpenDedup(SDFS) for a long
>>>
>>> time.
>>>
>>> OpenDedup
>>> >> has a Dokan version. And the author of
>>> OpenDedup said
>>> >>
>>> >> The Dokan library is quite flakey and
>>> testing should be
>>> performed before
>>> >> putting into production
>>> >>
>>> >> So what do you think about this?
>>> And if there is
>>>
>>> another
>>>
>>> solution of
>>> >> fuse-like filesystem dev framwork on
>>> Windows?
>>> >>
>>> >> Best Wish!
>>> >>
>>> >>
>>> >>
>>> >> On Thu, Nov 7, 2013 at 5:47 AM, Alphe
>>> Salas Michels
>>> <asalas@kepler.cl <mailto:asalas@kepler.cl>
>>> <mailto:asalas@kepler.cl <mailto:asalas@kepler.cl>>>
>>>
>>> >> wrote:
>>> >>>
>>> >>> Hello I created the github repository
>>> for this project
>>> >>>https://github.com/alphe/Ceph4Win
>>> >>>
>>> >>> Regards,
>>> >>>
>>> >>> signature
>>> >>>
>>> >>> *Alphé Salas*
>>> >>> Ingeniero T.I
>>> >>>
>>> >>>asalas@kepler.cl
>>> <mailto:asalas@kepler.cl>
>>> <mailto:asalas@kepler.cl <mailto:asalas@kepler.cl>>
>>>
>>>
>>> >>> *<http://www.kepler.cl>*
>>> >>>
>>> >>> On 11/05/13 21:00, Sage Weil wrote:
>>> >>>>
>>> >>>> Hi Alphe,
>>> >>>>
>>> >>>> On Tue, 5 Nov 2013, Alphe Salas
>>> Michels wrote:
>>> >>>>>
>>> >>>>> signature *Hi, Sage !
>>> >>>>> thank you for you enthousiast reply.
>>> >>>>> I sure want to make the best use of
>>> everything or
>>>
>>> anything
>>>
>>> previously
>>> >>>>> done to
>>> >>>>> tend to
>>> >>>>> write ceph cliente for windows.
>>> >>>>>
>>> >>>>> Apart using libre tools for building
>>> the future ceph
>>>
>>> cliente
>>>
>>> I am open
>>> >>>>> to
>>> >>>>> anything.
>>> >>>>> I would recommand eclipse CDT or
>>> Code::BLocks they are
>>>
>>> based
>>>
>>> on mingwin
>>> >>>>> open
>>> >>>>> and easyly enhanceable.**
>>> >>>>>
>>> >>>>> more free tools can be found here:
>>>
>>>>>>>>http://www.freebyte.com/programming/cpp/#cppcompilers
>>> >>>>>
>>> >>>>>
>>> >>>>> I will read libcephfs source code
>>> and take some notes
>>>
>>> about the
>>>
>>> >>>>> protocol.
>>> >>>>
>>> >>>> I think you don't need to worry about
>>> hte protocol at all,
>>>
>>> since
>>>
>>> >>>> libcephs
>>> >>>> implements it for you (and will
>>> capture any future
>>>
>>> changes).
>>>
>>> >>>>
>>> >>>>> I was more going from what I know
>>> and trying to track down
>>>
>>> how
>>>
>>> >>>>> mount.ceph work
>>> >>>>> with the parameters passed to it.
>>> >>>>> since it point finally to
>>> Kernel/fs/ceph and that I don t
>>>
>>> really
>>>
>>> >>>>> understand
>>> >>>>> how that module work and that it
>>> probably points to some
>>>
>>> other
>>>
>>> >>>>> dependencies
>>> >>>>> Reading libcephfs source code could
>>> be a big gain of
>>>
>>> time.
>>>
>>> >>>>
>>> >>>> (I would also ignore mount.ceph as
>>> everything it does it
>>> specific to
>>> >>>> how Linux mounts work.)
>>> >>>>
>>> >>>>> basically on the protocol what is
>>> need are:
>>> >>>>>
>>> >>>>> 1) open and maintain a connection
>>> (socket open, auth, etc
>>>
>>> )
>>>
>>> >>>>> 2) retreive a map of directories and
>>> disk Quota (disk
>>>
>>> sizing
>>>
>>> Y TB free,
>>> >>>>> Z TB
>>> >>>>> total)
>>> >>>>> 3) procedure to send files /
>>> directories in a maner that
>>>
>>> it
>>>
>>> will allow
>>> >>>>> our
>>> >>>>> client to fit ceph transmission
>>> protocols
>>> >>>>> (limit bandwith for stability?,
>>> limit connection amount?,
>>> limit cpu
>>> >>>>> use?,
>>> >>>>> Cache for preparing data transfer (a
>>> FIFO cache)?)
>>> >>>>> 4)Procedure to retreive files /
>>> directory from ceph
>>>
>>> cluster
>>>
>>> >>>>> 5) Management copy/move files
>>> /Directories, FS stats,
>>> Connection Stats.
>>> >>>>> logging.
>>> >>>>>
>>> >>>>> My idea to progress is to take those
>>> main bulletpoint in
>>>
>>> ceph
>>>
>>> protocol
>>> >>>>> based
>>> >>>>> on general ideas of what ceph file
>>> system does and start
>>> identifying
>>> >>>>> parts
>>> >>>>> from libcephfs to match those "needs".
>>> >>>>
>>> >>>> Instead, I would look at
>>> include/cephfs/libcephfs.h, the
>>> interface that
>>> >>>> libcephfs provides, and try to map
>>> that to what the fuse
>>>
>>> layer
>>>
>>> expects.
>>> >>>> There is both a path-based that I
>>> suspsect lends itself
>>>
>>> well
>>>
>>> to the
>>> >>>> Windows interface and (very soon now)
>>> a handle based API
>>>
>>> that is
>>>
>>> >>>> targetted
>>> >>>> at the Unix-style VFS layers. I'm
>>> mostly guessing,
>>>
>>> though,
>>>
>>> since I've
>>> >>>> never seen any low-level fs code in
>>> windows before.
>>> >>>>
>>> >>>> In this case, the analogous code for
>>> Linux should be
>>> client/fuse_ll.cc
>>> >>>> itself (and not much else), although
>>> there will probably be
>>>
>>> a
>>>
>>> few tricks
>>> >>>> necessary to map cleanly onto how the
>>> windows interfaces
>>>
>>> work.
>>>
>>> >>>>
>>> >>>> Does that make sense?
>>> >>>>
>>> >>>> Cheers!
>>> >>>> sage
>>> >>>>
>>> >>>>
>>> >>>>> Any suggestion and contributions are
>>> welcome.
>>> >>>>>
>>> >>>>>
>>> >>>>> *
>>> >>>>> On 11/05/13 11:23, Sage Weil wrote:
>>> >>>>>>
>>> >>>>>> Hi Alphe,
>>> >>>>>>
>>> >>>>>> On Mon, 4 Nov 2013, Alphe Salas
>>> Michels wrote:
>>> >>>>>>>
>>> >>>>>>> Good day developers!
>>> >>>>>>>
>>> >>>>>>> I would like to propose to the one
>>> interested work with
>>>
>>> me to
>>>
>>> >>>>>>> develop a
>>> >>>>>>> ceph
>>> >>>>>>> cliente for MS windows world,
>>> Basing us on dokanFS.
>>> >>>>>>>
>>> >>>>>>> My company is a ceph enthousiast
>>> that use on a dayly
>>>
>>> basis
>>>
>>> ceph and
>>> >>>>>>> that
>>> >>>>>>> need
>>> >>>>>>> both transfer speed and big
>>> expendable and cheap
>>>
>>> storage.
>>>
>>> >>>>>>> My company is specialised in data
>>> recovery and we want
>>>
>>> to
>>>
>>> participate
>>> >>>>>>> to
>>> >>>>>>> ceph
>>> >>>>>>> effort by bringing a ceph cliente
>>> for windows.
>>> >>>>>>
>>> >>>>>> Awesome!
>>> >>>>>>
>>> >>>>>>> Our experience shows us that the
>>> best gateway is each
>>> clientes being
>>> >>>>>>> its
>>> >>>>>>> own
>>> >>>>>>> gateway, instead of having a
>>> bottle neck server or a
>>>
>>> cluster of
>>>
>>> >>>>>>> bottle
>>> >>>>>>> neck
>>> >>>>>>> servers as gateway (FTP, samba,
>>> SFTP,webdav, s3,
>>>
>>> etc..).
>>>
>>> >>>>>>>
>>> >>>>>>> We already did some research in
>>> that domain.
>>> >>>>>>>
>>> >>>>>>> Dokan FS is an intent to write an
>>> opensource fuse like
>>> cliente for
>>> >>>>>>> MS
>>> >>>>>>> windows.
>>> >>>>>>>
>>> >>>>>>> More information on DOKANFS can be
>>> triggered here
>>> >>>>>>>http://dokan-dev.net/en/download/
>>> >>>>>>>
>>> >>>>>>> Positive points of using DOKANFS.
>>> >>>>>>>
>>> >>>>>>> - its opensourced and well
>>> licenced mit licence, gpl
>>> licence and lgpl
>>> >>>>>>> licence.
>>> >>>>>>>
>>> >>>>>>> Negative point of using DOKAN FS.
>>> >>>>>>> - unreachable author
>>> >>>>>>> - Poor documentation . Dev
>>> comments in japanese.
>>> >>>>>>> - Work in progress so it is
>>> unstable and needs to be
>>>
>>> updated,
>>>
>>> >>>>>>> debugged and
>>> >>>>>>> maintained by a MS Windows file
>>> system expert
>>>
>>> developper.
>>>
>>> >>>>>>
>>> >>>>>> I am not very familiar with windows
>>> storage APIs, but
>>> somebody told me
>>> >>>>>> at once point there were several
>>> interfaces against which
>>>
>>> a
>>>
>>> new file
>>> >>>>>> system could be implemented,
>>> everything from a full
>>> in-kernel driver
>>> >>>>>> to
>>> >>>>>> something that is explorer-based.
>>> Are any of those
>>> suitable? Using a
>>> >>>>>> potentially abandoned fuse-like
>>> layer makes me a bit
>>>
>>> nervous.
>>>
>>> >>>>>>
>>> >>>>>> That said,
>>> >>>>>>
>>> >>>>>>>
>>> >>>>>>> I try past year to do a merge from
>>> ceph-fuse to dokanfs
>>> >>>>>>> here are what I learnt.
>>> >>>>>>> - Ceph-fuse and related source
>>> code is around 60 000
>>>
>>> lines
>>>
>>> of code.
>>> >>>>>>> - Ceph protocol isn t documented
>>> so it is like trying
>>>
>>> to
>>>
>>> draw a map
>>> >>>>>>> of
>>> >>>>>>> america
>>> >>>>>>> using only a sextan and a compass.
>>> >>>>>>>
>>> >>>>>>> Those led me to those conclusions:
>>> >>>>>>> - I can t do it alone.
>>> >>>>>>> - It is easier to draw down the
>>> ceph protocol way to
>>>
>>> work from
>>>
>>> >>>>>>> kernel/fs/ceph
>>> >>>>>>> sources and mount.ceph
>>> >>>>>>> - Ceph depending libraries may be
>>> unexistant or not up
>>>
>>> to
>>>
>>> date in
>>> >>>>>>> their
>>> >>>>>>> port
>>> >>>>>>> on MS Windows (cygwin)
>>> >>>>>>
>>> >>>>>> I think the most sane path should
>>> be to make libcephfs
>>> sufficiently
>>> >>>>>> portable to build on windows (or
>>> cygwin). For the bits
>>>
>>> used
>>>
>>> by the
>>> >>>>>> client-side coe, I don't think
>>> there should be much in
>>>
>>> the
>>>
>>> way of
>>> >>>>>> dependencies, and the main
>>> challenge would be untangling
>>>
>>> the
>>>
>>> build for
>>> >>>>>> the necessary pieces out from the
>>> rest of Ceph.
>>> >>>>>>
>>> >>>>>> Have you seen the wip-port
>>> portability work that is
>>> currently underway
>>> >>>>>> by
>>> >>>>>> Noah and Alan? That may solve many
>>> of the cygwin
>>>
>>> problems
>>>
>>> you are
>>> >>>>>> seeing
>>> >>>>>> today.
>>> >>>>>>
>>> >>>>>>> - MS file system specialist are
>>> hard do find in the
>>>
>>> "open
>>>
>>> source
>>> >>>>>>> libre
>>> >>>>>>> world"
>>> >>>>>>> so I will try in the commercial world.
>>> >>>>>>>
>>> >>>>>>> The commercial world has some
>>> problems too. They need
>>>
>>> ceph
>>>
>>> protocol
>>> >>>>>>> draft
>>> >>>>>>> to
>>> >>>>>>> implemente it to their own product
>>> They will have
>>>
>>> licencing
>>>
>>> >>>>>>> /commercial
>>> >>>>>>> politics that infringe lgpl, and
>>> hide that most of the
>>>
>>> work
>>>
>>> is done
>>> >>>>>>> by
>>> >>>>>>> people
>>> >>>>>>> other than them. They will not
>>> participate in a
>>>
>>> financial
>>>
>>> way to ceph
>>> >>>>>>> enhancement and growth.
>>> >>>>>>
>>> >>>>>> I don't think reimplementing the
>>> client code is an
>>>
>>> efficient way
>>>
>>> >>>>>> forward.
>>> >>>>>> Unless the goal is a pure kernel
>>> implementation...but a
>>> significant
>>> >>>>>> ongoing investment in development
>>> resources would be
>>>
>>> needed
>>>
>>> for that
>>> >>>>>> going
>>> >>>>>> forward. I suspect that is a
>>> challenge for a platform
>>>
>>> that
>>>
>>> does not
>>> >>>>>> typically rally that sort of
>>> community effort.
>>> >>>>>>
>>> >>>>>> The easiest thing is of course just
>>> to use CIFS and
>>>
>>> Samba
>>>
>>> (which works
>>> >>>>>> today). A fuse-like approach is
>>> probably a reasonably
>>> middle ground
>>> >>>>>> (both
>>> >>>>>> in initial effort and
>>> maintainability going forward)...
>>> >>>>>>
>>> >>>>>> sage
>>> >>>>>>
>>> >>>>>>
>>> >>>>>
>>> >>>
>>> >>> --
>>> >>> To unsubscribe from this list: send
>>> the line "unsubscribe
>>> ceph-devel" in
>>> >>> the body of a message
>>> tomajordomo@vger.kernel.org
>>> <mailto:tomajordomo@vger.kernel.org>
>>> <mailto:majordomo@vger.kernel.org
>>> <mailto:majordomo@vger.kernel.org>>
>>>
>>> >>> More majordomo info at
>>>
>>> http://vger.kernel.org/majordomo-info.html
>>>
>>> >>
>>> >>
>>> >
>>>
>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line
>>> "unsubscribe ceph-devel"
>>> in
>>> the body of a message tomajordomo@vger.kernel.org
>>> <mailto:tomajordomo@vger.kernel.org>
>>> More majordomo info
>>> athttp://vger.kernel.org/majordomo-info.html
>>> <http://vger.kernel.org/majordomo-info.html>
>>>
>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe
>>> ceph-devel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> <mailto:majordomo@vger.kernel.org>
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
>
> --
> Dong Yuan
> Email:yuandong1222@gmail.com
--
Dong Yuan
Email:yuandong1222@gmail.com
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2013-11-06 0:00 ` Sage Weil
2013-11-06 14:37 ` Alphe Salas Michels
@ 2013-11-06 21:47 ` Alphe Salas Michels
1 sibling, 0 replies; 21+ messages in thread
From: Alphe Salas Michels @ 2013-11-06 21:47 UTC (permalink / raw)
To: Sage Weil; +Cc: ceph-devel
Hello I created the github repository for this project
https://github.com/alphe/Ceph4Win
Regards,
signature
*Alphé Salas*
Ingeniero T.I
asalas@kepler.cl
*<http://www.kepler.cl>*
On 11/05/13 21:00, Sage Weil wrote:
> Hi Alphe,
>
> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
>> signature *Hi, Sage !
>> thank you for you enthousiast reply.
>> I sure want to make the best use of everything or anything previously done to
>> tend to
>> write ceph cliente for windows.
>>
>> Apart using libre tools for building the future ceph cliente I am open to
>> anything.
>> I would recommand eclipse CDT or Code::BLocks they are based on mingwin open
>> and easyly enhanceable.**
>>
>> more free tools can be found here:
>> http://www.freebyte.com/programming/cpp/#cppcompilers
>>
>>
>> I will read libcephfs source code and take some notes about the protocol.
> I think you don't need to worry about hte protocol at all, since libcephs
> implements it for you (and will capture any future changes).
>
>> I was more going from what I know and trying to track down how mount.ceph work
>> with the parameters passed to it.
>> since it point finally to Kernel/fs/ceph and that I don t really understand
>> how that module work and that it probably points to some other dependencies
>> Reading libcephfs source code could be a big gain of time.
> (I would also ignore mount.ceph as everything it does it specific to
> how Linux mounts work.)
>
>> basically on the protocol what is need are:
>>
>> 1) open and maintain a connection (socket open, auth, etc )
>> 2) retreive a map of directories and disk Quota (disk sizing Y TB free, Z TB
>> total)
>> 3) procedure to send files / directories in a maner that it will allow our
>> client to fit ceph transmission protocols
>> (limit bandwith for stability?, limit connection amount?, limit cpu use?,
>> Cache for preparing data transfer (a FIFO cache)?)
>> 4)Procedure to retreive files / directory from ceph cluster
>> 5) Management copy/move files /Directories, FS stats, Connection Stats.
>> logging.
>>
>> My idea to progress is to take those main bulletpoint in ceph protocol based
>> on general ideas of what ceph file system does and start identifying parts
>> from libcephfs to match those "needs".
> Instead, I would look at include/cephfs/libcephfs.h, the interface that
> libcephfs provides, and try to map that to what the fuse layer expects.
> There is both a path-based that I suspsect lends itself well to the
> Windows interface and (very soon now) a handle based API that is targetted
> at the Unix-style VFS layers. I'm mostly guessing, though, since I've
> never seen any low-level fs code in windows before.
>
> In this case, the analogous code for Linux should be client/fuse_ll.cc
> itself (and not much else), although there will probably be a few tricks
> necessary to map cleanly onto how the windows interfaces work.
>
> Does that make sense?
>
> Cheers!
> sage
>
>
>> Any suggestion and contributions are welcome.
>>
>>
>> *
>> On 11/05/13 11:23, Sage Weil wrote:
>>> Hi Alphe,
>>>
>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
>>>> Good day developers!
>>>>
>>>> I would like to propose to the one interested work with me to develop a
>>>> ceph
>>>> cliente for MS windows world, Basing us on dokanFS.
>>>>
>>>> My company is a ceph enthousiast that use on a dayly basis ceph and that
>>>> need
>>>> both transfer speed and big expendable and cheap storage.
>>>> My company is specialised in data recovery and we want to participate to
>>>> ceph
>>>> effort by bringing a ceph cliente for windows.
>>> Awesome!
>>>
>>>> Our experience shows us that the best gateway is each clientes being its
>>>> own
>>>> gateway, instead of having a bottle neck server or a cluster of bottle
>>>> neck
>>>> servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
>>>>
>>>> We already did some research in that domain.
>>>>
>>>> Dokan FS is an intent to write an opensource fuse like cliente for MS
>>>> windows.
>>>>
>>>> More information on DOKANFS can be triggered here
>>>> http://dokan-dev.net/en/download/
>>>>
>>>> Positive points of using DOKANFS.
>>>>
>>>> - its opensourced and well licenced mit licence, gpl licence and lgpl
>>>> licence.
>>>>
>>>> Negative point of using DOKAN FS.
>>>> - unreachable author
>>>> - Poor documentation . Dev comments in japanese.
>>>> - Work in progress so it is unstable and needs to be updated, debugged and
>>>> maintained by a MS Windows file system expert developper.
>>> I am not very familiar with windows storage APIs, but somebody told me
>>> at once point there were several interfaces against which a new file
>>> system could be implemented, everything from a full in-kernel driver to
>>> something that is explorer-based. Are any of those suitable? Using a
>>> potentially abandoned fuse-like layer makes me a bit nervous.
>>>
>>> That said,
>>>
>>>> I try past year to do a merge from ceph-fuse to dokanfs
>>>> here are what I learnt.
>>>> - Ceph-fuse and related source code is around 60 000 lines of code.
>>>> - Ceph protocol isn t documented so it is like trying to draw a map of
>>>> america
>>>> using only a sextan and a compass.
>>>>
>>>> Those led me to those conclusions:
>>>> - I can t do it alone.
>>>> - It is easier to draw down the ceph protocol way to work from
>>>> kernel/fs/ceph
>>>> sources and mount.ceph
>>>> - Ceph depending libraries may be unexistant or not up to date in their
>>>> port
>>>> on MS Windows (cygwin)
>>> I think the most sane path should be to make libcephfs sufficiently
>>> portable to build on windows (or cygwin). For the bits used by the
>>> client-side coe, I don't think there should be much in the way of
>>> dependencies, and the main challenge would be untangling the build for
>>> the necessary pieces out from the rest of Ceph.
>>>
>>> Have you seen the wip-port portability work that is currently underway by
>>> Noah and Alan? That may solve many of the cygwin problems you are seeing
>>> today.
>>>
>>>> - MS file system specialist are hard do find in the "open source libre
>>>> world"
>>>> so I will try in the commercial world.
>>>>
>>>> The commercial world has some problems too. They need ceph protocol draft
>>>> to
>>>> implemente it to their own product They will have licencing /commercial
>>>> politics that infringe lgpl, and hide that most of the work is done by
>>>> people
>>>> other than them. They will not participate in a financial way to ceph
>>>> enhancement and growth.
>>> I don't think reimplementing the client code is an efficient way forward.
>>> Unless the goal is a pure kernel implementation...but a significant
>>> ongoing investment in development resources would be needed for that going
>>> forward. I suspect that is a challenge for a platform that does not
>>> typically rally that sort of community effort.
>>>
>>> The easiest thing is of course just to use CIFS and Samba (which works
>>> today). A fuse-like approach is probably a reasonably middle ground (both
>>> in initial effort and maintainability going forward)...
>>>
>>> sage
>>>
>>>
>>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2013-11-06 0:00 ` Sage Weil
@ 2013-11-06 14:37 ` Alphe Salas Michels
2013-11-06 21:47 ` Alphe Salas Michels
1 sibling, 0 replies; 21+ messages in thread
From: Alphe Salas Michels @ 2013-11-06 14:37 UTC (permalink / raw)
To: Sage Weil; +Cc: ceph-devel
Hi Sage,
When I initially planned past year to do a client for ceph on windows
what I inspected was ceph_fuse.cc
from there I tracked all the related files using a script that read the
includes precompiler instructions and
retreive the related files from the ceph source code git master branch.
In the end the work seemed to me gigantic.
Now that we have an open discussion on that topic and that you giveme
the libcephfs.h and fuseII.cc hints I will start
by isolate the source code related. I will modify my previous isolate
script to do so. I will create a github branch under ceph if that is ok
with you, to put the documentation, the isolated files and the dev tools
related.
Regards,
--- For notes my script working from content and related content
ceph_fuse.cc produced this:
find . -name '*' | xargs wc -l
wc: .: Is a directory
0 .
wc: ./common: Is a directory
0 ./common
29 ./common/compiler_extensions.h
39 ./common/simple_spin.h
31 ./common/config_obs.h
481 ./common/admin_socket.cc
33 ./common/pipe.h
119 ./common/LogEntry.h
135 ./common/DoutStreambuf.h
111 ./common/Cond.h
42 ./common/code_environment.h
112 ./common/Formatter.h
760 ./common/config.cc
52 ./common/escape.h
644 ./common/DoutStreambuf.cc
202 ./common/LogClient.cc
573 ./common/ConfUtils.cc
199 ./common/escape.c
84 ./common/Timer.h
25 ./common/hex.h
395 ./common/Formatter.cc
52 ./common/safe_io.h
82 ./common/HeartbeatMap.h
74 ./common/code_environment.cc
17 ./common/errno.cc
42 ./common/signal.h
144 ./common/Mutex.h
96 ./common/admin_socket.h
83 ./common/common_init.h
34 ./common/BackTrace.h
29 ./common/version.h
74 ./common/snap_types.h
238 ./common/ceph_context.cc
84 ./common/Finisher.cc
74 ./common/ceph_argparse.h
187 ./common/utf8.c
39 ./common/Clock.cc
50 ./common/Thread.h
148 ./common/HeartbeatMap.cc
51 ./common/utf8.h
160 ./common/Thread.cc
9 ./common/errno.h
62 ./common/BackTrace.cc
39 ./common/hex.cc
120 ./common/safe_io.c
27 ./common/Clock.h
161 ./common/DecayCounter.h
189 ./common/Timer.cc
119 ./common/Throttle.h
126 ./common/strtol.cc
320 ./common/perf_counters.cc
119 ./common/LogClient.h
33 ./common/debug.h
81 ./common/signal.cc
44 ./common/pipe.c
210 ./common/config.h
24 ./common/static_assert.h
186 ./common/entity_name.cc
124 ./common/armor.c
24 ./common/likely.h
43 ./common/simple_spin.cc
73 ./common/ceph_crypto.cc
79 ./common/entity_name.h
41 ./common/version.cc
106 ./common/dout.h
28 ./common/strtol.h
86 ./common/ConfUtils.h
86 ./common/Finisher.h
108 ./common/LogEntry.cc
179 ./common/perf_counters.h
16 ./common/armor.h
93 ./common/snap_types.cc
409 ./common/ceph_argparse.cc
110 ./common/common_init.cc
114 ./common/ceph_context.h
160 ./common/ceph_crypto.h
wc: ./msg: Is a directory
0 ./msg
623 ./msg/SimpleMessenger.h
673 ./msg/Message.cc
263 ./msg/Messenger.h
478 ./msg/Message.h
153 ./msg/msg_types.cc
64 ./msg/Dispatcher.h
2824 ./msg/SimpleMessenger.cc
425 ./msg/msg_types.h
wc: ./messages: Is a directory
0 ./messages
81 ./messages/MOSDPGCreate.h
67 ./messages/MRoute.h
72 ./messages/MForward.h
83 ./messages/MMonElection.h
44 ./messages/MPGStatsAck.h
67 ./messages/MClientSession.h
65 ./messages/MOSDBoot.h
90 ./messages/MClientReconnect.h
69 ./messages/MOSDFailure.h
78 ./messages/MOSDPing.h
47 ./messages/MLogAck.h
206 ./messages/MClientRequest.h
51 ./messages/MClientCapRelease.h
57 ./messages/MMonObserve.h
50 ./messages/MMonSubscribeAck.h
57 ./messages/MOSDPGMissing.h
66 ./messages/MClientSnap.h
55 ./messages/MCommandReply.h
106 ./messages/MMonProbe.h
64 ./messages/MOSDPGQuery.h
85 ./messages/MOSDPGNotify.h
95 ./messages/MMonSubscribe.h
223 ./messages/MOSDSubOp.h
55 ./messages/MGetPoolStatsReply.h
62 ./messages/MMonObserveNotify.h
64 ./messages/PaxosServiceMessage.h
59 ./messages/MMonJoin.h
101 ./messages/MPoolOp.h
58 ./messages/MLog.h
57 ./messages/MAuth.h
44 ./messages/MStatfsReply.h
54 ./messages/MMonCommandAck.h
35 ./messages/MMonGetMap.h
78 ./messages/MOSDPGLog.h
52 ./messages/MStatfs.h
54 ./messages/MOSDPGTrim.h
172 ./messages/MClientCaps.h
130 ./messages/MMonPaxos.h
143 ./messages/MOSDMap.h
222 ./messages/MOSDOpReply.h
62 ./messages/MOSDPGRemove.h
81 ./messages/MOSDPGScan.h
39 ./messages/MGenericMessage.h
146 ./messages/MOSDSubOpReply.h
54 ./messages/MOSDPGInfo.h
52 ./messages/MOSDPGTemp.h
70 ./messages/MOSDRepScrub.h
388 ./messages/MOSDOp.h
63 ./messages/MClientRequestForward.h
64 ./messages/MMonGetVersionReply.h
66 ./messages/MOSDScrub.h
59 ./messages/MCommand.h
266 ./messages/MClientReply.h
58 ./messages/MMonGetVersion.h
61 ./messages/MPGStats.h
49 ./messages/MOSDAlive.h
51 ./messages/MRemoveSnaps.h
83 ./messages/MClientLease.h
35 ./messages/MPing.h
83 ./messages/MOSDPGBackfill.h
60 ./messages/MMonCommand.h
80 ./messages/MPoolOpReply.h
57 ./messages/MGetPoolStats.h
96 ./messages/MMDSMap.h
43 ./messages/MMonMap.h
65 ./messages/MAuthReply.h
12 ./ceph_ver
186 ./acconfig
wc: ./osdc: Is a directory
0 ./osdc
283 ./osdc/Filer.h
2023 ./osdc/Objecter.cc
1774 ./osdc/ObjectCacher.cc
1479 ./osdc/Objecter.h
571 ./osdc/rados_bencher.h
388 ./osdc/Filer.cc
694 ./osdc/ObjectCacher.h
wc: ./include: Is a directory
0 ./include
600 ./include/frag.h
28 ./include/addr_parsing.h
43 ./include/ceph_features.h
18 ./include/page.h
22 ./include/compat.h
192 ./include/CompatSet.h
335 ./include/lru.h
253 ./include/utime.h
113 ./include/assert.h
47 ./include/blobhash.h
83 ./include/ceph_fs.cc
786 ./include/encoding.h
547 ./include/interval_set.h
312 ./include/Context.h
499 ./include/buffer.h
750 ./include/ceph_fs.h
183 ./include/msgr.h
124 ./include/atomic.h
29 ./include/err.h
225 ./include/filepath.h
38 ./include/intarith.h
14 ./include/str_list.h
13 ./include/color.h
28 ./include/inttypes.h
428 ./include/types.h
150 ./include/addr_parsing.c
105 ./include/cmp.h
411 ./include/rados.h
192 ./include/object.h
165 ./include/xlist.h
wc: ./client: Is a directory
0 ./client
627 ./client/fuse_ll.cc
661 ./client/Client.h
15 ./client/fuse_ll.h
6923 ./client/Client.cc
177 ./ceph_fuse
wc: ./none: Is a directory
0 ./none
wc: ./mds: Is a directory
0 ./mds
253 ./mds/MDSMap.cc
74 ./mds/inode_backtrace.h
1581 ./mds/mdstypes.h
622 ./mds/MDSMap.h
wc: ./global: Is a directory
0 ./global
28 ./global/pidfile.h
98 ./global/pidfile.cc
45 ./global/signal_handler.h
338 ./global/signal_handler.cc
226 ./global/global_init.cc
23 ./global/global_context.cc
64 ./global/global_init.h
33 ./global/global_context.h
wc: ./json_spirit: Is a directory
0 ./json_spirit
8 ./json_spirit/json_spirit_value.cpp
585 ./json_spirit/json_spirit_value.h
wc: ./cephx: Is a directory
0 ./cephx
wc: ./os: Is a directory
0 ./os
79 ./os/hobject.cc
639 ./os/ObjectStore.cc
129 ./os/hobject.h
738 ./os/ObjectStore.h
1378 ./rados
wc: ./crush: Is a directory
0 ./crush
518 ./crush/CrushWrapper.cc
47 ./crush/CrushWrapper.i
427 ./crush/CrushWrapper.h
wc: ./mon: Is a directory
0 ./mon
406 ./mon/MonCaps.cc
106 ./mon/MonCaps.h
40 ./mon/mon_types.h
158 ./mon/Session.h
810 ./mon/MonClient.cc
267 ./mon/MonClient.h
130 ./mon/MonMap.cc
204 ./mon/MonMap.h
wc: ./osd: Is a directory
0 ./osd
694 ./osd/OSDMap.h
1755 ./osd/osd_types.h
1310 ./osd/OSDMap.cc
2464 ./osd/osd_types.cc
wc: ./auth: Is a directory
0 ./auth
33 ./auth/AuthServiceHandler.cc
118 ./auth/Crypto.h
37 ./auth/AuthSupported.h
44 ./auth/AuthServiceHandler.h
85 ./auth/KeyRing.h
437 ./auth/Crypto.cc
54 ./auth/RotatingKeyRing.h
wc: ./auth/none: Is a directory
0 ./auth/none
52 ./auth/none/AuthNoneClientHandler.h
31 ./auth/none/AuthNoneAuthorizeHandler.h
41 ./auth/none/AuthNoneServiceHandler.h
32 ./auth/none/AuthNoneProtocol.h
39 ./auth/none/AuthNoneAuthorizeHandler.cc
77 ./auth/RotatingKeyRing.cc
259 ./auth/KeyRing.cc
39 ./auth/AuthClientHandler.cc
89 ./auth/AuthClientHandler.h
wc: ./auth/cephx: Is a directory
0 ./auth/cephx
69 ./auth/cephx/CephxClientHandler.h
505 ./auth/cephx/CephxProtocol.cc
209 ./auth/cephx/CephxClientHandler.cc
418 ./auth/cephx/CephxKeyServer.cc
33 ./auth/cephx/CephxAuthorizeHandler.cc
203 ./auth/cephx/CephxServiceHandler.cc
487 ./auth/cephx/CephxProtocol.h
31 ./auth/cephx/CephxAuthorizeHandler.h
298 ./auth/cephx/CephxKeyServer.h
37 ./auth/cephx/CephxServiceHandler.h
246 ./auth/Auth.h
51 ./auth/AuthSupported.cc
63501 total
So ceph_fuse.cc and related files /dependencies as a ceph related only
layer involves 63501 lines of source code.
note that fuseII.cc is part of that extraction but libcephfs no.
signature
*Alphé Salas*
Ingeniero T.I
asalas@kepler.cl
On 11/05/13 21:00, Sage Weil wrote:
> Hi Alphe,
>
> On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
>> signature *Hi, Sage !
>> thank you for you enthousiast reply.
>> I sure want to make the best use of everything or anything previously done to
>> tend to
>> write ceph cliente for windows.
>>
>> Apart using libre tools for building the future ceph cliente I am open to
>> anything.
>> I would recommand eclipse CDT or Code::BLocks they are based on mingwin open
>> and easyly enhanceable.**
>>
>> more free tools can be found here:
>> http://www.freebyte.com/programming/cpp/#cppcompilers
>>
>>
>> I will read libcephfs source code and take some notes about the protocol.
> I think you don't need to worry about hte protocol at all, since libcephs
> implements it for you (and will capture any future changes).
>
>> I was more going from what I know and trying to track down how mount.ceph work
>> with the parameters passed to it.
>> since it point finally to Kernel/fs/ceph and that I don t really understand
>> how that module work and that it probably points to some other dependencies
>> Reading libcephfs source code could be a big gain of time.
> (I would also ignore mount.ceph as everything it does it specific to
> how Linux mounts work.)
>
>> basically on the protocol what is need are:
>>
>> 1) open and maintain a connection (socket open, auth, etc )
>> 2) retreive a map of directories and disk Quota (disk sizing Y TB free, Z TB
>> total)
>> 3) procedure to send files / directories in a maner that it will allow our
>> client to fit ceph transmission protocols
>> (limit bandwith for stability?, limit connection amount?, limit cpu use?,
>> Cache for preparing data transfer (a FIFO cache)?)
>> 4)Procedure to retreive files / directory from ceph cluster
>> 5) Management copy/move files /Directories, FS stats, Connection Stats.
>> logging.
>>
>> My idea to progress is to take those main bulletpoint in ceph protocol based
>> on general ideas of what ceph file system does and start identifying parts
>> from libcephfs to match those "needs".
> Instead, I would look at include/cephfs/libcephfs.h, the interface that
> libcephfs provides, and try to map that to what the fuse layer expects.
> There is both a path-based that I suspsect lends itself well to the
> Windows interface and (very soon now) a handle based API that is targetted
> at the Unix-style VFS layers. I'm mostly guessing, though, since I've
> never seen any low-level fs code in windows before.
>
> In this case, the analogous code for Linux should be client/fuse_ll.cc
> itself (and not much else), although there will probably be a few tricks
> necessary to map cleanly onto how the windows interfaces work.
>
> Does that make sense?
>
> Cheers!
> sage
>
>
>> Any suggestion and contributions are welcome.
>>
>>
>> *
>> On 11/05/13 11:23, Sage Weil wrote:
>>> Hi Alphe,
>>>
>>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
>>>> Good day developers!
>>>>
>>>> I would like to propose to the one interested work with me to develop a
>>>> ceph
>>>> cliente for MS windows world, Basing us on dokanFS.
>>>>
>>>> My company is a ceph enthousiast that use on a dayly basis ceph and that
>>>> need
>>>> both transfer speed and big expendable and cheap storage.
>>>> My company is specialised in data recovery and we want to participate to
>>>> ceph
>>>> effort by bringing a ceph cliente for windows.
>>> Awesome!
>>>
>>>> Our experience shows us that the best gateway is each clientes being its
>>>> own
>>>> gateway, instead of having a bottle neck server or a cluster of bottle
>>>> neck
>>>> servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
>>>>
>>>> We already did some research in that domain.
>>>>
>>>> Dokan FS is an intent to write an opensource fuse like cliente for MS
>>>> windows.
>>>>
>>>> More information on DOKANFS can be triggered here
>>>> http://dokan-dev.net/en/download/
>>>>
>>>> Positive points of using DOKANFS.
>>>>
>>>> - its opensourced and well licenced mit licence, gpl licence and lgpl
>>>> licence.
>>>>
>>>> Negative point of using DOKAN FS.
>>>> - unreachable author
>>>> - Poor documentation . Dev comments in japanese.
>>>> - Work in progress so it is unstable and needs to be updated, debugged and
>>>> maintained by a MS Windows file system expert developper.
>>> I am not very familiar with windows storage APIs, but somebody told me
>>> at once point there were several interfaces against which a new file
>>> system could be implemented, everything from a full in-kernel driver to
>>> something that is explorer-based. Are any of those suitable? Using a
>>> potentially abandoned fuse-like layer makes me a bit nervous.
>>>
>>> That said,
>>>
>>>> I try past year to do a merge from ceph-fuse to dokanfs
>>>> here are what I learnt.
>>>> - Ceph-fuse and related source code is around 60 000 lines of code.
>>>> - Ceph protocol isn t documented so it is like trying to draw a map of
>>>> america
>>>> using only a sextan and a compass.
>>>>
>>>> Those led me to those conclusions:
>>>> - I can t do it alone.
>>>> - It is easier to draw down the ceph protocol way to work from
>>>> kernel/fs/ceph
>>>> sources and mount.ceph
>>>> - Ceph depending libraries may be unexistant or not up to date in their
>>>> port
>>>> on MS Windows (cygwin)
>>> I think the most sane path should be to make libcephfs sufficiently
>>> portable to build on windows (or cygwin). For the bits used by the
>>> client-side coe, I don't think there should be much in the way of
>>> dependencies, and the main challenge would be untangling the build for
>>> the necessary pieces out from the rest of Ceph.
>>>
>>> Have you seen the wip-port portability work that is currently underway by
>>> Noah and Alan? That may solve many of the cygwin problems you are seeing
>>> today.
>>>
>>>> - MS file system specialist are hard do find in the "open source libre
>>>> world"
>>>> so I will try in the commercial world.
>>>>
>>>> The commercial world has some problems too. They need ceph protocol draft
>>>> to
>>>> implemente it to their own product They will have licencing /commercial
>>>> politics that infringe lgpl, and hide that most of the work is done by
>>>> people
>>>> other than them. They will not participate in a financial way to ceph
>>>> enhancement and growth.
>>> I don't think reimplementing the client code is an efficient way forward.
>>> Unless the goal is a pure kernel implementation...but a significant
>>> ongoing investment in development resources would be needed for that going
>>> forward. I suspect that is a challenge for a platform that does not
>>> typically rally that sort of community effort.
>>>
>>> The easiest thing is of course just to use CIFS and Samba (which works
>>> today). A fuse-like approach is probably a reasonably middle ground (both
>>> in initial effort and maintainability going forward)...
>>>
>>> sage
>>>
>>>
>>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2013-11-05 17:40 ` Alphe Salas Michels
2013-11-05 21:49 ` Alphe Salas Michels
@ 2013-11-06 0:00 ` Sage Weil
2013-11-06 14:37 ` Alphe Salas Michels
2013-11-06 21:47 ` Alphe Salas Michels
1 sibling, 2 replies; 21+ messages in thread
From: Sage Weil @ 2013-11-06 0:00 UTC (permalink / raw)
To: Alphe Salas Michels; +Cc: ceph-devel
Hi Alphe,
On Tue, 5 Nov 2013, Alphe Salas Michels wrote:
>
> signature *Hi, Sage !
> thank you for you enthousiast reply.
> I sure want to make the best use of everything or anything previously done to
> tend to
> write ceph cliente for windows.
>
> Apart using libre tools for building the future ceph cliente I am open to
> anything.
> I would recommand eclipse CDT or Code::BLocks they are based on mingwin open
> and easyly enhanceable.**
>
> more free tools can be found here:
> http://www.freebyte.com/programming/cpp/#cppcompilers
>
>
> I will read libcephfs source code and take some notes about the protocol.
I think you don't need to worry about hte protocol at all, since libcephs
implements it for you (and will capture any future changes).
> I was more going from what I know and trying to track down how mount.ceph work
> with the parameters passed to it.
> since it point finally to Kernel/fs/ceph and that I don t really understand
> how that module work and that it probably points to some other dependencies
> Reading libcephfs source code could be a big gain of time.
(I would also ignore mount.ceph as everything it does it specific to
how Linux mounts work.)
> basically on the protocol what is need are:
>
> 1) open and maintain a connection (socket open, auth, etc )
> 2) retreive a map of directories and disk Quota (disk sizing Y TB free, Z TB
> total)
> 3) procedure to send files / directories in a maner that it will allow our
> client to fit ceph transmission protocols
> (limit bandwith for stability?, limit connection amount?, limit cpu use?,
> Cache for preparing data transfer (a FIFO cache)?)
> 4)Procedure to retreive files / directory from ceph cluster
> 5) Management copy/move files /Directories, FS stats, Connection Stats.
> logging.
>
> My idea to progress is to take those main bulletpoint in ceph protocol based
> on general ideas of what ceph file system does and start identifying parts
> from libcephfs to match those "needs".
Instead, I would look at include/cephfs/libcephfs.h, the interface that
libcephfs provides, and try to map that to what the fuse layer expects.
There is both a path-based that I suspsect lends itself well to the
Windows interface and (very soon now) a handle based API that is targetted
at the Unix-style VFS layers. I'm mostly guessing, though, since I've
never seen any low-level fs code in windows before.
In this case, the analogous code for Linux should be client/fuse_ll.cc
itself (and not much else), although there will probably be a few tricks
necessary to map cleanly onto how the windows interfaces work.
Does that make sense?
Cheers!
sage
>
> Any suggestion and contributions are welcome.
>
>
> *
> On 11/05/13 11:23, Sage Weil wrote:
> > Hi Alphe,
> >
> > On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
> > > Good day developers!
> > >
> > > I would like to propose to the one interested work with me to develop a
> > > ceph
> > > cliente for MS windows world, Basing us on dokanFS.
> > >
> > > My company is a ceph enthousiast that use on a dayly basis ceph and that
> > > need
> > > both transfer speed and big expendable and cheap storage.
> > > My company is specialised in data recovery and we want to participate to
> > > ceph
> > > effort by bringing a ceph cliente for windows.
> > Awesome!
> >
> > > Our experience shows us that the best gateway is each clientes being its
> > > own
> > > gateway, instead of having a bottle neck server or a cluster of bottle
> > > neck
> > > servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
> > >
> > > We already did some research in that domain.
> > >
> > > Dokan FS is an intent to write an opensource fuse like cliente for MS
> > > windows.
> > >
> > > More information on DOKANFS can be triggered here
> > > http://dokan-dev.net/en/download/
> > >
> > > Positive points of using DOKANFS.
> > >
> > > - its opensourced and well licenced mit licence, gpl licence and lgpl
> > > licence.
> > >
> > > Negative point of using DOKAN FS.
> > > - unreachable author
> > > - Poor documentation . Dev comments in japanese.
> > > - Work in progress so it is unstable and needs to be updated, debugged and
> > > maintained by a MS Windows file system expert developper.
> > I am not very familiar with windows storage APIs, but somebody told me
> > at once point there were several interfaces against which a new file
> > system could be implemented, everything from a full in-kernel driver to
> > something that is explorer-based. Are any of those suitable? Using a
> > potentially abandoned fuse-like layer makes me a bit nervous.
> >
> > That said,
> >
> > > I try past year to do a merge from ceph-fuse to dokanfs
> > > here are what I learnt.
> > > - Ceph-fuse and related source code is around 60 000 lines of code.
> > > - Ceph protocol isn t documented so it is like trying to draw a map of
> > > america
> > > using only a sextan and a compass.
> > >
> > > Those led me to those conclusions:
> > > - I can t do it alone.
> > > - It is easier to draw down the ceph protocol way to work from
> > > kernel/fs/ceph
> > > sources and mount.ceph
> > > - Ceph depending libraries may be unexistant or not up to date in their
> > > port
> > > on MS Windows (cygwin)
> > I think the most sane path should be to make libcephfs sufficiently
> > portable to build on windows (or cygwin). For the bits used by the
> > client-side coe, I don't think there should be much in the way of
> > dependencies, and the main challenge would be untangling the build for
> > the necessary pieces out from the rest of Ceph.
> >
> > Have you seen the wip-port portability work that is currently underway by
> > Noah and Alan? That may solve many of the cygwin problems you are seeing
> > today.
> >
> > > - MS file system specialist are hard do find in the "open source libre
> > > world"
> > > so I will try in the commercial world.
> > >
> > > The commercial world has some problems too. They need ceph protocol draft
> > > to
> > > implemente it to their own product They will have licencing /commercial
> > > politics that infringe lgpl, and hide that most of the work is done by
> > > people
> > > other than them. They will not participate in a financial way to ceph
> > > enhancement and growth.
> > I don't think reimplementing the client code is an efficient way forward.
> > Unless the goal is a pure kernel implementation...but a significant
> > ongoing investment in development resources would be needed for that going
> > forward. I suspect that is a challenge for a platform that does not
> > typically rally that sort of community effort.
> >
> > The easiest thing is of course just to use CIFS and Samba (which works
> > today). A fuse-like approach is probably a reasonably middle ground (both
> > in initial effort and maintainability going forward)...
> >
> > sage
> >
> >
>
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* RE: writing a ceph cliente for MS windows
2013-11-04 20:19 Alphe Salas Michels
2013-11-05 14:23 ` Sage Weil
@ 2013-11-05 23:33 ` James Harper
1 sibling, 0 replies; 21+ messages in thread
From: James Harper @ 2013-11-05 23:33 UTC (permalink / raw)
To: Alphe Salas Michels, ceph-devel
> Good day developers!
>
> I would like to propose to the one interested work with me to develop a
> ceph cliente for MS windows world, Basing us on dokanFS.
>
I've looked at porting the rbd client to windows a little while back. That would require a kernel driver and all the rbd stuff is C++ (windows kernel is strictly C) so it was looking like a lot of work. Porting the rbd kernel implementation from Linux would have been easier.
An alternative would be to write a fuse-like kernel driver that talks to a userspace implementation of librbd. That would have been slower though, and have some restrictions like no swap space and no booting (although booting would have been a hard problem to solve even with a kernel-only implementation...)
Good luck!
James
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2013-11-05 21:59 ` Yehuda Sadeh
@ 2013-11-05 22:31 ` Alphe Salas Michels
0 siblings, 0 replies; 21+ messages in thread
From: Alphe Salas Michels @ 2013-11-05 22:31 UTC (permalink / raw)
To: Yehuda Sadeh; +Cc: Sage Weil, ceph-devel
Hello sage and Yehuda,
yes nwat told me that u64 was an alias for unsigned long which is
default C compiler data type.
I wasn't cautious.
I read some other things like the cygwin mount.cc /mount.h files ...
they include cifs and nfs filetype things like samba too. We could
integrate a new data type in it with our ceph-win32-client.
Obviously everyone would prefere something with a nice point and click
window to setup their ceph-win32-client. But as a starter... maybe we
could understand better the way windows handle file systems starting
from cygwin/mount files.
Regards,
signature
*Alphé Salas*
Ingeniero T.I
asalas@kepler.cl*<http://www.kepler.cl>*
On 11/05/13 18:59, Yehuda Sadeh wrote:
> On Tue, Nov 5, 2013 at 1:49 PM, Alphe Salas Michels<asalas@kepler.cl> wrote:
>> Hello sage,
>> I followed your lead and went a bit further in my source code reading and I
>> notice serveral problems:
>> first most of the id use in ceph osd_clients, mds_clients, mon_client use
>> the data type u64 which will be a problem
>> as most of the windows in use are windows XP or Windows server 2003 in 32
>> bits. As they are used for id tag for
>> things like snapshot pages (if I understand well) that can be a problem for
>> a port no?
>>
>> What I read so far was
>> mount.ceph files > kernel/fs/ceph files which contains the mds_client files
>> and the kernel ioctl interface ... > include / linux / ceph files which
>> integrates libcephfs.h. and all the needed files
>>
>> there is three clients, there is auth.h a messenger, a msgpool, buffer,
>> rados that depends on msgr that include in every aspect of the messages
>> formation one or more __le64 message.
>>
>> I m far from understanding the whole code and the interactions betwin all
>> the files and which we can skip and which we can keep.
>>
>> how can we translate those data type into 32bit in order for ceph cluster to
>> understand them and ceph-windows-client to transmit them?
> It is possible to define 64 bit variables on 32 bit architectures. The
> compiler handles it for you.
>
> Yehuda
>
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2013-11-05 21:49 ` Alphe Salas Michels
@ 2013-11-05 21:59 ` Yehuda Sadeh
2013-11-05 22:31 ` Alphe Salas Michels
0 siblings, 1 reply; 21+ messages in thread
From: Yehuda Sadeh @ 2013-11-05 21:59 UTC (permalink / raw)
To: Alphe Salas Michels; +Cc: Sage Weil, ceph-devel
On Tue, Nov 5, 2013 at 1:49 PM, Alphe Salas Michels <asalas@kepler.cl> wrote:
> Hello sage,
> I followed your lead and went a bit further in my source code reading and I
> notice serveral problems:
> first most of the id use in ceph osd_clients, mds_clients, mon_client use
> the data type u64 which will be a problem
> as most of the windows in use are windows XP or Windows server 2003 in 32
> bits. As they are used for id tag for
> things like snapshot pages (if I understand well) that can be a problem for
> a port no?
>
> What I read so far was
> mount.ceph files > kernel/fs/ceph files which contains the mds_client files
> and the kernel ioctl interface ... > include / linux / ceph files which
> integrates libcephfs.h. and all the needed files
>
> there is three clients, there is auth.h a messenger, a msgpool, buffer,
> rados that depends on msgr that include in every aspect of the messages
> formation one or more __le64 message.
>
> I m far from understanding the whole code and the interactions betwin all
> the files and which we can skip and which we can keep.
>
> how can we translate those data type into 32bit in order for ceph cluster to
> understand them and ceph-windows-client to transmit them?
It is possible to define 64 bit variables on 32 bit architectures. The
compiler handles it for you.
Yehuda
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2013-11-05 17:40 ` Alphe Salas Michels
@ 2013-11-05 21:49 ` Alphe Salas Michels
2013-11-05 21:59 ` Yehuda Sadeh
2013-11-06 0:00 ` Sage Weil
1 sibling, 1 reply; 21+ messages in thread
From: Alphe Salas Michels @ 2013-11-05 21:49 UTC (permalink / raw)
To: Alphe Salas Michels, Sage Weil; +Cc: ceph-devel
Hello sage,
I followed your lead and went a bit further in my source code reading
and I notice serveral problems:
first most of the id use in ceph osd_clients, mds_clients, mon_client
use the data type u64 which will be a problem
as most of the windows in use are windows XP or Windows server 2003 in
32 bits. As they are used for id tag for
things like snapshot pages (if I understand well) that can be a problem
for a port no?
What I read so far was
mount.ceph files > kernel/fs/ceph files which contains the mds_client
files and the kernel ioctl interface ... > include / linux / ceph
files which integrates libcephfs.h. and all the needed files
there is three clients, there is auth.h a messenger, a msgpool, buffer,
rados that depends on msgr that include in every aspect of the messages
formation one or more __le64 message.
I m far from understanding the whole code and the interactions betwin
all the files and which we can skip and which we can keep.
how can we translate those data type into 32bit in order for ceph
cluster to understand them and ceph-windows-client to transmit them?
Regards
signature
*Alphé Salas*
Ingeniero T.I
asalas@kepler.cl
On 11/05/13 14:40, Alphe Salas Michels wrote:
>
> signature *Hi, Sage !
> thank you for you enthousiast reply.
> I sure want to make the best use of everything or anything previously
> done to tend to
> write ceph cliente for windows.
>
> Apart using libre tools for building the future ceph cliente I am open
> to anything.
> I would recommand eclipse CDT or Code::BLocks they are based on
> mingwin open and easyly enhanceable.**
>
> more free tools can be found here:
> http://www.freebyte.com/programming/cpp/#cppcompilers
>
>
> I will read libcephfs source code and take some notes about the protocol.
> I was more going from what I know and trying to track down how
> mount.ceph work with the parameters passed to it.
> since it point finally to Kernel/fs/ceph and that I don t really
> understand how that module work and that it probably points to some
> other dependencies Reading libcephfs source code could be a big gain
> of time.
>
> basically on the protocol what is need are:
>
> 1) open and maintain a connection (socket open, auth, etc )
> 2) retreive a map of directories and disk Quota (disk sizing Y TB
> free, Z TB total)
> 3) procedure to send files / directories in a maner that it will allow
> our client to fit ceph transmission protocols
> (limit bandwith for stability?, limit connection amount?, limit cpu
> use?, Cache for preparing data transfer (a FIFO cache)?)
> 4)Procedure to retreive files / directory from ceph cluster
> 5) Management copy/move files /Directories, FS stats, Connection
> Stats. logging.
>
> My idea to progress is to take those main bulletpoint in ceph protocol
> based on general ideas of what ceph file system does and start
> identifying parts from libcephfs to match those "needs".
>
> Any suggestion and contributions are welcome.
>
>
> *
> On 11/05/13 11:23, Sage Weil wrote:
>> Hi Alphe,
>>
>> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
>>> Good day developers!
>>>
>>> I would like to propose to the one interested work with me to
>>> develop a ceph
>>> cliente for MS windows world, Basing us on dokanFS.
>>>
>>> My company is a ceph enthousiast that use on a dayly basis ceph and
>>> that need
>>> both transfer speed and big expendable and cheap storage.
>>> My company is specialised in data recovery and we want to
>>> participate to ceph
>>> effort by bringing a ceph cliente for windows.
>> Awesome!
>>
>>> Our experience shows us that the best gateway is each clientes being
>>> its own
>>> gateway, instead of having a bottle neck server or a cluster of
>>> bottle neck
>>> servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
>>>
>>> We already did some research in that domain.
>>>
>>> Dokan FS is an intent to write an opensource fuse like cliente for MS
>>> windows.
>>>
>>> More information on DOKANFS can be triggered here
>>> http://dokan-dev.net/en/download/
>>>
>>> Positive points of using DOKANFS.
>>>
>>> - its opensourced and well licenced mit licence, gpl licence and
>>> lgpl licence.
>>>
>>> Negative point of using DOKAN FS.
>>> - unreachable author
>>> - Poor documentation . Dev comments in japanese.
>>> - Work in progress so it is unstable and needs to be updated,
>>> debugged and
>>> maintained by a MS Windows file system expert developper.
>> I am not very familiar with windows storage APIs, but somebody told me
>> at once point there were several interfaces against which a new file
>> system could be implemented, everything from a full in-kernel driver to
>> something that is explorer-based. Are any of those suitable? Using a
>> potentially abandoned fuse-like layer makes me a bit nervous.
>>
>> That said,
>>> I try past year to do a merge from ceph-fuse to dokanfs
>>> here are what I learnt.
>>> - Ceph-fuse and related source code is around 60 000 lines of code.
>>> - Ceph protocol isn t documented so it is like trying to draw a map
>>> of america
>>> using only a sextan and a compass.
>>>
>>> Those led me to those conclusions:
>>> - I can t do it alone.
>>> - It is easier to draw down the ceph protocol way to work from
>>> kernel/fs/ceph
>>> sources and mount.ceph
>>> - Ceph depending libraries may be unexistant or not up to date in
>>> their port
>>> on MS Windows (cygwin)
>> I think the most sane path should be to make libcephfs sufficiently
>> portable to build on windows (or cygwin). For the bits used by the
>> client-side coe, I don't think there should be much in the way of
>> dependencies, and the main challenge would be untangling the build for
>> the necessary pieces out from the rest of Ceph.
>>
>> Have you seen the wip-port portability work that is currently
>> underway by
>> Noah and Alan? That may solve many of the cygwin problems you are
>> seeing
>> today.
>>
>>> - MS file system specialist are hard do find in the "open source
>>> libre world"
>>> so I will try in the commercial world.
>>>
>>> The commercial world has some problems too. They need ceph protocol
>>> draft to
>>> implemente it to their own product They will have licencing /commercial
>>> politics that infringe lgpl, and hide that most of the work is done
>>> by people
>>> other than them. They will not participate in a financial way to ceph
>>> enhancement and growth.
>> I don't think reimplementing the client code is an efficient way
>> forward.
>> Unless the goal is a pure kernel implementation...but a significant
>> ongoing investment in development resources would be needed for that
>> going
>> forward. I suspect that is a challenge for a platform that does not
>> typically rally that sort of community effort.
>>
>> The easiest thing is of course just to use CIFS and Samba (which works
>> today). A fuse-like approach is probably a reasonably middle ground
>> (both
>> in initial effort and maintainability going forward)...
>>
>> sage
>>
>>
>
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2013-11-05 14:23 ` Sage Weil
@ 2013-11-05 17:40 ` Alphe Salas Michels
2013-11-05 21:49 ` Alphe Salas Michels
2013-11-06 0:00 ` Sage Weil
0 siblings, 2 replies; 21+ messages in thread
From: Alphe Salas Michels @ 2013-11-05 17:40 UTC (permalink / raw)
To: Sage Weil; +Cc: ceph-devel
signature *Hi, Sage !
thank you for you enthousiast reply.
I sure want to make the best use of everything or anything previously
done to tend to
write ceph cliente for windows.
Apart using libre tools for building the future ceph cliente I am open
to anything.
I would recommand eclipse CDT or Code::BLocks they are based on mingwin
open and easyly enhanceable.**
more free tools can be found here:
http://www.freebyte.com/programming/cpp/#cppcompilers
I will read libcephfs source code and take some notes about the protocol.
I was more going from what I know and trying to track down how
mount.ceph work with the parameters passed to it.
since it point finally to Kernel/fs/ceph and that I don t really
understand how that module work and that it probably points to some
other dependencies Reading libcephfs source code could be a big gain of
time.
basically on the protocol what is need are:
1) open and maintain a connection (socket open, auth, etc )
2) retreive a map of directories and disk Quota (disk sizing Y TB free,
Z TB total)
3) procedure to send files / directories in a maner that it will allow
our client to fit ceph transmission protocols
(limit bandwith for stability?, limit connection amount?, limit cpu
use?, Cache for preparing data transfer (a FIFO cache)?)
4)Procedure to retreive files / directory from ceph cluster
5) Management copy/move files /Directories, FS stats, Connection Stats.
logging.
My idea to progress is to take those main bulletpoint in ceph protocol
based on general ideas of what ceph file system does and start
identifying parts from libcephfs to match those "needs".
Any suggestion and contributions are welcome.
*
On 11/05/13 11:23, Sage Weil wrote:
> Hi Alphe,
>
> On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
>> Good day developers!
>>
>> I would like to propose to the one interested work with me to develop a ceph
>> cliente for MS windows world, Basing us on dokanFS.
>>
>> My company is a ceph enthousiast that use on a dayly basis ceph and that need
>> both transfer speed and big expendable and cheap storage.
>> My company is specialised in data recovery and we want to participate to ceph
>> effort by bringing a ceph cliente for windows.
> Awesome!
>
>> Our experience shows us that the best gateway is each clientes being its own
>> gateway, instead of having a bottle neck server or a cluster of bottle neck
>> servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
>>
>> We already did some research in that domain.
>>
>> Dokan FS is an intent to write an opensource fuse like cliente for MS
>> windows.
>>
>> More information on DOKANFS can be triggered here
>> http://dokan-dev.net/en/download/
>>
>> Positive points of using DOKANFS.
>>
>> - its opensourced and well licenced mit licence, gpl licence and lgpl licence.
>>
>> Negative point of using DOKAN FS.
>> - unreachable author
>> - Poor documentation . Dev comments in japanese.
>> - Work in progress so it is unstable and needs to be updated, debugged and
>> maintained by a MS Windows file system expert developper.
> I am not very familiar with windows storage APIs, but somebody told me
> at once point there were several interfaces against which a new file
> system could be implemented, everything from a full in-kernel driver to
> something that is explorer-based. Are any of those suitable? Using a
> potentially abandoned fuse-like layer makes me a bit nervous.
>
> That said,
>
>> I try past year to do a merge from ceph-fuse to dokanfs
>> here are what I learnt.
>> - Ceph-fuse and related source code is around 60 000 lines of code.
>> - Ceph protocol isn t documented so it is like trying to draw a map of america
>> using only a sextan and a compass.
>>
>> Those led me to those conclusions:
>> - I can t do it alone.
>> - It is easier to draw down the ceph protocol way to work from kernel/fs/ceph
>> sources and mount.ceph
>> - Ceph depending libraries may be unexistant or not up to date in their port
>> on MS Windows (cygwin)
> I think the most sane path should be to make libcephfs sufficiently
> portable to build on windows (or cygwin). For the bits used by the
> client-side coe, I don't think there should be much in the way of
> dependencies, and the main challenge would be untangling the build for
> the necessary pieces out from the rest of Ceph.
>
> Have you seen the wip-port portability work that is currently underway by
> Noah and Alan? That may solve many of the cygwin problems you are seeing
> today.
>
>> - MS file system specialist are hard do find in the "open source libre world"
>> so I will try in the commercial world.
>>
>> The commercial world has some problems too. They need ceph protocol draft to
>> implemente it to their own product They will have licencing /commercial
>> politics that infringe lgpl, and hide that most of the work is done by people
>> other than them. They will not participate in a financial way to ceph
>> enhancement and growth.
> I don't think reimplementing the client code is an efficient way forward.
> Unless the goal is a pure kernel implementation...but a significant
> ongoing investment in development resources would be needed for that going
> forward. I suspect that is a challenge for a platform that does not
> typically rally that sort of community effort.
>
> The easiest thing is of course just to use CIFS and Samba (which works
> today). A fuse-like approach is probably a reasonably middle ground (both
> in initial effort and maintainability going forward)...
>
> sage
>
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: writing a ceph cliente for MS windows
2013-11-04 20:19 Alphe Salas Michels
@ 2013-11-05 14:23 ` Sage Weil
2013-11-05 17:40 ` Alphe Salas Michels
2013-11-05 23:33 ` James Harper
1 sibling, 1 reply; 21+ messages in thread
From: Sage Weil @ 2013-11-05 14:23 UTC (permalink / raw)
To: Alphe Salas Michels; +Cc: ceph-devel
Hi Alphe,
On Mon, 4 Nov 2013, Alphe Salas Michels wrote:
> Good day developers!
>
> I would like to propose to the one interested work with me to develop a ceph
> cliente for MS windows world, Basing us on dokanFS.
>
> My company is a ceph enthousiast that use on a dayly basis ceph and that need
> both transfer speed and big expendable and cheap storage.
> My company is specialised in data recovery and we want to participate to ceph
> effort by bringing a ceph cliente for windows.
Awesome!
> Our experience shows us that the best gateway is each clientes being its own
> gateway, instead of having a bottle neck server or a cluster of bottle neck
> servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
>
> We already did some research in that domain.
>
> Dokan FS is an intent to write an opensource fuse like cliente for MS
> windows.
>
> More information on DOKANFS can be triggered here
> http://dokan-dev.net/en/download/
>
> Positive points of using DOKANFS.
>
> - its opensourced and well licenced mit licence, gpl licence and lgpl licence.
>
> Negative point of using DOKAN FS.
> - unreachable author
> - Poor documentation . Dev comments in japanese.
> - Work in progress so it is unstable and needs to be updated, debugged and
> maintained by a MS Windows file system expert developper.
I am not very familiar with windows storage APIs, but somebody told me
at once point there were several interfaces against which a new file
system could be implemented, everything from a full in-kernel driver to
something that is explorer-based. Are any of those suitable? Using a
potentially abandoned fuse-like layer makes me a bit nervous.
That said,
> I try past year to do a merge from ceph-fuse to dokanfs
> here are what I learnt.
> - Ceph-fuse and related source code is around 60 000 lines of code.
> - Ceph protocol isn t documented so it is like trying to draw a map of america
> using only a sextan and a compass.
>
> Those led me to those conclusions:
> - I can t do it alone.
> - It is easier to draw down the ceph protocol way to work from kernel/fs/ceph
> sources and mount.ceph
> - Ceph depending libraries may be unexistant or not up to date in their port
> on MS Windows (cygwin)
I think the most sane path should be to make libcephfs sufficiently
portable to build on windows (or cygwin). For the bits used by the
client-side coe, I don't think there should be much in the way of
dependencies, and the main challenge would be untangling the build for
the necessary pieces out from the rest of Ceph.
Have you seen the wip-port portability work that is currently underway by
Noah and Alan? That may solve many of the cygwin problems you are seeing
today.
> - MS file system specialist are hard do find in the "open source libre world"
> so I will try in the commercial world.
>
> The commercial world has some problems too. They need ceph protocol draft to
> implemente it to their own product They will have licencing /commercial
> politics that infringe lgpl, and hide that most of the work is done by people
> other than them. They will not participate in a financial way to ceph
> enhancement and growth.
I don't think reimplementing the client code is an efficient way forward.
Unless the goal is a pure kernel implementation...but a significant
ongoing investment in development resources would be needed for that going
forward. I suspect that is a challenge for a platform that does not
typically rally that sort of community effort.
The easiest thing is of course just to use CIFS and Samba (which works
today). A fuse-like approach is probably a reasonably middle ground (both
in initial effort and maintainability going forward)...
sage
^ permalink raw reply [flat|nested] 21+ messages in thread
* writing a ceph cliente for MS windows
@ 2013-11-04 20:19 Alphe Salas Michels
2013-11-05 14:23 ` Sage Weil
2013-11-05 23:33 ` James Harper
0 siblings, 2 replies; 21+ messages in thread
From: Alphe Salas Michels @ 2013-11-04 20:19 UTC (permalink / raw)
To: ceph-devel
Good day developers!
I would like to propose to the one interested work with me to develop a
ceph cliente for MS windows world, Basing us on dokanFS.
My company is a ceph enthousiast that use on a dayly basis ceph and that
need both transfer speed and big expendable and cheap storage.
My company is specialised in data recovery and we want to participate to
ceph effort by bringing a ceph cliente for windows.
Our experience shows us that the best gateway is each clientes being its
own gateway, instead of having a bottle neck server or a cluster of
bottle neck servers as gateway (FTP, samba, SFTP,webdav, s3, etc..).
We already did some research in that domain.
Dokan FS is an intent to write an opensource fuse like cliente for MS
windows.
More information on DOKANFS can be triggered here
http://dokan-dev.net/en/download/
Positive points of using DOKANFS.
- its opensourced and well licenced mit licence, gpl licence and lgpl
licence.
Negative point of using DOKAN FS.
- unreachable author
- Poor documentation . Dev comments in japanese.
- Work in progress so it is unstable and needs to be updated, debugged
and maintained by a MS Windows file system expert developper.
I try past year to do a merge from ceph-fuse to dokanfs
here are what I learnt.
- Ceph-fuse and related source code is around 60 000 lines of code.
- Ceph protocol isn t documented so it is like trying to draw a map of
america using only a sextan and a compass.
Those led me to those conclusions:
- I can t do it alone.
- It is easier to draw down the ceph protocol way to work from
kernel/fs/ceph sources and mount.ceph
- Ceph depending libraries may be unexistant or not up to date in their
port on MS Windows (cygwin)
- MS file system specialist are hard do find in the "open source libre
world" so I will try in the commercial world.
The commercial world has some problems too. They need ceph protocol
draft to implemente it to their own product They will have licencing
/commercial politics that infringe lgpl, and hide that most of the work
is done by people other than them. They will not participate in a
financial way to ceph enhancement and growth.
As for the modalities of the work if we come to work togather on that
topic, it up to you. We can / should use the most possible libre tools
for that work (eclipse, mingwin/cygwin etc...) Sharpening the dev tools
in libre can already be a big problem. But my way to see this is the
more libre are our tools on MS Windows the most people can easyly join us.
Best Regards,
Alphe Salas
asalas@kepler.cl
I.T ingeneer.
*<http://www.kepler.cl>*
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2014-12-27 5:14 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <CAME-gASugOE=-ZoY3-sAa4vCzOd+TXBiEavjoUH3gDLG+1K38w@mail.gmail.com>
2013-11-07 12:13 ` writing a ceph cliente for MS windows Alphe Salas Michels
2013-11-07 14:29 ` Ketor D
2013-11-07 14:50 ` Matt W. Benjamin
2013-11-07 18:02 ` Alphe Salas Michels
2013-11-07 20:47 ` Alphe Salas Michels
2013-11-08 0:11 ` Malcolm Haak
2013-11-08 0:31 ` Matt W. Benjamin
[not found] ` <CAME-gARbjR++qXsx9Sx4KbuggthONrscHToxCTUKrci_MLMDzw@mail.gmail.com>
2013-11-08 14:15 ` Alphe Salas Michels
2014-12-26 17:10 ` Ketor D
2014-12-27 5:14 ` Dong Yuan
2014-12-27 5:14 ` Dong Yuan
2013-11-04 20:19 Alphe Salas Michels
2013-11-05 14:23 ` Sage Weil
2013-11-05 17:40 ` Alphe Salas Michels
2013-11-05 21:49 ` Alphe Salas Michels
2013-11-05 21:59 ` Yehuda Sadeh
2013-11-05 22:31 ` Alphe Salas Michels
2013-11-06 0:00 ` Sage Weil
2013-11-06 14:37 ` Alphe Salas Michels
2013-11-06 21:47 ` Alphe Salas Michels
2013-11-05 23:33 ` James Harper
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.